Search Results: "ron"

3 July 2013

Petter Reinholdtsen: Fourth alpha release of Debian Edu/Skolelinux based on Debian Wheezy

The fourth wheezy based alpha release of Debian Edu was wrapped up today. This is the release announcement: New features for Debian Edu 7.1+edu0~alpha3 released 2013-07-03 These are the release notes for for Debian Edu / Skolelinux 7.1+edu0~alpha3, based on Debian with codename "Wheezy". About Debian Edu and Skolelinux Debian Edu, also known as Skolelinux, is a Linux distribution based on Debian providing an out-of-the box environment of a completely configured school network. Immediately after installation a school server running all services needed for a school network is set up just waiting for users and machines being added via GOsa , a comfortable Web-UI. A netbooting environment is prepared using PXE, so after initial installation of the main server from CD, DVD or USB stick all other machines can be installed via the network. The provided school server provides LDAP database and Kerberos authentication service, centralized home directories, DHCP server, web proxy and many other services. The desktop contains more than 60 educational software packages and more are available from the Debian archive, and schools can choose between KDE, Gnome, LXDE and Xfce desktop environment. This is the fourth test release based on Debian Wheezy. Basically this is an updated and slightly improved version compared to the Squeeze release. Software updates Other changes Known issues Where to get it To download the multiarch netinstall CD release you can use The MD5SUM of this image is: 2b161a99d2a848c376d8d04e3854e30c
The SHA1SUM of this image is: 498922e9c508c0a7ee9dbe1dfe5bf830d779c3c8 To download the multiarch USB stick ISO release you can use The MD5SUM of this image is: 25e808e403a4c15dbef1d13c37d572ac
The SHA1SUM of this image is: 15ecfc93eb6b4f453b7eb0bc04b6a279262d9721 How to report bugs http://wiki.debian.org/DebianEdu/HowTo/ReportBugs

2 July 2013

Dominique Dumont: Seeking help to update OpenSSH configuration editor (Config::Model)

Hello OpenSsh 6.0 has been out for more than a year and the OpenSSH configuration editor still lacks the new parameters introduced by this release (like AllowAgentForwarding). Technically, the task is not difficult, but I lack time to address it: I m swamped by real life job, maintenance of Config::Model, Debian packaging activities So I m looking for volunteer(s) to help me on Config::Model. Updating OpenSSH model is a great way to start ! If you want to help, please: Be sure that you won t be forgotten in the change log ;-) Feel free to contact config-model-users at lists.sourceforge.net to get help, exchange ideas, or to discuss how to handle deprecation or upgrade of OpenSSH parameters All the best update: OpenSSH project name was corrected after Rafal s comment.
Tagged: Config::Model, OpenSsh, Perl

Ondřej Čertík: My impressions from the SciPy 2013 conference

I have attended the SciPy 2013 conference in Austin, Texas. Here are my impressions.

Number one is the fact that the IPython notebook was used by pretty much everyone. I use it a lot myself, but I didn't realize how ubiquitous it has become. It is quickly becoming the standard now. The IPython notebook is using Markdown and in fact it is better than Rest. The way to remember the "[]()" syntax for links is that in regular text you put links into () parentheses, so you do the same in Markdown, and append [] for the text of the link. The other way to remember is that [] feel more serious and thus are used for the text of the link. I stressed several times to +Fernando Perez and +Brian Granger how awesome it would be to have interactive widgets in the notebook. Fortunately that was pretty much preaching to the choir, as that's one of the first things they plan to implement good foundations for and I just can't wait to use that.

It is now clear, that the IPython notebook is the way to store computations that I want to share with other people, or to use it as a "lab notebook" for myself, so that I can remember what exactly I did to obtain the results (for example how exactly I obtained some figures from raw data). In other words --- instead of having sets of scripts and manual bash commands that have to be executed in particular order to do what I want, just use IPython notebook and put everything in there.

Number two is that how big the conference has become since the last time I attended (couple years ago), yet it still has the friendly feeling. Unfortunately, I had to miss a lot of talks, due to scheduling conflicts (there were three parallel sessions), so I look forward to seeing them on video.

+Aaron Meurer and I have done the SymPy tutorial (see the link for videos and other tutorial materials). It's been nice to finally meet +Matthew Rocklin (very active SymPy contributor) in person. He also had an interesting presentation
about symbolic matrices + Lapack code generation. +Jason Moore presented PyDy.
It's been a great pleasure for us to invite +David Li (still a high school student) to attend the conference and give a presentation about his work on sympygamma.com and live.sympy.org.

It was nice to meet the Julia guys, +Jeff Bezanson and +Stefan Karpinski. I contributed the Fortran benchmarks on the Julia's website some time ago, but I had the feeling that a lot of them are quite artificial and not very meaningful. I think Jeff and Stefan confirmed my feeling. Julia seems to have quite interesting type system and multiple dispatch, that SymPy should learn from.

I met the VTK guys +Matthew McCormick and +Pat Marion. One of the keynotes was given by +Will Schroeder from Kitware about publishing. I remember him stressing to manage dependencies well as well as to use BSD like license (as opposed to viral licenses like GPL or LGPL). That opensource has pretty much won (i.e. it is now clear that that is the way to go).

I had great discussions with +Francesc Alted, +Andy Terrel, +Brett Murphy, +Jonathan Rocher, +Eric Jones, +Travis Oliphant, +Mark Wiebe, +Ilan Schnell, +St fan van der Walt, +David Cournapeau, +Anthony Scopatz, +Paul Ivanov, +Michael Droettboom, +Wes McKinney, +Jake Vanderplas, +Kurt Smith, +Aron Ahmadia, +Kyle Mandli, +Benjamin Root and others.


It's also been nice to have a chat with +Jason Vertrees and other guys from Schr dinger.

One other thing that I realized last week at the conference is that pretty much everyone agreed on the fact that NumPy should act as the default way to represent memory (no matter if the array was created in Fortran or other code) and allow manipulations on it. Faster libraries like Blaze or ODIN should then hook themselves up into NumPy using multiple dispatch. Also SymPy would then hook itself up so that it can be used with array operations natively. Currently SymPy does work with NumPy (see our tests for some examples what works), but the solution is a bit fragile (it is not possible to override NumPy behavior, but because NumPy supports general objects, we simply give it SymPy objects and things mostly work).

Similar to this, I would like to create multiple dispatch in SymPy core itself, so that other (faster) libraries for symbolic manipulation can hook themselves up, so that their own (faster) multiplication, expansion or series expansion would get called instead of the SymPy default one implemented in pure Python.

Other blog posts from the conference:

Bits from Debian: all Debian source are belong to us

This is a verbatim repost from Stefano Zacchiroli's post TL;DR: go to http://sources.debian.net and enjoy.
Debsources is a new toy I've been working on at IRILL together with Matthieu Caneill. In essence, debsources is a simple web application that allows to publish an unpacked Debian source mirror on the Web. You can deploy Debsources where you please, but there is a main instance at http://sources.debian.net (sources.d.n for short) that you will probably find interesting. sources.d.n follows closely the Debian archive in two ways:
  1. it is updated 4 times a day to reflect the content of the Debian archive
  2. it contains sources coming from official Debian suites: the usual ones (from oldstable to experimental), *-updates (ex volatile), *-proposed-updates, and *-backports (from Wheezy on)
Via sources.d.n you can therefore browse the content of Debian source packages with usual code viewing features like syntax highlighting. More interestingly, you can search through the source code (of unstable only, though) via integration with http://codesearch.debian.net. You can also use sources.d.n programmatically to query available versions or link to specific lines, with the possibility of adding contextual pop-up messages (example). In fact, you might have stumbled upon sources.d.n already in the past few days, via other popular Debian services where it has already been integrated. In particular: codesearch.d.n now defaults to show results via sources.d.n, and the PTS has grown new "browse source code" hyperlinks that point to it. If you've ideas of other Debian services where sources.d.n should be integrated, please let me know. I find Debsources and sources.d.n already quite useful but, as it often happens, there is still a lot TODO. Obviously, it is all Free Software (released under GNU AGPLv3). Do not hesitate to report new bugs and, better, to submit patches for the outstanding ones. Acknowledgements PS in case you were wondering: at present sources.d.n requires ~381 GB of disk space to hold all uncompressed source packages, plus ~83 GB for the local (compressed) source mirror

Stefano Zacchiroli: introducing sources.debian.net

all Debian source are belong to us TL;DR: go to http://sources.debian.net and enjoy.
Debsources is a new toy I've been working on at IRILL together with Matthieu Caneill. In essence, debsources is a simple web application that allows to publish an unpacked Debian source mirror on the Web. You can deploy Debsources where you please, but there is a main instance at http://sources.debian.net (sources.d.n for short) that you will probably find interesting. sources.d.n follows closely the Debian archive in two ways:
  1. it is updated 4 times a day to reflect the content of the Debian archive
  2. it contains sources coming from official Debian suites: the usual ones (from oldstable to experimental), *-updates (ex volatile), *-proposed-updates, and *-backports (from Wheezy on)
Via sources.d.n you can therefore browse the content of Debian source packages with usual code viewing features like syntax highlighting. More interestingly, you can search through the source code (of unstable only, though) via integration with http://codesearch.debian.net. You can also use sources.d.n programmatically to query available versions or link to specific lines, with the possibility of adding contextual pop-up messages (example). In fact, you might have stumbled upon sources.d.n already in the past few days, via other popular Debian services where it has already been integrated. In particular: codesearch.d.n now defaults to show results via sources.d.n, and the PTS has grown new "browse source code" hyperlinks that point to it. If you've ideas of other Debian services where sources.d.n should be integrated, please let me know. I find Debsources and sources.d.n already quite useful but, as it often happens, there is still a lot TODO. Obviously, it is all Free Software (released under GNU AGPLv3). Do not hesitate to report new bugs and, better, to submit patches for the outstanding ones. Acknowledgements PS in case you were wondering: at present sources.d.n requires ~381 GB of disk space to hold all uncompressed source packages, plus ~83 GB for the local (compressed) source mirror

Lars Wirzenius: Consistency in backups

A backup run takes time. During a backup run the live data may change. This can mean that the snapshot of the live data may not be internally consistent. For example, suppose you're writing an e-mail when the backup starts automatically. If the timing goes wrong in exactly the right way, the backup might capture both the draft e-mail you were writing when the backup started, and the sent e-mail, when you finished the draft and sent the mail before the backup ended. Or, worse, the backup might miss both the draft and the sent e-mail. The next backup run will, certainly, get the sent e-mail, but what if you have a disaster before that happens? Internal consistency of data can be quite difficult to achive. Some programs, such as MySQL, don't do a good job of keeping the data consistent on-disk, making it difficult for the backup program to get a proper setup. (In that case, a standard trick is to "dump" or "export" the data in a consistent form, and back that up instead.) In an ideal situation, when the backup starts, the system would have a way to tell all running programs to store all their state on disk in a recoverable manner (save all open documents, dump data base content, etc), and once that is done, the backup program creates a snapshot of the filesystem. Unfortunately, I don't know of any operating system that can request consistent state from running applications. Snapshotting, however, is easy-peasy. It can be achieved by the filesystem (e.g., btrfs), or using LVM. Doing the snapshots requires root privileges, but that can probably be arranged. However, snapshotting like this, without getting all programs to commit their state to disk first, is not a guarantee of consistency. A thorough way to be consistent is to shut down the computer, and then reboot it into a special backup mode (similar to single-user mode), where no services or applications are running. However, this is, for most people, much too heavyweight a procedure to be worth it. Does consistency really matter? It matters a lot when the live data is being changed actively and consistency of the data is critical. For example, a bank's database of the amount of money in each account is quite critical: if data is not backed up consistently, money can be lost or created spontaneously, and that's probably a bad thing. Most people, however, don't need to worry a lot, and merely need to be aware of the issue, and perhaps mitigate it by backing up when the computer is mostly idle, and run backups often enough. Oh dear. This blog post contains too few jokes. I seem to have run out of jokes to make, so that will conclude this series of blog posts. I hope it was useful to some.

1 July 2013

Michael Stapelberg: Survey answers part 2: the transition

This blog post is the second of a series of posts dealing with the results of the Debian systemd survey. I intend to give a presentation at DebConf 2013, too, so you could either read my posts, or watch the talk, or both :-). It seems that it is unclear how Debian s transition to systemd is intended to work. By transition , we mean going from the current state (sysvinit is the default and fully supported) to systemd is fully supported. Then by merely installing systemd by default and letting it provide /sbin/init, we can make it the default init system. If and when that happens is a different matter and it s not necessary for all packages to have systemd support. sysvinit compatibility systemd natively supports sysvinit scripts, meaning your existing package will work as-is but you cannot utilize all the features that systemd provides. The sysvinit support works very well, as you can try in a fresh Debian wheezy VM. In the output of systemctl list-units , every entry which has an LSB: prefix is actually a sysvinit script. The mechanism with which systemd decides whether to use an init script or a service file is by looking whether a service file with a corresponding name exists. That is, if e.g. apache2.service exists, systemd will prefer it over /etc/init.d/apache2. To make this crystal clear: it is not necessary to ship service files for all services in some kind of flag day. systemd supports a mixed installation where some services use init scripts and some services use service files. Adding systemd support to your package In a nutshell, it usually works like this:
  1. Install a service file to /lib/systemd/system/foo.service.
    Often, upstream already provides and installs a .service file.
    If not, you can place your file at debian/package.service
    Make sure that your service file name corresponds to the sysvinit script name
    (e.g. apache2.service for /etc/init.d/apache2)
  2. Ensure your service file(s) are enabled and started.
    We strongly recommend you to use our package dh-systemd.
    If you use dh(1), add --with=systemd in debian/rules and Build-Dep on dh-systemd
  3. Test your package, see the next section.
For details see wiki.debian.org/Systemd/Packaging. Testing systemd We carefully made sure that you can install the systemd Debian package on your machine alongside sysvinit without breaking anything. The systemd package does not conflict with any other packages, it will not replace /sbin/init and systemd will not be enabled right away. It is only after you specify the kernel parameter init=/bin/systemd in /etc/default/grub that you switch to systemd. In case you want to go back, simply boot without this kernel parameter. Conclusion In conclusion, the transition is straight-forward and the necessary infrastructure is in place. systemd is available in Debian and can be used today. Packages can add systemd support whenever their maintainer(s) feel like it. There is no need for a flag day. We can switch the default whenever we think we are ready.

Russ Allbery: Review: Dark Lies the Island

Review: Dark Lies the Island, by Kevin Barry
Publisher: Greywolf
Copyright: 2012
Printing: 2013
ISBN: 1-55597-651-4
Format: ARC
Pages: 185
This is a literary short story collection by a mainstream Irish writer, rather far from my normal fare. It was the second book in a shipment of Powell's Indiespensable, which is generally (when there is one) a proof or Advanced Reading Copy of something relatively obscure. I'm not really the target audience, not being much of a short story aficionado, but the whole point of doing this Indiespensable experiment is to stretch my reading range. First off, as I would generally expect by mainstream fiction, the quality of the writing is excellent. One thing I particularly like about Barry is that he's refreshingly and unhesitatingly profane. His characters curse like I expect people to curse, some of them at the drop of a hat, and for me it gives a feel of solid realism to his dialogue. He's also very good with eye dialect, giving the impression of accents without making the text hard to read. Some of these stories I liked a great deal, rather more than I had expected. But if there was a general undermining flaw for me, it was that I prefer stories that go somewhere, that have a plot or lead up to a definite point. Some of Barry's do, but a lot of them seem more vignettes or just scenes, sketches of characters, that drift off without resolution. I can admire those as writing exercises, but for me they don't fill the desire I have when I sit down with a story. I'm not sure how useful these reviews will be, given that I don't have a lot of mainstream fiction background to compare them with, but here are some reactions, for whatever they're worth. "Across the Rooftops": This is one of those sketches, but it's one of the best of them. It's a single moment on the rooftops, an attempt to capture the moment of possibility around the first romantic overture. It's sadly quite conventional in gender roles (nervous man initiating, woman as relationship gatekeeper), but apart from that it's very well-written. This kind of thing is normally not to my taste, but Barry does such a great job capturing the sense of uncertainty and the sharp focus on detail that I couldn't help but like it. (7) "Wifey Redux": This is by far my favorite story of this book. I think it's both hilarious and brilliant, right from the starting line.
This is the story of a happy marriage but before you throw up and turn the page let me say that it will end with my face pressed hard into the cold metal of the Volvo's bonnet, my hands cuffed behind my back, and my rights droned into my ear this will occur in the car park of a big-box retail unit on the Naas Road in Dublin.
It's the story of a marriage and how it changes over time, but also the story of being a father and his baffled, flailing reactions to his daughter's sexual explorations. But that's not what makes the story. What makes the story is what leads up to the arrest at the end, which had me laughing out-loud in delight. Barry is great at writing a sort of devilish humor; I wish more of the collection was like this. (9) "Fjord of Killary": This is another one of the ones I enjoyed. It's about an author who buys a hotel (with a bar) on the fjord of Killary thinking that the local culture and contact along with the wild beauty would inspire his poetry. At the start of the story, he's thoroughly sick of both the climate and the people, and the collision between his idyllic dreams and the reality, as well as his acerbic first-person complaints, is quite entertaining. A huge storm and resulting flood provides the conflict and impetus for change, and a few moments of humor in the understated local reactions. I thought the closing epiphany was a bit too obvious and expected, and the ending didn't quite work for me, but it has some good moments. (7) "A Cruelty": Here's where the collection started losing me. This is still well-written and closely observed, but in the case of this story, I wish it weren't, since I found it extremely uncomfortable. The viewpoint character is vulnerable to an encounter of unalloyed nastiness. I'm sure that's the point, but, similar to how I avoid horror, it's the sort of story I'd rather not read. (3) "Beer Trip to Llandudno": This was my second-favorite story of the collection. It's about a group of men who take periodic beer-tasting trips and rate beers. They're just normal men, from a variety of backgrounds and professions, and I thought Barry did a great job capturing the sort of camaraderie that comes from long association with a hobby. That includes, here, the awkward limits on what one talks about and the tendency to use the hobby as a shield from anything that drifts into less certain territory. I thought the story suffered somewhat from not having much of a plot or conclusion, but that's also necessary from the setup. This group of people isn't actually going to solve problems, but will be there for each other in their own way. (8) "Ernestine and Kit": Another very dark story, although that's not quite as obvious at the start. As with "A Cruelty," it's well-written but disturbing and lacks any sort of positive resolution. Here, I think Barry went a bit far into portraying people who are a large part of the popular imagination but who are exceptionally rare in reality. It's the kind of story that, at least for me, plays badly with my brain's threat analysis: too memorable for the level of actual risk, and feeling like it was playing a bit too much with popular boogeymen. (4) "The Mainland Campaign": This one was just too subtle for me. Some of the summaries for this book hint at a meaning for the story that I didn't pick up at all on my first reading. On re-reading, I suspect that analysis is right, but even re-reading the story with that interpretation in mind (this is the sort of story where I'm not sure the average American would pick it up on first reading, but others might), I still don't understand exactly what happened. I think some mainstream writing, and some short stories, tends to err on the side of subtle. (4) "Wistful England": Another one that's way over on the sketch of a moment side of stories. Unlike "Across the Rooftops," it failed to make any impression on me whatsoever; I had to skim it just now to remind myself of any details at all, and by the time I edited this review for posting, I'd forgotten it again. I think it was aiming at capturing a mood, but the mopey viewpoint character didn't mean anything to me. (3) "Doctor Sot": This is another odd one, and I'm not quite sure what I think of it. Either it's another character sketch without much of a plot, or the plot was too subtle. The protagonist is a rather unlikable drunk who doubles as a small town's incompetent doctor. The story involves his encounter with a local sort of hippie encampment, which is moderately interesting even though I couldn't stand the man. Then it leads to an encounter that clearly meant something, but which was entirely lost on me. (4) "The Girls and the Dogs": And from an absence of plot back to the deeply disturbing. This story is about a moderately unlikable drug dealer who gets refuge from an extremely unlikable crazy person in a weird relationship with a woman and her sister. (As is sadly typical, polyamory only makes an appearance when the author wants to paint a scenario that's utterly fucked up and psychotic.) There's a pseudo-fantasy element here in that the characters talk about spells, but I think the only spell involved is authorial fiat to create a deeply broken and sick game that plays into a host of negative stereotypes. Despite myself, I did get drawn into the suspense of the story (Barry's undoubtedly a good writer), but it left me feeling dirty on several levels. (4) "White Hitachi": Finally, another story that I liked, although not as good as the ones earlier in the collection. This one follows a con man and two-bit thief as he gets his brother out of lockup and tries to resolve, or at least stay ahead of, various problems in his life. Again, no real ending, but I thought Barry did a good job at taking a snapshot of a life from a perspective that I don't read much in literature (along with all the sexism and racism that comes along with it). It sort of drifts off rather than ends (although the last sentence is a great bit of characterization), but I found it oddly enjoyable. (6) "Dark Lies the Island": I'm not sure what to make of this. It's another character sketch, a moment of deep psychological significance for a character, floating somewhat alone but described in detail. But the character is a girl struggling with cutting, from a male author whose other stories did not fill me with confidence in his handling of female characters. Barry makes this psychologically believable, but I'm a little nervous about trusting his portrayal of that mindset. It's also, again, just a bit too subtle; I had a hard time getting an emotional feel for how the events of the story connect. (5) "Berlin Arkonaplatz My Lesbian Summer": The final story of the collection is fairly typical of the problem I had with most of it. It seems reasonably well-written, I had a hard time connecting with the protagonist or understanding his motivations. It looks at a slice of life with which I have no familiarity at all (and is interesting for that reason), but I don't quite trust the story enough to submerge in it. Barry seems to write a lot of protagonists who seem aimless, befuddled, or just largely out of control, and I like a bit more agency in the primary character. Silvija is an interesting character, even shown from another's perspective, but the story is almost purely observational: showing her without comment, without obvious motive, and mostly without plot. It's strangely compelling, but not quite compelling enough that I'd actually seek it out. Which, really, is the whole book in a nutshell. (6) Rating: 6 out of 10

30 June 2013

Russell Coker: Links June 2013

Cory Doctorow published a letter from a 14yo who had just read his novel Homeland [1]. I haven t had anything insightful to say about Aaron Swartz, so I think that this link will do [2]. Seth Godin gave an interesting TED talk about leading tribes [3]. I think everyone who is active in the FOSS community should watch this talk. Ron Garrett wrote an interesting post about the risk of being hit by a dinosaur killer [4]. We really need to do something about this and the cost of defending against asteroids is almost nothing compared to defence spending. Afra Raymond gave an interesting TED talk about corruption [5]. He focussed on his country Trinidad and Tobago but the lessons apply everywhere. Wikihouse is an interesting project that is based around sharing designs for houses that can be implemented using CNC milling machines [6]. It seems to be at the early stages but it has a lot of potential to change the building industry. Here is a TED blog post summarising Dan Pallotta s TED talk about fundraising for nonprofits [7]. His key point is that moral objections to advertising for charities significantly reduce their ability to raise funds and impacts the charitable mission. I don t entirely agree with his talk which is very positive towards spending on promotion but I think that he makes some good points which people should consider. Here is a TED blog post summarising Peter Singer s TED talk about effective altruism [8]. His focus seems to be on ways of cheaply making a significant difference which doesn t seem to agree with Dan Pallotta s ideas. Patton Oswalt wrote an insightful article about the culture of stand-up comedians which starts with joke stealing and heckling and ends with the issue of rape jokes [9]. Karen Eng wrote an interesting TED blog post about Anthony Vipin s invention of HAPTIC shoes for blind people [10]. The vibration of the shoes tells the person which way to walk and a computer sees obstacles that need to be avoided. David Blaine gave an interesting TED talk about how he prepared for a stunt of holding his breath for 17 minutes [11].
Related posts:
  1. Links March 2013 Russ Allbery wrote an informative post about how to determine...
  2. Links May 2013 Cameron Russell (who works as an underwear model) gave an...
  3. Links January 2013 AreWomenHuman has an interesting article about ViolentAcrez and the wide...

29 June 2013

Russ Allbery: remctl 3.5

remctl is the middleware layer that we use all over the place at Stanford. It's a simple GSS-API client/server application that provides a very low-weight way of running a specific command using GSS-API authentication and strong ACL checking. The primary point of this release is to fix a very long-standing occasional data corruption issue. Due to the way that the remctl server checked for the completion of a command, a command that produced a lot of output immediately before exiting could have the end of its output cut off when sending it back to the client. That's fixed in this release. Also in this release, I finally figured out the thread-safe way of specifying to the GSS-API layer which Kerberos ticket cache to use for authentication. This means that the remctl_set_ccache API is now thread-safe if there were Kerberos libraries available during the remctl build and gss_krb5_import_cred was supported. I also fixed the way that the Perl modules are versioned, which resulted in a one-time move backwards of the module versions from Perl's perspective (from 3.4 to 3.05). Hopefully that won't be a problem for anyone. The Perl module version will match the package version with an adjustment so that it sorts properly as a number from here on. You can get the latest version from the remctl distribution page.

28 June 2013

Daniel Leidert: Idea: A new toy (ein neues Spielzeug) ... HP Microserver N54L

Ich fertige regelm ig Backups meiner Systeme an. Diese werden auf der Systemplatte meines Notebooks abgelegt und via rsync auf mobilen Speicher dupliziert. Hierzu verwende ich eine USB-Festplatte. Diese enth lt auch Medien-Dateien und wird regelm ig an den Fernseher angeschlossen. Prinzipiell halte ich meine Daten daher f r sicher. Aber vor kurzem stie ich an die Grenzen ihrer Kapazit t. Schon l nger habe ich nach einer Alternative gesucht, nicht zuletzt da heute viel gr ere Festplatten m glich sind und mein Laptop ber einen eSATA-Anschluss verf gt, der schneller als USB2.0 ist. Meine bevorzugte Variante war ein FANTEC DB-ALU3e Geh use mit einer WD Red WD20EFRX 2TB (5400 RPM) Festplatte, die f r den 24/7 Betrieb zertifiziert ist (und zudem ber eine ausgezeichnete Reputation verf gt). Die Kombination lief sehr gut und schnell, sieht edel aus, ben tigt aber eine externe Stromversorgung. Ich kann Sie als Speicherl sung absolut empfehlen. Allerdings hatte ich zu diesem Zeitpunkt noch weitere Anspr che, die mit der o.g. L sung nicht zu befriedigen sind. So trage ich mich bereits l nger mit dem Gedanken an ein RAID-1-NAS. Au erdem spiegelt sich die Beanspruchung meiner Notebook-Festplatte durch das Pakete-Bauen f r Debian im S.M.A.R.T.-Status wieder. Daher wollte ich diese Arbeit an einen robusten lokalen buildd-Boliden abgeben und habe ber den Kauf eines g nstigen Rechners nachgedacht. Ein NAS verbraucht aber deutlich weniger Strom als ein Desktop-Rechner. Also wie l sst sich ein buildd und ein energiesparendes NAS vereinen? Per Zufall stie ich bei einem lokalen H ndler auf den HP ProLiant MicroServer N40L. Das Angebot klang super und so entschied ich mich zum Kauf meines neuen Spielzeuges: ein HP ProLiant MicroServer N54L, der zuk nftig folgende Aufgaben verrichten soll:
Datensicherung
Die Sicherung der Daten erfolgt cron-gesteuert auf den RAID-Verbund in eine gesonderte (verschl sselte) Partition. Der S.M.A.R.T.-Status der Festplatten wird via smartd berwacht. Sollte eine Platte kaputt gehen, bestehen gute Aussichten, die Daten zu retten. Eine zuk nftige Option w re auch noch ein RAID-6 Verbund.
NAS / File-Server
Das Ger t verf gt ber bis zu 6 SATA Anschl sse. Davon werden vier standardm ig via Wechselrahmen belegt. Die mitgelieferte 250GB Festplatte wird vorerst das Betriebssystem aufnehmen und an den drei verbleibenden Anschl ssen werden zun chst drei WD Red WD20EFRX 2TB (5400 RPM) Festplatten als RAID-5-Verbund f r den notwendigen Platz sorgen. Letzterer l sst sich ohne Erweiterung nur via Software-Raid und mdadm realisieren.
buildd
Betriebssystem wird Debian GNU/Linux. Der Hauptspeicher wird auf mindestens 8GB ECC-Ram aufger stet.
HTPC (XBMC)
Der Microserver l sst sich nicht als Massenspeicher an einen Fernseher anschlie en. Daher soll vorr. XBMC in Verbindung mit einem USB3.0 BR/DVD-Player den Server zum Entertainment-Ger t erheben.
Das ganze soll m glichst wenig Strom verbrauchen und leise sein. Zum Anschluss an das lokale Netzwerk habe ich mich f r WLAN entschieden, da kein Gigabit-Ethernet vorhanden ist. Folgende Teile ben tige ich f r "meinen" Server:
Server
HP ProLiant N54L MicroServer mit Turion II Neo 2,2 GHz, 2GB RAM/250GB HDD - ca. 200 EUR (lokal)
Bel ftung / Lautst rke
Scythe Slip Stream Geh usel fter 120mm 800RPM 11dB - ca. 9 EUR (SY1225SL12L)
Scythe Slip Stream Geh usel fter 120mm 500RPM 7,5dB - ca. 8 EUR (SY1225SL12SL)
Netzwerk
TP-Link TL-WN722N(C) 150Mbps USB-Adapter - ca. 15 EUR (TL-WN722N(C))
File-Server
3x WD Red WD20EFRX 2TB 5400 RPM SATA600 f r NAS 24/7 - ca. 95 EUR / St. (WD20EFRX)
buildd
8GB (2x4GB) Kingston ValueRAM DDR3-1333 CL9 ECC Modul RAM-Kit - ca. 85 EUR (KVR1333D3E9SK2/8G)
16GB (2x8GB) Kingston ValueRAM DDR3-1333 CL9 ECC Modul RAM-Kit - ca. 145 EUR (KVR1333D3E9SK2/16G)
HTPC
Sapphire Radeon HD 5450/6450/6570/6670/7750 PCIe 16x Low-Profile passiv/aktiv - ca. 25..100 EUR (11166-45-20G, 11190-09-20G, 11191-27-20G, 11191-02-20G, 11192-18-20G, 11202-10-20G)
SILVERSTONE PCIe 1x USB3.0 2xInt 2xExt - ca. 21 EUR (SST-EC04-P)
Logitech K400 od. Keysonic ACK-540RF - ca. 40 EUR (920-003100 bzw. ACK-540 RF)
BR/DVD-Player od. Brenner mit USB3.0 Anschluss - 50..100 EUR
LCD-Mod
LDC Display Modul mind. 4x20 - ca. 10 EUR
Interessant ist auch noch die Option einer echten RAID-Karte. Ich stie dabei auf die IBM ServeRAID M1015 (46M0831) und diesen Hinweis. Kauft man stattdessen den "Schl ssel" zur Freischaltung des vollen Funktionsumfanges, dann bezahlt man (lokal) zus tzlich ca. 150 EUR! Aber das nur BTW. N tzliche Links:

Vincent Bernat: lan o: a task launcher powered by cgroups

A few months ago, I was looking for a piece of software to spawn long-running tasks on behalf of some daemon with the benefit of the tasks not being interrupted when this daemon is restarted. Here are the specifications: The last requirement explains why tasks are not spawned directly by the daemon requesting them: its complexity or the way it needs to be operated may need frequent restarts. Even if it is possible to re-execute a daemon while keeping its children, like stateful re-exec support in Upstart, this is quite difficult: internal state should be serialized and restored. Part of this state can be contained into a third-party library. While Upstart or systemd seemed to be good candidates for this purpose, I didn t find a straightforward way to run arbitrary tasks without any privilege1. Here comes lan o2. It is a very simple task launcher. It can run any tasks, stop them and check if they are still running. It leverages cgroups in recent Linux kernels and avoids the use of any daemon.

Quick tour Before looking at how lan o works, let s have a look at how it can be used. To avoid usage conflicts, each task is run in the context of a namespace that needs to be initialized: 
$ sudo lanco testns init -u $(id -un) -g $(id -gn)
This is the only command that needs to be run as root. Subsequent ones can be run as a normal user. Let s run some task: 
$ lanco testns run first-task openssl speed aes
$ lanco testns check first-task && echo "Still running"
Still running
$ lanco testns ls
testns
   first-task
      28456 openssl speed aes
The output of the task is logged into a file: 
$ head -3 /var/log/lanco-testns/task-first-task.log
Doing aes-128 cbc for 3s on 16 size blocks: 8678442 aes-128 cbc's in 2.85s
Doing aes-128 cbc for 3s on 64 size blocks: 2478283 aes-128 cbc's in 2.99s
Doing aes-128 cbc for 3s on 256 size blocks: 628105 aes-128 cbc's in 3.00s
If the task is too long, we can kill it: 
$ lanco testns run first-task openssl speed aes
$ lanco testns stop first-task
You cannot run a task that already exists or kill a task that does not exist: 
$ lanco testns run first-task openssl speed aes
2013-06-09T22:50:34 [WARN/run] task first-task is already running
$ lanco testns stop second-task
2013-06-09T22:50:45 [WARN/stop] task second-task is not running
Thanks to the use of cgroups, lan o is able to track multiple processes even when they are forking away3: 
$ lanco testns run second-task sh -c \
>  "while true; do (sleep 30 &)& sleep 1; done"
$ lanco testns ls
testns
   first-task
      28456 openssl speed aes 
   second-task
      29572 sh -c while true; do (sleep 30 &)& sleep 1; done 
      29575 sleep 30 
      29593 sleep 30 
      29596 sleep 30 
      29599 sleep 30 
      29622 sleep 30 
      29644 sleep 1 
      29645 sleep 30 
  
$ lanco testns stop second-task
$ lanco testns check second-task   echo "Killed!"
Killed!
Also, there is a top-like command (lanco testns top): lanco top

Using cgroups Control groups (cgroups) is a mechanism to partition a set of tasks and their future children into hierarchical groups with a set of parameters.

Hierarchy Let s start with the hierarchical stuff first. To create a new hierarchy, you have to mount the cgroup filesystem on some empty directory. Usually, cgroups hierarchies are setup in /sys/fs/cgroups which is a tmpfs filesystem: 
# mount -t tmpfs tmpfs /sys/fs/cgroup -o nosuid,nodev,noexec,relatime,mode=755
Now, we can create our first hierarchy: 
# cd /sys/fs/cgroup
# mkdir my-first-hierarchy
# mount -t cgroup cgroup my-first-hierarchy -o name=my-first-hierarchy,none
# ls -1 my-first-hierarchy
cgroup.clone_children
cgroup.event_control
cgroup.procs
notify_on_release
release_agent
tasks
We ll see later the purpose of none. The most interesting file is tasks. It contains the PIDs of all processes in our group. Since there are currently no sub-group, all processes are part of it. Let s create a sub-group and attach a process to it: 
# mkdir first-child
# cd first-child
# ls -1
cgroup.clone_children
cgroup.event_control
cgroup.procs
notify_on_release
tasks
# echo $$ > tasks
# cat tasks
23184
23311
# cat /proc/$$/cgroup 
9:name=my-first-hierarchy:/first-child
8:perf_event:/
7:blkio:/
6:net_cls:/
5:freezer:/
4:devices:/
3:cpuacct,cpu:/
2:cpuset:/
1:name=systemd:/user/bernat/1
We have added our shell to the new cgroup. Moreover, all its children will also be part of this group. This explains why we have two tasks: the shell and cat. The last command is quite interesting: for each hierarchy, it shows which cgroup the task belongs to. For a given hierarchy, each task is exactly in one group. The most useful features of lan o are done using just this: a namespace is a named hierarchy and each task is enclosed into a cgroup so it can be tracked properly.

Subsystems A subsystem makes use of the task grouping provided by cgroups to assign resources to a set of tasks. Each available subsystem can only be in one hierarchy, so the usual way to setup things is to assign a hierarchy for each subsystem4. Let s have a look at the cpuset subsystem: it allows to assign tasks to specific CPUs and memory nodes (for NUMA systems). Let s suppose we have 4 cores and we want to assign the first core to common system tasks and the three remaining ones to nginx: 
# cd /sys/fs/cgroup
# mkdir cpuset
# mount -t cgroup cgroup cpuset -o cpuset
# echo 0-3 > cpuset/cpuset.cpus
# echo 0 > cpuset/cpuset.mems
# mkdir cpuset/system
# echo 0 > cpuset/system/cpuset.cpus
# echo 0 > cpuset/system/cpuset.mems
# for task in $(cat cpuset/tasks); do
>    echo $task > cpuset/system/tasks
> done
# mkdir cpuset/nginx
# echo 1-3 > cpuset/nginx/cpuset.cpus
# echo 0 > cpuset/nginx/cpuset.mems
# for task in $(pidof nginx); do
>    echo $task > cpuset/nginx/tasks
> done
The first mount is not needed if something else (like systemd) has setup the appropriate subsystem. Now, even if some system process is going crazy, it won t affect the performance of your webserver. The kernel comes with a more complete documentation if you need additional details on cgroups. There are high level tools to manipulate them, like the tools provided by libcg, but they are currently quite buggy and invasive.

Use in lan o Here are the features used by lan o:
  • For each namespace, a specific named hierarchy is created.
  • By setting the appropriate permissions on the hierarchy, an unprivileged user can create subgroups.
  • Each submitted task has its own sub-group for tracking purpose.
  • Actions can be executed when a task terminates by using the release agent mechanism5.
  • The cpuacct subsystem is used to track CPU usage: a group is created in this subsystem for this purpose.
  1. Both systemd and Upstart are difficult to run without being PID 1 and without being root. They both support user sessions but this requires to use them as PID 1 as well. I discovered later than runit would have been a good fit: it does not require to be PID 1, it does not need to be run as root and the service directory can be specified by an environment variable. It could have been tweaked to meet the above requirements.
  2. Lan o means launcher in esperanto. The unusual diacritic on the letter c is not present in many web fonts, so the rendering may be a little odd.
  3. Linux cgroups do not provide any facility to kill a whole group. We need to iterate through the processes and kill them one by one. lan o does not try to freeze processes before killing them and therefore may be inefficient against violent fork bombs.
  4. The cpu and cpuacct subsystems are the notable exception. There seems to be little use to keep them in separate hierarchies, so they are part of the same hierarchy.
  5. Unfortunately, only one global release agent per hierarchy can be used. Since lan o provides with the ability to execute an arbitrary command when a task terminates, the command needs to be stored on the filesystem for the release agent of the namespace to execute it.

Justus Winter: Getting into the mood - my second week

tl; dr version: shutdown works, Debian GNU/Hurd booting using sysvinit: 
Loading GNU Mach ...
Loading the Hurd ...
GNU Mach 1.3.99-486
AT386 boot: physical memory map from 0x0 to 0x9f400
AT386 boot: physical memory map from 0x100000 to 0x1fffe000
AT386 boot: physical memory from 0x0 to 0x1fffe000
Enabling FXSR
pcibios_init : BIOS32 Service Directory structure at 0xfcff0
pcibios_init : BIOS32 Service Directory entry at 0xfc7ba
pcibios_init : PCI BIOS revision 2.10 entry at 0xfc78c
Probing PCI hardware.
ide: Intel 82371 PIIX3 (dual FIFO) DMA Bus Mastering IDE
    Controller on PCI bus 0 function 9
ide: BM-DMA feature is not enabled (BIOS), enabling
    ide0: BM-DMA at 0xc100-0xc107
    ide1: BM-DMA at 0xc108-0xc10f
hd0: got CHS=762/128/63 CTL=c8 from BIOS
hd0: QEMU HARDDISK, 3001MB w/256kB Cache, CHS=762/128/63, DMA
hd2: QEMU DVD-ROM, ATAPI CDROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
ide1 at 0x170-0x177,0x376 on irq 15
Floppy drive(s): fd0 is 1.44M
FDC 0 is a S82078B
probing scsi 5/22: Adapteco1542FInvalidiaddresseforishpnt.with 1542.
Invalid address for shpnt with 1542.
probing scsi 18/22:AWesternxDigitalAWD-7000nFailed9initialization of WD-7000 SCSI card!
probing eata on/1f0EATA0:Daddress 0x1f04in3use, skipping probe.
probing eata on 170EATA0: address 0x170 in use, skipping probe.
probing scsi 21/22: Iomega7parport ZIP drive ppa: Version 1.42
ppa: Probing port 03bc
ppa: Probing port 0378
ppa:         SPP port present
ppa:         PS/2 bidirectional port present
ppa: Probing port 0278
done
scsi : 0 hosts.
scsi : detected total.
Partition check (DOS partitions):
 hd0: hd0s1 hd0s2 < hd0s5 >
lpr0: at atbus0, port = 378, spl = 6, pic = 7.
2omultibootxmodulesd/execl$(exec-task=task-create)ne=$ kernel-command-line  --host-priv-port=$ host-port  --device-master-port=$ device-port  --exec-server-task=$ exec-task  -T typed $ root  $(task-create) $(task-resume)               task loaded: ext2fs --readonly --multiboot-command-line=root=device:hd0s1 console=com0 --host-priv-port=1 --device-master-port=2 --exec-server-task=3 -T typed device:hd0s1
task loaded: exec /hurd/exec
start ext2fs: Hurd server bootstrap: ext2fs[device:hd0s1] exec init proc auth
INIT: version 2.88 booting
Using makefile-style concurrent boot in runlevel S.
Activating swap...done.
Checking root file system...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
ext2fs_check_if_mount: Can't check if filesystem is mounted due to missing mtab file while determining whether /dev/hd0s1 is mounted.
/dev/hd0s1: clean, 43930/181056 files, 281248/723200 blocks
done.
Cleaning up temporary files... /tmp.
/etc/rcS.d/S06mtab.sh: 48: /etc/rcS.d/S06mtab.sh: cannot open /proc/mounts: No such file
Activating lvm and md swap...(default pager): Already paging to partition hd0s5!
done.
Checking file systems...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
done.
Mounting local filesystems...mount: invalid option -- 'O'
Try  mount --help' or  mount --usage' for more information.
failed.
Activating swapfile swap...(default pager): Already paging to partition hd0s5!
done.
df: Warning: cannot read table of mounted file systems: No such file or directory
Cleaning up temporary files....
Configuring network interfaces...inetutils-ifconfig: invalid arguments
ifup: failed to open pid file /run/network/ifup-/dev/eth0.pid: No such file or directory
Internet Systems Consortium DHCP Client 4.2.2
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
Listening on Socket//dev/eth0
Sending on   Socket//dev/eth0
DHCPDISCOVER on /dev/eth0 to 255.255.255.255 port 67 interval 6
DHCPREQUEST on /dev/eth0 to 255.255.255.255 port 67
DHCPOFFER from 10.0.2.2
DHCPACK from 10.0.2.2
can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
bound to 10.0.2.15 -- renewal in 38544 seconds.
done.
Cleaning up temporary files....
Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix.
INIT: Entering runlevel: 2
Using makefile-style concurrent boot in runlevel 2.
Starting enhanced syslogd: rsyslogd.
Starting deferred execution scheduler: atd.
Starting periodic command scheduler: cron.
Starting system message bus: dbusFailed to set socket option"/var/run/dbus/system_bus_socket": Protocol not available.
Starting OpenBSD Secure Shell server: sshd.
GNU 0.3 (debian) (console)
login:
This is the result of: This yields a surprisingly functional Debian/Hurd system with sysvinit. There are still lot's of loose ends, I populated my status page with them. I spent quite some time in #hurd on freenode and Pino Toscano offered to send me some old notes of his about the sysvinit issue. This turned out to be most helpful as his notes were about plugging sysvinits init into the boot process. This was the next logical thing to do for me, so first thing Monday morning I just put his runsystem.sysv file with some very minor tweaks (my version) into my overlay and had a working system in front of my eyes. That certainly was a nice way to start a week of work :), so many thanks again Pino! For your convenience I've created an overlay with all necessary changes and recompiled binaries. Be careful though, you might want to backup /sbin/reboot first or you won't be able to reboot your system! Of course my tool does that for you, and I've managed to create a reasonably small (253 mb) image thanks to the zerofree tool, the process of compacting images is implemented in hurdtest as the compact subcommand. I also added sysvinit specific tests. So if you want to try hurdtest, download the image and unxz it, my tool expects the image to be in the current working directory and be named image.qcow2. Download and untar an overlay and execute: 
% hurdtest -logfile log -sysvinit overlay path/to/second-week-overlay
If you specify -logfile you can execute tail -f path/to/log to see what the VM is printing to the console in real time. The sysvinit specific tests demonstrate that indeed sysvinit is managing the system and that e. g. init 0 or shutdown -r now works. There are at least three major problems that I will need to tackle for this project: I think the last issue is what I'm going to look at next week. There's information about this in the Hurd wiki and I've spoken with Richard Braun about it and he drilled me to think about the hurdish way to solve this. I think I've got a pretty good picture of how to implement this in my head and am eager to give it a go. This will involve adding a function to Hurds filesystem interface that allows one to query an active translator about his mount options and "source" and about all active and passive translators that are attached to one of its nodes. This is where the fun actually begins. This is changing an integral part of an operating system, but this is okay since it is just another program written in c, operating in userspace. But hey I know that, that's just c and if something goes wrong (and it always does) I fire up gdb and have a look. And mind you, I'm running all this as an unprivileged user, no need to jeopardize the entire system for that. Besides implementing /proc/mounts I will also add -O, --test-opts to mount, and -v, --verbose to swapo n,ff . PS: My blog is still not properly registered to planet.d.o. I asked planet@ about that, but got no response. Any help? I never imagined blogging being so stressful, I care about losing potential readers :/

Roland Mas: FusionForge news, June 2013

Once again, it's been some time since the last FusionForge update in here. The main explanation is that news is slow on that front. Debian Wheezy was released without FusionForge packages, as previously announced, and even Sid hadn't seen any update on the package for way too long. The latter has just been fixed though: the freshly-released 5.2.2 upstream version is on its way to Sid (via NEW, since it adds a new plugin to allow authenticating against a CAS server). If and when it reaches Debian Jessie (the current testing ), I'll work on backporting it for Wheezy. At some point I'll also start uploading snapshots of the upstream master branch to experimental , to give adventurous users a glimpse of what is to come in the future releases, although I'll stick to only uploading when the automated tests all pass.

Russ Allbery: Review: This Alien Shore

Review: This Alien Shore, by C.S. Friedman
Publisher: DAW
Copyright: 1998
Printing: July 1999
ISBN: 0-88677-799-2
Format: Mass market
Pages: 564
Jamisia is an orphan taken in by the Shido corporation, but she has a fairly comfortable and happy life in Shido Station. She lives in the care of her tutor, has friends, has something that feels like a relatively normal life. Then the station is attacked and destroyed by a rival corporation and she barely escapes in a one-person lifeboat, sent inexplicably by her tutor with a fake identity to rendezvous with a spaceship heading for the up-and-out. That begins a long and quite confusing flight away from everything she's known, and a discovery that her life is far more complicated than she thought. Meanwhile, far from the backwater of corporate tyranny that Earth has become, a new computer virus is wreaking havoc on Guild outpilots. The Guild navigate the ainniq: a sort of subspace that allows ships to slip between nodes without regard to distance. This is the only acceptable method of interstellar travel now that humans know that the more conventional Hausman Drive causes massive genetic mutations, and only Guerans have the secret of navigating it safely. But a virus of a sophistication no one has seen before appears to be targeting outpilot brains through their implants. This Alien Shore is told in two threads but multiple characters and viewpoints. Jamisia (yes, I stumbled over that name the entire book) spends nearly the whole book running while discovering that her head is a very complicated place for very bad reasons. Meanwhile, various other people (most notably the careful and meticulous Kio Masada) are analyzing and tracing the virus, which is of deep concern to the powers that be in the Guaran-controlled network of human colonies. The threads do come together eventually, helped by a young and much more impulsive hacker named Phoenix who is also on the trail of the virus, but it takes much longer than I expected. The book is a bit of cyberpunk, a bit of space opera, and is stuffed to the brim with world-building ideas. There's always something that feels slightly emotionally off about C.S. Friedman novels. Her characters are a bit too intense, a bit too full of primary colors and inherent natures. They don't do a lot of changing; they're just a little too much purely themselves to feel quite accurate. Sometimes it matters and sometimes it doesn't. This is one of the books where it mostly doesn't, partly because it's full of such an amazing variety of characters, and partly because Friedman does a great job adding situational complexity by playing them off of each other and showing them through each other's eyes. Pyschological realism, though, this isn't, nor would I look to Friedman for that. That's somewhat unfortunate given how central psychology is to this book. The focus that slowly emerges from this novel is the whole-hearted embrace of psychological difference as a force for creativity, power, and perspective. Acceptance, respect for, and integration of difference borders on a religious belief for Guerans, which makes this book one of the most wholeheartedly inclusive stories (at least for psychology) I've read. I loved that message enough to overlook the fact that those psychological differences are not treated with much subtlety. The most central of the differences (a very mild spoiler, but one that's essentially given away on the back cover and that is easy to guess early in the book) is multiple personalities. Sadly, I thought that was also the least successful. Friedman treats the topic respectfully and as a difficult asset, which I liked, but it's bog-standard fictional multiple personalities. The presentation owes much more to popular fiction than to the reality of disassociative identity, and the notes that Friedman hits (and the characteristics that the multiples embody) are predictable. However, her second major psychological idea is simply brilliant. The Guerans have developed a coding system for people to communicate their psychological profile to anyone who sees them, and all of the Gueran characters in the novel actively engage with that system and make extensive use of it. Friedman hints at a great deal of subtlety: a wealth of primary patterns (many of which are described in quite enjoyable animal metaphors as chapter epigraphs), secondary patterns that can be layered underneath, and a complex interaction of base patterns and patterns for daily mood. Here, I think Friedman's mild tendency towards archetypal characters helps, since it makes the psychological coding system clearer for the reader. I wish it had been more central to the story and explained further, although I can see the risk of turning the novel into an RPG sourcebook. (Although, now that I mention it, I would love to read an RPG sourcebook for this universe.) The other point for which I have to give Friedman huge credit is that at least half of the book is about a computer virus, the analysis of that virus, and the hacking culture of her future universe, and at no point during the story did I want to start throwing things. There are no completely unbelievable magic computer programs, the sense of movement of data and virtual presence through computer networks seemed vaguely plausible and less steeped in unrealistic VR metaphors, the sense of aesthetics of code is captured remarkably well for a story of this sort, and Friedman even brings some variety to her computer-savvy characters. There are a few mildly stereotyped young hackers (although even those have a tendency to have more depth than the stereotype), but Friedman also populates her electronic world with the other types you rarely see in these novels: thoughtful, brilliant, but meticulous researchers, competent technicians who don't engage as much on the aesthetic aspects, and even moderately-intelligent users who pay attention to the computers their lives are built around. I could actually believe in both these people and the software they use with only the suspension of disbelief required for any future SF. That's not to say there are no flaws, quibbles, or places where computers probably won't ever work that way. But that's part of what makes me so impressed. Friedman is still using computers and hacking in the direct service of a fairly typical SF plot to achieve specific dramatic ends and to create a mystery. She's not building the story around actual technology; she's staying fairly close to the sort of technology and plots that a lot of cyberpunk and hacking novels have. And yet she manages not to make the computing entirely stupid or unbelievable. It's existence proof that an SF author can pay attention to experts and research without compromising the story they want to tell. I want to take this book and smack Robert Sawyer upside the head with it. This is a great idea book with a heavily computer-driven plot strand that I quite enjoyed. It is, however, not without its flaws. The virus-hunting part of the plot is significantly more interesting than Jamisia's flight, particularly given that she keeps running, and running, and running, encountering various people who seem important and end up not being important at all. It takes more than half the book before anything particularly interesting happens to her; up until then, I was dragging myself through her segments so that I could get back to Masada and the other Guerans. She's also annoyingly dim about some things. I figured out what the deep secret in her brain was within the first four or five chapters (it's entirely obvious from the political structure of Friedman's universe), and I have no idea why it took everyone else so long to figure it out. There is also, as mentioned, the slightly off feeling of the characters that I find typical of Friedman, and a cut-throat power struggle at the heart of the book that just didn't have enough psychological believability to fully engage me. But the ideas are wonderful: technology-driven political structures, corporate-controlled states, ideological clashes, lots of believable cultural interactions, and a neat two-option interstellar travel system (even if the nature of the Hausman Drive damage bears no resemblence to any genetic science known to our reality). And that's on top of a wonderful visual language of psychology and discussions of computer security that I actually enjoyed reading. Friedman's the sort of author who won't be to everyone's tastes. You have to have some tolerance for her tendency to write grand clashes of power between highly flawed, aggressively emotional people. Sometimes you have to put up with rather a lot of setup. But I really liked this novel. I suspect I'm editing out a lot of the boring bits in my memory, but I fondly remember that edited version as a ton of fun. Rating: 8 out of 10

Paul Wise: OpenPGP keysigning: alternate encodings for fingerprint exchange

I think that hexadecimal is a fairly poor pre-encoding for information exchange via data to speech and speech to data engines (aka voice boxes, brains and fingers). Reading out and typing long strings of hexadecimal digits at OpenPGP keysignings is tedious and annoying. There have been some experiments using photography and QR codes for this, which I think is pretty cool but not always practical since not everyone has a camera and QR code software installed. An alternative to this might be to pre-encode using a different scheme that encodes to less words in English speech. Diceware is one possibility that I recently experimented with. Diceware is a password generation scheme that encodes data from a random number generator (aka some dice) using a list of 7776 words. Each word thus represents a 5 digit number in base 6. Diceware is mainly used for generating strong and easier to remember passwords. So Diceware is the coupling between a non-digital random number generator and an interesting encoding scheme. Below are my fingerprints in Diceware and Hexadecimal form. The Diceware form is longer to type at 69 characters, 40 for hex. The Diceware form has the advantage that it is only 16 words to say while the hexadecimal form is 40. I don't know if this will be more practical than hex but I can almost remember my entire fingerprint after reading it a few times so hopefully that will translate to practical use. A rough script for encoding your fingerprint in the Diceware encoding is available but I haven't implemented the reverse yet. I would be glad if someone could check it for correctness. 
Diceware:    frame maze bear usgs deter wag prissy bush hoyt mayo upton child indy
Hexadecimal: 610B 28B5 5CFC FE45 EA1B  563B 3116 BA5E 9FFA 69A3
If you want to discuss this topic and try it out in person and attempt to understand my accent, I'll be at DebConf13 and OHM2013. Some downsides that I can think of are accents, multiple spellings and the inclusion of non-words and special characters in the wordlist. These can be solved by using a different wordlist created specifically for OpenPGP fingerprint exchange that only includes suitable words. Update: there is a standard called PGP Word List that is much better than Diceware for this purpose. It maps 256 words to individual bytes, with different lists for odd and even bytes. Unfortunately it doesn't appear to be supported by GnuPG or signing-party. This post was inspired by the screenshots for RedPhone. You can comment on this post on debian-project.

Matt Zimmerman: Scaling Human Systems: Making and keeping commitments

This is part 3 in a series on organizational design and growth. What it means

Taking responsibility for action, and following through. Every time. Why it s important

Commitments are a cornerstone of collaborative teamwork. Knowing what to expect from others makes it possible to plan beyond your own work. If you aren t sure whether a task will be completed, you carry that uncertainty as low-grade anxiety, which accumulates and creates cognitive load. You may even spend extra time checking back to make sure the need has been met. Old status quo

Most work happens without an explicit commitment at all. Consistency is highly variable from one individual, team or circumstance to another. Tasks fall through the cracks, and we may not even realize it until much later. New status quo

When something needs to get done, someone agrees to take responsibility for it. When this happens, everyone involved can trust that it will get done. When exceptions happen, we acknowledge them, and seek to understand what happened so that we can do better in the future. Behaviors that help

Obstacles that stand in our way Further reading

27 June 2013

Matthew Garrett: Automating HP server BIOS setup

One of the goals of our work at Nebula is making it as easy as possible for someone to set up a private cloud. In an ideal world that would involve being able to just plug in the controller hardware, wire up a rack of servers and turn everything on, but right now there are cases where the default firmware configuration on the servers doesn't match our desired configuration. Plugging a console into individual servers just to set some BIOS options is (based on personal experience) about as much fun as writing a doctoral thesis on the experience of watching paint dry, so it seemed worth trying to find a way to avoid people having to deal with that. Thankfully, it turns out that the industry has come to a similar set of conclusions. Recent Dell hardware lets you use WS-MAN, which makes it easy to do things like enable security features as long as you have an authenticated connection to the iDRAC management system. This actually works out wonderfully - the first time a Dell node sends a PXE request to the Cloud Controller, it can push back some configuration changes, reboot the system and then boot it in a known good configuration. Thanks, Dell![1].

HP's slightly trickier. As far as I've been able to work out, the remotely-available programmatic interfaces only provide configuration interfaces to the iLO device itself, along with a range of chassis-level monitoring. Instead, HP provide a couple of utilities that allow the OS to change values. The first of these is called conrep. Conrep is able to modify BIOS settings, but needs an XML file which tells it which configuration values are at which addresses. That's a bit of a pain. Thankfully it's being replaced with a new tool called hprcu which is able to directly query the firmware in order to figure out which configuration options are available and which values have to be stored where in order to set them. You run hprcu once and it spits out a file containing your existing settings. You modify that, feed it back into hprcu, reboot and you're set.

Sounds pretty ideal. What's the problem? The first is that hprcu is currently only shipped as a 32-bit binary, and right now we don't deploy any 32-bit support code on our nodes. It'd be irritating to have to do so just to configure the firmware. The second is that it works by doing raw port IO, and that's not going to be possible from userspace once we've moved to a UEFI Secure Boot setup.

So, obviously, I've been working on reimplementing it. The first step was to figure out what it was actually doing. The first step was to strace it to see if it was using the kernel IPMI interface. strace showed no accesses to /dev/ipmi*. My next thought was that it could be using some sort of shared memory segment with the iLO hardware, so I used MMIOTrace to dump the memory accesses it made. Turned out there weren't many, and certainly not enough to do anything interesting. Then I went back to the strace logs and saw that it was calling iopl() a bunch, and then I got sad.

iopl() allows a userspace application to change its io privilege levels. The Linux manpage for the iopl() call is amazingly helpful, informing us with a straight face that "This call is necessary to allow 8514-compatible X servers to run under Linux." What it actually does is grant userspace access to the full range of io ports.

So, what's an io port? The x86 architecture (and some others) provide two ways to communicate with hardware. The one that's mostly used these days is to map the hardware into the same address space as memory (memory-mapped io), but x86 has an entirely separate address range that can be used. This is significantly more limited, as only 65,536 addresses are available for the entire system. Further, you can only read or write a maximum of 4 bytes in a single transaction. However, while slower and less generally useful than memory-mapped io, port io has the advantage that it's much simpler to implement. As a result, a lot of the original PC hardware was intended to be accessed via port io, and a bunch of that's still present. Want to reprogram your real-time clock? Port io. Want to read from the legacy keyboard controller? Port io. For low-bandwidth transactions, it's a completely reasonable way to implement hardware.

Applications[2] perform port io by executing in and out instructions. Since these instructions allow you to do things like, say, hit the keyboard controller directly[3], userspace is generally forbidden from calling them. Sufficiently privileged applications can call iopl() to raise their privileges and gain access to the full set of io ports. But, since in and out are CPU-level instructions, the actual port io accesses won't show up in strace. So I had to take another approach.

The correct way of doing this would be to LD_PRELOAD something that intercepted iopl(), didn't call it, let the application perform an in or out instruction, caught the fault, looked at the stack frame, called iopl(), performed the access, dumped debug details, dropped iopl() again and restored state. Because that seemed like a bunch of work, I took advantage of the fact that hprcu had separate in and out functions and used gdb. I told gdb to break on every call to in or out, dump the register state and then continue. Then I hacked up a script to parse the register dumps and tell me where the reads and writes were going. Astonishingly, it worked. And then I read the results and got really sad.

PCI setup is hard, if you're a BIOS. You've got PCI devices with large memory windows and you've got to arrange them all somehow and look you've only got 640K of RAM you can access while you're doing this and come on, seriously? So PC-type systems define a port IO interface to performing PCI configuration. Each PCI device has an address made up of a bus, a device and a function. You take a 32-bit integer, set the top bit to indicate that you want PCI access, put the bus in bits 16-23, the device in bits 11-15, the function in bits 8-10 and the configuration register on that device you want to access in bits 0-7. Then you write that to io port 0xcf8. Reads or writes to io port 0xcfc then read or write from that configuration register on that device. This lets the BIOS tell each PCI device where it's going to live in mmio address space without having to actually perform any mmio. Good work, BIOS developers.

This works fine in the BIOS, because nothing else is running while the BIOS is. It even works fine in the kernel, which uses this approach on older hardware that doesn't provide a memory-mapped mechanism to access PCI configuration registers. But it works really badly if you're doing it in userspace, because it shares a problem with all other indexed accesses. You write an address to 0xcf8. You write a value to 0xcfc. What happens if the kernel writes to 0xcf8 before you write to 0xcfc? Your access goes to some other register on some other device and suddenly you've just mapped a PCI device over the top of RAM and well I sure hope you saved all your work.

So, I wasn't thrilled to discover that hprcu was communicating with the iLO by using 0xcf8/0xcfc port io accesses. Not only was it going to stop working once we started deploying UEFI Secure Boot, it had the potential to cause really annoyingly hard to debug problems. Working out how to reimplement it became something of a priority.

By looking at the XML files it generated, and by following the port IO accesses it was performing, I could figure out pretty much how it worked. Some configuration values are stored in the real-time clock CMOS. These were just done in the standard way - write the address to port 0x70, read or write the value to port 0x71[4]. Other configuration values were stored in NVRAM, with accesses going via PCI. This was a little trickier to figure out, because sometimes different NVRAM addresses went to the same PCI address. I finally figured it out - there's a 48 byte window into NVRAM via the PCI configuration registers, and another register which chooses which set of NVRAM is visible. So, take the NVRAM address, divide by 48, write that value to register 0xa6, take the modulus, add it to 0x80 and write the desired value to that address.

So now I knew enough to be able to take an existing XML file and deploy it. Definitely progress, and I could even add support to the kernel. But I still wanted to know how hprcu generated that XML file in the first place. Running strings over the binary showed a bunch of debug output that it never actually printed, but immediately after the help text it also printed SsLlFf:HhAaTtOoDd. I've lost enough years of my life to this kind of thing to be able to identity that as a getopt format string, and there was that tantalising D option at the end that went entirely unmentioned in the help text. Re-running hprcu with the -D argument gave me huge piles of debug output, including references to a DMI entry and an $RBS table.

DMI is a standard for exposing system information, and one aspect of it is providing data from the firmware to the operating system. The OS can look for a defined signature in a fixed area of memory and then pull out a bunch of data about the hardware, including things like the vendor name, model, serial number, BIOS version and more. Some of these information tables are standardised and some are vendor-specific. hprcu was looking for a vendor-specific table and then scanning for it looking for a known signature. That table turned out to to contain a set of signatures and addresses. The address corresponding to the signature hprcu was contained a bunch of data starting with "$RBS". It also contained a huge quantity of ASCII that looked awfully like the strings in the XML file that hprcu had written. Success!

So, the rest of today was spent on working out the format of this table. This was only marginally less tedious than setting up BIOS settings on 20 servers by hand, but I've made enough progress to be able to figure out how to write a kernel-level driver for this. That's about half-done now - the parsing code is all implemented, I just need to add the sysfs glue and I am nowhere near drunk enough for that right now[5], so for now you're just getting a format description.

To start with, look for a DMI table with a type of 0xe5. This contains several entries of 4 bytes of signature, 8 bytes of address and 4 bytes of length. Find the entry with a signature of "$CRQ" and map the corresponding address. The first four bytes of the mapped region should be "$RBS". The first record is 21 bytes into the file. Each record starts with a single byte representing the type and two bytes representing the record length. Records of type 1 contain an ascii string that starts 8 bytes into the record, and have a further type number 6 bytes into the record that tells you what type of string it is. 0x05 indicates that it's a "Feature string", which defines the overall classification of the following features. 0x06 is an "Option string", which provides a human readable explanation of what this configuration value corresponds to. 0x60 is an "Optional warning string", which tells the user which further configuration may be required before the configuration change takes effect.

All other types appear to be configuration options. Bytes 3 to 6 provide the little-endian name of the configuration option. Bytes 7 and 8 provide a unique numberical identifier for the configuration option. Byte 13 indicates the type. 0x03 is stored in CMOS. 0x04 is stored in NVRAM. 0x05 appears to be some different kind of CMOS store. I haven't figured out the others. There then follows a set of choices. These vary in length depending on the type - 0x03 are 14 bytes long, 0x04 are 6 bytes long, 0x05 are 5 bytes long. Type 0x03 choices have the choice id at byte 0, the CMOS offset at byte 2, the mask at byte 3 and the value at byte 4. Type 0x04 choices have the choice id at byte 0, the nvram offset at bytes 1 and 2, the mask at byte 3 and the value at byte 4. Type 0x05 choices have the choice id at byte 0, the cmos offset at byte 1, the mask at byte 2 and the value at byte 3. There are optional flags following each choice - if the final byte of the choice isn't 0, there then follow 6 bytes of flags. The first four bytes provide the name of a configuration option (little-endian, again), the fifth byte refers to a choice on that option and the sixth byte indicates what kind of flag. 0x1 appears to indicate that the option is mutually exclusive with that other option. Examples include the embedded serial port and virtual serial port options, where it's impossible to map both to the same address at once. A choice can have multiple flags.

After a configuration option, there'll be a set of option string records. In turn, these correspond to the previous choices - if a configuration option had three choices, there'll be three option string records. I haven't figured out whether there's a stronger way of binding the option string records to the configuration choice yet. The other thing I haven't entirely figured out are the details of configuring the platform to perform a cold reboot on the next power cycle, but the io traces for that don't look too bad.

The aim is to provide a kernel driver that exposes all these configuration options via sysfs, including indicating the current value, the available values and letting the user set a new value. In an ideal world we'd have a wonderfully generic interface to this kind of functionality, but I'm (sadly) not sure that that's possible.

And that's how I spent the past two days.

[1] (Thell)
[2] Or, indeed, kernels
[3] And, hence, read your keystrokes
[4] Which has exactly the same problems as the PCI access, in that if you happen to ask the clock what the time is while hprcu is accessing CMOS, at least one of you is going to get very confused
[5] Obviously, I would never write kernel code while drunk. You're certainly not running any of it in your enterprise kernel.

comment count unavailable comments

Paul Tagliamonte: Migrating to a feed reader service

I ve moved to using newsblur, a hosted F/OSS RSS feed reader. I really like it a lot, and I highly recommend it to anyone looking for a new feed reader. I also strongly encourage you to donate / pay for an account, it s rare we get a hosted F/OSS solution like this, and I d like to see more of it. That s all, really.

26 June 2013

Benjamin Mako Hill: Lookalikes

sacher_pde Is Franz Sacher, the Inventor of the famous sachertorte, still alive and and working at the at the Electronic Frontier Foundation? Might this help explain why EFF Technology Project Director Peter Eckersley is so concerned about protecting privacy and pseudonymity?

Next.

Previous.