Search Results: "ron"

6 January 2015

Holger Levsen: 20150106-lts-december-2014

My LTS December In December 2014 I spent 11h on Debian LTS work and managed to get six DLAs released and another one almost done... I did: Thanks to everyone who is supporting Squeeze LTS in whatever form! Even just expressing that you or the company or project you're working with is using LTS, is useful, as it's always nice to hear once work is used and appreciated. If you can contribute more, please do so. If you can't, that's also fine. It's free software after all :-)

Russ Allbery: Review: Code Complete, Second Edition

Review: Code Complete, Second Edition, by Steve McConnell
Publisher: Microsoft
Copyright: June 2004
ISBN: 0-7356-1967-0
Format: Kindle
Pages: 960
As mentioned in the title, this is a review of the second edition of Code Complete, published in 2004. There doesn't appear to be a later edition at the time of this writing. I should say, as a prefix to this review, that I'm the sort of person who really likes style guides. When learning a language, a style guide is usually the second or third document I read. I enjoy debates over the clearest way to express a concept in code, trying to keep all the code in a large project consistent, and discussing the subtle trade-offs that sit on the boundary between mechanical style issues and the expressiveness of programming. I try to spend some time reading good code and getting better at expressing myself in code. Presumably, therefore, I'm the target audience for this book. It sounded good from the descriptions, so I picked it up during one of the Microsoft Press sales. The stated goal of Code Complete is to collect in one place as much as possible of the oral tradition and lore of the programming field, to try to document and communicate the techniques and approaches that make someone a good programmer. The table of contents sounds like a style guide, with entire sections on variables and statements in addition to topics like how to improve existing code and how to design a new program. If you're starting to think that a 960 page style guide sounds like a bad idea, you're wiser than I. (In my defense, I grabbed this as an ebook and didn't realize how large it was before I bought it.) I have not actually finished this book. I hate to do this: I don't like reviewing books I haven't finished (this will be the first), and I hate starting books and not finishing them. (This is probably not particularly wise, since some books aren't worth finishing, but I've gotten into a rhythm of reading and reviewing that works for me, so I try not to mess with it.) But I've been trying to finish this book off and on for about a year, I don't think it's worth the time investment, and I think I've gotten far enough in it to provide some warnings by others who are deceived by the very high ratings that it gets on Amazon and other places. The primarily problem with Code Complete is its sheer, mind-numbing comprehensiveness. It tries to provide a set of guidelines and a checklist to think about at each level of writing code. This is one of those ideas that might sound good on paper, but which completely doesn't work for me. There is no way I'm going to keep this many rules in my head, in the form of rules, while programming. Much of good style has to be done by feel, and the book I'm looking for is one that improves my feel and my sense of taste for code. What Code Complete seems to provide instead is a compilation of every thought that McConnell has ever had about programming. There's a lot of basic material, a few thoughtful statements, a ton of style advice, an endless compilation of trade-offs and concepts that one should keep in mind, and just a massive, overwhelming pile of stuff. Each chapter (and there are a lot of chapters) ends in a checklist of things that you should think about when doing a particular programming task. To give you a feel for the overwhelming level of trivia here, this is the checklist at the end of the chapter where I stopped reading, on quality assurance in software. This is one picked at random; a lot of them are longer than this.
  • Have you identified specific quality characteristics that are important to your project?
  • Have you made others aware of the project's quality objectives?
  • Have you differentiated between external and internal quality characteristics?
  • Have you thought about the ways in which some characteristics might compete with or complement others?
  • Does your project call for the use of several different error-detection techniques suited to finding several different kinds of errors?
  • Does your project include a plan to take steps to assure software quality during each stage of software development?
  • Is the quality measured in some way so that you can tell whether it's improving or degrading?
  • Does management understand that quality assurance incurs additional costs up front in order to save costs later?
I'm not saying those are bad things to think about with quality assurance, but you may notice a few issues immediately. They're very general and vague, they're not phrased in a particularly compelling or memorable way, and there are a lot of them. This falls between two stools: it's too much for the programmer who is thinking about quality as part of an overall project but not focusing on it (particularly when you consider that the book is full of checklists like this for everything from variable naming to how to structuring if statements to program debugging), but it's not nearly specific or actionable enough for someone who is focusing on quality assurance. It's not that the information isn't organized: there's a lot of structure here. And there are bits and pieces here that are occasionally interesting. McConnell is very data-driven and tries to back up recommendations with research on error rates and similar concrete measurements. It's just insufficiently filtered and without elegant or memorable summary. There is far too much here, an overwhelming quantity, and hopelessly mixed between useful tidbits and obvious observations that anyone who has been programming for a while would pick up, all presented in the same earnest but dry tone. It didn't help that there's a lot here I didn't agree with. Some of that is to be expected: I've never agreed completely with any style guide. But McConnell kept advocating variable and function naming conventions that I find rather ugly and tedious, and the general style of code he advocates feels very "bureaucratic" to me. It's not exactly wrong, but one of the things that I look for in style discussions is to be inspired by the elegant and simple way someone finds to phrase something in code. A lot of the code in this book just felt mind-numbing. It's functional, but uninteresting; perfectly adequate for a large project, but not the sort of discussion that inspires me to improve the quality of my craft. So, I didn't finish this. I gave up about halfway through. It's frustrating, since I was occasionally finding an interesting nugget of information. But they were too few and far between, and the rest of the book was mostly... boring. It's possible that I just know too much about programming to be the person for whom that McConnell was writing this book. It's certainly true that the book has not aged particularly well; it's focused on fairly old-school languages (C, C++, Java, and Visual Basic) and says almost nothing about modern language techniques, although it does have a bit about extreme programming. But whatever the reason is, it didn't work for me at all. I would rate it as one of the worst books about programming I've tried to read. And that's notably different enough from its reviews that it seems worth throwing this out there as a warning. I'm quite disappointed, since I'd heard nothing but praise for this book before picking it up. But it's not for me, and I'm now dubious of its value for any programmer outside of a fairly narrow, large-team, waterfall development process involving large numbers of people writing very large quantities of code in languages that aren't very expressive. And, well, in that situation I think one would get more benefit from changing that environment than reading this book. Rating: unfinished

5 January 2015

Russ Allbery: Review: Ancillary Sword

Review: Ancillary Sword, by Ann Leckie
Series: Imperial Radch #2
Publisher: Orbit
Copyright: October 2014
ISBN: 0-316-24665-4
Format: Trade paperback
Pages: 354
This is the second book in the Imperial Radch series and a direct sequel to Ancillary Justice. You don't want to read this book out of order, since the previous book sets up the background of everything that happens here. Besides, Ancillary Justice is an amazing book. It's going to be challenging to review Ancillary Sword without spoiling the previous book. If you're planning on reading Ancillary Justice but haven't gotten to it, you may want to stop here and come back to this review after you've read it. Or, even better, just read both books. They're some of the best science fiction I've read. Ancillary Justice started small, with one person and their quixotic search for revenge, and grew large, to encompass conflicts and confrontations that would shake the Radch. Ancillary Sword returns to a smaller scale and stays there. This means that much of what was left unresolved at the end of the previous book is still unresolved; Leckie does not continue escalating into large-scale conflict. It also means that we see a lot more of Breq making personal choices and trying to work out her own sense of morality, plus semi-adopting a couple more injured people along the way. One of my favorite types of stories is where I get to watch someone who is very good at something do the thing that they're very good at. Breq's unique background and experience makes her a wildcard outsider with vast experience in her new role. (Not to mention the special advantages she has from her implants.) Her long experience with people, similarly from a unique perspective, lets her use her power to effectively navigate political situations while keeping people slightly off-balance. And now she has some real power, made more potent for being somewhat ill-defined. In short, this is a story of political agency, given to someone who hasn't had it before but who is very good at using it. It's immensely satisfying, in part because it's not a simple wish fulfillment. Breq can't just reshape the world to her preferences; in fact, she can't do much about one of the social conflicts she runs into, except treat the people involved with unexpected respect. But she can occasionally do something, and she can always upset existing power structures in subtle ways, and the way Leckie writes this makes it so much fun to read. I think one of the reasons why I enjoyed this so much is that Breq is not relentlessly introspective. She just acts. Usually this sort of book involves lots of soul-searching and analysis, and the lack is refreshing. The other people in the story analyze Breq much more than she analyzes herself, sometimes incorrectly, and Breq finds the whole thing faintly amusing. Not only does this keep the story from bogging down in too much internal drama, it means that Breq frequently surprises the reader, usually in ways that had me grinning. And, despite not mulling things over incessantly, she is growing and developing, finding her own sense of morality and and ethics in a way that's sometimes only apparent in retrospect. The one caveat I will mention is that this is a book that concerns itself a great deal with colonialism and racial slavery, but it's a fantasy of political agency focused on someone who's part of the dominant culture. While it's not quite accurate to say that Breq is this world's equivalent of white, she can pass, and she's Radchaai. I thought the book handles the issues reasonably well, but it is still using oppressed cultures to focus on the agency and power of someone who is, comparatively, privileged. This didn't bother me while I was reading the story, but it started to bother me a little afterwards once it was pointed out. There's nothing inherently wrong with that story, but it's a rather common pattern, and I'm afraid Ancillary Sword doesn't do much to broaden the pattern. That said, it's a caveat rather than a fatal flaw, at least for me. Ancillary Sword is obviously the middle book of a trilogy, and normally the lack of forward progress on the overarching story and the sense of filling in background and setting the scene would undermine the book. But Breq and the other characters in this world are so fascinating that I didn't mind. The ending was not quite what I expected, but worked better the more that I thought about it. I'm really looking forward to the next book. Followed by Ancillary Mercy. Rating: 9 out of 10

4 January 2015

Russ Allbery: Review: Programming Ruby

Review: Programming Ruby, by Dave Thomas, et al.
Publisher: Pragmatic Bookshelf
Copyright: 2005
Printing: May 2006
ISBN: 0-9745140-5-5
Format: Trade paperback
Pages: 785
There are a few different editions of this book. The version I read is the second edition, written by Dave Thomas with Chad Fowler and Andy Hunt and published in 2005, covering Ruby 1.8.2. There's now a fourth edition, covering Ruby 1.9 and 2.0, which is probably what you'd want if you were buying this book today. This book, in whatever edition, is called the Pickaxe in the Ruby community after its cover. I've used a lot of different programming languages, so I can usually pick one up on the fly reasonably well, but I still like to read a good introductory book before using one seriously. It's a bit too easy to get lost or to fall into habits that don't match the best practices of the language community without a solid introduction. I've been using a bit of Ruby off and on since I started using Puppet, but I'm looking at doing more serious development using Chef, so I decided it was time to get that introduction. (It helped that I had this book sitting around, although that's also why I read an older edition.) Programming Ruby starts with the obligatory introduction to installing and running Ruby, and then provides a high-level introduction to the language and its basic types just enough to make Ruby comprehensible before starting into the object system. Everything is an object in Ruby, so the book introduces the object system as early as possible, and then shows the rest of the language from constants up in the light of that object system. The rest of part one follows the normal language introduction path, building up from constants and methods to exceptions, modules, and basic IO. It closes with chapters about threads and processes, unit testing, and the debugger. Part two is a grab-bag of one-chapter topics describing how to use Ruby in a particular setting, or showing one angle of the language. The best of those chapters for me was the one on RDoc, partly because I'm quite impressed by Ruby's documentation system. A few of these chapters are oddly in-depth for an introductory book I doubt I'm ever going to use all the details about special irb configuration, and if I do, I'd just look them up but I greatly appreciated the solid chapter on how to write Ruby extensions in C. There is also the obligatory chapter on writing GUI applications with Tk, which always seems to show up in these sorts of introductions and which always baffles me. Does anyone actually do this any more instead of writing a web application? Part three dives back into the language and provides a more complete and formal description. The authors aren't afraid to get into some of the internals, which I appreciated. There is a good chapter here on the details of the type system and how objects and classes interact, and a much-needed extended discussion of duck typing. This type of weak typing and runtime binding is fundamental to how Ruby approaches objects, for better or worse. (I have mixed opinions; it makes some things easier, but I increasingly appreciate strong typing and more formal interface definitions.) Some discussion of marshalling and introspection closes out the discussion portion of the book. That's about 420 pages of the material. The rest of the book is a detailed reference on all of the core classes, and a quicker overview of the standard library. Normally, this sort of thing is thrown into language introductions to pad out the page count, but usually the language's official documentation is better at this sort of reference. But I found Programming Ruby to be an exception. The reference is succinct, sticking to a paragraph or two for each method, and did a great job of providing enough cross-reference and discussion to put each class into a broader perspective. It's the most useful example of this type of reference section I've seen. I still probably won't use it after this initial reading, but I think I got a better feel for the language from reading through it. It's hard to review a book like this without reviewing the language it documents, at least a little bit. I'll indulge: it entertains me how much Ruby is obviously based on Perl, including borrowing some of Perl's more dubious ideas. The global punctuation variables will look familiar to any Perl programmer, and the oddly-named global variables for the interpreter flags are in the same spirit. The language unfortunately has similar problems as Perl with safely running commands without using the shell; it's possible, but not the default and not what the built-ins do. There are places where I wish Ruby were a little less like Perl. The plus side for an experienced Perl programmer is that Ruby feels quite familiar and has made some clear improvements. The ? and ! convention for methods that return booleans or modify objects in-place is brilliant in its simplicity, and something I'd love to see in more languages. And the way Ruby implements ubiquitous code blocks for both iterators and for any temporary objects is lovely once one gets used to it. It's similar to Python's context managers, except more general and built deeper into the language. Returning to the review of the book, rather than the topic, Programming Ruby has a good, clear explanation of blocks, iterators, and yield. If you're interested in getting a grounding in Ruby, this book still feels like a solid introduction. The edition I read is getting a bit long in the tooth now that we're on Ruby 2.1, but the pace of language change has slowed, and most of the book is still applicable. (If you're buying it new, you should, of course, get the later edition.) The table of contents makes it seem like the book is covering the same ground multiple times, but that organizational strategy worked better than I expected. Ruby is not the most organized language in the world, so I still felt a bit overwhelmed with random method names in places, but I never felt lost in the mechanics of the language. In short, recommended if you want a good introduction to the language, although probably in a later edition. Rating: 8 out of 10

3 January 2015

Russ Allbery: Review: The Ring of Charon

Review: The Ring of Charon, by Roger MacBride Allen
Series: Hunted Earth #1
Publisher: Tor
Copyright: December 1990
ISBN: 0-8125-3014-4
Format: Mass market
Pages: 500
Larry Chao is a junior scientist at a gravity research on Pluto, at the very outer limits of human reach in the solar system. The facility is for researching artificial gravity, which is one reason it's in the middle of nowhere. Another is that their experimental generator is built in a ring around Charon, and the close proximity of the two bodies is useful for scientific observation. Unfortunately, there hasn't been much to observe. They can create very short-lived gravity fields in very small areas, but nothing like the artificial gravity that was the original promise of the facility. As a result, the government is shutting the facility down. The authoritarian director, Simon Raphael, is... not exactly happy with that decision, but resigned to it and running the facility to completion with a sullen anger. When Larry makes a startling breakthrough at nearly the last minute for the society, Simon is uninterested and hostile. This leads Larry and his fellow scientist Sondra Berghoff to attempt a more radical demonstration of Larry's success and prove to the rest of the solar system that the facility should be kept open. That decision has a far deeper impact on humanity and the solar system than they could have possibly imagined. The Ring of Charon and its sequel, Shattered Sphere, were recommended to me as good harder science fiction. It took me a while to track down copies in fact, it took an in-person trip to Powell's. Once I found them, a relatively straightforward, old-school science fiction novel seemed like just the thing to read during my commute. Allen delivers there. I'm not spoiling the main plot driver of the book even though it's given away on the back cover, since it's some time in coming in the novel. But The Ring of Charon turns into a multi-viewpoint cross between a disaster novel and a scientific investigation. Larry and Sondra stay central to the plot, of course, but Allen adds a variety of other characters who are attempting to figure out what happened to the solar system and then deal with the consequences: everyone from scientists to pilots to communications officers in weird surrealistic stations. The science is mostly believable, apart from the scientific breakthrough that motivates the plot. Characterization isn't absent completely, but it's simple and unsubtle; dialogue is a bit wooden, characters aren't entirely multidimensional, but Allen does a reasonably good job with both pacing and the sense of mystery and investigation, and a surprisingly good job portraying organizational politics. As you might guess from the tone of my review, this is not the book to reach for if you want something ground-breaking. It's a very conventional, multi-viewpoint SF novel full of scientists and investigation of unknown and possibly hostile phenomena. If you've read much science fiction, you've read books like this before. But one thing that Allen does surprisingly well, which makes The Ring of Charon stand a bit above the pack, is that he doesn't write villains. Even Simon, who goes out of his way to make the reader hate him at the start of the book, becomes a surprisingly sympathetic character. The characters who are usually villains or at least foils in books like this the smooth PR person, the religious man, the blindly-focused scientist who isn't interested in anyone else's theories never turn into caricatures, play important roles in the plot, and turn out to be competent in their fields of expertise. There are actual villains, sort of, but I found myself feeling sympathetic even towards them, at least in places. Allen takes a rather old SF dodge to achieve conflict without an evil enemy, and, because of that, the end of the book felt like a bit of an anticlimax. But I did like the feel of the book where there isn't a good versus evil fight, just a wide variety of people (and others) trying to understand and control the universe in the best ways they know how. I'm not sure I can quite recommend this book. The quality of the writing is not particularly high, and I'm not generally a fan of the disaster novel style of storytelling. But despite not being very original, there's just something likable about it. It moves along reasonably well for a 500 page book, and it's refreshingly light on irritating stereotypes. I think one has to be in the right mood when reading it and set expectations accordingly, but it fit what I was looking for when I picked it up. One warning, though: although The Ring of Charon reaches a climax, the major plot conflict is not resolved at the end of this book, so you may want to have the sequel around. Followed by Shattered Sphere. Rating: 6 out of 10

2 January 2015

Simon Josefsson: OpenPGP Smartcards and GNOME

The combination of GnuPG and a OpenPGP smartcard (such as the YubiKey NEO) has been implemented and working well for around a decade. I recall starting to use it when I received a FSFE Fellowship card long time ago. Sadly there has been some regressions when using them under GNOME recently. I reinstalled my laptop with Debian Jessie (beta2) recently, and now took the time to work through the issue and write down a workaround. To work with GnuPG and smartcards you install GnuPG agent, scdaemon, pscsd and pcsc-tools. On Debian you can do it like this:
apt-get install gnupg-agent scdaemon pcscd pcsc-tools
Use the pcsc_scan command line tool to make sure pcscd recognize the smartcard before continuing, if that doesn t recognize the smartcard nothing beyond this point will work. The next step is to make sure you have the following line in ~/.gnupg/gpg.conf:
use-agent
Logging out and into GNOME should start gpg-agent for you, through the /etc/X11/Xsession.d/90gpg-agent script. In theory, this should be all that is required. However, when you start a terminal and attempt to use the smartcard through GnuPG you would get an error like this:
jas@latte:~$ gpg --card-status
gpg: selecting openpgp failed: unknown command
gpg: OpenPGP card not available: general error
jas@latte:~$
The reason is that the GNOME Keyring hijacks the GnuPG agent s environment variables and effectively replaces gpg-agent with gnome-keyring-daemon which does not support smartcard commands (Debian bug #773304). GnuPG uses the environment variable GPG_AGENT_INFO to find the location of the agent socket, and when the GNOME Keyring is active it will typically look like this:
jas@latte:~$ echo $GPG_AGENT_INFO 
/run/user/1000/keyring/gpg:0:1
jas@latte:~$ 
If you use GnuPG with a smartcard, I recommend to disable GNOME Keyring s GnuPG and SSH agent emulation code. This used to be easy to achieve in older GNOME releases (e.g., the one included in Debian Wheezy), through the gnome-session-properties GUI. Sadly there is no longer any GUI for disabling this functionality (Debian bug #760102). The GNOME Keyring GnuPG/SSH agent replacement functionality is invoked through the XDG autostart mechanism, and the documented way to disable system-wide services for a normal user account is to invoke the following commands.
jas@latte:~$ mkdir ~/.config/autostart
jas@latte:~$ cp /etc/xdg/autostart/gnome-keyring-gpg.desktop ~/.config/autostart/
jas@latte:~$ echo 'Hidden=true' >> ~/.config/autostart/gnome-keyring-gpg.desktop 
jas@latte:~$ cp /etc/xdg/autostart/gnome-keyring-ssh.desktop ~/.config/autostart/
jas@latte:~$ echo 'Hidden=true' >> ~/.config/autostart/gnome-keyring-ssh.desktop 
jas@latte:~$ 
You now need to logout and login again. When you start a terminal, you can look at the GPG_AGENT_INFO environment variable again and everything should be working again.
jas@latte:~$ echo $GPG_AGENT_INFO 
/tmp/gpg-dqR4L7/S.gpg-agent:1890:1
jas@latte:~$ echo $SSH_AUTH_SOCK 
/tmp/gpg-54VfLs/S.gpg-agent.ssh
jas@latte:~$ gpg --card-status
Application ID ...: D2760001240102000060000000420000
...
jas@latte:~$ ssh-add -L
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFP+UOTZJ+OXydpmbKmdGOVoJJz8se7lMs139T+TNLryk3EEWF+GqbB4VgzxzrGjwAMSjeQkAMb7Sbn+VpbJf1JDPFBHoYJQmg6CX4kFRaGZT6DHbYjgia59WkdkEYTtB7KPkbFWleo/RZT2u3f8eTedrP7dhSX0azN0lDuu/wBrwedzSV+AiPr10rQaCTp1V8sKbhz5ryOXHQW0Gcps6JraRzMW+ooKFX3lPq0pZa7qL9F6sE4sDFvtOdbRJoZS1b88aZrENGx8KSrcMzARq9UBn1plsEG4/3BRv/BgHHaF+d97by52R0VVyIXpLlkdp1Uk4D9cQptgaH4UAyI1vr cardno:006000000042
jas@latte:~$ 
That s it. Resolving this properly involves 1) adding smartcard code to the GNOME Keyring, 2) disabling the GnuPG/SSH replacement code in GNOME Keyring completely, 3) reorder the startup so that gpg-agent supersedes gnome-keyring-daemon instead of vice versa, so that people who installed the gpg-agent really gets it instead of the GNOME default, or 4) something else. I don t have a strong opinion on how to solve this, but 3) sounds like a simple way forward.

Gunnar Wolf: Status of the OpenPGP keyring: 1024D is a thing of the past!

Having seen the end of December and the beginning of January, this is the time of year where we say "Happy new year!" But this is a very interesting new year: We have also went past our much announced deadline for the <2048 bit keys to be removed from the Debian keyrings. And yes, our highly efficient keyring-maint team managed to deliver on the promised time And, I'd say, with much success. Lets see the numbers Only before that, refer to Jonathan's mail to debian-devel-announce for further, fuller information. So, first of all, how do overall numbers look? Just remember, the following are not the number of DDs, just the number of active keys. That is, the holders to the 252 DD and 35 DM keys we removed are still valid Debian Developers/Maintainers, but have to get a new key accepted to perform many of their tasks in the project. The graph above shows the sharp change between tags 2014.12.31 and 2015.01.01. But my definition of success is that we managed to get the number down to just 252+35=287 from what we had back in August, when we did our DebConf presentation and started the aggressive push: 490 DD keys and 49 DM keys. Since then, 34 DDs requested their retirement, becoming emeritus, and practically all of the rest managed to get their key transition done! So, lets go again easiest-to-hardest. First, the Non-uploading Debian Developers keyring: As this is the newest keyring in existence, and is also the smallest one, we were already without <2048 keys since 2011. Nothing to see, move along. Then, as for the Debian Maintainers: We did have a sensible migration from weaker to stronger keys, but it was not as sharp as I'd have liked. That makes sense, after all, since DMs have less involvement and compromise in the project in regard to DDs. So, we only processed 15 DM keys since August, which is almost a third of the keys we needed to process to reach the ideal 100% migration. Now, as for our biggest and oldest keyring, and the one that denotes more project involvement, here is the graph for the uploading Debian Developers: And yes, here you can see the sharp turn we saw in the second half of this year: By DebConf time, we were happy because the red and yellow lines had just crossed. But we were still sitting at 490 DD keys needing to be migrated. Half of the DD keys (compared to almost a fourth for the DM keys). I'm almost sure we anticipated in our presentation (I know, I should check the video) that, by January 1st, we would have to retire around 300 keys. And I'm very, very happy and proud that we managed to get the number down to 252. And, yes, people leave things to the end: We already have some more pending requests in the Request Tracker to introduce new keys for our fellow friends who were disabled. We will be working to make keyring pushes more frequent than our usual monthly uploads until requests go back to a sane level. So, if everything runs smoothly, this will probably be the last of my posts in this regard. This has been quite an interesting (and exhausting!) experience!

John Goerzen: Sound players: Adventures with Ampache, mpd, pulseaudio, Raspberry Pi, and Logitech Media Server

I finally decided it was about time to get my whole-house sound project off the ground. As an added bonus, I d like to be able to stream music from my house to my Android phone. Some Background It was about 2.5 years ago that I last revisited the music-listening picture on Linux. I used Spotify for awhile, but the buggy nature of its support for local music eventually drove me up the wall. I have a large collection of music that will never be on Spotify (local choirs, for instance) and this was an important feature. When Google Play Music added the feature of uploading your local collection, I used that; it let me stream music from my phone in my car (using the Bluetooh link to the car). I could also listen at home on a PC, or plug my phone into various devices to play. But that was a hassle, and didn t let me have music throughout the house. Google Play is reasonable for that, but it has a number of really glaring issues. One is that it often gets album artwork wrong; it doesn t use the ID3 tags embedded in the files, but rather tries to guess . Another is that the sync is only add . Move files to another place in your collection, or re-rip them to FLAC and replace your old MP3s, and suddenly they re in Google Play twice. It won t ever see updated metadata, either quite a hassle for someone that uses Musicbrainz and carefully curates metadata. My Hardware I already have an oldish PC set up as a entertainment box running MythTV. It is a diskless system (boots PAE over the network and has NFS root) and very quiet. It has video output to my TV, and audio output via S/PDIF to my receiver. It is one logical audio frontend. My workstation in my office is another obvious place. My kitchen has a radio with a line in jack, and I also have a small portable speaker with a line in jack to make the last two options. I also have a Raspberry Pi model B that I bought awhile ago and was looking for a purpose, so I thought this should be cheap and easy, right? Well. Cheap, yes. Easy, not so much. First attempt: Ampache The Ampache project produces quite a nice piece of software. Ampache has matured significantly in the last 2.5 years, and the usability of its web-interface with HTML5 and Flash players as options is quite impressive. It is easily as usable as Google s, though its learning curve is rather more steep. There are multiple Android apps for Ampache to stream remotely. And, while most are terribly buggy and broken, there is at least one that seems to work well. Ampache can also output m3u/pls files for a standalone player. It does on-the-fly transcoding. There are some chinks in the armor, however. The set of codecs that are transcoded or passed through is a global setting, not a per-device setting. The bitrates are per-account, so you can t easily have it transcode FLAC into 320Kbps mp3 for streaming on your LAN and 128Kbps MP3 for streaming to your phone. (There are some hacks involving IP address ranges and multiple accounts, but they are poorly documented and cumbersome.) Ampache also has a feature called localplay in which it drives local players instead of remote ones. I tried to use this in combination with mpd to drive music to the whole house. Ampache s mpd interface is a bit odd; it actually loads things up into mpd s queue. Sadly it shares the same global configs as the rest. Even though mpd is perfectly capable of handling FLAC audio, the Ampache web player isn t, so you have to either make it transcode mp3 for everything or forego the web player (or use a second account that has transcode everything set). Frustratingly, not one of the Android clients for Ampache is even remotely compatible with Localplay, and some will fail in surprising ways if you have been using Localplay on the web client. So let s see how this mpd thing worked out. Ampache with mpd for whole-house audio The primary method here is to use mpd s pulseaudio driver. I configured it like so:
audio_output
type "pulse"
name "MPD stream"
#server "remote_server" # optional
sink "rtp" # optional
mixer_type "software"

Then in /etc/pulse/system.pa:
load-module module-null-sink sink_name=rtp format=s16be channels=2 rate=44100 sink_properties="device.description='RTP Multicast Sink'"
load-module module-rtp-send source=rtp.monitor
This tells Pulse to use multicast streaming to the LAN for the audio packets. Pulse is supposed to have latency synchronization to achieve perfect audio everywhere. In practice, this works somewhat poorly. Plus I have to install pulse everywhere, which inserts its tentacles way too deeply into the ALSA stack for my taste. (alsamixer suddenly turns useless by default, for instance.) But I gave it a try. After much fiddling Pulse is rather poorly documented and the interactions of configuration tools with it even more so I found a working configuration. On my MythTV box, I added:
load-module module-rtp-recv
load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1;x.x.x.x
The real annoyance was getting it to set the output through the S/PDIF digital port. Finally I figured out the magic potion:
set-card-profile 0 output:iec958-stereo+input:analog-stereo
This worked reasonably well. The Raspberry Pi was a much bigger challenge, however. I put Raspbian on it easily enough, and installed Pulse. But apparently it is well-known in the Pulse community that Pulse s RTP does not work well with wifi. Multicast itself works poorly with wifi in general, and Pulse won t do unicast RTP. However, Pulse assumes very low latency and just won t work well with wifi at all. Morever, pulseaudio on the rpi is something of a difficult beast to tame. It has crackling audio, etc. I eventually got it working decently with: daemon.conf:
realtime-scheduling = yes
realtime-priority = 5
resample-method = src-sinc-fastest
default-sample-rate = 44100
default-fragments = 4
default-fragment-size-msec=20
and in system.pa, commented out the module-udev-detect and module-detect, and added:
load-module module-alsa-card device_id=0 tsched=0 fragments=10 fragment_size=640 tsched_buffer_size=4194384 tsched_buffer_watermark=262144
plus the rtp-recv and tcp protocol lines as before. This got it working with decent quality, but it was always out of sync by at least a few tenths of a second, if not a whole second, with the other rooms. Brief diversion: mplayer Many players besides pulseaudio can play the RTP streams that pulseaudio generates. mplayer, for instance, can. I found this worked on the Raspberry Pi:
mplayer -really-quiet -cache 64 -cache-min 95 -demuxer rawaudio -rawaudio format=0x20776172 rtp://224.0.0.56:46510
But this produced even worse sync. It became clear that Ampache was not going to be the right solution. Logitech Media Server I have looked at this a few times over the years, but I ve somehow skipped it. But I looked at it again now. It is an open source (GPL) server, and was originally designed to work with Logitech hardware. There are now all sorts of software clients out there. There is a helpful wiki about it, although Logitech has rebranded the thing so many times, it s not even consistent internally on what the heck it s called (is it LMS? or Squeezebox Server? or Slimserver? The answer is: yes.) LMS does no web-based streaming. At all. But I thought I d give it a try. Installation is a little weird (its .deb packages up binary perl modules for things that are already in Debian, for half a dozen architectures and many Perl versions.) I had some odd issues but eventually it worked. It scanned my media collection. I installed squeezelite on my workstation as my first player, and things worked reasonably well out of the box. I proceeded to install squeezelite on the Raspberry Pi and MythTV server (where I had to compile libsoxr for wheezy, due to an obscure error I couldn t find the cause of until I used strace). Log in to the LMS web interface, tell it to enable synchronization, and bam! Perfect synchronization! Incredible. But I ran into two issues: one was that it used a lot of CPU even on my workstation (50% or more, even when idle). Strangely, it used far less CPU on the pi than did pulseaudio. Secondly, I d get annoying clicks from it from time to time. Some debugging and investigation revealed that they were both somewhat related; it was getting out of sync with the pi and correcting. Initially I assumed this to be an issue with the pi, but tracked it down to something else. I told it to upsample to 48kHz and this made the problem go away. My command on PC hardware is:
squeezelite -a 200:10 -o dmix -u hLX -z
And on the Raspberry Pi:
squeezelite -a 800:10:16:0 -n kitchen -z
It is working perfectly now. LMS even has an Android remote control app that can control all devices together, queue up playlists, etc. Very slick. The Raspberry Pi The Raspberry Pi, even the model B+, apparently has notoriously bad audio output. I noticed this first as very low volume on the output. I found a $11 USB DAC on Amazon that seems to do the trick, so we will see. I am also using the $10 Edimax USB Wifi and it seems to work well. Other Options The big-name other option here is Sonos. They have perfect sync audio already working. But they re pricy, closed, and proprietary; you have to buy Sonos speakers, transmitters, etc. for the entire house. Their receivers start at $200, and it s at least $350 to integrate an existing stereo into the system. I d be shelling out over $1000 to use Sonos for this setup. As it is, I ve bought less than $100 of equipment (a second Pi and its accessories) and getting output that, while no doubt not quite as pristine, it still quite nice and acceptable. Quite nice. Update: LMS with Apache It turns out that Logitech Media Server can automatically save created playlists to a spot where Ampache can find them. So an Ampache streaming player can be used to access the same collection LMS uses, with full features. Nice!

1 January 2015

Lars Wirzenius: The shape of a solution

Each software tool exists to solve some problem. For each problem, there are many possible solutions. Even when different programs basically do the same thing, they can have quite different shapes. As an example, this morning I was wondering if it would be possible for me to use notmuch to index my entire mail archive. For that, I needed to convert a number of mbox folders to Maildir format. That's a resonably easy problem, given access to suitable programming libraries, but there's an existing tool for that, called mb2md. Unfortunately, it has the wrong shape for my needs. mb2md doesn't just convert one mbox to one maildir. It's designed to for a mail admin converting all server-side mbox folders for a user into a corresponding structure of Maildir folders. This seems to be necessary when switching IMAP servers. That's a fairly specialised problem, and the program has been written to make it easy for a mail admin to do that. What I need is part of the problem solved by mb2md and indeed it can do just that part. However, the overall shape of mb2md is such that my part is hard to do. The incantation is quite unintuitive and requires careful reading of the documentation. The shape of a solution matters. mb2md could easily have been written in a way that provides a simple tool for the single folder conversion, and then a more complex tool for the mail admin's more complicated problem. This would have resulted in a much more general tool, and that would make it easier for more people to use it without much effort. Mail folder format conversions are a fairly esoteric thing to do. However, the lack of generality is a frequent issue with how programs are designed. It is easy to fall into the trap of writing a highly specialised tool, instead of taking a step back and making a more general purpose tool. The specialised tool will help a small number of people. The general tool will help many people. Examples of this are fairly common. Debian has a set of tools for making Debian live CDs; they are not quite able to make a bootable hard disk image as well (thus, vmdebootstrap). There's programs for computing cyclomatic complexity, which produce HTML reports, rather than something that can be processed by other programs without too much effort. There's tools for managing address books that are limited to specific cultures, e.g., by hardcoding assumptions of what a person's name looks like (thus, clab). One of my favourite examples is xargs, which by default does the wrong thing by assuming its input is whitespace delimited. Any whitespace, not just newlines. Any sensible use requires adding the -0 option, which makes xargs that much more tedious to use. Furthermore, I've often found that the more general tool is simpler. It's functional specification is simpler; it's implementation is simpler, and has fewer special cases; it's user experience is simpler. That's not always true, but often it is. Sometimes the general solution shape is not worth it. But it's always worth considering whether it might be. One of the parts of the Unix culture I really like is the preference for general tools that are easy to combine together.

31 December 2014

Chris Lamb: 2014: Selected highlights

Previously: 2012 & 2013.


January https://d1icoid1cnixnp.cloudfront.net/yadt/blog.Image/image/cropped/15.jpeg Was lent a 15-course baroque lute. February https://d1icoid1cnixnp.cloudfront.net/yadt/blog.Image/image/cropped/19.jpeg Grandpa's funeral. In December he was posthumously awarded the Ushakov Medal (pictured) for his service in the Royal Navy's Arctic Convoys during the Second World War. March https://d1icoid1cnixnp.cloudfront.net/yadt/blog.Image/image/cropped/14.jpeg A lot of triathlon training but also got back into cooking. April https://d1icoid1cnixnp.cloudfront.net/yadt/blog.Image/image/cropped/2.jpeg Returned to the Cambridge Duathlon. May https://d1icoid1cnixnp.cloudfront.net/yadt/blog.Image/image/cropped/13.jpeg Raced 50 and 100 mile cycling time trials & visited the Stratford Olympic pool (pictured). June https://d1icoid1cnixnp.cloudfront.net/yadt/blog.Image/image/cropped/1.jpeg Ironman Austria. July https://d1icoid1cnixnp.cloudfront.net/yadt/blog.Image/image/cropped/9.jpeg Paced my sister at the Downtow-Upflow Half-marathon. Also released the first version of the Strava Enhancement Suite. August https://d1icoid1cnixnp.cloudfront.net/yadt/blog.Image/image/cropped/3.jpeg Visited Cornwall for my cousin's wedding (pictured). Another month for sport including my first ultramarathon and my first sub-20 minute 5k. September https://d1icoid1cnixnp.cloudfront.net/yadt/blog.Image/image/cropped/11.jpeg Entered a London Oxford London cycling brevet, my longest single-ride to date (269 km). Also visited the Tour of Britain and the Sri Chomnoy 24-hour endurance race. October https://d1icoid1cnixnp.cloudfront.net/yadt/blog.Image/image/cropped/10.jpeg London Paris London cycling tour (588 km). November https://d1icoid1cnixnp.cloudfront.net/yadt/blog.Image/image/cropped/5.jpeg Performed Handel's Messiah in Kettering. December https://d1icoid1cnixnp.cloudfront.net/yadt/blog.Image/image/cropped/18.jpeg Left Thread.com.

Michal &#268;iha&#345;: No Windows builds for Gammu and Wammu

For quite some time I used to produce Windows builds for both Wammu and Gammu using cross compiling on Linux. But this has proven to produce some errors and needed my time to maintain the cross compilation environment. I've decided to stop producing Windows binaries and I don't expect to get back to that anytime soon. This is actually no news for Wammu, where I've removed Windows builds some about two years ago as they proven to be too broken for normal usage, but for Gammu it's new as previous release had Windows builds. I've lost the cross compilation environment due to hard drive failure and restoring it is simply too much of work and still will not allow me to build complete release (I've not managed to build Python modules properly). So if anybody is interested in Windows binaries, he needs to produce them on Windows. I can help with fixing code or existing setup scripts (they probably need adjustments as they were tweaked for cross compiling), but somebody has to setup the environment with all dependencies and test the build on Windows.

Filed under: English Gammu Wammu 0 comments Flattr this!

30 December 2014

Benjamin Mako Hill: Consider the Redirect

In wikis, redirects are special pages that silently take readers from the page they are visiting to another page. Although their presence is noted in tiny gray text (see the image below) most people use them all the time and never know they exist. Redirects exist to make linking between pages easier, they populate Wikipedia s search autocomplete list, and are generally helpful in organizing information. In the English Wikipedia, redirects make up more than half of all article pages. seattle_redirectOver the years, I ve spent some time contributing to to Redirects for Discussion (RfD). I think of RfD as like an ultra-low stakes version of Articles for Deletion where Wikipedians decide whether to delete or keep articles. If a redirect is deleted, viewers are taken to a search results page and almost nobody notices. That said, because redirects are almost never viewed directly, almost nobody notices if a redirect is kept either! I ve told people that if they want to understand the soul of a Wikipedian, they should spend time participating in RfD. When you understand why arguing about and working hard to come to consensus solutions for how Wikipedia should handle individual redirects is an enjoyable way to spend your spare time where any outcome is invisible you understand what it means to be a Wikipedian. That said, wiki researchers rarely take redirects into account. For years, I ve suspected that accounting for redirects was important for Wikipedia research and that several classes of findings were noisy or misleading because most people haven t done so. As a result, I worked with my colleague Aaron Shaw at Northwestern earlier this year to build a longitudinal dataset of redirects that can capture the dynamic nature of redirects. Our work was published as a short paper at OpenSym several months ago. It turns out, taking redirects into account correctly (especially if you are looking at activity over time) is tricky because redirects are stored as normal pages by MediaWiki except that they happen to start with special redirect text. Like other pages, redirects can be updated and changed over time are frequently are. As a result, taking redirects into account for any study that looks at activity over time requires looking at the text of every revision of every page. Using our dataset, Aaron and I showed that the distribution of edits across pages in English Wikipedia (a relationships that is used in many research projects) looks pretty close to log normal when we remove redirects and very different when you don t. After all, half of articles are really just redirects and, and because they are just redirects, these articles are almost never edited. edits_over_pagesAnother puzzling finding that s been reported in a few places and that I repeated myself several times is that edits and views are surprisingly uncorrelated. I ll write more about this later but the short version is that we found that a big chunk of this can, in fact, be explained by considering redirects. We ve published our code and data and the article itself is online because we paid the ACM s open access fee to ransom the article.

Steve Kemp: Reducing, or redirecting at least, charitable donations.

This is the time of year when there are lots of adverts shown on TV solicating donations for charities, which frequently end with the two words "thank you". I've always felt there were too many charities in the world, and that it was hard to half-heartedly give money to one charity this month, one the next, and still another next year. On that basis I decided long ago to give my money solely to three charities. If I had money that was spare, or I felt generous that month, I would give it to one of "my" charities. Any other appeals I just ignored (with minor exceptions for one-off events like tsunamis, etc). I won't claim credit for this idea, it came directly from my mom who does the same thing. I've given money to the same three charities for twenty years now. Maybe not thousands, but hopefully enough to be useful. Certainly more than I'd have given if my donation were split between more recipients. Now I'm changing. As of next year only one charitable organization will get my pennies. The other two haven't done anything bad, wrong, or failed/succeeded (sadly), but it feels better for me to stick to a single recipient. Happy Christmas. (Details shouldn't matter, but to answer the obvious question the charity I've kept is the RNLI.)

28 December 2014

Russ Allbery: Review: Some Remarks

Review: Some Remarks, by Neal Stephenson
Publisher: William Morrow
Copyright: June 2013
ISBN: 0-06-202444-2
Format: Trade paperback
Pages: 336
This is going to be another weird review, since I read this essay collection about three months ago, and I borrowed it from a friend. So this is both from fading memory and without a handy reference other than Internet searches. Apologies in advance for any important details that I miss. The advantage is that you'll see what parts of this collection stuck in my memory. Some Remarks is, as you might guess from the title, a rather random collection of material. There's one long essay that for me was the heart of the book (more on that in a moment), two other longer essays, two short stories, and thirteen other bits of miscellaneous writing of varying lengths. I found most of the short essays unremarkable. Stephenson uses a walking desk because sitting is bad for you that sentence contains basically all of the interesting content of one of the essays. I think it takes a large topic and some running room before Stephenson can get up to speed and produce something that's more satisfying than technological boosterism. That means the most interesting parts of this book are the three longer works. "In the Kingdom of Mao Bell" was previously published in Wired and is still available. Some Remarks contains only excerpts; Stephenson says that some of the original essay is no longer that interesting. I had mixed feelings about this one. Some of the sense of place he creates was fun to read, but Stephenson can't seem to quite believe that the Chinese don't care about "freedom" according to his definitions in the same way and therefore don't have the same political reaction to hacker culture that he does. This could have been an opportunity for him to question assumptions, but instead it's mostly an exercise in dubious, sweeping cultural evaluation, such as "the country has a long history of coming up with technologies before anyone else and then not doing a lot with them." A reminder that the detail with which Stephenson crams his writing is not always... true. Stronger is "Atoms of Cognition: Metaphysics in the Royal Society 1715 2010," which covers material familiar to readers of Stephenson's Baroque Cycle. The story of Newton, Leibniz, their rivalry, and the competing approaches to thinking about mathematics and science was my favorite part of that series, and in some ways the non-fiction treatment is better than the fictional one. If you liked the Baroque Cycle, this is worth reading. But the highlight of the book for me was "Mother Earth Mother Board." This is a long essay (50,000 words, practically a small book and the largest part of this collection) about the laying of undersea fiber-optic cables. Those who have read Cryptonomicon will recognize some of the themes here, but there's way more to this essay than there was to the bits about undersea cables in Cryptonomicon. It's mostly about technology, rather than people, which puts Stephenson on firmer ground. The bit about people reads more like a travelogue, full of Stephenson's whole-hearted admiration of people who build things and make things work. There's a bit of politics, a bit of history, a bit of tourism, and a lot of neat trivia about a part of the technological world that I'd not known much about before. I would say this is worth the price of the collection, but it too was previously published in Wired, so you can just read it online. Those reading this review on my web site will notice that I filed it in non-fiction. There are a couple of stories, but they're entirely forgettable (in fact, I had entirely forgotten them, and had to skim them again). But, for the record, here are short reviews of those: "Spew": This originally appeared in Wired and can still be read on-line. The protagonist takes a job as a sort of Internet marketing inspector who looks for deviations from expected profiles. While tracing down an anomaly, though, he finds another use of the Internet that's outside of the marketing framework he's using. It's unlikely that anyone who's been online for long will find much new in this story. Some of that is because it was originally published in 1994, but most of it is just that this isn't a very good story. Stephenson seems to have turned up his normal manic infodump to 11 to satisfy the early Wired aesthetic, and the result is a train wreck of jargon, bad slang, and superficial social observation. (3) "The Great Simoleon Caper": Originally published in TIME, this story too is still available online. It's primarily interesting because it's a story about Bitcoin (basically), written in 1995. And it's irritating for exactly the same reason that Bitcoin enthusiasm often tends to be irritating: the assumption that cryptocurrency is somehow a revolutionary attack on government-run currency systems. I'm not going to get into the ways in which this doesn't make sense given how money is used socially (David Graeber's Debt is the book to read if you want more information); just know that the story follows that path and doesn't engage with any of the social reasons why that outcome is highly unlikely. Indeed, the lengths to which the government tries to go to discredit cryptocurrency in this story are rather silly. Apart from that, this is typical early Stephenson writing. It's very in love with ideas, not as much with characterization, and consists mostly of people explaining things to each other. Sometimes this is fun, but when focused on topics about which considerably more information has become available, it doesn't age very well. (5) Overall, there was one great essay and a few interesting bits, but I wouldn't have felt I was missing much if I'd never read this collection. I borrowed Some Remarks from a friend, and I think that's about the right level of effort. If it falls into your hands, or you see it in a library, some of the essays, particularly "Mother Earth Mother Board," are worth reading, but given that the best parts are available on-line for free, I don't think it's worth a purchase. Rating: 6 out of 10

26 December 2014

Ritesh Raj Sarraf: Linux Containers and Productization

Linux has improved many many things over the last couple of years. Of the many improvements, the one that I've started leveraging the most today, are Control Groups. In the past, when there was a need to build a prototype for a solution, we needed hardware. Then came the virtualization richness to Linux. It came in 2 major flavors, KVM (Full Virtualization) and Xen (Para Virtualization). Over the years, the difference of para vs full, for both the implementations, is almost none. KVM now has support for Para-Virtualizaiton, with para-virtualized drviers for most resource intensive tasks, like network and I/O. Similarly, Xen has Full Virtualization support with the help of Qemu-KVM. But, if you had to build a prototype implementation comprising of a multi node setup, virtualization could still be resource hungry. Otherwise too, if your focus was an application (say like a web framework), virtualization was an overkill. All thanks to Linux Containers, prototyping applicaiton based solutions, is now a breeze in Linux. The LXC project is very well designed, and well balanced, in terms of features (as compared to the recently introduced Docker implementation). From an application's point of view, linux containers provide virtualization for namespace, network and resources. Thus making more than 90% of your application's needs fulfilled. For some apps, where a dependency on the kernel is needed, linux containers will not serve the need. Beyond the defaults provided by the distribution, I like to create a base container with my customizations, as a template. This allows me to quickly create environements, without too much housekeeping to do for the initial setup. My base config, looks like:
rrs@learner:~$ sudo cat /var/lib/lxc/deb-template/config
[sudo] password for rrs:
# Template used to create this container: /usr/share/lxc/templates/lxc-debian
# Parameters passed to the template:
# For additional config options, please look at lxc.container.conf(5)
# CPU
lxc.cgroup.cpuset.cpus = 0,1
lxc.cgroup.cpu.shares = 1234
# Mem
lxc.cgroup.memory.limit_in_bytes = 2000M
lxc.cgroup.memory.soft_limit_in_bytes = 1500M
# Network
lxc.network.type = veth
lxc.network.hwaddr = 00:16:3e:0c:c5:d4
lxc.network.flags = up
lxc.network.link = lxcbr0
# Root file system
lxc.rootfs = /var/lib/lxc/deb-template/rootfs
# Common configuration
lxc.include = /usr/share/lxc/config/debian.common.conf
# Container specific configuration
lxc.mount = /var/lib/lxc/deb-template/fstab
lxc.utsname = deb-template
lxc.arch = amd64
# For apt
lxc.mount.entry = /var/cache/apt/archives var/cache/apt/archives none defaults,bind 0 0
23:07          
rrs@learner:~$
Some of the important settings to have in the templace are the mount point, to point to your local apt cache, and CPU and Memory limits. If there was one feature request to ask the LXC developers, I'd ask them to provide a util-lxc tools suite. Currently, to know the memory (soft/hard) allocation for the container, one needs to do the following:
rrs@learner:/sys/fs/cgroup/memory/lxc/deb-template$ cat memory.soft_limit_in_bytes memory.limit_in_bytes
1572864000
2097152000
23:21          
rrs@learner:/sys/fs/cgroup/memory/lxc/deb-template$ bc
bc 1.06.95
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type  warranty'.
1572864000/1024/1024
1500
quit
23:21          
rrs@learner:/sys/fs/cgroup/memory/lxc/deb-template$
Tools like lxc-cpuinfo, lxc-free would be much better. Finally, there's been a lot of buzz about Docker. Docker is an alternate product offering, like LXC, for Linux Containers. From what I have briefly looked at, docker doesn't seem to be providing any ground breaking new interface than what is already possible with LXC. It does take all the tidbit tools, and presents you with a unified docker interface. But other than that, I couldn't find it much appealing. And the assumption that the profiles should be pulled off the internet (Github ?) is not very exciting. I am hoping they do have other options, where dependence on the network is not really required.

Categories:

Keywords:

Lars Wirzenius: 'It Will Never Work in Theory' and software engineering research

It Will Never Work in Theory is a web site that blogs, though slowly, of important research and findings about software development. It's one of the most interesting sites I've found recently, possibly for a long time. I disagree with the term "software engineering" to describe the software development that happens today. I don't think it's accurate, and indeed I think the concept's too much of a fantasy for the term to be used seriously about practicing developers do. For software development to be an engineering discipline, it needs a strong foundation based on actual research. In short, we need to know what works, what doesn't work, and preferably why in both cases. We don't have much of that. This website is one example of how that's now changing, and that's good. As a practicing software developer, I want to know, for example, whether code review actually helps improve software quality, the speed of software development, and the total cost of a software project, and also under what the limits of code review are, how it should be done well, and what kind of review doesn't work. Once I know that, I can decide whether and how to do reviews in my development teams. The software development field is full of anecdotal evidence about these things. It's also full of people who've done something once, and then want to sell books, seminars, and lectures about it. That's not been working too well: it makes research be mostly about fads, and that's no way to build a strong foundation. Now I just need the time to read everything, and the brain to understand big words.

Vasudev Kamath: Notes: LXC How-To

LXC - Linux Containers allows us to run multiple isolated Linux system under same control host. This will be useful for testing application without changing our existing system. To create an LXC container we use the lxc-create command, it can accepts the template option, with which we can choose the OS we would like to run under the virtual isolated environment. On a Debian system I see following templates supported
[vasudev@rudra: ~/ ]% ls /usr/share/lxc/templates
lxc-alpine*    lxc-archlinux*  lxc-centos*  lxc-debian*    lxc-fedora*  lxc-openmandriva*  lxc-oracle*  lxc-sshd*    lxc-ubuntu-cloud*
lxc-altlinux*  lxc-busybox*    lxc-cirros*  lxc-download*  lxc-gentoo*  lxc-opensuse*      lxc-plamo*   lxc-ubuntu*
For my application testing I wanted to create a Debian container for my By default the template provided by lxc package creates Debian stable container. This can be changed by passing the option to debootstrap after -- as shown below.
sudo MIRROR=http://localhost:9999/debian lxc-create -t debian \
     -f   container.conf -n container-name -- -r sid
-r switch is used to specify the release, MIRROR environment variable is used to choose the required Debian mirror. I wanted to use my own local approx installation, so I can save some bandwidth. container.conf is the configuration file used for creating the LXC, in my case it contains basic information on how container networking should b setup. The configuration is basically taken from LXC Debian wiki
lxc.utsname = aptoffline-dev
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0.1
lxc.network.name = eth0
lxc.network.ipv4 = 192.168.3.2/24
lxc.network.veth.pair = vethvm1
I'm using VLAN setup described in Debian wiki: LXC VLAN Networking page. Below is my interfaces file.
iface eth0.1 inet manual
iface br0.1  inet manual
   bridge_ports eth0.1
   bridge_fd 0
   bridge_maxwait 0
Before launching the LXC make sure you run below command
sudo ifup eth0.1
sudo ifup br0.1
# Also give ip to bridge in same subnet as lxc.network.ipv4
sudo ip addr add 192.168.3.1/24 dev br0.1
I'm giving ip address to bridge so that I can communicate with container from my control host, once it comes up. Now start the container using below command
sudo lxc-start -n container -d -c tty8
We are starting lxc in daemon mode and attaching it to console tty8. If you want, you can drop -d and -c option to start lxc in foreground. But its better you start it in background and attach using lxc-console command shown below.
sudo lxc-console -n container -t tty8
You can detach from console using Ctrl+a q combination and let lxc execute in background. Its also possible to simply ssh into the running container since we have enabled networking. Stopping the container should be done using lxc-stop command, but without -k switch (kill) this command never returned. Even with timeout container is not stopped.
sudo lxc-stop -n container
-r can be used for reboot of container. Since I couldn't get clean shutdown I normally attach the console and issue a halt command in container itself. Not sure if this is the right way, but it gets the thing done. I consider Linux container as a better alternative for spawning a virtual Linux environment instead of running a full blown VM like Virtualbox or VMware.

25 December 2014

Russ Allbery: C TAP Harness 3.2

This is a fairly minor release, mostly to have a release of files that I'm updating in rra-c-util. Most of the changes are typos and other cosmetic fixes. But I hate releasing a new version of something without a little bit of new functionality, so I implemented comment support in test lists. The runtests driver now ignores comments (lines starting with #) and blank lines in test list files, and leading whitespace in front of test names. You can get the latest version from the C TAP Harness distribution page.

24 December 2014

Gunnar Wolf: More on Debian as a social, quasi-familiar project

I have long wanted to echo Gregor's beautiful Debian Advent Calendar posts. Gregor is a dear project member and a dear friend to many of us Debianers, who has shown an amount of stamina and care for the project that inspires everybody; this year, after many harsh flamefests in the project (despite which we are moving at a great rate towards a great release!), many people have felt the need to echo how Debian even as often seen from the outside as a hostile mass of blabbering geeks is actually a great place to work together and to create a deep, strong social fabric And that's quite probably what binds the project together and ensures it will continue existing and excelling for a long time. As for the personal part: This year, my Debian involvement has once again reduced. Not because I care less about Debian, much to the contrary, but because I have taken several responsabilities which require my attention and time. Technically, I'm basically maintaining a couple of PHP-based packages I use for work (most prominently, Drupal7). I have stepped back of most of my DebConf responsabilities, although I stay (and will stay, as it's an area of the project I deeply enjoy doing) involved. And, of course, my current main area of involvement is keyring-maint (for which I have posted here several status updates). I have to say that we expected having a much harder time (read: Stronger opposition and discussions) regarding the expiry of 1024D keys. Of course, many people do have a hard time connecting anew to the web of trust, and we will still face quite a bit of work after January 1st, but the migration has been a mostly pleasant (although clearly intensive) work. Jonathan has just told me we are down to only 306 1024D keys in the keyring (which almost exactly matches the "200-300" I expected back at DC14). Anyway: People predicting doomsday scenarios for Debian do it because they are not familiar with how deep the project runs in us, how important it is socially, almost at a family level, to us that have been long involved in it. Debian is stronger than a technical or political discussion, no matter how harsh it is. And, as a personal thank-you: Gregor, your actions (the GDAC, the RC bug reports) inspire us to stay active, to do our volunteer work better, and remind us of how great is it to be a part of a global, distributed will to Do It Right. Thanks a lot!

Gunnar Wolf: On rogue states disrupting foreign networks

Much ink has been spilled lately (well, more likely, lots of electrons have changed their paths lately as most of these communications have surely been electronic) on the effects, blame, assurance and everything related to the (allegedly) North Korean attack on Sony's networks. And yes, the list of components and affectations is huge. Technically it was a very interesting feat, but it's quite more interesting socially. Say, the not-so-few people wanting to wipe North Korea from the face of the Earth, as... Well, how did such a puny nation dare touch a private company that's based in the USA? Of course, there's no strong evidence the attack did originate in (or was funded by) North Korea. And... I have read very few people talking about the parallels to the infamous Stuxnet, malware written by USA and Israel (not officially admitted, but with quite a bit of evidence pointing to it, and no denial attempts after quite a wide media exposure). In 2010, this worm derailed Iran's nuclear program. Iran, a sovereign nation. Yes, many people doubt such a nuclear program would be used "for good, not for evil" But since when have those two words had an unambiguous meaning? And when did it become accepted as international law to operate based on hunches and a "everybody knows" mentality? So, how can the same people repudiate NK's alleged actions and applaud Stuxnet as a perfect weapon for peace?

Next.

Previous.