Search Results: "beh"

20 January 2014

Dirk Eddelbuettel: RProtoBuf 0.4.0: A whole lot of goodies and Windoze support

A new major release 0.4.0 of RProtoBuf, is now on CRAN. RProtoBuf provides GNU R bindings for the Google Protocol Buffers ("Protobuf") data encoding library used and released by Google, and deployed as a language and operating-system agnostic protocol by numerous projects. With this release, we are welcoming Jeroen Ooms to the team. Jeroen had already worked on some RProtoBuf extensions in the context of his OpenCPU project; we have now integrated those Protocol Buffers functions. And Jeroen pushed all the right buttons to finally get RProtoBuf built on everybody's least-favourite operating system by provding a static library for use by win-builder and CRAN. Murray once again did a lot of work on internals. His use of the LLVM tool llvm-format was particular helpful to make our coding style a little more consistent. The complete NEWS file entry for this release follows:
Changes in RProtoBuf version 0.4.0 (2014-01-14)
  • Changes to support CRAN builds for MS Windows.
  • Added functions serialize_pb, unserialize_pb, and can_serialize_pb plus documentation from Jeroen Ooms RProtoBufUtils package.
  • New dir inst/python with some Python examples.
  • Added Jeroen Ooms as author.
  • Vignettes have been converted to the R 3.0.0 or later use of external vignette builders, no longer need a Makefile
  • Added missing methods to dollar completion list for Message, Descriptor, EnumValueDescriptor, and FileDescriptor classes.
  • Add missing export for .DollarNames EnumValueDescriptor to allow completion on that class.
  • Add more than 15 additional pages to the main Intro vignette documenting better all of the S4 classes implemented by RProtoBuf, updating the type mapping tables to take into account 64-bit support, and documenting advanced features such as Extensions.
  • Added better error checking in EnumDescriptors to catch the case when wrong types are provided.
  • Updated the FileDescriptor name() method to accept a boolean for full paths just like the generic name() method.
  • Correct a bug that incorrectly dispatched as.character() when as.list() was called on Descriptor objects.
  • Update FileDescriptor $ dispatch to work properly for the names of fields defined in the FileDescriptor, instead of just returning NULL even for types returned by $ completion.
  • Added a reservation for extension fields in the example tutorial.Person schema.
  • Support setting int32 fields with character representations and raise an R-level stop() error if the provided string can not be parsed as a 32-bit integer, rather than crashing the R instance.
  • Update the project TODO file.
  • Add better documentation and tests for all of the above.
  • Corrected the handling of uint32 and fixed32 types in protocol buffers to ensure that they work with numbers as large as 2^32 - 1, which is larger than an integer can hold in R since R does not have an unsigned integer class. These values are stored as doubles internally now to avoid losing precision.
  • Added unit tests to verify behavior of RProtoBuf with extreme values for uint32 types.
  • Removed old exception handling code and instead rely on the more modern Rcpp::stop method maintained in Rcpp.
  • Add better error messages when setting a repeated field of messages to inform the user which element index was of the wrong type and what the expected type was.
  • Add an optional 'partial' argument to readASCII allowing uninitialized message fragments to be read in.
  • (internal) Added const qualifiers in more places throughout the C++ code for type safety.
  • (internal) Standardize coding conventions of the C++ files and run them through clang-format for consistency. A STYLE file has been submitted to R-Forge with details about the coding standards and how they are enforced with Emacs and clang-format.
  • Applied changes suggested by Kevin Ushey to the S4 class handling to support both the currently released Rcpp as well as the currently pending next version.
CRANberries also provides a diff to the previous release 0.3.2. More information is at the RProtoBuf page which has a draft package vignette, a 'quick' overview vignette and a unit test summary vignette. Questions, comments etc should go to the rprotobuf mailing list off the RProtoBuf page at R-Forge.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

19 January 2014

Matthias Klumpp: Tanglu Beta2 now available!

While writing this post, I recognize that we might need a news-blog for tanglu logo pureTanglu :-) Since yesterday, we have an (installable!) Beta2 release of Tanglu! Compared to the previous snapshot, it comes with a huge amount of improvements and bugfixes: Known issues: You can download the release from one of these mirrors: Mirror1 Mirror2 Mirror3 Additionally, there is now a release schedule for Tanglu Aequorea, and we will do a vote on the name for the next release soon (proposed names). As a note to Debian: systemd is working pretty well for Tanglu so far ;-) I am pretty happy with this Beta2 release, because Tanglu is shaping out to be the distribution we imagined in the beginning. Have fun! And as bonus, here are some images from Beta2: tanglu-beta2-kde0 tanglu-beta2-kde1 tanglu-beta2-kde2 tanglu-beta2-gnome2tanglu-beta2-gnome1

17 January 2014

Chris Lamb: Captain Phillips: Pontius Pirate

https://chris-lamb.co.uk/wp-content/2014/phillips_b.jpg Somalia's chief exports appear to be morally-ambiguous Salon articles about piracy and sophomoric evidence against libertarianism. However, it is the former topic that Captain Phillips concerns itself with, inspired by the hijacking of the Maersk Alabama container ship in 2009. What is truth? In the end, Captain Phillips does not rise above Pontius Pilate in providing an answer, but it certainly tries using more refined instruments than irony or leaden sarcasm. This motif pervades the film. Obviously, it is based on a "true story" and brings aboard that baggage, but it also permeates the plot in a much deeper sense. For example, Phillips and the US Navy lie almost compulsively to the pirates, whilst the pirates only really lie once (where they put Phillips in greater danger). Notice further that Phillips only starts to tell the truth when he thinks all hope is lost. These telling observations become even more fascinating when you realise that they must be based on the testimony of the, well, liars. Clearly, deception is a weapon to be monopolised and there are few limits on what good guys can or should lie about if they believe they can save lives.
https://chris-lamb.co.uk/wp-content/2014/phillips_c.jpg

Even Phillip's nickname ("Irish") is a falsehood he straight-up admits he is an American citizen.

Futhermore, there is an utterly disarming epilogue where Phillips is being treated for shock by clinical efficient medical staff. Not only will this scuttle any "blanket around the shoulders" clich but is probably a highly accurate portrayal of what actually happens post-trauma. This echoes the kind of truth Werner Herzog aims for in his filmmaking as well his guilt-inducing duality between uncomfortable lingering and compulsive viewing. Lastly, a starter for a meta-discussion: can a film based on real-world events even be "spoilered"? Hearing headlines on the radio before you read your newspaper hardly robs you of a literary journey... Captain Phillips does have some quotidian problems. Firstly, the only tool for ratcheting up tension is for the Somalians to launch verbal broadsides at the Americans, with each compromise somehow escalating the situation. This technique is effective but well before the climatic rescue scene where it is really needed it has been subject to the most extreme diminishing returns. (I cannot be the first to notice the "Africans festooned with guns shouting incomphensively" trope I hope it is based on a Babel-esque mechanism of disorientation from miscommunication rather than anything more unsavoury.)
https://chris-lamb.co.uk/wp-content/2014/phillips_a.jpg

The racist idea that Africans prefer an AK-47 rotated about the Z-axis is socially constructed.

Secondly, the US Navy acts like a teacher with an Ofsted inspector observing quietly from the corner of the classroom; far too well-behaved it suspends belief, with no post-kill gloating or even the tiniest of post-arrest congratulations. Whilst nobody wants to see the Navy overreact badly to other military branches getting all the glory, nobody wants to see a suspiciously bland recruitment vehicle either. Paradoxically, this hermetic treatment made me unduly fascinated by them as if they were part of some military "uncanny valley". Two quick observations:
https://chris-lamb.co.uk/wp-content/2014/phillips_d.jpg

The drone footage: I'd love to write an essay about how Call of Duty might have influenced (or even be) cinema.

Finally, despite the title, the film is actually about two captains; the skillful liar Phillips and ... well, that's the real problem. Whilst Captain Muse is certainly no caricatured Hook, we are offered little depth beyond a "You're not just a fisherman" faux-revelation that leads nowhere. I was left inventing reasons for his akrasia so that he made any sense whatsoever. One could charitably argue that the film attempts to stay objective on Muse, but the inability for the film to take any obvious ethical stance actually seems to confuse and then compromise the narrative. What deeper truth is actually being revealed? Is this film or documentary? Worse still, the moral vacuum is invariably filled by the viewer's existing political outlook: are Somali pirates victims of circumstance who are forced into (alas, regrettable) swashbuckling adventures to pacify plank-threatening warlords? Or are they violent and dangerous criminals who habour an irrational resentment against the West, flimsily represented by material goods in shipping containers? Your improvised answer to this Rorschach test will always sit more haphazardly in the film than any pre-constructed treatment ever could. 6/10

Chris Lamb: Review: Captain Phillips (2013)

https://chris-lamb.co.uk/wp-content/2014/phillips_b.jpg Somalia's chief exports appear to be morally-ambiguous Salon articles about piracy and sophomoric evidence against libertarianism. However, it is the former topic that Captain Phillips concerns itself with, inspired by the hijacking of the Maersk Alabama container ship in 2009. What is truth? In the end, Captain Phillips does not rise above Pontius Pilate in providing an answer, but it certainly tries using more refined instruments than irony or leaden sarcasm. This motif pervades the film. Obviously, it is based on a "true story" and brings aboard all that well-travelled baggage, but it also permeates the plot in a much deeper sense. For example, Phillips and the US Navy lie almost compulsively to the pirates, whilst the pirates only really lie once where they put Phillips in greater danger. Notice further that Phillips only starts to tell the truth when he thinks all hope is lost. These telling observations become even more fascinating when you realise that they must be based on the testimony of the, well, liars. Clearly, deception is a weapon to be monopolised and there are few limits on what good guys can or should lie about if they believe they can save lives.
https://chris-lamb.co.uk/wp-content/2014/phillips_c.jpg

Even Phillip's nickname ("Irish") is a falsehood he straight-up admits he is an American citizen.

Lastly, there is an utterly disarming epilogue where Phillips is being treated for shock by clinical efficient medical staff. Not only will it scuttle any "blanket around the shoulders" clich but is probably a highly accurate portrayal of what actually happens post-trauma. This echoes the kind of truth Werner Herzog often aims for in his filmmaking as well his guilt-inducing duality between uncomfortable lingering and compulsive viewing. Another angle worthy of discussion: can a film based on real-world events even be "spoilered"? Hearing headlines on the before you read the newspaper hardly robs you of a literary journey... Captain Phillips does have some quotidian problems. Firstly, the only tool for ratcheting up tension is for the Somalians to launch verbal broadsides at the Americans, with each compromise somehow escalating the situation further. This technique is genuinely effective but well before the climatic rescue scene where it is really needed it has been subject to the most extreme diminishing returns. (I cannot be the first to notice the "Africans festooned with guns shouting incomphensively" trope I hope it is based on a Babel-esque mechanism of disorientation and miscommunication rather than anything, frankly, unsavoury.)
https://chris-lamb.co.uk/wp-content/2014/phillips_a.jpg

The racist idea that Africans prefer a AK-47 rotated about the Z-axis is socially constructed.

Secondly, the US Navy acts like a teacher with an Ofsted inspector observing quietly from the corner of the classroom; far too well-behaved it suspends belief, with no post-kill gloating or even the tiniest of post-arrest congratulations. Whilst nobody wants to see the Navy overreact badly to other military branches getting all the glory, nobody wants to see a suspiciously bland recruitment vehicle either. Paradoxically, this hermetic treatment made me unduly fascinated by them, as if they were part of some military "uncanny valley". Two quick observations:
https://chris-lamb.co.uk/wp-content/2014/phillips_d.jpg

The drone footage: I'd love to read (or write) an essay about how Call of Duty might have influenced cinema.

Finally, despite the title, the film is actually about two captains; the skillful liar Phillips and ... well, that's the real problem. Whilst Captain Muse is certainly no caricatured Hook, we are offered little depth beyond a "You're not just a fisherman" faux-revelation that leads nowhere. I was left inventing reasons for his akrasia so that he made any sense whatsoever. One could charitably argue that the film attempts to stay objective on Muse, but the inability for the film to take any obvious ethical stance actually seems to confuse and then compromise the narrative. What deeper truth is actually being revealed? Is this film or documentary? Worse still, the moral vacuum is invariably filled by the viewer's existing political outlook: are Somali pirates victims of circumstance who are forced into (alas, regrettable) swashbuckling adventures to pacify plank-threatening warlords? Or are they violent and dangerous criminals who habour an irrational resentment against the West, flimsily represented by material goods in shipping containers? Your improvised answer to this Rorschach test will always sit more haphazardly in the film than any pre-constructed treatment ever could. 6/10

14 January 2014

Petter Reinholdtsen: New chrpath release 0.16

Coverity is a nice tool to find problems in C, C++ and Java code using static source code analysis. It can detect a lot of different problems, and is very useful to find memory and locking bugs in the error handling part of the source. The company behind it provide check of free software projects as a community service, and many hundred free software projects are already checked. A few days ago I decided to have a closer look at the Coverity system, and discovered that the gnash and ipmitool projects I am involved with was already registered. But these are fairly big, and I would also like to have a small and easy project to check, and decided to request checking of the chrpath project. It was added to the checker and discovered seven potential defects. Six of these were real, mostly resource "leak" when the program detected an error. Nothing serious, as the resources would be released a fraction of a second later when the program exited because of the error, but it is nice to do it right in case the source of the program some time in the future end up in a library. Having fixed all defects and added a mailing list for the chrpath developers, I decided it was time to publish a new release. These are the release notes: New in 0.16 released 2014-01-14: You can download the new version 0.16 from alioth. Please let us know via the Alioth project if something is wrong with the new release. The test suite did not discover any old errors, so if you find a new one, please also include a test suite check.

11 January 2014

Johannes Schauer: Why do I need superuser privileges when I just want to write to a regular file

I have written a number of scripts to create Debian foreign architecture (mostly armel and armhf) rootfs images for SD cards or NAND flashing. I started with putting Debian on my Openmoko gta01 and gta02 and continued with devices like the qi nanonote, a marvel kirkwood based device, the Always Innovating Touchbook (close to the Beagleboard), the Notion Ink Adam and most recently the Golden Delicious gta04. Once it has been manufactured, I will surely also get my hands dirty with the Neo900 whose creators are currently looking for potential donors/customers to increase the size of the first batch and get the price per unit further down. Creating a Debian rootfs disk image for all these devices basically follows the same steps:
  1. create an disk image file, partition it, format the partitions and mount the / partition into a directory
  2. use debootstrap or multistrap to extract a selection of armel or armhf packages into the directory
  3. copy over /usr/bin/qemu-arm-static for qemu user mode emulation
  4. chroot into the directory to execute package maintainer scripts with dpkg --configure -a
  5. copy the disk image onto the sd card
It was not long until I started wondering why I had to run all of the above steps with superuser privileges even though everything except the final step (which I will not cover here) was in principle nothing else than writing some magic bytes to files I had write access to (the disk image file) in some more or less fancy ways. So I tried using fakeroot+fakechroot and after some initial troubles I managed to build a foreign architecture rootfs without needing root priviliges for steps two, three and four. I wrote about my solution which still included some workarounds in another article here. These workarounds were soon not needed anymore as upstream fixed the outstanding issues. As a result I wrote the polystrap tool which combines multistrap, fakeroot, fakechroot and qemu user mode emulation. Recently I managed to integrate proot support in a separate branch of polystrap. Last year I got the LEGO ev3 robot for christmas and since it runs Linux I also wanted to put Debian on it by following the instructions given by the ev3dev project. Even though ev3dev calls itself a "distribution" it only deviates from pure Debian by its kernel, some configuration options and its initial package selection. Otherwise it's vanilla Debian. The project also supplies some multistrap based scripts which create the rootfs and then partition and populate an SD card. All of this is of course done as the superuser. While the creation of the file/directory structure of the foreign Debian armel rootfs can by now easily be done without superuser priviliges by running multistrap under fakeroot/fakechroot/proot, creating the SD card image still seems to be a bit more tricky. While it is no problem to write a partition table to a regular file, it turned out to be tricky to mount these partition because tools like kpartx and losetup require superuser permissions. Tools like mkfs.ext3 and fuse-ext2 which otherwise would be able to work on a regular file without superuser privileges do not seem to allow to specify the required offsets that the partitions have within the disk image. With fuseloop there exists a tool which allows to "loop-mount" parts of a file in userspace to a new file and thus allows tools like mkfs.ext3 and fuse-ext2 to work as they normally do. But fuseloop is not packaged for Debian yet and thus also not in the current Debian stable. An obvious workaround would be to create and fill each partition in a separate file and concatenate them together. But why do I have to write my data twice just because I do not want to become the superuser? Even worse: because parted refuses to write a partition table to a file which is too small to hold the specified partitions, one spends twice the disk space of the final image: the image with the partition table plus the image with the main partition's content. So lets summarize: a bootable foreign architecture SD card disk image is nothing else than a regular file representing the contents of the SD card as a block device. This disk image is created in my home directory and given enough free disk space there is nothing stopping me from writing any possible permutation of bits to that file. Obviously I'm interested in a permutation representing a valid partition table and file systems with sensible content. Why do I need superuser privileges to generate such a sensible permutation of bits? Gladly it seems that the (at least in my opinion) hardest part of faking chroot and executing foreign architecture package maintainer scripts is already possible without superuser privileges by using fakeroot and fakechroot or proot together with qemu user mode emulation. But then there is still the blocker of creating the disk image itself through some user mode loop mounting of a filesystem occupying a virtual "partition" in the disk image. Why has all this only become available so very recently and still requires a number of workarounds to fully work in userspace? There exists a surprising amount of scripts which wrap debootstrap/multistrap. Most of them require superuser privileges. Does everybody just accept that they have to put a sudo in front of every invocation and hope for the best? While this might be okay for well tested code like debootstrap and multistrap the countless wrapper scripts might accidentally (be it a bug in the code or a typo in the given command line arguments) write to your primary hard disk instead of your SD card. Such behavior can easily be mitigated by not executing any such script with superuser privileges in the first place. Operations like loop mounting affect the whole system. Why do I have to touch anything outside of my home directory (/dev/loop in this case) to populate a file in it with some meaningful bits? Virtualization is no option because every virtualization solution again requires root privileges. One might argue that a number of solutions just require some initial setup by root to then later be used by a regular user (for example /etc/fstab configuration or the schroot approach). But then again: why do I have to write anything outside of my home directory (even if it is only once) to be able to write something meaningful to a file in it? The latter approach also does not work if one cannot become root in the first place or is limited by a virtualized environment. Imagine you are trying to build a Debian rootfs on a machine where you just have a regular user account. Or a situation I was recently in: I had a virtual server which denied me operations like loop mounting. Given all these downsides, why is it still so common to just assume that one is able and willing to use sudo and be done with it in most cases? I really wonder why technologies like fakeroot and fakechroot have only been developed this late. Has this problem not been around since the earliest days of Linux/Unix? Am I missing something and rambling around for nothing? Is this idea a lost cause or something that is worth spending time and energy on to extend and fix the required tools?

9 January 2014

Gerfried Fuchs: Clawfinger

It's almost a month since I last blogged something, and one of my new year's resolution is to change that, a bit. Let's see how it goes. I've been listening a lot to this great band from Sweden recently again, put all their songs onto my mobile phone. It might sound weird because they have a rather aggressive style, both soundwise but also lyricswise, but it helps me to get things off my chest and stay relaxed in the rest of my life. The band I want to present to you was already twice mentioned in some other articles of my blog I noticed, but this is the proper post about them: I'm talking about Clawfinger. They came up in the nineties during the crossover phase and did blend in pretty well, but it's mostly their direct and political statements they carry in their lyrics that did let them stand out. One warning though: the direct language they use might be considered blunt and maybe even offensive by some. The message behind it though should rather get you thinking of your own doings if you consider it to be offensive. Here are the songs: Like always, enjoy! And maybe also think about it a bit. :)

/music permanent link Comments: 4 Flattr this

8 January 2014

Gunnar Wolf: [review] Packt's Instant Haml

Packt Publishing sent me Krzysztof Niki ski's Instant Haml to do a short blog review... Thing is, and those who read me know, I cannot write short reviews ;-) So lets see how this goes. Packt's Instant series First, as for the format this book follows and the series it fits in: They are very short books. I was the tech reviewer for their Instant Debian Build a Web Server , and when I opened it for the first time, I thought I had received only the first chapter. This series' motto is Short Fast Focused so, yes, the books are ~50 pages long, with ample margins and many big screenshots. I told the editors I was quite unsure about the value this kind of publications add, as they are very similar to online tutorials, but given they have so many books in this series (printed and electronic), there must be a market for them. They surely know their business better than me! There is one other detail regarding the publisher I'll stick in this section, although it is not specific to their Instant series: I do most of my reading in my Kindle. Packt seems to have chosen to specify a very small font for their texts. Besides the two books mentioned, I also have their Object Oriented JavaScript book. For all of the other books I read, I use the smallest font setting. Packt books in mobi format (what they present as Amazon format in their web site) are basically illegible without bumping up the font size through the roof. As I understand it, publishers should use industry-wide default sizes, and let the reader adjust their reading experience. Yes, books of technical nature (such as Packt's) often benefit from having more displayed information than what an ebook reader normally shows... But that's for me to decide! And no, this is not a Kindle-specific glitch: I also get a similar experience when opening it from Calibre. And as for the book... I am a long-time Haml convert/fan; I packaged its Ruby implementation for Debian (and should update it soon :-/ as new versions have appeared), the corresponding Emacs Lisp mode, and I use it for basically all of my static HTML sites and Ruby-based Web systems. Haml is a great templating language, meant to make dynamic HTML generation leaner and easier. It is a clean and elegant technology, and can clearly sell itself. And the project webpage has very good documentation, ranging from a short tutorial and a full API documentation for people to contribute to the project. The first part of this book, of course, covers the "why is Haml so great?" part, presenting the basic equivalences with HTML. In my opinion, the book could be structured a bit differently and would generate a stronger following. Why? Because, even given that Haml is greatly benefited from its Ruby integration and has many hooks to be tightly integrated in a Rails workflow, it can also be used by people who don't want to do Ruby. I have some static websites that are just a bunch of Haml files, compiled by a very simple Makefile and calling the haml command from the shell. I think there would be some value in presenting this invocation before learning how to call it from the Gems environment (i.e. bundle install). Using Haml as a standalone tool is only presented almost as an appendix ("Using HAML outside of Rails" section). Even more so if the author presents four different static-site solutions (probably more non-programmer-friendly than my trusty Makefile). Oh, and this little section uses the (old? obsolete?) uppercase HAML notation for the project's name instead of the now-standard Haml. Dirty screenshots About the screenshots presented in this first section (and I refer mainly to the scaffolding examples), they are meant to show how easy things are, but have an important stylistic mistake: Terminals with transparency might look cool (I hate using them, but then again I'm a very boring person when it comes to my computer habits ;-) ), but transparencies have no place in a book (be it printed or e-book), as the images appear very uncrisp. In the Kindle, they just seemed dirty Now that I'm browsing the PDF from my desktop, I see that the screenshot in "Step 1" has a photo of Earth from space behind the terminal, and even worse, in "Step 2" it is shown over a Web page detailing the installation! These elements are just distracting, and the text would be much cleaerer if it were presented typeset just as a regular code fragment. Opening many of those screenshots in the Kindle was plain useless. Editor integration It is perfectly clear from this text that the author strongly prefers Sublime Text, sometimes uses TextMate, and has just heard reports that Haml has some support in Vim, Rubymine, Emacs and Coda. Of course, devoting a full page (as it is for the first case) for each editor would be out of proportion for this size of book... I just felt the imbalance too big, specially given that most editors will sport approximately the same feature set. Text redundancy If the Instant series is about making short books, I would expect them to be loaded with easy to follow content (as they are), but free of repetition. I felt, however, a large chunk of step 1 of the "Quick start" chapter to be repeated with the the first section of the "Top 6 features" chapter. Yes, the second one adds some depth, but very little. Still, in the later parts of this "Top 6 features" chapter, a more thorough explanation could be used for the Filters, Multiline attributes/code/strings and helpers and extensions sections. It seems the author was on a hurry trying to get it finished, as it is barely explained Explaining a bit more deeply would surely benefit readers. Wrapping up Reading my review, I'm mostly talking about negative issues. IMO, the book is not excellent, but it is clearly not bad. Specially given it fits very well the format for which it was designed. There are many items that could be fixed, and I hope some of them can be fixed (i.e. the screenshots) before the book goes to print If I understand Packt's page correctly, it is currently only available as ebook, contrary to most of the Instant series' titles.

Jamie McClelland: New OpenPGP key

After playing around with How secure is my password, I dediced to start changing the handful of passwords I have committed to memory to being very long phrases rather than shorter, concentrated collections of random-seeming characters (another interesting blog post would be to understand the algorythm behind that site but for now I've just taken it at face value). Despite my usual practice of introducing new passphrases by changing my laptop login (which can always be reset if necessary), I went ahead and changed by GPG key instead to a new 50 character phrase. Fifteen minutes passed before I needed access to the key and... I couldn't repeat the passphrase. I tried to be clever. I opened a text file and typed in this passphrase over and over again, ultimately generating 49 unique typo-ridden versions of the passphrase. Then, I wrote a simple script that took each of these versions and generated additional versions by: In the end I had about 350,000 versions of my passphrase. None of them worked. What in the world did I type?? Could I just ask the NSA? In any event, my new OpenPGP key fingerprint is:
1F9C30CB3CFC5DA9987FA035A014C05A607B7535
Please note: I've changed the "real name" part of my User Id from "Jamie McClelland" to "James McClelland" becuase James is on my government issued id. I'll be working on collecting new signatures for the rest of my life. Some interesting information on how this made my life difficult:
Postscript: Thanks to Jordan Uggla, I was directed to passe-partout, a tool that can extract an RSA private key out of memory when ssh-agent is running with the key loaded. Since I am using the Monkeysphere, my OpenPGP key has an authentication-capable subkey, which is fed to an ssh-agent and used to grant me access to remote computers using ssh. After losing my passphrase, I was not able to access that subkey. With the help of pass-partout, and the fact that the subkey was loaded into ssh-agent before I lost the passphrase, I was able to extract the private RSA key from my ssh-agent and save it to a file. Then, I could restart my ssh-agent with my ''new'' authentication-capable OpenPGP subkey, thus allowing me to access servers that have granted me access via my new key and the handful of servers that still only grant me access via my old key.

3 January 2014

Russ Allbery: A quick copy-paste UNIX trick

I ran across this trick a long time ago and now use it regularly, but it occurred to me today that I don't remember seeing anyone else mention it. So, just in case it helps someone else: I often find myself wanting to copy and paste the contents of several small files into an email message, particularly when explaining Debian packaging. I want to show all the file contents with little headers showing which lines come from each file. This gets tedious if I actually do this file-by-file. The obvious approaches, from including them into my editor to cat'ing them one-by-one and pasting them, are tedious and require writing the headers by hand. Obviously, cat doesn't help, since it obliterates the difference between the files. What does help, however, is good old head (or tail, for that matter). People normally use those utilities with standard input or a single file, but you can give them multiple files, and they do just about exactly what you want. Consider this shell transcript:
    % head -n 20 *.install
    ==> liblog4shib-dev.install <==
    usr/bin/log4shib-config
    usr/include/*
    usr/lib/*/lib*.a
    usr/lib/*/lib*.so
    usr/lib/*/pkgconfig/log4shib.pc
    usr/share/aclocal/log4shib.m4
    debian/log4shib-config.1 usr/share/man/man1
    ==> liblog4shib-doc.install <==
    usr/share/doc/log4shib-*/api/html usr/share/doc/liblog4shib-doc
    ==> liblog4shib1.install <==
    usr/lib/*/lib*.so.*
Just pick some value for -n that's longer than any of the files (I normally want to show the contents of small files anyway), run head on all the files whose contents you want to show, and then you can cut and paste its full output directly into your email message. Bonus: This behavior is even guaranteed by POSIX (something that I didn't realize until I checked while writing this post).

2 January 2014

Luke Faraone: Unstandardized standards are the worst: sendmail

Implementing software to replace legacy systems is always a challenge, especially when you're dealing with a system with as much legacy as sendmail, which was first introduced as delivermail in 1979.ref

Each UNIX vendor, it seems, rewrote or heavily customized sendmail. This has lead to sometimes conflicting implementations.
Case in point: -tNormally, you invoke sendmail(8) with a series of arguments indicating the subject of a message, the recipients, etc. When invoked this way, the command expects a message on standard input, waits for EOF, and then sends your message along.

However, sometimes you don't want to have to fiddle with command-line parameters; you've already written a perfectly fine message with headers.

-t is generally passed to sendmail when you want to build a message envelope from an already-formatted message, with headers, etc. For example, if you had a file foo.txt with a body like this:
From: Luke Faraone 
To: John Smith
Subject: Hello, world!

Hi there.

you could send the message with a simple invocation of cat foo.txt sendmail -t. The system would take care of ensuring a Message-id was appended if appropriate, and queue the message to be sent. However, it is when you do slightly more complex invocations of sendmail that things get ambiguous.

It turns out that implementations differ on what exactly it means when you use -t in combination with naming destination addresses after the arguments to sendmail. exim4's documentation describes the situation in greater detail:
extract_addresses_remove_ argumentsUse: mainType: booleanDefault: true
According to some Sendmail documentation (Sun, IRIX, HP-UX), if any addresses are present on the command line when the -t option is used to build an envelope from a message s To:, Cc: and Bcc: headers, the command line addresses are removed from the recipients list. This is also how Smail behaves. However, other Sendmail documentation (the O Reilly book) states that command line addresses are added to those obtained from the header lines. When extract_addresses_remove_arguments is true (the default), Exim subtracts argument headers. If it is set false, Exim adds rather than removes argument addresses.

Thus, there's basically no mechanism for a program to know which behaviour to expect. God forbid two programs are installed on a system that expect different behaviours!

It appears that the default behaviour of Ruby is the opposite of what exim4 (Debian's default mail client) expects. This has resulted in numerous bug reports. Some replies suggest changing exim4's defaults, while others advocate overriding ActionMailer and friends to use sendmail -i instead, without -t.

That said, its not really clear who's wrong here; at no point does there appear to have been a definitive specification for sendmail, and as such we can hope for defined behaviour by common custom at best, and a sea of incompatibility bugs at worst. Amusingly, POSIX standards have nothing to say on this subject of sendmail at all; it defines that a mailx command must exist, but says that its sending mode may be implementation-specific.

As Matthew Garrett writes, there's not enough gin in the world.

1 January 2014

Russ Allbery: Review: The Female Man

Review: The Female Man, by Joanna Russ
Publisher: Beacon
Copyright: 1975
Printing: 1986
ISBN: 0-8070-6299-5
Format: Trade paperback
Pages: 214
The Female Man is one of the most famous novels of feminist science fiction, written by a giant of the field. Joanna Russ was one of the strongest writers of the first era of explicitly feminist SF and one of the most challenging. Along with Alice Sheldon (James Tiptree, Jr.) and Sheri Tepper, she's probably the best known author whose approach tended towards the directly confrontational. She was also a noted essayist, critic, and reviewer; her book How to Suppress Women's Writing is one of the most famous works of feminist literary criticism. That said, I wasn't sure what to expect from this book. I'd heard that it was about four different women: Jeannine, Joanna, Janet, and Jael. They're parallels of sorts but from far different cultures, alternate worlds displaced in time. I'd also heard that it was full of biting, accurate sarcasm, which is certainly the case. This is a book written from a place of righteous anger combined with keen social observation and a desire to point out how different the world could be. And how toxic male behavior towards women currently is. What it's not, however, is much of a novel, in the sense of having a clear plot that carries the reader through the book. There is a story of sorts, which revolves around the appearance of Janet in Joanna's time. Janet is a native of Whileaway, an alternate future Earth in which there are no men at all and social organization is significantly different. We get a fairly detailed description of her life and of some Whileaway customs over the course of the book. But they're mixed with excerpts from the lives of both Jeannine (from an alternate past in which the Great Depression never ended) and Joanna. If Janet's story is a sort of utopian alternate future, following the basic model of female separatist societies, Jeannine's is the dystopia drawn from our past. She's locked entirely into traditional gender roles and the desire to catch a man and be a wife. I found her the hardest character to read about by far, largely because she locks herself into the worst constraints of that role. She's almost entirely passive, and she's devoted to a man for whom it's difficult to have any respect at all. Janet is interesting in part because she's not particularly exceptional in either ability or intelligence. She just reflects the Whileaway culture, which leads to some sharp conflicts with the norms of Joanna's time. It's a very effective tactic for alienating the reader from current gender roles (or, at least, roles from the 1970s) and showing how absurd they appear with a different set of starting assumptions. Russ makes the point clearly; this is not a book that tends towards the subtle, although it can be a little hard to track the characters' progress through alternate timelines. Towards the end of the book, they meet the fourth character. (This is not really the sort of book that is possible to spoil by summary.) Jael is by far the most aggressive of any of them, the closest that any of them come to the caricature of a "man-hating feminist." One of the strengths of The Female Man is that, by the time the story reaches her, I at least was feeling quite a lot of sympathy towards her aggressively violent approach, after watching Joanna be put down and Jeannine simper ineffectually. However, it's worth nothing that Janet, the representative of the supposed utopia, does not agree with Jael or her methods, and the relationship between their worlds is interestingly complex. None of the four characters are exactly wrong (although Jeannine comes close); none of them are entirely right either. They're all struggling in different ways with how to define femininity. While the quiet and simplicity of Janet's world has definite appeal, it's the sort of utopia that one isn't sure one would actually want. And Jael has some pointed commentary on how wanting such a world is one thing, but the acts required to create it is quite another. I can see why this book is so well-known. Many of the scenes that show gender relations and male behavior are both painful and funny. Reading it forty years after it was written, it's clear that social norms have changed; men do and say things here that they would no longer be able to get away with, that are clearly from another era. But it's also clear that some of the underlying attitudes have not changed, just been cloaked in different language. And Russ's bluntness here is refreshing: this book does not pull its punches, both in openly ridiculing sexist behavior, and in showing how oppressive it is for women. That said, it didn't work as a story for me, in part because the plot is sketchy at best. It's a collection of individual excerpts and commentary, only some of which are shaped like a story. Much of it feels stream of consciousness, and when cut between three different viewpoints, it's also occasionally confusing. There are also a few too-obvious characters: some of the men, particularly in Jael's section, but also Jeannine, who I never liked even after she starts changing her mind about women's roles. I wanted a bit more story and a bit less introspection. I think this is an important book that's worth reading, and sadly it's not as obsolete as one would like. But I don't think it's entirely successful as a novel. Best treated as a piece of literary history. Rating: 6 out of 10

Andrew Pollock: [life] It's a new year, it's a new me

Well, it's 2014. The annus horribilus of 2013 is behind me. I'm entering 2014 in a much better place (on multiple levels) than I did for 2013. I'm positively excited to see what 2014 brings me. I was very fortunate to have someone I looked up to reach out to help me when my marriage fell apart. One of the things he said to me was "A year from now, you won't recognise yourself". He was right. I am such a different person today than I was a year ago. I'm happier, fitter, more self-confident. I'm focusing on different things in my life. One thing that has been a casualty, has been my geeky pursuits. There were a few times last year where I questioned if I was still a geek. I am. I've just deprioritised it. Instead, I've been putting my own self-care and my daughter above all else, meaning that my running has probably been the one thing I'd call a "hobby" at the moment. I've got some stuff in the pipeline that I'm not quite ready to write about yet, but I'm expecting a rearrangement of my time that will hopefully allow for some more geeky endeavours in 2014. I'm also hoping I can regain some momentum on blogging. Here's what I'd say my goals are, going into 2014: plus some other stuff I'm not ready to talk about yet.

31 December 2013

Paul Tagliamonte: Hy 0.9.12 released

Good morning all my hungover friends. New Hy release - sounds like the perfect thing to do while you re waiting for your headaches to go away. Here s a short-list of the changes (from NEWS) - enjoy!
Changes from Hy 0.9.11
   tl;dr:
    0.9.12 comes with some massive changes,
    We finally took the time to implement gensym, as well as a few
    other bits that help macro writing. Check the changelog for
    what exactly was added. 
    
    The biggest feature, Reader Macros, landed later
    in the cycle, but were big enough to warrent a release on it's
    own. A huge thanks goes to Foxboron for implementing them
    and a massive hug goes out to olasd for providing ongoing
    reviews during the development.
    
    Welcome to the new Hy contributors, Henrique Carvalho Alves,
    Kevin Zita and Kenan B l kba . Thanks for your work so far,
    folks!
    
    Hope y'all enjoy the finest that 2013 has to offer,
      - Hy Society
    * Special thanks goes to Willyfrog, Foxboron and theanalyst for writing
      0.9.12's NEWS. Thanks, y'all! (PT)
    [ Language Changes ]
    * Translate foo? -> is_foo, for better Python interop. (PT)
    * Reader Macros!
    * Operators + and * now can work without arguments
    * Define kwapply as a macro
    * Added apply as a function
    * Instant symbol generation with gensym
    * Allow macros to return None
    * Add a method for casting into byte string or unicode depending on python version
    * flatten function added to language
    * Add a method for casting into byte string or unicode depending on python version
    * Added type coercing to the right integer for the platform
    [ Misc. Fixes ]
    * Added information about core team members
    * Documentation fixed and extended
    * Add astor to install_requires to fix hy --spy failing on hy 0.9.11.
    * Convert stdout and stderr to UTF-8 properly in the run_cmd helper.
    * Update requirements.txt and setup.py to use rply upstream.
    * tryhy link added in documentation and README
    * Command line options documented
    * Adding support for coverage tests at coveralls.io
    * Added info about tox, so people can use it prior to a PR
    * Added the start of hacking rules
    * Halting Problem removed from example as it was nonfree
    * Fixed PyPI is now behind a CDN. The --use-mirrors option is deprecated.
    * Badges for pypi version and downloads.
    [ Syntax Fixes ]
    * get allows multiple arguments
    [ Bug Fixes ]
    *  OSX: Fixes for readline Repl problem which caused HyREPL not allowing 'b'
    * Fix REPL completions on OSX
    *  Make HyObject.replace more resilient to prevent compiler breakage.
    [ Contrib changes ]
    * Anaphoric macros added to contrib
    * Modified eg/twisted to follow the newer hy syntax
    * Added (experimental) profile module

29 December 2013

Russ Allbery: Review: The Incrementalists

Review: The Incrementalists, by Steven Brust & Skyler White
Publisher: Tor
Copyright: September 2013
ISBN: 0-7653-3422-4
Format: Hardcover
Pages: 304
The Incrementalists are a secret society that has been in continuous existence for forty thousand years. The members are immortal... sort of (more on that in a moment). They are extremely good at determining what signals, triggers, and actions have the most impact on people, and adept at using those triggers to influence people's behavior. And they're making the world better. Not in any huge, noticeable way, just a little bit, here and there. Incrementally. Brust and White completely had me with this concept. There are several SF novels about immortals, but (apart from the vampire subgenre) they mostly stay underground and find ways to survive without trying to change the surrounding human society. I love the idea of small, incremental changes, and I was looking forward to reading a book about how that would work. How do they choose what to do? How do they find the right place to push? What long-term goals would such an organization pursue, and what governance structure would they use? I'm still wondering about most of those things, sadly, since that isn't what this novel is about at all. I should be fair: a few of those questions hover in the background. There are some political arguments (that parallel arguments on Brust's blog), and a tiny bit about governance. But mostly this is a sort of romance, a fight over identity, and an extensive exploration of the mental "garden" that the Incrementalists use and build to hold their memories and share information with each other. The story revolves around two Incrementalists: Phil, who is one of the leaders and has one of the longest continuous identities of any of the group, and Renee, a new recruit by Phil who picks up the memories and capabilities (sort of) of the recently-deceased Incrementalist Celeste. Phil is the long-term insider, although a fairly laid-back one. Renee is the outsider, the one to which the story is happening at the beginning, who is taught how the Incrementalists' system works. The Incrementalists is told in alternating first-person viewpoint sections by Phil and Renee (making me wonder if the characters were each written by one of the authors). Once I got past my disappointment over this book not being quite what I wanted, the identity puzzles that Brust and White play with here caught my attention. The underlying "magic" of the Incrementalists is based on the "memory palace" concept of storing memories via visualizations, but takes it into something akin to personal alternate worlds. Their immortality isn't through physical immortality, but rather the ability to store their memories and personality in their garden, which is their term for the memory palace, and then have it imposed on someone else: a combination of reincarnation, mental takeover, and mental merging. This makes for complex interpersonal relationships and complex interactions between memory, belief, and identity, not to mention some major ethical issues, which come to a head in Phil and Renee's relationship and Celeste's lingering meddling. I think Brust and White are digging into some interesting questions of character here. There's a lot of emphasis in The Incrementalists on the ease with which people can be manipulated, and the Incrementalists themselves are far from immune. Choice and personal definition are both questionable concepts given how much influence other people have, how many vulnerabilities everyone carries with them, and how much one's opinions are governed by one's life history. That makes identity complex, and raises the question of whether one can truly define oneself. But, while all these ideas are raised, I think The Incrementalists dances around them more than engages them directly. It's similar to the background hook: yes, these people are slowly improving the world, and we see a little bit of that (and a little bit of them manipulating people for their own convenience), but the story doesn't fully engage with the idea. There's a romance, a few arguments, some tension, some world-building, but I don't feel like this book ever fully committed to any of them. One of the occasional failure modes of Brust for me is insufficient explanation and insufficient clarity, and I hit that here. My final impression of The Incrementalists is interesting, enjoyable, but vaguely disappointing. It's still a good story with some interesting characters and nice use of the memory palace concept, and I liked Renee throughout (although I think the development of the love story is disturbingly easy and a little weird). But I can't strongly recommend it, and I'm not sure if it's worth seeking out. Rating: 7 out of 10

24 December 2013

Gergely Nagy: Some free software are really free

I read a post titled "Free software is NOT Free", and that triggered a few thoughts, which I would like to share here. While the post has some good thoughts, there are points in it which I disagree with strongly. You see, I've been writing free software almost half my life, most of the things I contributed to various projects, and the vast majority of my own projects, were made because it made me feel good. Because I scratched an itch, and sharing filled me with a sense of accomplishment.I always had bills to pay, and I have a day job to get that kind of thing sorted, yet, I do not believe that the quality of my free software work suffers because of that. Do I have less time? Yes, I do. But slower pace does not mean lesser quality - more often than not, I found that slower pace leads to better considerations, more care given to each change, because fixing it later, fast, is much more costly, so you make sure that your improvements will stand the trial of time.But I digress. I want to share a different view on the topic, based on my own experiences.When I first got into free software, I was young, bright-eyed, full of hopes and illusions of grandieur. I did my part of small contributions here and there, and every bit of "Thanks!", every bit of feedback - be it criticism or rejection - contributed to my development. This part of the free software culture is still important to me, and remains a major factor behind my desire to work on free software. Later on, there came another revelation: people actually use the stuff I write! That's an awesome feeling too, but with it, comes a burden too: supporting users, and trying to cater to every need that arises. To combat this, to not drown, you need to learn to say "No". It is hard, but as maintainer of a project, it's your task to keep it maintainable, and you need to know your limits. Would it be better if you could fulfil every wish? If you could do everything you ever dreamed of? You may think the answer is yes, but you're wrong. The answer is "No".Not everything needs to be done, not every wish needs to come true. As always, there's a balance. Limiting the amount of time that can be spent on a project helps keep that balance. And when balance is found, the project becomes more mature, maintenance stops being a chore you do not want to do, but have to, and instead becomes something you actually enjoy. We are human beings, we dream and wish for a lot, and we have a hard time realising our limits. That makes us feel sad, sometimes even inadequate. Having less time puts us into a different position, where were are forced to reevaluate how we spend the little free time we have. In the long run, I believe that is good for free software.But there's an even better way to approach the problem! You want to work on your projects, right? Yet, you also want to pay your bills. And you are a fan of free (as in freedom) software. When I found myself in a position like that, when I was looking for work, I was looking at companies which are friendly towards free software, and even have a few on their portfolio. Working for such a company allows me to be happy with myself, by not betraying my own beliefs, and on the side, still spend time on my own things.When I spend my day working on free software, I feel energised, and during lunch break, I can look through my plans for my own projects (some of which are related to my work too, which makes it even easier to spend time on them), plan ahead, and go home with a head full of ideas. No exhaustion after a daunting day of hacking proprietary software, with a sinking heart. No feeling guilty of not taking good care of your projects, because you do take care! You just do it on your own terms.So this is my suggestion to independent free software hackers: find a free software friendly company, that allows - or even encourages - you to work on your own things too. That way, you can pay your bills, and work on your hobbies, and best of all: keep them as hobbies, so it does not become a burden. When a hobby becomes your job, that often leaves a sour taste behind, when you can't fulfil every need, or fix every issue.Get a free software day-job, and keep your hobby as such, and everyone benefits. You get to keep your sanity, and users will receive better care, because the developers do what they love, and not what they have to.You give up some of your freedoms, yes, by not being independent. But being independent is not always the right choice. For some people, it may well be, for some, not so much. My experience over the past 15 years or so, is that independence brings a lot more headache than I'm willing to deal with. I'd rather work for a free software friendly company, where, after three years, I get paid to write and contribute to free software, and take care of my personal projects on the side, than try do all of it on my own.None of the software I wrote is worth a full salary, and none of them requires me to work full time on them, either. I could, mind you, I have enough ideas lined up to keep me busy for years. But do I want to? Do I want to burn myself out?No. The day job satisfies my hunger for contributing to a widely used piece of software, and my hobby projects satisfy my exploratory needs. I'm happy this way, and I'm very happy that thanks to the work and dedication of some amazing people, we got this far, that one can have a day job and independent projects too, that are all free software, without having to depend on fairly unreliable sources, such as donations (mind you, donations are a nice extra).There are, of course, different situations, and different people. What works for one, may not work for the other. This is what worked for me, and I felt the need to share it, lest anyone thinks that the world's darker than it really is.

19 December 2013

Joachim Breitner: My contribution to XKCD s #949 is not needed

A few hours ago I posted about share-file, a small tool to make a local file available at a public URL so that it is transferred from my machine, without being stored at some central server. I wanted to create something like FileTea, but usable from the command line. My post made it to HackerNews (which brought my server to its knees for a while; let s say I had the opportunity to find out what setting for apache s MaxClient option works on my machine...), where the comments (besides some misunderstanding about the notable features of share-file) contained interesting links related to solving #949:

18 December 2013

Ingo Juergensmann: Debian donation to m68k arrived

The Debian m68k port has been entitled by the DPL to receive a donation of five memory expansion cards for the m68k autobuilders. The cards arrived two weeks ago and are now being shipped to the appropriate buildd admins. Adding those 256 MB memory expansion will have a huge effect to the m68k buildds, because most of them are Amigas that are currently running with "just" 128 MB. The problem with those expansion cards is to make use of them. Sounds strange but this is the story behind it.... Those memory expansion cards, namely it is the BigRamPlus from Individual Computers, are Zorro III bus cards, which has some speed limitations. The Amiga memory model is best described in the Amiga Hardware Reference Manual. For Zorro III based Amigas this is described in the section "A3000 memory map", where you can see that the memory model is divided into different address space. The most important address space is the "Coprocessor Slot Expansion" space, starting at $0800 0000. This is where the memory on the CPU accelerator cards will be found and which runs at full CPU speed. The BigRamPlus, however, is located within "Zorro III Expansion" address space at $1000 0000 and will have transfer rates of about 13 MB/s. Then again there's still the motherboard expansion memory and others like Zorro II expansion memory. Unfortunately the current kernel does not support SPARSEMEM on m68k, but is using DISCONTIGMEM as Geert Uytterhoven explained. In short: we need SPARSEMEM support to easily make use of all available memory chunks that can be found. To make it a little more difficult, Amigas do use some kind of memory priority. Memory on accelerator cards usually has a priority of 40, motherboard expansion memory has a priority of, let's say, 20 and chip memory a pri of 0. This priority usually is equivalent to speed of memory. So, we want to have the kernel loaded to accelerator memory, of course. Basically we could do that by using a memfile and define the different memory chunks in the appropriate priority list like this one:
2097152
0x08000000 67108864
0x07400000 12582912
0x05000000 268435424
Would be an easy solution, right? Except that this doesn't work out. Currently the kernel will be loaded into the first memory chunk that is defined and ignore all memory chunks before that address space. As you can see 0x07400000 and 0x05000000 would be ignored because of this. Getting confused? No problem! It will get worse! ;) There's another method of accessing memory for Amigas: it's called z2ram and will use Zorro II as, let's say, swapping area. But maybe you guessed it: z2ram does not work for Zorro III memory (yet). So, this won't work either. Geert suggested to use that Zorro III memory as mtd device and finally this worked out! You'll need these modules in your kernel:
CONFIG_MTD=m
CONFIG_MTD_CMDLINE_PARTS=m
CONFIG_MTD_BLKDEVS=m
CONFIG_MTD_SWAP=m
CONFIG_MTD_MAP_BANK_WIDTH_1=y
CONFIG_MTD_MAP_BANK_WIDTH_2=y
CONFIG_MTD_MAP_BANK_WIDTH_4=y
CONFIG_MTD_CFI_I1=y
CONFIG_MTD_CFI_I2=y
CONFIG_MTD_SLRAM=m
CONFIG_MTD_PHRAM=m
Then you just need to create the mtd device and configure it as swap space:
/sbin/modprobe phram phram=bigram0,0x50000000,0xfffffe0
/sbin/modprobe mtdblock
/sbin/mkswap /dev/mtdblock0
/sbin/swapon -p 5 /dev/mtdblock0
And then you're done:
# swapon -s
Filename Type Size Used Priority
/dev/sda3 partition 205932 8 1
/dev/sdb3 partition 875536 16 1
/dev/mtdblock0 partition 262136 53952 5
To make it even worse (yes, there's still room for that! ;)) you can put two memory expansion cards into one box:
# lszorro -v
00: MacroSystems USA Warp Engine 40xx [Accelerator, SCSI Host Adapter and RAM Expansion]
40000000 (512K) 01: Unknown device 0e3b:20:00
50000000 (256M) 02: Village Tronic Picasso II/II+ RAM [Graphics Card]
00200000 (2M) 03: Village Tronic Picasso II/II+ [Graphics Card]
00e90000 (64K) 04: Hydra Systems Amiganet [Ethernet Card]
00ea0000 (64K) 05: Unknown device 0e3b:20:00
60000000 (256M)
The two "Unknown device" entries are the two BigRamPlus cards. As you can see card #1 starts at 0x50000000 and card #2 starts at 0x60000000. Unfortunately the phram kernel module can be loaded twice with different start addresses, but the idea to start at 0x50000000 with a size of 512M won't work either as there seems to be a reserved 0x20 bytes a range at the beginning of each card. Anyway... So, to make a very long and weird story short: the donated memory cards from Debian can now be used as additional and fast swap space for the buildds as long as it takes to get SPARSEMEM support working. Thanks again for donating the money for those memory expansion cards for the good old m68k port. Once done SPARSEMEM support in the m68k will benefit not only these cards in Amigas but Ataris as well.
Kategorie:

15 December 2013

Alessandro Ghedini: Building Debian packages using Linux namespaces

In the past few days I have been messing around with Linux namespaces, and developed a little tool (pflask) that automates the creation of simple Linux containers based on them (a sort of chroot(8) on steroids if you will). While the whole raison d' tre behind this project was "just because", and many more mature solutions exist, I decided that it'd be nice to find an actual use case for this (otherwise I tend to lose interest pretty quickly) so I wrote a lil (and rather dumb) pbuilder clone that uses pflask instead of chroot. The nice thing about pflask is that, differently from e.g. LXC, it doesn't need any pre-configuration and can be used directly on a vanilla debootstrap(8)ed Debian system:
$ sudo mkdir -p /var/cache/pflask
$ sudo debootstrap --variant=buildd $DIST /var/cache/pflask/base-$DIST-$ARCH
Where $DIST and $ARCH are e.g. unstable and amd64. Once that's done just run pflask-debuild on the package sources:
$ apt-get source somepackage
$ cd somepackage-XYX
$ pflask-debuild
The script will take care of creating a new container, chroot(2)ing into it, installing all the required dependencies, building and signing the package (it also runs lintian!). The main difference from pbuilder is that pflask will mount a copy-on-write filesystem (using AuFS) on the / of the container so that any modification (e.g. installation of packages) can be easily discarded once the container terminates (similarly to what cowbuilder(8) does, modulo the hardlinks hack). Additionally, thanks to the mount namespace created inside the container, all of this will be isolated from the host system and other containers, so that multiple packages can be built simultaneously on the same base debootstrapped directory. Another possibility would be that of disabling the network inside the container using a network namespace, in order to prevent the package build system from downloading stuff from Internet while at the same time maintaining the network active on the host system, but I haven't done any experiment in this direction yet. Note though that all of this is rather crude and experimental, but as a little hack it seems to work rather well (YMMV).

11 December 2013

Daniel Pocock: Get WebRTC going faster

On Saturday, Lumicall began offering free calls from browser to mobile using the free and open WebRTC technology. It should be no surprise that the service has been popular. Is it really free and open? The only way to prove this technology is free is to help people implement this for themself. On Monday, I uploaded reSIProcate v1.9.0 beta7 packages to Debian. The reSIProcate SIP proxy, repro, is one of the core components of the solution behind the free Lumicall service. Simply install the repro and resiprocate-turn-server packages using apt-get and make the following changes to the configuration (use your own IP addresses of course). I've taken this diff from my own runtime environment, only hiding my passwords, so that you can see exactly how I got it working:
--- repro.config.orig	2013-12-11 17:36:27.179228324 +0100
+++ repro-ws.sip5060.net.config	2013-12-11 17:48:24.159938649 +0100
@@ -143,6 +143,41 @@
 # Transport6TlsClientVerification = None
 # Transport6RecordRouteUri = sip:h1.sipdomain.com;transport=WS
 
+Transport1Interface = 195.8.117.57:80
+Transport1Type = WS
+Transport1RecordRouteUri = auto
+
+Transport2Interface = 2001:67c:1388:1000::57:80
+Transport2Type = WS
+Transport2RecordRouteUri = auto
+
+Transport3Interface = 195.8.117.57:5060
+Transport3Type = TCP
+Transport3RecordRouteUri = auto
+
+Transport4Interface = 2001:67c:1388:1000::57:5060
+Transport4Type = TCP
+Transport4RecordRouteUri = auto
+
+Transport5Interface = 195.8.117.57:443
+Transport5Type = WSS
+#Transport5RecordRouteUri = auto
+Transport5TlsDomain = ws.sip5060.net
+Transport5TlsClientVerification = None
+Transport5RecordRouteUri = sip:ws.sip5060.net;transport=WSS
+Transport5TlsCertificate = /etc/ssl/ssl.crt/ws.sip5060.net-bundle.crt
+Transport5TlsPrivateKey = /etc/ssl/private/ws.sip5060.net-key.pem
+
+Transport6Interface = 2001:67c:1388:1000::57:443
+Transport6Type = WSS
+#Transport6RecordRouteUri = auto
+Transport6TlsDomain = ws.sip5060.net
+Transport6TlsClientVerification = None
+Transport6RecordRouteUri = sip:ws.sip5060.net;transport=WSS
+Transport6TlsCertificate = /etc/ssl/ssl.crt/ws.sip5060.net-bundle.crt
+Transport6TlsPrivateKey = /etc/ssl/private/ws.sip5060.net-key.pem
+
+
 # Comma separated list of DNS servers, overrides default OS detected list (leave blank 
 # for default)
 DNSServers =
@@ -455,7 +490,7 @@
 ForceRecordRouting = false
 
 # Assume path option
-AssumePath = false
+AssumePath = true
 
 # Disable registrar
 DisableRegistrar = false
@@ -481,7 +516,7 @@
 # WARNING: Before enabling this, ensure you have a RecordRouteUri setup, or are using
 # the alternate transport specification mechanism and defining a RecordRouteUri per
 # transport: TransportXRecordRouteUri
-DisableOutbound = true
+DisableOutbound = false
 
 # Set the draft version of outbound to support (default: RFC5626)
 # Other accepted values are the versions of the IETF drafts, before RFC5626 was issued
@@ -505,7 +540,7 @@
 # WARNING: Before enabling this, ensure you have a RecordRouteUri setup, or are using
 # the alternate transport specification mechanism and defining a RecordRouteUri per
 # transport: TransportXRecordRouteUri
-EnableFlowTokens = false
+EnableFlowTokens = true
 
 # Enable use of flow-tokens in non-outbound cases for clients detected to be behind a NAT.  
 # This a more selective flow token hack mode for clients not supporting RFC5626.  The 

This is a diff against the /etc/repro/repro.config file distributed in the Debian package version 1.9.0~beta7-1. In the example above, I've included WSS transport defintions for WebSockets over TLS. Use the standard procedure for creating webserver SSL certificates to create certificates for repro and make sure you insert the correct filenames in the TLS parameters above. I've also duplicated every transport for IPv6. If you don't want TLS/WSS or IPv6, just comment those entries out (and renumber the remaining transports). Web-based SIP proxy setup Once you have repro running, go to the web admin interface (port 5080, username: admin, password: admin) and finish the setup using the web UI. The following steps are essential:
  • Add your domain (and restart the repro daemon after adding your domain)
  • Add some users
  • Add any routes to external services (optional - in my next blog I'll demonstrate how to route WebRTC calls to Asterisk using the Debian packages and less than 20 lines of configuration)
Set up reTurn, the TURN server Just put your IP addresses in /etc/reTurnServer.config and add the following line to /etc/reTurnServer-users.txt:
test:notasecret:reTurn:authorized
IMPORTANT: the realm in the users file (reTurn in the example and default config) must be identical to the AuthenticationRealm in the /etc/reTurnServer.config file. On your own web site Simply install your own apache server and clone the webrtc.lumicall.org demo site. Modify the file js/custom.js and include the settings for your own server.
# cd /var/www
# mkdir webcall
# cd webcall
# wget -r -nH http://webrtc.lumicall.org
# vi js/custom.js
In the custom.js, make sure you use a ws:// URL if you didn't set up SSL certificates and use a wss:// URL if you did. The IP or domain of your repro server must be in the ws:// or wss:// URL. Now navigate to the URL ending with /webcall on your server. For RHEL, Fedora and other RPM users Can somebody please assist with the review of the cajun-jsonapi dependency package so I can upload this new version of reSIProcate to Fedora? I'm also planning to make v1.9.0 available in EPEL6 when it is released in January. Questions? Please come and ask questions or discuss your experiences on the Free RTC mailing list sponsored by FSF Europe.

Next.

Previous.