Search Results: "ron"

5 February 2014

Wouter Verhelst: "Nuclear precision weapons"

Now that the madness is mostly over, I have some time to catch up on reading my newspaper. Last week, one article in De Standaard talks about the nuclear weapons on Belgian soil which officially didn't exist (until wikileaks proved otherwise) and which are now apparently going to be modernized. The newer model would be "nuclear precision weapons". Ignoring the question of whether today's world still requires nuclear bombs (this may or may not be true, I don't care), I question the logic which leads to that phrase. A nuclear weapon is a weapon of mass destruction. By definition, a weapon of mass destruction causes collateral damage. By definition, a precision weapon is a weapon that does not cause collateral damage or, at the very lease, where every effort is made to limit the amount of collateral damage. Even the very first nuclear bombs were capable of destroying entire cities. Today's nuclear weapons, even the smaller ones, are far more powerful than those. Don't get me wrong: I'm not a peace activist. In fact, I have been contracted by companies who produce military equipment, and don't feel bad about that. But to claim that it is possible to create "nuclear precision weapons" is to deceive oneself. A nuclear weapon is not very precise.

4 February 2014

Enrico Zini: debops

Debops What I like the most about being a Developer is building tools to (hopefully) make someone's life better. I like it when my software gets used, and people thank me for it, because there was a need they had that wasn't met before, and thanks to my software now it is being met. I am maintaining software for meteorological research that is soon going to be 10 years old, and is still evolving and getting Real Work done. I like to develop software as if it is going to become a part of human cultural heritage, developing beyond my capacity, eventually surviving me, allowing society to declare that the need, small as it was, is now met, and move on to worry about some other problem. I feel that if I'm not thinking of my software in that way, then I am not being serious. Then I am not developing something fit for other people to use and rely on. This involves Development as much as it involves Operations: tracking security updates for all the components that make up a system. Testing. Quality assurance. Scalability. Stability. Hardening. Monitoring. Maintenance requirements. Deployment and upgrade workflows. Security. I came to learn that the requirements put forward by sysadmins are to be taken seriously, because they are the ones whose phone will ring in the middle of the night when your software breaks. I am also involved in more than one software project. I am responsible for about a dozen web applications deployed out there in the wild, and possibly another dozen of non-web projects, from terabyte-sized specialised archival tools to little utilities that are essential links in someone's complex toolchain. I build my software targeting Debian Stable + Backports. At FOSDEM I noticed that some people consider it uncool. I was perplexed. It provides me with a vast and reasonably recent set of parts to use to build my systems. It provides me with a single bug tracking system for all of them, and tools to track known issues in the systems I deployed. It provides me with a stable platform, with a well documented upgrade path to the next version. It gives me a release rhythm that allows me to enjoy the sweet hum of spinning fans thinking about my next mischief, instead of spending my waking time chasing configuration file changes and API changes deep down in my dependency chain. It allows me to rely on Debian for security updates, so I don't have to track upstream activity for each one of the building blocks of the systems I deploy. It allows me not to worry about a lot of obscure domain specific integration issues. Coinstallability of libraries with different ABI versions. Flawless support for different versions of Python, or Lua, or for different versions of C++ compilers. It has often happened to me to hear someone rant about a frustrating situation, wonder how come it had never happened to me, and realise that someone in Debian, who happens to be more expert than I can possibly be, had thought hard about how to deal with that issue, years before. I know I cannot be an expert of the entire stack from bare iron all the way up, and I have learnt to stand on the shoulders of giants. 'Devops' makes sense for me in that it hints at this cooperation between developers and operators, having constructive communication, knowing that each side has their own needs, trying their best to meet them all. It hints at a perfect world where developers and operators finally come to understand and trust each other's judgement. I don't know that perfect world, but I, a developer, do like to try to understand and trust the judgement of sysadmins. I sympathise with my sysadmin friends who feel that devops is turning into a trend of developers thinking they can do without sysadmins. Reinventing package managers. Bundling dependencies. Building "apps" instead of components. I wish that people who deploy a system built on such premises, have it become so successful that they end up being paid to maintain them for their whole career. That is certainly what I wish and strive for, for me and my own projects. In my experience, a sustainable and maintainable system won't come out of the startup mindset of building something quick&dirty, then sell it and move on to something else. In my experience, the basis for having sustainable and maintainable systems have been well known and tested in Debian, and several other distributions, for over two decades. At FOSDEM, we thought that we need a name for such a mindset. Between beers, that name came to be "debops". (It's not just Debian, though: many other distributions get it right, too)

Martín Ferrari: Telegram

This weekend I attended FOSDEM, as I did for the past 4 years. As always, it was a great experience, even if most of the talks I attended were not so interesting. One of the great things about these events, is to get together with other Free Software enthusiasts. One of them introduced me to the new kid on the block: Telegram. It is advertised as a free and secure replacement for WhatsApp, a mobile application I have been refusing to use for months, much to the anger of friends and family. People use WhatsApp because it saves them money to send SMS-like messages to other people. Which in my mind does not make much sense, as I can use email or XMPP for that same purpose, without having to enter just another walled garden. Sadly, people nowadays are using email less and less, and think that it is easier to send a Facebook message (it is not), and XMPP has became a de-facto walled garden since Google betrayed its users by dropping federation. WhatsApp has the advantage of having millions of users already inside their walled garden, like it happened with Facebook, MSN, or ICQ back in the day. Their success can possibly be attributed to the foolproof system it uses to discover contacts: it just sends your entire address book to their servers for matching against already registered users, which is in itself enough reason not to use it. So, when I was told of a secure, free and open alternative, I was eager to try it out. I started by opening their website to see what it was about. There, I found the first signs that something was wrong: no mention of licenses, almost no technical detail of how the protocol works, or how security is achieved. Still, my friend wanted to talk to me, so I rushed to install it and accept a laundry list of permissions in Android. That was a big mistake. The first thing the application did was to check my address book for contacts, without my permission or knowledge. I got greeted by being told that some of my contacts already have Telegram installed, and since then I keep getting notification that some more of my geek friends are installing it. So it is obvious that this company got all my records, breaking my privacy and security. This is enough for me to remove this piece of software from my phone, but it is not the end of the story. When checking it in more detail I found several other problems. The supposed "secret chat" cannot be possibly secure, as there is no verification of the remote party. It "just works", who cares if there is a man in the middle or not. Supposedly it employs a peer-to-peer connection, but I haven't verified that. On the other hand, the non-"secret" conversations are all routed to the main server. The client-to-server communication is supposedly encrypted, but it uses a home-made protocol (which everybody knows is a recipe for disaster), and the server has access to the cleartext of all your communications. Then, browsing the website, I've found that this "open" and "free" offer does not even have all the source code released. In particular, the server code is not public, nor you can set up your own server. Of course, it does not have federation, so even with the server code you wouldn't be able to talk with your friends. It all depends on two men funding and maintaining the project, so when the funds run out, one can only expect that all the users will be left in the dark. I think these arguments are enough to realise that Telegram is only marginally better than WhatsApp. It offers encryption at the transport layer, but you still are contributing to another walled garden, you are at the mercy of a company which does not have a funding plan, and the security practises range from weak to disturbing. In closing, if you are going to compromise your privacy, give away all your contacts' information, and rely on a single company to keep in touch with people, you might as well go and use what everybody else is using. Update: Even before submitting this post, I've found that some more qualified people has already dissected Telegram and concluded that it is basically snake oil. Please see this article. and proceed to uninstall Telegram.

Russell Coker: Clothing and Phone Cameras

In 2012 I wrote about my jeans from Rivers that fit the largest available phones (and the smaller tablets) in their pockets [1]. Those jeans are still working well for me, I can add the fact that they don t wear out quickly to the list of positive attributes. Recently my sister asked for advice on getting a new phone, she was considering the Samsung Galaxy Note 2 (the phone I m using now) because it apparently takes better pictures than the Nexus 4 she s using. I ve used both those phones and I hadn t noticed a difference in picture quality, but there is some variation in manufacturing and it could be that I ve got a below average Note 2 and a better than average Nexus 4 so I ll assume for the sake of discussion that my sister would actually get an improvement in picture quality by using a Note 2. If you have a phone that doesn t have the picture quality you desire then one option is to buy a phone with a better camera, but you will be limited by issues of physics. A thin phone has a short focal length which means that the lens has to be small and therefore the amount of light that gets to the sensor is small. The Nokia Lumia 1020 has some of the best camera hardware that you ll find in a phone, but it s still only 14.5mm thick where the camera is and that will limit the quality a lot. Any compact camera should be able to beat all phone cameras in terms of picture quality in most areas. The Samsung Galaxy Camera [2] is also worth considering, it has more features than a typical compact camera and good GUI that allows novice photographers to take advantage of it. Also being able to blog your photos directly from the camera could be a useful feature. But the big down-side of a compact camera is that it s not that compact. Most people won t find it convenient to carry a compact camera with them at all times and therefore they might miss a good opportunity to take a photo. The Galaxy Note series of phones also suffer in this regard because they are larger than most phones. If your phone won t fit in your pocket and you have it in your backpack when on the move or on a bench at home then you will probably miss some good photos. As I was at a Rivers store recently I tested my Note 2 in the pockets of women s jeans. Rivers scored very poorly in this regard, one pair of women s jeans had fake pockets (this is just wrong for working clothes) and of the rest only one pair could fit a Note 2. The pair that fit a Note 2 didn t completely enclose the phone, one corner was sticking out, this would probably give a risk of having the phone fall out of the pocket and cause some discomfort to the wearer. I have a pair of shorts with similar size pockets and find it very annoying with the Note 2 in the pocket (for about 10 months of the year I wear jeans so this isn t a big deal). Rivers jeans only count as geeky jeans for male geeks. It s disappointing that with about a dozen different styles of women s jeans there didn t seem to be a single one with pockets of comparable size to the men s jeans. I had to recommend that my sister not get a phone from the Galaxy Note series if taking pictures is a priority due to the apparent difficulty in getting it to fit in a pocket and the probability that she would miss good photos due to this. In past discussions of phone size there have been mentions of the possibility of getting clothing altered. Does anyone have a good experience in getting clothes altered to have bigger pockets or in the case of women s clothing to have fake pockets replaced with real ones?

Andrew Pollock: [life] Day 8, another Kindergarten day

Yesterday was pretty crazy busy. I woke up a little bit late because I didn't have Zoe, and decided to head down to Jaycar to grab some components and other bits and pieces for my little spontaneous electronics project. I got home from Jaycar and fiddled around for a bit, and then biked down to Bulimba to get a ferry across to a bus so I could get into the city for lunch with my accountant. I ate my fill of sashimi while discussing the best corporate structure for my endeavours this year. Lunch ran a bit long, and it was going to be tight trying to bus it back to the ferry to retrieve my bike, so I decided to cab it back to Bulimba. I retrieved my bike and raced home and jumped in the car. It was just going to be too tight to bike over there. It probably worked out for the best, because I got to the Kindergarten and Zoe was fast asleep still. She took quite a lot of effort to wake. I think they'd only got her down for a nap not long before. I eventually roused her, but she was very groggy and a little bit sad about the whole thing. I think she would have been a sad panda if she'd had to then get on the bike. I asked her if she wanted to make some play dough or go swimming when we got home, and she chose swimming. I asked her if she wanted to bike or drive, and she wanted to drive. We took the new pool noodle that I picked up the other week and headed down to Colmslie Pool The pool was pretty busy with a lot of after school swim squads, but there was still enough room for Zoe to play around. Her sinkies were very popular and were "borrowed" extensively, and she really enjoyed riding the pool noodle like a horse. We got home and Sarah was waiting to pick up Zoe and I headed off for a massage, and then my girlfriend came over for dinner.

3 February 2014

Evgeni Golov: Monitoring your Puppet nodes using PuppetDB

When you run Puppet, it is very important to monitor whether all nodes have an uptodate catalog and did not miss the last year of changes because of a typo in a manifest or a broken cron-script. The most common solution to this is a script that checks /var/lib/puppet/state/last_run_summary.yaml on each node. While this is nice and easy in a small setup, it can get a bit messy in a bigger environment as you have to do an NRPE call for every node (or integrate the check as a local check into check_mk). Given a slightly bigger Puppet environment, I guess you already have PuppetDB running. Bonuspoints if you already let it save the reports of the nodes via reports = store,puppetdb. Given a central knowledgebase about your Puppet environment one could ask PuppetDB about the last node runs, right? I did not find any such script on the web, so I wrote my own: check_puppetdb_nodes. The script requires a recent (1.5) PuppetDB and a couple of Perl modules (JSON, LWP, Date::Parse, Nagios::Plugin) installed. When run, the script will contact the PuppetDB via HTTP on localhost:8080 (obviously configurable via -H and -p, HTTPS is available via -s) and ask for a list of nodes from the /nodes endpoint of the API. PuppetDB will answer with a list of all nodes, their catalog timestamps and whether the node is deactivated. Based on this result, check_puppetdb_nodes will check the last catalog run of all not deactivated nodes and issue a WARNING notification if there was none in the last 2 hours (-w) or a CRITICAL notification if there was none for 24 hours (-c). As a fresh catalog does not mean that the node was able to apply it, check_puppetdb_nodes will also query the /event-counts endpoint for each node and verify that the node did not report any failures in the last run (for this feature to work, you need reports stored in PuppetDB). You can modify the thresholds for the number of failures that trigger a WARNING/CRITICAL with -W and -C, but I think 1 is quite a reasonable default for a CRITICAL in this case. Using check_puppetdb_nodes you can monitor the health of ALL your Puppet nodes with a singe NRPE call. Or even with zero, if your monitoring host can access PuppetDB directly.

Wouter Verhelst: Waking up

Today is the monday after FOSDEM 2014, and I'm slowly waking up. Being in charge of the video team, and after having spent a whole week preparing followed by a whole weekend stressing about, today is a day of slowly waking up from hibernation and returning gear to various rental companies. At FOSDEM 2013, we had recorded five rooms. Even though we did lose a few talks, I felt pretty good about the whole thing, and I think we did pretty well then. It had been my plan to increase the number of rooms to be recorded from 5 to a reasonably higher number say, 10 or so but someone convinced me to do all of them. I'll readily admit I was afraid. Scratch that, I was terrified. Petrified, that we wouldn't be able to pull it off. For previous editions, we'd kept the number conservative, since video work is a high-risk business: if something goes wrong with volunteers work (say, the heralding volunteer doesn't show up), you find a replacement, or you have a staff member introduce a speaker, and the show will go on. Not so with video work: if something minor goes wrong, you will most likely lose content. If something minor goes wrong in one devroom, I'll deal with it. If something minor goes wrong in 20% of devrooms, that's 4 to 5 devrooms, and I can't deal with it. Moreover, finding enough volunteers to manage the video for five rooms was a challenge last year; I couldn't even begin to imagine how to do so for 22 of them. We did work out a way that didn't seem too far-fetched, and that might work: why not look for volunteers from the devrooms themselves? But then, they couldn't be expected to have the required experience, so we'd have to do a lot of handholding. The week before FOSDEM, four of us sat down in an office in Mechelen, and started preparing. We had a lot to do; and when you do, more jobs keep being added to the pile; TODO lists tend to grow longer, not shorter. Installing servers, testing cameras, setting twinpact dipswitches, configuring laptops. Changing a few tests in the config management system, and having to go over the laptops again. Buying extra gear. Buying yet more extra gear, several times. Buying cardboard boxes when it appeared that the plastic foldable ones we'd ordered wouldn't make it in time, and unloading the pallet of them when it did, after all, so we could load them into the van with all of the other gear. Laminating instruction sheets. Receiving shipments of rented gear. Calling another supplier last minute when one of our other suppliers had contacted us to let us know one of their cameras that they'd promised us had broken down, and couldn't be repaired in time. Calling that last-minute supplier again when we found out that some of the cameras we'd rented could do HDV only (and not DV), and we needed to find more than the ones we'd already asked for. Discovering that his cameras were of the same problematic model and so also couldn't be used. The feeling of relief when he called me a few hours later and told me he'd managed to scrounge up enough cameras to help us out after all. And then it was time to go to the ULB and start setting up everything. On friday, we had a little bit of help, but did have to do most of it ourselves. After setting up everything in every room and testing as much as possible within the time constraints that we had, we picked up the most expensive bits of the gear, brought it down to a safe place, and went home. On saturday and sunday, for me it was mostly a matter of handing out gear to various devrooms, running around to fix various issues, and then in the evening receiving boxes back, using a checklist to ensure everything expected was in the box (and that things not needed would remain in the room, or be sent back). It was exhausting, to the extent that on saturday afternoon, I had to take a bit of time off so I could go and rest. I think it's safe to say that something like this hasn't been done before. FOSDEM is a huge conference; most conferences with multiple rooms don't have more somewhere between 5 and 10 of them. The fact that we have so many more, and the fact that we record all of them, puts FOSDEM video work in a class of its own, to the extent that the professional cameramen I talked to in order to get the right cameras would incredulously ask "what do you need that for?" when I gave them the numbers we needed. As such, I didn't expect, nor was aiming for, perfection. Unfortunately, with video work it is close to impossible to attain perfection; I have told multiple people during the event that I would be happy if we reached 85% of talks recorded at acceptable quality; and though it is much too soon to say for certain, my gut feeling tells m that we've probably achieved that. One thing we hadn't planned for was streaming. During the past three years, FOSDEM enlisted the help and sponsorship of flumotion to get streaming going. Unfortunately, that did not work out this year; and since we already had far too much on our plates with everything else, we decided to forego streaming and focus on recording, only. A few days before the event, however, Steinar Gunderson took it upon himself to fix that. While we couldn't support him a great deal, we could give him access to the secondary laptops (which weren't otherwise doing much anyway) on which he could then do his thing. This was mostly transparent to us; we did communicate to some extent (e.g., when a machine that had been down had fixed and was working again), but he mostly did his thing while we did ours. Full details, for those who want it, in the linked blog post. Today, then, I spent most of my time handling hardware: waiting for other members of FOSDEM to drive the van up to my office; waiting for the laptop rental company to retrieve most of the laptops; waiting for two of the camera rental companies to retrieve their cameras and tripods; driving over to the final camera rental company to return their cameras and tripods; and finally, driving up to the ULB to retrieve the last laptops who had been copying files all night. As I write this, some video files have already been uploaded to the FOSDEM video archive. However, these are extremely low-quality renditions of video snippets that need to be reviewed; they are not ready yet for public consumption. When those files exist, expect an announcement through the main FOSDEM website.

Dirk Eddelbuettel: Rcpp 0.11.0

A new major release 0.11.0 of Rcpp is now on the CRAN network for GNU R and binaries for Debian have been uploaded as well. Before you read on, please note that this release will most likely require a reinstallation of all your packages using Rcpp as it now works without a user-facing shared library. The biggest change in this version is that it is now effectively headers-only. Before you wonder, there is still compiled code provided by Rcpp for use by packages. But this uses the function registration mechanism of GNU R instead via instantiation at package startup. This does make package building easier, will remove the need to query GNU R for the Rcpp library file in src/Makevars when compiling, and should generally avoid building issues such as the dreaded fails on paths with spaces still annoying users of a certain OS. There are a number of other fixes, upgrades and other extensions detailed in NEWS file extract below, in the ChangeLog file in the package and on the Rcpp Changelog page as well as in a release announcement I'll post later.
Changes in Rcpp version 0.11.0 (2014-02-02)
  • Changes in Rcpp API:
    • Functions provided/used by Rcpp are now registered with R and instantiated by client package alleviating the need for explicit linking against libRcpp which is therefore no longer created.
    • Updated the Rcpp.package.skeleton() function accordingly.
    • New class StretchyList for pair lists with fast addition of elements at the front and back. This abstracts the 3 functions NewList, GrowList and Insert used in various packages and in parsers in R.
    • The function dnt, pnt, qnt sugar functions were incorrectly expanding to the no-degree-of-freedoms variant.
    • Unit tests for pnt were added.
    • The sugar table function did not handle NAs and NaNs properly for numeric vectors. Fixed and tests added.
    • The internal coercion mechanism mapping numerics to strings has been updated to better match R (specifically with Inf, -Inf, and NaN.)
    • Applied two bug fixes to Vector sort() and RObject definition spotted and corrected by Kevin Ushey
    • New checkUserInterrupt() function that provides a C++ friendly implementation of R_CheckUserInterrupt.
  • Changes in Rcpp attributes:
    • Embedded R code chunks in sourceCpp are now executed within the working directory of the C++ source file.
    • Embedded R code chunks in sourceCpp can now be disabled.
  • Changes in Rcpp documentation:
    • The Rcpp-FAQ and Rcpp-package vignettes have been updated and expanded.
    • Vignettes are now typeset with grey background for code boxes.
    • The bibtex reference file has been update to reflexct current package versions.
  • Changes in Rcpp unit tests:
    • The file tests/doRUnit.R was rewritten following the pattern deployed in RProtoBuf which is due to Murray Stokely
    • The function test() was rewritten; it provides an easy entry point to running unit tests of the installed package
Thanks to CRANberries, you can also look at a diff to the previous release 0.10.6. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Andrew Pollock: [geek] Yep, I'm still a geek

I did some soldering tonight, and it felt good. I haven't really done much soldering since I built my cat water bowl monitor, and that's going back to the last decade. My apartment building has an intercom-triggered release to the common front door, and the latch has been sticking a bit lately. It stuck today when Megan's Dad came to pick her up, and it stuck when Sarah came to pick up Zoe. It's annoying to have to trek all the way downstairs to let them in and then walk back up again. I decided I'd try and look into what the problem was today after everyone left. A visual inspection of the latch showed nothing obvious, and no one else was home to push the door unlock button for me while I was at the front door, so I decided to have a look at how it worked from within my apartment. I pulled the intercom phone off the wall and identified the two wires that got shorted when you pressed the unlock button, and shorted them by hand with a wire, and sure enough, I could hear the door solenoid releasing. I was thinking to myself, "if only there was a way to trigger this with my phone". I have a RaspberryPi and an Arduino board at my disposal, but I didn't really want to risk frying either of them. Then I remembered that I had a BoArduino that I'd bought at the Maker Faire years ago, but never assembled. I figured it'd make a good sacrificial lamb. So I rummaged around and found it, and busted out the 240v soldering iron I bought recently, but haven't used yet, and set about slapping it together. Man, it was fun. I had it half built before yoga, and finished it off when I got home. It even worked first go. I felt such a sense of accomplishment. The problem is, my general electronics fu is so weak, I'm not sure where to go from here, and my multimeter needs a new battery. I think the voltage on the wire for the intercom is higher than the BoArduino likes. I'd naively thought I could just connect the two wires of interest, one to ground and the other to one of the digital I/O pins, and have the BoArduino act as a switch, but that's not working. If I wire it one way, it unlocks the door instantly, even with the board powered off, and if I wire it the other way, it seems to half power up the board just using the power of the intercom system. So I'm obviously doing something wrong. My plan is to just have the BoArduino simulate pressing the button for 5 seconds at a time, every 10 seconds or so. Then I can be downstairs wiggling the latch to see what's up with it. Or I could just place a maintenance request with the Body Corporate. At least I had fun assembling the BoArduino, and now I have a spare one. It was nice to be able to go from problem to idea to execution in one night, even if the execution failed.

Andrew Pollock: [life] Day 7, Kindergarten again

Today was a pretty quiet day, because Zoe had Kindergarten again. Nominally, I'm supposed to be doing Debian stuff today, but I had some work presentations I wanted to watch while I still had the chance, so I mostly watched them for four hours while blowing the cobwebs out of my development environment. Zoe slept well last night and slept in a little bit to a whopping 6:30am. We got out the door at about 8:05am and by the time I'd got the bike trailer on the bike and made it to Kindergarten it was about 8:30am. Drop off went pretty smoothly. I decided to go for a run after I got back home, so I banged out 5km. My running fitness has been a bit off of late, partially because of the heat, and partially because I just haven't been doing it enough. I need to get back up to 10km as soon as possible so I can try increasing the distance. Both of Megan's parents work, and so I'm helping them out with picking her up on Mondays and Wednesdays. Today was the day of doing that. It was certainly an interesting experience having to wrangle two kids instead of one. Megan's a good kid, so it was no trouble. To kill time, we went to the car wash, where I bought the girls babyccinos while we waited, and then we headed into the Valley so I could check my post office box. After we got home, I attempted to do something I'd seen on Mister Maker. He basically squirted some PVA glue, some paint and some detergent into a cup and then mixed it all up and then blew into it with a straw until the bubbles rose over the top of the cup and pressed down on it with some paper. I used bubble mix instead of detergent because I figured it would have to be as good, but only Zoe's cup really worked as intended. Then, despite much discussion about it, Zoe decided to suck on the straw instead of blow it and so I had to race her off to the bathroom to rinse her mouth out. After that the girls played a bit more on their own, then Zoe wanted to watch TV but Megan didn't, so we painted a butterfly together and then Megan's Dad arrived to pick her up, and Sarah arrived not long after that. I killed some time doing a spontaneous electronics project before yoga and that was my day.

2 February 2014

Gunnar Wolf: CuBox-i4Pro

CuBox-i4Pro
Somewhere back in August or September, I pre-ordered a CuBox-i A nicely finished, completely hackable, and reasonably powerful ARM system, nicely packaged and meant to be used to hack on. A sweet deal! There are four models (you can see the different models' specs here) I went for the top one, and bought a CuBox-i4Pro. That means, I have a US$130 nice little box, with 4 ARM7 cores, 2GB RAM, WiFi, and... well, all of its basic goodies and features. For some more details, look at the CuBox-i block diagram. I got it delivered by early January, and (with no real ARM experience on my side) I finally got to a point where I can, I believe, contribute something to its adoption/usage: How to get a basic Debian system installed and running in it. The ARM world is quite different to the x86 one: Compatibility is much harder, the computing platform does not self-describe properly, and a kernel must first understand how a specific subarchitecture is before being able to boot on it. Somewhere in the CuBox forums (or was it the IRC channel?) I learnt that the upstream Linux kernel does not yet boot on the i.MX6 chip (although support is rumored to be merged for the 3.14 release), so I am using both a kernel and an uBoot bootloader not built for (or by) Debian people. Besides that, the result I will describe is a kosher Debian install. Yes, I know that my orthodox friends and family will say that 99% kosher is taref... But remember I'm never ever that dogmatic. (yeah, right!) Note that there is a prebuilt image you can run if you are so inclined: In the CuBox-i forums and wiki, you will find links to a pre-installed Debian image you can use... But I cannot advise to do so. First, it is IMO quite bloated (you need a 4GB card for a very basic Debian install? Seriously?) Second, it has a whole desktop environment (LXDE, if I recall correctly) and a whole set of packages I will probably not use in this little box. Third, there is a preinstalled user, and that's a no-no (user: debian, password: debian). But, most importantly, fourth: It is a nightly build of the Testing (Jessie) suite... Built back in December. So no, as a Debian Developer, it's not something we should recommend our users to run! So, in the end and after quite a bit of frustration due to my lack of knowledge, here goes the list of steps I followed:
Using the CuBox
On the i2 and i4 models, you can use it either with a USB keyboard and a HDMI monitor, or by a serial consoles (smaller models do not have a serial console). I don't have a HDMI monitor handy (only a projector), so I prefer to use the serial terminal. Important details to avoid frustration: The USB keyboard has to be connected to the lower USB port, or it will be ignored during the boot process. And make sure your serial terminal is configured not to use hardware flow control. Minicom is configured by default to use hardware flow control, so it was not sending any characters to the CuBox. ^A-O gets you to the Minicom configuration, select Serial port setup, and disable it.
Set up the SD card
I created a 2GB partition, but much less can suffice; I'd leave it at least to 1GB to do the base install, although it can be less once the system is set up (more on this later). Partition and format using your usual tools (fdisk+mke2fs, or gparted, or whatever suits your style).
Install the bootloader
I followed up the instructions on this CuBox-i forums thread to get the SPL and uBoot bootloader running. In short, from this Google Drive folder, download the SPL-U-Boot.img.xz file, uncompress it (xz --decompress SPL-U-Boot.img.xz), and write it to the SD card just after the partition map: As root,
# dd if=SPL-U-Boot.img of=/dev/mmcblk0 bs=1024 seek=1.
Actually, to be honest: As I wanted something basic to be able to debug from, I downloaded (from the same Google Drive) the busybox.img.gz file. That's a bit easier to install from: xz --decompress busybox.img.xz, and just dump it into the SD from the beginning (as it does already include a partition table):
# dd if=busybox.img of=/dev/mmcblk0
This card is already bootable and minimal, and allows to debug some bits from the CuBox-i itself (as we will see shortly).
After this step, I created a second partition, as I said earlier. So, my mmcblk0p1 partition holds Busybox, and the second will hold Debian. We are still working from the x86 system, so we mount the SD card in /media/mmcblk0p2
Installing the base system
Without debian-installer to do the heavy lifting, I went for debootstrap. As I ran it from my PC, debootstrap's role will be for this first stage only to download and do a very initial pre-unpacking of the files: Bootstrapping a foreign architecture implies, right, using the --foreign switch:
debootstrap --foreign --arch=armhf wheezy /media/mmcblk0p2 http://http.debian.net/debian
You can add some packages you often use by specifying --include=foo,bar,baz
So, take note notes: This board is capable of running the armhf architecture (HF for Hardware Float). It can also run armel, but I understand it is way slower.
First boot (with busybox)
So, once debootstrap finishes, you are good to go to the real hardware! Unmount the SD card, put it in the little guy, plug your favorite console in (I'm using the serial port), and plug the power in! You should immediately see something like:
  1. U-Boot SPL 2013.10-rc4-gd05c5c7-dirty (Jan 12 2014 - 02:18:28)
  2. Boot Device: SD1
  3. reading u-boot.img
  4. Load image from RAW...
  5. U-Boot 2013.10-rc4-gd05c5c7-dirty (Jan 12 2014 - 02:18:28)
  6. CPU: Freescale i.MX6Q rev1.2 at 792 MHz
  7. Reset cause: POR
  8. Board: MX6-CuBox-i
  9. DRAM: 2 GiB
  10. MMC: FSL_SDHC: 0
  11. In: serial
  12. Out: vga
  13. Err: vga
  14. Net: phydev = 0x0
  15. Phy not found
  16. PHY reset timed out
  17. FEC
  18. (Re)start USB...
  19. USB0: USB EHCI 1.00
  20. scanning bus 0 for devices... 1 USB Device(s) found
  21. scanning usb for storage devices... 0 Storage Device(s) found
  22. scanning usb for ethernet devices... 0 Ethernet Device(s) found
  23. Hit any key to stop autoboot: 3

Let it boot (that means, don't stop autoboot), and you will soon see a familiar #, showing you are root in the busybox environment. Great! Now, mount the Debian partition:
# mount /dev/mmcblk0p2 /mnt
Finishing debootstrap's task
With everything in place, it's time for debootstrap to work. Chroot into the Debian partition:
# chroot /mnt
And ask Debootstrap to finish what it started:
# debootstrap --second-stage
Be patient, as this step takes quite a bit to be finished.
Some extra touches...
After this is done, your Debian system is almost ready to be booted into. Why almost? Because it still does not have any users, does not know its own name nor knows I want to use it via a serial terminal. Three very simple tasks to fix. First two:
  1. # passwd
  2. Enter new UNIX password:
  3. Retype new UNIX password:
  4. passwd: password updated successfully
  5. # echo cubox-i.gwolf.org > /etc/hostname

For the second one, add a line to /etc/inittab specifying the details of the serial console. You can just do this:
# echo 'T0:23:respawn:/sbin/getty -L ttymxc0 115200 vt100' >> /etc/inittab
Boot into Debian!
So, ready to boot Debian? Ok, first exit the chroot shell, to go back to the Busybox shell, unmount the Debian partition, and set the root partition read-only:
  1. # exit
  2. # umount /mnt
  3. # mount / -o remount,ro

Disconnect and connect power, and now, do interrupt the boot process when you see the Hit any key to stop automount prompt. To see the configuration of uboot, you can type printenv We will only modify the parameters given to the kernel:
  1. CuBox-i U-Boot > setenv root /dev/mmcblk0p2 rootfstype=ext3 ro rootwait
  2. CuBox-i U-Boot > boot

So, the kernel will load, and a minimal Debian system will be initialized. In my case, I get the following output:
  1. ** File not found /boot/busyEnv.txt **
  2. 4703740 bytes read in 390 ms (11.5 MiB/s)
  3. ## Booting kernel from Legacy Image at 10000000 ...
  4. Image Name: Linux-3.0.35-8
  5. Image Type: ARM Linux Kernel Image (uncompressed)
  6. Data Size: 4703676 Bytes = 4.5 MiB
  7. Load Address: 10008000
  8. Entry Point: 10008000
  9. Verifying Checksum ... OK
  10. Loading Kernel Image ... OK
  11. Starting kernel ...
  12. Unable to get enet.0 clock
  13. pwm-backlight pwm-backlight.0: unable to request PWM for backlight
  14. pwm-backlight pwm-backlight.1: unable to request PWM for backlight
  15. _regulator_get: get() with no identifier
  16. mxc_sdc_fb mxc_sdc_fb.2: NO mxc display driver found!
  17. INIT: version 2.88 booting
  18. [info] Using makefile-style concurrent boot in runlevel S.
  19. [....] Starting the hotplug events dispatcher: udevd. ok
  20. [....] Synthesizing the initial hotplug events...done.
  21. [....] Waiting for /dev to be fully populated...done.
  22. [....] Activating swap...done.
  23. [....] Cleaning up temporary files... /tmp. ok
  24. [....] Activating lvm and md swap...done.
  25. [....] Checking file systems...fsck from util-linux 2.20.1
  26. done.
  27. [....] Mounting local filesystems...done.
  28. [....] Activating swapfile swap...done.
  29. [....] Cleaning up temporary files.... ok
  30. [....] Setting kernel variables ...done.
  31. [....] Configuring network interfaces...done.
  32. [....] Cleaning up temporary files.... ok
  33. [....] Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix. ok
  34. INIT: Entering runlevel: 2
  35. [info] Using makefile-style concurrent boot in runlevel 2.
  36. [....] Starting enhanced syslogd: rsyslogd. ok
  37. [....] Starting periodic command scheduler: cron. ok
  38. Debian GNU/Linux 7 cubox-i.gwolf.org ttymxc0
  39. cubox-i login:

And that's it, the system is live and ready for my commands!
So, how big is this minimal Debian installed system? I cheated a bit on this, as I had already added emacs and screen to the system, so yours will be a small bit smaller. But anyway Lets clear our cache of downloaded packages, and see the disk usage information:
  1. root@cubox-i:~# apt-get clean
  2. root@cubox-i:~# df -h
  3. Filesystem Size Used Avail Use% Mounted on
  4. rootfs 1008M 347M 611M 37% /
  5. /dev/root 1008M 347M 611M 37% /
  6. devtmpfs 881M 0 881M 0% /dev
  7. tmpfs 177M 132K 177M 1% /run
  8. tmpfs 5.0M 0 5.0M 0% /run/lock
  9. tmpfs 353M 0 353M 0% /run/shm

So, instead of a 4GB install, we have a 350MB one. Great improvement! Now, lets get it to do something useful, in a most Debianic way!

1 February 2014

Paul Tagliamonte: Iron-Blogger starts again

After years under Mako s loving care, iron-blogger is now under new leadership. The triumvirate consists of Me, Molly and John. If any interesting Boston-area people want to join, just email us!

30 January 2014

Rudy Godoy: What have we done?

Couple of weeks ago I was in the situation of having to setup up a new laptop. I decided to go with wheezy s DVD installer. So far so good. I didn t expect (somewhere in the lies-I-tell-myself dept.) to have GNOME as the default Debian desktop. However after install I ve figured it was the new GNOME3 everybody was talking about. Before that I ve seen it in Ubuntu systems used by my classmates. I thought yeah, OK GNOME3, I think that s fine for them as a Linux desktop. Turns out that I ve started using the Debian desktop aka GNOME3 and noticed that it was not as bloated as the Ubuntu desktop I ve seen before looked, so I sticked with it, (I thought for a while). Turns out that I did like this new so called GNOME3, the non-window based but an application-based system (that is something that sticks in my head). I liked the way it makes sense as a desktop system, like when you look for applications, documents, connect to networks, use pluggable devices or just configure stuff every time with less and less effort. Good practices and concepts learned from Mac OS X-like environments and for sure taking advantage of the new features the Linux kernel and user-space environment got over the years. So, like one month later I stick with it and it makes sense for me to keep it. I had no chance to try the latest XFCE or KDE, my default choices before this experience. Kudos GNOME team, even after the depictions you guys had on GNOME Shell; as I learned. This whole situation got me into some pondering about the past of the Linux user experience and how we in the community lead people into joining. I remember that when I guy asked: how do I configure this new monitor/VGA card/network card/Etc? the answer was in the lines of: what is the exact chipset model and specific whole product code number that your device has? Putting myself in the shoes of such people or today s people I d say: what are you talking about? what it is a chipset? I mean, like it was too technical that only one guy with more than average knowledge could grasp. From a product perspective this is similar for a car manufacturer tell to a customer to look for the exact layout or design your car s engine has, so that they are able to tell whether is the 82 s model A or 83 s model C. Simplicity on naming and identification was not in the mindset of most of us. This is funny because as technology advances it also becomes more transparent to the customer. So, for instance, today s kids can become really power users of any new technology as if they had, indeed, many pre-set chipsets in their brain. But when going into the details we had to grasp few years ago they have some hard time figuring out the complexity of the product that presents itself on this clean and simple interface. New times, interesting times. Always good to look back. No, I m not getting old.

Russell Coker: The Movie Experience

Phandroid has one of many articles about a man being detained for wearing Google Glass in a cinema [1]. The article states as a fact that it s probably not smart to bring a recording device into a movie theater which is totally bogus. I ve visited a government office where recording devices were prohibited, they provided a locker for me to store everything that could be used for electronic storage outside their main security zone, that s what you do when you ban recording devices. Any place that doesn t have such facilities really isn t banning recording. The Gadgeteer has the original story with more detail with an update showing that the Department of Homeland Security were responsible for detaining the victim [2]. There are lots of issues here with DHS continuing to do nothing good and more bad things than most people suspect and with the music and film industry organisations attacking innocent people. But one thing that seems to be ignored is that movies are a recreational activity, so it s an experience that they are selling not just a movie. Any organisation that wants to make money out of movies really should be trying to make movies fun. The movie experience has always involved queuing, paying a lot of money for tickets ($20 per seat seems common), buying expensive drinks/snacks, and having to waste time on anti-piracy adverts. Now they are adding the risk of assault, false-arrest, and harassment under color of law to the down-sides of watching a movie. Downloading a movie via Bittorrent takes between 20 minutes and a few hours (depending on size and internet connectivity). Sometimes it can be quicker to download a movie than to drive to a cinema and if you are organising a group to watch a movie it will definitely be easier to download it. When you watch a movie at home you can pause it for a toilet break and consume alcoholic drinks while watching (I miss the Dutch cinemas where an intermission and a bar were standard features). It s just a better experience to download a movie via Bittorrent. I ve previously written about the way that downloading movies is better than buying a DVD [3], now they are making the cinema a worse experience too. I sometimes wonder if groups like the MPAA are actually trying to make money from movies or whether they just want to oppress their audiences for fun or psychological research. I could imagine someone like the young Phillip Zimbardo working for the MPAA and doing experiments to determine how badly movie industry employees can treat their customers before the customers revolt. Anyone who watches a Jack Ryan movie (or any movie with a Marty-Stu/Gary-Stu character) obviously doesn t even want to experience the stress of an unhappy ending to a movie. It seems obvious that such people won t want the stress of potentially being assaulted in the cinema. In terms of economics it seems a bad idea to do anything about recording in the cinema. When I was 11 I was offered the opportunity to watch a movie that had been recorded by a video camera in the US before it was released in Australia, I wasn t interested because watching a low quality recording wouldn t be fun. It seems to me that if The Pirate Bay (the main site for Bittorrent downloads of movies) [4] was filled with awful camera recordings of movies then it would discourage people from using it. A quick search shows some camera recordings on The Pirate Bay, it seems that if you want to download a movie of reasonable quality then you have to read the Wikipedia page about Pirated Movie Release Types [5] to make sure that you get a good quality download. But if you buy a DVD in a store or visit a cinema then you are assured of image and sound quality. If the movie industry were smarter they would start uploading camera recordings of movies described as Blue-Ray rips to mess with Bittorrent users and put newbies off downloading movies.

29 January 2014

Benjamin Mako Hill: Aaron Swartz A Year Later

My friend Aaron Swartz died a little more than a year ago. This time last year, I was spending much of my time speaking with journalists and reading what they were writing about Aaron. Since the anniversary of his death, I have tried to take time to remember Aaron. I ve returned to the things I wrote and the things I said including this short article published last year in Red Pepper that SJ Klein and I wrote together but that I forgot to mention on my blog. I m also excited to see that a documentary film about Aaron premiered at the Sundance Film Festival last week. I was interviewed for the film but am not in it. As I said last year at a memorial for Aaron, I think about Aaron frequently and often think about my own decisions in terms of what Aaron would have done. I continued to be optimistic about the potential for Aaron-inspired action.

Emanuele Rocca: Antifeatures and Antibugs

Software Engineering distinguishes between software features and software bugs. It is usually understood that features are positive, expected characteristics of a computer program. Features make users happy by allowing them to do something useful, interesting, or fun. Something good, anyways. Bugs are instead undesirable and annoying. You're sitting there at your computer writing a long email and the software crashes right before your email is sent. Bad stuff. Features are generally implemented by programmers on purpose, whereas bugs are purely unintentional. They are mistakes. You don't make a mistake on purpose. We might at this point be inclined to think that: i) what is good for users is done on purpose by software manufacturers; ii) what is bad for users was not meant to be. It happened by mistake. Here is a handy table to visualize this idea:
On purpose By mistake
Good Feature
Bad Bug
It seems to make a lot of sense. But you might have noticed that two cells of the table are empty. Right! In a great talk titled When Free Software isn't better, Benjamin Mako Hill mentions the concept of antifeatures, and how they relate to Free Software. Antifeatures are features that make the software do something users will hate. Something they will hate so much they would pay to have those features removed, if that's an option. Microsoft Windows 7 is used in the talk to provide some examples of software antifeatures: the Starter Edition does not allow users to change their background image. Also, it limits the amount of usable memory on the computer to 2GBs, regardless of how much memory the system actually has. Two antifeatures engineered to afflict users to the point that they will purchase a more expensive version of the software, if they have the means to do that. I have another nice example. The Spotify music streaming service plays advertisements between songs every now and then. To make sure users are annoyed as much as possible, Spotify automatically pauses an advertisement if it detects that the volume is being lowered. A poor Spotify user even tried to report the bug on The Spotify Community forum, only to find out that what she naively considered as a software error was "intentional behavior". A spectacular antifeature indeed. Whenever a piece of technology does something you most definitely do not want it to do, such as allowing the NSA to take complete control of your Apple iPhone, including turning on its microphone and camera against your will, that's an antifeature.
On purpose By mistake
Good Feature
Bad Antifeature Bug
Both bugs and antifeatures are bad for users. The difference between them is that antifeatures are engineered. Time and money are spent to make sure the goal is reached. A testing methodology is followed. "Are we really sure customers cannot change their wallpaper even if they try very very hard?" Engineering processes, of course, can fail. If the poor devils at Microsoft who implemented those harassments would have made a mistake that allows users to somehow change their wallpaper on Windows Starter... Well, I would call that a glorious antibug.
On purpose By mistake
Good Feature Antibug
Bad Antifeature Bug
There is no place for antifeatures in Free and Open Source Software. Free Software gives users control over what their software does. Imagine Mozilla adding a feature to Firefox that sets your speakers volume to 11 and starts playing a random song from the black metal artist Burzum every time you add a bookmark, unless you pay for Mozilla Firefox Premium Edition. The source code for Firefox is available under a free license. People who are not into Burzum's music would immediately remove this neat antifeature. I have spent many years of my life advocating Free and Open Source Software, perhaps using the wrong arguments. Mako's talk made me think about all this (thanks mate!). All these years I've been preaching about the technical superiority of Free Software, despite evidence of thousands of bugs and usability issues in the very programs I am using, and contributing to develop. Free Software is not better than Proprietary Software per se. Sometimes it is, sometimes it's not. But it gives you control, and freedom. When it annoys you, when it doesn't do what you expect and want, you can be sure it's not on purpose. And we can fix it together.

Petter Reinholdtsen: A fist full of non-anonymous Bitcoins

Bitcoin is a incredible use of peer to peer communication and encryption, allowing direct and immediate money transfer without any central control. It is sometimes claimed to be ideal for illegal activity, which I believe is quite a long way from the truth. At least I would not conduct illegal money transfers using a system where the details of every transaction are kept forever. This point is investigated in USENIX ;login: from December 2013, in the article "A Fistful of Bitcoins - Characterizing Payments Among Men with No Names" by Sarah Meiklejohn, Marjori Pomarole,Grant Jordan, Kirill Levchenko, Damon McCoy, Geoffrey M. Voelker, and Stefan Savage. They analyse the transaction log in the Bitcoin system, using it to find addresses belong to individuals and organisations and follow the flow of money from both Bitcoin theft and trades on Silk Road to where the money end up. This is how they wrap up their article:
"To demonstrate the usefulness of this type of analysis, we turned our attention to criminal activity. In the Bitcoin economy, criminal activity can appear in a number of forms, such as dealing drugs on Silk Road or simply stealing someone else s bitcoins. We followed the flow of bitcoins out of Silk Road (in particular, from one notorious address) and from a number of highly publicized thefts to see whether we could track the bitcoins to known services. Although some of the thieves attempted to use sophisticated mixing techniques (or possibly mix services) to obscure the flow of bitcoins, for the most part tracking the bitcoins was quite straightforward, and we ultimately saw large quantities of bitcoins flow to a variety of exchanges directly from the point of theft (or the withdrawal from Silk Road). As acknowledged above, following stolen bitcoins to the point at which they are deposited into an exchange does not in itself identify the thief; however, it does enable further de-anonymization in the case in which certain agencies can determine (through, for example, subpoena power) the real-world owner of the account into which the stolen bitcoins were deposited. Because such exchanges seem to serve as chokepoints into and out of the Bitcoin economy (i.e., there are few alternative ways to cash out), we conclude that using Bitcoin for money laundering or other illicit purposes does not (at least at present) seem to be particularly attractive."
These researches are not the first to analyse the Bitcoin transaction log. The 2011 paper "An Analysis of Anonymity in the Bitcoin System" by Fergal Reid and Martin Harrigan is summarized like this:
"Anonymity in Bitcoin, a peer-to-peer electronic currency system, is a complicated issue. Within the system, users are identified by public-keys only. An attacker wishing to de-anonymize its users will attempt to construct the one-to-many mapping between users and public-keys and associate information external to the system with the users. Bitcoin tries to prevent this attack by storing the mapping of a user to his or her public-keys on that user's node only and by allowing each user to generate as many public-keys as required. In this chapter we consider the topological structure of two networks derived from Bitcoin's public transaction history. We show that the two networks have a non-trivial topological structure, provide complementary views of the Bitcoin system and have implications for anonymity. We combine these structures with external information and techniques such as context discovery and flow analysis to investigate an alleged theft of Bitcoins, which, at the time of the theft, had a market value of approximately half a million U.S. dollars."
I hope these references can help kill the urban myth that Bitcoin is anonymous. It isn't really a good fit for illegal activites. Use cash if you need to stay anonymous, at least until regular DNA sampling of notes and coins become the norm. :) As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

28 January 2014

Andrea Colangelo: The (brand new) Debian 3D-Printing Team is hiring!

The 3D-printing technology has gained quite a big momentum in recent times. Its use is spreading widely even among hobbyists and many sources report daily several (sometimes weird) examples of its usage. The thing I like most, as a Free Software activist, is that several 3D-printing-related free and open source software are already available and quite good, and some printers are even released as free hardware. Unfortunately, Debian is quite lagged behind with regard to the availability of such software in our archive, especially compared to other distributions who already ship much of it. Hopefully this situation will change drastically thanks to the brand new Debian 3D-Printing team, whose purpose is to make Debian a rocking platform to do some serious 3D-printing. The team is bootstrapping right now, and we are looking for volunteers who are willing to help us. Packagers are greatly welcomed, but also triagers, users and passionate people with some experience in the field are extremely valuable. People willing to join the team can take a look at our (in-progress) wiki page and apply on the team page on Alioth. As with any good packaging team, a mailing list is available to share ideas, feel free to subscribe it, and feel free to join #debian-3dprinting on OFTC if you want to hang out with other team members.

Dirk Eddelbuettel: RQuantLib 0.3.11

A new minor / maintenance release RQuantLib 0.3.11 is now on CRAN and in Debian. Like the RcppClassic upload two days ago and the RcppZiggurat and RcppEigen uploads yesterday, this release was motivated at least in part by an upcoming Rcpp release about which I should have more to day soon. A few things got polished and updated, and we added a plugin which should make use via Rcpp Attribute easier. RQuantLib combines (some of) the quantitative analytics of QuantLib with the R statistical computing environment and language. Thanks to CRANberries, there is also a diff to the previous release 0.3.10. Full changelog details, examples and more details about this package are at my RQuantLib page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

27 January 2014

Kees Cook: -fstack-protector-strong

There will be a new option in gcc 4.9 named -fstack-protector-strong , which offers an improved version of -fstack-protector without going all the way to -fstack-protector-all . The stack protector feature itself adds a known canary to the stack during function preamble, and checks it when the function returns. If it changed, there was a stack overflow, and the program aborts. This is fine, but figuring out when to include it is the reason behind the various options. Since traditionally stack overflows happen with string-based manipulations, the default (-fstack-protector), only includes the canary code when a function defines an 8 (--param=ssp-buffer-size=N, N=8 by default) or more byte local character array. This means just a few functions get the checking, but they re probably the most likely to need it, so it s an okay balance. Various distributions ended up lowering their default --param=ssp-buffer-size option down to 4, since there were still cases of functions that should have been protected but the conservative gcc upstream default of 8 wasn t covering them. However, even with the increased function coverage, there are rare cases when a stack overflow happens on other kinds of stack variables. To handle this more paranoid concern, -fstack-protector-all was defined to add the canary to all functions. This results in substantial use of stack space for saving the canary on deep stack users, and measurable (though surprisingly still relatively low) performance hit due to all the saving/checking. For a long time, Chrome OS used this, since we re paranoid. :) In the interest of gaining back some of the lost performance and not hitting our Chrome OS build images with such a giant stack-protector hammer, Han Shen from the Chrome OS compiler team created the new option -fstack-protector-strong, which enables the canary in many more conditions: This meant we were covering all the more paranoid conditions that might lead to a stack overflow. Chrome OS has been using this option instead of -fstack-protector-all for about 10 months now. As a quick demonstration of the options, you can see this example program under various conditions. It tries to show off an example of shoving serialized data into a non-character variable, like might happen in some network address manipulations or streaming data parsing. Since I m using memcpy here for clarity, the builds will need to turn off FORTIFY_SOURCE, which would also notice the overflow.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
struct no_chars  
    unsigned int len;
    unsigned int data;
 ;
int main(int argc, char * argv[])
 
    struct no_chars info =    ;
    if (argc < 3)  
        fprintf(stderr, "Usage: %s LENGTH DATA...\n", argv[0]);
        return 1;
     
    info.len = atoi(argv[1]);
    memcpy(&info.data, argv[2], info.len);
    return 0;
 
Built with everything disabled, this faults trying to return to an invalid VMA: Built with FORTIFY_SOURCE enabled, we see the expected catch of the overflow in memcpy: So, we ll leave FORTIFY_SOURCE disabled for our comparisons. With pre-4.9 gcc, we can see that -fstack-protector does not get triggered to protect this function: However, using -fstack-protector-all does trigger the protection, as expected: And finally, using the gcc snapshot of 4.9, here is -fstack-protector-strong doing its job: For Linux 3.14, I ve added support for -fstack-protector-strong via the new CONFIG_CC_STACKPROTECTOR_STRONG option. The old CONFIG_CC_STACKPROTECTOR will be available as CONFIG_CC_STACKPROTECTOR_REGULAR. When comparing the results on builds via size and objdump -d analysis, here s what I found with gcc 4.9: A normal x86_64 defconfig build, without stack protector had a kernel text size of 11430641 bytes with 36110 function bodies. Adding CONFIG_CC_STACKPROTECTOR_REGULAR increased the kernel text size to 11468490 (a +0.33% change), with 1015 of 36110 functions stack-protected (2.81%). Using CONFIG_CC_STACKPROTECTOR_STRONG increased the kernel text size to 11692790 (+2.24%), with 7401 of 36110 functions stack-protected (20.5%). And 20% is a far-cry from 100% if support for -fstack-protector-all was added back to the kernel. The next bit of work will be figuring out the best way to detect the version of gcc in use when doing Debian package builds, and using -fstack-protector-strong instead of -fstack-protector. For Ubuntu, it s much simpler because it ll just be the compiler default.

2014, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.
Creative Commons License

Next.

Previous.