Search Results: "will"

28 February 2015

Zlatan Todori : Interviews with FLOSS developers: Joey Hess

Edit: Now translated to Chinese. Thanks zhang wei! There is really hardly a better way to open a series of interviewing with developers behind Free Libre Open Source Software project, then with incredible mind such as Joey Hess. To write his contributions to Free software ecosystem, especially in Debian, would be a book by itself. His impact exceeds even his projects - people literally follow his blog posts to see what he is doing and how is he living. A hacker from cabin. If you really need to have a picture of true hacker, then Joey is the one. As this isn't a book I will just mention few projects that he has been behind - git-annex, ikiwiki, etckeeper, debian installer, parts of dpkg, debhelper, devscripts, taskel. So without further waiting here it is. Picture of Joey Hess me: Who are you? joeyh: I'm Joey -- https://joeyh.name/ me: How did you start programming? joeyh: Atari 130XE which came with BASIC and a boring word processor and not much else. No other friends had one, so the only way to get software was to type in demo programs from manual and then begin to change and write my own. So, the easy way to learn. Also some Logo in school. me: How would you now advise others to start programming? joeyh: Difficult question, it seems much harder to get an intimate understanding of things than when I started, and much harder to be motivated to program when there's so much stuff easily available. Maybe simple bare-metal systems like Arduino coupled with real-world interaction are the answer. I've recently been mentoring my nephew who is learning python and Python the Hard Way has gotten him far impressively fast. me: Setup of your development machine? joeyh: Lenovo laptop de-spywared with Debian unstable, xmonad, xfce, vim. me: Your thoughts on Purism (the open hardware laptop initiative that got recently funded on CrowdSupply)? joeyh: I don't know much about that one, but it seems that consumer level hardware has gotten so low quality, and so closed and untrustworthy that it makes sense to either build alternatives that are open, or pick out, as a community, the stuff we can adapt to our needs and concentrate on it. Several projects are trying, I hope they succeed. me: How do you see future of Debian development? joeyh: Well, I've mostly stopped worrying about it. If you look back at my presentations at the past 2 or 3 DebConfs, you'll find my best thoughts on the matter. me: You retired as Debian developer - do you intend sometime soon to come back and/or do you plan to join some other communities? joeyh: It would be glorious to come back, wouldn't it? But I don't think I will. Can't step in the same river twice, and all. Instead, Debian will probably have to put up with me as an annoying upstream author who doesn't ship tarballs, but does ship debian/ directories, and as a bug reporter who enjoys reporting amusing bugs like -0 NaN. I seem to have more time to spend in other online communities since I left Debian, but in a more diffuse way. Maybe that's just what it's like, to be involved in Free Software but not in the embrace of a big project like Debian. me: Some memorable moments from Debian conferences? joeyh: There are so many! Picnicing on berries and tamales at the Portland farmer's market right outside the venue; rainbows and bonfire in Switzerland after crazy busy days; impromptu pipe organ repair in a weird night venue in Edinburgh; walking through Porto Alegre at night with Ian Murdock and how humble he was about what he'd started; hacking all night in Spain; failing to sleep through midnight sun and incessent partying Finland; hanging out in the hotel lobby in Atlanta where we designed Build-Depends. me: Are you a gamer? Valve Steam games are offered for free to Debian Developers - do you use steam and play Valve games? joeyh: I've played through Half Life and Portal, but nethack has claimed more of my time. I mostly enjoy short, indie games, or games that tell us something new about the medium of games, A recent favorite was A Dark Room. But really, I have more pure fun playing real world Tabletop games with friends, like Carcassanne Discovery and Hive. In March, I am going to try to write a roguelike game in one week, in Haskell, for the Seven Day Roguelike Challenge and I'll be blogging about my progress daily. me: You are nowdays a Haskell hacker (git-annex) - what would you like to say about this language and how does it compare to Python, C, JavaScript, Ruby and Perl? joeyh: Not just git-annex; all my current projects are written in Haskell. I think it's amazing how much we expect programmers to keep in their heads while writing code. Is that buffer going to overflow? Is changing the value of that global variable going to break some other part of the code? Is that input sanitized yet? Did that interface change? Haskell solves some of these outright, but more, it makes you start noticing this kind of pervasive issue, and it provides ways to completely eliminate a class of problems from your code. For example http://joeyh.name/blog/entry/making_propellor_safer_with_GADTs_and_type_families/. The class of bugs I avoided there had never affected my code even once, but it was still worth preventing that whole class of bugs, so I don't have to worry about them ever again. me: Would you suggest Haskell as first language to learn especially for those that have an itch for mathematics? joeyh: I think that can work well. Or it can go other the way -- I had an affinity to mathematics when I was young, but it got knocked out of me in the way that happens to many people, and languages like perl and C don't do much to make you want to learn more about higher-order math. I've been picking up a bit more here and there via Haskell. me: How do you compare your productivity in Haskell compared to your Perl days? joeyh: It's very different; I'm a very different programmer now. I probably would bang out quick hacks more quickly when I was writing Perl. But, they tended to stay quick hacks. Now, I might take a little longer to get there, but the code seems a lot more solid, while also being more malleable to turn into larger or different programs. I'm also a lot more drawn toward writing software libraries. me: Can you describe your philosophy of life (you live in cabin, in forest, using a lot of solar power - many people are intrigued (including myself) what drives you towards that kind of life and how does it impact your overall quality of life and happiness. Looking the todays modern predator capitalistic society, in which you could easily earn more then $10.000 a month, you seem to be an anarchist and very humble human)? joeyh: I want to build worthwhile things that might last. Which is super hard in the world of software, both because it's hard to think far ahead at all, and because most jobs don't emphasize that kind of real value. I've been lucky and bootstrapped up to a point where I've been able to work full time on free software for years, and I'm willing to forgo a lot to continue that. Living in the woods without modern conveniences is great, because it's quiet and you can think as much as you like; the internet is just as close as it is anywhere else (maybe a bit slower); and when you've spent too much time quietly thinking you'll need to go chop wood, or haul water, or jump in the river to cool off, depending on the season. (Humble? Like most programmers, I am internally a flaming tower of ego...)
Vote on Hacker News

Mathieu Parent: Hello Planet Debian

After more than five years of being a Debian developer, here is my first post on Planet Debian! I currently maintain 165 packages. My focus has changed since 2009, but those are still mostly sysadmin packages: Now that the first post is online, I will try to keep up!

Gunnar Wolf: Welcome to the world, little ones!

Welcome to the world, little ones!
Welcome little babies! Yesterday night, we entered the hospital. Nervous, heavy, and... Well, would we ever be ready? As ready as we could. A couple of hours later, Alan and Elena Wolf Daichman became individuals on their own right. As is often the case in the case of twins, they were brought to this world after a relatively short preparation (34 weeks, that's about 7.5 months). At 1.820 and 1.980Kg, they are considerably smaller than either of the parents... But we will be working on that! Regina is recovering from the operation, the babies are under observation. As far as we were told, they seem to be quite healthy, with just minor issues to work on during neonatal care. We are waiting for our doctors to come today and allow us to spend time with them. And as for us... It's a shocking change to finally see the so long expected babies. We are very very very happy... And the new reality is hard to grasp, to even begin understanding :) PS- Many people have told me that my blog often errors out under load. I expect it to happen today :) So, if you cannot do it here, there are many other ways to contact us. Use them! :)

27 February 2015

Dirk Eddelbuettel: RcppArmadillo 0.4.650.1.1 (and also 0.4.650.2.0)

A new Armadillo release 4.650.1 was released by Conrad a few days ago. Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. It turned out that this release had one shortcoming with respect to the C++11 RNG initializations in the R use case (where we need to protect the users from the C++98 RNG deemed unsuitable by the CRAN gatekeepers). And this lead to upstream release 4.650.1 which we wrapped into RcppArmadillo 0.4.650.1.1. As before this, was tested against all 107 reverse dependencies of RcppArmadillo on the CRAN repo. This version is now on CRAN, and was just uploaded to Debian. Its changes are summarized below based on the NEWS.Rd file.
Changes in RcppArmadillo version 0.4.650.1.1 (2015-02-25)
  • Upgraded to Armadillo release Version 4.650.1 ("Intravenous Caffeine Injector")
    • added randg() for generating random values from gamma distributions (C++11 only)
    • added .head_rows() and .tail_rows() to submatrix views
    • added .head_cols() and .tail_cols() to submatrix views
    • expanded eigs_sym() to optionally calculate eigenvalues with smallest/largest algebraic values fixes for handling of sparse matrices
  • Applied small correction to main header file to set up C++11 RNG whether or not the alternate RNG (based on R, our default) is used
Now, it turns out that another small fix was needed for the corner case of a submatrix within a submatrix, ie V.subvec(1,10).tail(5). I decided not to re-release this to CRAN given the CRAN Repository Policy preference for releases no more than every 1 2 months . But fear not, for we now have drat. I created a drat package repository in the RcppCore account (to not put a larger package into my main drat repository often used via a fork to initialize a drat). So now with these two simple commands
## if needed, first install 'drat' via:   install.packages("drat")
drat:::add("RcppCore")
update.packages()
you will get the newest RcppArmadillo via this drat package repository. And course install.packages("RcppArmadillo") would also work, but takes longer to type :) Lastly, courtesy of CRANberries, there is also a diffstat report for the most recent CRAN release. As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

26 February 2015

EvolvisForge blog: tomcat7 init script is asynchronous

TIL: the init script of tomcat7 in Debian is asynchronous. For some piece of software, our rollout (install and upgrade) process works like this: The first tomcat7 start here is just to unzip the *.war files. For some reason, people like to let tomcat7 do that. This failed today; there were two webapps. Manually unzipping it also did not work for some reason. Re-doing it, inserting a sleep 30 after the here , made it work. In a perfect world, initscripts only return when the service is running, so that the next one started in a nice sequential (not parallel!) init or manual start sequence can do what it needs to, assuming the previous command has fully finished. In this perfect world, those who do wish for faster startup times use a different init system, one that starts things in parallel, for example. Even there, dependencies will wish for the depended-on service to be fully running when they are started; even more so, since the delays between starting things seem to be less for that other init system. So, this is not about the init system, but about the init script; a change that would be a win-win for users of both init schemes. Update: Someone already contacted me with feedback: they suggested to wait until the shutdown port is listened on by tomcat7. We ll look at this later. In the meantime, we re trying to also get rid of the config (and logs) in webapps/ part
PS: If someone is interested in an init script (Debian/LSB sysvinit, I made the effort to finally learn that some months before the other system came) that starts Wildfly (formerly known as JBoss AS) synchronously, waiting until all *.?ar files are fully deployed before returning (though with a timeout in case it won t ever finish), just ask (maybe it will become a dialogue, in which we can improve it together). (We have two versions of it, the more actively maintained one is in a secret internal project though, so I d have to merge it and ready it for publication though, plus the older one is AGPLv3, the newer one was relicenced to a BSDish licence.)

25 February 2015

Holger Levsen: 20150225-apparmor-user-stories

Developing is a use case too For whatever reason Ulrike's blog post about AppArmor user stories and user tags was not syndicated to planet.d.o, despite it should have been and despite planet admins nicely having looked into it. Whatever... As you might have guessed by now, the user stories referred to in this blog post are about developers supporting AppArmor (a kernel module for restricting capabilities of processes) in their Debian packages. So if you're maintaining packages and have always been pondering to look into this apparmor thingy, go read that blog post! Hopefully the next post will "magically" appear on planet again ;-)

Andrew Cater: Cubietruck now running Debian :)

Following a debootstrap build of sid on one machine to complete the cross-compilation of mainline u-boot, I managed to get vanilla Debian installed on my Cubietruck

A USB-serial cable is a must for the install and for any subsequent major reconfiguration as the stock Debian installer does not have drivers for the video / audio. Various Cubietruck derivative distributions do - but the Sunxi kernel appears flaky

All was fine for a few days, then I decided to try and configure the Wifi by hand configuring /etc/network/interfaces and wpasupplicant files. I managed to break the network connectivity by doing things in a hurry and typing blind. I'd put it into the appropriate closed metal case so was rather stuck.

A friend carefully took the case apart by easing off the metal cover plates, removed two screws holding the whole thing together and precision drilled the metal cover plates on one side so that four screws can be undone and the entire inner part of the case can slide out as one while the other metal clover plate remains captive. He will follow this procedure with his two later.

Very pleased with the way it's turned out. The WiFi driver has non-free firmware but I now have a tiny, silent machine, drawing about 3W tops and both interfaces are now working.


23 February 2015

Simon Josefsson: Laptop Buying Advice?

My current Lenovo X201 laptop has been with me for over four years. I ve been looking at new laptop models over the years thinking that I should upgrade. Every time, after checking performance numbers, I ve always reached the conclusion that it is not worth it. The most performant Intel Broadwell processor is the the Core i7 5600U and it is only about 1.5 times the performance of my current Intel Core i7 620M. Meanwhile disk performance has increased more rapidly, but changing the disk on a laptop is usually simple. Two years ago I upgraded to the Samsung 840 Pro 256GB disk, and this year I swapped that for the Samsung 850 Pro 1TB, and both have been good investments. Recently my laptop usage patterns have changed slightly, and instead of carrying one laptop around, I have decided to aim for multiple semi-permanent laptops at different locations, coupled with a mobile device that right now is just my phone. The X201 will remain one of my normal work machines. What remains is to decide on a new laptop, and there begins the fun. My requirements are relatively easy to summarize. The laptop will run a GNU/Linux distribution like Debian, so it has to work well with it. I ve decided that my preferred CPU is the Intel Core i7 5600U. The screen size, keyboard and mouse is mostly irrelevant as I never work longer periods of time directly on the laptop. Even though the laptop will be semi-permanent, I know there will be times when I take it with me. Thus it has to be as lightweight as possible. If there would be significant advantages in going with a heavier laptop, I might reconsider this, but as far as I can see the only advantage with a heavier machine is bigger/better screen, keyboard (all of which I find irrelevant) and maximum memory capacity (which I would find useful, but not enough of an argument for me). The sub-1.5kg laptops with the 5600U CPU on the market that I have found are:
Lenovo X250 1.42kg 12.5 1366 768
Lenovo X1 Carbon (3rd gen) 1.44kg 14 2560 1440
Dell Latitude E7250 1.25kg 12.5 1366 768
Dell XPS 13 1.26kg 13.3 3200 1800
HP EliteBook Folio 1040 G2 1.49kg 14 1920 1080
HP EliteBook Revolve 810 G3 1.4kg 11.6 1366 768
I find it interesting that Lenovo, Dell and HP each have two models that meets my 5600U/sub-1.5kg criteria. Regarding screen, possibly there exists models with other screen resolutions. The XPS 13, HP 810 and X1 models I looked had touch screens, the others did not. As screen is not important to me, I didn t evaluate this further. I think all of them would suffice, and there are only subtle differences. All except the XPS 13 can be connected to peripherals using one cable, which I find convenient to avoid a cable mess. All of them have DisplayPort, but HP uses DisplayPort Standard and the rest uses miniDP. The E7250 and X1 have HDMI output. The X250 boosts a 15-pin VGA connector, none of the others have it I m not sure if that is a advantage or disadvantage these days. All of them have 2 USB v3.0 ports except the E7250 which has 3 ports. The HP 1040, XPS 13 and X1 Carbon do not have RJ45 Ethernet connectors, which is a significant disadvantage to me. Ironically, only the smallest one of these, the HP 810, can be memory upgraded to 12GB with the others being stuck at 8GB. HP and the E7250 supports NFC, although Debian support is not certain. The E7250 and X250 have a smartcard reader, and again, Debian support is not certain. The X1, X250 and 810 have a 3G/4G card. Right now, I m leaning towards rejecting the XPS 13, X1 and HP 1040 because of lack of RJ45 ethernet port. That leaves me with the E7250, X250 and the 810. Of these, the E7250 seems like the winner: lightest, 1 extra USB port, HDMI, NFC, SmartCard-reader. However, it has no 3G/4G-card and no memory upgrade options. Looking for compatibility problems, it seems you have to be careful to not end up with the Dell Wireless card and the E7250 appears to come in a docking and non-docking variant but I m not sure what that means. Are there other models I should consider? Other thoughts?

Dirk Eddelbuettel: drat Tutorial: Publishing a package

Introduction The drat package was released earlier this month, and described in a first blog post. I received some helpful feedback about what works and what doesn't. For example, Jenny Bryan pointed out that I was not making a clear enough distinction between the role of using drat to publish code, and using drat to receive/install code. Very fair point, and somewhat tricky as R aims to blur the line between being a user and developer of statistical analyses, and hence packages. Many of us are both. Both the main point is well taken, and this note aims to clarify this issue a little by focusing on the former. Another point make by Jenny concerns the double use of repository. And indeed, I conflated repository (in the sense of a GitHub code repository) with repository for a package store used by a package manager. The former, a GitHub repository, is something we use to implement a personal drat with: A GitHub repository happens to be uniquely identifiable just by its account name, and given an (optional) gh-pages branch also offers a stable and performant webserver we use to deliver packages for R. A (personal) code repository on the other hand is something we implement somewhere---possibly via drat which supports local directories, possibly on a network share, as well as anywhere web-accessible, e.g. via a GitHub repository. It is a little confusing, but I will aim to make the distinction clearer.

Just once: Setting up a drat repository So let us for the remainder of this post assume the role of a code publisher. Assume you have a package you would like to make available, which may not be on CRAN and for which you would like to make installation by others easier via drat. The example below will use an interim version of drat which I pushed out yesterday (after fixing a bug noticed when pushing the very new RcppAPT package). For the following, all we assume (apart from having a package to publish) is that you have a drat directory setup within your git / GitHub repository. This is not an onerous restriction. First off, you don't have to use git or GitHub to publish via drat: local file stores and other web servers work just as well (and are documented). GitHub simply makes it easiest. Second, bootstrapping one is trivial: just fork my drat GitHub repository and then create a local clone of the fork. There is one additional requirement: you need a gh-pages branch. Using the fork-and-clone approach ensures this. Otherwise, if you know your way around git you already know how to create a gh-pages branch. Enough of the prerequisities. And on towards real fun. Let's ensure we are in the gh-pages branch:
edd@max:~/git/drat(master)$ git checkout gh-pages
Switched to branch 'gh-pages'
Your branch is up-to-date with 'origin/gh-pages'.
edd@max:~/git/drat(gh-pages)$ 

Publish: Run one drat command to insert a package Now, let us assume you have a package to publish. In my case this was version 0.0.1.2 of drat itself as it contains a fix for the very command I am showing here. So if you want to run this, ensure you have this version of drat as the CRAN version is currently behind at release 0.0.1 (though I plan to correct that in the next few days). To publish an R package into a code repository created via drat running on a drat GitHub repository, just run insertPackage(packagefile) which we show here with the optional commit=TRUE. The path to the package can be absolute are relative; the easists is often to go up one directory from the sources to where R CMD build ... has created the package file.
edd@max:~/git$ Rscript -e 'library(drat); insertPackage("drat_0.0.1.2.tar.gz", commit=TRUE)'
[gh-pages 0d2093a] adding drat_0.0.1.2.tar.gz to drat
 3 files changed, 2 insertions(+), 2 deletions(-)
 create mode 100644 src/contrib/drat_0.0.1.2.tar.gz
Counting objects: 7, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (7/7), 7.37 KiB   0 bytes/s, done.
Total 7 (delta 1), reused 0 (delta 0)
To git@github.com:eddelbuettel/drat.git
   206d2fa..0d2093a  gh-pages -> gh-pages
edd@max:~/git$ 
You can equally well run this as insertPackage("drat_0.0.1.2.tar.gz"), then inspect the repo and only then run the git commands add, commit and push. Also note that future versions of drat will most likely support git operations directly by relying on the very promising git2r package. But this just affect package internals, the user-facing call of e.g. insertPackage("drat_0.0.1.2.tar.gz", commit=TRUE) will remain unchanged. And in a nutshell that really is all there is to it. With the newly drat-ed package pushed to your GitHub repository with a single function call), it is available via the automatically-provided gh-pages webserver access to anyone in the world. All they need to do is to point R's package management code (which is built into R itself and used for e.g._ CRAN and BioConductor R package repositories) to the new repo---and that is also just a single drat command. We showed this in the first blog post and may expand on it again in a follow-up. So in summary, that really is all there is to it. After a one-time setup / ensuring you are on the gh-pages branch, all it takes is a single function call from the drat package to publish your package to your drat GitHub repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

21 February 2015

Vasudev Kamath: Running Plan9 using 9vx - using vx32 sandboxing library

Now a days I'm more and more attracted towards Plan9, an Operating System meant to be the successor of UNIX and created by same people who created original UNIX. I'm always baffled by the simplicity of Plan9. Sadly Plan9 never took off for whatever reasons. I've been for a while trying to run Plan9, I ran Plan9 on Raspberry Pi model B using 9pi, but I couldn't experiment with it more due to some restrictions in my home setup. I installed original Plan9 4th Edition from Bell labs (now part of Alcatel-Lucent), I will write about it in on different post. But running virtual machine on my system is again PITA as system is already old (3 and half year). I came across the 9vx which is port of Plan9 for FreeBSD, Linux and Mac OSX by Russ Cox. I downloaded original 9vx version 0.9.12 from Russ's page linked above. The archive contains a Plan9 rootfs along with precompiled 9vx binaries for Linux, FreeBSD and Mac OS X. I ran the Linux binary but it crashed.
./9vx.Linux -u glenda
I was seeing some illegal instruction error in dmesg. I didn't bother to do more investigation. A bit of googling showed me Arch Linux's wiki page on 9vx. I got errors trying to compile the original vx32 from rsc's repository but later saw that AUR 9vx package is built from different repository forked from rsc's found here. I cloned the repository to local and compiled it, I don't really remember if I had installed any additional packages. But if you get error you will know what additional thing is required. After compilation the 9vx binary is found inside src/9vx/9vx. I used this newly compiled 9vx to run the the rootfs I downloaded from Russ's website.
9vx -u glenda -r /path/to/extracted/9vx-0.9.12/
This launches Plan9 and allows you to work inside Plan9. The good part is its not resource hungry and still looks like you have a VM running with Plan9 on it. But there seems to be a better way to do this directly from plan9 iso from bell labs. It can be found on 9fans list. Now I'm going to try that out too :-). And in next post I will share my experience of using Plan9 on Qemu.

20 February 2015

David Bremner: Dear Lenovo, it's not me, it's you.

I've been a mostly happy Thinkpad owner for almost 15 years. My first Thinkpad was a 570, followed by an X40, an X61s, and an X220. There might have been one more in there, my archives only go back a decade. Although it's lately gotten harder to buy Thinkpads at UNB as Dell gets better contracts with our purchasing people, I've persevered, mainly because I'm used to the Trackpoint, and I like the availability of hardware service manuals. Overall I've been pleased with the engineering of the X series. Over the last few days I learned about the installation of the superfish malware on new Lenovo systems, and Lenovo's completely inadequate response to the revelation. I don't use Windows, so this malware would not have directly affected me (unless I had the misfortune to use this system to download installation media for some GNU/Linux distribution). Nonetheless, how can I trust the firmware installed by a company that seems to value its users' security and privacy so little? Unless Lenovo can show some sign of understanding the gravity of this mistake, and undertake not to repeat it, then I'm afraid you will be joining Sony on my list of vendors I used to consider buying from. Sure, it's only a gross income loss of $500 a year or so, if you assume I'm alone in this reaction. I don't think I'm alone in being disgusted and angered by this incident.

Wouter Verhelst: LOADays 2015

Looks like I'll be speaking at LOADays again. This time around, at the suggestion of one of the organisers, I'll be speaking about the Belgian electronic ID card, for which I'm currently employed as a contractor to help maintain the end-user software. While this hasn't been officially confirmed yet, I've been hearing some positive signals from some of the organisers. So, under the assumption that my talk will be accepted, I've started working on my slides. The intent is to explain how the eID middleware works (in general terms), how the Linux support is supposed to work, and what to do when things fail. If my talk doesn't get rejected at the final hour, I will continue my uninterrupted "speaker at loadays" streak, which has started since loadays' first edition...

19 February 2015

Matthew Garrett: It has been 0 days since the last significant security failure. It always will be.

So blah blah Superfish blah blah trivial MITM everything's broken.

Lenovo deserve criticism. The level of incompetence involved here is so staggering that it wouldn't be a gross injustice for the company to go under as a result[1]. But let's not pretend that this is some sort of isolated incident. As an industry, we don't care about user security. We will gladly ship products with known security failings and no plans to update them. We will produce devices that are locked down such that it's impossible for anybody else to fix our failures. We will hide behind vague denials, we will obfuscate the impact of flaws and we will deflect criticisms with announcements of new and shinier products that will make everything better.

It'd be wonderful to say that this is limited to the proprietary software industry. I would love to be able to argue that we respect users more in the free software world. But there are too many cases that demonstrate otherwise, even where we should have the opportunity to prove the benefits of open development. An obvious example is the smartphone market. Hardware vendors will frequently fail to provide timely security updates, and will cease to update devices entirely after a very short period of time. Fortunately there's a huge community of people willing to produce updated firmware. Phone manufacturer is never going to fix the latest OpenSSL flaw? As long as your phone can be unlocked, there's a reasonable chance that there's an updated version on the internet.

But this is let down by a kind of callous disregard for any deeper level of security. Almost every single third-party Android image is either unsigned or signed with the "test keys", a set of keys distributed with the Android source code. These keys are publicly available, and as such anybody can sign anything with them. If you configure your phone to allow you to install these images, anybody with physical access to your phone can replace your operating system. You've gained some level of security at the application level by giving up any real ability to trust your operating system.

This is symptomatic of our entire ecosystem. We're happy to tell people to disable security features in order to install third-party software. We're happy to tell people to download and build source code without providing any meaningful way to verify that it hasn't been tampered with. Install methods for popular utilities often still start "curl sudo bash". This isn't good enough.

We can laugh at proprietary vendors engaging in dreadful security practices. We can feel smug about giving users the tools to choose their own level of security. But until we're actually making it straightforward for users to choose freedom without giving up security, we're not providing something meaningfully better - we're just providing the same shit sandwich on different bread.

[1] I don't see any way that they will, but it wouldn't upset me

comment count unavailable comments

Niels Thykier: Partial rewrite of lintian s reporting setup

I had the mixed pleasure of doing a partial rewrite of lintian s reporting framework. It started as a problem with generating the graphs, which turned out to be not enough memory . On the plus side, I am actually quite pleased with the end result. I managed to scope-creep myself quite a bit and I ended up getting rid of a lot of old issues. The major changes in summary: There are also some nice minor features : As you can probably tell, I am quite pleased with the end result. The reporting framework lacks behind in development, since it just sits there and takes care of itself . Also with the complete lack of testing, it also suffers from the if it is not broken, then do not fix it paradigm (because we will not notice if we broke until it is too late). Of course, I managed to break the setup a couple of times in the process. However, a bonus feature of the reporting setup is that if you break it, it simply leaves an outdated report on the website. Anyway, enjoy. :)
Filed under: Debian, Lintian

Nicolas Dandrimont: We need your help to make GSoC and Outreachy in Debian a success this summer!

Hi everyone, A quick announcement: Debian has applied to the Google Summer of Code, and will also participate in Outreachy (formerly known as the Outreach Program for Women) for the Summer 2015 round! Those two mentoring programs are a great way for our project to bootstrap new ideas, give an new impulse to some old ones, and of course to welcome an outstanding team of motivated, curious, lively new people among us. We need projects and mentors to sign up really soon (before February 27th, that s next week), as our project list is what Google uses to evaluate our application to GSoC. Projects proposals should be described on our wiki page. We have three sections:
  1. Coding projects with confirmed mentors are proposed to both GSoC and Outreachy applicants
  2. Non-Coding projects with confirmed mentors are proposed only to Outreachy applicants
  3. Project ideas without confirmed mentors will only happen if a mentor appears. They are kept on the wiki page until the application period starts, as we don t want to give applicants false hopes of being picked for a project that won t happen.
Once you re done, or if you have any questions, drop us a line on our mailing-list (soc-coordination@lists.alioth.debian.org), or on #debian-soc on OFTC. We also would LOVE to be able to welcome more Outreachy interns. So far, and thanks to our DPL, Debian has committed to fund one internship (US$6500). If we want more Outreachy interns, we need your help :). If you, or your company, have some money to put towards an internship, please drop us a line at opw@debian.org and we ll be in touch. Some of the successes of our Outreachy alumni include the localization of the Debian Installer to a new locale, improvements in the sources.debian.net service, documentation of the debbugs codebase, and a better integration of AppArmor profiles in Debian. Thanks a lot for your help!

18 February 2015

Philipp Kern: Caveats of the HP MicroServer Gen8

If you intend to buy the HP MicroServer Gen8 as a home server there are a few caveats that I didn't find on the interwebs before I bought the device:
If you try to boot FreeBSD with its zfsloader you will likely need to apply a workaround patch, because the BIOS seems to do something odd. Linux works as expected.

17 February 2015

Lucas Nussbaum: Some email organization tips and tricks

I d like to share a few tips that were useful to strengthen my personal email organization. Most of what follows is probably not very new nor special, but hey, let s document it anyway. Many people have an inbox folder that just grow over time. It s actually similar to a twitter or RSS feed (except they probably agree that they are supposed to read more of their email feed ). When I send an email to them, it sometimes happen that they don t notice it, if the email arrives at a bad time. Of course, as long as they don t receive too many emails, and there aren t too many people relying on them, it might just work. But from time to time, it s super-painful for those interacting with them, when they miss an email and they need to be pinged again. So let s try not to be like them. :-) Tip #1: do Inbox Zero (or your own variant of it) Inbox Zero is an email management methodology inspired from David Allen s Getting Things Done book. It s best described in this video. The idea is to turn one s Inbox into an area that is only temporary storage, where every email will get processed at some point. Processing can mean deleting an email, archiving it, doing the action described in the email (and replying to it), etc. Put differently, it basically means implementing the Getting Things Done workflow on one s email. Tip #1.1: archive everything One of the time-consuming decisions in the original GTD workflow is to decide whether something should be eliminated (deleted) or stored for reference. Given that disk space is quite cheap, it s much easier to never decide about that, and just archive everything (by duplicating the email to an archive folder when it is received). To retrieve archived emails when needed, I then use notmuch within mutt to easily search through recent (< 2 year) archives. I use archivemail to archive older email in compressed mboxes from time to time, and grepmail to search through those mboxes when needed. I don t archive most Debian mailing lists though, as they are easy to fetch from master.d.o with the following script:
#!/bin/sh
rsync -vP master.debian.org:~debian/*/*$1/*$1.$ 2:-$(date +%Y%m) * .
Then I can fetch a specific list archive with getlist devel 201502, or a set of archives with e.g. getlist devel 2014, or the current month with e.g. getlist devel. Note that to use grepmail on XZ-compressed archives, you need libmail-mbox-messageparser-perl version 1.5002-3 (only in unstable I was using a locally-patched version for ages, but finally made a patch last week, which Gregor kindly uploaded). Tip #1.2: split your inbox (Yes, this one looks obvious but I m always surprised at how many people don t do that.) Like me, you probably receive various kinds of emails: Splitting those into separate folders has several advantages: I ve seen some people splitting their inbox into too many folders. There s not much point in having a per-sender folder organization (unless there s really a recurring flow of emails from a specific person), as it increases the risk of missing an email. I use procmail to organize my email into folders. I know that there are several more modern alternatives, but I haven t looked at them since procmail does the job for me. Resulting organization I use one folder for my day-job email, one for my DPL email, one for all other email directed or Cced to me. Then, I have a few folders for automated notifications of stuff. My Debian mailing list folders are auto-managed by procmail s $MATCH:
:0:
 * ^X-Mailing-List: <.*@lists\.debian\.org>
 * ^X-Mailing-List: <debian-\/[^@]*
 .ml.debian.$MATCH/
Some other mailing lists are in they separate folders, and there s a catch-all folder for the remaining ones. Ah, and since I use feed2imap, I have folders for the RSS/Atom feeds I follow. I have two different commands to start mutt. One only shows a restricted number of (important) folders. The other one shows the full list of (non-empty) folders. This is a good trick to avoid spending time reading email when I am supposed to do something more important. :) As for many people probably, my own organization is loosely based on GTD and Inbox Zero. It sometimes happen that some emails stay in my Inbox for several days or weeks, but I very rarely have more than 20 or 30 emails in one of my main inbox folders. I also do reviews of the whole content of my main inbox folders once or twice a week, to ensure that I did not miss an email that could be acted on quickly. A last trick is that I have a special folder replies, where procmail copies emails that are replies to a mail I sent, but which do not Cc me. That s useful to work-around Debian s no Cc on reply to mailing list posts policy. I receive email using offlineimap (over SSH to my mail server), and send it using nullmailer (through a SSH tunnel). The main advantage of offlineimap over using IMAP directly in mutt is that using IMAP to a remote server feels much more sluggish. Another advantage is that I only need SSH access to get my full email setup to work. Tip #2: tracking sent emails Two recurring needs I had was: I developed a short script to address that. It scans the content of my Sent maildir and my archive maildirs, and, for each email address I use, displays (see example output in README) the list of emails sent with this address (one email per line). It also detects if an email was replied to ( R column after the date), and abbreviates common parts in email addresses (debian-project@lists.debian.org becomes d-project@l.d.o). It also generates a temporary maildir with symlinks to all emails, so that I can just open the maildir if needed.

John Goerzen: Has Linux lost its way? comments prompt a Debian developer to revisit FreeBSD after 20 years

I ll admit it. I have a soft spot for FreeBSD. FreeBSD was the first Unix I ran, and it was somewhere around 20 years ago that I did so, before I switched to Debian. Even then, I still used some of the FreeBSD Handbook to learn Linux, because Debian didn t have the great Reference that it does now. Anyhow, some comments in my recent posts ( Has modern Linux lost its way? and Reactions to that, and the value of simplicity), plus a latent desire to see how ZFS fares in FreeBSD, caused me to try it out. I installed it both in VirtualBox under Debian, and in an old 64-bit Thinkpad sitting in my basement that previously ran Debian. The results? A mixture of amazing and disappointing. I will say that I am quite glad that both exist; there is plenty of innovation happening everywhere and neat features exist everywhere, too. But I can also come right out and say that the statement that FreeBSD doesn t have issues like Linux does is false and misleading. In many cases, it s running the exact same stack. In others, it s better, but there are also others where it s worse. Perhaps this article might dispell a bit of the FUD surrounding jessie, while also showing off some of the nice things FreeBSD does. My conclusion: Both jessie and FreeBSD 10.1 are awesome Free operating systems, but both have their warts. This article is more about FreeBSD than Debian, but it will discuss a few of Debian s warts as well. The experience My initial reaction to FreeBSD was: wow, this feels so familiar. It reminds me of a commercial Unix, or maybe of Linux from a few years ago. A minimal, well-documented base system, everything pretty much in logical places in the filesystem, and solid memory management. I felt right at home. It was almost reassuring, even. Putting together a FreeBSD box is a lot of package installing and config file editing. The FreeBSD Handbook, describing how to install X, talks about editing this or that file for this or that feature. I like being able to learn directly how things fit together by doing this. But then you start remembering the reasons you didn t like Linux a few years ago, or the commercial Unixes: maybe it s that programs like apache are still not as well supported, or maybe it s that the default vi has this tendency to corrupt the terminal periodically, or perhaps it s that root s default shell is csh. Or perhaps it s that I have to do a lot of package installing and config file editing. It is not quite the learning experience it once was, either; now there are things like paste this XML file into some obscure polkit location to make your mouse work or something. Overall, there are some areas where FreeBSD kills it in a way no other OS does. It is unquestionably awesome in several areas. But there are a whole bunch of areas where it s about 80% as good as Linux, a number of areas (even polkit, dbus, and hal) where it s using the exact same stack Linux is (so all these comments about FreeBSD being so differently put together strike me as hollow), and frankly some areas that need a lot of work and make it hard to manage systems in a secure and stable way. The amazing Let s get this out there: I ve used ZFS too much to use any OS that doesn t support it or something like it. Right now, I m not aware of anything like ZFS that is generally stable and doesn t cost a fortune, so pretty much: if your Unix doesn t do ZFS, I m not interested. (btrfs isn t there yet, but will be awesome when it is.) That s why I picked FreeBSD for this, rather than NetBSD or OpenBSD. ZFS on FreeBSD is simply awesome. They have integreated it extremely well. The installer supports root on zfs, even encrypted root on zfs (though neither is a default). top on a FreeBSD system shows a line of ZFS ARC (cache) stats right alongside everything else. The ZFS defaults for maximum cache size, readahead, etc. auto-tune themselves at boot (unless overridden) based on the amount of RAM in a system and the system type. Seriously, these folks have thought of everything and it just reeks of solid. I haven t seen ZFS this well integrated outside the Solaris-type OSs. I have been using ZFSOnLinux for some time now, but it is just not as mature as ZFS on FreeBSD. ZoL, for instance, still has some memory tuning issues, and is not really suggested for 32-bit machines. FreeBSD just nails it. ZFS on FreeBSD even supports TRIM, which is not available in ZoL and I think fairly unique even among OpenZFS platforms. It also supports delegated administration of the filesystem, both to users and to jails on the system, seemingly very similar to Solaris zones. FreeBSD also supports beadm, which is like a similar tool on Solaris. This lets you basically use ZFS snapshots to make lightweight boot environments , so you can select which to boot into. This is useful, say, before doing upgrades. Then there are jails. Linux has tried so hard to get this right, and fallen on its face so many times, a person just wants to take pity sometimes. We ve had linux-vserver, openvz, lxc, and still none of them match what FreeBSD jails have done for a long time. Linux s current jail-du-jour is LXC, though it is extremely difficult to configure in a secure way. Even its author comments that you won t hear any of the LXC maintainers tell you that LXC is secure and that it pretty much requires AppArmor profiles to achieve reasonable security. These are still rather in flux, as I found out last time I tried LXC a few months ago. My confidence in LXC being as secure as, say, KVM or FreeBSD is simply very low. FreeBSD s jails are simple and well-documented where LXC is complex and hard to figure out. Its security is fairly transparent and easy to control and they just work well. I do think LXC is moving in the right direction and might even get there in a couple years, but I am quite skeptical that even Docker is getting the security completely right. The simply different People have been throwing around the word distribution with respect to FreeBSD, PC-BSD, etc. in recent years. There is an analogy there, but it s not perfect. In the Linux ecosystem, there is a kernel project, a libc project, a coreutils project, a udev project, a systemd/sysvinit/whatever project, etc. You get the idea. In FreeBSD, there is a base system project. This one project covers the kernel and the base userland. Some of what they use in the base system is code pulled in from elsewhere but maintained in their tree (ssh), some is completely homegrown (kernel), etc. But in the end, they have a nicely-integrated base system that always gets upgraded in sync. In the Linux world, the distribution makers are responsible for integrating the bits from everywhere into a coherent whole. FreeBSD is something of a toolkit to build up your system. Gentoo might be an analogy in the Linux side. On the other end of the spectrum, Ubuntu is a just install it and it works, tweak later sort of setup. Debian straddles the middle ground, offering both approaches in many cases. There are pros and cons to each approach. Generally, I don t think either one is better. They are just different. The not-quite-there I said that there are a lot of things in FreeBSD that are about 80% of where Linux is. Let me touch on them here. Its laptop support leaves something to be desired. I installed it on a few-years-old Thinkpad basically the best possible platform for working suspend in a Free OS. It has worked perfectly out of the box in Debian for years. In FreeBSD, suspend only works if it s in text mode. If X is running, the video gets corrupted and the system hangs. I have not tried to debug it further, but would also note that suspend on closed lid is not automatic in FreeBSD; the somewhat obscure instuctions tell you what policykit pkla file to edit to make suspend work in XFCE. (Incidentally, it also says what policykit file to edit to make the shutdown/restart options work). Its storage subsystem also has some surprising misses. Its rough version of LVM, LUKS, and md-raid is called GEOM. GEOM, however, supports only RAID0, RAID1, and RAID3. It does not support RAID5 or RAID6 in software RAID configurations! Linux s md-raid, by comparison, supports RAID0, RAID1, RAID4, RAID5, RAID6, etc. There seems to be a highly experimental RAID5 patchset floating around for many years, but it is certainly not integrated into the latest release kernel. The current documentation makes no mention of RAID5, although it seems that a dated logical volume manager supported it. In any case, RAID5 does not seem to be well-supported in software like it is in Linux. ZFS does have its raidz1 level, which is roughly the same as RAID5. However, that requires full use of ZFS. ZFS also does not support some common operations, like adding a single disk to an existing RAID5 group (which is possible with md-raid and many other implementations.) This is a ZFS limitation on all platforms. FreeBSD s filesystem support is rather a miss. They once had support for Linux ext* filesystems using the actual Linux code, but ripped it out because it was in GPL and rewrote it so it had a BSD license. The resulting driver really only works with ext2 filesystems, as it doesn t work with ext3/ext4 in many situations. Frankly I don t see why they bothered; they now have something that is BSD-licensed but only works with a filesystem so old nobody uses it anymore. There are only two FreeBSD filesystems that are really useable: UFS2 and ZFS. Virtualization under FreeBSD is also not all that present. Although it does support the VirtualBox Open Source Edition, this is not really a full-featured or fast enough virtualization environment for a server. Its other option is bhyve, which looks to be something of a Xen clone. bhyve, however, does not support Windows guests, and requires some hoops to even boot Linux guest installers. It will be several years at least before it reaches feature-parity with where KVM is today, I suspect. One can run FreeBSD as a guest under a number of different virtualization systems, but their instructions for making the mouse work best under VirtualBox did not work. There may have been some X.Org reshuffle in FreeBSD that wasn t taken into account. The installer can be nice and fast in some situations, but one wonders a little bit about QA. I had it lock up on my twice. Turns out this is a known bug reported 2 months ago with no activity, in which the installer attempts to use a package manger that it hasn t set up yet to install optional docs. I guess the devs aren t installing the docs in testing. There is nothing like Dropbox for FreeBSD. Apparently this is because FreeBSD has nothing like Linux s inotify. The Linux Dropbox does not work in FreeBSD s Linux mode. There are sketchy reports of people getting an OwnCloud client to work, but in something more akin to rsync rather than instant-sync mode, if they get it working at all. Some run Dropbox under wine, apparently. The desktop environments tend to need a lot more configuration work to get them going than on Linux. There s a lot of editing of polkit, hal, dbus, etc. config files mentioned in various places. So, not only does FreeBSD use a lot of the same components that cause confusion in Linux, it doesn t really configure them for you as much out of the box. FreeBSD doesn t support as many platforms as Linux. FreeBSD has only two platforms that are fully supported: i386 and amd64. But you ll see people refer to a list of other platforms that are supported , but they don t have security support, official releases, or even built packages. They includ arm, ia64, powerpc, and sparc64. The bad: package management Roughly 20 years ago, this was one of the things that pulled me to Debian. Perhaps I am spolied from running the distribution that has been the gold standard for package management for so long, but I find FreeBSD s package management even pkg-ng in 10.1-RELEASE to be lacking in a number of important ways. To start with, FreeBSD actually has two different package management systems: one for the base system, and one for what they call the ports/packages collection ( ports being the way to install from source, and packages being the way to install from binaries, but both related to the same tree.) For the base system, there is freebsd-update which can install patches and major upgrades. It also has a cron option to automate this. Sadly, it has no way of automatically indicating to a calling script whether a reboot is necessary. freebsd-update really manages less than a dozen packages though. The rest are managed by pkg. And pkg, it turns out, has a number of issues. The biggest: it can take a week to get security updates. The FreeBSD handbook explains pkg audit -F which will look at your installed packages (but NOT the ones in the base system) and alert you to packages that need to be updates, similar to a stripped-down version of Debian s debsecan. I discovered this myself, when pkg audit -F showed a vulnerability in xorg, but pkg upgrade showed my system was up-to-date. It is not documented in the Handbook, but people on the mailing list explained it to me. There are workarounds, but they can be laborious. If that s not bad enough, FreeBSD has no way to automatically install security patches for things in the packages collection. Debian has several (unattended-upgrades, cron-apt, etc.) There is pkg upgrade , but it upgrades everything on the system, which may be quite a bit more than you want to be upgraded. So: if you want to run Apache with PHP, and want it to just always apply security patches, FreeBSD packages are not up to the job like Debian s are. The pkg tool doesn t have very good error-handling. In fact, its error handling seems to be nonexistent at times. I noticed that some packages had failures during install time, but pkg ignored them and marked the package as correctly installed. I only noticed there was a problem because I happened to glance at the screen at the right moment during messages about hundreds of packages. In Debian, by contrast, if there are any failures, at the end of the run, you get a nice report of which packages failed, and an exit status to use in scripts. It also has another issue that Debian resolved about a decade ago: package scripts displaying messages that are important for the administrator, but showing so many of them that they scroll off the screen and are never seen. I submitted a bug report for this one also. Some of these things just make me question the design of pkg. If I can t trust it to accurately report if the installation succeeded, or show me the important info I need to see, then to what extent can I trust it? Then there is the question of testing of the ports/packages. It seems that, automated tests aside, basically everyone is running off the master branch of the ports/packages. That s like running Debian unstable on your servers. I am distinctly uncomfortable with this notion, though it seems FreeBSD people report it mostly works well. There are some other issues, too: FreeBSD ports make no distinction between development and runtime files like Debian s packages do. So, just by virtue of wanting to run a graphical desktop, you get all of the static libraries, include files, build scripts, etc for XOrg installed. For a package as concerned about licensing as FreeBSD, the packages collection does not have separate sections like Debian s main, contrib, and non-free. It s all in one big pot: BSD-license, GPL-license, proprietary without source license. There is /usr/local/share/licenses where you can look up a license for each package, but there is no way with FreeBSD, like there is with Debian, to say never even show me packages that aren t DFSG-free. This is useful, for instance, when running in a company to make sure you never install packages that are for personal use only or something. The bad: ABI stability I m used to being able to run binaries I compiled years ago on a modern system. This is generally possible in Linux, assuming you have the correct shared libraries available. In FreeBSD, this is explicitly NOT possible. After every major version upgrade, you must reinstall or recompile every binary on your system. This is not necessarily a showstopper for me, but it is a hassle for a lot of people. Update 2015-02-17: Some people in the comments are pointing out compat packages in the ports that may help with this situation. My comment was based on advice in the FreeBSD Handbook stating After a major version upgrade, all installed packages and ports need to be upgraded . I have not directly tried this, so if the Handbook is overstating the need, then this point may be in error. Conclusions As I said above, I found little validation to the comments that the Debian ecosystem is noticeably worse than the FreeBSD one. Debian has its warts too particularly with keeping software up-to-date. You can see that the two projects are designed around a different passion: FreeBSD s around the base system, and Debian s around an integrated whole system. It would be wrong to say that either of those is always better. FreeBSD s approach clearly produces some leading features, especially jails and ZFS integration. Yet Debian s approach also produces some leading features in the way of package management and security maintainability beyond the small base. My criticism of excessive complexity in the polkit/cgmanager/dbus area still stands. But to those people commenting that FreeBSD hasn t lost its way like Linux has, I would point out that FreeBSD mostly uses these same components also, and FreeBSD has excessive complexity in its ports/package system and system management tools. I think it s a draw. You pick the best for your use case. If you re looking for a platform to run a single custom app then perhaps all of the Debian package management benefits don t apply to you (you may not even need FreeBSD s packages, or just a few). The FreeBSD ZFS support or jails may well appeal. If you re looking to run a desktop environment, or a server with some application that needs a ton of PHP, Python, Perl, or C libraries, then Debian s package management and security handling may well be attractive. I am disappointed that Debian GNU/kFreeBSD will not be a release architecture in jessie. That project had the promise to provide a best of both worlds for those that want jails or tight ZFS integration.

16 February 2015

Matthew Garrett: Intel Boot Guard, Coreboot and user freedom

PC World wrote an article on how the use of Intel Boot Guard by PC manufacturers is making it impossible for end-users to install replacement firmware such as Coreboot on their hardware. It's easy to interpret this as Intel acting to restrict competition in the firmware market, but the reality is actually a little more subtle than that.

UEFI Secure Boot as a specification is still unbroken, which makes attacking the underlying firmware much more attractive. We've seen several presentations at security conferences lately that have demonstrated vulnerabilities that permit modification of the firmware itself. Once you can insert arbitrary code in the firmware, Secure Boot doesn't do a great deal to protect you - the firmware could be modified to boot unsigned code, or even to modify your signed bootloader such that it backdoors the kernel on the fly.

But that's not all. Someone with physical access to your system could reflash your system. Even if you're paranoid enough that you X-ray your machine after every border crossing and verify that no additional components have been inserted, modified firmware could still be grabbing your disk encryption passphrase and stashing it somewhere for later examination.

Intel Boot Guard is intended to protect against this scenario. When your CPU starts up, it reads some code out of flash and executes it. With Intel Boot Guard, the CPU verifies a signature on that code before executing it[1]. The hash of the public half of the signing key is flashed into fuses on the CPU. It is the system vendor that owns this key and chooses to flash it into the CPU, not Intel.

This has genuine security benefits. It's no longer possible for an attacker to simply modify or replace the firmware - they have to find some other way to trick it into executing arbitrary code, and over time these will be closed off. But in the process, the system vendor has prevented the user from being able to make an informed choice to replace their system firmware.

The usual argument here is that in an increasingly hostile environment, opt-in security isn't sufficient - it's the role of the vendor to ensure that users are as protected as possible by default, and in this case all that's sacrificed is the ability for a few hobbyists to replace their system firmware. But this is a false dichotomy - UEFI Secure Boot demonstrated that it was entirely possible to produce a security solution that provided security benefits and still gave the user ultimate control over the code that their machine would execute.

To an extent the market will provide solutions to this. Vendors such as Purism will sell modern hardware without enabling Boot Guard. However, many people will buy hardware without consideration of this feature and only later become aware of what they've given up. It should never be necessary for someone to spend more money to purchase new hardware in order to obtain the freedom to run their choice of software. A future where users are obliged to run proprietary code because they can't afford another laptop is a dystopian one.

Intel should be congratulated for taking steps to make it more difficult for attackers to compromise system firmware, but criticised for doing so in such a way that vendors are forced to choose between security and freedom. The ability to control the software that your system runs is fundamental to Free Software, and we must reject solutions that provide security at the expense of that ability. As an industry we should endeavour to identify solutions that provide both freedom and security and work with vendors to make those solutions available, and as a movement we should be doing a better job of articulating why this freedom is a fundamental part of users being able to place trust in their property.

[1] It's slightly more complicated than that in reality, but the specifics really aren't that interesting.

comment count unavailable comments

Julien Danjou: Hacking Python AST: checking methods declaration

A few months ago, I wrote the definitive guide about Python method declaration, which had quite a good success. I still fight every day in OpenStack to have the developers declare their methods correctly in the patches they submit. Automation plan The thing is, I really dislike doing the same things over and over again. Furthermore, I'm not perfect either, and I miss a lot of these kind of problems in the reviews I made. So I decided to replace me by a program a more scalable and less error-prone version of my brain. In OpenStack, we rely on flake8 to do static analysis of our Python code in order to spot common programming mistakes. But we are really pedantic, so we wrote some extra hacking rules that we enforce on our code. To that end, we wrote a flake8 extension called hacking. I really like these rules, I even recommend to apply them in your own project. Though I might be biased or victim of Stockholm syndrome. Your call. Anyway, it's pretty clear that I need to add a check for method declaration in hacking. Let's write a flake8 extension! Typical error The typical error I spot is the following:
class Foo(object):
# self is not used, the method does not need
# to be bound, it should be declared static
def bar(self, a, b, c):
return a + b - c

That would be the correct version:
class Foo(object):
@staticmethod
def bar(a, b, c):
return a + b - c

This kind of mistake is not a show-stopper. It's just not optimized. Why you have to manually declare static or class methods might be a language issue, but I don't want to debate about Python misfeatures or design flaws. Strategy We could probably use some big magical regular expression to catch this problem. flake8 is based on the pep8 tool, which can do a line by line analysis of the code. But this method would make it very hard and error prone to detect this pattern. Though it's also possible to do an AST based analysis on on a per-file basis with pep8. So that's the method I pick as it's the most solid. AST analysis I won't dive deeply into Python AST and how it works. You can find plenty of sources on the Internet, and I even talk about it a bit in my book The Hacker's Guide to Python. To check correctly if all the methods in a Python file are correctly declared, we need to do the following: Flake8 plugin In order to register a new plugin in flake8 via hacking, we just need to add an entry in setup.cfg:
[entry_points]
flake8.extension =
[ ]
H904 = hacking.checks.other:StaticmethodChecker
H905 = hacking.checks.other:StaticmethodChecker

We register 2 hacking codes here. As you will notice later, we are actually going to add an extra check in our code for the same price. Stay tuned. The next step is to write the actual plugin. Since we are using an AST based check, the plugin needs to be a class following a certain signature:
@core.flake8ext
class StaticmethodChecker(object):
def __init__(self, tree, filename):
self.tree = tree

def run(self):
pass

So far, so good and pretty easy. We store the tree locally, then we just need to use it in run() and yield the problem we discover following pep8 expected signature, which is a tuple of (lineno, col_offset, error_string, code). This AST is made for walking The ast module provides the walk function, that allow to iterate easily on a tree. We'll use that to run through the AST. First, let's write a loop that ignores the statement that are not class definition.
@core.flake8ext
class StaticmethodChecker(object):
def __init__(self, tree, filename):
self.tree = tree

def run(self):
for stmt in ast.walk(self.tree):
# Ignore non-class
if not isinstance(stmt, ast.ClassDef):
continue

We still don't check for anything, but we know how to ignore statement that are not class definitions. The next step need to be to ignore what is not function definition. We just iterate over the attributes of the class definition.
for stmt in ast.walk(self.tree):
# Ignore non-class
if not isinstance(stmt, ast.ClassDef):
continue
# If it's a class, iterate over its body member to find methods
for body_item in stmt.body:
# Not a method, skip
if not isinstance(body_item, ast.FunctionDef):
continue

We're all set for checking the method, which is body_item. First, we need to check if it's already declared as static. If so, we don't have to do any further check and we can bail out.
for stmt in ast.walk(self.tree):
# Ignore non-class
if not isinstance(stmt, ast.ClassDef):
continue
# If it's a class, iterate over its body member to find methods
for body_item in stmt.body:
# Not a method, skip
if not isinstance(body_item, ast.FunctionDef):
continue
# Check that it has a decorator
for decorator in body_item.decorator_list:
if (isinstance(decorator, ast.Name)
and decorator.id == 'staticmethod'):
# It's a static function, it's OK
break
else:
# Function is not static, we do nothing for now
pass

Note that we use the special for/else form of Python, where the else is evaluated unless we used break to exit the for loop.
for stmt in ast.walk(self.tree):
# Ignore non-class
if not isinstance(stmt, ast.ClassDef):
continue
# If it's a class, iterate over its body member to find methods
for body_item in stmt.body:
# Not a method, skip
if not isinstance(body_item, ast.FunctionDef):
continue
# Check that it has a decorator
for decorator in body_item.decorator_list:
if (isinstance(decorator, ast.Name)
and decorator.id == 'staticmethod'):
# It's a static function, it's OK
break
else:
try:
first_arg = body_item.args.args[0]
except IndexError:
yield (
body_item.lineno,
body_item.col_offset,
"H905: method misses first argument",
"H905",
)
# Check next method
continue

We finally added some check! We grab the first argument from the method signature. Unless it fails, and in that case, we know there's a problem: you can't have a bound method without the self argument, therefore we raise the H905 code to signal a method that misses its first argument. Now you know why we registered this second pep8 code along with H904 in setup.cfg. We have here a good opportunity to kill two birds with one stone. The next step is to check if that first argument is used in the code of the method.
for stmt in ast.walk(self.tree):
# Ignore non-class
if not isinstance(stmt, ast.ClassDef):
continue
# If it's a class, iterate over its body member to find methods
for body_item in stmt.body:
# Not a method, skip
if not isinstance(body_item, ast.FunctionDef):
continue
# Check that it has a decorator
for decorator in body_item.decorator_list:
if (isinstance(decorator, ast.Name)
and decorator.id == 'staticmethod'):
# It's a static function, it's OK
break
else:
try:
first_arg = body_item.args.args[0]
except IndexError:
yield (
body_item.lineno,
body_item.col_offset,
"H905: method misses first argument",
"H905",
)
# Check next method
continue
for func_stmt in ast.walk(body_item):
if six.PY3:
if (isinstance(func_stmt, ast.Name)
and first_arg.arg == func_stmt.id):
# The first argument is used, it's OK
break
else:
if (func_stmt != first_arg
and isinstance(func_stmt, ast.Name)
and func_stmt.id == first_arg.id):
# The first argument is used, it's OK
break
else:
yield (
body_item.lineno,
body_item.col_offset,
"H904: method should be declared static",
"H904",
)

To that end, we iterate using ast.walk again and we look for the use of the same variable named (usually self, but if could be anything, like cls for @classmethod) in the body of the function. If not found, we finally yield the H904 error code. Otherwise, we're good. Conclusion I've submitted this patch to hacking, and, finger crossed, it might be merged one day. If it's not I'll create a new Python package with that check for flake8. The actual submitted code is a bit more complex to take into account the use of abc module and include some tests. As you may have notice, the code walks over the module AST definition several times. There might be a couple of optimization to browse the AST in only one pass, but I'm not sure it's worth it considering the actual usage of the tool. I'll let that as an exercise for the reader interested in contributing to OpenStack. Happy hacking!
The Hacker's Guide to Python
A book I wrote talking about designing Python applications, state of the art, advice to apply when building your application, various Python tips, etc. Interested? Check it out.

Next.

Previous.