Search Results: "reg"

30 May 2011

Christoph Egger: Maintaining kFreeBSD buildds since one month

At April 30, I took over maintenance of of Debian's kFreeBSD autobuilders. Means getting something like 4,5k e-Mails this month (gladly no need to sign all those 4k successful builds any more!), filling nearly 30 RC Bugs (quite a lot of which got fixed just within hours after filling, wow!), investigating some of the more strange build failures and such stuff. In general it turned out to be quite some fun. Quite interesting which libraries turn out to be rather central to the Archive. I wouldn't have guessed that a uninstallable libmtp would prevent a reasonable fraction of the builds to fail -- including packages like subversion. Packages builds failing because the disk space is exhausted may be something most of us have already witnessed, especially those here that use one of these small notebook hard drives. Build failures caused by a lack of RAM might certainly be imaginable as well, especially on highly parallel builds. But have you ever seen gcc fail because the virtual address space was exhausted on 32 bit architectures? Also there's a interesting lot of packages with misspelled build dependencies which sbuild can't find and therefore can't build the package. Maybe having a lintian check for some of these problems would be a good idea? I'm also regularly seeing build failures that look easy enough to fix -- like some glob in a *.install for some python package matching lib.linux*. I try to fix some of these as I see them but my time is unfortunately limited as well. Someone interested in quick&easy noticed about these kind of issues? I could put URLs to build-logs on identi.ca or somewhere on IRC. There are also some really strange failures like llvm, which builds flawlessly on my local kFreeBSD boxes all the time, inside and outside schroot but hangs all the time in the same testcase when building on any of the autobuilders (any hints welcome!) or perl failing on kfreebsd-amd64 selectively but all the time.

Sam Dunne: Diversions Control File

So after some discussion I can share the first draft syntax of the control file that ll be used for declarative diversions
One diversion per line Blank lines and lines beginning with # are comments Two fields per diversion seperated by whitespace (Source Destination)
Hopefully the full design draft for the project will be available on Friday of this week

EDIT (30/05/2011 11AM GMT+1) I realise there will be some issues regarding spaces within file names. This will be addressed hopefully by the end of the week. If anyone happens to see anything obvious that I ve overlooked you can catch me on IRC as SCD[Eire]. I d appreciate all and any input

~Sam

27 May 2011

Matt Zimmerman: Why I m excited about joining Singly

This summer, I ll be taking a bit of time off, moving back to San Francisco and starting a new job. I can t wait to get back to work. Here s why. Me and my data
I have a singular relationship with my data. I have a copy of every email I ve sent since I first got an Internet email address in 1994 (82,000 messages and counting). I have even older files downloaded from BBSes, and passed between friends on floppy disks. Chat logs, text messages, voicemail I hold onto them all. Anything which is relevant to me personally, I tend to save. This must seem banal to people who are first getting online today. In the age of Gmail and Flickr, it s easy to assume that all of your data will be preserved indefinitely, with little or no effort on your part. But for me, it has been hard work over the years, because I ve done it myself. I ve carried my data with me to countless new computers, operating systems, storage technologies, file formats and cities over the years. Everywhere I ve lived, I ve brought it with me. Physically. Really? Why do I do this? Why have I gone to such trouble for a collection of bits? Especially now, why is most of my data still at home?
One pragmatic answer is that I can simply do more with my data when I have a copy. I can work with it using any software I want, including software that I write myself. I don t have to worry about whether I can transfer it from one web service to another. I m never stuck using yesterday s services because my data is never trapped in them. My personal data is always available to me me, always raw, ready and waiting for the next wave of software to come along. When it does, I can load my data into it and keep going. The fact that Facebook and Google disagree over sharing their users data doesn t bother me in the least. Another reason is that I want to be in control of it. I decide who to share my data with, and when. Some of it, I prefer not to share at all, with any person or company, and I have that choice. Even if a powerful government wants to access my data, I am afforded certain protection under the law, at least in the countries where I ve lived. If I turned over my data to service providers, my choices and protection would likely be much more limited. I have a deeper emotional attachment to my data as well. Enfolded within that vast pattern of bits is some part of my self. By sharing my personal data with other people, I show them something of who I am. Increasingly, my personal data is part of my identity. This is more than just a state of mind: it s been shown that even our non-identifying personal data can reveal who we are. In other words, it s not just my data it s me data . Singly I m joining Singly because I want to take this concept much further, and combine people, data and software into a different shape with people at the center. Today, we are creating vastly greater amounts of personal data, and it s stored in many more places. We leave our trail on the Internet in the form of activity streams, messages and content, spread across different web sites, each with their own inscrutable terms of service and (if we re lucky) their own API. These disconnected silos prevent us from using all of this information effectively. Meanwhile, we want and need to connect with each other in more ways than ever before. We need applications which can connect us, through our personal data, to the services we need. Singly is building the technology to make this possible. It will be designed with the deepest respect for the relationship that we have with our personal data, and with a vision for truly personal computing. Singly is This opportunity is a great fit for my interests and experience. Singly aims to be the commercial part of a vibrant open source community, and I m looking forward to building on what I ve learned in Debian, Canonical and Ubuntu to help make it a success. I ll have lots more to say about it as time goes on. Meanwhile, if you re interested in following what we re doing, here s where:

Lucas Nussbaum: Changes to Ruby in Debian (and Ubuntu)

(this is a copy of an mail sent to ruby-core@ and ruby-talk@) Since the beginning of 2011, the Debian Ruby team has been working on several big changes. Those changes all are available in Debian unstable, some of them are also available in Debian testing, and they should all be available in the next Debian and Ubuntu releases. I think that it addresses most of the reasonable concerns about Ruby in Debian. Using alternatives to switch between Ruby implementations The alternatives system is now used to manage the ruby symlink and the other related symlinks, making it easy to switch between Ruby implementations (only Ruby 1.8 and 1.9.X at the moment) (see this mail for details). The default choice for Ruby is still 1.8, but this change will make it easy for us to make a switch to 1.9.X by default (likely by the release of Debian wheezy). Installing gems executables to /usr/local/bin Rubygems (both as a standalone package, and as shipped with Ruby 1.9.X) now install executables to /usr/local/bin. (The other files still get installed under /var, see this mail) Enabling gem update system gem update system has been re-enabled. Since upgrading rubygems to a version that may not have been properly tested with the rest of the Debian system may cause issues in the user s system, there s a big warning about that. The user can confirm and upgrade rubygems anyway by defining an environment variable. New gem2deb packaging helper There s a new packaging helper, named gem2deb, that makes it very easy to generate Debian source packages from Rubygems. We are in the process of migrating all ruby libraries packaged in Debian to that new helper. It will take some time, though (help is welcomed). Transition status: http://pkg-ruby-extras.alioth.debian.org/wheezy/ One big benefit of the switch to gem2deb for the Ruby community is that, in the process, we are enabling test suites at build time for each package and each Ruby implementation. This should make it easy to detect regressions in new interpreter versions. Ruby 1.9.3 We will switch to Ruby 1.9.3 ASAP (probably when it is branched off trunk, with a package first in Debian experimental). Since the Ruby compatibility version issue is likely to stay around, we will re-evaluate how we are dealing with it (to avoid the ruby1.9.1 package <=> ruby -v = 1.9.2 problem that confuses many users). This is likely by switching the package name to ruby1.9.3 (keep a ruby1.9.1 package for compatibility). The package containing the shared library will stay libruby1.9.1. Links

26 May 2011

Dirk Eddelbuettel: RcppArmadillo 0.2.20

Conrad Sanderson is preparing version 2.0.0 of his Armadillo templated C++ library for linear algebra and has released a first public beta version 1.99.2. This has been folded into a new release of the RcppArmadillo wrapper for R based on our Rcpp library. Also added in this version is a new example of a simulation of a vector autoregression process which I had blogged about earlier. The example had been prepared for the Rcpp workshop / class at last month's R/Finance conference, and demonstrates a rather nice speed gain from using Rcpp and RcppArmadillo. The short NEWS file extract follows below.
0.2.20  2011-05-25
    o   Upgraded to Armadillo release 1.99.2 "v2.0 beta 2" (and 1.99.1 before)
          * faster inverse of symmetric matrices
          * faster element access for fixed size matrices
          * faster multiplication of tiny matrices (eg. 4x4)
          * faster compund expressions containing submatrices
          * added handling of arbitrarily sized empty matrices (eg. 5x0)
          * added syl()
          * added strans()
          * added symmatu()/symmatl()
          * added submatrices of submatrices
          * htrans() has been deprecated; use trans() instead
          * trans() now takes the complex conjugate when transposing a
            complex matrix
          * .is_vec() now outputs true for empty matrices
          * most functions with matrix inputs no longer throw exceptions when
            given empty matrices (eg. 5x0)
    o   Added a new subdirectory examples/ seeded with a nice Vector
        Autoregression simulation simulation example by Lance Bachmeier
    o   Rewrote armadillo_version as to no longer require an instance 
        of arma::arma_version, with tanks to Conrad for the suggestion
And courtesy of CRANberries, here is the diff to the previous release.
Diff between RcppArmadillo versions 0.2.19 dated 2011-04-24 and 0.2.20 dated 2011-05-26
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/Col_proto.hpp               only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/Cube_proto.hpp              only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/GlueCube_proto.hpp          only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/Glue_proto.hpp              only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/Mat_proto.hpp               only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/OpCube_proto.hpp            only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/Op_proto.hpp                only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/Row_proto.hpp               only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/arma_ostream_proto.hpp      only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/arrayops_proto.hpp          only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/atlas_proto.hpp             only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/auxlib_proto.hpp            only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/blas_proto.hpp              only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/diagview_proto.hpp          only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/diskio_proto.hpp            only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/eGlueCube_proto.hpp         only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/eGlue_proto.hpp             only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/eOpCube_proto.hpp           only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/eOp_proto.hpp               only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/eglue_core_proto.hpp        only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/eop_core_proto.hpp          only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/field_proto.hpp             only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/fn_htrans.hpp               only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/forward_proto.hpp           only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/glue_conv_proto.hpp         only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/glue_cor_proto.hpp          only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/glue_cov_proto.hpp          only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/glue_cross_proto.hpp        only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/glue_join_proto.hpp         only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/glue_kron_proto.hpp         only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/glue_mixed_proto.hpp        only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/glue_relational_proto.hpp   only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/glue_solve_proto.hpp        only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/glue_times_proto.hpp        only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/glue_toeplitz_proto.hpp     only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/injector_proto.hpp          only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/lapack_proto.hpp            only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/mtGlueCube_proto.hpp        only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/mtGlue_proto.hpp            only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/mtOpCube_proto.hpp          only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/mtOp_proto.hpp              only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_chol_proto.hpp           only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_cor_proto.hpp            only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_cov_proto.hpp            only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_cumsum_proto.hpp         only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_cx_scalar_proto.hpp      only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_diagmat_proto.hpp        only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_diagvec_proto.hpp        only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_dot_proto.hpp            only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_dotext_proto.hpp         only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_find_proto.hpp           only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_flip_proto.hpp           only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_htrans_proto.hpp         only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_inv_proto.hpp            only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_max_proto.hpp            only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_mean_proto.hpp           only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_median_proto.hpp         only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_min_proto.hpp            only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_misc_proto.hpp           only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_pinv_proto.hpp           only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_princomp_cov_proto.hpp   only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_princomp_proto.hpp       only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_prod_proto.hpp           only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_relational_proto.hpp     only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_repmat_proto.hpp         only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_reshape_proto.hpp        only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_shuffle_proto.hpp        only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_sort_proto.hpp           only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_stddev_proto.hpp         only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_sum_proto.hpp            only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_trans_meat.hpp           only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_trans_proto.hpp          only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_trimat_proto.hpp         only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/op_var_proto.hpp            only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/podarray_proto.hpp          only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/running_stat_proto.hpp      only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/running_stat_vec_proto.hpp  only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/subview_cube_proto.hpp      only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/subview_elem1_proto.hpp     only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/subview_field_proto.hpp     only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/subview_proto.hpp           only
 RcppArmadillo-0.2.19/RcppArmadillo/inst/include/armadillo_bits/wall_clock_proto.hpp        only
 RcppArmadillo-0.2.20/RcppArmadillo/ChangeLog                                                  20 
 RcppArmadillo-0.2.20/RcppArmadillo/DESCRIPTION                                                10 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/NEWS                                                  26 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/examples                                           only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/RcppArmadillo/Mat_meat.h                       8 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo                                    199 -
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/Col_bones.hpp               only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/Col_meat.hpp                  519 +++
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/Cube_bones.hpp              only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/Cube_meat.hpp                  96 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/GlueCube_bones.hpp          only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/Glue_bones.hpp              only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/Mat_bones.hpp               only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/Mat_meat.hpp                  673 +++-
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/OpCube_bones.hpp            only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/Op_bones.hpp                only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/Proxy.hpp                      28 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/ProxyCube.hpp                  20 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/Row_bones.hpp               only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/Row_meat.hpp                  491 +++
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/arma_ostream_bones.hpp      only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/arma_ostream_meat.hpp          87 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/arma_static_assert.hpp         31 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/arma_version.hpp                6 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/arrayops_bones.hpp          only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/arrayops_meat.hpp             143 -
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/atlas_bones.hpp             only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/auxlib_bones.hpp            only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/auxlib_meat.hpp               772 ++++-
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/blas_bones.hpp              only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/compiler_setup.hpp             17 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/debug.hpp                      24 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/diagmat_proxy.hpp              93 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/diagview_bones.hpp          only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/diagview_meat.hpp             128 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/diskio_bones.hpp            only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/diskio_meat.hpp               104 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/eGlueCube_bones.hpp         only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/eGlueCube_meat.hpp             14 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/eGlue_bones.hpp             only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/eGlue_meat.hpp                 14 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/eOpCube_bones.hpp           only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/eOp_bones.hpp               only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/eglue_core_bones.hpp        only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/eglue_core_meat.hpp           519 ++-
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/eop_core_bones.hpp          only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/eop_core_meat.hpp             327 +-
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/field_bones.hpp             only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_accu.hpp                    40 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_det.hpp                      7 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_diagmat.hpp                  7 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_elem.hpp                    62 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_eye.hpp                  only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_inv.hpp                     18 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_log_det.hpp                  8 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_max.hpp                     38 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_mean.hpp                    42 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_median.hpp                  53 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_min.hpp                     38 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_ones.hpp                    45 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_prod.hpp                    20 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_randn.hpp                   10 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_randu.hpp                   12 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_stddev.hpp                  20 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_strans.hpp               only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_syl_lyap.hpp             only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_symmat.hpp               only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_trans.hpp                   35 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_var.hpp                     20 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/fn_zeros.hpp                   12 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/forward_bones.hpp           only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/gemm.hpp                      337 --
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/gemm_mixed.hpp                 59 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/gemv.hpp                      208 +
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/glue_conv_bones.hpp         only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/glue_conv_meat.hpp             16 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/glue_cor_bones.hpp          only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/glue_cor_meat.hpp              12 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/glue_cov_bones.hpp          only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/glue_cross_bones.hpp        only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/glue_cross_meat.hpp            47 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/glue_join_bones.hpp         only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/glue_join_meat.hpp             46 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/glue_kron_bones.hpp         only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/glue_mixed_bones.hpp        only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/glue_relational_bones.hpp   only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/glue_solve_bones.hpp        only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/glue_times_bones.hpp        only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/glue_times_meat.hpp           493 +--
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/glue_toeplitz_bones.hpp     only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/glue_toeplitz_meat.hpp          5 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/injector_bones.hpp          only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/lapack_bones.hpp            only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/mtGlueCube_bones.hpp        only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/mtGlue_bones.hpp            only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/mtOpCube_bones.hpp          only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/mtOp_bones.hpp              only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_chol_bones.hpp           only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_cor_bones.hpp            only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_cor_meat.hpp                14 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_cov_bones.hpp            only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_cumsum_bones.hpp         only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_cumsum_meat.hpp              9 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_cx_scalar_bones.hpp      only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_diagmat_bones.hpp        only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_diagmat_meat.hpp            10 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_diagvec_bones.hpp        only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_dot_bones.hpp            only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_dotext_bones.hpp         only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_find_bones.hpp           only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_flip_bones.hpp           only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_htrans_bones.hpp         only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_htrans_meat.hpp             98 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_inv_bones.hpp            only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_inv_meat.hpp                13 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_max_bones.hpp            only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_max_meat.hpp                57 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_mean_bones.hpp           only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_mean_meat.hpp               34 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_median_bones.hpp         only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_median_meat.hpp            214 -
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_min_bones.hpp            only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_min_meat.hpp                33 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_misc_bones.hpp           only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_pinv_bones.hpp           only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_pinv_meat.hpp               10 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_princomp_bones.hpp       only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_princomp_cov_bones.hpp   only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_prod_bones.hpp           only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_prod_meat.hpp               34 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_relational_bones.hpp     only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_repmat_bones.hpp         only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_repmat_meat.hpp             29 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_reshape_bones.hpp        only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_reshape_meat.hpp            32 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_shuffle_bones.hpp        only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_shuffle_meat.hpp            10 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_sort_bones.hpp           only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_stddev_bones.hpp         only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_stddev_meat.hpp             34 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_strans_bones.hpp         only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_strans_meat.hpp          only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_sum_bones.hpp            only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_sum_meat.hpp                38 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_symmat_bones.hpp         only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_symmat_meat.hpp          only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_trimat_bones.hpp         only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_trimat_meat.hpp            190 +
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_var_bones.hpp            only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/op_var_meat.hpp                37 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/operator_times.hpp             12 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/podarray_bones.hpp          only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/podarray_meat.hpp              57 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/promote_type.hpp               18 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/running_stat_bones.hpp      only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/running_stat_meat.hpp           2 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/running_stat_vec_bones.hpp  only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/subview_bones.hpp           only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/subview_cube_bones.hpp      only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/subview_cube_meat.hpp          50 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/subview_elem1_bones.hpp     only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/subview_elem1_meat.hpp         24 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/subview_field_bones.hpp     only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/subview_field_meat.hpp         37 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/subview_meat.hpp             1423 +++++++++-
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/traits.hpp                     69 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/typedef_u64.hpp             only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/undefine_conflicts.hpp      only
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/unwrap.hpp                     64 
 RcppArmadillo-0.2.20/RcppArmadillo/inst/include/armadillo_bits/wall_clock_bones.hpp        only
 RcppArmadillo-0.2.20/RcppArmadillo/src/RcppArmadillo.cpp                                      27 
 253 files changed, 6400 insertions(+), 2358 deletions(-)
More information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

25 May 2011

Matthew Garrett: Trials and tribulations with EFI

I wrote about some EFI implementation issues I'd seen on Macs a while back. Shortly afterwards we started seeing approximately identical bugs on some Intel reference platforms, and fixing it actually became more of a priority.

The fundamental problem is the same. We take the EFI memory map, identify the virtual addresses of the regions that will be required for runtime (mapping them into virtual address space if needed) and then call the firmware's SetVirtualAddressMap() implementation in order to let the firmware convert all its pointers. Sadly it seems that some firmware implementations call into sections of boot services code to do this, which is unfortunate because we've already taken that back to use as RAM. So, given that this is clearly against the spec, how does it ever work?

The tediously dull version is that Linux typically calls SetVirtualAddressMap() in the kernel, and everyone else does it in their bootloaders. The bootloader hasn't set up NX bits or anything, so it just happens to work there. We could just do it in the bootloader in Linux, but that makes doing things like kernel address space randomisation trickier, so it's not the favoured approach. So, instead, we can probably just reserve those ranges until after we've switched to virtual mode, and make sure the pages are executable. This ought to land in 2.6.40, or whatever it ends up being called.

(The alternative approach, of just never transitioning to physical mode, turns out to mysteriously fail on various machines. Calls to SetVariable() just give errors. We just don't know)

That still leaves the problem of SetVariable() on the test Mac trying to access a random address. That one turned out to be easier. There's 2MB of flash at the top of physical address space, and this was being presented as being broken into four separate EFI regions. While physically contiguous, Linux was mapping these to discontiguous virtual addresses. Apple's firmware appeared to assume that a pointer into one region could just be incremented into another. So because it's still easier to change the kernel than change Apple, 2.6.39 merges these regions to ensure they're contiguous.

Remaining problems include some machines seemingly not booting if they have 4GB of RAM or more and this Apple failing to communicate with its panel over the eDP auxchannel. Anyone got any idea how to dump the bios compatibility module out of a running EFI session?

( Comments Comment on this)

Russell Coker: Links May 2011

John W. Dean wrote in insightful series of three articles for Findlaw about Authoritarian Conservatives [1]. In summary there are Authoritarian Followers who follow their leader blindly and Authoritarian Leaders who do whatever it takes to gain and maintain power. The Authoritarian mindset lends itself towards right-wing politics. Mick Ebeling gave an inspiring TED talk about his work developing a system to produce art that is controlled by eye movements [2]. The development work was started to support the quadriplegic graffiti artist TEMPT1. Mick s most noteworthy point is that all the hardware design and software are free so anyone can implement it without asking an insurance company or hospital (this is one of the few occasions when a TED speaker has received a standing ovation during a talk). The Eyewriter.org site has the designs and source which is licensed under the GPL [3]. Morgan Spurlock (who is famous for Supersize Me ) gave an amusing TED talk titled The Greatest TED Talk Ever Sold [4]. He provides some interesting information about the brand sponsorship process and his new movie The Greatest Movie Ever Sold . Ralph Langner gave an interesting TED talk about reverse-engineering the Stuxnet worm and discovering that it was targetted at the Iranian nuclear program [5]. The fact that the Stuxnet environment could be turned to other uses such as disrupting power plants is a great concern, particularly as it has special code to prevent automatic safety systems from activating. Angela Belcher gave an interesting TED talk about using nature to grow batteries [6]. She is evolving and engineering viruses to manufacture parts of batteries and assemble them, the aim is to scale up the process to manufacture batteries for the Prius and other large devices at room temperature with no toxic materials. She is also working on biological methods of splitting water into hydrogen and oxygen which has the obvious potential for fuel-cell power and also solar PV cells. As an aside she mentions giving a copy of the Periodic Table to Barack Obama and he told her that he will look at it periodically . Bruce Schnier gave a good overview of the issues related to human perceptions of security in his TED talk about The Security Mirage [7]. There isn t much new in that for people who have been doing computer work but it s good to have an overview of lots of issues. TED has an interesting interview with Gerry Douglas about his work developing touch-screen computer systems for processing medical data in Malawi [8]. This is worth reading by everyone who is involved in software design, many of the things that he has done go against traditional design methods. Mike Matas gave an interesting demo at TED of the first proper digital book [9]. The book is by Al Gore and is run on the iPad/iPhone platform (hopefully they will have an Android version soon). His company is in the business of licensing software for creating digital books. The demonstration featured a mixture of pictures, video, audio, and maps with the pinch interface to move them around. Dr Sommers of Tufts University wrote an interesting post for Psychology Today titled Why it s Never About Race [10]. It seems that there are lots of patterns of people being treated differently on the basis of race but for every specific case no-one wants to believe that racial bias was involved. The Register has an amusing article about what might have happened if Kate had left Prince William at the altar [11]. Fiorenzo Omenetto gave an interesting TED talk about synthetic silk [12]. He is working on developing artificial fibers and solids based on the same proteins as silk which can be used for storing information (DVDs and holograms), medical implants (which can be re-absorbed into the body and which don t trigger an immune response), and cups among other things. Maybe my next tie will have a no pupae were harmed in the production notice. ;) The CDC has released a guide to preparing for a Zombie apocalypse [13], while it s unlikely that Zombies will attack, the same suggestions will help people prepare for the other medical emergencies that involve the CDC. Salon has an interesting article by Glenn Greenwald who interviewed Benjamin Ferencz about aggressive warfare [14]. Benjamin was a prosecutor for war crimes at Nuremberg after WW2 and compares the US actions since 9-11 with what was deemed to be illegal by the standards of WW2. Eli Pariser gave an interesting TED talk about Online Filter Bubbles [15]. He claims that services such as Facebook and Google should give more of a mixture of results rather than targetting for what people want. The problem with this idea is that presenting links that someone doesn t want to click doesn t do any good. It s not as if the filter bubble effect relies on modern media or can be easily solved. Terry Moore gave a TED talk about how to tie shoelaces [16]. Basically he advocates using a doubly-slipped Reef Knot instead of a doubly-slipped Granny Knot. Now I just need to figure out how to tie a doubly-slipped Reef Knot quickly and reliably. Terry uses this as a mathaphor for other ways in which one might habitually do something in a non-optimal way.

24 May 2011

Nathan Handler: Hello Planet Debian

Hello everyone. Per tradition, I will use my first post to Planet Debian as a bit of an introduction. My name is Nathan Handler. I am Ubuntu Developer and a member of freenode staff. Currently, I am participating as a student in the Google Summer of Code program with Debian. I am working on Backend Tools and Infrastructure for DEX (aka dextools ) with Matt Zimmerman as my mentor. To quote the project description:
DEX is a new program designed to help improve Debian and its derivatives by merging in changes made downstream and encouraging discussions between the various projects. As this is a new project, most of the infrastructure does not exist (or is rather hackish and incomplete). This project will create the necessary backend tools and infrastructure so that all Debian derivatives can easily make use of the DEX project.
While I have not been great with updating my blog in the past, I am going to try and produce regular posts about my Summer of Code work. If you have any questions about DEX or are interested in getting involved, please check out the Get Involved section on the website.

Russell Coker: Are Assholes Essential to a Free Software Project?

What do Assholes do? Rusty just wrote a post titled If you didn t run code written by assholes, your machine wouldn t boot [1] about some of the anti-social tendencies demonstrated by programmers, including some that are implied to be fairly important. His post seems to imply that there are really great programmers who are anti-social and that we should just put up with it because of their great code. One of the problems with his post is that he doesn t define asshole . Holocaust deniers and all other Nazi supporters seem to clearly by assholes by any definition. People who have silly ideas about medicine and tell others seem to be merely misguided (although my dictionary gives stupid or irritating or ridiculous in the definition of asshole so technically they can meet the criteria). In the comments Rusty states that nuttiness is relative . While that is correct it doesn t seem to have much bearing on whether someone is an asshole. For example I know some very nice people who are utterly convinced by creationism. Is Anyone Essential to Free Software? Most projects have one person or several people in leadership positions, usually there seems to be a combination of project management and lead-programmer positions. Such people are obviously quite important to a project. But we have seen examples of people dying, being employed by Microsoft, retiring, relicensing the source in a bad way, and just losing interest without the project ceasing. It seems clear that in most cases when a project which has a significant amount of use has significant bugs and no maintainer then someone will step in. The cases where no-one takes over the project are often resolved by someone creating a competing project. If no-one takes over an abandoned Free Software project then it s a strong indication that the project wasn t particularly important anyway. I have no doubt that if any bug or missing feature made Linux systems stop booting then someone would fix it quite quickly. In a more general sense it seems that every time someone takes a position in a project that is of wide interest they are displacing someone else who might have done the job. When you volunteer to do significant work for a project you may be displacing someone who is more skillful than you this isn t necessarily a bad thing as there are plenty of other projects to work on, some of which require more skill. Growing More Programmers It seems to me that a large part of becoming a great programmer is facing great challenges. People who could be described as optimistic or arrogant will tend to take on more challenging tasks and therefore learn more. I m sure that there are a lot of people out there who have the potential to be great programmers apart from not taking on the challenging work, this seems to be an unfortunate waste of talent. Given a large enough population if someone leaves a senior position there should be someone else who can obtain the skills needed to take over. One advantage of this for Free Software development is that even if the best person to take on the challenge isn t living in the most convenient continent that won t be an obstacle, while with proprietary software development projects the teams are small and it s common that no-one else is capable of stepping up to a lead position. Another advantage is that when the lead developer leaves there are generally many candidates to replace them, all of whom can start work and be judged on the work that they do. I think that the best claims that can be made regarding essential people are not in regard to technical skill alone but to a combination of technical and people skills. Getting a group of programmers to work together is really hard but it s something that needs to be done for any significant project. Also the larger projects tend not to stand alone, being able to get changes included in other projects requires some skill. Ben Collins-Sussman and Brian Fitzpatrick gave an insightful talk at Google IO 2008 titled How to Protect Your Open Source Project From Poisonous People [2]. The first half of their talk is mostly about people who are misguided or difficult rather than what most people would consider poisonous and the second half is more about people who are actively poisonous and need to be removed. They advocate a community based on Politeness, Respect, Trust, and Humility. They describe in detail how the methods they advocate result in the members of their community being more productive, it seems obvious that those principles will lead to better career growth for people within the community and more friendly people wanting to join. When is being an Asshole OK? I once worked for a company that apparently had a team consisting solely of assholes. Apparently one asshole got promoted to management and after some internal transfers they ended up with all the assholes in the company on one team. I guess that when someone has negative interactions with everyone they won t notice the difference if they are put in a team where everyone is difficult. For a corporate environment that lacks a no jerks hiring policy this is probably a good way of improving productivity overall. I am not aware of any significant Free Software project that was comprised of mostly jerks although I have seen a few with dysfunctional environments that encourage the worst behavior from their members. The smaller Free Software projects have less need for people who can relate to other people. There are many useful Free Software projects which have only one developer, in most cases anyone can take the source code and use it without dealing with the author. But even for a single-developer project an asshole can cause some serious problems. One example I know of concerns a developer who had unclear licenses and started making legal threats in response to a request for a clear license. Another example is of a developer who released code that was designed to not work when one particular user compiled it and redistributed the binaries. Both of them caused some significant amounts of time to be wasted by people who were unfortunate enough to develop systems that interacted with the code in question, and even more time was wasted when some misguided people defended them in the inevitable flame-wars. Even for a project with only one developer it s still better for everyone if that developer isn t an asshole. One comment I ve seen related to this issue suggesting that some types of asshole behavior shouldn t be a problem an example that was cited is a colleague who cheats on a romantic partner. Jeremy Clyman (who is currently doing a Ph.D in Psychology) has written an interesting article about this for Psychology Today [3]. He reviews the movie The Dilemma which deals with someone catching their colleague s wife cheating. Jeremy analyses the psychological issues involved and how they can (among other things) impact the ability for such people to work. I once worked in an office where two married employees were very open about having an affair and we were all apparently expected to lie on their behalf if necessary, it really affected the quality of the working environment. Extreme Assholes There are lots of people involved in Free Software development who are difficult and many who are to some extent assholes. But some of them take being an asshole to the extreme, such as Holocaust deniers (an example which Rusty used). In the comments on his post the Westboro Baptist Church is also mentioned. It is possible to entirely disagree with someone on a contentious issue such as abortion but still be able to get along with them. But when someone supports a hate-based organisation such as the WBC or supports Nazis in any way then there will be many people who just can t tolerate them and no-one should be expected to tolerate such people. I have seen two instances where Free Software developers advocated pro-Nazi positions (one had an archive of neo-Nazi propaganda and the other claimed that Nazis were not responsible for the Holocaust). Neither of the pro-Nazi programmers was evicted for defending Nazis, but both of them ended up leaving the community in adverse ways after causing other damage in the mean time. I don t think it takes any great ability to predict the future to determine that someone who defends Nazis will eventually end up doing something that requires expulsion and drive away users and developers in the mean-time. There is no possibility that someone can support the Nazi or WBC ideology only when not associated with your project, it will affect all aspects of their life. When a Holocaust denier is allowed to be a member of a community it also sends out a message that members of the groups which were persecuted by Nazis aren t particularly welcome in the community. Helping Minor Assholes There are a lot of people who don t have malevolent aims but who unintentionally cause some difficulty (it seems that the truly malicious are a tiny minority). I don t think that excusing the bad things that they do on the basis of writing good code helps them in the long term. Many of the suggestions that Ben Collins-Sussman and Brian Fitzpatrick make seem likely to help people who don t want to be assholes and direct them towards positive involvement in the community. One trend that seems apparent is the non-linear response to certain types of bad behavior. There is often little difference in severity between something that gets almost no attention and something that results in a large and extremely hostile reaction. If someone persists in acting like an asshole for long enough it seems to be inevitable that they will eventually exceed some threshold for what is tolerated and get a very significant negative response. It would be good if things didn t need to get to that stage. I think that the most unfortunate aspect of Rusty s blog post is that most people will probably interpret it as encouragement to write better code as a way of getting a free pass for being an asshole. I know that this isn t what Rusty intended, but most people on the Internet don t know Rusty as well as I do. Conclusion Ben Collins-Sussman and Brian Fitzpatrick seem to have some of the best ideas for how to deal with these issues when you control a project, but most of us aren t in that position. Everyone can advocate better behavior. Extreme assholes need to be removed quickly and without a great debate about their contributions, freedom of speech, or other issues. Since considering this issue I ve been wondering about when one should avoid the lesser assholes and asshole-positive environments. People tend to adapt to their environment, so if you associate with assholes a lot then there s a good chance you will start to become like them. Is

23 May 2011

Ben Hutchings: Testing new hardware support for Debian 6.0.2

The Debian kernel team regularly backports driver updates to the Linux kernel in stable releases to add support for new hardware, and I've prepared several updates intended for point release 6.0.2. Since the kernel team does not have a large collection of hardware on which to test driver changes, we would appreciate test reports from users. It is important to test not just that new devices are supported properly, but that there are no regressions in support for older devices. Changes I have updated these drivers to the versions found in Linux 2.6.38, modulo driver API changes: I have also cherry-picked some small changes: There are more drivers that I think should be added or updated (see #624794) but they will probably have to wait for release 6.0.3. Packages The source package and binary packages for i386 and amd64 are available on people.debian.org. They can be verified by the checksums in the signed changes file. The current packages are version 2.6.32-35~test1, but there may be further test versions before an official stable update. How to test For network drivers, I suggest the following regression tests:
  1. If the driver tries to load firmware (only required for some chips), does this work once the firmware file(s) are installed?
  2. Can you receive and transmit VLAN-tagged frames after creating a VLAN interface?
  3. Does the interface work after suspend and resume?
  4. Does the interface work after removing the cable for 10 seconds and reinserting it?
  5. Does multicast configuration work? (IPv6 autoconfiguration or mDNS will cover this.)
  6. Can the interface send and receive TCP/IP across a LAN at the same speed, before and after these changes? (Use e.g. netperf to test this, but don't forget to remove the netperf package after use.)
  7. Are any warnings or errors logged by the kernel during the preceding tests?
For storage drivers, unfortunately I don't have a good idea of what tests would be suitable. In any case, please don't test on disks storing valuable data! Please send test reports to the bug reports linked above, stating the driver name, the PCI ID for the device you tested (from lspci -n) and any other device identification that the kernel log (for example, r8169 logs the 'XID' of the device).

David Welton: Summary: Built To Sell - Creating A Business That Can Thrive Without You

<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="http://rcm.amazon.com/e/cm?lt1=_blank&amp;bc1=000000&amp;IS2=1&amp;bg1=FFFFFF&amp;fc1=000000&amp;lc1=0000FF&amp;t=dedasys-20&amp;o=1&amp;p=8&amp;l=as4&amp;m=amazon&amp;f=ifr&amp;ref=ss_til&amp;asins=B004IYISQW" style="width: 120px; height: 240px; float: left; margin-right: 20px; margin-bottom: 8px;"></iframe> Many small business owners start a business with the idea of greater freedom in mind, yet end up chaining themselves to something that takes even more dedication than a regular job. This book discusses the dos and don'ts that business owners need to be conscious of in order to create a business that is independent of its owner(s) in order to be able to sell it. It does so in an easy to read format that's a bit of a gimmick - a fictional story illustrating the authors points. Like most business books, the core idea isn't that hard to explain. the book even includes a handy summary of the main points, which I'll paraphrase here:
  1. Focus on one thing you can do really well; even a service business can be 'productized' by creating a standard service that is the focus of the business.
  2. If your business revolves around one or two key clients, the risk inherent in that approach will lower the value or scare of potential buyers completely.
  3. Put a process in place, from sales through production.
  4. Don't "be" your company. If the company is all about you, what's a potential buyer really getting if you sell it and leave?
  5. By focusing on products, you can charge up front rather than having poor cash flow. If you pay people to work on a project for 3 months, and don't get paid for another month, you are, by the end, out 4 months salary while waiting for the payment.
  6. Say no to projects outside the scope of your business. Only by focusing narrowly can you really excel at what you do.
  7. Spend time doing some research and calculations to estimate your potential market size; buyers will want to know this.
  8. If you have a business that has sales people, hire at least two so that they'll compete with one another.
  9. You want people who are good at selling products, not services; the latter will want to tweak your offering for each and every client, rather than selling it as-is and trying to find how it can meet the customer's needs that way, which is the best strategy for a product.
  10. If you were previously running a more 'generic' services company, and you switch to a more productized approach, be prepared to take a hit the year when you switch.
  11. Potential acquirers will want to see at least a couple of years of steady growth with the new model after making the switch.
  12. If you grow, you'll need a management team that can work without you. Put an incentive system into place to reward their loyalty and results.
  13. When looking for an adviser to sell your company, aim for one where you will not be the largest or smallest client.
  14. If your adviser really only has one company in mind to sell yours to, they may be trying to sell you off cheap as a favor.
  15. Consider how much you could grow with the resources of a buyer. Think big, and show them what kind of growth could happen with the right backing.
  16. Think and speak like a product business with 'customers' rather than 'clients'.
  17. Stock options are more complex than simpler options like bonuses that are paid out over a period of time, in terms of ways to give people an incentive to stay.
All in all, it was an enjoyable read, with solid points. Even if you have no intention of selling your business, thinking of the business itself as a sort of "product" that is not dependent on you to work correctly is a sensible way to go about creating and growing a business. He's certainly not, nor claims to be the first one to discuss this idea, and indeed references the well known E-Myth Revisited which focuses less on selling a business, and more on how to go about extricating you and your skills from the business.

Gregor Herrmann: RC bugs 2011/11 - 2011/20

long time no post about my RC bug fixing activities. & indeed, I haven't been very active during the last weeks. here's a quick update; it includes two NMUs done for the perl 5.12 transition, the rest is just "normal" maintainance work (interesting to see how much stuff can break behind our backs ...). I still hope to find some more time in the future to look at other RC bugs; after all we want to have a not-too-long freeze for wheezy, don't we?

Olivier Berger: I'm now a Debian Developer

As of today, I'm now obergix@debian.org. I've been a Debian contributor (more recently as Debian Maintainer) for quite a while, and after a few months of procedures, I've now been accepted as a registered Debian Developer, i.e. a Debian project's member. Thanks for all the good folks that have helped me doing so, by advocating, reviewing my application, or just inspiring me ;) Now, that's not an achievement, just a start, and I hope I'll be able to help improve Debian a bit.

John Goerzen: Baptism

Today I was baptized and joined the Mennonite Church. I imagine this might surprise some of you for various reasons, so let me provide a bit of background. I ve had a lot of people, both in person and in comments responding to my blog post, express surprise with statements I have made. The view of Christianity that many people have is of a group that devalues scientific inquiry and places a lot of emphasis on things like opposition to gay marriage, evolution, and abortion, and enjoys political leaders that say bring it on. While I know some Christians that fall into that mold, there are quite a few that don t as well. Some churches, such as the one we attend, have a surprisingly diverse set of people and yet still function and get along well. Today they accepted me with joy. Nobody was concerned that I started my statement with a reference to ancient Greek philosophy, wound up suggesting that the church ought to make sure to make illegal immigrants, gays and lesbians, and prostitutes feel welcome, and embraced both religion and scientific inquiry, feeling them complimentary. Mennonites practice adult baptism rather than infant baptism. A traditional age for people to be baptized is during high school, though emphasis is placed more on the individual than their age, so it s not unheard-of for someone to do so a bit later in life as I have. Part of the baptism involves the candidate sharing their faith story. These are typically intensely personal, widely varied, heartfelt, and honest. Some people s stories involve struggles with depression, physical disabilities, or the place of religion in their lives, while others reflect little struggle at all. Mine involves letting go of a lot of things, and also seeing some things, such as serious intellectual inquiry or existential questions about God (such as is there a god? ), as a positive rather than a negative feature of a Christian life. But it also involves a recognition and deep respect for those that don t approach things in this way. Though given publicly in a church, baptism testimonies are rarely published or shared more broadly. But I m going to share mine here. I have edited it only very lightly to remove a few local references that wouldn t make sense out of the context of this community. This was delivered in front of a rather different audience than is likely to be reading this post, so if you have questions, do feel free to ask in the comments. But before I begin, I just have to pass along a joke a friend told me: John, you picked quite a day for your baptism the day after the rapture. Baptism Testimony John Goerzen May 22, 2011 Those of you that know me well will probably not be surprised that I will begin my Christian baptism testimony with a story about an ancient Greek philosopher, and also touch on the philosophical nature of truth. These are key parts of my story. As legend goes, Socrates famously said, I know that I know nothing in other words, he believed that nothing could be known with absolute certainty. The Greek Oracle thought to be infallibly wise said that Socrates, the man that thought he knew nothing, was the wisest man in all Athens. An interesting paradox, and one that sheds light on my own religious story. My story involves coming to grips with the understanding that I know very little, that no person can ultimately know much about God, and finding a way to make peace with that situation. Growing up in this community, I thought I knew some things about faith and Christianity. At a certain age, it all seemed so simple. We took the Bible to be literally true. We marveled in Sunday School at how the apostles could sometimes appear so blind. And, modeled both implicitly and explicitly, was this notion: the stronger our faith, the fewer inner questions or doubt we have about the nature of God, the literal accuracy of the entire Bible, or our relationship to God. Those I perceived held up as examples never seemed to question any of these things, and showed outwardly, at least complete certainty about them. Moreover, holding certain intellectual beliefs was key to Christian identity, and even more importantly, to eternal life. This model has quite obviously worked well for many people for many years. The good that has happened, and continues to happen, from people that have that kind of certainty is manifest all around us. And yet, it didn t seem to work out for me. I m not the kind of person that accepts a lot of things at face value. It is helpful to be able to examine and challenge ideas and even more helpful to have other people challenge my ideas. From well before I was in high school, I was questioning some things about the Bible, God, and religion in general. My thoughts ranged from the impact of evolution on religion to the apparently vengeful God of the Old Testament to the very existence of God. In the Christian context, I perceived having these questions as a personal failing, something that I ought to repress. The more I tried to repress them, the more troublesome they became. Why, for instance, should a loving merciful God decide whether to let us into heaven based on whether we hold certain intellectual beliefs? In high school, I participated in the catechism class here at this church, and was frustrated because it didn t tackle deeper meanings or the kinds of questions I had. I wasn t yet able to articulate all my thoughts and questions very well, and I probably had an overactive case of teenage cynicism. As a result, I didn t get baptized like most others my age did. As I learned more about the early history of Judaism and Christianity, I only found more reasons to question the model of faith I thought I had received the one in which Biblical literalism and a divine guarantee of sorts was key. More recently, I gradually became aware that the model of Christianity I had in mind was one of many views. Christians, Mennonites, and even this church are incredibly diverse groups, and in retrospect, I am surprised that it took me so long to realize this. Three major steps led me to baptism. The first step was the realization that, whatever our understanding of the literal accuracy of the Bible, literal truth is often inferior to metaphorical truth. As an example, many of us have read the works of Shakespeare. They are fictional, but the reason they have been revered for so many centuries is that they are true. They teach us things about ourselves and our world in a way that no history book can. By placing such an emphasis on literal truth in the Bible, I was missing out on the message right there for me. By concerning myself with creationism or evolution, I missed out on thinking about what the story in Genesis meant for the Jews, and what it could mean for me. Evolution ceased to be a threat to religion; it became simply a tool for learning about a different sort of truth than we get from religion. N. T. Wright mentions an incident that illustrates this point. A woman attending a religious conference asked the speaker, a prominent theologian, Is it true that the serpent in Genesis actually spoke like you and I speak? The theologian answered, It doesn t matter whether the serpent could speak. What matters is what the serpent said. The second step I took towards being baptized was realizing where the real core of Christianity lies. It s not some debate about Genesis, but rather the death to one s old self, and the rebirth and continual remaking of oneself in the example of Jesus. Given that, a lot of questions seem unimportant or even irrelevant. Good can be expressed in many ways, and if one person achieves a remarkably Christian life via a literal understanding of the Bible, and another via a more metaphorical reading, then it is my place only to affirm both and say that they both got it right. If we say that the task of remaking ourselves is like climbing a mountain, then what matters is not how we are climbing the mountain, but simply that we are climbing it. I used to equate faith with an intellectual belief. I have come to see that was a narrow view. Faith, to me now, is more about vision: do we see the world like Christ did? Where do our loyalties and our trust lie in God or in human institutions? What are our goals in life? We humans have failed to understand God, and probably always will. I too share in that incomplete understanding, but I have come to accept that it is OK. I know enough to know that I want my loyalty to lie in Jesus, to know what kind of vision of the world I want to have, and I have learned to accept that intellectual questions can even be a form of meditation, enlightenment, and prayer. The third step toward baptism was moving past my own shortcomings. For a long time, I thought I didn t believe the right things, didn t believe them strong enough, wasn t certain enough about God, didn t pray enough, didn t read enough, didn t understand enough, didn t love enough and ultimately, that I wasn t good enough. I request baptism today understanding that, despite the various imperfections I still have, and we all have, everyone is good enough and deserving of love and peace. God s love is for everyone. No exceptions! Following Jesus boils down to this: I too must be an agent of love and peace to everyone, without exception. My vision must be centered around the fact that we are to emulate the God that loves the entire world, sinners all, and therefore I should as well. I hope I can show others the kind of love that has been shown me. I hope, too, to share with this congregation and the global church in the vision of love. I hope that we can continually strive to re-focus on Christ s vision. As an example, we all know that many different viewpoints about whether homosexuality is sinful exist in Mennonite Church USA. These opinions are deeply held and personal for many, and have been discussed over and over and over. But ultimately, they aren t terribly relevant to the church s mission. The example of Jesus unites us all: he embraced everyone. He accepted criminals and prostitutes and showed them love and kindness. Our difficult task, which is also my difficult task, is to show this same love to absolutely everyone, regardless of our varying opinions about them and their conduct. The forgotten and repressed of our day perhaps criminals, homeless, Muslims, addicts, gays and lesbians, illegal immigrants, ethnic minorities, and still prostitutes deserve the love of Christ s church and all its members. My hope is that any one of the earth s 7 billion people could step through the doors of our church, or any church, and immediately feel Christ s love, and the unconditional, non-judgmental, welcome and love of Christ s followers, no matter what. This prayer from Ephesians 3 sums up my hope and my vision, for myself and everyone:
I pray that out of his glorious riches he may strengthen you with power through his Spirit in your inner being. I pray that Christ may dwell in your hearts through faith. And I pray that you, being rooted and established in love, may have power, together with all the Lord s people, to grasp how wide and long and high and deep is the love of Christ. And I pray that you will know this love that surpasses knowledge that you may be filled to the measure of all the fullness of God. (3.16-19)

21 May 2011

Vincent Bernat: Network lab with User Mode Linux

To setup a virtual network lab, you get a lot of alternatives. You can for example use GNS3, Netkit, Marionnet or VNUML. Some of those tools, like GNS3, allow to add some closed source equipments like a Cisco 7200 or a Juniper router. All those tools are a great way to setup your network lab. Look at them! If you want to setup a virtual network lab for educational purpose, one of those should fit your purpose. However, none of those solution were a perfect match for me. I did not want to maintain some root filesystem. I wanted my lab to start in a few seconds. I wanted to keep all configuration files (including the ones for the virtual hosts) into one subdirectory of my home and be able to modify them while the lab was running. I also wanted to be able to plug some Cisco router using Dynamips/Dynagen. None of the listed solution above matched all those criteria. Therefore, I setup my own lab script with User Mode Linux. This is not a complete solution, but is more like a home-made solution to match one particular need. You cannot use the final result without tweaking it. Again, look at the other solutions first. We will go step by step to understand how the lab was built. If you are in a hurry, you can look at the result which is published on GitHub.

User Mode Linux User Mode Linux (or UML) is a Linux kernel running as a process instead of running on top of some processor. With Debian, you can install it using apt-get install user-mode-linux. You get a linux command that will run your Linux kernel as a simple process. No need to be root.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
$ linux
Core dump limits :
    soft - 0
    hard - NONE
Checking that ptrace can change system call numbers...OK
Checking syscall emulation patch for ptrace...OK
Checking advanced syscall emulation patch for ptrace...OK
Checking for tmpfs mount on /dev/shm...nothing mounted on /dev/shm
Checking PROT_EXEC mmap in /tmp/user/500/...OK
Checking for the skas3 patch in the host:
  - /proc/mm...not found: No such file or directory
  - PTRACE_FAULTINFO...not found
  - PTRACE_LDT...not found
UML running in SKAS0 mode
Adding 5632000 bytes to physical memory to account for exec-shield gap
Initializing cgroup subsys cpuset
Linux version 2.6.32 (2.6.32) (root@tito) (gcc version 4.4.5 (Debian 4.4.5-10) ) #2 Thu Jan 27 12:49:46 UTC 2011
[...]
console [mc-1] enabled
Couldn't stat "root_fs" : err = 2
Failed to initialize ubd device 0 :Couldn't determine size of device's file
registered taskstats version 1
VFS: Cannot open root device "98:0" or unknown-block(98,0)
Please append a correct "root=" boot option; here are the available partitions:
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(98,0)
You get a Linux kernel that will just panic because it does not find a disk to mount to run init from it. Usually, you build some root file system and give it as an argument to your UML kernel.

Basic setup However, you can also use your host filesystem as a root filesystem. We also specify /bin/sh as init instead of /sbin/init. This will be the first process to be started by our UML kernel.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
$ linux init=/bin/sh rootfstype=hostfs
[...]
Linux version 2.6.32 (2.6.32) (root@tito) (gcc version 4.4.5 (Debian 4.4.5-10) ) #2 Thu Jan 27 12:49:46 UTC 2011
[...]
VFS: Mounted root (hostfs filesystem) readonly on device 0:12.
IRQ 3/console-write: IRQF_DISABLED is not guaranteed on shared IRQs
IRQ 2/console: IRQF_DISABLED is not guaranteed on shared IRQs
IRQ 10/winch: IRQF_DISABLED is not guaranteed on shared IRQs
/bin/sh: can't access tty; job control turned off
# uname -a
Linux (none) 2.6.32 #2 Thu Jan 27 12:49:46 UTC 2011 x86_64 GNU/Linux
# echo $$
1
You may want to setup a bit your environment:
1
2
3
4
5
6
7
8
9
# hostname -b R1
# export TERM=xterm
# export PATH=/usr/local/bin:/usr/bin:/bin:/sbin:/usr/local/sbin:/usr/sbin
# mount -t proc proc /proc
# mount -t sysfs sysfs /sys
# mount -t tmpfs tmpfs /var/run -o rw,nosuid,nodev
# mount -t tmpfs tmpfs /var/log -o rw,nosuid,nodev
# mount -o bind /usr/lib/uml/modules /lib/modules
# mount -t hostfs hostfs /home/bernat/mylab -o /home/bernat/mylab
We setup /proc and /sys correctly. We also turn /var/run and /var/log as tmpfs filesystems. This will allow most daemons to run correctly: they will be able to write runtime data in /var/run and to write logs in /var/log. You can even launch your favorite syslog daemon. The line about /lib/modules is to ensure that our UML kernel is able to access its kernel modules. For example, we can now use modprobe tun. The line about /home allows to mount the home directory of your host inside UML. This was already the case, but it was mounted as read-only. One important thing to notice is that while you seem to be root inside UML, you did not run linux as root. Therefore, when the UML kernel tries to write a file it will still be bound by the permissions granted by your host kernel. See this example:
1
2
3
4
# mount -o remount,rw /
# touch /tmp/test1
# touch /etc/test1
touch: cannot touch  /etc/test1': Permission denied
This is a great way to ensure that your lab will not destroy your host. Do not run linux as root. What is interesting with such a setup is that you share the filesystem with your host. You can modify some configuration file on your host and your UML will see the modification right away. You can also modify a file inside UML and see the modification on your host too.

Console & job control There is a slight problem with this environment. You may be used to it if you already started your own GNU/Linux system with init=/bin/sh. The problem is that you don t have job control. Job control allows you to put tasks in background and interrupt them while they are running. It is job control that allows you to stop a program with ^C. Here, you don t have job control. See:
1
2
# cat
^C^C^C^C
You are stuck. The only way to go out is to kill the linux process on your host. Job control is a great mystery for me. I don t know how to enable it by hand. However, getty is able to enable job control for you. Just type this command:
1
# exec getty -n -l /bin/sh 38400 /dev/tty0
You get job control. We keep /bin/sh, but you may prefer /bin/bash since on most systems, /bin/sh is rather basic.

Writable root filesystem Let s get back to our root filesystem. Most daemons can run just fine with our setup. For example, we can start Nginx, a popular web server:
1
2
3
# mkdir /var/log/nginx
# /etc/init.d/nginx start
Starting nginx: nginx.
However, it will use the configuration file in /etc/nginx. You don t want to modify this directory for all your labs. The best way to handle this is to start Nginx with a custom configuration file:
1
2
# /etc/init.d/nginx stop
# nginx -c /home/bernat/mylab/nginx.conf
While keeping the configuration files related to your lab in your home is an important objective (all your lab related files are in the same place and you can modify them at will), it may be more convenient to still use /etc/init.d/nginx script to control Nginx. Moreover, some daemons do not allow to specify a configuration file. You will then need to modify your root filesystem. This is a pretty common problem with Live CD and it is solved by using an union mount. This is a special kind of mount that will merge two filesystems into one. One of the two filesystems can be read-only and all the changes will be reported to the second one. We will use AUFS. Ensure that you have aufs-tools package installed. Let s start from scratch for this example.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
$ linux init=/bin/sh rootfstype=hostfs
[...]
# mount -n -t proc proc /proc
# mount -n -t sysfs sysfs /sys
# mount -o bind /usr/lib/uml/modules /lib/modules
# mount -n -t tmpfs tmpfs /tmp -o rw,nosuid,nodev
# mkdir /tmp/ro /tmp/rw /tmp/aufs
# mount -n -t hostfs hostfs /tmp/ro -o /,ro
# mount -n -t aufs aufs /tmp/aufs -o noatime,dirs=/tmp/rw:/tmp/ro=ro
# exec chroot /tmp/aufs /bin/bash
It seems that AUFS with hostfs is rather fragile. We could have merged our root filesystem with some directory in our lab using hostfs like this:
1
2
3
4
# mount -n -t hostfs hostfs /tmp/rw -o /home/bernat/mylab/root,rw
# mount -n -t hostfs hostfs /tmp/ro -o /,ro
# mount -n -t aufs aufs /tmp/aufs -o noatime,dirs=/tmp/rw:/tmp/ro=ro
# exec chroot /tmp/aufs /bin/bash
However, the kernel panics on most operations. Therefore, /tmp/rw stays in a tmpfs filesystem. This means that any change to the root filesystem will be lost. I also was unable to issue a correct pivot_root before chroot. This is not really important here. UPDATED: It seems that there are other drawbacks to this setup. For some reason, if you have a separate /usr partition on your host system, you won t be able to write to it until you specify the noxino option to AUFS. Another issue is that AUFS does not notice most changes on the hostfs filesystem. You can fix this with option udba=inotify. However, this is incompatible with the noxino option. Therefore, you need to choose what is best for your lab. You can also mount some parts of your home with hostfs (for example, the source directory of the piece of software that you would like to test). Let s consider that Nginx configuration is in /home/bernat/lab/nginx. We simply uses a symbolic link:
1
2
# rm -rf /etc/nginx
# ln -s /home/bernat/mylab/nginx /etc/nginx
Since we lose the modifications to the root filesystem, the symbolic link should be done each time we start the lab. However, we can modify directly the configuration files in the host or in the UML.

Networking We know how to start our UML. Let s see how we can add some wires to it. UML supports several network adapters. The two most useful are TAP and VDE backend.

TAP A TAP interface is a virtual network kernel device which can be used by userland applications to inject layer 2 frames in it. If the kernel sends frames in a TAP device, they will be received by the application listening to it. This also works the other way around. A TAP interface is a regular ethernet device for the kernel. This means that you can bridge it or use tcpdump on it. The drawback of such interfaces is that you need to be root to create them. The application using them does not need to be run as root. I usually use this nifty shell function to setup TAP interfaces:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
__add_to_bridge()  
    # Optionally, add it to given bridge
    [ -z "$2" ]    
        [ -f /sys/class/net/$2/brforward ]    
            sudo brctl addbr $2
            sudo brctl stp $2 off
            sudo ip link set $2 up
         
        [ -f /sys/class/net/$2/brif/$1 ]    
            # We need to check if it is in another bridge
            bridge=$(echo /sys/class/net/*/brif/$1 2> /dev/null   \
                sed 's+/sys/class/net/\([^/]*\)/.*+\1+') 2> /dev/null
            [ -n "$bridge" ] && \
                sudo brctl delif $bridge $1
            sudo brctl addif $2 $1
         
     
 
tap()  
    sudo tunctl -b -u $(whoami) -t $1 > /dev/null
    sudo ip link set up dev $1
    __add_to_bridge $1 $2
 
It will create the interface and put it into a bridge. For example, you can create two interfaces linked together through a bridge to allow two UML to talk each other:
1
2
$ tap tap-R1 br-R1R2
$ tap tap-R2 br-R1R2
We can setup two UML which will use those interfaces. The first one is setup like this:
1
2
3
4
$ linux init=/bin/sh rootfstype=hostfs eth0=tuntap,tap-R1
[...]
# ip link set up dev eth0
# ip addr add 192.168.0.1/24 dev eth0
The second one:
1
2
3
4
5
6
7
8
9
$ linux init=/bin/sh rootfstype=hostfs eth0=tuntap,tap-R2
[...]
# ip link set up dev eth0
# ip addr add 192.168.0.2/24 dev eth0
# ping 192.168.0.1
PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data.
Warning: time of day goes back (-13832us), taking countermeasures.
64 bytes from 192.168.0.1: icmp_req=20 ttl=64 time=0.633 ms
64 bytes from 192.168.0.1: icmp_req=21 ttl=64 time=0.157 ms
If you are running a firewall on your host, ensure that you let your host forward those packets. By default, bridging code forwards the frames through Netfilter.

VDE A VDE switch is a software emulation of a regular network switch. It is a userland component and does not need to be run as root. You need to install vde2 package to make use of it. You can run a switch using vde_switch command. You will get a console by pressing Enter. From this console, you can configure the switch: add ports, configure VLAN, etc. Like for TAP interfaces, let s setup two UML connected to this switch. The first one is like this:
1
2
3
4
$ linux init=/bin/sh rootfstype=hostfs eth0=vde
[...]
# ip link set up dev eth0
# ip addr add 192.168.0.1/24 dev eth0
And the second one:
1
2
3
4
5
6
7
8
9
$ linux init=/bin/sh rootfstype=hostfs eth0=vde
[...]
# ip link set up dev eth0
# ip addr add 192.168.0.2/24 dev eth0
# ping 192.168.0.1
PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data.
64 bytes from 192.168.0.1: icmp_req=1 ttl=64 time=2.93 ms
Warning: time of day goes back (-8101us), taking countermeasures.
64 bytes from 192.168.0.1: icmp_req=2 ttl=64 time=0.503 ms

The lab So far, we had a look at what can be used to setup a lab. We need to put everything together into some script to setup the lab in a quick way. For each lab that I want to setup, I copy my latest script to a new location and adapt to the topology I want. I have put everything described below in GitHub. Feel free to browse and fork. Our lab will be about setting up a redundant VPN between two remote sites of the same company. This company is using OSPF as an IGP on each site and we will use BGP to let each site know the routes advertised in the other site. We will only use VDE switches for the layer 2 part. Redundant VPN lab Let s implement this lab. You should grab the sources first:
1
2
$ git clone https://github.com/vincentbernat/network-lab.git
$ cd network-lab/lab-redundant-vpn
You have a setup script which will setup the whole lab for you. You also have one directory for each UML. Those directories only contain configuration for the various daemons used inside UML. To make this lab work, ensure that you have quagga (Quagga, an Internet routing daemon supporting both OSPF and BGP) and racoon packages.

UML configuration First look at the end of script. There is something like this:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
case $$ in
    1)
        # Inside UML. Three states:
        [...]
        ;;
    *)
        TMP=$(mktemp -d)
        trap "rm -rf $TMP" EXIT
        check_dependencies
        setup_screen
        # Setup switches
        setup_switch site1
        setup_switch site101
        setup_switch internet
        # Start VM
        start_vm R1 eth0=vde,$TMP/switch-site1.sock
        start_vm R2 eth0=vde,$TMP/switch-site101.sock
        start_vm V1 eth0=vde,$TMP/switch-site1.sock eth1=vde,$TMP/switch-internet.sock
        start_vm V2 eth0=vde,$TMP/switch-site1.sock eth1=vde,$TMP/switch-internet.sock
        start_vm V3 eth0=vde,$TMP/switch-site101.sock eth1=vde,$TMP/switch-internet.sock
        start_vm V4 eth0=vde,$TMP/switch-site101.sock eth1=vde,$TMP/switch-internet.sock
        start_vm I1 eth0=vde,$TMP/switch-internet.sock
        display_help
        cleanup
        screen -X quit
        ;;
esac
The script will check its PID ($$). If it is 1, this means it has been invoked as init. Otherwise, it means it has been launched by the user. We will see the init part later. When called, we first check some dependencies, then we setup screen. Everything is run inside a screen session. No multiple windows. The next step is to setup the various switches needed for our lab. We take one switch for each site and one switch for Internet . The function that setups the switch just invokes vde_switch with the help of start-stop-daemon to record the PID (to shutdown the lab properly). Then, we start our various UML. For each of them, we need to specify which network adapters we need. For R1 and R2, we don t use a network adapter to connect to the internal network. We will use a dummy0 interface. To emulate Internet, we use another UML, I1. For each UML, our script will be used as init. In this case, its PID will be 1. As we have seen earlier, the script checks if it is called as init. If it is the case, it will setup the UML. During this setup, host specific configuration will be applied:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
echo "[+] Setup UML"
sysctl -w net.ipv4.ip_forward=1
case $ uts  in
    R1)
        modprobe dummy
        ip link set up dev dummy0
        ip addr add 192.168.15.1/24 dev dummy0
        ip addr add 192.168.1.10/24 dev eth0
        setup_quagga
        ;;
    R2)
        modprobe dummy
        ip link set up dev dummy0
        ip addr add 192.168.115.1/24 dev dummy0
        ip addr add 192.168.101.10/24 dev eth0
        setup_quagga
        ;;
    V1)
        ip addr add 192.168.1.11/24 dev eth0
        ip addr add 1.1.2.1/24 dev eth1
        ip route add default via 1.1.2.10
        setup_quagga
        setup_racoon 1.1.2.1 1.1.1.1 192.168.0.0/19-192.168.100.0/19
        ;;
    V2)
        ip addr add 192.168.1.12/24 dev eth0
        ip addr add 1.1.2.2/24 dev eth1
        ip route add default via 1.1.2.10
        setup_quagga
        setup_racoon 1.1.2.2 1.1.1.2 192.168.0.0/19-192.168.100.0/19
        ;;
    V3)
        ip addr add 192.168.101.13/24 dev eth0
        ip addr add 1.1.1.1/24 dev eth1
        ip route add default via 1.1.1.10
        setup_quagga
        setup_racoon 1.1.1.1 1.1.2.1 192.168.100.0/19-192.168.0.0/19
        ;;
    V4)
        ip addr add 192.168.101.14/24 dev eth0
        ip addr add 1.1.1.2/24 dev eth1
        ip route add default via 1.1.1.10
        setup_quagga
        setup_racoon 1.1.1.2 1.1.2.2 192.168.100.0/19-192.168.0.0/19
        ;;
    I1)
        ip addr add 1.1.1.10/24 dev eth0
        ip addr add 1.1.2.10/24 dev eth0
        ;;
esac
After all this, we drop to a shell to allow you to issue additional commands interactively.

Testing We need to wait one minute after the lab has been started. We can then check the routing table of various equipments. For example, here is R1 routing table (you need to type vtysh to get Quagga console):
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
R1# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
       I - ISIS, B - BGP, > - selected route, * - FIB route
C>* 127.0.0.0/8 is directly connected, lo
O   192.168.1.0/24 [110/10] is directly connected, eth0, 00:25:42
C>* 192.168.1.0/24 is directly connected, eth0
O   192.168.15.0/24 [110/10] is directly connected, dummy0, 00:25:42
C>* 192.168.15.0/24 is directly connected, dummy0
O>* 192.168.115.0/24 [110/20] via 192.168.1.11, eth0, 00:02:24
  *                           via 192.168.1.12, eth0, 00:02:24
We can see that if this router wants to contact R2, it has learned a multipath route with OSPF and can then sends its packets through V1 or V2, as expected. Here is the routing table of V3:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
V3# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
       I - ISIS, B - BGP, > - selected route, * - FIB route
K>* 0.0.0.0/0 via 1.1.1.10, eth1
C>* 1.1.1.0/24 is directly connected, eth1
C>* 127.0.0.0/8 is directly connected, lo
O   192.168.15.0/24 [110/20] via 192.168.101.14, eth0, 00:04:26
B>* 192.168.15.0/24 [20/20] via 192.168.1.11 (recursive via 1.1.1.10), 00:07:16
O   192.168.101.0/24 [110/10] is directly connected, eth0, 00:07:30
C>* 192.168.101.0/24 is directly connected, eth0
O>* 192.168.115.0/24 [110/20] via 192.168.101.10, eth0, 00:07:18
We see that to contact R1, V3 knows two routes. One with BGP through the VPN. This is the one that will be used. The other one is through V4 but will not be used unless the BGP route is lost. We can ping R2 from R1:
1
2
3
4
R1# ping -I 192.168.15.1 192.168.115.1
PING 192.168.115.1 (192.168.115.1) from 192.168.15.1 : 56(84) bytes of data.
Warning: time of day goes back (-459129us), taking countermeasures.
64 bytes from 192.168.115.1: icmp_req=1 ttl=62 time=1.10 ms
Let s suppose we break the link between V1 and the Internet (using ip link set down eth1 on V1). The previous multipath route on R1 is now a single route through V2. Moreover, V3 has lost its BGP route to V1 and is now using its OSPF route through V4:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
V3# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
       I - ISIS, B - BGP, > - selected route, * - FIB route
K>* 0.0.0.0/0 via 1.1.1.10, eth1
C>* 1.1.1.0/24 is directly connected, eth1
C>* 127.0.0.0/8 is directly connected, lo
O>* 192.168.15.0/24 [110/20] via 192.168.101.14, eth0, 00:10:13
O   192.168.101.0/24 [110/10] is directly connected, eth0, 00:13:17
C>* 192.168.101.0/24 is directly connected, eth0
O>* 192.168.115.0/24 [110/20] via 192.168.101.10, eth0, 00:13:05
We could enhance the resiliance of such a setup by adding a dedicated link between V1 and V2 and between V3 and V4 and running OSPF on this link (or iBGP). This would allow us to support several outages on our network.

Conclusion The script used to setup this lab can be adapted to setup other labs with little effort. Please, note that it depends on the configuration of your environment. However, the whole lab is contained in a single directory and is very small. You can email it to your friends or publish it on GitHub. It can also be extended to support Dynagen to include some Cisco equipments in your lab (for example, to experiment with VRF or MPLS).

Craig Small: JFFNMS 0.9.0 Released

After 3 release candidates jffnms is now at version 0.9.0. Both the web frontend and backend (engines) have had extensive re-work done to them to cleanup and tighten up the code. There should be a lot less warnings and errors from PHP when you set to a higher error level.

20 May 2011

Tanguy Ortolo: cpio > tar

Large package icon tar and cpio tar(5) and cpio(5) are two competing archive formats, that provide almost identical features. Both are streamed formats, originally designed to be used on tapes. Their major practical difference is the style of their standard command line utilities: Peace! There is a traditional opposition between these two formats and utilities and their respective supporters. To fix this, the POSIX specification deprecated both commands in 2001, replacing them by a single one called pax(1), that can be used either in cpio-like or tar-like style. This specification also defined a pax interchange format, which these vile trolls derived from tar! Now, considering the actual practice, you can notice that tar has won, and that almost nobody uses pax. Almost every free software project I know release tarballs, not cpio archives. This is probably the reason why the GNU tar utility has evolved much more than GNU cpio: tar now integrates seamlessly with compressors such as gzip, bzip2 or xz, autodetecting the compression format when needed whereas cpio has no such features as far as I know. However, cpio is notably used by the Linux kernel for its initial memory file system, aka initrd. tar and cpio drawbacks Now I come to the title of this article: I personally prefer the cpio format over tar, although I usually use tar because of the advanced features of GNU tar. In fact, tar suffers from a drawback that usually has to be worked around: it cannot store any type of file. There exist seven file types: regular files, directories, symbolic links, character devices, block devices, FIFOs and Unix sockets. Well, tar can only represent six of them, and misses the ability to store sockets. Thus, if you try to archive your entire filesystem with tar, you may get the following output:
tar: /var/spool/postfix/private/rewrite: socket ignored
tar: /var/spool/postfix/private/bsmtp: socket ignored
If you try the same with cpio, you will see that sockets get archived without any problem. Given that the tar format has a type byte, with one reserved value, I am surprised that its 2001 revision did not use it for sockets, but for some reason it did not. In practice, programs that use sockets work around this problem by creating them at startup. To be honest, I must mention that cpio also has one significant drawback: when a file has several links, it gets copied several times in the archive. Edit: It appears that archiving sockets is completely pointless, given that they must be unlinked if they exist, and then created before they are used by a program. Thus, the question becomes: why does cpio support sockets?

19 May 2011

Mark Brown: ASoC updates in 2.6.39

Linux 2.6.39 was released earlier today. This release includes a few updates, the main user visible one being that machine drivers can now be registered as regular devices rather than using the soc-audio device.

18 May 2011

Sandro Tosi: I'm going to EuroPython 2011

I just got confirmation my company will sponsor me for EuroPython 2011 (thanks Register.it), so I'll be able to attend the whole week; a lot of amazing talks and the code sprints in the weekend: this is going to be a great time!

Are you coming?

17 May 2011

Gerfried Fuchs: It's MY Life

Sometimes people will tell you what you should do. Sometimes they will even shout at you for simply asking a question on why they want something done because it isn't clear just from itself. And others likes to jump the boat and join in just for the fun of it... Gladly, this is MY life, and I choose how much abuse I'm willing to take, especially for a voluntary work that I didn't even enroll for but got put into. Sometimes through my dedication to getting quality into things and seeing that others simply neglect these areas, but they need to get addressed anyway, no matter how little respect is shown for people investing in these boring areas. The topic of It's MY Life is an old one and thus it is no surprise that a fair amount of songs surrounding it popped up over time. In my previous blog entry I wrote about different interpretations, some responses seem to hint that I wasn't clear enough about that I really meant different interpretations of the same lyrics, not just regular cover versions. The following set of songs is special in a different sense: It is about the same song title and thus does also cover different bands. Like always, enjoy! And think about how you interact with others. I know that I'm sometimes crossing a line myself too, no one is perfect. What though makes the difference is the willingness to learn, and especially: To excuse. But in the end: It's MY life!

/music permanent link Comments: 3 Flattr this

Next.

Previous.