const char *foo = "foo";

This was recently mentioned on bugzilla, and the problem is usually underestimated, so I thought I would give some details about what is wrong with the code above. The first common mistake here is to believe foo is a constant. It is a pointer to a constant. In practical ELF terms, this means the pointer lives in the .data section, and the string constant in .rodata. The following code defines a constant pointer to a constant:

const char * const foo = "foo";

The above code will put both the pointer and the string constant in .rodata. But keeping a constant pointer to a constant string is pointless. In the above examples, the string itself is 4 bytes (3 characters and a zero termination). On 32-bits architectures, a pointer is 4 bytes, so storing the pointer and the string takes 8 bytes. A 100% overhead. On 64-bits architectures, a pointer is 8 bytes, putting the total weight at 12 bytes, a 200% overhead. The overhead is always the same size, though, so the longer the string, the smaller the overhead, relatively to the string size. But there is another, not well known, hidden overhead: relocations. When loading a library in memory, its base address varies depending on how many other libraries were loaded beforehand, or depending on the use of address space layout randomization (ASLR). This also applies to programs built as position independent executables (PIE). For pointers embedded in the library or program image to point to the appropriate place, they need to be adjusted to the base address where the program or library is loaded. This process is called relocation. The relocation process requires information which is stored in .rel.* or .rela.* ELF sections. Each pointer needs one relocation. The relocation overhead varies depending on the relocation type and the architecture. REL relocations use 2 words, and RELA relocations use 3 words, where a word is 4 bytes on 32-bits architectures and 8 bytes on 64-bits architectures. On x86 and ARM, to mention the most popular 32-bits architectures nowadays, REL relocations are used, which makes a relocation weigh 8 bytes. This puts the pointer overhead for our example string to 12 bytes, or 300% of the string size. On x86-64, RELA relocations are used, making a relocation weigh 24 bytes! This puts the pointer overhead for our example string to 32 bytes, or 800% of the string size! Another hidden cost of using a pointer to a constant is that every time it is used in the code, there will be pointer dereference. A function as simple as

int bar() return foo;

weighs one instruction more when foo is defined const char *. On x86, that instruction weighs 2 bytes. On x86-64, 3 bytes. On ARM, 4 bytes (or 2 in Thumb). That weight can vary depending on the additional instructions required, but you get the idea: using a pointer to a constant also adds overhead to the code, both in time and space. Also, if the string is defined as a constant instead of being used as a literal in the code, chances are it s used several times, multiplying the number of such instructions. Update: Note that in the case of const char * const, the compiler will optimize these instruction and avoid reading the pointer, since it s never going to change. The symbol for foo is also exported, making it available from other libraries or programs, which might not be required, but also adds its own overhead: an entry in the symbols table (5 words), an entry in the string table for the symbol name (strlen("foo") + 1) and an entry in the symbols hash chain table (4 bytes if only one type of hash table (sysv or GNU) is present, 8 if both are present), and possibly an entry in the symbols hash bucket table, depending on the other exported symbols (4 or 8 bytes, as chain table). It can also affect the size of the bloom filter table in the GNU symbol hash table. So here we are, with a seemingly tiny 3 character string possibly taking 64 bytes or more! Now imagine what happens when you have an array of such tiny strings. This also doesn t only apply to strings, it applies to any kind of global pointer to constants. In conclusion, using a definition like

const char *foo = "foo";

is almost never what you want. Instead, you want to use one of the following forms:

• For a string meant to be exported:
  

  const char foo[] = "foo";

• For a string meant to be used in the same source file:
  

  static const char foo[] = "foo";

• For a string meant to be used across several source files for the same library:
  

  __attribute__((visibility("hidden"))) const char foo[] = "foo";

For various reasons, having vmware running on my desktop would be kind of useful. VMware provide a Free (as in beer) version of their software called VMware Player. I downloaded the file VMware-Player-4.0.2-591240.x86_64.bundle off their website and tried to build it. It failed to build. Given my previous lack of success with VMware server, I wasn t too surprised. What was surprising was it wasn t too hard to fix it. The problem was that the vmnet module would not compile and that was due to three things:

• net device ops no longer has set_multicast_list (in netif.c)
• the linux module header needs to be included to define THIS_MODULE
• skb_frag_t has been redefined and needs an adjustment

The patch is only a few lines and means I can compile vmware on my Debian sid computer running kernel 3.2.0-1 vmnet.patch To use it, you will need to find where the modules are built, for me it is /usr/lib/vmware/modules/source

1. mv vmnet.tar vmnet2.tar
2. tar xf vmnet2.tar
3. patch -p1 < vmnet.patch
4. tar cf vmnet.tar vmnet-only

With that you can run the player which will try to build the modules and you re done!

Related articles

• imabonehead: Cross-compiling VMWare View for ARM Linux (in Debian/Ubuntu) CNXSoft Embedded Software Development (cnx-software.com)
• Virtually Run Another OS in your System (tech2copycat.wordpress.com)
• Upgrade to Oneiric breaks VMWare Player (aslakjohansen.wordpress.com)

Now that Rcpp 0.9.10 is released and on CRAN, other packages can take advantage of a small change needed to make use of the quasi-output stream Rcpp::Rcout. So the new release 0.2.35 of RcppArmadillo does just that---and input/output from Armadillo, the wonderful linear algebra / math library for C++ written chiefly by Conrad Sanderson now comes through in a coordinated buffered fashion. As as std::cout is no longer used by default, the R CMD check no longer flags this. There are two more bugfixes in the releases for issues noticed by attentive RcppArmadillo users. Teo Guo Ci spotted a missing semicolon when C++0x code was activated, and Gershon Bialer suggested a fix for an issue plagueing users of the jurassic g++ 4.2.1 compiler which Apple forces on users of OS X. We thank Martin Renner for testing these fixes. The NEWS entry below summarises the changes.

0.2.35  2012-02-17
    o   Upgraded to Armadillo release 2.4.3
          * Support for ARMA_DEFAULT_OSTREAM using Rcpp::Rcout added
    o   Minor bug fix release improving corner cases affecting builds:
          * Missing semicolon added in Mat_meat (when in C++0x mode), with
            thanks to Teo Guo Ci 
          * Armadillo version vars now instantiated in RcppArmadillo.cpp
            which helps older g++ versions, with thanks to Gershon Bialer
          * Thanks also to Martin Renner for testing these changes
          * Unit tests output fallback directory changed per Brian Ripley's
            request to not ever use /tmp
          * Minor update to version numbers in RcppArmadillo-package.Rd

Courtesy of CRANberries, there is also a diffstat reports for 0.2.35 relative to 0.2.34 As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

You might find yourself at some moment when your git repository imported from CVS does not contain all the correct names and email addresses of the commits which were once in CVS but are now part of your project history in your git repo. Or you might do a cvsimport which missed a few authors.

Let's suppose you first import the cvs repo into git, but then you realise you missed some authors.

Before being able to do a git cvsimport, you need a checkout of the module or cvs subdir that you want to turn into its own git repo.

For ease of use I defined CVSCMD as

cvs -z9 -d :pserver:my_cvs_id@cvs.server.com:/root_dir

You will need to replace the items written in italics according to you situation, more exactly, you need to define 'my_cvs_id', 'cvs.server.com' and 'root_dir'. If your acces method to the server is not pserver, you should change that accordingly. This information should be available from your project admin or pages.


Check out the desired module or even subdir of a module

$CVSCMD checkout -d localdirname MODULE/path/to/subdir

git cvsimport -A ../authors -m -z 600 -C ../new-git-repo -R

How to find out the commits which do need rewriting

The way to limit yourself only to the commits that had no cvs-git author and commit information on git-cvsimport time is to use a filter like this:

git log -E --author='^[^@]*$' --pretty=format:%h

This tells git log to print only the abbreviated hashes (%h) for the commits that have NO '@' sign in the 'Author:', which happens if no cvs user id to git author and email was provided in the authors file and git cvsimport time.

We will use this command's output to tell later git filter-branch which commits need rewriting. *

But before that...

How do we find if our authors file is complete?

For this task we'll use a slighly modified form of the previous command and some shell script magic.

git log -E --author='^[^@]*$' --pretty=format:%an sort -u > all-leftout-cvs-authors

And now in all-leftout-cvs-authors we'll have a sorted list of all cvs id's which were not handled in the original git-cvsimport. In my case there are only 19 such ids:

$ wc -l all-leftout-cvs-authors
19 all-leftout-cvs-authors

Nice, that will be easy to fix. Now edit your all-leftout-cvs-authors file to add the relevant information in a format similar to this:

john = John van Code <john@code.temple.tld>
jimmy = Jimmy O'Document <jimmy@documenting.com>

In case you can't make a complete cvs-user-to-name-and-email map, you might want to use stubs of the following form in order to be able to easily identify later such commits, if you prefer (or you could let them unaltered at al ;-):

cvsid=cvsid <cvsid@cvs.server.com>

How to actually do the filtering to fix history (using git-filter-branch and a script)

After this is done, we'll need just one more piece, the command to do the altering itself which reads as follow (note that my final authors file is called new-authors and that I placed this in a script in order to be able to easily run it without trying to escape all spaces and such madness):

[ "$authors_file" ]   export authors_file=$HOME/new-authors

#git filter-branch -f --remap-cvs --env-filter '
git filter-branch -f --env-filter '

get_name ()  
    grep "^$1=" "$authors_file"   sed "s/^.*=\(.*\)\ .*$/\1/"
 

get_email ()  
    grep "^$1=" "$authors_file"   sed "s/^.*\ <\(.*\)>$/\1/"
 

if grep -q "^$GIT_COMMITTER_NAME" "$authors_file" ; then
    GIT_AUTHOR_NAME=$(get_name "$GIT_COMMITTER_NAME") &&
    GIT_AUTHOR_EMAIL=$(get_email "$GIT_COMMITTER_NAME") &&
    GIT_COMMITTER_NAME="$GIT_AUTHOR_NAME" &&
    GIT_COMMITTER_EMAIL="$GIT_AUTHOR_EMAIL" &&
    export GIT_AUTHOR_NAME GIT_AUTHOR_EMAIL &&
    export GIT_COMMITTER_NAME GIT_COMMITTER_EMAIL
fi
' -- --all

You might wonder what's up with the commented git filter-branch line with the --remap-cvs option. This script will NOT work for you as long as you have the stock git-filter-branch script and keep the option --remap-cvs while not patching your git-filter-branch script (/usr/lib/git-core/git-filter-branch), but that option will provide a file with the mappings from the old to the new commit ids. If you want that function, too, you'll want to apply this patch to git-filter-branch:

diff --git a/git-filter-branch b/git-filter-branch
old mode 100644
new mode 100755
index ae602e3..d1f7ef6
--- a/git-filter-branch
+++ b/git-filter-branch
@@ -149,6 +149,11 @@ do
   prune_empty=t
   continue
   ;;
+ --remap-cvs)
+  shift
+  remap_cvs=t
+  continue
+  ;;
  -*)
   ;;
  *)
@@ -368,6 +373,33 @@ while read commit parents; do
    die "could not write rewritten commit"
 done <../revs
 
+# Rewrite the cvs-revisions file, if requested and the file exists
+
+ORIG_CVS_REVS_FILE="$ GIT_DIR /cvs-revisions"
+if [ -f "$ORIG_CVS_REVS_FILE" ]; then
+ if [ "$remap_cvs" ]; then
+  printf "CVS remapping requested\n"
+
+  CVS_REVS_FILE="$tempdir/cvs-revisions"
+  cp "$ORIG_CVS_REVS_FILE" "$CVS_REVS_FILE"
+  printf "\nFound $ORIG_CVS_REVS_FILE; will copy and alter it as $CVS_REVS_FILE\n"
+  cvs_remap__commit_count=0
+  newcommits="$(ls ../map/   wc -l)"
+  for commit in ../map/* ; do
+   cvs_remap__commit_count=$(($cvs_remap__commit_count+1))
+   printf "\rRemap CVS commit $commit ($cvs_remap__commit_count/$newcommits)"
+
+   oldsha1="$(basename $commit)"
+   read newsha1 < $commit
+   sed -i "s@$oldsha1\$@$newsha1@" "$CVS_REVS_FILE"
+  done
+ else
+  warn "\nNo CVS remapping requested, but cvs-revisions file found. All CVS mappings will be lost.\n"
+ fi
+elif [ "$remap_cvs" ]; then
+ warn "\nWARNING: CVS remap was ignored, since no original cvs-revisions file was found\n"
+fi
+
 # If we are filtering for paths, as in the case of a subdirectory
 # filter, it is possible that a specified head is not in the set of
 # rewritten commits, because it was pruned by the revision walker.
@@ -491,6 +523,11 @@ if [ "$filter_tag_name" ]; then
  done
 fi
 
+if [ "$remap_cvs" -a -f "$CVS_REVS_FILE" ]; then
+ mv "$ORIG_CVS_REVS_FILE" "$ORIG_CVS_REVS_FILE.original"
+ cp "$CVS_REVS_FILE" "$ORIG_CVS_REVS_FILE"
+fi
+
 cd ../..
 rm -rf "$tempdir"
  

Then, after running this script, let's call it filter, you should have a brand new git repo with the appropriate authors and their emails set.


P.S.: I have started writing this post some time ago but stopped just before the last part, the one with the filter script. I realise I might be missing something in the explanation, but if you have problems, please comment so I can help you fixing them.

P.P.S.: * I realised in the filter script at some point I wanted to do something like:

for R in $(git log -E --author='^[^@]*$' --pretty=format:%H   head -n 2) ; do
[the same git filter branch command above but ending in ...]
' $R
done

But I think I remember that $R didn't work on the whole history, but only on that revision, or some other weird of that sort. I know I ended up not filtering explicitly those revisions, but the entire history. I hope this helps.

I ve been working on enum again recently and got completely stuck in getopt_long() return values. I don t want to bore you with details, so let me cut to the chase: I needed a way to look into what getopt_long() is actually doing. Now, there are a few ways to figure out the doings of a function. If it s your own code, you could just go ahead and add a few printf()s to see if your variables fill up as intended. Then you could use gdb to walk through your running code. But what if you want to do so in functions outside the scope of your own code? In my case: getopt, i.e. libc? Obviously, you can download the library, add your printf() stuff, compile and install it, and see it all happening. But libc isn t exactly small, fast-compiling, and you really don t want to mess it up and have it installed on your system. So how about just taking what you actually care for and push it in? This is nothing experimental or particularly hacky . I just don t want to forget about it and maybe someone can make use of it. So here we go Download your libc sources to a new directory and copy the files you need. Try something like

mkdir -p /tmp/getopt/mygetopt
cd /tmp/getopt
apt-get source eglibc
cp eglibc-*/posix/getopt* mygetopt/
cd mygetopt

Now write up a primitive Makefile:

all: libmygetopt.so
libmygetopt.so: getopt.c getopt1.c getopt_int.h getopt.h
	gcc -shared -Wl,-soname,libmygetopt.so.1 -fPIC -o libmygetopt.so.1.0.0 getopt1.c getopt.c
	ln -s libmygetopt.so.1.0.0 libmygetopt.so.1
	ln -s libmygetopt.so.1 libmygetopt.so
clean:
	$(RM) libmygetopt*

In order for this to actually work, I fiddled around with a few #ifdef s in getopt.c. But that s something you ll probably find out yourselves. You should at least be able to compile it ( make it). Now put a printf statement somewhere in the function you want to inspect. Recompile the lib and then go back to your actual program. Running the program ist easy.

src/enum -p1 1 3
1.0
2.0
3.0

Surprisingly, running it using your own getopt version is just as easy. Just push your library in:

LD_PRELOAD=libmygetopt.so LD_LIBRARY_PATH=/tmp/getopt/mygetopt:$LD_LIBRARY_PATH src/enum -p1 1 3
Yo, Ulrich, my getopt version is waaaay cooler than yours!
Yo, Ulrich, my getopt version is waaaay cooler than yours!
1.0
2.0
3.0

If you re disappointed now, that s your own fault. I said it in the beginning, this is nothing special. It s not my idea. It s something that developers all use all the time or something like that. In case you didn t know about it, good for you. Otherwise, sorry for reading my memory dump.

I didn t realize that I had two of my machines attached to openvpn from the same l2 segment. It caused a network outage for about 24 hours. Sorry for the downtime folks. I believe that if I turn on stp for all nodes which have two legs in to the same l2, I can avoid this type of problem in the future. But I don t think the broadcast network overhead for managing the spanning tree is worth it. Especially for vlans which span counties.

---

Wait, I think LD1 is in the same county as the vlan hub. How about vlans which span legislative districts. http://wa-demchairs.org/kcdems/districts.php Hello from LD40, Orcas3! Reporting from F5 HQ/LD36 in Seattle.

---

I use the following tools to manage my vlans and bridges: vlan, bridge-utils and openvpn I verify my tls certs with perl.

#!/usr/bin/perl -wT
use strict;
use Log::Log4perl;
Log::Log4perl->init('/etc/openvpn/log4perl.conf') or die "couldn't init logger: $!";
my $logger = Log::Log4perl->get_logger('tls-verify');
$logger->debug("verifying tls cert for config [$ENV config ]");
my($network) = ($ENV config  =~ m:^/etc/openvpn/(.*?).conf$:);
my($certificate_depth) = ($ARGV[0] =~ /^(\d+)$/);
my($locality,$organization,$org_unit,$common_name) =
  ($ARGV[1] =~ m /C=US/ST=Washington/L=(.*?)/O=(.*?)/(?:OU=(.*?)/)?CN=(.*)$ );
$logger->debug("X509_NAME_oneline: [$ARGV[1]");
my %network_cn =
  (
   vl3  =>  [ 'vl3.colliertech.org',
              'Collier_Technologies_Server_CA',
              'Collier_Technologies_Root_CA',
   ],
   vl12 =>  [ 'vl12.colliertech.org',
              'Collier_Technologies_Server_CA',
              'Collier_Technologies_Root_CA',
   ],
   vl245 => [ 'vl245.colliertech.org',
              'Collier_Technologies_Server_CA',
              'Collier_Technologies_Root_CA',
   ],
  );
if( exists $network_cn $network  ) 
  exit 0 if $network_cn $network ->[$certificate_depth] eq $common_name;
  $logger->error("common name: [$common_name], network: [$network], depth: [$certificate_depth]");
 else 
  $logger->error("network: [$network]");
 
exit 1;

Sociological Images has an interesting article about the attempts to apply the word Camping to OWS and framing the issues [1]. Lester Macgurdy wrote an insightful article about the snake , a new technique for OWS protesters to beat riot police [2]. Ron Barassi suggests that Australia Day be celebrated on the 27th of May to commemorate the day in 1967 when the Australian constitution was amended to not be racist [3]. The current Australia Day is often referred to as Invasion Day . IMHO Ron deserves another Best and Fairest award. Stefon Harris gave an entertaining TED talk about improv Jazz music titled There Are No Mistakes on the Bandstand [4]. It seems that his concepts can apply to some extent to many collaborative projects. John Robb wrote an interesting article about the future of drone (UAV) warfare [5]. He suggests that having one person control each drone is a temporary thing and that the future is to have a cloud of cheap autonomous drones taking strategic control from one person. His comparison of Starcraft players to future drone fighters is interesting. The OWS movement is branching out into other related areas, OccupyYourHomes.org is one of the latest ones [6]. When banks try to forclose on homes without good cause the OWS people are protesting. Cory Doctorow wrote an important article for The Guardian about corporations using the Youtube ContentID system to pirate works that other people have uploaded [7]. Matt Taibbi s description of Goldman Sachs as a great vampire squid wrapped around the face of humanity, relentlessly jamming its blood funnel into anything that smells like money will never die [8]. It has spawned many other creative descriptions of the evil and greed of Goldman Sachs and even Lloyd Blankfein of Goldman Sachs describes his company as having burned down the Reichstag, shot the Archduke Ferdinand and fired on Fort Sumter he was trying to use satire, but I don t think that Goldman Sachs people would act differently to Fritz Thyssen. Keith Packard wrote an interesting article about the Calypso CalDAV system which he uses with Android [9]. He makes lots of good points about how to improve calendaring and contacts on Android, unfortunately I lack time to fiddle with such things at the moment so I ll stick with Google in spite of the risks. Asheesh Laroia wrote a great article about the problems with short (32bit) GPG keys [10]. It seems that creating keys with matching ID numbers isn t particularly difficult and that GPG doesn t handle them as well as we would like giving the possibility of at best annoying DoS attacks and at worse security problems due to using the wrong key. Sociological Images has an interesting article about when game show audiences are trustworthy [11]. It seems that French people don t want an undeserving person to win so they will intentionally advocate the wrong answer if the contestant should know it. Paul Wayper gave a great lecture titled SE Linux for Everyone [12]. He covers the basics of SE Linux in a user-friendly way and explains some simple solutions to common problems which don t involve compromising system security. Paul Tassi wrote an insightful article for Forbes about piracy [13]. His conclusion is that the media companies should make it cheaper and easier to be a customer and not spend insane amounts of money on low quality products. The Reid Report has an interesting article about Ron Paul s racism [14]. Ron Paul is generally well regarded outside the US because he wants the US government to stop meddling in the affairs of other countries, but while he s less bad than other US politicians in terms of foreign policy that doesn t make him a good person. Anonymous hacked some mailboxes belonging to a neo-Nazi group and found links to Ron Paul [15]. I ve always been suspicious of the way Ron Paul wanted to avoid anti-racism legislation on supposed Libertarian principles. The Reid Report has an interesting summary of Ron Paul news plus some criticism of Glenn Greenwald and others who associate with him [16].

• [1] http://thesocietypages.org/socimages/2011/12/06/on-camping-and-the-framing-of-occupy-movement-strategies/
• [2] http://www.portlandoccupier.org/2011/12/15/occupy-portland-outsmarts-police-creating-blueprint-for-other-occupations/
• [3] http://tinyurl.com/6mr95hu
• [4] http://www.ted.com/talks/stefon_harris_there_are_no_mistakes_on_the_bandstand.html
• [5] http://globalguerrillas.typepad.com/globalguerrillas/2011/12/drone-bonjwas.html
• [6] http://occupyourhomes.org/
• [7] http://www.guardian.co.uk/technology/2011/dec/12/pirates-of-youtube-cory-doctorow
• [8] http://dealbook.nytimes.com/2011/12/13/the-long-life-of-the-vampire-squid-metaphor/
• [9] http://keithp.com/blogs/calypso/
• [10] http://www.asheesh.org/note/debian/short-key-ids-are-bad-news.html
• [11] http://thesocietypages.org/socimages/2011/12/26/are-game-show-audiences-trustworthy/
• [12] http://www.youtube.com/watch?v=ZThVfm3JXdM
• [13] http://www.forbes.com/sites/insertcoin/2012/02/03/you-will-never-kill-piracy-and-piracy-will-never-kill-you/
• [14] http://blog.reidreport.com/2011/12/the-real-bottom-line-on-the-paul-newsletters-race-baiting-for-power-and-profit/
• [15] http://www.care2.com/causes/anonymous-hacks-neo-nazis-finds-ron-paul.html
• [16] http://blog.reidreport.com/2011/12/should-glenn-greenwald-have-to-own-the-ron-paul-blue-plate-special/

Related posts:

1. Links February 2011 Australia s Department of Finance has mandated that the MS-Office document...
2. Links January 2012 Cops in Tennessee routinely steal cash from citizens [1]. They...
3. Links February 2009 Michael Anissimov writes about the theft of computers from the...

For the past 5 days, I have worked on replacing a part of Britney (our beloved testing migration tool). The part I am trying replacing is her installability tester , which is probably one of the most used parts of Britney. I know premature optimization is the root of all evil, but I felt compelled to be a bit evil this time. The reason is that a na ve algorithm could easily spend decades on computing the result of our most complete tests (each consists of roughly 2 * 35 000 packages). Within a couple of days, I felt I had a reasonable candidate to use on the large tests. On the large tests, it ran with a noticeable speed regression and it produced one different result. The result difference was about 20-40 source packages and their binaries. You may have noticed I used difference and not regression . In the given test, our current implementation at some point gives up with an AIEEE error. So I decided to turn my attention to the speed regression first. As any good programmer, I decided to use a profiler (cProfile) to determine my bottle-necks. This turned out to be a good idea, as the bottleneck was not quite where I thought it was. I wish I could say I just patched it out the issue, but unfortunately not. I played around with various changes such as using implied for loops rather than explicit ones etc. Some times I would manage to cut the runtime in half only to be unable to reproduce it later. Yesterday I finally realized what caused this. I was working in 3 different chroots, two 64-bit sid chroots[1] and a 32-bit stable chroot. Turns out that my replacement is only slow in the stable chroot. As soon as I moved to a sid chroot, the runtime was more or less cut in half. I have not bothered to check if it is the 32 vs 64 bit, the stable vs sid part or maybe the python profile in stable is just slow[2]. I simply moved to my sid chroots for now and continued working. With that taken care of, I figured I would remind myself of what I was up against. On the largest test I have available, I profiled my implementation to around 4:30 5:00 minutes. Regardless of my changes, I always got times in that range. I did manage to patch a bug in my code that reduced by only diff to 10 source packages (plus their binaries) at the price of 30 seconds. So I was back to 5 minutes according to the profiler, but I noticed that my test runner disagreed. According to my test runner my implementation had a runtime of approx. 3 minutes and 40 seconds. For comparison my test runner puts the original implementation at 3 minutes and 20 seconds for that test. In short, I have spent most of my day trying to optimize my code to compensate for the +20% overhead the profiler introduced. Admittedly, my replacement still needs some speed improvement on some other tests where it still has a factor 2 runtime regression. I also need to fix the last diff, which now I suspect is in my replacement. [1] In case you were wondering, yes those chroots are on two different machines. [2] Of course, it could be a combination of that as well Anyhow, I am too lazy to research it at the moment.

Assume the following situation: You set problem sheets for a class, and at the end of the semester, you have 13 individual LaTeX files, including the preamble (the stuff from \documentclass to \begin document ). Now you want to provide a file that contains all problems, followed by all problems with solutions. You could just concatenate the resulting PDFs, but that would waste quite a bit of paper, as most problem sheets do not fill the whole page. You could just copy n paste the LaTeX code, but that is not neat either. So you want to include the 13 individual documents in one LaTeX document. LaTeX provides \include and \input commands, but these would require that the content of the individual files, i.e. the stuff between \begin document and \end document , is put in a file of its own. You cannot just \input the full problem sheet document, as there must be only one \documentclass command, and a few preamble commands are invalid within the document. There are various solutions suggested, e.g. on Wikibooks and StackOverflow, and there are packages helping with that functionality, such as combine or subfiles. I chose a rather rough, hands-on solution that worked great in my case: The idea is to simply re-define all commands that you have in your preamble to do nothing, including the document environment, before including the other files and constraint to a group. In my case, the preambles were relatively small, as the common definitions of the problem sheets were in a file of their own. I am also using the \foreach command provided by the TikZ package for a convenient loop over all problem sheets:

\foreach \x in  0,...,13   
   
    \excludecomment solution 
    
    \DeclareDocumentCommand \documentclass om 
    \DeclareDocumentCommand \usepackage om 
    \newcommand \Blattnummer [1] 
    \DeclareDocumentCommand \FirstDueDate mmm 
    \DeclareDocumentCommand \ThisDueDate mmm 
    \newcommand \SkippedWeeks [1] 
    \renewenvironment document 
    \section* Problem Sheet \x 
    \setcounter problem 0 
    \input GraphTheoryProblems\x 
   
 
\clearpage
\label sols 
\foreach \x in  0,...,13   
   
    \newenvironment solution 
      \par\addvspace 1em 
     
     
    \DeclareDocumentCommand \documentclass om 
    \DeclareDocumentCommand \usepackage om 
    \newcommand \Blattnummer [1] 
    \DeclareDocumentCommand \FirstDueDate mmm 
    \DeclareDocumentCommand \ThisDueDate mmm 
    \newcommand \SkippedWeeks [1] 
    \renewenvironment document 
    \section* Solution Sheet \x 
    \setcounter problem 0 
    \input GraphTheoryProblems\x 
   
 

To redefine commands taking optional arguments, using the xparse package is convenient. You can see how I include every file twice, first with the solution environment turned into a comment and then again with the solution environment actually showing its content. I create a section header based on the counter of the foreach loop, but I could have easily redefined \title and use that for a header, if my problem sheets had used that command (They don t, as they calculate the problem sheet number and the due date based on the number in the filename very convenient, and maybe worth a blog post of its own). The problem counter is reset for each file, so that the problems are numbered individually. I could have left this out, then the problems would be numbered consecutively across all included problem sheets. The individual and resulting files can be found on the Graph Theory course website.
Flattr this post

Last week I went along to the food co-ops networking event at the Southville Centre in Bristol. It was a useful event and very inspiring and informative to meet people from so many other co-ops, as well as attend some useful workshops: the two I went to were Good meetings and communication and Starting and developing a food co-op, while there were also ones on funding and Simply Legal available. There was some time for networking, as well as a relaxed end to the day which let me catch up with a few more people. I would have preferred a little more time for the workshops and a little less on case studies (every food co-op is different and I don t think any of the featured ones were quite what I was looking for), but that s a very minor thing and didn t really reduce the usefulness of the whole day. Our co-op is a tech worker co-op and not a food co-op, so I didn t know that much about how to start one before the event. Now I ve got a much better idea of what I need to do when I eventually move back out to what may be a co-op desert in King s Lynn. Are you a member of a food co-op or buying group? If so, what would you say about it? Were you involved in its start-up?

Once in a while my home server have disk problems. Thanks to Linux Software RAID, I have not lost data yet (but I was close this summer :). But once a disk is starting to behave funny, a practical problem present itself. How to get from the Linux device name (like /dev/sdd) to something that can be used to identify the disk when the computer is turned off? In my case I have SATA disks with a unique ID printed on the label. All I need is a way to figure out how to query the disk to get the ID out. After fumbling a bit, I found that hdparm -I will report the disk serial number, which is printed on the disk label. The following (almost) one-liner can be used to look up the ID of all the failed disks:

for d in $(cat /proc/mdstat  grep '(F)' tr ' ' "\n" grep '(F)' cut -d\[ -f1 sort -u);
do
    printf "Failed disk $d: "
    hdparm -I /dev/$d  grep 'Serial Num'
done

Putting it here to make sure I do not have to search for it the next time, and in case other find it useful. At the moment I have two failing disk. :(

Failed disk sdd1:       Serial Number:      WD-WCASJ1860823
Failed disk sdd2:       Serial Number:      WD-WCASJ1860823
Failed disk sde2:       Serial Number:      WD-WCASJ1840589

The last time I had failing disks, I added the serial number on labels I printed and stuck on the short sides of each disk, to be able to figure out which disk to take out of the box without having to remove each disk to look at the physical vendor label. The vendor label is at the top of the disk, which is hidden when the disks are mounted inside my box. I really wish the check_linux_raid Nagios plugin for checking Linux Software RAID in the nagios-plugins-standard debian package would look up this value automatically, as it would make the plugin a lot more useful when my disks fail. At the moment it only report a failure when there are no more spares left (it really should warn as soon as a disk is failing), and it do not tell me which disk(s) is failing when the RAID is running short on disks.

Over recent months, I have been pondering the relevance of my Free Software involvement a lot: More than anything, it constantly feels more and more like several important distributions and projects are moving in directions that break with the old ways far too radically, by breaking software usability rules such as "least surprise", "works out of the box", and "if it ain't broken, don't fix it" either to try new approaches to addressing existing needs or simply for change's sake. Another aspect lies in my usual motivation for packaging or adopting software: because I need it and nobody else bothered doing it. Further reflection made me ponder what, correspondingly, are my usual reasons for dropping software: because what exists already works and because everybody and their grand-mother already invented a million of ways to handle the same issue. Which brings us to today's news. What prompted me to hand the dictionary packages I maintained over to someone else was a combination of "no longer need it" and "it already works fine as it is." For instance, the Estonian dictionaries haven't seen any upstream release since the initial one in 2003. While I still use Estonian daily in my online communications, the last 9 years have mostly been spent making minor changes to the dependencies and maintainer scripts, just to keep the package compliant with current Debian packaging policies and with ongoing improvements to the dictionary-common maintainer tools. This is not the sort of work that requires any knowledge of the Estonian language, so I felt that having dictionary-common developers handle those technical transitions in a clean and across-the-board way for as many dictionary packages as possible would be more productive than me trying to keep track of those changes by myself. A similar case applies to the Latvian dictionaries. While there have been occasional upstream releases, some of which required patching the source code, maintaining the package has mostly been about tracking Debian policy changes and dictionary-common functional changes. Additionally, my interest for learning Latvian has dramatically dropped over the years, so I no longer saw any point in remaining involved in maintaining the package. Ditto for the Russian dictionaries: occasional upstream releases, occasional patches, regular packaging upgrades to keep up with the Debian policy and with dictionary-common functionality, but no longer much of anything than a passive interest in practicing my Russian. One of the developers behind dictionary-common, Agustin Martin Domingo, frequently helped me make sense of the changes I needed to track in the past, so he gladly accepted taking over the technical maintenance of all 3 packages. The Estonian dictionaries, while extremely skim in the breadth of vocabulary they cover, remain useful, but are essentially deprecated, as the upstream author is working on a spell checking engine similar to what Voikko does for Finnish, which is why maintaining them will be a rather easy task for Agustin. Meanwhile, Aigars Mahinovs passively remains on board for the Latvian dictionaries, while the Russian dictionaries have been adopted by Mikhail Gusarov, in both cases with Agustin assisting on technical matters. Basically, I feel that passing the maintenance over to people whose motivation remains high is a better way to guarantee those packages' future than leaving them in my unmotivated hands and I'm glad that I found someone to keep on packaging them. PS: happy Valentine's day to everyone!

Public Resource republishes many court documents. Although these documents are all part of the public record and PR will not take them down because someone finds their publication uncomfortable, PR will evaluate and honor some requests to remove documents from search engine results. Public Resources does so using a robots.txt file or "robot exclusion protocol" which websites use to, among other things, tell search engine's web crawling "robots" which pages they do not want to be indexed and included in search results. Originally, the files were mostly used to keep robots from abusing server resources by walking through infinite lists of automatically generated pages or to block search engines from including user-contributed content that might include spam. The result for Public Resource, however, is that PR is now publishing, in the form of its robots.txt, a list of all of the cases that people have successfully requested to be made less visible! In Public Resource's case, this is is the result of a careful decision; PR makes the arrangement clear in on their website. The robots.txt home page also explains the situation saying, "the /robots.txt file is a publicly available file. Anyone can see what sections of your server you don't want robots to use,", and "don't try to use /robots.txt to hide information." That said, I've looked at a bunch of robots.txt files on websites I have visited recently and, sadly, I've found many sites that use robots.txt as a form of weak security. This is very dangerous. Some poorly designed robots simply ignore the robots.txt file. But one can also imagine an evil search engine that uses a web-crawler that does the opposite of what it's told and only indexes these "hidden" pages. This evil crawler might look for particular keywords or use existing search engine data to check for incoming links in order to construct a list of pages whose existence is only made public through a file meant to keep people away. Check your own robots.txt and ask yourself what it might reveal. By advertising the existence and locations of your secrets, the act of "hiding" might make your data even less private.

As many would know, Debian GNU/Linux is one of the oldest, and the largest Linux distributions that is available for free. Since it was first released in 1993, several people have analysed the size and produced cost estimates for the project. In 2001, Jes s M. Gonz lez-Barahona et al produced an article entitled Counting Potatoes , an analysis of Debian 2.2 (code named Potato). When Potato was released in June 2003, it contained 2,800 source packages of software, totalling around 55 million lines of source code. When using David A. Wheeler s sloccount tool to apply the COCOMO model of development, and an average developer salary of US$56,000, the projected development cost that Gonz lez-Barahona calculated to start-from-scratch and build Debian 2.2 in 2003 was US$1.9 billion. In 2007 an analysis entitled Macro-level software evolution: a case study of a large software compilation by Jes s M. Gonz lez-Barahona, Gregorio Robles, Martin Michlmayr, Juan Jos Amor and Daniel M. German was released. It found that Debian 4.0 (codename Etch released April 2007) had just over 10,000 source packages of software and 288 million lines of source code. This analysis also delved into the dependencies of software packages, and the update flow between Debian release (not all packages are updated with each release). Today (February 2012) the current development version of Debian, codenamed Wheezy, contains some 17,141 source packages of software, but as it s still in development this number may change over the coming months. I analysied the source code in Wheezy, looking at the content from the original software that Debian distributes from its upstream authors without including the additional patches that Debian Developers apply to this software, or the package management scripts (used to install, configure and de-install packages). One might argue that these patches and configuration scripts are the added value of Debian, however the in my analysis I only examined the pristine upstream source code. By using David A Wheeler s sloccount tool and average wage of a developer of US$72,533 (using median estimates from Salary.com and PayScale.com for 2011) I summed the individual results to find a total of 419,776,604 source lines of code for the pristine upstream sources, in 31 programming languages including 429 lines of Cobol and 1933 lines of Modula3! In my analysis the projected cost of producing Debian Wheezy in February 2012 is US$19,070,177,727 (AU$17.7B, EUR 14.4B, GBP 12.11B), making each package s upstream source code wrth an average of US$1,112,547.56 (AU$837K) to produce. Impressively, this is all free (of cost). Zooming in on the Linux Kernel In 2004 David A. Wheeler did a cost analysis of the Linux Kernel project by itself. He found 4,000,000 source lines of code (SLOC), and a projected cost between US$175M and US$611M depending on the complexity rating of the software. Within my analysis above, I used the standard (default) complexity with the adjusted salary for 2011 (US$72K), and deducted that Kernel version 3.1.8 with almost 10,000,000 lines of source code would be worth US$540M at standard complexity, or US$1,877M when rated as complex . Another Kernel Costing in 2011 put this figure at US$3 billion, so perhaps there s some more variance in here to play with. Individual Projects Other highlights by project included:

Project	Version	Thousands of SLOC	Projected cost at US$72,533/developer/year
Samba	3.6.1	2,000	US$101 (AU$93M)
Apache	2.2.9	693	US$33.5M (AU$31M)
MySQL	5.5.17	1,200	US$64.2M (AU$59.7M)
Perl	5.14.2	669	US$32.3M (AU$30M)
PHP	5.3.9	693	US$33.5M (AU$31.1M)
Bind	9.7.3	319	US$14.8M (AU$13.8M)
Moodle	1.9.9	396	US$18.6M (AU$17.3M)
Dasher	4.11	109	US$4.8M (AU$4.4M)
DVSwitch	0.8.3.6	6	US$250K (AU$232K)

Debian Wheezy by Programming Language The upstream code that Debian distributes is written in many different languages. ANSI C with 168,536,758 is the dominant language (40% of all lines), followed by C++ at 83,187,329 (20%) and Java with 34,698,990 (8%). If you are intersted in finding the line count and cost projections for any of the 17,000+ projects, you will find them in the raw data CSV. Other Tools and Comparisons Ohcount is another source code cost analysis tool. In March 2011 Ohcount was run across Debian Sid: its results are here. In comparison, its results appear much lower than the sloccount tool. There s also the Ohloh.net Debian Estimate which only finds 55 Million source lines of code and a projected cost of US$1B. However Ohloh uses Ohcount for its estimates, and seems to be to be around 370 million SLOC missing compared to my recent analysis. Summary Over the last 10 years the cost to develop Debian has increased ten-fold. It s intersting to know that US$19 billion of software is available to use, review, extend, and share, for the bargain price of $0. If we were to add in Debian patches and install scripts then this projected figure would increase. If only more organisations would realise the potential they have before them. Need help with Linux (including Debian), Perl, or AWS? See www.jamesbromberger.com.

Some days ago, reading my local Couchsurfing groups, I stumbled across an announcement by Australian Peter Davies to go to each of the 148 stations in the Mexico City Metro system, take some photos of the environment, and document on his impressions. I have followed and enjoyed the Mexico City Metro blog since I learnt of it, and have grown used to looking forward to the daily post-or-two. Peter writes each of his entries both in English and Spanish (you can tell it's not a native Spanish, but it's a good effort). He has been doing the stations in a very well distributed order (I cannot say it's completely random, but it's surely not lineal or methodical). I connected wiht his project as I love discovering the city more or less the same way, but with a different system: I try to have at least one long bike ride every two weeks (being "long" something over 40Km). I usually go either to the North or to downtown and to the East by the good old route I always take, and on my way back, at some point I decide just to turn right or left and discover yet another village slurped by the city. I don't usually take pictures, as I'm too much into the cycling thrill, left-right-left-right... But cycling has led me to appropriate my city (I don't know if that's proper English), to make my city really mine, to get to know parts of it I'd never otherwise go to. Anyway, Peter's is a great way to document urban life. I'm in love with my city, and with expressions of urban appropriation. I loved his project, and if you are interested by what I say, go take a look at his wanderings in the city. I have suggested him two bits to check, but the work is very much an artist's He accepts my input, but quite probably he will do whatever he pleases ;-) In case any of you is interested in contacting him, I can tell you for a fact he replies :-) [*] And what is CouchSurfing? Oh, a great community where you can offer a space to crash at your house for unknown people from all around the world. I have never requested a couch, as the Free Software community is much more tightly knit, but I have offered it to several interesting people.

I'm working on porting FreeGLUT to Android! Since I don't have access so the repository (I'm not a regular contributor), and since it will be a BIG patch, I thought I'd try a git-svn branch. So this won't be a post on OpenGL for once I'm not an expert at managing Git branch but here's how I set it up. In this case, let's forget about importing tags: they don't have the same directory branch point in the hierarchy (sometimes the freeglut/ module, sometimes the parent directory), and git-svn only correctly supports set of sane tags that track the same initial directory. SVN is SLOW, especially remotely. I suggest you make a rsync copy of the repository first, it will greatly speed up the import :

rsync -avHSP freeglut.svn.sourceforge.net::svn/freeglut/ freeglut.svn/

Then start the import with git-svn :

git svn clone --trunk=trunk/freeglut/freeglut file://$(pwd)/freeglut.svn/ freeglut-git/

And edit .git/config to point to https://freeglut.svn.sourceforge.net/svnroot/freeglut :

[svn-remote "svn"]
    url = https://freeglut.svn.sourceforge.net/svnroot/freeglut
    fetch = trunk/freeglut/freeglut:refs/remotes/trunk

I work with 2 branches:

• master : the SVN on-going sync - may be rebased to push with git svn dcommit
• android : my feature branch - not rebased and hence trackable by contributors

Here's how I synchronize the upstream repository and merge it regularly with my feature branch:

git svn fetch
git checkout master
git merge remotes/trunk
git checkout android  # my feature branch
git merge master
# if conflict
git add .../sthing.c
git commit -m "Merge master"
git push

Any better way? The ongoing work is available at https://gitorious.org/freeglut/android

Talking about Secure Boot again, I'm afraid. One of the things that's made discussion of this difficult is that, while the specification isn't overly complicated, some of the outcomes aren't obvious at all until you spend a long time thinking about it. So here's some clarification on a few points.
Secure Boot provides no additional securityUntrue. Attacks against the pre-boot environment are real and increasingly common - this is a recent proof of concept, and this has been seen in the wild. Once something has got into the MBR, all bets are off. It can modify your bootloader or kernel, inserting its own code to return valid results whenever any kind of malware checker scans for it. The only way to reliably identify it is to either move the disk to another system or to cold boot off verified media. By restricting pre-OS code to something that's been signed, Secure Boot does provide enhanced security.
Secure Boot is just another name for Trusted BootUntrue. Trusted Boot requires the ability to measure the entire boot process, which gives the OS the ability to figure out everything that's been run before OS startup. The root of trust is in the hardware and a TPM is required. Secure Boot is simply a way to limit the applications that are run before the OS. Once booted, there is no way for the OS to identify what was previously booted, or even if the system was booted securely.
Microsoft are just requiring that vendors implement the specificationUntrue. Quoting from the Windows 8 Hardware Certification Requirements:

MANDATORY: No in-line mechanism is provided whereby a user can bypass Secure Boot failures and boot anyway Signature verification override during boot when Secure Boot is enabled is not allowed. A physically present user override is not permitted for UEFI images that fail signature verification during boot. If a user wants to boot an image that does not pass signature verification, they must explicitly disable Secure Boot on the target system.

Section 27.7.3.3 of version 2.3.1A of the UEFI spec explicitly permits implementations to provide a physically present user override. Whether this is a good thing or not is obviously open to argument, but the fact remains that Microsoft forbid behaviour that the specification permits.
Secure Boot can be used to implement DRMUntrue. The argument here is that Secure Boot can be used to restrict the software that a machine can run, and so can limit a system to running code that implements effective copy protection mechanisms. This isn't the case. For that to be true, there would need to be a mechanism for the OS to identify which code had been run in the pre-OS environment. There isn't. The only communication channel between the firmware and the OS is via a single UEFI variable called "SecureBoot", which may be either "1" or "0". Additionally, the firmware may provide a table to the OS containing a list of UEFI executables that failed to authenticate. It is not required to provide any information about the executables that authenticated correctly.

In both these cases, the OS is required to trust the firmware. If the firmware has been compromised in any way (such as the user turning off Secure Boot), the data provided by the firmware can be trivially modified and the OS told that everything is fine. As long as machines exist where users are permitted to disable Secure Boot, Secure Boot is not any kind of DRM scheme.
Secure Boot provides physical securityUntrue. Secure Boot does not in any way claim to improve security against attackers who have physical access, for the same reasons as the DRM case. The OS has no way to determine that the firmware's behaviour has been modified. A physically-present attacker can simply disable Secure Boot and install a piece of malware that lies to the OS about platform security. The "Evil Maid" attack still exists.
Secure Boot only defines the interaction between the firmware and the bootloader. It doesn't specify any higher policyMisleading. It's true that Secure Boot only specifies the authentication of pre-OS code. However, if you implement Secure Boot it's because you want to ensure that only authenticated code has run before your OS. If there is any way for unauthenticated code to touch the hardware before your OS starts, you can't ensure that. If an authenticated Linux kernel is booted and then loads an unsigned driver, that unsigned driver can fabricate a fake UEFI environment and then launch Windows from it. Windows would falsely believe that it has booted securely. If that authenticated Linux kernel were widely distributed, attackers could use it as an attack vector for Windows. The logical response from Microsoft would be to blacklist that kernel.

The inevitable outcome of implementing Secure Boot is that every component that can touch hardware must be signed. Anyone who implements Secure Boot without requiring that is adding no additional security whatsoever.
Only machines that want to boot Windows need to carry Microsoft's keysAgain, misleading. Microsoft only require one signing key to be installed, and the Windows bootloader will be signed with a key that chains back to this one. However, the bootloader is not the only component that must be signed. Any drivers that are carried on ROMs on plug-in cards must also be signed. One approach here would have been for all hardware vendors to have their own keys. This would have been unscalable - any shipped machine would have to carry keys for every vendor who produces PCI cards. If a machine carried an nvidia key but not an AMD one, swapping a geforce for a radeon would have resulted in the firmware graphics driver failing to load. Instead, Microsoft are providing a signing service. Vendors will be able to sign up for WHQL membership and have their UEFI drivers signed by Microsoft.

This leads to the problem. The Authenticode format used for signing UEFI objects only allows for a single signature. If a driver is signed by Microsoft, it can't be signed by anybody else. Therefore, if a system vendor wants to support off-the-shelf PCI devices with Microsoft-signed drivers, the system must carry Microsoft's key. If the same key is used as the root of trust for the driver signing and for the bootloader signing, that also means that the system will boot Windows.
This is only a problem for clients, not serversNot strictly true. While Microsoft's current requirements don't mandate the presence of Secure Boot on server hardware, if present it must be enabled and locked down as it is for clients. It's not valid for servers to ship with disabled Secure Boot support, or for it to be shipped in a mode allowing users to make automated policy modifications at OS install time.
Secure Boot is required by NISTThis is one that I've heard from multiple people working on Secure Boot. It's amazingly untrue. The document they're referring to is NIST SP800-147, which is a document that describes guidelines for firmware security - that is, what has to be done to ensure that the firmware itself is secure. This includes making sure that the OS can't overwrite the firmware and that firmware updates must be signed. It says absolutely nothing about the firmware only running signed software. Secure Boot depends upon the firmware being trusted, so these guidelines are effectively a required part of Secure Boot. But Secure Boot is not within the scope of SP800-147 at all.
It's easy for Linux to implement Secure BootMisleading. From a technical perspective, sure. From a practical perspective, not at all. I wrote about the details here.
It's only a problem for hobbyist Linux, not the real Linux marketUntrue. It's unclear whether even the significant Linux vendors can implement Secure Boot in a way that meets the needs of their customers and still allows them to boot on commodity hardware. A naive implementation removes many of the benefits of Linux for enterprise customers, such as the ability to use local modifications to micro-optimise systems for specific workloads. One of the key selling points of Linux is the ability to make use of local expertise when adapting the product for your needs. Secure Boot makes that more difficult.

ConclusionMuch reporting on the issues surrounding Secure Boot so far has been inaccurate, leading to misunderstandings about the (genuine) benefits and the (genuine) drawbacks. Any useful discussion must be based around an accurate understanding of the specification rather than statements from analysts with no real understanding of the Linux market or security.

comments

but I ve already forseen the unwanted meet for some time . My elder daughter does need to run educational software from school. And my wife wants to run education software for our younger son. And making all that running under different emulation environments does eat my time, and result is not always good. And also children start to want games Issue 1. Need hardware to run more or less modern windows on. For last 6 years, my home network consists of a more or less powerful headless server, and 3 terminals to access that. Terminals are definitely too weak for windows. And server is needed in 24/7 mode so sorry no windows there. Likely solution for this is one of 3 terminals is actually dying (it is doing so already for months but still works, just hangs sometimes) and needs a replacement, so I can just replace it now and use more powerful hardware. Issue 2. Windows host or windows VM? While windows VM feels preferable for a such a deep-linux person as myself, I tend to think that dual-boot host is better here. If the very reason of allowing windows into this house is make it easy to install and run random windows software, then doing so in VM environment is explicitly against the goal. Issue 3. Use unlicensed windows or buy a license? Although absolute majority of windows users here is Russia use unlicensed windows, it is somewhat unfair to promote free software and use unlicensed windows at the same time. And windows home basic license is not that expensive after all. The setup that makes me less uncomfortable than others is:
- obtain powerful enough hardware, and install that instead of failing terminal,
- obtain windows 7 home basic license
- install dual boot linux + windows, and under linux install windows VM
- when needed, try windows VM first, and in case of any issues, immediately reboot into windows host. What I can t find out for some time already is does microsoft retail license allow to have both host and VM installation on the same computer? These will be physically different installations, but on the same computer, and will never run at the same time. License wording does not give exact answer on this. I wanted to ask on Microsoft s support forum but failed to register there. After asking me quite a few things it claimed that there is some temporary problem and suggested to retry later. I retried later and got the same. I then asked here but looks like nobody knows. At least I want to know, what will happen if I will try to activate windows in VM after I ve activated windows (with the same key) on host. Will this render both installations non-genuine ? Also I d appreciate any comments on the whole situation

Perf stats for "doing nothing" I've recently discovered the perf Linux tool. I heard that oprofile was deprecated and that there is a new tool, and I noted down to try it sometime. Updated: more languages, fixed typos, more details, some graphs. Apologies if this shows twice in your feed. The problem with perf stats is that I hate bloat, or even perceived bloat. Even when it doesn't affect me in any way, the concept of wasted cycles makes me really sad. You probably can guess where this is going I said, well, let's see what perf says about a simple "null" program. Surely doing nothing should be just a small number of instructions, right? Note: I think that perf also records kernel-side code, because the lowest I could get was about ~50K instructions for starting a null program in assembler that doesn't use libc and just executes the syscall asm instruction. However, these ~50K instructions are noise the moment you start to use more high-level languages. Yes, this is expected, but the I was still shocked. And there's lots of delta between languages I'd expected to behave somewhat identical. Again, this is not important in the real world. At all. They are just numbers, and probably the noise (due to short runtime) has lots of influence on the resulting numbers. And I might have screwed up the measurements somehow. Test setup Each program was the equivalent of 'exit 0' in the appropriate form for the language. During the measurements, the machine was as much as possible idle (single-user mode, measurements run at real-time priority, etc.). For compiled languages, -O2 was used. For scripts, a simple #!/path/to/interpreter (without options, except in the case of Python, see below) was used. Each program/script was run 500 times (perf's -r 500) and I've checked that the variations were small ( 0.80% on the metrics I used). You can find all the programs I've used at http://git.k1024.org/perf-null.git/, the current tests are for the tag version perf-null-0.1. The raw data for the below tables/graphs is at log-4. Results Compiled languages

Language	Cycles	Instructions
asm	63K	51K
c-dietlibc	74K	57K
c-libc-static	177K	107K
c-libc-shared	506K	300K
c++-static	178K	107K
c++-dynamic	1,750K	1,675K
haskell-single	2,229K	1,338K
haskell-threaded	2,629K	1,522K
ocaml-bytecode	3,271K	2,741K
ocaml-native	1,042K	666K

Going from dietlibc to glibc doubles the number of instructions, and for libc going from static to dynamic linking again roughly doubles it. I didn't manage to compile a program dynamically-linked against dietlibc. C++ is interesting. Linked statically, it is in the same ballpark as C, but when linked dynamically, it executes an order of magnitude more instructions. I would guess that the initialisation of the standard C++ library is complex? Haskell, which has a GC and quite a complex runtime, executes slightly less instructions than C++, but uses more cycles. Not bad, given the capabilities of the runtime. The two versions of the Haskell program are with the single-threaded runtime and with the multi-threaded one; not much difference. A fully statically-linked Haskell binary (not recommended usually) goes below 1M instructions, but not by much. OCaml is a very nice surprise. The bytecode runtime is a bit slow to startup, but the (native) compiled version is quite fast to start: only 2 number of instructions and cycles compared to C, for an advanced language. And twice as fast as Haskell . Nice! Shells

Language	Cycles	Instructions
dash	766K	469K
bash	1,680K	1,044K
mksh	1,258K	942K
mksh-static	504K	322K

So, dash takes ~470K instructions to start, which is way below the C++ count and a bit higher than the C one. Hence, I'd guess that dash is implemented in C . Next, bash is indeed slower on startup than dash, and by slightly more than 2 (both instructions and cycles). So yes, switching /bin/sh from bash to dash makes sense. I wasn't aware of mksh, so thanks for the comments. It is, in the static variant, more efficient that dash, by about 1.5 . However, the dynamically linked version doesn't look too great (dash is also dynamically linked; I would guess a statically-linked dash "beats" mksh-static). Text processing I've added perl here (even though it's a 'full' language) just for comparison; it's also in the next section.

Language	Cycles	Instructions
mawk	849K	514K
gawk	1,363K	980K
perl	2,946K	2,213K

A normal spread. I knew the reason why mawk is Priority: required is that it's faster than gawk, but I wouldn't have guessed it's almost twice as fast. Interpreted languages Here is where the fun starts

Language	Cycles	Instructions
lua 5.1	1,947K	1,485K
lua 5.2	1,724K	1,335K
lua jit	1,209K	803K
perl	2,946K	2,213K
tcl 8.4	5,011K	4,552K
tcl 8.5	6,888K	6,022K
tcl 8.6	8,196K	7,236K
ruby 1.8	7,013K	6,128K
ruby 1.9.3	35,870K	35,022K
python 2.6 -S	11,752K	10,247K
python 2.7 -S	11,438K	10,198K
python 3.2 -S	29,003K	27,409K
pypy -S	21,106K	10,036K
python 2.6	25,143K	21,989K
python 2.7	47,325K	50,217K
python 2.7 -O	47,341K	50,185K
python 3.2	113,567K	124,133K
python 3.2 -O	113,424K	124,133K
pypy	90,779K	68,455K

The numbers here are not quite what I expected. There's a huge delta between the fastest (hi Lua!) and the slowest (bye Python!). I wasn't familiar with Lua, so I tested it thanks to the comments. It is, I think, the only language which actually improves from one version to the next (bonus points), and where the JIT version also make is faster. In context, lua jit starts faster than C++. Perl is the one that goes above C++'s instructions count, but not by much. From the point of view of the system, a Perl 'hello world' is only about 1.3 -1.6x slower than a C++ one. Not bad, not bad. Next category is composed of TCL and Ruby, both of which had older versions 2-3 slower than Perl, but whose most recent versions are even more slower. TCL has an almost constant slowdown across versions (5M, 6.9M, 8.2M cycles), but Ruby seems to have taken a significant step backwards: 1.9.3 is 5 slower than 1.8. I wonder why? As for TCL, I didn't expect it to be slower to startup than Perl; good to know. Last category is Python. Oh my. If you run perf stat python -c 'pass' you get some unbelievable numbers, like 50M instructions to do, well, nothing. Yes, it has a GC, yes, it does import modules at runtime, but still On closer investigation, the site module and the imports it does do eat a lot of time. Running a simpler python -S brings it back to a more reasonable 10M instructions, which is in-line with the other interpreted languages. However, even with the -S taken into account, Python also slows down across versions: a tiny improvement from 2.6 to 2.7, but (like Ruby) a 3 slowdown from 2.7 to 3.2. Trying the optimised version (-O) doesn't help at all. Trying pypy, which was based on Python 2.7, makes it around 2 slower to startup (both with and without -S). So in the interpreted languages, it seems only Lua is trying to improve, the rest of the languages are piling up bloat with every version. Note: I should have tried multiple perl versions too. Java Java is in its own category; you guess why , right? GCJ was version 4.6, whereas by java below I mean OpenJDK Runtime Environment (IcedTea6 1.11) (6b24-1.11-4).

Language	Cycles	Instructions
null-gcj	97,156K	74,576K
java -jamvm	85,535K	80,102K
java -server	147,174K	136,803K
java -zero	132,967K	124,977K
java -cacao	229,799K	205,312K

Using gcj to compile to native code (not sure whether that's native-native or something else) results in a binary that uses less than 100M cycles to start, but the jamvm VM is faster than that (85M cycles). Not bad for java! Python 3.2 is slower to startup yes, I think the world has gone crazy. However, the other VMs are a few times slower: server (the default one) is ~150M cycles, and cacao is ~230M cycles. Wow. The other thing about java is that it was the only one that couldn't be put nicely in a file that you just exec (there is binfmt_misc indeed, but that doesn't allow different Java classes to use different Java VMs, so I don't count this), as opposed to every single other thing I tested here. Someone didn't grow on Unix? Comparative analysis Since there are almost 4 orders of magnitude difference between all the things tested here, a graph of cycles or instructions is not really useful. However, cycles/instruction, branches percentage and branches miss-predicted percentage can be. Hence first the cycles/instructions: Pypy is jumping out of the graph here, with the top value of over 2 cycles/instruction. Lua JIT is also bigger than Lua non-JIT, so maybe there's something to this (mostly joking, two data points don't make a series). On the other hand, Python wins as best cycles/instruction (0.91). Lots of ILP, to get below 1? Java gets, irrespective of VM, consistently near 1.0-1.1. C++ gets very different numbers between static linking (1.666) and dynamic linking (1.045), whereas C has basically identical numbers. mksh also has a difference between dynamic and static linking. Hmm Ruby, TCL and Python have consistent values across versions. And that's about what I can see from that graph. Next up, percentage of branches out of total instructions and percentage of branches missed: Note that the two lines shouldn't really be on the same graph; for the branch %, the 100% is the total instructions count, but for the branch miss %, the 100% is the total branch count. Anyway. There are two low-value outliers:

• dynamically-linked C++ has a low branch percentage (17.46%) and a very low branch miss percentage (only 4.32%)
• gcj-compiled java has a very low branch miss percentage (only 2.82%!!!), even though is has a regular branch percentage (20.85%)

So it seems the gcj libraries are well optimised? I'm not familiar enough with this topic, but on the graph it does indeed stand out. On the other end, mksh-static has a high branch miss percentage: 11.60%, which jumps clearly ahead of all the others; this might be why it has a high cycles/instruction count, due to all the stalls in misprediction; one has to wonder why it confuses the branch predictor? I find it interesting that the overall branch count is very similar across languages, both when most of the cost is in the kernel (e.g. asm) and when the user-space cost heavily over-weighs the kernel (e.g. Java). The average is 20.85%, minimum is 17.46%, max 22.93%, standard deviation (if I used gnumeric correctly) is just 0.01. This seems a bit suspicious to me . On the other hand, the mispredicted branches percentage varies much more: from a measly 2.82% to 11.60% (5x difference). Summary So to recap, counting just instructions:

• going from dietlibc to glibc: 2 increase
• going from statically-linked libc to dynamically-linked libc: doubles it again
• going from C to C++: 5 increase
• C++ to Perl: 1.3
• Perl to Ruby: 3
• Ruby to Python (-S): 1.6x
• Python -S to regular Python: 5x
• Python to Java: 1 -2 , depending on version/runtime
• branch percentage (per total instructions) is quite consistent across all of the programs

Overall, you get roughly three orders of magnitude slower startup between a plain C program using dietlibc and Python. And all, to do basically nothing. On the other hand, I learned some interesting things while doing it, so it wasn't quite for nothing .

When Jacob was just born, I wondered how I might introduce them to computing. I thought over various things, but that wasn t really the most pressing thing right then. I don t suppose that I could have predicted installing an XMPP IM server (Prosody) for the boys. And I certainly couldn t have predicted creating accounts named: jacob, oliver, butterfly, bear. Because, as Jacob pointed out to me, if (Jacob s favorite toy) butterfly is typing with his wings, then he shouldn t be logged in as Jacob. I admire my 5-year-old s security consciousness Anyhow, as I mentioned yesterday, Jacob and Oliver enjoy their computer, which I recently put on the LAN. The firewall does not pass any of its traffic to the Internet, though, with very limited exceptions. Jacob can read, and is starting to enjoy typing as well. So I thought he would enjoy sending IMs to me. As his computer has no GUI, I needed a text-mode client. Something with an IRC-like interface that could be scripted to open up a window with me directly sounded perfect. Initially I tried irssi s XMPP plugin, but it proved to be too buggy (wanting to always latch on to a particular resource on the remote end, not having very predictable window behavior, etc.) So I switched to mcabber. With a couple of quick configuration bits to get him automatically logged in, remove superflous windows, and connect him directly to a chat with me, it was set. And well-loved. He sent me a mix of real words and random things he created by replacing letters in Jacob or by holding down keys. In the mcabberrc, besides the obvious setting of username and password, there is:
set log_win_height = 1
set hook-post-connect = source ~/.mcabber/post-connect.rc
The hook is simply:
roster search Dad
roster hide
After awhile, Jacob wanted to switch computers. He wanted to use my laptop, and me use his computer. He refused to switch back. I asked him why. Because on your computer, my name is red. I should have known. I set it to bright white on his computer, but I think tomorrow we may need to upgrade him to the color monitor I ve been saving for just such an occasion It will be a whole new set of discoveries, I m sure. Update: I also tried out freetalk, which looked like it would meet my goals nicely. The problem was it didn t have a dedicated everything typed goes to this person mode. It did have a mode where it put the person s JID on the command line by default, but excessive use of backspace key by a 5-year-old could wipe that out and leave it in a state where he d be confused.