Well I suppose its a bit heretical running something such as VMware, but its an important piece of software at my workplace, it also allows me to run some important VMKs or modules. But at first it just wouldn't compile.

And then I found this wonderful blog about Installing VMware on Ubuntu and it worked wonderfully. The author patched the code and it all installed nicely.

The next problem was one of the services would not start. Port 8308 would refuse to work and when I went to the management screen and said Service Unavailable. I tracked this down to the Java program dying at socket binding time.

The absolute first thing you should check if you are having TCP/IP problems with Java is the sysctl path net.ipv6.bindv6only which you can check with
sysctl net.ipv6.bindv6only.

If it is 1, it might mean bad Java code network problems. And in fact this time it was the problem, changing it to 0 and the Java daemon started and stayed running and all was good.

Incidently if you use the Cisco ASA firewall Java client and it dies, use this trick for it too. On Debian systems, edit the file /etc/sysctl.d/bindv6only and set that option to 0. I don't think its the fault of the key, but bad Java code (but is there anything but bad Java code?)

My relationship with Puppet is one of love and hate. I am forced to use it simply because there is no better tool around, but I hate it in so many ways that I don t even want to start to enumerate (hint: most have to do with Ruby, actually). Today I decided to put an end to one thing that has been driving me insane: the fact that puppetd (the client) and puppetmasterd (the server) use the same working directory, /var/lib/puppet. Since I consider and would like to treatthe machine on which puppetmasterd is running just another puppet client, I was running into funky issues related to SSL certificate confusion, obscure errors, and SSL revocation horrors. The following hence assumes that you have installed or are planning to install puppetd on the machine running your puppetmaster, and that you have two fully-qualified domain names for the machine. For instance, I run puppetmaster on vera.madduck.net, and puppetmaster.madduck.net is an alias for the same machine. I ll use these names in the following as examples. The following may be Debian-specific, as I am solely using the puppet and puppetmaster packages for my experimentation and verification. Your mileage may vary, but the concept shall be the same.

1. Stop everything:

   /etc/init.d/puppetmaster stop
   /etc/init.d/puppet stop
   
   

   (also verify that you have not instructed cron to restart these services)
2. Rename the working directory:

   mv /var/lib/puppet /var/lib/puppetmaster
   
   

   and amend /etc/puppet/puppet.conf accordingly:

   [main]
   #  
   vardir=/var/lib/puppetmaster
   ssldir=$vardir/ssl
   #  
   [puppetmasterd]
   certname=puppetmaster.madduck.net
   #  
   
   

   I am doing this in [main], planning to override it for puppetd later, because puppetd is the only program which makes sense to be separated from the rest. Since only the puppetmaster needs a special certificate name, that is set specifically in the [puppetmasterd] section. If you use apache2 or nginx in front of your puppetmasters, make sure to amend the SSL file locations in the virtual host definition and restart (!) the service. You can verify that the configuration has been amended by making sure that there is no output from the following command:

   # puppetmasterd --genconfig   grep -q '/var/lib/puppet/' && echo SOMETHING IS WRONG
   
   

3. Now restart puppetmaster:

   /etc/init.d/puppetmaster start
   
   

   and verify that it starts. If your puppetmaster previously ran under a different name, it will create itself a new certificate and sign it. Since the client will get its own working directory (and thus a new SSL certificate), you want to remove all records of the old certificate:

   # puppetca --list --all
   + puppetmaster.madduck.net
   + vera.madduck.net
   # puppetca --clean vera.madduck.net
   
   

4. Change the configuration file to tell puppetd about its working directory:

   [puppetd]
   server=puppetmaster.madduck.net
   vardir=/var/lib/puppetmaster
   ssldir=$vardir/ssl
   #  
   
   

   This you can verify with the following command, which should not print anything:

   # puppetd --genconfig   grep -q '/var/lib/puppet[^/]' && echo SOMETHING IS WRONG
   
   

5. Now install puppet, or (re)start it if it s already installed:

   # /etc/init.d/puppet stop
   # puppetd --no-daemonize --onetime --verbose --waitforcert 30 &
   info: Creating a new SSL key for vera.madduck.net
   warning: peer certificate won't be verified in this SSL session
   info: Caching certificate for ca
   info: Creating a new SSL certificate request for vera.madduck.net
   # puppetca --list
   vera.madduck.net
   # puppetca --sign vera.madduck.net
   notice: Signed certificate request for vera.madduck.net
   notice: Removing file Puppet::SSL::CertificateRequest vera.madduck.net at '/var/lib/puppetmaster/ssl/ca/requests/vera.madduck.net.pem'
   # fg
   info: Caching certificate for vera.madduck.net
   info: Caching certificate_revocation_list for ca
   [ ]
   # puppetca --list --all
   + puppetmaster.madduck.net
   + vera.madduck.net
   # /etc/init.d/puppet start
   
   

   Do yourself the favour and check that it s all working.
6. Optionally, you can now clean up the client stuff in the server s working directory, for instance like this (it worked for me, but this is the sledgehammer approach):

   # /etc/init.d/puppetmaster stop
   # cd /var/lib/puppetmaster
   # tar -cf /tmp/puppetmaster.workingdir-backup.tar .
   # find ../puppet -type f -printf '%P\n'   xargs rm
   # /etc/init.d/puppetmaster start
   
   

7. If you stopped cron before (and your puppet recipes have not since restarted it):

   /etc/init.d/cron start
   
   

All done. I wish puppet, or at least Debian s puppet packages would do this by default. Please let me know if the above conversion works for you. Then I might start working on an automated migration. NP: Genesis: Selling England by the Pound

Version 0.1.0 of RcppExamples, a simple demo package for Rcpp should appear on CRAN some time tomorrow. As mentioned in the post about release 0.7.8 of Rcpp, Romain and I carved this out of Rcpp itself to provide a cleaner separation of code that implements our R / C++ interfaces (which remain in Rcpp) and code that illustrates how to use it --- which is now in RcppExamples. This also provides an easier template for people wanting to use Rcpp in their packages as it will be easier to wrap one's head around the much smaller RcppExamples package. A simple example (using the newer API) may illustrate this:

#include <Rcpp.h>
RcppExport SEXP newRcppVectorExample(SEXP vector)  
    Rcpp::NumericVector orig(vector);			// keep a copy (as the classic version does)
    Rcpp::NumericVector vec(orig.size());		// create a target vector of the same size
    // we could query size via
    //   int n = vec.size();
    // and loop over the vector, but using the STL is so much nicer
    // so we use a STL transform() algorithm on each element
    std::transform(orig.begin(), orig.end(), vec.begin(), sqrt);
    Rcpp::Pairlist res(Rcpp::Named( "result", vec),
                       Rcpp::Named( "original", orig));
    return res;
 

With essentially five lines of code, we provide a function that takes any numeric vector and returns both the original vector and a tranformed version---here by applying a square root operation. Even the looping along the vector is implicit thanks to the generic programming idioms of the Standard Template Library. Nicer still, even on misuse, exceptions get caught cleanly and we get returned to the R prompt without any explicit coding on the part of the user:

R> library(RcppExamples)
Loading required package: Rcpp
R> print(RcppVectorExample( 1:5, "new" )) # select new API
$result
[1] 1.000 1.414 1.732 2.000 2.236
$original
[1] 1 2 3 4 5
R> RcppVectorExample( c("foo", "bar"), "new" )
Error in RcppVectorExample(c("foo", "bar"), "new") :
  not compatible with INTSXP
R>

There is also analogous code for the older API in the package, but it is about three times as long, has to loop over the vector and needs to set up the execption handling explicitly. As of right now, RcppExamples does not document every class but it should already provide a fairly decent start for using Rcpp. And many more actual usage examples are ... in the over two-hundred unit tests in Rcpp.

Update: Now actually showing new rather than classic API.

I'm looking for a way to do free hosting. But I mean free as in freedom, not free as in beer. Let me explain.

When I'm using a piece of free and open-source software such as OpenOffice.org, Evolution, or anything else, I have certain possibilities, freedoms if you will, of interacting with the software beyond just consuming it. I can look at the source code to study how it works. I can rebuild it to have a higher degree of confidence that I'm actually running that code. I can fix a bug or create an enhancement. I can send the patch upstream and wait for the next release, or in important cases I can create a local build. With the emerge of new project hosting sites such as GitHub, it's getting even easier to share one's modifications so others can use them. And so on.

As a lot of software moves to the web, how will this work in the future? There are those that say that it won't, and that it will be a big problem, and that's why you shouldn't use such services. Which is what probably a lot of free-software conscious users are doing right now. But I think that in the longer run, resisting isn't going to win over the masses to free software.

First of all, of course, the software would need to be written. So a free web office suite, a free web mail suite that matches the capabilities of the leading nonfree provider, and so on. We have good starts with Identi.ca and OpenStreetMap, for example, but we'd need a lot more. Then you throw it on a machine, and people can use it. Now as a user of this service, how do I get the source code? Of course you could offer a tarball for download, and that is the approach that the AGPL license takes. One problem with that is, if you are used to apt-get source or something similar for getting the source, everyone putting a tarball on their web site in a different place isn't going to make you happy. A standardized packaging-type thing ought to be wrapped around that. Another problem is that even if you trust the site's operator that that's the source code that's actually running on your site (even without malice, it could for example be outdated against the deployed version), it probably won't contain the local configuration files and setup scripts that would allow me to duplicate the service. And if I just want to study how the program is running in actuality, there is not much I can do.

Giving everyone SSH access to the box is probably not a good idea, and won't really solve all the issues anyway. In the future, when virtualization is standardized, ubiquitous, and awesome, one might imagine that a packaging of a web service won't be "put these files on the file system and reload a daemon" but instead "put these files and this configuration on that machine and activate it". This might give rise to a new generation of Linux "distributors". Getting the source tarball or "source package" might then involve getting a snapshot of that image, which you can examine, modify, and redeploy elsewhere. That could work for OpenStreetMap, for example, modulo the space and time required for their massive database. (But you might chose to fork only the code, not the data.) But it won't be easy to do the right thing in many cases, because with a web service, there is usually other people's data on the machine as well, which would need to be masked out or something. Maybe this really can't be done correctly, and the future will be more distributed, like in the way Jabber attempted to supplant centralized services such as ICQ. Distributed web mail makes sense, distributed OpenStreetMap perhaps less so.

Ideas anyone? Does anyone perhaps have experiences with running a web service that attempts to give users the freedoms and practical benefits that are usually associated with locally installed software?

Thanks! PS: Things like that shouldn t be a problem :-)

There is an interesting conversation over at about the nature of peer production, and crowd based production over at PBS. Thankfully folks right off the bat noted that the types of activities they are addressing that range from 4chan to open source are so freaken distinct that perhaps it is not all that useful to use one moniker for them. The comments I am most fascinated by are danah s who notes:

We assume that the collective voice will be populist and, more importantly, that it will reflect the diversity of the populous. Yet, as we ve seen time and time again, certain values and attitudes and voices are over-represented in crowd-sourced activities. Who is looking out for those who aren t represented? In what ways are we reinforcing structural inequalities? What are the implications of this?

And then Clay s response:

So, to re-ask your question in a non-rhetorical way, under what
circumstances would we want to make the population of Deviant Art,
say, less white, or Linux less male, and if we wanted to do so, what
would need to happen?

What I find interesting about this discussion (and will be talking about this topic here, next week) is there not enough recognition of two related things: 1) the efforts are there (more on this soon) 2) that perhaps hacking and F/OSS in particular are not fully accessible to all and everyone because they are full-fledged, full-bodied, cultural worlds and all cultural worlds are to some degree not fully accessible and transparent for there are built on particularities, often invisible and unarticulated, forms of value. That is, just as some norms and values of Indo-Guyanese to take one random example, are not of my world, so too is hacking partially inaccessible for the fact that it is culturally configured. But I am starting to suspect that the culture-ness of these domains are often overlooked because they are overwhelmingly white, male, and chock full of computers (and so economically lucrative). All three, I suspect are (incorrectly) seen as lacking culture, as domains of rationality. (I stand rightly corrected and also forget this very fact, though I know it well from all the Brazil/Latin America Debconfs, as this diversity gets a bit lost from a pure US-European perspective, which I was assuming). Other historical factors have also produced certain distortions that don t allow us to see (easily at least) these worlds as culture-full. First is the fact that so many folks outside of this world lobbed onto F/OSS for being radical (and this is partially right in so far as its challenge to intellectual property can be seen as radical). But the portrayal or mere suggestion of these worlds as uber-democratic and populist, made people expect these groups to behave as radical egalitarian collectives. For the most part, they don t and yet never portrayed their own politics and forms of organization as such (openness comes in the form of code and technical merit). But this vision stuck and when some folks realized that larger projects, for example are very organized (which many people addressed only very late), have hierarchies (which are flexible and also allow them to function, which is I think is a good thing), and are not as diverse, there was deep disappointment that they did not conform to the sense that there was something extremely radical going on as opposed to a cultural group really into producing free software. But if I am offering a cultural alibi of sorts in which barriers to participation are to some degree a function of culture, one of the great things about the norms, values, ideas that compose culture is that there are dynamic and changing. They are alive and historical. They are pushed and pulled upon by insiders and outsiders based on wider social values. And there is an answer to these questions about diversity for there has been a dramatic, noticeable, and noteworthy push within this world, one that really started to coalesce I would say in the last year or so, to address these issues and it ranges from Python s mammoth efforts at addressing diversity (and I have been told that there was a great speech on the topic at Pycon recently), the geek feminism wiki, and smaller but increasingly common efforts such as Libre Planet s women s caucus and their funding of women to participate. So while I do think that culture goes at least part of the way to explain why these worlds are not fully open for culture limits this very domain has grown dissatisfied with its representational make-up and are leading some efforts for cultural change.

There is an interesting conversation over at about the nature of peer production, and crowd based production over at PBS. Thankfully folks right off the bat noted that the types of activities they are addressing that range from 4chan to open source are so freaken distinct that perhaps it is not all that useful to use one moniker for them. The comments I am most fascinated by are danah s who notes:

We assume that the collective voice will be populist and, more importantly, that it will reflect the diversity of the populous. Yet, as we ve seen time and time again, certain values and attitudes and voices are over-represented in crowd-sourced activities. Who is looking out for those who aren t represented? In what ways are we reinforcing structural inequalities? What are the implications of this?

And then Clay s response:

So, to re-ask your question in a non-rhetorical way, under what
circumstances would we want to make the population of Deviant Art,
say, less white, or Linux less male, and if we wanted to do so, what
would need to happen?

What I find interesting about this discussion (and will be talking about this topic here, next week) is there not enough recognition of two related things: 1) the efforts are there (more on this soon) 2) that perhaps hacking and F/OSS in particular are not fully accessible to all and everyone because they are full-fledged, full-bodied, cultural worlds and all cultural worlds are to some degree not fully accessible and transparent for there are built on particularities, often invisible and unarticulated, forms of value. That is, just as some norms and values of Indo-Guyanese to take one random example, are not of my world, so too is hacking partially inaccessible for the fact that it is culturally configured. But I am starting to suspect that the culture-ness of these domains are often overlooked because they are overwhelmingly white, male, and chock full of computers (and so economically lucrative). All three, I suspect are (incorrectly) seen as lacking culture, as domains of rationality. (I stand rightly corrected and also forget this very fact, though I know it well from all the Brazil/Latin America Debconfs, as this diversity gets a bit lost from a pure US-European perspective, which I was assuming). Other historical factors have also produced certain distortions that don t allow us to see (easily at least) these worlds as culture-full. First is the fact that so many folks outside of this world lobbed onto F/OSS for being radical (and this is partially right in so far as its challenge to intellectual property can be seen as radical). But the portrayal or mere suggestion of these worlds as uber-democratic and populist, made people expect these groups to behave as radical egalitarian collectives. For the most part, they don t and yet never portrayed their own politics and forms of organization as such (openness comes in the form of code and technical merit). But this vision stuck and when some folks realized that larger projects, for example are very organized (which many people addressed only very late), have hierarchies (which are flexible and also allow them to function, which is I think is a good thing), and are not as diverse, there was deep disappointment that they did not conform to the sense that there was something extremely radical going on as opposed to a cultural group really into producing free software. But if I am offering a cultural alibi of sorts in which barriers to participation are to some degree a function of culture, one of the great things about the norms, values, ideas that compose culture is that there are dynamic and changing. They are alive and historical. They are pushed and pulled upon by insiders and outsiders based on wider social values. And there is an answer to these questions about diversity for there has been a dramatic, noticeable, and noteworthy push within this world, one that really started to coalesce I would say in the last year or so, to address these issues and it ranges from Python s mammoth efforts at addressing diversity (and I have been told that there was a great speech on the topic at Pycon recently), the geek feminism wiki, and smaller but increasingly common efforts such as Libre Planet s women s caucus and their funding of women to participate. So while I do think that culture goes at least part of the way to explain why these worlds are not fully open for culture limits this very domain has grown dissatisfied with its representational make-up and are leading some efforts for cultural change.

Last night, in two different instances I read the claim that the England s first copyright act, the statute of Anne passed in 1710 was never intended to protect authors but to protect the reproducers like printing houses and presses investing in authors implying that printing houses loved the act. After pouring through hundreds of pages of Adrian John s history of piracy, that statement is pretty off and in fact I don t think the Statute was really about printers/booksellers or authors but the public. While licensing had all together lapsed for a period before this statute was passed, and the printing houses and book sellers were indeed clamoring loudly for an official recognition of property in literary works, they wanted a perpetuall right in literary property rooted in common and natural law. Like I am talking here about forever, not like a measly, paltry 14 years. They were not exactly thrilled at this statute (in fact, they were downright pissssssssed off) for it severely limited how long they held a property right over books. In fact, so pissed were they, they challenged the statute, went to court in 1769 (Millar v Taylor) and got what they wanted: a perpetual right to literary work. It took s a fiery Scot and bookseller by the name of Alexander Donaldson (I kind of think of him as the RMS of booksellers; he was quite a rabble rouser) to challenge Millar and he finally got his day in the highest court of the land in 1774 in Donaldson v Beckett and the outcome was that a perpetual right in books was tossed out the window. The court ruled that copyright was a limited statute. One of the lords in the case even stated Knowledge has no value or use for the solitary owner: to be enjoyed it must be communicated. Adrian John s explains the significance of this case in the following way: Copyright, they decided, was not a right of man at all. Indeed, it was almost the very opposite: an artifact, and one that replaced a prior right established by an author s work of creation. . . In terms of revolution principles, liberty won out over property Again the printers booksellers (minus the pirate ones) were not happy a bunch. Unfortunately the subsequent history is one we all know well, one in which booksellers and others with vested interests in copyrights pushed to extend property rights in all sorts of ways to get to where we are today (obviously with a lot of different historical developments), a land, time, period where perpetuity may not be forever but it is long enough to nullify the very public domain envisioned by the first copyright act. However, I think it is nonetheless important to recognize how radical in many respects the first copyright act was: given what the book printers and sellers wanted (and they were a powerful bunch). For those interested in learning more about Alexander Donaldson, I would check out his Some Thoughts on the State of Literary Property, where he rails against the London booksellers for being monopolistic and calling for a limited property right in books.

Last night, in two different instances I read the claim that the England s first copyright act, the statute of Anne passed in 1710 was never intended to protect authors but to protect the reproducers like printing houses and presses investing in authors implying that printing houses loved the act. After pouring through hundreds of pages of Adrian John s history of piracy, that statement is pretty off and in fact I don t think the Statute was really about printers/booksellers or authors but the public. While licensing had all together lapsed for a period before this statute was passed, and the printing houses and book sellers were indeed clamoring loudly for an official recognition of property in literary works, they wanted a perpetuall right in literary property rooted in common and natural law. Like I am talking here about forever, not like a measly, paltry 14 years. They were not exactly thrilled at this statute (in fact, they were downright pissssssssed off) for it severely limited how long they held a property right over books. In fact, so pissed were they, they challenged the statute, went to court in 1769 (Millar v Taylor) and got what they wanted: a perpetual right to literary work. It took s a fiery Scot and bookseller by the name of Alexander Donaldson (I kind of think of him as the RMS of booksellers; he was quite a rabble rouser) to challenge Millar and he finally got his day in the highest court of the land in 1774 in Donaldson v Beckett and the outcome was that a perpetual right in books was tossed out the window. The court ruled that copyright was a limited statute. One of the lords in the case even stated Knowledge has no value or use for the solitary owner: to be enjoyed it must be communicated. Adrian John s explains the significance of this case in the following way: Copyright, they decided, was not a right of man at all. Indeed, it was almost the very opposite: an artifact, and one that replaced a prior right established by an author s work of creation. . . In terms of revolution principles, liberty won out over property Again the printers booksellers (minus the pirate ones) were not happy a bunch. Unfortunately the subsequent history is one we all know well, one in which booksellers and others with vested interests in copyrights pushed to extend property rights in all sorts of ways to get to where we are today (obviously with a lot of different historical developments), a land, time, period where perpetuity may not be forever but it is long enough to nullify the very public domain envisioned by the first copyright act. However, I think it is nonetheless important to recognize how radical in many respects the first copyright act was: given what the book printers and sellers wanted (and they were a powerful bunch). For those interested in learning more about Alexander Donaldson, I would check out his Some Thoughts on the State of Literary Property, where he rails against the London booksellers for being monopolistic and calling for a limited property right in books.

It s been awhile since I ve posted some family/home updates, so here they are, sorta random-like Jacob is 3.5 now, and Oliver is about 8 months old at this point. Jacob Jacob has a few trends going on lately: pretending to be a cat, and inventing new games to play. He has one game he likes to play with me called tunnels and slippers. He will have me sit on the couch, put my feet up on the footstool, and crawl on top of my legs. He ll want me to bounce him, then he can crawl underneath the tunnel. The slippers part came because he started to steal my slippers one time during winter. He d struggle furiously to get it off, then shriek with delight as he ran away, listing to my indignation at having a cold foot now. Then I was supposed to run after him, pick him up in the air, and get the slipper back. Here he is, trying to steal a slipper while I happened to be holding a camera: He s said quite a few memorable quotes this winter: Dec. 8: Jacob looked outside this morning and yelled, WOW! Look at that! , ran across the room and yelled, MOM! WOW! There is snow outside this window too! WOW! Snow is ALL OVER THE PLACE! January: When told his grandparents were going back to Indiana: The kitty doesn t like that. (the pretending to be a cat thing) The next morning when he remembered they were gone: Oh. My kitty is so very sad now. February: My name is kitty. Jacob Cat! March: I didn t realize the basketball could go so far! He s been starting to make his first attempts at sounding out words. He s been interested in 2- and 3-digit numbers, trying to figure out how to tell time, and asking me the German words for all sorts of things. (And showing some disappointment with my German vocabulary.) Jacob talks about Oliver as a sweet baby, and is often very caring about him. Jacob wants to make sure Oliver has *plenty* of toys. Though there have been some times when Jacob has pushed Oliver over. Here s the Christmas photo of the two of them: And another: Jacob has also started to pick up an interest in singing: his favorite song is one that he made up, and it goes Engine, engine, passenger car I still sing a song to Jacob at bedtime each night, but Home On the Range doesn t cut it anymore. No, now I have a make up a song on the spot on a topic of Jacob s choosing each night. Last night he chose a song about the ceiling light and the smoke detector. Other nights, it s been about his toy butterfly, his night light, various smoke detectors (that s a popular one), or something from his day. Oliver Oliver recently has started being able to sit up by himself. He can also scoot all over the room. I think he s saying dad , but Terah disagrees and thinks he s just babbling. Her case wasn t helped the day I asked him, Are you saying Dad? He got a big smile after the question and laughed at me. He loves playing peek-a-boo, playing with us or playing by himself. Oliver has been eating the mashed-up baby food for awhile. Terah recently gave him a rice puff, and he put it in his mouth but wasn t sure what to do with it. He has been taking quite an interest in watching the rest of us eat, though, and sits at the table with us in his high chair.

The meeting agenda is already posted for tonight s (Wednesday s) SPI board IRC meeting which will be at 2000 UTC. It s another pretty lean meeting, with only some minutes to approve, so why not come along and let the board know what you think they should be doing to promote free and open source software?

Exactly 1 month ago ;-)

read more

Inspired by a conversation about interview coding tasks from a list I'm on, I present the following - I considered it too long to email there. It took me longer than I expected to write; my x86 assembly is quite rusty. I'm not claiming it's pretty, but it fits in a single sector and most of the overhead is actually ELF structures.

; nasm -f elf fizzbuzz.asm
; ld -melf_i386 -s -o fizzbuzz fizzbuzz.o
; ./fizzbuzz
section .data
fizz	db	" fizz"
fizzlen	equ	$ - fizz
buzz	db	" buzz"
buzzlen	equ	$ - buzz
num	db	"   "
numend	equ	$ - 1
numlen	equ	$ - num
nl	db	0xa
nllen	equ	$ - nl
curnum	db 1
section .text
	global _start
_start:
	mov ax, [curnum]
	call printnum
	mov ax, [curnum]
	mov cx, 3
	xor dx, dx
	div cx
	cmp dx, 0
	jnz notfizz
	mov edx, fizzlen
	mov ecx, fizz
	call printstr
notfizz:
	mov ax, [curnum]
	mov cx, 5
	xor dx, dx
	div cx
	cmp dx, 0
	jnz notbuzz
	mov edx, buzzlen
	mov ecx, buzz
	call printstr
notbuzz:
	mov edx, nllen
	mov ecx, nl
	call printstr
	inc BYTE [curnum]
	cmp BYTE [curnum], 100
	jle _start
	xor ebx, ebx
	mov eax, 1
	int 0x80
printnum:
	mov edi, numend
	mov cx, 10
p1:
	xor edx, edx
	div cx
	add dx, '0'
	mov [edi], dl
	dec edi
	cmp ax, 0
	jne p1
	mov ecx, num
	mov edx, numlen
printstr:
	mov ebx, 1
	mov eax, 4
	int 0x80
	ret

Since I last mentioned music back in January I have accumulated another ten albums and unlike last time where there were only a couple of stand outs, this time I have he opposite problem.

Last week I was having a conversation with a friend, and while we were discussing some things I mentioned some Ubuntu tools that were completely natural for me since I use them every day in Ubuntu development and for my work, but he was completely amazed by them, so I decided to blog on the subject so more people can know about them. I m going to start with the one that impressed him the most: qa-regression-testing branch The QA and security team maintain a test suite to check for regressions in packages they are updating. These tests are written with python unittest. Most of this test suite can be a little harmful for a production system, so it s recommended to run it using a chroot environment or a virtual machine and to help with that there is a make-test-tarball script. From the script:

export HOSTS= sec-intrepid-amd64 sec-hardy-amd64 sec-dapper-amd64
export TEST=test-glibc-security
./make-test-tarball $TEST.py
for i in $HOSTS; do
scp /tmp/qrt-$TEST.tar.gz $i.local:
done
for i in $HOSTS; do
ssh -t $i.local hostname; rm -rf qrt-$TEST; tar zxf qrt-$TEST.tar.gz; cd qrt-$TEST && ./$TEST.py -v
done

As you can imagine HOSTS are the hostnames of the virtual machines where the tests are going to run. Writing those test isn t rocket science, you just need a little knowledge of the package and the functionality you want to test and python unittesting, there is even a skeleton script that can help you start writing your test. Hope this information is useful for you! And as usual, patches are welcome!

Every year, at least one person asks me to run for DPL. I don't remember quite when this started, but it was more than five years ago. I have some standard responses: I don't have time to do a good job, and why commit if you know you can't do a good job? That's just irresponsible. Usually I get some feeble relativism back for that one. Oddly enough, this is the one year that I could actually make the time for it, but it is still not a good idea. Also I do not wish to self-nominate. I am told that everyone is too autistic to understand why, so I assume that by observing the timing of nominations (in specific, who stands at the beginning of the nomination period and who waits until close to the last minute), no moro-ethical conclusions will be drawn. To belabor this point: if the culture of self-nomination were fixed, I would actually nominate people I think would do a good job. At present, I cannot, and I resent that. Furthermore, you do not really want me as DPL. There are oodles and oodles of things I think are fundamentally flawed and I would try to fix them. I would not take advice from predecessors because I think that only perpetuates years of flawed DPL behavior. I would not entertain requests in private because there is far, far, far too much backroom discussion right now, and unfortunately it is actively encouraged and promoted by many people in power. Such departures would be extremely unpopular. I would be pleased if Stefano runs unopposed. I think that there is a good chance he will do a decent job.

Well my last post elicited a response from Mr Lougher the squashfs author. Just not one I was expecting. Apparently he did receive one of my emails (I sent five in total) to which he has not replied as I have accidentally come across as critical.

Temporarily disabling file caching Does it happen to you that you cp a big, big file (say, similar in order of magnitude to the amount of RAM) and the system becomes rather unusable? It looks like Linux is saying "let's cache this", and as you copy it will try to free more and more ram in order to cache the big file you're copying. In the end, all the RAM is full with file data that you are not going to need. This varies according to how /proc/sys/vm/swappiness is set. I learnt about posix_fadvise and I tried to play with it. The result is this preloadable library that hooks into open(2) and fadvises everything as POSIX_FADV_DONTNEED. It is all rather awkward. fadvise in that way will discard existing cache pages if the file is already cached, which is too much. Ideally one would like to say "don't cache this because of me" without stepping on the feet of other system activities. Also, I found I need to also hook into write(2) and run fadvise after every single write, because you can't fadvise a file to be written in its entirety, unless you pass fadvise the file size in advance. But the size of the output file cannot be known by the preloaded library, so meh. So, now I can run: nocache cp bigfile someplace/ without trashing the existing caches. I can also run nocache tar zxf foo.tar.gz and so on. I wish, of course, that there were no need to do so in the first place. Here is the nocache library source code, for reference:

/*
 * nocache - LD_PRELOAD library to fadvise written files to not be cached
 *
 * Copyright (C) 2009--2010 Enrico Zini <enrico@enricozini.org>
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program; if not, write to the Free Software
 * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 */
#define _XOPEN_SOURCE 600
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <dlfcn.h>
#include <stdarg.h>
#include <errno.h>
#include <stdio.h>
typedef int (*open_t)(const char*, int, ...);
typedef int (*write_t)(int fd, const void *buf, size_t count);
int open(const char *pathname, int flags, ...)
 
    static open_t func = 0;
    int res;
    if (!func)
        func = (open_t)dlsym(RTLD_NEXT, "open");
    // Note: I wanted to add O_DIRECT, but it imposes restriction on buffer
    // alignment
    if (flags & O_CREAT)
     
        va_list ap;
        va_start(ap, flags);
        mode_t mode = va_arg(ap, mode_t);
        res = func(pathname, flags, mode);
        va_end(ap);
      else
        res = func(pathname, flags);
    if (res >= 0)
     
        int saved_errno = errno;
        int z = posix_fadvise(res, 0, 0, POSIX_FADV_DONTNEED);
        if (z != 0) fprintf(stderr, "Cannot fadvise on %s: %m\n", pathname);
        errno = saved_errno;
     
    return res;
 
int write(int fd, const void *buf, size_t count)
 
    static write_t func = 0;
    int res;
    if (!func)
        func = (write_t)dlsym(RTLD_NEXT, "write");
    res = func(fd, buf, count);
    if (res > 0)
     
        int saved_errno = errno;
        int z = posix_fadvise(fd, 0, 0, POSIX_FADV_DONTNEED);
        if (z != 0) fprintf(stderr, "Cannot fadvise during write: %m\n");
        errno = saved_errno;
     
    return res;
 

Updates Steve Schnepp writes:

Robert Love did a O_STREAMING patch for 2.4. It wasn't merged in 2.6 since POSIX_FADV_NOREUSE should be used instead. But unfortunatly it's currently mapped either as WILLNEED or as a noop. It seems that there is a google code project that has spawned to control this.

The Threat Bruce Schneier s blog post about the Mariposa Botnet has an interesting discussion in the comments about how to make a secure system [1]. Note that the threat is considered to be remote attackers, that means viruses and trojan horses which includes infected files run from USB devices (IE you aren t safe just because you aren t on the Internet). The threat we are considering is not people who can replace hardware in the computer (people who have physical access to it which includes people who have access to where it is located or who are employed to repair it). This is the most common case, the risk involved in stealing a typical PC is far greater than the whatever benefit might be obtained from the data on it a typical computer user is at risk of theft only for the resale value of a second-hand computer. So the question is, how do can we most effectively use free software to protect against such threats? The first restriction is that the hardware in common use is cheap and has little special functionality for security. Systems that have a TPM seem unlikely to provide a useful benefit due to the TPM being designed more for Digital Restrictions Management than for protecting the user and due to TPM not being widely enough used. The BIOS and the Bootloader It seems that the first thing that is needed is a BIOS that is reliable. If an attacker manages to replace the BIOS then it could do exciting things like modifying the code of the kernel at boot time. It seems quite plausible for the real-mode boot loader code to be run in a VM86 session and to then have it s memory modified before it starts switches to protected mode. Every BIOS update is a potential attack. Coreboot replaces the default PC BIOS, it initialises the basic hardware and then executes an OS kernel or boot loader [2] (the Coreboot Wikipedia page has a good summary). The hardest part of the system startup process is initialising the hardware, Coreboot has that solved for 213 different motherboards. If engineers were allowed to freely design hardware without interference then probably a significant portion of the computers in the market would have a little switch to disable the write line for the flash BIOS. I heard a rumor that in the days of 286 systems a vendor of a secure OS shipped a scalpel to disable the hardware ability to leave protected mode, cutting a track on the motherboard is probably still an option. Usually once a system is working you don t want to upgrade the BIOS. One of the payloads for Coreboot is GRUB. The Grub Feature Requests page has as it s first entry Option to check signatures of the bootchain up to the cryptsetup/luksOpen: MBR, grub partition, kernel, initramfs [3]. Presumably this would allow a GPG signature to be checked so that a kernel and initrd would only be used if they came from a known good source. With this feature we could only boot a known good kernel. How to run User Space The next issue is how to run the user-space. There has been no shortage of Linux kernel exploits and I think it s reasonable to assume that there will continue to be a large number of exploits. Some of the kernel flaws will be known by the bad guys for some time before there are patches, some of them will have patches which don t get applied as quickly as desired. I think we have to assume that the Linux kernel will be compromised. Therefore the regular user applications can t be run against a kernel that has direct hardware access. It seems to me that the best way to go is to have the Linux kernel run in a virtual environment such as Xen or KVM. That means you have a hypervisor (Xen+Linux or Linux+KVM+QEMU) that controls the hardware and creates the environment for the OS image that the user interacts with. The hypervisor could create multiple virtual machines for different levels of data in a similar manner to the NSA NetTop project, not that this is really a required part of solving the general secure Internet terminal problem but as it would be a tiny bit of extra work you might as well do it. One problem with using a hypervisor is that the video hardware tends to want to use features such as bus-mastering to give best performance. Apparently KVM has IOMMU support so it should be possible to grant a virtual machine enough hardware access to run 3D graphics at full speed without allowing it to break free. Maintaining the Virtual Machine Image Google has a good design for the ChromiumOS in terms of security [4]. They are using CGroups [5] to control access to device nodes in jails, RAM, CPU time, and other resources. They also have some intrusion detection which can prompt a user to perform a hardware reset. Some of the features would need to be implemented in a different manner for a full desktop system but most of the Google design features would work well. For an OS running in a virtual machine when an intrusion is detected it would be best to have the hypervisor receive a message by some defined interface (maybe a line of text printed on the console ) and then terminate and restart the virtual machine. Dumping the entire address space of the virtual machine would be a good idea too, with typical RAM sizes at around 4G for laptops and desktops and typical storage sizes at around 200G for laptops and 2T for new desktops it should be easy to store a few dumps in case they are needed. The amount of data received by a typical ADSL link is not that great. Apart from the occasional big thing (like downloading a movie or listening to Internet radio for a long time) most data transfers are from casual web browsing which doesn t involve that much data. A hypervisor could potentially store the last few gigabytes of data that were received which would then permit forensic analysis if the virtual machine was believed to be compromised. With cheap SATA disks in excess of 1TB it would be conceivable to store the last few years of data transfer (with downloaded movies excluded) but such long-term storage would probably involve risks that would outweigh the rewards, probably storing no more than 24 hours of data would be best. Finally in terms of applying updates and installing new software the only way to do this would be via the hypervisor as you don t want any part of the virtual machine to be able to write to it s data files or programs. So if the user selects to install a new application then the request please install application X would have to be passed to the hypervisor. After the application is installed a reboot of the virtual machine would be needed to apply the change. This is a common experience for mobile phones (where you even have to reboot if the telco changes some of their network settings) and it s something that MS-Windows users have become used to but it would get a negative reaction from the more skilled Linux users. Would this be Accepted? The question is, if we built this would people want to use it? The NetTop functionality of having two OSs interchangeable on the one desktop would attract some people. But most users don t desire greater security and would find some reason to avoid this. They would claim that it lowered the performance (even for aspects of performance where benchmarks revealed no difference) and claim that they don t need it. At this time it seems that computer security isn t regarded as a big enough problem for users. It seems that the same people who will avoid catching a train because one mugging made it to the TV news will happily keep using insecure computers in spite of the huge number of cases of fraud that are reported all the time.

• [1] http://www.schneier.com/blog/archives/2010/03/mariposa_botnet.html
• [2] http://www.coreboot.org/Welcome_to_coreboot
• [3] http://grub.enbug.org/FeatureRequests
• [4] http://sites.google.com/a/chromium.org/dev/chromium-os/chromiumos-design-docs/system-hardening
• [5] http://lxr.linux.no/linux+v2.6.30/Documentation/cgroups/