So I've got combat, I've got basic potions that restore health and I've got items such as weapons or shields which can affect your combat stats. Try as I might, I cannot get ncurses to give me a yellow background for some unfathomable reason. It just comes out brown no matter what. Ironically, in the next Konsole over, I have the alpine ncurses-based mua taunting me with a yellow background. I'll look in its source (yay open source!) to see how its done sometime but for now brown will have to do. C++ is proving a joy to work with as always. Ok It's probably just me; I use it so infrequently, I keep having to relearn how to do multiple inheritance and abstract base classes each time. I am rather proud of figuring out how to access class members from a signal handler without having to look it up. Now if I can quickly tie up some loose ends, I can spend the last two days maybe making the dungeon generation more roguelike.

There is an interesting conversation over at about the nature of peer production, and crowd based production over at PBS. Thankfully folks right off the bat noted that the types of activities they are addressing that range from 4chan to open source are so freaken distinct that perhaps it is not all that useful to use one moniker for them. The comments I am most fascinated by are danah s who notes:

We assume that the collective voice will be populist and, more importantly, that it will reflect the diversity of the populous. Yet, as we ve seen time and time again, certain values and attitudes and voices are over-represented in crowd-sourced activities. Who is looking out for those who aren t represented? In what ways are we reinforcing structural inequalities? What are the implications of this?

And then Clay s response:

So, to re-ask your question in a non-rhetorical way, under what
circumstances would we want to make the population of Deviant Art,
say, less white, or Linux less male, and if we wanted to do so, what
would need to happen?

What I find interesting about this discussion (and will be talking about this topic here, next week) is there not enough recognition of two related things: 1) the efforts are there (more on this soon) 2) that perhaps hacking and F/OSS in particular are not fully accessible to all and everyone because they are full-fledged, full-bodied, cultural worlds and all cultural worlds are to some degree not fully accessible and transparent for there are built on particularities, often invisible and unarticulated, forms of value. That is, just as some norms and values of Indo-Guyanese to take one random example, are not of my world, so too is hacking partially inaccessible for the fact that it is culturally configured. But I am starting to suspect that the culture-ness of these domains are often overlooked because they are overwhelmingly white, male, and chock full of computers (and so economically lucrative). All three, I suspect are (incorrectly) seen as lacking culture, as domains of rationality. (I stand rightly corrected and also forget this very fact, though I know it well from all the Brazil/Latin America Debconfs, as this diversity gets a bit lost from a pure US-European perspective, which I was assuming). Other historical factors have also produced certain distortions that don t allow us to see (easily at least) these worlds as culture-full. First is the fact that so many folks outside of this world lobbed onto F/OSS for being radical (and this is partially right in so far as its challenge to intellectual property can be seen as radical). But the portrayal or mere suggestion of these worlds as uber-democratic and populist, made people expect these groups to behave as radical egalitarian collectives. For the most part, they don t and yet never portrayed their own politics and forms of organization as such (openness comes in the form of code and technical merit). But this vision stuck and when some folks realized that larger projects, for example are very organized (which many people addressed only very late), have hierarchies (which are flexible and also allow them to function, which is I think is a good thing), and are not as diverse, there was deep disappointment that they did not conform to the sense that there was something extremely radical going on as opposed to a cultural group really into producing free software. But if I am offering a cultural alibi of sorts in which barriers to participation are to some degree a function of culture, one of the great things about the norms, values, ideas that compose culture is that there are dynamic and changing. They are alive and historical. They are pushed and pulled upon by insiders and outsiders based on wider social values. And there is an answer to these questions about diversity for there has been a dramatic, noticeable, and noteworthy push within this world, one that really started to coalesce I would say in the last year or so, to address these issues and it ranges from Python s mammoth efforts at addressing diversity (and I have been told that there was a great speech on the topic at Pycon recently), the geek feminism wiki, and smaller but increasingly common efforts such as Libre Planet s women s caucus and their funding of women to participate. So while I do think that culture goes at least part of the way to explain why these worlds are not fully open for culture limits this very domain has grown dissatisfied with its representational make-up and are leading some efforts for cultural change.

There is an interesting conversation over at about the nature of peer production, and crowd based production over at PBS. Thankfully folks right off the bat noted that the types of activities they are addressing that range from 4chan to open source are so freaken distinct that perhaps it is not all that useful to use one moniker for them. The comments I am most fascinated by are danah s who notes:

We assume that the collective voice will be populist and, more importantly, that it will reflect the diversity of the populous. Yet, as we ve seen time and time again, certain values and attitudes and voices are over-represented in crowd-sourced activities. Who is looking out for those who aren t represented? In what ways are we reinforcing structural inequalities? What are the implications of this?

And then Clay s response:

So, to re-ask your question in a non-rhetorical way, under what
circumstances would we want to make the population of Deviant Art,
say, less white, or Linux less male, and if we wanted to do so, what
would need to happen?

What I find interesting about this discussion (and will be talking about this topic here, next week) is there not enough recognition of two related things: 1) the efforts are there (more on this soon) 2) that perhaps hacking and F/OSS in particular are not fully accessible to all and everyone because they are full-fledged, full-bodied, cultural worlds and all cultural worlds are to some degree not fully accessible and transparent for there are built on particularities, often invisible and unarticulated, forms of value. That is, just as some norms and values of Indo-Guyanese to take one random example, are not of my world, so too is hacking partially inaccessible for the fact that it is culturally configured. But I am starting to suspect that the culture-ness of these domains are often overlooked because they are overwhelmingly white, male, and chock full of computers (and so economically lucrative). All three, I suspect are (incorrectly) seen as lacking culture, as domains of rationality. (I stand rightly corrected and also forget this very fact, though I know it well from all the Brazil/Latin America Debconfs, as this diversity gets a bit lost from a pure US-European perspective, which I was assuming). Other historical factors have also produced certain distortions that don t allow us to see (easily at least) these worlds as culture-full. First is the fact that so many folks outside of this world lobbed onto F/OSS for being radical (and this is partially right in so far as its challenge to intellectual property can be seen as radical). But the portrayal or mere suggestion of these worlds as uber-democratic and populist, made people expect these groups to behave as radical egalitarian collectives. For the most part, they don t and yet never portrayed their own politics and forms of organization as such (openness comes in the form of code and technical merit). But this vision stuck and when some folks realized that larger projects, for example are very organized (which many people addressed only very late), have hierarchies (which are flexible and also allow them to function, which is I think is a good thing), and are not as diverse, there was deep disappointment that they did not conform to the sense that there was something extremely radical going on as opposed to a cultural group really into producing free software. But if I am offering a cultural alibi of sorts in which barriers to participation are to some degree a function of culture, one of the great things about the norms, values, ideas that compose culture is that there are dynamic and changing. They are alive and historical. They are pushed and pulled upon by insiders and outsiders based on wider social values. And there is an answer to these questions about diversity for there has been a dramatic, noticeable, and noteworthy push within this world, one that really started to coalesce I would say in the last year or so, to address these issues and it ranges from Python s mammoth efforts at addressing diversity (and I have been told that there was a great speech on the topic at Pycon recently), the geek feminism wiki, and smaller but increasingly common efforts such as Libre Planet s women s caucus and their funding of women to participate. So while I do think that culture goes at least part of the way to explain why these worlds are not fully open for culture limits this very domain has grown dissatisfied with its representational make-up and are leading some efforts for cultural change.

Version 0.7.8 of the Rcpp R / C++ interface classes is now on CRAN and in Debian. As of right now. Debian has already built packages for eight more architectures; and CRAN has built the Windows binary. Oh, and cran2deb had Debian packages for 'testing' before I was done with the blog entry. This is a minor feature release based on a over three weeks of changes that are summarised below in the extract from the NEWS file. Some noteworthy highlights are

• something that isn't there: we have split most of the example code and their manual pages off into a new package RcppExamples which can now be released given that 0.7.8 is out
• another new package RcppArmadillo will also be forthcoming shortly: it shows how to use Rcpp with Conrad Sanderson's excellent Armadillo C++ library for linear algebra; this required some internal code changes to seamlessly pass data from R via Rcpp to Armadillo and back;
• there is a new example fastLm using Armadillo for faster (than lm() or lm.fit()) linear model fits
• yet more internal improvements to the class hierarchy as detailed below; more support for STL iterators and algorithms;
• more build fixes; paths with spaces in the name should now be tolerated
• and last but not least a new introduction / overview vignette based on a just-submitted paper on Rcpp.

The full NEWS entry for this release follows:

0.7.8   2010-03-09
    o	All vector classes are now generated from the same template class
    	Rcpp::Vector<int rtype="RTYPE"> where RTYPE is one of LGLSXP, RAWSXP, STRSXP,
    	INTSXP, REALSXP, CPLXSXP, VECSXP and EXPRSXP. typedef are still 
    	available : IntegerVector, ... All vector classes gain methods 
    	inspired from the std::vector template : push_back, push_front, 
    	erase, insert
    	
    o	New template class Rcpp::Matrix<rtype> deriving from 
    	Rcpp::Vector<rtype>. These classes have the same functionality
    	as Vector but have a different set of constructors which checks
    	that the input SEXP is a matrix. Matrix<rtype> however does/can not
    	guarantee that the object will allways be a matrix. typedef 
    	are defined for convenience: Matrix<intsxp> is IntegerMatrix, etc...
    	
    o	New class Rcpp::Row<int rtype="RTYPE"> that represents a row of a matrix
    	of the same type. Row contains a reference to the underlying 
    	Vector and exposes a nested iterator type that allows use of 
    	STL algorithms on each element of a matrix row. The Vector class
    	gains a row(int) method that returns a Row instance. Usage 
    	examples are available in the runit.Row.R unit test file
    	
    o	New class Rcpp::Column<int rtype="RTYPE"> that represents a column of a 
    	matrix. (similar to Rcpp::Row<int rtype="RTYPE">). Usage examples are 
    	available in the runit.Column.R unit test file
    o	The Rcpp::as template function has been reworked to be more 
    	generic. It now handles more STL containers, such as deque and 
    	list, and the genericity can be used to implement as for more
    	types. The package RcppArmadillo has examples of this
    o   new template class Rcpp::fixed_call that can be used in STL algorithms
	such as std::generate.
    o	RcppExample et al have been moved to a new package RcppExamples;
        src/Makevars and src/Makevars.win simplified accordingly
    o	New class Rcpp::StringTransformer and helper function 
    	Rcpp::make_string_transformer that can be used to create a function
    	that transforms a string character by character. For example
    	Rcpp::make_string_transformer(tolower) transforms each character
    	using tolower. The RcppExamples package has an example of this.
        
    o	Improved src/Makevars.win thanks to Brian Ripley
    o	New examples for 'fast lm' using compiled code: 
        - using GNU GSL and a C interface
        - using Armadillo (http://arma.sf.net) and a C++ interface
        Armadillo is seen as faster for lack of extra copying
    o	A new package RcppArmadillo (to be released shortly) now serves 
        as a concrete example on how to extend Rcpp to work with a modern 
	C++ library such as the heavily-templated Armadillo library
    o	Added a new vignette 'Rcpp-introduction' based on a just-submitted 
        overview article on Rcpp

As always, even fuller details are in the ChangeLog on the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

Update: Two links corrected.

Last month I converted my laptop to btrfs to get some experience using this new, still experimental file system. The bugs tell the story:

• #567065: initramfs-tools: avoid mount -t unknown
  I provided a patch which has been accepted. The underlying bug in klibc still remains to be fixed (#548047).
• #540786: update-grub doesn't work with root fs on btrfs
  There are patches, but not acceptably clean ones. (For values of "acceptably clean" that are somehow more important than "it boots", anyway.)

At this point I had a booting, working btrfs system. The bugs seen over the next weeks were strange:

• #567135: python-dbus went missing from python-support directory on upgrade
  Was the filesystem losing information? Maybe.
• #569058: dbus: helper not setuid; breaks gnome-power-manager
  Both the package maintainer and I am fairly sure it's not really a bug in the package. And another btrfs user also saw the problem.
• #568908: gnome-terminal: uses iceweasel for urls despite gconf being configured to use chromium
  Hard to say if btrfs was really involved since I can't point at a file that was wrong.

In the meantime, I did some initial work toward supporting btrfs in d-i, writing a basic partman-btrfs. One of the interesting open questions is how best to use subvolumes to split up a single btrfs filesystem. If the installer defines the right set of subvolumes, upgrades to be rolled back, without also rolling back /home and /srv. The remaining parts needed for d-i to support btrfs are fairly tractable, but are waiting until root on btrfs is bootable, and until the filesystem is not experimental. I gave up on the current experiement when I ran debsums on my laptop. (So that long thread was good for something!) It didn't find any corrupt files. But it did find a lot of md5sums files that were missing. Some of the packages with missing md5sums were upgraded just this week, and the md5sums files must have gone missing since. I have not seen anything that I can say with certainty was caused by btrfs, and not by some other cause, but since I can't trust the data on this laptop now (although git tells me everything in my home directory is ok), I'm reinstalling it, and saying aloha to btrfs, for now.

Just a kind thank you note to the gThumb authors for completely breaking my workflow by re-inventing the paradigm used to save imported pictures. Until now, all my pictures landed in a predictable location, using a predictable filename pattern that was easily searchable. Not anymore. Now, files land into my home directory, according to some recursive folder pattern that further complicates searching for files and requires a few more clicks to accomplish. Dammit! Couldn't you at least make this configurable, so that those of us who prefer to retain the old paradigm can?! Note: re-inventing an application's paradigms is always a very bad idea. If you're a software developer who is reading this, please keep it in mind and go scratch your itch to change the world somewhere else. Thank you. Post Scriptum Many thanks to Damon Lynch for pointing me to his own professional picture importer called Rapid. This is an extremely configurable importing tool and, lo and behold, Damon even offers builds for Ubuntu via his PPA! Still, the consequence of this mess is that migrating to Rapid means that I'll be loosing gThumb's simple but extremely efficient editing tools. To me, one strength of gTthumb was this unique combination of picture importing with basic editing tools. Now, I'm forced to split these interconnected tasks, simply because someone chose to completely rethink gThumb's paradigms. I'm of course aware of Gimp's existence, but repeated attempts at mastering it made me conclude that it's entirely the wrong software for my needs and essentially overkill. By contrast, gThumb offers just enough tools to enable someone to crop images to useful sizes and to adjust color balances in easy steps; it does the job without hassle, which is not the case with Gimp. Thinking out loud, it is precisely on days like these that the urge to create my own Linux distribution keeps on coming to mind. Retaining consistent paradigms in the desktop environment and applications that I use, not to mention maintaining the number of duplicate libraries to a bare minimal, has been a constant struggle and, noticing how some developers' urge to re-invent the wheel every other day, using whatever new programming language of the day, persistently takes precedence over keeping system resource consumption to a bare minimum and over preserving user sanity, I'm slowly coming to the conclusion that Free Software has veered way too far into the bazaar and urgently needs a copious amount of cathedral to make it usable for mere mortals again. Some dpkg-based distribution where the only scripting language allowed is Bourne shell and the only programming languages C or C++ comes to mind. Of course, this would also require porting popular application from e.g. Java, Python, etc. which would be a colossal amount of work. Still, I think that the time has come for this to happen. As an added bonus, this would make applications usable again on embedded devices with spartan CPU, RAM and storage resources, so this project could generate huge benefits to the embedded Linux industry. Based on my experience at my previous jobs, I have a rather clear picture (pun intended) of what needs to be done and of who I would hire to make it happen. What I'm missing are investors. Who's with me?

I know I ve been on something of a religion streak on the blog of late, and this will be the last such post for awhile. I first hear of Harvey Cox s book The Future of Faith during an excellent hour-long interview with NPR s Diane Rehm. It was intriguing enough that I bought the Kindle edition of the book and read it. The title of the book is both very accurate and rather misleading. A lot of the book and, to me, the most fascinating parts of it focus on the history of faith. Cox s repeated point is that we are only now regaining a notion of faith that the earliest Christians had, and it is a notion that happens to be compatible with modern science and incompatible with fundamentalism and intolerance in all its stripes. Throughout this post, it should be understood that quotes or passages are from the book. Cox is so quotable that a good chunk of this review will be showing you some of his quotes, with a bit of discussion around them. I very much enjoyed this book, and highly recommend it. Faith vs. Belief

It is true that for many people faith and belief are just two words for the same thing. But they are not the same and it is important to clarify the difference. Faith is about deep-seated confidence. In everyday speech we usually apply it to people we trust or the values we treasure a matter of what the Hebrews spoke of as the heart. Belief, on the other hand, is more like opinion. We often use the term to express a degree of uncertainty We can believe something to be true without it making much difference to us, but we place our faith only in something that is vital for the way we live.

This is an important distinction, and if you stop and think about it, Cox is arguing with a common notion about faith almost from page 1. Faith isn t about intellectual assent to a set of propositions. It s about what we hold dear, what we think works for us in life. Creeds

Creeds are clusters of beliefs. But Christianity is not a history of creeds. It is the story of a people of faith who sometimes cobbled together creeds out of beliefs. It is also the history of equally faithful people who questioned, altered, and discarded those same creeds But both the doctrinal canons and the architectural constructions are means to an end. Making either the defining element warps the underlying reality of faith.

Cox here reinforces the point that Christianity isn t about believing certain statements, and it isn t even about a literal (or not) reading of the Bible. It s what C. S. Lewis talked about as the inward transformation in onesself. Creeds, such as the Nicene Creed, are rather irrelevant to him. Cox separates the history of Christianity into three periods: the age of faith, stretching from the time of Jesus only a few centuries until Constantine; the age of belief, stretching from Constantine until the 20th century; and the age of the spirit, now dawning. During the age of faith, their sharing in the living Spirit of Christ united Christians with each other, and faith meant hope and assurance in the dawning of a new era of freedom, healing, and compassion that Jesus had demonstrated. Cox makes the point that doctrinal questions just weren t all that important back then, and though differences existed, they weren t considered to be fundamental to the religion. Confidence in Christ was their primary orientation, and hope for his [earthly] Kingdom their motivating drive. Further, he argues that the age of the spirit is a return to this earlier age, albeit with modern twists.

Christianity is growing faster than it ever has before, but mainly outside the West and in movements that accent spiritual experience, discipleship, and hope; pay scant attention to creeds; and flourish without hierarchies. We are now witnessing the beginning of a post-Constantinian era.

Cox describes a person that described himself as a practicing Christian, not always a believing one. He suggests that the belief/non-believer statement is a disservice to Christianity and to other religions. He then quoted a Catholic bishop as saying: The line between belief and unbelief runs through the middle of each one of us, including myself, a bishop of the church. In other words, The experience of the divine is displacing theories about it. Faith and Belief in Bible reading

Creation myths such as the first chapters of Genesis were not primarily composed to answer the how or when questions. They are not scientific accounts, even though their poetical language, when read literally (which is always a mistake), may sound that way. Rather, they grapple with the linked mysteries of both why there is a universe and what our place in it is They are more like lyrical cantatas, symphonies of symbols through which humans have tried to make sense of their place in the world This is where the distinction between faith and belief is vital. These stories are literally not to be believed. They are, rather, artifacts human beings have crafted to try to wring some meaning from the mystery. They are not themselves the mystery.

I liken this to Michael Crichton s novel Jurassic Park. If you were to read it 1000 years in the future, it might not have been conveniently shelved above the word fiction. Would a reader in the future know that it was not meant to be a literal description of facts? I think sometimes we make this mistake when we read the Bible. Note, though, that although we all understand that Jurassic Park wasn t meant to be a literal description of facts, it seems to have been valued by quite a large part of society. And it didn t even address big mysteries. Cox argues against ridding ourselves of the creation myths, suggesting that they are an important reminder that we are similar to humans who grappled with the same big questions centuries ago as we do today.

The ill-advised transmuting of symbols into a curious kind of facts has created an immense obstacle to faith for many thoughtful people. Instead of helping them confront the great mystery, it has effectively prevented them from doing so the objective knowledge science rightly insists on is not the only kind of knowledge human beings need Faith, although it is evoked by the mystery that surrounds us, is not the mystery itself.

Constantine and the Age of Belief

One of the most devastating blunders made by the church, especially as the Age of Belief began, was to insist that the Spirit is present only in believers.

Cox spends a lot of time covering the very interesting topic of how and why the church moved to the Age of Belief. His central thesis is that money, power, and prestige were primarily responsible, and that an unrighteous collusion between bishops and Constantine, each using Christianity for their own purposes, finally made it happen. This is very interesting stuff, but this post is too long already, so I will not spend a lot of time on it. I found the Council of Nicea to be particularly interesting, considering that the Nicean Creed came about partially by exile or execution of those Christians that disagreed with it. Cox also points out that there never was a single early Christianity ; there were many, and the idea of heresy was unknown.

The time is ripe to retrieve the term Way for Christianity and followers of the Way for Christians. It is at once more accurate, more original, and more contemporary than believers.

To the future Cox describes attending a meeting of the church in Hong Kong in 2003, and uses it as a metaphor for the future of faith:

Their idea of interfaith dialogue was to work with their fellow Asians of whatever religion to advance the Kingdom that Jesus had inspired them, as Christians, to strive for, regardless of what the others called it. They were neither fundamentalist nor modernist. They seemed more attuned to the element of mystery at the core of Christianity and to its vision of justice. They were also clearly impatient with many of the disputes that preoccupy the different wings of the American churches.

Conclusion I found this book to be both enlightening and informative. I highly recommend it, even if you disagree with some of Cox s conclusions. It is a fascinating view into how the world s largest religion evolved over the years, and a candid look at the mistakes it has made in that time.

Last week I was having a conversation with a friend, and while we were discussing some things I mentioned some Ubuntu tools that were completely natural for me since I use them every day in Ubuntu development and for my work, but he was completely amazed by them, so I decided to blog on the subject so more people can know about them. I m going to start with the one that impressed him the most: qa-regression-testing branch The QA and security team maintain a test suite to check for regressions in packages they are updating. These tests are written with python unittest. Most of this test suite can be a little harmful for a production system, so it s recommended to run it using a chroot environment or a virtual machine and to help with that there is a make-test-tarball script. From the script:

export HOSTS= sec-intrepid-amd64 sec-hardy-amd64 sec-dapper-amd64
export TEST=test-glibc-security
./make-test-tarball $TEST.py
for i in $HOSTS; do
scp /tmp/qrt-$TEST.tar.gz $i.local:
done
for i in $HOSTS; do
ssh -t $i.local hostname; rm -rf qrt-$TEST; tar zxf qrt-$TEST.tar.gz; cd qrt-$TEST && ./$TEST.py -v
done

As you can imagine HOSTS are the hostnames of the virtual machines where the tests are going to run. Writing those test isn t rocket science, you just need a little knowledge of the package and the functionality you want to test and python unittesting, there is even a skeleton script that can help you start writing your test. Hope this information is useful for you! And as usual, patches are welcome!

Tomorrow, all being well, I'll receive a new computer. I've always run Debian unstable upon my desktop in the past, partly because I wanted to have "new stuff" and partly because I needed a Debian unstable system for building Debian packages with. However I'm strongly tempted to just install Lenny. I use that upon my work desktop and it does me just fine for surfing, building tools, and similar. I can use pbuilder, sbuildd, or similar to build packages for upload to Debian, and if I want to experiment with new-hotness I can use a KVM guest or two. Providing the hardware works with Lenny (and I have no reason to believe it won't) then there's no obvious downside I can think of. The only potential complication will be restoring my backups, it is possible that my firefox databases, and similar things, might not work on older version. Still we shall see. I plan to install software RAID, and run the system on LVM because quite frankly it rocks. Unless my current system fails in the next 24 hours I can use that to do the installation (My current desktop acts as a TFTP/DHCP/NFS server so I can use it to PXE-boot). Anyway now I need to go eat food, tidy my desk, and decide what to call the machine .. At the moment the choice is between "march.my.flat" and birthday.my.flat, as my 34th birthday is on March 10th. ObFilm: 300

The Threat Bruce Schneier s blog post about the Mariposa Botnet has an interesting discussion in the comments about how to make a secure system [1]. Note that the threat is considered to be remote attackers, that means viruses and trojan horses which includes infected files run from USB devices (IE you aren t safe just because you aren t on the Internet). The threat we are considering is not people who can replace hardware in the computer (people who have physical access to it which includes people who have access to where it is located or who are employed to repair it). This is the most common case, the risk involved in stealing a typical PC is far greater than the whatever benefit might be obtained from the data on it a typical computer user is at risk of theft only for the resale value of a second-hand computer. So the question is, how do can we most effectively use free software to protect against such threats? The first restriction is that the hardware in common use is cheap and has little special functionality for security. Systems that have a TPM seem unlikely to provide a useful benefit due to the TPM being designed more for Digital Restrictions Management than for protecting the user and due to TPM not being widely enough used. The BIOS and the Bootloader It seems that the first thing that is needed is a BIOS that is reliable. If an attacker manages to replace the BIOS then it could do exciting things like modifying the code of the kernel at boot time. It seems quite plausible for the real-mode boot loader code to be run in a VM86 session and to then have it s memory modified before it starts switches to protected mode. Every BIOS update is a potential attack. Coreboot replaces the default PC BIOS, it initialises the basic hardware and then executes an OS kernel or boot loader [2] (the Coreboot Wikipedia page has a good summary). The hardest part of the system startup process is initialising the hardware, Coreboot has that solved for 213 different motherboards. If engineers were allowed to freely design hardware without interference then probably a significant portion of the computers in the market would have a little switch to disable the write line for the flash BIOS. I heard a rumor that in the days of 286 systems a vendor of a secure OS shipped a scalpel to disable the hardware ability to leave protected mode, cutting a track on the motherboard is probably still an option. Usually once a system is working you don t want to upgrade the BIOS. One of the payloads for Coreboot is GRUB. The Grub Feature Requests page has as it s first entry Option to check signatures of the bootchain up to the cryptsetup/luksOpen: MBR, grub partition, kernel, initramfs [3]. Presumably this would allow a GPG signature to be checked so that a kernel and initrd would only be used if they came from a known good source. With this feature we could only boot a known good kernel. How to run User Space The next issue is how to run the user-space. There has been no shortage of Linux kernel exploits and I think it s reasonable to assume that there will continue to be a large number of exploits. Some of the kernel flaws will be known by the bad guys for some time before there are patches, some of them will have patches which don t get applied as quickly as desired. I think we have to assume that the Linux kernel will be compromised. Therefore the regular user applications can t be run against a kernel that has direct hardware access. It seems to me that the best way to go is to have the Linux kernel run in a virtual environment such as Xen or KVM. That means you have a hypervisor (Xen+Linux or Linux+KVM+QEMU) that controls the hardware and creates the environment for the OS image that the user interacts with. The hypervisor could create multiple virtual machines for different levels of data in a similar manner to the NSA NetTop project, not that this is really a required part of solving the general secure Internet terminal problem but as it would be a tiny bit of extra work you might as well do it. One problem with using a hypervisor is that the video hardware tends to want to use features such as bus-mastering to give best performance. Apparently KVM has IOMMU support so it should be possible to grant a virtual machine enough hardware access to run 3D graphics at full speed without allowing it to break free. Maintaining the Virtual Machine Image Google has a good design for the ChromiumOS in terms of security [4]. They are using CGroups [5] to control access to device nodes in jails, RAM, CPU time, and other resources. They also have some intrusion detection which can prompt a user to perform a hardware reset. Some of the features would need to be implemented in a different manner for a full desktop system but most of the Google design features would work well. For an OS running in a virtual machine when an intrusion is detected it would be best to have the hypervisor receive a message by some defined interface (maybe a line of text printed on the console ) and then terminate and restart the virtual machine. Dumping the entire address space of the virtual machine would be a good idea too, with typical RAM sizes at around 4G for laptops and desktops and typical storage sizes at around 200G for laptops and 2T for new desktops it should be easy to store a few dumps in case they are needed. The amount of data received by a typical ADSL link is not that great. Apart from the occasional big thing (like downloading a movie or listening to Internet radio for a long time) most data transfers are from casual web browsing which doesn t involve that much data. A hypervisor could potentially store the last few gigabytes of data that were received which would then permit forensic analysis if the virtual machine was believed to be compromised. With cheap SATA disks in excess of 1TB it would be conceivable to store the last few years of data transfer (with downloaded movies excluded) but such long-term storage would probably involve risks that would outweigh the rewards, probably storing no more than 24 hours of data would be best. Finally in terms of applying updates and installing new software the only way to do this would be via the hypervisor as you don t want any part of the virtual machine to be able to write to it s data files or programs. So if the user selects to install a new application then the request please install application X would have to be passed to the hypervisor. After the application is installed a reboot of the virtual machine would be needed to apply the change. This is a common experience for mobile phones (where you even have to reboot if the telco changes some of their network settings) and it s something that MS-Windows users have become used to but it would get a negative reaction from the more skilled Linux users. Would this be Accepted? The question is, if we built this would people want to use it? The NetTop functionality of having two OSs interchangeable on the one desktop would attract some people. But most users don t desire greater security and would find some reason to avoid this. They would claim that it lowered the performance (even for aspects of performance where benchmarks revealed no difference) and claim that they don t need it. At this time it seems that computer security isn t regarded as a big enough problem for users. It seems that the same people who will avoid catching a train because one mugging made it to the TV news will happily keep using insecure computers in spite of the huge number of cases of fraud that are reported all the time.

• [1] http://www.schneier.com/blog/archives/2010/03/mariposa_botnet.html
• [2] http://www.coreboot.org/Welcome_to_coreboot
• [3] http://grub.enbug.org/FeatureRequests
• [4] http://sites.google.com/a/chromium.org/dev/chromium-os/chromiumos-design-docs/system-hardening
• [5] http://lxr.linux.no/linux+v2.6.30/Documentation/cgroups/

I've made a quick release of dh-make, now up to 0.52. Besides a minor dh_make.1 manual page fix this release will put the right value in the debian/source/format file. 0.51 will make it a native source package no matter what flags you use.

If you get weird lintian errors about native source formats and version numbers wrong on your brand-new Debian package you just made, you might of been bitten by this bug.

That leaves a few bugs left, there are two I need help with:

• Bug 328692 - If you have a plan ASCII name like me, then your name looks fine in the dh-make generated man pages. This bug requires converting names that aren't plain ASCII (e.g. that use UTF-8) into something groff understands. Someone suggested decomposed unicode but unicode and groff are pretty much a black art to me, let alone combining them.
• Bug 533117 - This one is all about making dh-make make your python packages. You'll need to understand the new debhelper v7 rules files. Again, python is not something I use so the bug is stuck here.
  

Related articles by Zemanta

• UTF-8: Bits, Bytes, and Benefits (research.swtch.com)

Has anyone managed to get a standard version of mod_fastcgi work correctly with FastCGIExternalServer? There seems to be a complete lack of documentation on how to get this to work. I have managed to get it working by removing some code which appears to completely break AddHandler. However, people on the FastCGI list told me I was wrong for making it work. So, if anyone has managed to get it to work, please show me some working config.

Getting the special permission, that you don't need to wear a suit on the booth, just to wear one nonetheless, as it is to cold in your cool Debian T-Shirts...

W00t. Again below 1h45 and beating my best time by more than 2 minutes. Paris half-marathon was great. Freaking cold (-3 C with a strong North-East wind) but great weather with the sun in a blue sky. I didn't see Ralf (and, Ralf, sorry but I was finally wearing an orange suite..:-)) nor Debian swirls....but, apparently, that wasn't missing for having a great race. Elisabeth cam along with me to Paris and I managed to see her (and she manage to spot me) twice: once in Bastille at km 10 and later on at Charenton at km 14 The interesting challenge now is what to set as goal for the marathon. Of course, I have to run below 4h (which I missed by 49 seconds in Cologne) but I apparently can do better. I think I'll be reasonable and set 3h50 and we'll see.

Daniel, I also recently installed a WRT on the Linksys WRT54GL. But I used DD-WRT instructions went to http://www.dd-wrt.com/site/support/router-database, entered my model number and got direct download links to the firmware along with a README. One very important points in flashing these routers is to clear the NVRAM by doing a hard reset BEFORE and AFTER an upgrade to a different type of software on the device. Also, starting with a micro build is strongly recommended. In any case the hardware looks to be very solid, especially if you don t need gigabit ethernet and n wireless, but might need some advanced networking features.

I finally got a few hours free this morning to check an item off my home system administration checklist: upgrading the wireless router's firmware to OpenWRT. There were a couple motivations for this, including the fact that my SoundBridge Radio couldn't maintain a connection to my firefly server using the built-in firmware, and I had read that OpenWRT would work better. Since this is posted on Planet Debian, I should mention why I didn't use DebianWRT. The basic answer is this text at the top of the DebianWRT homepage:

Currently the most common methods used to run Debian on these systems is to install OpenWRT or a similar firmware, add disk space either by USB storage or NFS, create a debian chroot by either running cdebootstrap from inside OpenWRT or debootstrap --foreign on a PC, and running Debian from this chroot. For example, instructions for the WLHDD.

I'm not sure whether this is because DebianWRT is experimental, or because its goal is to use routers as cheap general computers. Either way, it sounds way too complicated and/or fragile for what I'm interested in (i.e., a wireless router with better software). The goal here is to get something that does a better job than the built-in WRT firmware, but doesn't require too much tinkering to get working or to maintain. I have plenty of outlets for my tinkering urges already. Nothing here was exactly difficult, but it was hard to find all the information I needed to get things working. Hopefully documenting the steps I went through here will save someone else some time. Step 1: acquire the firmware This was trickier than you might think. If you click on the download link at the OpenWRT web site, you end up at the top of an FTP server populated with a vast quantity of stuff. Worse, when you find the actual firmware images, you'll quickly discover that there are piles and piles of them divided between sixteen directories, and no guidance as to which one to pick. And picking the wrong one will turn your beautiful router into a doorstop. Luckily, the OpenWRT documentation contains a section called "Getting Started". Unluckily, that section consists of the following text:

1.1 Getting started 1.1.1 Installation 1.1.2 Initial configuration 1.1.3 Failsafe mode

Whoops, someone forgot to execute on a TODO. Undeterred, I consulted the usual fallback reference, Google. It pointed me at several references, some of which were hidden on other parts of the OpenWRT site. Armed with these, I was able to determine that:

1. My WRT54GL v1.1 probably uses a Broadcom 5352 chipset.
2. The correct firmware, according to multiple sources, is probably in the kamikaze top-level director, the latest version's subdirectory, and the bcrm-2.4 directory under the version (the link is to 8.09.2, which is current as of March 6, 2010). Apparently the bcrm47xx directory doesn't have wireless support; it helpfully contains a file called NO-BROADCOM-WIRELESS to warn you off, but unhelpfully doesn't include any additional information in that file (like what exactly is broken or that you can find a working firmware in a sibling directory) ... oh well.
3. The firmware file that you want is openwrt-wrt54g-squashfs.bin, even though it doesn't match the model number of the router. This directory could use a README file explaining which hardware is supported by each of the dozen or so firmware files it contains.

Step 2: install the firmware After Step 1, this was a real relief. I just used the built-in Linksys firmware installer, pointed it at the .bin file, and it went. Step 3: configure the router The installation guide I followed was pretty much silent about what to do after I got the firmware on. Luckily, this is just a software problem, meaning it's much more familiar territory for me.

1. First things first: I checked that I could still get a DHCP lease. It worked.
2. Armed with that, I tried telneting to the router. I used the resulting root prompt to set a password on the root account.
3. I logged out and tried telneting in. Apparently the router is configured to disallow root logins over telnet if you don't have a password. Good for them (although why allow telnet at all?); oops for me.
4. Luckily, an ssh server was installed by default. I like using keys to log in, so I tracked down the documentation on configuring the server to use public-key authentication; it turns out there's a single global key file named /etc/dropbear/authorized_keys that's exactly like OpenSSH's per-user authorized_keys file. No idea what would happen if I had multiple users, but I won't.
5. The next obstacle: I didn't have an Internet connection. For some reason, my cable modem didn't want to give the router a DHCP lease. On the off-chance that it was remembering too much, I rebooted it and ran ifup wan. That fixed the problem. I still don't know why.
6. Not a step, but a useful note: in the process of figuring the above out, I found readlog. It's basically dmesg for syslog files; it shows the most recent lines written to syslog. This is useful because there isn't a real syslog file, due to the fact that there would be no room to store it on the 54GL.
7. Finally, I had to get wireless working. The documentation is very helpful when it comes to describing the syntax of the wireless configuration. Unfortunately, I read the list of encryption options and missed the section right below where their meanings are explained (although, to be honest, I might not have understood the implications of the explanation without the research I did anyway).

   option encryption none, wep, psk, psk2, wpa, wpa2

   I wanted WPA2 encryption, so I entered wpa2. And nothing worked. After a good hour of trying different options on the client, swapping software components in and out on the router, experimenting with the encryption key syntax, and crawling Google, I finally found my answer. If you just want WPA2 encryption, you must not use wpa2 as the encryption type. Instead, use psk2. It turns out out that wpa2 actually means use WPA2 and also use an external RADIUS server for authentication. psk2 is the system you're familiar with from a typical consumer wireless router.

Step 4: enjoy And with that, it works. Unfortunately, contrary to what I wrote here originally, my Roku still doesn't work. On the other hand, having a real Linux installation is helpful for debugging it. Currently my suspicion is that the router isn't passing multicast packets between the wired and wireless interfaces (broadcast works fine, multicast doesn't). That said, it seems like if I restart Firefly just before I start playing music, I can play reasonably reliably -- as long as I don't stop, because if I do, the Roku forgets that the music server is there. Either way, I've spent about as much time fighting this as I can afford. One lingering worry I have is security; unlike Debian, which has both a security mailing list and tools to inform me when I need to install a security update, the OpenWRT firmware doesn't seem to have any mechanism for distributing security notices. True, there is an openwrt-security-announce list, but it appears to be entirely unused, as is openwrt-announce. Something to keep an eye on, then. Also (file under note to self), I need to remember to verify that the router isn't exposing services to the outside world. The default iptables configuration is hideously complex; with just a quick glance, it could be setting up a floral shop for all I can tell. I'll need to test this empirically and maybe analyze the rules in more depth.

It's been a while since I made an upload of anything DHCP-related, so I thought a general update was deserved. The lovely test infrastructure that I built had to be cannibalised to stand in for some other hardware that failed on me, so that prevented me from being able to test as easily. Add to that, the distractions of moving house, and I just haven't had the time to do any work on the DHCP 4.1 packaging. That said, I haven't been completely idle. I've been pressing the ISC DHCP developers to incorporate the LDAP patch, which is extremely popular in some quarters. At the same time, I was able to flush the author of the LDAP patch (he'd seemed to have disappeared) and connected the two parties together. Hopefully we'll see something in 4.2 Today, I updated the 4.1 packages in experimental to 4.1.1, which includes a reintroduced LDAP patch. I think if there's no unfavourable feedback, I'll look at uploading this to unstable in the next month or so, and the great transition to DHCP 4 can commence (assuming the release time is cool with this). If I could just figure out how to do bridged networking with KVM and still use NetworkManager for my WiFi, I could probably set up a similar test environment to what I had before, on my laptop...

Inspired by S. Lott and his blog post (and by the pure genius that xkcd is giving us these days, today included) I gave a look to Project Euler problem 14, that's about the Collatz conjecture.

The straightforward recursive solution:

def collatz(n):
 if n == 1:
     return 1
 if n % 2 == 0:
    return 1 + collatz(n/2)
 else:
    return 1 + collatz(3*n + 1)

very rapidly converges to... an error:

RuntimeError: maximum recursion depth exceeded

So I've recoded it a bit using a cache dictionary to store all the intermediate values in it:

cache =  1: 1 

def collatz(n, res):
 nn = n

 while True:
     if n in cache:
         if nn != n:
             cache[nn] = cache[n]
         return cache[nn]

     if n % 2 == 0:
         cache[n] = 1 + collatz(n/2, cache)
     else:
         cache[n] = 1 + collatz(3*n + 1, cache)

Now loping for all the numbers lower than 1 million we find the solution to the problem: the number with the longest path is 837799 with a path length of 524. The code runs in about 4secs.

Since I got that cache dict around to play with, I graphed with Matplotlib (strange, ha ;) ) the frequencies of each path length:
There are few short paths, several long paths but rare to occur, and most of the paths have length between 100 and 200 (more or less).

UPDATE: I've removed length variable, not needed (left from a previous version of the code) and fixed the path length for 837799, wrongly reported.

I've been thinking about something for a while, and to be honest, still haven't reached any firm conclusions: what to think about self-published "e-books"? I'm curious to hear your opinions. For instance:

• http://gettingreal.37signals.com/
• http://www.railsupgradehandbook.com/
• http://createyourproglang.com/

These are all: electronic, in that they aren't distributed as real, paper books, have no ISBN number, and are generally only available via the author's web site (you won't find them on Amazon.com). They aren't simply on-line, PDF versions of real books. They're certainly a departure from the traditional model of having a publisher, editor, and real, physical books that could be purchased from book stores. They don't appear to have been through any kind of formal editing or quality control process. The prices seem to differ quite a bit; the first one is $19, the second one is $12, and the last one is $30.77. For the authors, the appeal is obvious: they get to keep all of the money, and don't have to fool around with a lot of "process". Consumers, on the other hand, have to consider different aspects: with a "real book", the bureacracy and process exist to guarantee some minimum standards of quality. If you buy an O'Reilly book, you know that it's probably like many of the other books they sell: perhaps it won't stand the test of time like something written by Knuth, but it'll be a pretty good guide to the technology it describes, most likely be someone who is indeed an expert in that area. If I buy some random PDF on the internet, it may come from someone who really knows their stuff, or it may be junk. On the other hand, were this market to grow, theoretically prices could come down. Since the people who are authoring the book don't have to fool around with editing, printing, and so on, and get to keep all the money themselves, they could in theory keep their prices significantly lower than someone creating a more 'traditional' book with a lot of overhead. That is, of course, if the book is one where there is competition in its niche. Right now a lot of these books that pop up on my radar are written by domain experts. However, what's to prevent a lot of people from jumping in and attempting to make a quick buck with a flashy looking web site? Buying books based only on reputation? That might lead to people who are really good authors, but perhaps not well known as "doers" (they didn't invent the technology in question) being left out in the cold. Also, there is something of an unknown quantity about "pdf books". For instance, after raking in a bunch of cash with theirs, 37signals put it on their web site, completely for free. That had to leave the guy who bought it the day before it went free feeling like a bit of a chump. At least with a 'real book', even if the contents are posted on the internet, you have a physical object that belongs to you. I wonder how bad piracy is, and how bad it might be were these to become more popular? Another thing worth noting is that, via services like Lulu.com, it *is* possible to print these out. In any case, I think things are likely to change with time, as we aren't dealing with a static situation, but rather one where a changing landscape maylead to different outcomes, as the key variables... vary. I am honestly unsure of what to make of this development. How do you see the market for "home brewed" pdf ebooks evolving?

From RFC3164, which is otherwise about syslog formats:

6. Security Considerations An odor may be considered to be a message that does not require any acknowledgement. People tend to avoid bad odors but are drawn to odors that they associate with good food. The acknowledgement of the receipt of the odor or scent is not required and indeed it may be the height of discretion to totally ignore some odors. On the other hand, it is usually considered good civility to acknowledge the prowess of the cook merely from the ambiance wafting from the kitchen. Similarly, various species have been found to utilize odors to attract mates. One species of moth uses this scent to find each other. However, it has been found that bolas spiders can mimic the odor of the female moths of this species. This scent will then attract male moths, which will follow it with the expectation of finding a mate. Instead, when they arrive at the source of the scent, they will be eaten [8]. This is a case of a false message being sent out with inimical intent. ... Along the lines of the analogy, computer event messages may be sent accidentally, erroneously and even maliciously.

This smells more like "I bet nobody ever really reads this RFC, let's put some stuff in the middle to see if they do!".