Most of us developers use metaphors to help convey an understanding of how our software works. Sadly, many developers choose the "master" and "slave" metaphor. Why would we voluntarily choose such an ugly human human interaction with a real and present history in many of our lives to describe software that we've written? I wouldn't rule out the possibility that someone will (or already has) invented software for which the master/slave metaphor aptly conveys its function, however, I'd prefer not to use it. Also, many people intentionally play with master/slave relationships because of the history and connotations it provokes. However, use of this metaphor in software like MySQL databases or Bind Name servers seems pointless, offensive and mis-leading. I've never seen the use of the master/slave metaphor in software that conveyed more than the idea that one piece of the software essentially controls, wholly or in part, another piece of software. So, why master/slave? There are many metaphors to choose from - parent/infant, boss/worker, guard/prisoner, king/subject, landowner/serf, top/bottom, etc. What does master/slave convey that the others don't convey? Or, maybe they all convey the same thing, so it doesn't make a difference which one you choose (after all, it's just a metaphor). In that case, how about replacing master/slave with the metaphor husband/wife? Whoops. Now that's offensive (if you don't think that's offensive you can stop reading here and please don't post any comments - I won't change your mind). But, that's different right? At one point in history, the role of the wife was to do as her husband says, but it's 2011, and we've evolved beyond that, so by using husband/wife we'd be re-enforcing that ugly, sexist stereotype. However, how is that different from master/slave? It is 2011 after all. Isn't the role of the master to abdicate and the role of the slave to revolt? Maybe that's why Bind has so many security problems...

Puppet labs's use of the term puppetmaster is rather clever (in contrast to other un-necessarily offensive uses of "master" in the software world). While I appreciate the clever name, I'm less impressed with the concept. At May First/People Link we've spent the last several years (including the last couple months in earnest) working to transition management of our 90-some servers from a collection of hand-written bash scripts to puppet. Over the years, we've worked hard to keep our servers as secure as possible. We have a team of about a half dozen people who all have root access on all servers. It's all key-based access. To help mitigate a disaster if one person's keys were compromised, we've implemented monkeysphere on all servers, allowing us to easily revoke access. After spending so much time thinking through our root-access strategy and fully implementing the monkeysphere to reduce our exposure to a single point of vulnerability, I was disappointed by puppet's use of a puppet master. For those less familiar with puppet, it goes something like this: One server (or god forbid multiple servers), run an externally accessible daemon. Each and every server on your network runs a daemon as root that periodically communicates with the puppet master, receives new instructions, and then (again, as root) executes these instructions. In other words, if your puppet master is compromised, I'm not sure exactly what you would need to do, short of rebuilding every server in your network. To make matters worse, it seems as though some users generate and store all server ssh keys (private and public) on the puppet master and then push the private keys to their respective nodes. That means an intruder doesn't need to write to the puppet master, just reading these keys would be enough to compromise all servers in your network. There must be a better way. Puppet without masters After some web-searching, I found a promising thread on the puppet list asking what's lost without a puppet master. This thread lead to a couple other blogs by people who have worked out a system for using puppet without a master. It turns out that there are two distinct points of centralization with puppet. One is the puppet master (as described above). In addition, there is a concept called storeconfigs - which allows each node in the network to store information in a central database. For example, one server can store a request for an account to be setup on a backup server. The next time the backup server runs, it checks the storeconfigs, finds the request, and creates the user. It's possible to run puppet with storeconfigs but without running a puppet master (that avoids the hassle of running the puppet daemons, while providing the convenience of centralization). For our purposes, however, we decided to forego both the puppet master and storeconfigs. We did not want any form of centralization that would provide an additional point of vulnerability. As is common with puppet, we are storing our puppet recipes in a git repository. And, we are publishing to a single, canonical git repository on the Internet. On each node, we have two git repositories - one is a bare repo (that we can push to) and the other is a checked out repo (in /etc/puppet) that is read by puppet. The bare repo has a post-update hook the changes into the /etc/puppet directory, pulls in the changes from the bare repository, and runs puppet against the newly checked out files. Therefore, we can apply new puppet recipes to any server on the network with

git push <server>

No daemons: neither a master daemon nor a puppet daemon running on the node using up memory or providing a potential security hole. The git push happens over an ssh connection - since all system administrators already have root-level ssh access on every server - there is no need to grant any additional access above what we already have. Pushing works great - but with 90 nodes we don't want to have to push to 90 servers everytime we want a change made. That's where the canonical git repository comes in. A cron job runs a script on each node once an hour that runs git remote update from /etc/puppet. The script then checks the time stamp on the most recent gpg-signed tag and compares it with the time stamp of the current commit. If the most recent gpg-signed tag is newer, it verifies that the tag came from a list of authorized gpg keys (the very same gpg keys used by the monkeysphere to grant root level ssh access). If the gpg signature of the tag can be properly verified, then the changes are merged and puppet is run on the new recipes. What about privacy? One of the benefits of a puppet master setup is that nodes get configuration details on a need-to-know basis. The puppet master doesn't share the entire puppet repo - only the compiled manifest for the node with which it's communicating. Our solution to this problem was to go screaming in the other direction. As you might notice from our support wiki and ticket system, we generally favor transparency. Since we are publishing our entire puppet git repo publicly, there seems little point in trying to hide one node's configuration details from another node. That also means each node carries around about 4Mb of extra weight in the form of disk space for the git repo. That seems like a small price to pay for the resource savings of not running a puppetd process all the time. More differences As I've read the puppet lists, faqs and documentation, I've found yet more ways our use of puppet diverges from the norm. The first is a little thing really - most people seem to store all their node configurations in a single nodes.pp file. I'm not sure why. Fortunately, puppet's include syntax allows globbing, so we've created a directory and gave each server it's own .pp file. This arrangement makes it much easier to parse the configuration with tools other than puppet (like, Q. How many servers do we have? A. ls wc -l). Backup and Nagios monitoring without storeconfigs More significantly - there are some things we can't do since we are not using storeconfigs. While many puppet users add a variable, like $nagios = true before including their sshd class (which then causes the sshd class to store a configuration for the nagios server to monitor ssh on the node in question), we were forced to come up with alternatives. My first solution was to simply list all the servers that needed to be monitored in the server node configuration file for the nagios server. Ditto for the backup servers. This approach, however, proved cumbersome and error prone. When adding a new node, I now have to edit three files instead of one. And, how can I easily tell if all nodes have their nagios and/or backup configurations set? The solution was rather simple - there's more than one way to store a config for another node. Our nagios server is called jojobe.mayfirst.org and our backup server is luisa.mayfirst.org. A typical node.pp file looks like this:

node pietri.mayfirst.org  
  # node config goes here
 
if ( $fqdn == "jojobe.mayfirst.org )  
  nagios_montior   "pietri":  
 
if ( $fqdn == "luisa.mayfirst.org )  
  backup_access   "pietri":  
 

This way all configuration related to pietri stays in a single file. Host keys and granting access between servers storeconfigs is commonly used to distribute host ssh keys. Every node that is added to puppet has it's ssh host key stored centrally and then re-distributed to every other node. That way, you can ssh from node to node without ever getting the ssh fingerprint verification. Avoiding that prompt is particularly important when backing up from one server to another via automated scripts. storeconfigs can additionally be used to copy user's public ssh keys - thus granting user access between servers. Our solution to this problem: monkeysphere. Rather than maintaining our own private data store of keys, we publish (and sign) our ssh keys via the web of trust. In addition to server keys, each one of our servers' root user has an ssh-enabled gpg key (also publicly signed by us). By configuring each server to trust our system administrators' gpg keys for verifying other keys, we can both avoid the ssh fingerprint manual verification step and we can grant a root user on one server access to another server by simply dropping root@$server.mayfirst.org into an authorizeduserids file on the target server. There's no question - the setup was rather tedious (we're using runit to maintain an ssh-agent for each root user), however, now that's in place (and configured via puppet), it's a breeze to add new servers. The only extra step we have to take is to confirm and sign each new server's keys. This "extra" step not only allows our servers to verify each other, but also allows our users to verify the servers as well, so it's hardly an extra step at all. Shared modules There's a vibrant community of third party module developers for puppet. Rather than figure out the intricacies of having puppet configure sshd, for example, you can install a contributed sshd module and then you simply add:

include sshd

And you get a default sshd setup. Many of these modules are fairly well developed, offering the ability to easily customize your setup in a number of different ways. Unfortunately, most of the modules assume you are using storeconfigs and if you are not, they will either fail to work right or you will get noisy errors. At first, this seemed like a problem. However, as I built our puppet recipes, I found myself increasingly frustrated with the third party modules that we could use. Configuring servers is hard - and requires constant debugging and trouble shooting. puppet already provides a layer of abstraction between you and the server you are setting up. Given the benefits of puppet, I'm willing to spend the time learning the puppet syntax and asking the rest of our system administrators to do the same. This layer of abstraction is further compounded by our use of git to store the configurations (not a problem if you are git hero - but most of us are already struggling to get a handle on using git). Again, all seems worth it for the pay off. Now enter the puppet module. In addition to learning puppet syntax (and struggling with git) you now need to understand how the third party module works. With software programming, I typically don't need or want to learn how a library or class does what it does - that's the beauty of object-oriented programming: it hides the complexity. But when it comes to configuring the servers that I will be responsible for debugging and maintaining, I really need to know exactly what is happening. To further compound the problem, I found myself wading through third party module code designed to work on Debian, Ubuntu, CentOS, Redhat, gentoo... and more. We run entirely on Debian - we don't need any of this extra code. And, once I got rid of all the other operating systems, I was still left with a complex module that allows you to configure software in ways we'll never need. In the end, we tore out most of these third party modules and replaced them with file and exec puppet resources that did exactly what we needed them to do. Our code base is now much smaller and simpler. Not just a whiner I have a lot more to whine about (like why native resources for things like nagios that are so easily handled with the file resource?). However - the remarkable thing about puppet is that it's flexible. Despite some fairly substantial problems with the "typical" use of puppet, the program provides enough flexibility for us to use it in a way that fully meets our needs. After having built my own bash-based set of configuration scripts and deeply exploring puppet, I have a great appreciation for the difficulty of building system configuration software (we considered and rejected cf-engine and chef as not being any better). And, if you are still not convinced that puppet will work fo you ... you might consider a package I learned about after going down the puppet route: slack.

Nowadays, many websites provide a mobile version, like Wikipedia, which usually is a version tailored to device with less bandwidth and smaller screen. CSS Media Queries is a candidate recommendation from the W3C to support different stylesheets depending on the how the document is displayed. This allows to build a mobile version just by providing additional instructions in your CSS style sheet.

Using CSS Media Queries There are two ways to use CSS media queries. You can provide an additional CSS stylesheet and let the browser knows when to use it:

1	<link rel="stylesheet" media="screen and (max-width: 700px)" href="mobile.css">

When the width of the viewport is less than 700 pixels, the stylesheet mobile.css will be used. Another way to use CSS media queries is to declare a section inside your current CSS style sheet:

1 2 3 4 5

@media screen and (max-width: 700px) #article margin: 0.3em 0.2em;

While there are many available properties, the most two common ones will be max-width and min-width. You can use this feature to modify some aspects of your web site like reducing margins or hiding some extra navigational blocks.

Browser support Current versions of mobile browsers support CSS media queries: Safari Mobile, Android browser and Opera mobile will handle it without a glitch. If you use CSS media queries to provide a mobile version of your web site, you don t really have to worry about browser compatibility. There exists polyfills that will enable this feature on unsupported browsers like Internet Explorer 8. One of them is Respond which is a lightweight polyfill targeted at supporting only min-width and max-width in Internet Explorer 6, 7 and 8.

Example I have added CSS media queries to my website. Try it by resizing the window of your browser. Here are some of the changes depending on the width of the window:

• the sidebar transforms into a topbar
• the search box is hidden
• padding is reduced at various places

Here is the current look in a classic browser: If you reduce the size of your window, you will get something like this: Browse my CSS files for more examples. Look at the end of each file. For further information, have a look at the article from CSS Tricks.

Background: Lars Wirzenius was searching for a tool to create virtual disk images. Turned out I needed something similar to improve the automatic deployment process of a customer s platform installation. Being the author of the mentioned grml-debootstrap tool I started to work on deploying virtual disk images using grml-debootstrap. As a result I just uploaded grml-debootstrap version 0.46 which is the first release being capable of installing virtual disk images. Thanks to great help by Thorsten Glaser also proper Grub integration is working now. Instructions: Grab the most recent grml-debootstrap version (>=0.46) from the grml-testing repository and make sure to install its recommends (kpartx mksh parted qemu-utils). Then just use the vmfile option to activate deployment of a virtual disk image, combined with the target option as usual. To adjust the disk size use the vmsize option. Usage example:

# IMAGE_FILE="/mnt/sda1/qemu.img"
# grml-debootstrap --vmfile --vmsize 3G --target $IMAGE_FILE

I tested it from inside a running Grml live system, which is running inside a KVM image running on a Proxmox VE cluster. I can even invoke KVM/Qemu inside Grml inside KVM to test the image (using remote access e.g. via vncviewer $IP_OF_KVM_HOST:5901 ) running:

# kvm -hda $IMAGE_FILE -m 256 -vnc :1

And the best about it: it just works.
As usual: testing and feedback highly welcome.

I doubt I have ever read a book that had as significant an impact on me, and my act of reading it had on others, as this one. Conversations like this were frequent: Someone, upon seeing me reading my Kindle, would ask what I am reading. War and Peace. Oh. . . uhm, wow. Yeah, it s great. I m reading it for fun. Uhm, OK then, see you later. . . It seems to be so revered, and also so loathed, that I had to read it. At 1400 pages, it took me 8 months to finish, though I read several other entire books in that time. I feel rather unequal to the task of expressing how this book impacted me, let alone a review of it. And nonetheless, I also feel I would be remiss if I let it go past saying nothing. I was struck by so many things as I read War and Peace. Some of them I won t mention here and hope to turn into their own blog posts. Of the others, I felt I gained some sense of how the nobility and serfs in Russia (and, to a certain extent, Europe) thought about life, their position, and how things ran. Being a modern Kansan, this thought process was not familiar to me, and though I head read about it in history texts before, felt far more informed having read it in Tolstoy s novel. Although much of the novel centers around Napoleon s invasion of Russia in 1812, the work as a whole spans nearly two decades of time. Tolstoy s characters aren t static; they change over time. Some, yes, die; others go through hardships and triumphs that change them to their core. It evoked a feeling of nostalgia in me at times for the younger, childlike Natasha who was so full of simple delight in life. But then, a thousand pages later, the older Pierre finally was able to find simple delight in life too. Sometimes I have missed on the simple joy of being, and Jacob or Oliver or Terah remind me of that. Today Oliver and I read a book together, one that we read often, and we discovered an illustration of a tiny worm we had never noticed before. And the worm had a red hat ( hat is one of Oliver s favorite words right now.) The happy laughter as he pointed at the tiny hat, saying hat over and over, reminds me that sometimes children know how to live better than adults. Jacob later asked me how my day was, and I told him how I read a book on my Kindle, where I sat, and how I even read it lying down on the couch for a bit. At that he too laughed. As with some other wonderful, engrossing books, I was sad to reach the end of this one. I felt as if I was leaving a conversation early; fictional characters, yes, but their story wasn t over. And really, that was part of Tolstoy s point: things don t happen in isolation, and stories don t have clearly-defined start and end points. The novel touched on politics, religion, philosophy, free will, and just about every topic imaginable. It is, really, unfair to call it a just a novel. Here are some random quotes from the book, which I highlighted:

his heart was now overflowing with love, and by loving people without cause he discovered indubitable causes for loving them. How often we sin, how much we deceive, and all for what? All will end in death, all! A pleasant humming and whistling of bullets were often heard. Looking into Napoleon s eyes Prince Andrew thought of the insignificance of greatness, the unimportance of life which no one could understand, and the still greater unimportance of death, the meaning of which no one alive could understand or explain. If there is a God and future life, there is truth and good, and man s highest happiness consists in striving to attain them. We must live, we must love, and we must believe that we live not only today on this scrap of earth, but have lived and shall live forever, there, in the Whole, said Pierre, and he pointed to the sky. Prince Andrew stood leaning on the railing of the raft listening to Pierre, and he gazed with his eyes fixed on the red reflection of the sun gleaming on the blue waters. There was perfect stillness. Pierre became silent. The raft had long since stopped and only the waves of the current beat softly against it below. Prince Andrew felt as if the sound of the waves kept up a refrain to Pierre s words, whispering: It is true, believe it. He sighed, and glanced with a radiant, childlike, tender look at Pierre s face, flushed and rapturous, but yet shy before his superior friend. In Natasha Prince Andrew was conscious of a strange world completely alien to him and brimful of joys unknown to him, a different world, that in the Otradnoe avenue and at the window that moonlight night had already begun to disconcert him. Now this world disconcerted him no longer and was no longer alien to him, but he himself having entered it found in it a new enjoyment. All the kings, except the Chinese, wear military uniforms, and he who kills most people receives the highest rewards. But what is war? What is needed for success in warfare? What are the habits of the military? The aim of war is murder; the methods of war are spying, treachery, and their encouragement, the ruin of a country s inhabitants, robbing them or stealing to provision the army, and fraud and falsehood termed military craft. The habits of the military class are the absence of freedom, that is, discipline, idleness, ignorance, cruelty, debauchery, and drunkenness. And in spite of all this it is the highest class, respected by everyone. From the moment Pierre had witnessed those terrible murders committed by men who did not wish to commit them, it was as if the mainspring of his life, on which everything depended and which made everything appear alive, had suddenly been wrenched out and everything had collapsed into a heap of meaningless rubbish. Though he did not acknowledge it to himself, his faith in the right ordering of the universe, in humanity, in his own soul, and in God, had been destroyed. He had experienced this before, but never so strongly as now. When similar doubts had assailed him before, they had been the result of his own wrongdoing, and at the bottom of his heart he had felt that relief from his despair and from those doubts was to be found within himself. But now he felt that the universe had crumbled before his eyes and only meaningless ruins remained, and this not by any fault of his own. He felt that it was not in his power to regain faith in the meaning of life. When Princess Mary began to cry, he understood that she was crying at the thought that little Nicholas would be left without a father. With a great effort he tried to return to life and to see things from their point of view. Yes, to them it must seem sad! he thought. But how simple it is. The fowls of the air sow not, neither do they reap, yet your Father feedeth them, he said to himself and wished to say to Princess Mary; but no, they will take it their own way, they won t understand! They can t understand that all those feelings they prize so all our feelings, all those ideas that seem so important to us, are unnecessary. We cannot understand one another, and he remained silent. Love hinders death. Love is life. All, everything that I understand, I understand only because I love. Everything is, everything exists, only because I love. Everything is united by it alone. Love is God, and to die means that I, a particle of love, shall return to the general and eternal source. These thoughts seemed to him comforting. People speak of misfortunes and sufferings, remarked Pierre, but if at this moment I were asked: Would you rather be what you were before you were taken prisoner, or go through all this again? then for heaven s sake let me again have captivity and horseflesh! We imagine that when we are thrown out of our usual ruts all is lost, but it is only then that what is new and good begins. While there is life there is happiness. There is much, much before us. I say this to you, he added, turning to Natasha. During that twenty-year period an immense number of fields were left untilled, houses were burned, trade changed its direction, millions of men migrated, were impoverished, or were enriched, and millions of Christian men professing the law of love of their fellows slew one another. What does all this mean? Why did it happen? What made those people burn houses and slay their fellow men? What were the causes of these events? What force made men act so? These are the instinctive, plain, and most legitimate questions humanity asks itself when it encounters the monuments and tradition of that period.

Almost 4 years ago, I decided to set up a tiny website, mostly for blogging purposes. Given I was using ikiwiki and my nickname s KiBi, ikibiki.org sounded fun, so I picked that one. Now, it s a bit long to type, error-prone (how many times did I type ikiwiki instead ), and even my family has troubles remembering it. Other kibi.* domain names weren t released in the meanwhile, so looking into stuff I care about, something cat-related could do the job. And since I ve been using mraw in my signature for a while, I picked that one. So my blog lives at http://blog.mraw.org/ now. The old domain name will be kept for a while though.

This week saw again some progress for samba packages. May is indeed my samba month..:-). So, I uploaded samba 3.5.8~dfsg-3 and -4. For whatever reason, -3 was rejected three times and, as I already tagged it in our SVN then started adding changes for an upcoming -4, I finally uploaded that -4. It fixed about 10 more bugs and the bug count is now 62. Upstream also release 3.6.0-rc1, which I uploaded in experimental and which fixed an RC bug. I also began working on Pootle 2.1.6 as an attempt to resync with upstream. I integrated patches by Alaa Abd El Fattah to have it use dbconfig-common to setup a database. However, this revealed that more invasive work is needed to satisfy dependencies...that may depend on the database backend chosen by users. As I'm quite ignorant with these things, I might need some help, here. I uploaded a new release of xgridfit to fix an important bug and avoid an NMU. xgridfit is team-maintained in pkg-fonts. I'm not very active in this team ATM. I shoul dresume work for it as I proposed a talk about it at Debconf11..:-) I continued hard work on French DDTP. Martijn made changes to stats pages and it makes interesting challenges to complete. You know I like challenges.. Smith reviews continue to go on: I launched a call for translations for slapos.tool.format and started the review of ntfs-3g. Finally, on Saturday....I upgraded churro (aka i18n.debian.net) to squeeze. Really. All alone..:-)...on a server that lies in Spain. "Meme pas peur" (not even scared) as we say in French. I'm really highly confident in squeeze upgrades now....and I was right. After only 5 hours work (mostly looking at packages downloads and configuration), nearly everything was back to work. As of now, we got three glitches, with only one bignot yet fixed:

• the wiki that uses MoinMoin is severely broken. So, as of now, http://i18n.debian.net yields an error (but all other i18n.d.n URLs are working)
• HTTPS was broken for ddtp.debian.net, which prevented people to login to work on the DDTP. I partly fixed that today (Monday) and Felipe will have a closer look at VirtualHosts in Apache to fix the remaining glitches
• cron jobs for i18n robots were no longer working. After some headache and googling, it turned out one file in /etc/cron.d was group-writable, which cron translates to "WRONG INODE INFO".

Quite active week, finally. This week should be quieter as I'm on holidays with Elizabeth in our country house in Normandy, in Seine valley (this house belonged to Elizabeth parents and she inherited it). I'm in deep love with this house and there are tons of things to do there...which leaves few room for hacking. We'll see what will be next weekly summary!

Yesterday was my 21st birthday, and I received all Hitchhiker s Guide to the Galaxy novels, the five ones in one book, and the sixth one written by Eoin Colfer in another book. Needless to say, the first book weights more than an N900. I did not read them yet, so now is the perfect chance to do so. Yes, I did not know that 25th is towel day, sorry for that. I also bought a Toshiba AC100 before my birthday, a Tegra 2 based notebook/netbook/ web companion with 1 GHz dual core ARM Cortex A9 chip and 512 MB RAM. It runs Android by default, and had a price of 160 which is low compared to anything else with Cortex A9. It currently runs Ubuntu 11.04 with a specialised kernel 2.6.37 from time to time, without sound and accelerated video (and not functioning HDMI). Mostly waiting for Nvidia to release a new binary blob for the video part (And yes, if you just want to build packages, you can probably get happy without those things). Another thing happening last week is the upload of python-apt 0.8.0 to unstable, marking the beginning (or end) of the API transition I started more than a year ago. Almost all packages not supporting it have proper Breaks in python-apt [most of them already fixed, only 2 packages remaining, one of which is "maintained" (well, not really maintained right now) by me], but there may be some which do not work correctly despite being fixed (or at least thought to be fixed). If you know any other interesting thing I did last week, leave a comment, I wrote enough now. And yes, WordPress wants to write a multiplication sign instead of an x, so I had to use &#120 instead.
Filed under: Debian, General, Ubuntu

A new version of the digest package (which generates hash function summaries for arbitrary (and possibly nested) R objects using any of the standard md5, sha-1, sha-256 or crc32 algorithms) is now on CRAN. Thanks to a patch by Mario Frasca, digest now contains a second (exported) function hmac() to generate Hash-based Message Authentication Code as defined in RFC 2104. Reference output from RFC 2104 is used to validate this functionality as shown below where we run the example code from the hmac() help page:

R> library(digest)
R> example(hmac)
hmacR> ## Standard RFC 2104 test vectors
hmacR> current <- hmac('Jefe', 'what do ya want for nothing?', "md5")
hmacR> target <- '750c783e6ab0b503eaa86e310a5db738'
hmacR> stopifnot(identical(target, as.character(current)))
hmacR> current <- hmac(rep(0x0b, 16), 'Hi There', "md5")
hmacR> target <- '9294727a3638bb1c13f48ef8158bfc9d'
hmacR> stopifnot(identical(target, as.character(current)))
hmacR> current <- hmac(rep(0xaa, 16), rep(0xdd, 50), "md5")
hmacR> target <- '56be34521d144c88dbb8c733f0e8b3f6'
hmacR> stopifnot(identical(target, as.character(current)))
hmacR> ## SHA1 tests inspired to the RFC 2104 and checked against the python
hmacR> ## hmac implementation.
hmacR> current <- hmac('Jefe', 'what do ya want for nothing?', "sha1")
hmacR> target <- 'effcdf6ae5eb2fa2d27416d5f184df9c259a7c79'
hmacR> stopifnot(identical(target, as.character(current)))
hmacR> current <- hmac(rep(0x0b, 16), 'Hi There', "sha1")
hmacR> target <- '675b0b3a1b4ddf4e124872da6c2f632bfed957e9'
hmacR> stopifnot(identical(target, as.character(current)))
hmacR> current <- hmac(rep(0xaa, 16), rep(0xdd, 50), "sha1")
hmacR> target <- 'd730594d167e35d5956fd8003d0db3d3f46dc7bb'
hmacR> stopifnot(identical(target, as.character(current)))
R> 

Also, CRANberries shows the difference to the previsious release 0.4.2 as follows:

Diff between digest versions 0.4.2 dated 2009-12-06 and 0.5.0 dated 2011-05-30
 digest-0.4.2/digest/INDEX                       only
 digest-0.4.2/digest/R/zzz.R                     only
 digest-0.4.2/digest/inst                        only
 digest-0.5.0/digest/ChangeLog                   only
 digest-0.5.0/digest/DESCRIPTION                    33 ++++++++++++-------------
 digest-0.5.0/digest/NAMESPACE                       7 +++--
 digest-0.5.0/digest/R/hmac.R                    only
 digest-0.5.0/digest/man/hmac.Rd                 only
 digest-0.5.0/digest/tests/digestTest.Rout.save      2 -
 digest-0.5.0/digest/tests/hmacTest.R            only
 digest-0.5.0/digest/tests/hmacTest.Rout.save    only
 11 files changed, 22 insertions(+), 20 deletions(-)

With that, special thanks to Mario Frasca for the patch and to Henrik Bengtsson for helpful discussion. We took some care to ensure that the existing interface to the digest() function remains unaffected.

So earlier on I posted about my work on declarative diversions . I felt I needed to clarify a few things as a few people asked me different questions and I don t think I made the post clear enough and I also left out an important piece of information. The new control file will be located in control.tar.gz and it s syntax will be

One diversion per line Blank lines and lines with # are comments Two fields per diversion seperated by whitespace (SOURCE DESTINATION)

Now people have asked be about updating and removing diversions and how that will be accomplished. All diversions will be listed in this control file. If a diversion isn t listed in the control file on upgrade dpkg will automatically remove the diversion because it will see it is no longer needed. The same goes for changing diversions, i.i. list them just as though they re new and it will update them. This project will infer add, remove and package and will not allow you to specify them. The same is goes for rename, it is being made default and not optional (for now, I am aware of some special cases). *Removed, slight mistake here* Now, there are a number of things still left to do this week before a full draft is prepared. There is an email here by Ian Jackson which talks about ordering unpack in such a way to delay the the actual renaming of the file. If anyone has any suggestions for this process it would be greatly appreciated. Another two things that need to be done is the proper handling of diversions to non-existant directories and also escaping spaces within filenames for the control file. I should have another update tomorrow evening. Until then, ~Sam

Lately I have been working on the triggers feature of dpkg. I would like to share my plan and what I have done so far. I ll first explain what triggers are, the current problems, and the work I did to try to improve the situation. Introduction Dpkg triggers are a neat feature of dpkg that package can use to send/receive notifications to/from other installed packages. Those notifications take the form a simple string. This feature is heavily used to track changes of packaged files in a list of predefined directories, and to update other files based on this. For instance, man-db is watching the directories containing manual pages so that it can update its cache (in /var/cache/man/). install-info is updating the index of info pages when there have been changes in /usr/share/info. gnome-menus is updating its own copy of the menu hierarchy (with entries from /etc/gnome/menus.blacklist blacklisted) every time that a .desktop file is installed/updated/removed. From a user s perspective You see triggers in action very often during upgrades (in fact too often as we ll see it later):

Preparing to replace zim 0.52-1 (using .../archives/zim_0.52-1_all.deb) ...
Unpacking replacement zim ...
Processing triggers for shared-mime-info ...
Processing triggers for menu ...
Processing triggers for desktop-file-utils ...
Processing triggers for man-db ...
Processing triggers for hicolor-icon-theme ...
Processing triggers for python-support ...
Processing triggers for gnome-menus ...
Setting up zim (0.52-1) ...
Processing triggers for python-support ...
Processing triggers for menu ...

As you guessed it, those Processing triggers lines correspond to the packages which received (one or more) trigger notifications and which are doing the corresponding task. By default the triggers are processed at the end of the dpkg --unpack invocation which is often too soon because APT will often call dpkg --unpack repeatedly during important upgrades. There are some options to ask APT to use dpkg s --no-triggers option in order to defer the trigger processing at the end of the APT run. You can put this in /etc/apt/apt.conf.d/triggers:

// Trigger deferred
DPkg::NoTriggers "true";
PackageManager::Configure "smart";
DPkg::ConfigurePending "true";
DPkg::TriggersPending "true";

I have now asked APT maintainers to use those options by default, I filed bug #626599 to track this. At the same time I fixed bug #526774 reported by APT maintainers. This bug forced them to put a work-around in APT which resulted in running triggers sooner than expected. (And while writing this article I filed bug #628564 and #628574 because it was clearly not normal that the menu triggers was executed twice for the installation of a single package) From a packager s perspective The implementation of triggers has several consequences on the status that packages can have. Let s assume that the package A installs a file in a directory that is watched by package B (and that B is currently in the installed state). When A is unpacked, dpkg adds B to its Triggers-Awaited field and lists the activated trigger in B s Triggers-Pending field. Package A is in unpacked state, but B has been changed to triggers-pending . When A is configured, instead of going to the installed state, it will go to the triggers-awaited state. In that state the package is assumed to NOT fulfill dependencies. However, B which is still in triggers-pending state does fulfill dependencies. A and B will switch to installed at the same time when the trigger has been processed. The fact that the triggers-awaited status does not fulfill dependencies means that some common triggers like man-db have to be processed regularly just to be able to ensure dependencies are satisfied before running the postinst of other installed packages. But a package which ships a manual page can certainly be considered as configured when its postinst has been run even if man-db has not yet updated its cache to know about the new/updated manual page. When you activate a trigger with the dpkg-trigger command you have an option --no-await to avoid awaiting the trigger processing (and thus to go directly to installed state after the postinst has been run). But with file triggers or activate trigger directives, you do not have this option. My proposal to improve the situation This is the problem that I tried to solve during my last vacation. But before changing the inner working of triggers, I wrote a non-regression test suite for that feature (commit here) so I could hack with some confidence that I did not break everything. The result has been presented on the debian-dpkg mailling list: see the discussion here. I added new directives that can be used in triggers files that work exactly like the current triggers except that they do not put triggering packages in trigger-awaited status. I believe the code to be mostly ready, but in its current form the patch brings zero benefits until all packages have been converted to use the trigger variants that do not require awaiting trigger processing (and the change requires a pre-dependency on dpkg to ensure we have the required dpkg that understands the new kind of trigger directives). Remaining question Thus I wonder if I should not change the default semantic of triggers. The packages which really provide crucial functionality to awaiting packages through triggers would then have to be updated to switch to the new directives. If you re a packager using triggers, you can thus help me by answering this question: do you know some triggers where it s important that the awaiting packages are not considered as configured before the trigger processing? In most of the cases I checked, it s important for the triggered package rather than for the triggering package. In truth, a package in triggers-awaited status is usually in a good enough shape to be able to satisfy dependencies (i.e. requirements that other packages can have), but it would still be worth to record the fact that it s not entirely configured yet because it might be true from the user s point of view: for example if the menu trigger has not yet been processed, the software might not yet be visible in the application menu.

If you appreciate this kind of groundwork that benefits to the whole Debian ecosystem, please consider supporting my work. Click here and give it a look, there are many ways to contribute and to make a difference for me.

3 comments Liked this article? Click here. My blog is Flattr-enabled.

I m proud to be able to announce a new stable release of Grml, the Debian based Live system for system administrators. This release is a very special one for me. On the one hand of course because of the special release name Just Mari , being dedicated to my lovely wife. But it s also special because of the way the release management worked out. I ve been the release manager for Grml since the very beginning, which turned out to be more than 6 years since the first stable release already. I developed grml-live as build framework based on FAI for generating a Grml and Debian based Linux Live system to streamline the build process. Anyway it was mainly me who managed the release chroots, doing the update management during release freeze, editing main web page etc. As I want to make the project as independent from myself as much as possible and to keep the Bus factor in balance we started to improve our project infrastructure so it s not just me who can do this kind of release management. As a result Grml core developer Christian Hofstaedtler became the release sergeant of this release. The release candidate version was even released without myself being available (the release happened behind my back during our marriage). Major work on the final stable release was also done by Grml core developers Ulrich Dangel, Christian Hofstaedtler, Frank Terbeck and Alexander Wirt and I m very happy about that. Kudos guys for all your work and all the hidden efforts going on behind my back. The details regarding the new Grml release are available in the official release announcement, get the ISOs from grml.org/download. I hope you enjoy the release as much as I do. Happy hacking!

At April 30, I took over maintenance of of Debian's kFreeBSD autobuilders. Means getting something like 4,5k e-Mails this month (gladly no need to sign all those 4k successful builds any more!), filling nearly 30 RC Bugs (quite a lot of which got fixed just within hours after filling, wow!), investigating some of the more strange build failures and such stuff. In general it turned out to be quite some fun. Quite interesting which libraries turn out to be rather central to the Archive. I wouldn't have guessed that a uninstallable libmtp would prevent a reasonable fraction of the builds to fail -- including packages like subversion. Packages builds failing because the disk space is exhausted may be something most of us have already witnessed, especially those here that use one of these small notebook hard drives. Build failures caused by a lack of RAM might certainly be imaginable as well, especially on highly parallel builds. But have you ever seen gcc fail because the virtual address space was exhausted on 32 bit architectures? Also there's a interesting lot of packages with misspelled build dependencies which sbuild can't find and therefore can't build the package. Maybe having a lintian check for some of these problems would be a good idea? I'm also regularly seeing build failures that look easy enough to fix -- like some glob in a *.install for some python package matching lib.linux*. I try to fix some of these as I see them but my time is unfortunately limited as well. Someone interested in quick&easy noticed about these kind of issues? I could put URLs to build-logs on identi.ca or somewhere on IRC. There are also some really strange failures like llvm, which builds flawlessly on my local kFreeBSD boxes all the time, inside and outside schroot but hangs all the time in the same testcase when building on any of the autobuilders (any hints welcome!) or perl failing on kfreebsd-amd64 selectively but all the time.

I've got a few projects which are hosted in mercurial repositories. To deploy them I manually checkout the repository, create symlinks by hand, then update apache thttpd to make them work. When I want to update my applications I manually become the correct user, find the repository and run "hg pull --update". I think it is about time that I sat down and started doing things neatly. I made a start at this by writing a shell script for each site called .deploy then I drive it like so:

#!/bin/sh
#
# ~/bin/deploy  execute the .deploy file associated with this project.
#
while true; do
    #
    #  If we're at the root directory we're done.
    #
    if [ $PWD = "/" ]; then
        echo "Reached /"
        exit
    fi
    # found our file?
    #
    if [ -x ".deploy" ]; then
       ./.deploy
       exit
    fi
    cd ..
done

It seems the main candidate is capistrano, which was previously very Ruby on Rails centric, but these days seems to be divorced from it. Alternatively there is the python-based fabric project which has been stalled for two years, vlad the deployer (great name!) which is another Rake-based and thus Ruby-loving system, and finally whiskey disk which is limited to Git-based projects as far as I can tell. In short each of these projects is very similar, and each relies upon being able to do two things:

• SSH to remote machine(s) and run a command.
• Copy files to the remote command / pull a repository from a known location.

I've automated SSH before, and I've automed SCP/rsync. The hard part is doing both "copy" and "command" over one SSH channel - such that you don't get prompted for passwords multiple times - and handling the case of runnign sudo where appropriate. Still most of the initial stages are trivial. I wonder what project I should be using:

• I like perl. Perl is good.
• I use mercurial. Mercurial is good.
• Rake is perhaps permissable, but too ruby-centric == not for me.

Anything I've missed? Or pointers to good documentation? ObQuote: "We need to be a little more constructive here, okay? " - Terminator 2

So after some discussion I can share the first draft syntax of the control file that ll be used for declarative diversions

One diversion per line Blank lines and lines beginning with # are comments Two fields per diversion seperated by whitespace (Source Destination)

Hopefully the full design draft for the project will be available on Friday of this week

EDIT (30/05/2011 11AM GMT+1) I realise there will be some issues regarding spaces within file names. This will be addressed hopefully by the end of the week. If anyone happens to see anything obvious that I ve overlooked you can catch me on IRC as SCD[Eire]. I d appreciate all and any input

~Sam

The Mahara project has just moved to mandatory code reviews for every commit that gets applied to core code.

Here is a description of how Gerrit Code Review, the peer-review system used by Android, was retrofitted into our existing git repository on Gitorious.

(If you want to know more about Gerrit, listen to this FLOSS Weekly interview.)

Replacing existing Gitorious committers with a robotThe first thing to do was to log into Gitorious and remove commit rights from everyone in the main repository. Then I created a new maharabot account with a password-less SSH key (stored under /home/gerrit/.ssh/) and made that new account the sole committer.

This is to ensure that nobody pushes to the repository by mistake since all of these changes would be overwritten by Gerrit.

Basic Gerrit installationAfter going through the installation instructions, I logged into the Gerrit admin interface and created a new "mahara" project.

I picked the "merge if necessary" submit action because "cherry-pick" would disable dependency tracking which is quite a handy feature.

Reverse proxy using NginxSince we wanted to offer Gerrit over HTTPS, I decided to run it behind an Nginx proxy. This is the Nginx configuration I ended up with:

server  
     listen 443;
     server_name reviews.mahara.org;
     add_header Strict-Transport-Security max-age=15768000;

     ssl on;
     ssl_certificate  /etc/ssl/certs/reviews.mahara.org.crt;
     ssl_certificate_key  /etc/ssl/certs/reviews.mahara.org.pem;

     ssl_session_timeout  5m;
     ssl_session_cache shared:SSL:1m;

     ssl_protocols  TLSv1;
     ssl_ciphers  HIGH:!ADH;
     ssl_prefer_server_ciphers   on;

     location /  
             proxy_pass   http://127.0.0.1:8081;
             proxy_set_header X-Forwarded-For $remote_addr;
             proxy_set_header Host $host;
      
 

Things to note:

• An HTTP to HTTPS redirection is not provided.


• The HSTS headers indicates to modern browsers that this URL should only be accessed via HTTPS.


• Only strong SSL ciphers are enabled.


• Before proxying requests to Gerrit, Nginx adds a few headers to identify the origin of the request. The Host header in particular must be present otherwise built-in Gerrit redirections will fail.

Mail setupTo enable Gerrit to email reviewers and committers, I installed Postfix and used "reviews.mahara.org" as the "System mail name".

Then I added the following to /home/gerrit/mahara_reviews/etc/gerrit.config:

[user]
 email = "gerrit@reviews.mahara.org"

to fix the From address in outgoing emails.

Init script and cronFollowing the installation instructions, I created these symlinks:

ln -s /home/gerrit/mahara_reviews/bin/gerrit.sh /etc/init.d/gerrit
cd /etc/rc2.d && ln -s ../init.d/gerrit S19gerrit
cd /etc/rc3.d && ln -s ../init.d/gerrit S19gerrit
cd /etc/rc4.d && ln -s ../init.d/gerrit S19gerrit
cd /etc/rc5.d && ln -s ../init.d/gerrit S19gerrit
cd /etc/rc0.d && ln -s ../init.d/gerrit K21gerrit
cd /etc/rc1.d && ln -s ../init.d/gerrit K21gerrit
cd /etc/rc6.d && ln -s ../init.d/gerrit K21gerrit

and put the following settings into /etc/default/gerritcodereview:

GERRIT_SITE=/home/gerrit/mahara_reviews
GERRIT_USER=gerrit
GERRIT_WAR=/home/gerrit/gerrit.war

to automatically start and stop Gerrit.

I also added a cron job in /etc/cron.d/gitcleanup to ensure that the built-in git repository doesn't get bloated:

MAILTO=admin@example.com
20 4 * * * gerrit GIT_DIR=/home/gerrit/mahara_reviews/git/mahara.git git gc --quiet

Configuration enhancementsTo allow images in change requests to be displayed inside the browser, I marked them as safe in /home/gerrit/mahara_reviews/etc/gerrit.config:

[mimetype "image/*"]
 safe = true

Another thing I did to enhance the review experience was to enable the gitweb repository browser:

apt-get install gitweb

and to make checkouts faster by enabling anonymous Git access:

[gerrit]
  canonicalGitUrl = git://reviews.mahara.org/git/
[download]
  scheme = ssh
  scheme = anon_http
  scheme = anon_git

which requires that you have a git daemon running and listening on port 9418:

apt-get install git-daemon-run
ln -s /home/gerrit/mahara_reviews/git/mahara.git /var/cache/git/
touch /home/gerrit/mahara_reviews/git/mahara.git/git-daemon-export-ok

Finally, I included the Mahara branding in the header and footer of each page by providing valid XHTML fragments in /home/gerrit/mahara_reviews/etc/GerritSiteHeader.html and GerritSiteFooter.html.

Initial import and replicationOnce Gerrit was fully working, I performed the initial code import by using my administrator account to push the exiting Gitorious branches to the internal git repository:

git remote add gerrit ssh://username@reviews.mahara.org:29418/mahara
git push gerrit 1.2_STABLE
git push gerrit 1.3_STABLE
git push gerrit master

Note that I had to temporarily disable "Require Change IDs" in the project settings in order to import the old commits which didn't have these.

To replicate the internal Gerrit repository back to Gitorious, I created a new /home/gerrit/mahara_reviews/etc/replication.config file:

[remote "gitorious"]
 url = gitorious.org:mahara/$ name .git
 push = +refs/heads/*:refs/heads/*
 push = +refs/tags/*:refs/tags/*

(The $ name variable is required even when you have a single project.)

Contributor instructionsThis is how developers can get a working checkout of our code now:

git clone git://gitorious.org/mahara/mahara.git
cd mahara
git remote add gerrit ssh://username@reviews.mahara.org:29418/mahara
git fetch gerrit
scp -p -P 29418 reviews.mahara.org:hooks/commit-msg .git/hooks/

and this is how they can submit local changes to Gerrit:

git push gerrit HEAD:refs/for/master

Anybody can submit change requests or comment on them but make sure you do not have the Cookie Pie Firefox extension installed or you will be unable to log into Gerrit.

Navigating the openssl suite of subcommands is time consuming. Here's my list of frequently used commands for easy reference: Examining the certificates being offered by a host (replace imap with any service, -starttls imap can be eliminated if you are checking a service that provides tls from the start):

openssl s_client -starttls imap -showcerts -connect chavez.mayfirst.org:imap

Examine a certificate signing request:

openssl req -in /path/to/file.csr -text -verify -noout

Examine a x509 certificate:

openssl x509  -noout -text -purpose -in path/to/file.crt

Memorial Day weekend so following the annual JPMorgan Chase CC run on Thursday, it was once again time for the annual Bike The Drive on Chicago's beautiful Lakeshore Drive. The weather was craptastic---following a fog (!!) warning we had dense fog along the lake making it somewhat moist and chilly. Once we were done this was followed by an impressive thunderstorm warnings and rather heavy rain. But the girls were troopers and we did both halfes for a total of thirty miles as shown on this Google Maps record of the ride.

I happened to have some Arum Lillies on hand this week after taking some shots for use by a local funeral director. This is one of the shots I took once I had finished.