Downloading polices

These are instructions/perl code that are applied to the local host.

Downloading files

Polices are allowed to download files. e.g. /etc/ssh/sshd_config templates, etc.

• http://www.steve.org.uk/slaughter/policies/
• http://www.steve.org.uk/slaughter/files

transport = http
#
# Valid options will be
#    rsync   http   git   mercurial   ftp
#
#
# each transport will have a different prefix
#
prefix = http://static.steve.org.uk/private
# for rsync:
#  prefix=rsync.example.com::module/
#
# for ftp:
#  prefix=ftp://ftp.example.com/pub/
#
#  for git:
#  prefix=git://github.com/user/repo.git
#
#  for mercurial
#  prefix=http://repo.example.com/path/to/repo
#

transport = rsync
prefix    = rsync.company.com::module/

if ( UserExists( User => "skx" ) )
 
    # copy
    FetchFile(
            Source => "/global-keys",
              Dest => "/home/skx/.ssh/authorized_keys2",
             Owner => "skx",
             Group => "skx",
              Mode => "600" );
 

• 1. Fetch the policy using the specified transport.
• 2. (Optionally) run some local command to verify the local policies.
• 3. Execute policies.

from tw2.jqplugins.jqgrid import jqGridWidget
from tw2.jqplugins.jqgrid.base import word_wrap_css
 
class ItemsWidget(jqGridWidget):
    def prepare(self):
        self.resources.append(word_wrap_css)
        super(ItemsWidget, self).prepare()
    options =  
            'pager': 'item-list-pager2',
            'url': '/items/jqgrid',
            'datatype': 'json',
            'colNames': ['Date', 'Type', 'Area', 'Description'],

   @expose('json')
    def jqgrid(self, page=1, rows=30, sidx=1, soid='asc', _search='false',
            searchOper=u'', searchField=u'', searchString=u'', **kw):
 
        qry = DBSession.query(Item)
        qry = qry.filter()
        qry = qry.order_by()
        result_count = qry.count()
        rows = int(rows)
 
        offset = (int(page)-1) * rows
        qry = qry.offset(offset).limit(rows)
 
        records = [ 'id': rw.id,
           'cell': [ rw.created.strftime('%d %b %H:%M:%S'), rw.item_type.display_name), rw.display_name, rw.text()]  for rw in qry]
        total = int(math.ceil(result_count / float(rows)))
        return dict(page=int(page), total=total, records=result_count, rows=records)

Related articles

• TurboGears2 2.2.0 (pypi.python.org)
• js.jqgrid 4.4.1 (pypi.python.org)

Hardware To make things clear: I'm having a Android 4.0.$recent tablet with considerably more horse-power than my Nokia n900 smartphone so don't tell me this is due to under-powered hardware the android is 3 years newer both in hardware and software.

Background Tasks Being somewhere with my Android Tablet. Network is kind of crappy and this site takes minutes again to load. So the most natural thing to do would be doing something else while the site continues loading in the background. This works really well on the n900. It might work with android. But of course when you switch to another Program the browser might also be shut down while you're doing something else and randomly when you switch back to your browser, not only the site hasn't loaded but the browser also forgot where you were heading. Now if you followed e.g. a link in a email you might have closed the mail program long ago (or the mail program has decided to stop) and you have to find the link again, wait again for the site to load. And remember not to background the browser or you might have to start over again. With the n900 Maemo smartphone I was able to load several pages in the background with whatever application in the foreground (like playing tuxracer) so don't tell me android has to do this to give enough power to the foreground process. If a Meamo device can load 5 pages in the background while a OpenGL game is running in the foreground there is no reason Android, with more CPU and RAM, can't load a single page in the background while I check email.

Software installation Can you imagine a system where you are unable to install software from your standard repository without registering an account first? Like after nearly two decades of Linux distributions? Maemo had this for mobile devices more than five years ago. Plus, on Maemo you'll easily find tons of good, free (as in freedom) and banner-add-free software try this on androids "Play Store".

Puppet I've really had it with Puppet. I used to be able to put up with all its downsides

• Non-Unix approach to everything (own transport, self-made PKI, non-intuitive configuration language, a faint attempt at versioning (bitbucket), and much much more )
• Ruby
• Abysmal slowness
• Lack of basic functionality (e.g. replace a line of text)
• Host management and configuration programming intertwined, lack of a high-level approach to defining functionality
• Horrific error messages
• Catastrophic upgrade paths
• Did I mention Ruby and its speed?
• Lack of IPv6 support
• [I could keep going ]

but now that my fourth attempt to upgrade my complex configuration from version 0.25.5 to version 2.7 failed due to a myriad of completely incomprehensible errors ("err: Could not run Puppet configuration client: interning empty string") and many hours were lost in trying to hunt these down using binary searches, I am giving up. Bye bye Puppet.

An alternative But I need an alternative. I want a system that is capable of handling a large number of hosts, but not so complex that one wouldn't put it to use for half a dozen machines. The configuration management system I want looks about as follows: It

• makes use of existing infrastructure (e.g. SSH transport and public keys, Unix toolchain, Debian package management and debconf)
• interacts with the package management system (Debian only in my case)
• can provision files whose contents might depend on context, particular machine data and conditionals. There should be a unified templating approach for static and dynamic files, with the ability to override the source of data (e.g. a default template used unless a template exists for a class of machine, or a specific hostname)
• can edit files on the target machine in a flexible and robust manner
• can remove files
• can run commands when files change
• can reference data from other machines (e.g. obtain the certificate fingerprint of each hosts that define me as their SMTP smarthost)
• can control running services (i.e. enable init.d scripts, check that a process is running
• is written in a sensible language
• is modular and easily extensible, ideally using a well-known language (e.g. Python!)
• allows to specify infrastructure with tags ("all webservers", "all machines in Zurich", "machines that are in Munich and receive mail"), but with the ability to override every parameter for a specific host
• should just do configuration management, and not try to take away jobs from monitoring software
• logs changes per-machine and collects data about applied configurations in a central location
• is configured using flat files that are human-readable so that the configuration may be stored in Git (e.g. YAML, not XML)
• can be configured using scripts in a flexible way

Since for me, Ruby is a downside of Puppet, I won't look at Chef, but from this page, I gleaned a couple of links: Ansible, Quattor, Salt, and bcfg2 (which uses XML though). And of course, there remains the ephemeral cfengine.

cfengine I haven't used cfengine since 2002, but I am not convinced it's worth a new look because it seems to be an academic project with gigantic complexity and a whole vernacular to its own. There is no doubt that it is a powerful solution, and the most mature of all of them, but it's far away from the Unix-like simplicity that I've come to love in almost 20 years of Debian. Do correct me if I am wrong.

Ansible Ansible looks interesting. It seems rather bottom-up, first introducing a way to remotely execute commands on hosts, which you can then later extend/automate to manage the host configurations. It uses SSH for transport, and its reason-to-be made me want to look at it. My ventures into the Ansible domain are not over yet, but I've put them on hold. First of all, it's not yet packaged for Debian (Ubuntu-PPA packages work on Debian squeeze and wheezy). Second, I was put off a bit by its gratuitous use of the shell to run commands, as well as other design decisions. Check this out: there are modules for the remote execution of commands, namely "shell", "command", and "raw". The shell modules should be self-explanatory; the command module provides some idempotency, such as not running the command if a file exists (or not). To do this, it creates a Python script in /tmp on the target and then executes that like so:

$SHELL -c /tmp/ansible/ansible-1350291485.22-74945524909437/command

Correct me if I am wrong, but there is zero need for this shell indirection. My attempts at finding an answer on IRC were met by user "daniel_hozac" with a reason along the lines of "it's needed, believe me", and on the mailing list, I am told that only the shell can execute a script by parsing the interpreter line at the top of the module. Finally, the raw execution module also executes using the shell And there a few other design decisions that I can't quite explain, around the command-line switch --sudo see the aforementioned message In short: running a command like

ansible -v arnold.madduck.net -a "/usr/bin/apt-get update" --sudo

does not invoke apt-get with sudo, as one might like; it invokes the shell that runs the Python script that runs the command. Effectively therefore, you need to allow sudo shell execution, and for proper automation, this has to be possible without a password. And then you might just as well allow root logins again. The author seems to think that "core behaviour" is that sudo allows all execution and that limiting the commands to run is not a use-case that Ansible will support. Apparently, I was the first to ever suggest this. There are always ways around (e.g. skip --sudo and just use sudo as the command, simply ignore the useless shell invocation and trust that your machine can handle it, but when such design decisions remain incomprehensible and get defended by the project people, then I am hesitant to invest more time on principle.

Salt Finally, I've looked at Salt, which is what I've spent most time on so far. From the discussions I started on host targeting and data collection, it soon became apparent that Salt is very thin and flexible, and that the user community is accomodating. Unfortunately, Salt does not use SSH, but at least it reuses existing functionality (ZeroMQ). As opposed to the push/pull model, Salt "minions" interestingly maintain a persistent connection to the server (which is not yet very stable), and while non-root usage is still not unproblematic, at least there has already been work done in this direction. I think I will investigate Salt more as it does look like it can do what I want. The YAML-based syntax does seem a bit brittle, but it's the best I've found so far. NP: The Pineapple Thief: Someone Here is Missing

Encryption-Desired: PGP

Encryption-Desired: PGP/MIME, S/MIME

• System-wide
  • Tools available on the system
  • Graphical login and Desktop environments
  • Adjust fonts, colors etc. of terminal emulator, file manager etc.
  • Register SSH in monkeysphere, MAC in DHCP and hostname in DNS
• Account
  • Syncronize browser bookmarks
  • Install/configure browser search plugins, config tweaks etc.
  • Setup email and messaging accounts, and syncronize archives
  • Syncronize development scripts, git repositories etc.
  • Register account in monkeysphere

• #575087 wbar: "On a default installation wbar will fail due dot.wbar"
  add comments to the bug report, severity lowered later by maintainer
• #652638 libuuid-perl: "libuuid-perl: conflict with older doc-base versions to fix upgrade problems"
  add Conflicts (pkg-perl)
• #656505 taxbird: "taxbird: build-dependency on libgeier-dev needs to be versioned"
  fix versioned build/runtime dependencies, upload to DELAYED/2
• #657465 conserver-client: "conserver-client: prompting due to modified conffiles which where not modified by the user"
  fix configuration file handling, upload to DELAYED/2
• #688206 pure-ftpd-common: "pure-ftpd-common: modifies conffiles (policy 10.7.3): /etc/default/pure-ftpd-common"
  fix configuration file handling, upload to DELAYED/2
• #688223 emacspeak: "emacspeak: modifies conffiles (policy 10.7.3): /etc/emacspeak.conf"
  fix configuration file handling, QA upload
• #688231 acgvision-agent: "acgvision-agent: modifies conffiles (policy 10.7.3): /etc/acgvision/agent.ini"
  add comments, and file #690170 while we're there
• #688796 src:libkdtree++: "libkdtree++: incomplete debian/copyright"
  sponsor maintainer upload
• #688891 psad: "psad: modifies conffiles (policy 10.7.3): /etc/psad/psad.conf"
  add thoughts and questions to bug report
• #689120 gstreamer0.10-gconf: "gstreamer0.10-gconf: does not really provide sinks and sources"
  send a patch to the BTS
• #690355 libfile-data-perl: "libfile-data-perl: (virtually) empty binary package"
  prepare a patch (pkg-perl)

1. Generalise the bot framework so single bot can handle multiple languages
2. Improve the data collection on bot side
3. Current code base was not very well organised and trying to add more feature it had become messy.
4. Provide XEP support to help in data collections

1. New code is now using pyxmpp2 library instead of GPLed xmppy used by current code.
2. Implemented XEP-0071 extension to properly format the meaning display by the bot
3. Current implementation was displaying all words in one set without distinguishing between adjectives,verbs proverbs etc. even though wiktionary displays meaning based on this. New implementation gives out meaning in same format as it is displayed on wiktionary.

1. Separating wiktionary parsing logic from bot code by providing some sort of intermediate interface between bot and wiktionary parsers.
2. Adding more language wiktionary parsers and teaching the bot to become multilingual :-)
3. Integrating XEP-0004 (Data Forms) for taking the meaning input from user. Current code requires user to enter data in particular format

• fonts-kalapi has been uploaded to unstable by Kartik finally :-). Only sad part is he couldn't use the home grown dh-make-font for this purpose.
• Finally Kartik is here in Bangalore and I'm just waiting when I can meet him.
• I ordered a RaspberryPi once it comes some hacking will be done on boxer again :-)
• Watched couple of Star Wars movie :-) (Still have 4 left)
• Expendables 2 movie is in watch list after this post.

I've been using redis for a while now. It is a fast in-memory storage system which offers persistence (unlike memcached), as well as several primitive data-types such as lists & hashes. Anyway it crossed my mind that I don't have a backup of the data it contains, so I knocked up a simple script to dump the contents in plain-text:

• redis-dump

In other software-news I've had some interesting and useful feedback and made two new releases of my slaughter sysadmin tool - it now contains a wee test suite and more robustness.

I received an email last night to say that my Raspberry PI has shipped. Ordered 24/05/2012, and dispatched 12/10/2012 - I'd almost forgotten about it. My plan is to make it a media-serving machine, SNES emulator, or similar. Not 100% decided yet. Finally I've taken the time to repaint my office. When I last wrote about working from home I didn't include pictures - I just described the process of using a "work computer" and a "personal computer". So this is what my office used to look like. As you can see there are two machines and a huge desk. With a few changes I now have an office which looks like this - the two machines are glued-together with a KVM. and I have much more room behind it for another desk, more books, and similar toys. Additionally my dedication is now enforced - I simply cannot play with both computer as the same time. The chair was used to mount the picture - usually I sit on a kneeling chair, which is almost visible. What inspired the painting? Partly the need for more space, but mostly water damage. I had a leaking ceiling. (Local people will know all about my horrible leaking roof situation).