Search Results: "xam"

27 June 2010

Matt Zimmerman: Navigating the PolicyKit maze

I ve written a simple application which will automatically extract media from CDs and DVDs when they are inserted into the drive attached to my server. This makes it easy for me to compile all of my media in one place and access it anytime I like. The application uses the modern udisks API, formerly known as DeviceKit-disks, and I wrote it in part to learn get some experience working with udisks (which, it turns out, is rather nice indeed). Naturally, I wanted to grant this application the privileges necessary to mount, unmount and eject removable media. The server is headless, and the application runs as a daemon, so this would require explicit configuration. udisks uses PolicyKit for authorization, so I expected this to be very simple to do. In fact, it is very simple, but finding out exactly how to do it wasn t quite so easy. The Internet is full of web pages which recommend editing /etc/PolicyKit/PolicyKit.conf. As far as I can tell, nothing pays attention to this file anymore, and all of these instructions have been rendered meaningless. My system was also full of tools like polkit-auth, from the apparently-obsolete policykit package, which kept their configuration in some other ignored place, i.e. /var/lib/PolicyKit. It seems the configuration system has been through a revolution or two recently. In Ubuntu 10.04, the right place to configure these things seems to be /var/lib/polkit-1/localauthority, and this is documented in pklocalauthority(8). Authorization can be tested using pkcheck(1), and the default policy can be examined using pkaction(1). I solved my problem by creating a file in /var/lib/polkit-1/localauthority/50-local.d with a .pkla extension with the following contents:
[Access to removable media for the media group]
Identity=unix-group:media
Action=org.freedesktop.udisks.drive-eject;org.freedesktop.udisks.filesystem-mount
ResultAny=yes
This took effect immediately and did exactly what I needed. I lost quite some time trying to figure out why the other methods weren t working, so perhaps this post will save the next person a bit of time. It may also inspire some gratitude for the infrastructure which makes all of this work automatically for more typical usage scenarios, so that most people don t need to worry about any of this. Along the way, I whipped up a patch to add a --eject option to the handy udisks(1) tool, which made it easier for me to test along the way.

26 June 2010

Anand Kumria: Get Low

A story about getting people together to tell them a story of regret involving the main character, which we see at the start. The story itself isn't very dramatic. Which is why there isn't a film about that. Instead we have the story about getting people to tell the story. This is a great example of a movie with a mind-boggingly amazing cast (Bill Murray, Robert Duvell, Sissy Spacek, Lucas Black) having a script that is crap. Having the director present, to discuss the film, after the screening lifted this from a 1 to 2. Hearing Aaron talk about the different styles of Duvell, Murray and how the editing process was actually done (Final Cut Pro) was very entertaining. The story drifts, the soundtrack doesn't annoy and the filmography is alright. Oh, the period setting is done very well. Not much else to recommend it though. Get Low refers to burying in the film. Something I think that should have happened to this film.

25 June 2010

Anand Kumria: Beginning Django Development

Oddly, I have had four different people ask me about the best way to begin a Django project or setup their system to make it easy to develop with Django. Since I've now given my response via email so many times, I figure I might as well broadcast it and hopefully this will help others as well My assumptions: What we will do is create a private binary directory and another Python modules. Then take a checkout of the two most important Django projects, and make them work. First: create your user
$ sudo adduser newbie
$ ssh localhost -l newbie
$ whoami
newbie
Then we want to create some local bin and lib directories. First the binary directory $ mkdir ~/bin # On Ubuntu/Debian, this will automatically be in your PATH the next time you login And now a local directory for our various libraries
$ mkdir -p ~/lib/python
$ echo "PYTHONPATH=~/lib/python:" >> ~/.bashrc
$ echo "export PYTHONPATH" >> ~/.bashrc
This will add that directory to our Python path. If you happen to also use another language you can put things into ~/lib/ruby, ~/lib/perl as appropriate
$ mkdir ~/Projects
$ cd ~/Projects
Here is where we will store copies of upstream software. What is the reason for using the repositories rather than packages? This allows us to checkout specific versions to match what our clients might be using. Or test things against newer versions of the upstream project. First, let's setup Django. I tend to use git by default, especially if the upstream is using Subversion. If they are using Mercurial or Bazaar, I use those directly.
$ git svn clone -s http://code.djangoproject.com/svn/django/
Initialized empty Git repository in /home/newbie/Projects/django/.git/
Using higher level of URL: http://code.djangoproject.com/svn/django => http://code.djangoproject.com/svn
r1 = 5cda37203ffa6ea83da2958a95c377984482877f (refs/remotes/trunk)
	A	django-docs/images/flatfiles_admin.png
	A	django-docs/images/users_changelist.png
	A	django-docs/model-api.txt
	A	django-docs/build.py
	A	django-docs/db-api.txt
	A	django-docs/writing-apps-guide-outline.txt
	A	django-docs/templates.txt
r2 = b8249ac45e2154933b9649fd8181d5769e31c9fc (refs/remotes/trunk)
	A	django/utils/feedgenerator.py
	A	django/utils/datastructures.py
[...]
r13399 = f3902c67a3b8788de2145899e435a394c512b455 (refs/remotes/releases)
	M	tests/regressiontests/m2m_through_regress/tests.py
r13400 = cd72207306a5a4eecdf07f65c109f37c8317ed81 (refs/remotes/trunk)
$
Now we have Django, push it into our python path
$ cd ~/lib/python
$ ln -s ~/Projects/django/django/
$ python
Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
>>> django.VERSION
(1, 2, 1, 'final', 0)
Now, let's do the same for South. South is a database migration helper for Django. These are the two things you want to have in any Django at a minimum. There are also plenty of other amazing things like haystack, piston, satchmo, etc. You can follow the same recipie for them too.
$ cd ~/Projects
$ hg clone http://bitbucket.org/andrewgodwin/south/
destination directory: south
requesting all changes
adding changesets
adding manifests
adding file changes
added 802 changesets with 1340 changes to 183 files (+1 heads)
updating to branch default
143 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd ~/lib/python
$ ln -s ~/Projects/south/south
$ python 
Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import south
>>> south.__version__
'0.7.1'
Finally, we want to be able to create Django projects. There is a great helper called django-admin that will do this. Let's put it into our path
$ cd ~/bin
$ ln -s ~/Projects/django/django/bin/django-admin.py
$ hash -r
That last statment will cause your interpreter to re-read PATH and make anything new available for execution. Now, you should go ahead and create your projects. There are a variety of ways to do this. However what I like to do is keep Django applications separate from the Django project. That way, they can easily be re-used if required. So, for a project called 'foo', we might have
$ mkdir -p ~/Work/foo
$ cd ~/Work/foo
$ mkdir foo.example.com
$ mkdir templates
$ mkdir media
$ mkdir <individual apps>
For each individual app, put them into the Python library. And you then have your re-usability from within Django. For a application called 'bar', you would do:
$ cd ~/Work/foo
$ django-admin.py startapp bar
$ cd ~/lib/python
$ ln -s ~/Work/foo/bar
And now, in your Django foo.example.com setting.py's file you can put 'bar' as one the installed applications and things will Just Work Obviously there are many way to slice this particular mango, but I've found that this works pretty well for me. You can spruce it up by revision controlling each directory in your project (I do) and also take advantage of things like virtualenv and Fabric to make deploying just as easy as developing. But I'll leave those topics until a later date

24 June 2010

David Bremner: Yet another tale of converting Debian packaging to Git

racket (previously known as plt-scheme) is an interpreter/JIT-compiler/development environment with about 6 years of subversion history in a converted git repo. Debian packaging has been done in subversion, with only the contents of ./debian in version control. I wanted to merge these into a single git repository. The first step is to create a repo and fetch the relevant history.
TMPDIR=/var/tmp
export TMPDIR
ME= readlink -f $0 
AUTHORS= dirname $ME /authors
mkdir racket && cd racket && git init
git remote add racket git://git.racket-lang.org/plt
git fetch --tags racket
git config  merge.renameLimit 10000
git svn init  --stdlayout svn://svn.debian.org/svn/pkg-plt-scheme/plt-scheme/
git svn fetch -A$AUTHORS
git branch debian
A couple points to note: Now a couple complications arose about upstream's git repo.
  1. Upstream releases seperate source tarballs for unix, mac, and windows. Each of these is constructed by deleting a large number of files from version control, and occasionally some last minute fiddling with README files and so on.
  2. The history of the release tags is not completely linear. For example,
rocinante:~/projects/racket  (git-svn)-[master]-% git diff --shortstat v4.2.4  git merge-base v4.2.4 v5.0 
 48 files changed, 242 insertions(+), 393 deletions(-)
rocinante:~/projects/racket  (git-svn)-[master]-% git diff --shortstat v4.2.1  git merge-base v4.2.1 v4.2.4 
 76 files changed, 642 insertions(+), 1485 deletions(-)
The combination made my straight forward attempt at constructing a history synched with release tarballs generate many conflicts. I ended up importing each tarball on a temporary branch, and the merges went smoother. Note also the use of "git merge -s recursive -X theirs" to resolve conflicts in favour of the new upstream version. The repetitive bits of the merge are collected as shell functions.
import_tgz()  
    if [ -f $1 ]; then
        git clean -fxd;
        git ls-files -z   xargs -0 rm -f;
        tar --strip-components=1 -zxvf $1 ;
        git add -A;
        git commit -m'Importing ' basename $1 ;
    else
        echo "missing tarball $1";
    fi;
 
do_merge()  
    version=$1
    git checkout -b v$version-tarball v$version
    import_tgz ../plt-scheme_$version.orig.tar.gz
    git checkout upstream
    git merge -s recursive -X theirs v$version-tarball
 
post_merge()  
    version=$1
    git tag -f upstream/$version
    pristine-tar commit ../plt-scheme_$version.orig.tar.gz
    git branch -d v$version-tarball
 
The entire merge script is here. A typical step looks like
do_merge 5.0
git rm collects/tests/stepper/automatic-tests.ss
git add  git status -s   egrep ^UA   cut -f2 -d' ' 
git checkout v5.0-tarball doc/release-notes/teachpack/HISTORY.txt
git rm readme.txt
git add  collects/tests/web-server/info.rkt
git commit -m'Resolve conflicts from new upstream version 5.0'
post_merge 5.0
Finally, we have the comparatively easy task of merging the upstream and Debian branches. In one or two places git was confused by all of the copying and renaming of files and I had to manually fix things up with git rm.
cd racket   /bin/true
set -e
git checkout debian
git tag -f packaging/4.0.1-2  git svn find-rev r98 
git tag -f packaging/4.2.1-1  git svn find-rev r113 
git tag -f packaging/4.2.4-2  git svn find-rev r126 
git branch -f  master upstream/4.0.1
git checkout master
git merge packaging/4.0.1-2
git tag -f debian/4.0.1-2
git merge upstream/4.2.1
git merge packaging/4.2.1-1
git tag -f debian/4.2.1-1
git merge upstream/4.2.4
git merge packaging/4.2.4-2
git rm collects/tests/stxclass/more-tests.ss && git commit -m'fix false rename detection'
git tag -f debian/4.2.4-2
git merge -s recursive -X theirs upstream/5.0
git rm collects/tests/web-server/info.rkt
git commit -m 'Merge upstream 5.0'

23 June 2010

Bernd Zeimetz: creating a google sitemap for ikiwiki

ikiwiki is not yet able to create a Google sitemap internally, so I'm using google-sitemapgen. To run it automatically when the website is being updated, I've changed the git hook to run it after the ikiwiki hook. /path/to/myikiwiki.git/hooks/post-update
#!/bin/sh
/path/to/myikiwiki.git/hooks/post-update.ikiwiki
/usr/bin/google-sitemapgen --config=/path/to/mywikiconfig/sitemap_config.xml
exec git-update-server-info
/path/to/mywikiconfig/sitemap_config.xml
<?xml version="1.0" encoding="UTF-8"?>
<site
  base_url="http://bzed.de/"
  store_into="/path/to/bzed.de/sitemap.xml.gz"
  verbose="0"
  suppress_search_engine_notify="0"
  default_encoding="UTF-8"
  >
  <directory
    path="/path/to/bzed.de"
    url="http://bzed.de/"
    default_file="index.html"
  />
  <!-- Exclude URLs that end with a '~'   (IE: emacs backup files)      -->
  <filter  action="drop"  type="wildcard"  pattern="*~"           />
  <!-- Exclude URLs within UNIX-style hidden files or directories       -->
  <filter  action="drop"  type="regexp"    pattern="/\.*"     />
  <!-- Exclude ikiwiki directories -->
  <filter  action="drop"  type="regexp"  pattern="/helponformatting/*"           />
  <filter  action="drop"  type="regexp"  pattern="/ikiwiki/*"                    />
  <filter  action="drop"  type="regexp"  pattern="/markdown/*"                   />
  <filter  action="drop"  type="regexp"  pattern="/openid/*"                     />
  <filter  action="drop"  type="regexp"  pattern="/pagespec/*"                   />
  <filter  action="drop"  type="regexp"  pattern="/preprocessordirective/*"      />
  <filter  action="drop"  type="regexp"  pattern="/sandbox/*"                    />
  <filter  action="drop"  type="regexp"  pattern="/shortcuts/*"                  />
  <filter  action="drop"  type="regexp"  pattern="/smileys/*"                    />
  <filter  action="drop"  type="regexp"  pattern="/subpage/*"                    />
  <filter  action="drop"  type="regexp"  pattern="/templates/*"                  />
  <filter  action="drop"  type="regexp"  pattern="/theme/*"                      />
  <filter  action="drop"  type="regexp"  pattern="/wikiicons/*"                  />
  <filter  action="drop"  type="regexp"  pattern="/wikilink/*"                   />
  <filter  action="drop"  type="regexp"  pattern="/wmd/*"                        />
  <!-- Exclude css files, favicon and javascript -->
  <filter  action="drop"  type="wildcard"    pattern="*.css"                         />
  <filter  action="drop"  type="wildcard"    pattern="*favicon.ico"                  />
  <filter  action="drop"  type="wildcard"    pattern="*.js"                          />
  <!-- Exclude ikiwiki.cgi -->
  <filter  action="drop"  type="wildcard"    pattern="ikiwiki.cgi"                   />
</site>
See the examples and README files in /usr/share/doc/google-sitemapgen/ for an introduction into configuring google-sitemapgen. So far the generated sitemaps works very well, especially for search engines which are not able to use the rss feeds like Google does.

Steve Kemp: I am the edge!

Over the past few years I've amassed a collection of a few thousand images taken with a succession of digital cameras. I'm pretty good at organising images, in a directory hierarchy which makes sense to me, in a few simple and broad categories:
skx@birthday:~$ tree -L 1 ~/Images/
/home/skx/Images/
 -- Misc
 -- Parties
 -- People
 -- Pets & Animals
 -- Travel
Beneath ~/Images/People, for example, I have subdirectories for specific individuals (or a "Debian/" folder for Debian-people who've been snapped but don't warrant their own folder.) ~/Images/Travel has things like Travel/Local/2010, Travel/Vienna/2008, etc. In summary I have images of people, places, and things stored beneath what should be a reasonably discoverable directory hierarchy, however this just doesn't work. I still struggle to find images - for example images of myself might be located in ~/Images/People/Self/*, but in practise I'm often included in ~/Images/Travel/* as well. A few times I've looked at using f-spot, digikam, and similar tools to perform image-organisation (but not editing, or timelines, or anything else. Just organisation). I've found I didn't like being locked into their formats, didn't want them to copy my images to a second location, and other gripes. In the end I've forced myself to come up with a Steve-Specific-Solution. Not for the first time, but I think I have just cause... I'm now using the User-Comment field in the image's EXIF data to store tags. (When it comes to EXIF data I keep camera-generated fields, but sometimes update/set "Copyright", "Comment", and "Title" fields. So UserComment is one I've never used until now, and thus I run no risk of trashing existing meta-data.) I've put together a simple perl script, called itag, which will: This is enough for me to be able to see all images of "Edinburgh", via:
~$ qiv --fullscreen --slide --delay 5 $(itag --search=edinburgh)
Similarly I could find myself:
~$ itag --search=steve --search=people
I'm not sure it is useful to others, mostly on the basis that people probably fall into their own routine when it comes to filing, and I suspect that people with vast collections of images will just get annoyed by the obscenely slow indexing process I've got. (Hint: run "exiftool" on every /.jpe?g$/i file..) Still its a simple enough idea and I think it should scale in the future - I can even see myself writing a wee GUI to do tag exploration and similar. Just not today. ObFilm:Aeon Flux ObRandom: Apologies for people waiting on email - it's been that kind of week.

22 June 2010

Martin Zobel-Helas: Preview of updated page layout for lists.d.o available!

First of all, THANKS Rhonda for pushing me to do that! I did plan to integrate the Debian menubar for a long time, but it never made it high enough on my todo lists up to now.

So, what am i talking about? For those of you who don't follow debian-www@l.d.o too closely, there was a recent thread about debian.org's page layout, where Rhonda pointed the initial poster to the layout proposals from Kalle S dermann. Rhonda was so kind and mainly documented in the recent blog post how to convert the gitweb theme to that layout.

I took that documentation and sat down Friday evening and converted the layout of my local instance of the DSA internal wiki to that new layout. While not everything looks perfect yet, it took me less than an hour. WOW, that was fast.

Now i became megalomaniac. After copying over the current list archive to an other machine (to not destroy the current archive while playing with mhonarc), I started playing with the same layout for lists.debian.org. The whole scripts for lists.debian.org were a bit more complex than the ikiwiki code we use for dsa.debian.org, but i managed to render usefull pages yesterday early morning.

Not everything was easyly convertable, and I still have some smaller issues to work on, but if you compare for example the following posting using the old and the new layout, i think we can use my work as basis for further improvements of the layout. Eg. i am aware that the new HTML code does not fullly validate using the w3c validator.

Many thanks also go to Kalle, who responded to my problems with the CSS immediatly.

Not all lists are converted to the new layout yet, as a full list archive rebuild seems to run about 24 hours. Also i adjusted some minor stuff in the templates while the rebuild was running, so you will see some smaller differences in the breadcrumbs. That will go away when I start the next rebuild.

So what is next? Rhonda, do we want to see if we can take over qa.d.o? ;-)

PS: if someone wants to generate new icons for the thread view arrows, i am happy to integrate them.

14 June 2010

Sami Haahtinen: Bacula & OpenERP

I've been working on setting up OpenERP for my needs and today I decided it was time to work on backing up the beast. Since I've been running bacula at home to backup my environment, it was time to tweak it so that it made reasonable backups of OpenERP too. In the end I was able to build a really elegant solution for backing it all up. I decided to go for the bpipe plugin that allows one to pipe programs directly to the bacula file daemon. This allowed me to do a live dump of the database with pg_dump and store it directly to the backup set without writing it to the disk. Since the other examples in bacula wiki define methods that either use files or FIFO to do the backup, I documented my setup there too. The only thing that was left was to add the directories specific for OpenERP to the backup and I was all set.

Russell Coker: Should Passwords Expire?

It s widely regarded that passwords should be changed regularly. The Australian government declared last week the National Cyber Security Awareness Week [1] and has published a list of tips for online security which includes Get a stronger password and change it at least twice a year . Can a Password be Semi-Public? Generally I think of a password as being either secret or broken. If a password is known to someone other than the sysadmin and the user who is authorised to use the account in question then you have potentially already lost all your secret data. If a password is disclosed to an unauthorised person on one occasion then merely changing the password is not going to do any good unless the root cause is addressed, otherwise another anothorised person will probably get the password at some future time. Hitachi has published a good document covering many issues related to password management [2]. I think it does a reasonable job of making sysadmins aware of some of the issues but there are some things I disagree with. I think it should be used as a list of issues to consider rather than a list of answers. The Hitachi document lists a number of ways that passwords may be compromised and suggests changing them every 60 to 90 days to limit the use of stolen passwords. This seems to imply that a password is something that s value slowly degrades over time as it s increasingly exposed. I think that the right thing to do is to change a password if you suspect that it has been compromised. There s not much benefit in having a password if it s going to be known by unauthorised people for 89 days before being changed! Fundamentally a password is something that can have it s value rapidly drop to zero without warning. It doesn t wear out. Why are terms such as Three Months used for Maximum Password Ages? The Hitachi document gives some calculations on the probability of a brute-force attack succeeding against a random password with 90 days of attacking at a rate of 100 attempts per second [2]. I think that if a service is run by someone who wouldn t notice the load of 100 attempts per second then you have bigger security problems than the possibility of passwords being subject to brute-force attacks. Also it s not uncommon to have policies to lock accounts after as few as three failed login attempts. Rumor has it that in the early days of computing when the hashed password data was world readable someone calculated that more than 3 months of CPU time on a big computer would be needed to obtain a password by brute-force. But since then the power of individual CPUs has increased dramatically, computers have become cheap enough that anyone can easily gain legal access to dozens of systems and illegal access to a million systems, and it has become a design feature in every OS that hashed passwords are not readable by general users. So the limiting factor is to what degree the server restricts the frequency of password guesses. I don t think that specifying the minimum password length and maximum password age based on the fraction of the key space that could be subject to a brute-force attack makes sense. I don t think that any attempt to make an industry-wide standard for the frequency of password changes (as the government is trying to do) makes sense. Can there be a Delay between a Password being Compromised and being Used by an Attacker? Hypothetically speaking, if a password was likely to be compromised (EG by having the paper it was written on lost or stored insecurely) for some time before an attacker exploited it, then if the password was changed during that time period it could solve the problem. For example when a company moves office there is the possibility of notepaper with passwords to be lost. So if the sysadmin caused every user password to expire at the time of the move then a hostile party would be unable to gain access. Another possibility is the theft of backup tapes that contain the list of unencrypted passwords. If users change their passwords every three months then the theft of some four month old backup tapes will be less of a problem. Another possibility concerns the resale of old computers, phones, and other devices that may contain passwords. A reasonably intelligent user won t sell their old hardware as soon as the replacement device arrives, they will want to use the new device for some time to ensure that it works correctly. If passwords expire during this trial period with the new device then passwords stored in the old device won t have any value. The down-side to this idea is that people probably sell their old gear fairly quickly and making passwords expire every two weeks would not be accepted well by the users. It seems to me that having bulk password changes (all passwords for one user or for one company) based on circumstances that lead to potential insecurity would do more good than changing passwords at a fixed schedule. How are Passwords typically Compromised? Dinei Flor ncio and Cormac Herley of Microsoft Research and Baris Coskun of Brooklyn Polytechnic University wrote a paper titled Do Strong Web Passwords Accomplish Anything? [3] which discusses this issue. The first thing that they note is that nowadays passwords are most commonly compromised by phishing and keylogging. In those cases passwords are typically used shortly after they are stolen and the strength of a password never does any good. That paper suggests that banks should use stronger user-names rather than stronger passwords to combat the threat of bulk brute-force attacks. Can a Password Last Forever? If a password is entered in a secure manner, authenticated by a secure server, and all network links are encrypted or physically protected then there should never be a need to change it. Of course nothing is perfectly secure, so for some things with minimal short-term value or which can be used without anyone noticing there is a benefit in changing the password. But in the case of Internet banking if a hostile party gets your login details then you will probably know about it in a few days when the bank calls you about the unusual transactions from foreign countries long before a 90 day password change schedule would have done any good. Maybe one of the issues determining whether a password should be changed regularly is whether an attacker could use long-term read-only access to gain some benefit. Being able to read all the email someone received for a year could be a significant benefit if that person was a public figure, and there s usually no way for an ISP customer to know that someone else is downloading all their mail via POP or IMAP. Should a Password be the only Authentication Method? It is generally agreed that an authenitcation method should ideally involve something you have plus something you know. That means a password and a physical device such as a smart card, token with a changing sequential password, or a key such as a Yubikey [4]. If the physical device can t be cloned (through some combination of technical difficulty and physical access control) then it significantly improves security. When a physical device is used the purpose of the password is merely to stop someone who steals the physical device from being immediately exploit everything the password only has to be strong enough to keep the accounts secure until a new token can be issued. The combination of something you have and something you know is very strong. Having a physical token stored on the desk next to the PC that is used for logging in provides a significant benefit, then an attacker needs to break in to the house and can t sniff the password by compromising the PC remotely. Conclusion In all aspects of security you need to consider what threats you face. If an attacker is likely to launch an immediate noisy attack (such as transferring the maximum funds out of an Internet banking account) then changing the password regularly won t do any good. If a subtle long-term attack is expected then changing the password can do a lot of good but a physical token is the ideal if the account is valuable enough. But to put things in to perspective, it s technically possible to use a mobile phone camera at close range (or a SLR with a big lens at long range) to take a photo of keys that allow reproducing them. But this hasn t stopped people from carrying their house keys in an obvious manner that permits photography or leaving them on their desk at work. Also I ve never heard of anyone routinely changing the door locks in case a hostile party might have got a key although I m sure that such practices are common in highly secure locations. Few people even take their house keys off the key-ring when they have their car serviced! Related Posts Defense in Depth and Sudo when using sudo can increase security and when it can t.
Logging Shell Commands how to log what the sysadmin does and what benefits that provides you, it doesn t help if the sysadmin is hostile.
Logging in as Root should you login directly as root?

Biella Coleman: Baby Panda wants YOU to submit to DEBCONF



bad mood destroyer, originally uploaded by the biella.
People s the deadline is fast approaching. Don t worry though, the best ideas come under deadlines and your proposal need not be lengthy. So get those typing hands out and submit a proposal (a few sentences suffices) for a presentation!
The annual Debian conference is being held in NYC this year. First time
setting foot in the imperial empire. Are you doing something related to
free software that might be interesting to share, show off or talk about
at Debconf? I bet you are! We are looking for art installations,
performances, movie screenings, maybe you are wanting to host a
roundtable/panel discussion, coordinate a workshop (you dont have to
know the subject deeply to do this!). I know the deadline is soon, but don t let that scare you. The
submission requirement bar is very low, and we are flexible! here is the submission information: [1] ********************* Calling all potential contributors to DebConf10! One more week until
the final submission deadline! The DebConf team is excited to announce that we are now accepting
proposals for contributions to this year s Debian conference! We
invite you to submit proposals for how you would contribute until the
deadline: soon There are many ways you can contribute, you could present a technical
paper, host a panel discussion, put on a tutorial, do a performance,
an art installation, a debate, host a meeting (BoFS, or Birds of a
Feather Session), or other possibilities that you devise. This year we
are also accepting proposals for tracks=E2=80=94a thematic grouping around a
particular subject, and people to coordinate those tracks. If you are
looking for ideas of things that you could contribute, or have ideas
for things that you would like to see happen at DebConf, have a look
at the Contribution Brainstorm[0] page. DebConf talks will be broadcast live on the Internet where possible,
unless otherwise requested. Videos of the talks will be published on
the web along with the presentation slides and papers. =3D=3D Step 1. Submit Proposal =3D=3D Proposals should provide an overview of your proposed contribution,
similar to an abstract, and be no more than 600 words. Since
contributions can cover a range of issues, please describe what your
proposal covers, be it political (Free Software, law, advocacy,
access, etc.), technical (d-i, OpenPGP, etc.), social (Debian
structures and groups). It s okay to hit all three of these. Your
proposal should also include some information on why you re qualified
to present, or coordinate this topic. For example, you can include
links to posts you have made to Debian mailing lists or other material
that indicates your qualifications. Proposals and their abstracts will
be accepted until May 1st, 2010, 23h59 UTC. To submit your proposal, go to Penta[1] and register as an
attendee. Once you are registered, there is a Submit a proposal
link. You will see your proposal on the site. You can choose between
different presentation types. If you are unsure what type you should
use, please read our glossary of definitions[2]. If it s not there for
some reason, or you have questions, please contact us immediately at
mailto:talks@debconf.org. =3D=3D Step 2. Committee Review =3D=3D The review committee for this year will make its decision by May 1st,
2010. All correspondence will be done by email. =3D=3D Step 3. Paper Submission =3D=3D Like last year, we are asking for papers to be submitted along with
the presentations. A full paper is optional, however we strongly
encourage you to submit one if possible as we intend to provide
written information along with transcripts of the sessions for later
viewing. Having written papers in advance will allow us to get
translations done to help non-native-English speakers feel more
comfortable with the topics presented. Papers are due by July 1st,
2010, and should cover the topic in reasonable depth (3 pages A4 text,
plus pictures and diagrams). We will use LaTeX to typeset the
proceedings. Please submit your paper formatted in LaTeX. Should you
be unfamiliar with LaTeX earlier submission in plain text is also
fine. =3D=3D Step 4. Live Presentation =3D=3D Longer presentations may have a break in the middle and should include
workshop items that directly involve the participants. If using slides
or any other presentation, please consider that your audience will
consist of people who use free software, and your choice of
application to prepare and display the slides should reflect this if
at all possible. =3D=3D Fine Print Publication Rights =3D=3D DebConf requires non-exclusive publication rights to papers,
presentations, and any additional handouts or audio/visual materials
used in conjunction with the presentation. The authors have the
freedom to pick a DFSG-free license for the papers themselves and
retain all copyrights. The presentations will be recorded, and may be
broadcast over the Internet. Any copies of the presentation will be
made available =3D=3D Failure to Submit =3D=3D In the event that a deadline is missed we reserve the right to revoke
any offer to present. =3D=3D About DebConf =3D=3D The annual DebConf conference is a technical and social forum for
Debian developers, sponsors, affiliates, and friends. It allows
various groups within Debian a chance to come together, network, and
share their work. Until we see you in the big city,
DebConf Talks Team [1] introduction written by Micah Anderson

12 June 2010

Matt Zimmerman: How to decide what to read (and what not to read)?

Like you, dear Internet readers, I have no shortage of reading material. I have ready access to more engaging, high quality, informative and relevant information than I can possibly digest. Every day, I have to choose what to read, and what to pass by. This seems like an important thing to do well, and I wonder if I do a good enough job of it. This is just one example of a larger breadth/depth problem, but I m finding the general problem difficult to stomach, so I m focusing on reading for the moment. These are my primary sources of reading material on a day-to-day basis: How do you decide what to read, and what not to read? How does your experience differ between your primary information sources? How have you tried to improve?

10 June 2010

Dirk Eddelbuettel: RcppArmadillo 0.2.2

Following the Rcpp 0.8.1 release we made yesterday, we released RcppArmadillo release 0.2.2 this morning. RcppArmadillo uses Rcpp (and some 'glue' code) to provide a transparent interface from R to Conrad Sanderson's impressive Armadillo library for linear algebra. This release works well with the most recent inline release 0.3.5. One can now employ inlined R code as we generalized how/which headers are included and how library / linking information is added thanks a plugin mechanism. This is the first RcppArmadillo version to provide such a plugin, We also updated the included Armadillo headers to its most recent release 0.9.10, added some more operators and provide a utility function RcppArmadillo:::CxxFlags() to provide include directory information on the fly. An example of the direct inline approach for the fastLm function:
library(inline)
library(RcppArmadillo)
src <- '
	Rcpp::NumericVector yr(ys);			// creates Rcpp vector from SEXP
	Rcpp::NumericMatrix Xr(Xs);			// creates Rcpp matrix from SEXP
	int n = Xr.nrow(), k = Xr.ncol();

	arma::mat X(Xr.begin(), n, k, false);   	// reuses memory and avoids extra copy
	arma::colvec y(yr.begin(), yr.size(), false);

	arma::colvec coef = arma::solve(X, y);      	// fit model y ~ X
	arma::colvec res = y - X*coef;			// residuals

	double s2 = std::inner_product(res.begin(), res.end(), res.begin(), double())/(n - k);
							// std.errors of coefficients
	arma::colvec std_err = arma::sqrt(s2 * arma::diagvec( arma::inv(arma::trans(X)*X) ));

	return Rcpp::List::create(Rcpp::Named("coefficients") = coef,
				  Rcpp::Named("stderr")       = std_err,
				  Rcpp::Named("df")           = n - k
				  );
'
fun <- cxxfunction(signature(ys="numeric", Xs="numeric"), src, plugin="RcppArmadillo")
This creates a compiled function fun which, by using Armadillo, regresses a vector ys on a matrix Xs (just how the fastLmPure() function in the package does) --- yet is constructed on the fly using cxxfunction from inline. More information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

9 June 2010

David Welton: A Minor Erlang Rant

In an earlier post, I compared node.js to Erlang: http://journal.dedasys.com/2010/04/29/erlang-vs-node-js - which admittedly has a whiff of apples and oranges about it. Still, though, I think there's something to it. Node.js is creating lots of buzz for itself these days. Some of that will turn into actual production use, and at that point actual companies will have a vested interest in seeing the system improved. Currently, it is not as 'good' as Erlang. Erlang's VM has a built in scheduler, so you simply don't have to worry about manually creating bite-sized chunks of processing that feed into one another, it all "just works". For instance, my current Erlang project involves reading from twitter's stream, and distributing that to IRC and the web. It's pretty simple, works well, and is quite robust. I haven't had the best of days though, and one little annoyance of the many I dealt with today really caught my eye. I need to do an HTTP POST in Erlang, and:
  1. The documentation does not have examples.
  2. Here's an example of how to do a POST:
    http:request( post, "http://scream-it.com/win/tests/posttest.cfm", [], "application/x-www-form-urlencoded", "x=2&y=3&msg=Hello%20World" , [], [] ).
  3. Aside from being ugly and not very leggible, you'll note that he's passing the parameters as a string, and also has to manually include the "x-www-form-urlencoded" header.
  4. To create that string, you'd want to url encode it. Actually, ideally, you'd just pass in a list of key/value pairs and let the library module handle it.
  5. However, there's nothing in the http module that does that.
  6. If you look through various bits of Erlang code on the web, you'll note many, many versions of url encoding and decoding, because the http module makes no mention of how one might go about doing so.
  7. It turns out, that the edoc_lib module does have a uri encode function!
  8. That isn't documented in its own man page.
  9. And certainly isn't linked to in the http page.
So, in 2010, doing an HTTP POST in Erlang is still a pain in the neck. I'd be happy to put my money where my mouth is and submit a patch (at least one for the documentation), but you have to wonder why no one has fixed the problem so far - maybe they're not very accepting of patches?

Russell Coker: Mailing List Meta-Discussions

It seems that most mailing lists occasionally have meta-discussions about what is on-topic, the few that don t are the ones that have very strong moderation authoritarian moderators who jump on the first infraction and clearly specify the rules. I don t recall the list of acceptable topics for any mailing list including also discussions about what is on-topic . As this is the Internet I m sure that someone will immediately point out an example of such a list, but apart from the counter-example that someone will provide it seems obvious that for the majority of mailing lists a meta-discussion is not strictly on topic. Regardless of a meta-discussion not being on-topic I don t think there s anything wrong with such a discussion on occasion. But if a meta-discussion is to be based on the volume of off-topic messages it would be nice if the people who advocate such a position could try and encourage the discussion in a way that reduced the number of messages. Replying to lots of messages is not a good strategy if your position is that there are too many messages. If a meta-discussion is going to be about moving off-topic discussions to other forums that are more appropriate then it would be nice to have the meta-discussion move to another forum if possible. My previous post which advocates creating a separate mailing list for chatty messages was an attempt to move a discussion to a different forum [1]. Anyone who believes that such discussions don t belong on a list such as debian-private is free to commit their thoughts to some place that they consider more appropriate and provide the URL to any interested parties. I think that it s worth noting that the only comment on my previous post is one that describes how to filter mail to split the traffic from the debian-private list into different mailboxes. I had hoped that other people would write blog posts advocating their positions which would allow us to consider the merits of various ideas without the he-said-she-said element of mailing list discussions. Most mailing lists have a policy against profanity and some go further and ban personal abuse. Therefore it seems hypocritical to advocate a strict interpretation of the rules in regard to what is on-topic while also breaking the rules regarding profanity or personal abuse. I don t think it s asking a lot to suggest that the small minority of messages that someone writes on the topic of a list meta-discussion should obey the set of rules that they advocate I m not suggesting that someone should obey all the rules all the time, just when they are trying to enforce them. Also you can argue that a list policy against profanity doesn t preclude sending profane messages off-list, but if the off-list messages are for the purpose of promoting the list rules it still seems hypocritical to use profanity. It is a fair point that off-topic discussions and jokes can distract people from important issues and derail important discussions. It would be good if people who take such positions would implement them in terms of meta-discussions. If the purpose of a meta-discussion is to avoid distraction from important issues then it seems like a really good idea to try and avoid distraction in the meta-discussion thread. I wonder whether a meta-discussion can provide anything other than a source of lulz for all the people who don t care about the issue in question. The meta-discussions in the Debian project seem to always result in nothing changing, not even when the majority of people who comment agree that the current situation is not ideal. When an almost identical meta-discussion happens regularly it seems particularly pointless to start such a discussion for the purpose of reducing off-topic content. Revisiting an old discussion can do some good when circumstances change or when someone has some new insight. I know that it s difficult to avoid being sucked into such discussions, when I was diagnosed with AS [2] I decided to try and minimise my involvement in such discussions but I haven t been as successful at doing so as I had hoped.

David Welton: Some difficult decisions?

I am, by nature, a tinkerer. I love to build things to see how they will work or because I have an idea that I think is fun. I am not so much of a natural business guy. I go build something "because it's there". Or rather, it's there in my mind, and I want to make it. I earn money, however, via consulting work. There are problems with this line of business though, namely that it doesn't scale very well unless, besides being good at a skill, you've also got the skills to build up a consulting business, which is very much about adding and managing people, something I'm not all that passionate about and would rather avoid. So what happens is that you go through slumps with less than full capacity, and then things pick up again, and before you know it, you hit your upper limit and can't take on any new work, putting a ceiling over your potential earnings. Sure, you can raise prices a bit, but that'll only get you so far. For a while now, I've been considering this problem, and I think some sort of product would be an ideal way out. Stories like Balsamiq's are inspirational, and along the lines of what I think might be nice. Don't get me wrong, I like my clients and like to help people solve problems a lot, but it's a stressful job in that you're never very far from having no income. I'm not much of an entrepreneur, though, in the sense of someone with a nose for making money. I tend to think about building something cool first, and not really worrying about making money with it. Predictably, some of the sites I have created are popular, but don't make any money. Sure, they have some ads, but that doesn't really make much unless you get really huge page views or have some sort of content that attracts users who will click on very lucrative advertisements. I've started to think that perhaps I should constrain myself in new initiatives to creating things that have a direct source of revenue: you want to use it, you pay. In my daydream world, I'd probably just hack on open source all day long, but there's no money in that for someone like me who is not the author of some huge, famous piece of software, nor a particularly great hacker in any case (I've met some, like Andrew Tridgell, and it's a really humbling experience). So that brings up the question of what to do with the aforementioned sites. One option would be to sell them off via a site like flippa.com, another would be simply to just let them sit there and stew, but they do take up some resources on my server, and require some maintainance now and then, and they're a bit of a distraction too (I start wondering about things to add or how to fiddle with them). Maybe there's a way to make money from some of them, but I'm too blind to see it. The ones I'm thinking about, are, in no particular order:

7 June 2010

Mart&iacute;n Ferrari: Python WTF

Lately, I have recurring WTF moments when working with Python. I'm still more or less a novice in Python, as I started to use it in my day job last year, so sometimes it could be blamed to ignorance. Today I was writing some code that takes any file descriptor as argument, and thought it could be nice to accept also objects that represent one. But it seems there is no common ancestor to file objects and socket objects, for example. In Perl, I'd just check for IO::Handle. Then, I wrote code to check if a passed argument was a socket, or a file descriptor to one, with some obvious check: isinstance(s, socket.socket). WRONG, when you create sockets with socketpair, they are of an undocumented type which is not a descendant of socket, nor of SocketType! (There is a patch for this since 2007) In general, I have found a lack of any reasonable object hierarchy, and many inconsistencies in the low-level interfaces, which is a pity, because Python is in general quite tidy. Comparing with Perl is unavoidable, and I would have expected that Python had learnt more from it. Another thing that amazes me is the lack of anything like CPAN, which is one of the reasons Perl was so successful. The cheese factory doesn't come close, as you need to publish your files somewhere else, there is no automated testing, no automatically generated documentation, or anything that requires access to the source code. In Perl-land, many people loathe the venerable MakeMaker, and some of the alternatives are ugly as hell. The Python equivalent (or so) to Makemaker, distutils, looks to me like a bad joke: advanced customisation is black magic (if possible at all), and doesn't provide absolutely basic things like running your unit tests! After fighting with it for some days, I decided to go back writing my own Makefile and calling distutils just to build and install. Tags: Planet Debian, Planet Lugfi, Programaci n

Riku Voipio: WebGL on N900

With the recent "PR1.2" firmware upgrade of Nokia N900, a new feature was enabled in the browser - WebGL. WebGL is cool and scary. The cool part is that it is a chance to bring lots of games to to Linux users. The screenshot is from Match3D, a 3D tic-tac-toe game.

The best way to get started is with the Learning webgl tutorial. Passing through the lessons, some are clearly featuring buggy graphics. I don't know if the website, N900 browser or the OpenGL ES driver is being buggy. Which brings the scary part of WebGL - not only does one need to deal with buggy browsers, one has to deal with buggy 3D hardware/drivers! WebGL is still very much work in progress, and on N900 the webGL is rather an "easter egg" than a proper feature. For example the 3D graphics from the GPU does an extra roundtrip via CPU before appearing on the screen.

Other WebGL resources are planet WebGL for following blogs about WebGL, the Khronos Demo repository and more demos can be found at CubicVR site. The last CubicVR demos feature another scary new browser feature (not yet supported on N900) - audio processing in JavaScript.

5 June 2010

Jos? Parrella: Ditching PCI cards in Asterisk for fun and profit

In the past, I ve used PCI cards providing telephony TDM for my Asterisk-based projects. I ve only worked with Digium s FXO/FXS and T1/E1 cards, both in PCI and PCI-x formats, until a client asked for a gateway solution with FXO, FXS and E1, where using the PCI cards in a 1U IPBX was simply out of the question. Using Ethernet gateways has always been a problem in those cases, since entry-to-medium level solutions lack the processing power for massive call handling, and signalling is poor, making the architecture unfit for reporting purposes and specialized apps such as call centers. But hey, that s my experience with gateways. Therefore, I looked into Xorcom s Astribank. They pack FXO, FXS and PRI/BRI ports inside a convenient 19 1U case, which can connect to an Asterisk-based IPBX server via USB and gets recognized as an integral DAHDI device in modern setups. So I ordered a composite 8xFXO, 8xFXS and 1xE1 PRI from Astribank, labelled XR0072. Getting the equipment from Arizona to Florida was easy, and since I worked with DHL, getting it to Guayaquil in Ecuador was a breeze (next-day delivery) but passing customs is, as always in Latin America, a time-consuming operation. After paying 12.5% taxes and then some for DHL operations, and 15 days waiting, I got the equipment in Quito. To my surprise, the Astribank was extremely light and small, slightly near 3 Kg in 1U form factor and under 1/4 depth in the rack. While it had a rack-mounting kit, which I used, it could also rest securely over my IPBX without rack screws. My Astribank uses a single USB cable to connect to the master PBX it could also be connected to a secondary PBX and provide a fault tolerant setup. It also depends on a DC supply, which they included. Since I was using a modern Debian, with a modern Asterisk with already-compiled modern DAHDI (formerly called Zapata/Zaptel) drivers, I just connected the equipment via USB to the IPBX, issued an invoke-rc.d dahdi restart and we were good to go. Note: you do need the fxload package in Debian for firmware loading to the Astribank. And for good to go I mean you can check the registered spans with dahdi_registration and, as soon as you connect the E1 cable that comes from the CPE to the CPE port (which in my XR0072 is the first on the lower row for PRI) you get an OK status with dahdi_tool. Also note that I use an E1/CPE port, which is the default for the Astribank, otherwise I d have to set the XPP_PRI_SETUP variable as indicated on the User Manual. DAHDI insists on configuring a crc4 parameter for the detected spans, which I needed to remove and restart DAHDI so the equipment could sync with the CPE (that is, the box that the telco provided, with the balun and the crossover RJ45 cable) and avoid some nasty noise issues on the line, that also occur with Digium s Wildcard for PRI.
usb:008/005          xpp_usb+     e4e4:1162 Astribank-modular FPGA-firmware
 LABEL=[usb:...]       CONNECTOR=@usb-0000:00:1d.7-1
	XBUS-00/XPD-00: E1       (31)  Span 1  DAHDI-SYNC
	XBUS-00/XPD-10: FXS      (8)   Span 2
	XBUS-00/XPD-20: FXO      (8)   Span 3
Using DAHDI s dahdi_genconf command writes an /etc/asterisk/dahdi-channels.conf file with the group and channel configuration as detected by DAHDI. It should be included by the chan_dahdi.conf file, which, in my case, is not necessary since I only wanted to enable one DAHDI group (DAHDI/g0) with my 31 E1 channels (1-15,17-31)
group=0
context=from-pstn
channel => 1-15,17-31
That means that when I want to use the FXO/FXS ports, it d probably be easier to include the dahdi-channels.conf (FXS ports, for example, belong to a different context, from-internal) so I don t have to type more than my laziness permits. I also found that, in my setup, some Asterisk parameters were superfluous, such as the switchtype (either national or euroisdn work, and the former is the default) or the echo-cancelling stuff, so I assume Astribank s doing a great job in negotiating that with the CPE. Do take note, however, that in my case the CPE needed to be cold-restarted so I could get an Up status in Asterisk (pri show spans) and that my only non-default, channel related configuration in chan_dahdi.conf is signalling=pri_cpe. So far, so good. I m really happy with the Astribank and I can fully understand how useful it is in terms of price (quite competitive in comparison with traditional cards, specially when bought from resellers -like me-), size/weight and functionality/ease of setup and administration. Kudos!

4 June 2010

Adrian von Bidder: Filesystems Quo Vadis: Clients

Probably it's just a question of me not paying enough attention to the news... So: pointers welcome. There are quie a few shiny new filesystems for local storage, like btrfs or HAMMER (and nilfs, Tux3, ext4, Reiser4...) It seems the distributed storage side is covered as well, with ceph being merged recently. There are other systems (like, for example, Lustre), but they haven't appeared much in the news channels I tend to read. What I'd be curious is if any of these support hierarchical storage architectures like pushing out rarely used data to tape libraries. (But this is just idle speculation, I don't need this anywhere.) But what I would really need is a replacement for NFS (v3): a classical client-server filesystem. I'm not sure NFSv4 is the right solution (where I'd use it, I currently can't because we rely too much on POSIX ACLs there, making the transition to NFSv4 quite a chore.) I think POHMELFS might be a solution in the long term, or CRFS, but I'm not sure how much progress there is on these; apt-cache search is silent, not a good sign. There's Samba 4 I guess I'll have to look at it, since it's supposed to be much cleaner and nicer to use than its predecessors and might be a good solution even if no Microsoft systems are involved. Is this what small environments (a fileserver and 100 clients or so) will want to use? Comments are very welcome.

John Goerzen: Hardware/Machine Reporting Tools

We have a tool at work, Kaseya, for our Windows machines. It handles updates, but also can report back information gathered from them: OS version, patches installed, time since last checkin, last user logged in, user most frequently logged in, plus details about hardware: RAM, disk size, serial number, etc. I m looking for a tool that can do this on Linux as well. Of course, we could roll our own; all the above is readily cleaned from standard commands plus examining /proc and /sys. But before we roll our own, we want to be sure there s nothing else out there worth using. Down the road, we might also like it to grow in another direction: centralized configuration, storing links between machines and people in a database, and generating DNS, DHCP, etc. configurations out of it. Whether that would be the same tool as this is another question. There are, of course, expensive and overkilling well-known commercial packages to do this. We re after something simple that gets the job done. Any ideas?

Next.

Previous.