Oddly, I have had four different people ask me about the best way to begin a Django project or setup their system to make it easy to develop with Django. Since I've now given my response via email so many times, I figure I might as well broadcast it and hopefully this will help others as well My assumptions:

• Using Linux (or a Unix machine)
• Single developer with root access (quite possibly a laptop)
• Knowledge of DVCSes (e.g. git / hg / bzr )

What we will do is create a private binary directory and another Python modules. Then take a checkout of the two most important Django projects, and make them work. First: create your user

$ sudo adduser newbie
$ ssh localhost -l newbie
$ whoami
newbie

Then we want to create some local bin and lib directories. First the binary directory $ mkdir ~/bin # On Ubuntu/Debian, this will automatically be in your PATH the next time you login And now a local directory for our various libraries

$ mkdir -p ~/lib/python
$ echo "PYTHONPATH=~/lib/python:" >> ~/.bashrc
$ echo "export PYTHONPATH" >> ~/.bashrc

This will add that directory to our Python path. If you happen to also use another language you can put things into ~/lib/ruby, ~/lib/perl as appropriate

$ mkdir ~/Projects
$ cd ~/Projects

Here is where we will store copies of upstream software. What is the reason for using the repositories rather than packages? This allows us to checkout specific versions to match what our clients might be using. Or test things against newer versions of the upstream project. First, let's setup Django. I tend to use git by default, especially if the upstream is using Subversion. If they are using Mercurial or Bazaar, I use those directly.

$ git svn clone -s http://code.djangoproject.com/svn/django/
Initialized empty Git repository in /home/newbie/Projects/django/.git/
Using higher level of URL: http://code.djangoproject.com/svn/django => http://code.djangoproject.com/svn
r1 = 5cda37203ffa6ea83da2958a95c377984482877f (refs/remotes/trunk)
	A	django-docs/images/flatfiles_admin.png
	A	django-docs/images/users_changelist.png
	A	django-docs/model-api.txt
	A	django-docs/build.py
	A	django-docs/db-api.txt
	A	django-docs/writing-apps-guide-outline.txt
	A	django-docs/templates.txt
r2 = b8249ac45e2154933b9649fd8181d5769e31c9fc (refs/remotes/trunk)
	A	django/utils/feedgenerator.py
	A	django/utils/datastructures.py
[...]
r13399 = f3902c67a3b8788de2145899e435a394c512b455 (refs/remotes/releases)
	M	tests/regressiontests/m2m_through_regress/tests.py
r13400 = cd72207306a5a4eecdf07f65c109f37c8317ed81 (refs/remotes/trunk)
$

Now we have Django, push it into our python path

$ cd ~/lib/python
$ ln -s ~/Projects/django/django/
$ python
Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
>>> django.VERSION
(1, 2, 1, 'final', 0)

Now, let's do the same for South. South is a database migration helper for Django. These are the two things you want to have in any Django at a minimum. There are also plenty of other amazing things like haystack, piston, satchmo, etc. You can follow the same recipie for them too.

$ cd ~/Projects
$ hg clone http://bitbucket.org/andrewgodwin/south/
destination directory: south
requesting all changes
adding changesets
adding manifests
adding file changes
added 802 changesets with 1340 changes to 183 files (+1 heads)
updating to branch default
143 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd ~/lib/python
$ ln -s ~/Projects/south/south
$ python 
Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import south
>>> south.__version__
'0.7.1'

Finally, we want to be able to create Django projects. There is a great helper called django-admin that will do this. Let's put it into our path

$ cd ~/bin
$ ln -s ~/Projects/django/django/bin/django-admin.py
$ hash -r

That last statment will cause your interpreter to re-read PATH and make anything new available for execution. Now, you should go ahead and create your projects. There are a variety of ways to do this. However what I like to do is keep Django applications separate from the Django project. That way, they can easily be re-used if required. So, for a project called 'foo', we might have

$ mkdir -p ~/Work/foo
$ cd ~/Work/foo
$ mkdir foo.example.com
$ mkdir templates
$ mkdir media
$ mkdir <individual apps>

For each individual app, put them into the Python library. And you then have your re-usability from within Django. For a application called 'bar', you would do:

$ cd ~/Work/foo
$ django-admin.py startapp bar
$ cd ~/lib/python
$ ln -s ~/Work/foo/bar

And now, in your Django foo.example.com setting.py's file you can put 'bar' as one the installed applications and things will Just Work Obviously there are many way to slice this particular mango, but I've found that this works pretty well for me. You can spruce it up by revision controlling each directory in your project (I do) and also take advantage of things like virtualenv and Fabric to make deploying just as easy as developing. But I'll leave those topics until a later date

I ve been watching the June budget and pondering its effects on me, our co-op, co-ops in general and the wider community.

IT Taxes

I just want to flag this one up as a special interest: the telephone tax is killed off before it starts. Yippee! But now we get to wait and see how the government will support private investment to get universal provision of fast broadband. I ve no problem with tax relief for the video games industry being scrapped - why should any type of software get special treatment?

VAT up 2.5% to 20% from 4 January 2011

This seems bad for everyone. It s a bit less bad for our co-op because we have some international suppliers, whose sales tax rates won t change, but we buy a lot from the UK too and it hurts all our workers.

Personal Taxes

The basic income tax allowance rise of 1,000 is welcome, as is restoring the pension-earnings links. More widely, the freezes in various things, increase in Capital Gains Tax and drop in cider tax all seem broadly good ideas. The housing benefit 400 maximum is a bit mixed - how bad it is may depend how lenders react when borrowers get into difficulty.

Business Taxes

At a time when a VAT increase is called unavoidable , it stinks a bit to cut corporation tax, extend the 10% capital gains tax allowance and raise the employers National Insurance threshold. Of course, our co-op pays none of those, so it also hurts us by giving our competitors an advantage.

Business Incentives

George Osborne said he wants to tackle regional economic differences, but the big change in this budget is bad for all existing businesses outside London, the South-East and East: National Insurance exemptions for new businesses. Once again, existing co-ops take it in the neck from another government obsessed with capitalism and employment instead of businesses and work. Meanwhile, increasing the Enterprise Finance Guarantee props up debt-laden business and a new Growth Capital Fund encourages capitalist businesses, while both appear useless to good co-ops at first glance. I don t mind being ignored, but could we please elect a government which doesn t actively hinder co-ops? Where s the fabled commitment to fairness ? Promises around the Green Investment Bank and Green Deal sound good, but are in the future.

Council Tax

Council Tax will be frozen for a year. I wonder if that applies to parish councils, because our village planned a cut after a one-year project-based increase last year. The Budget Document has pretty much no detail.

So, how was it for you? Have I missed some friendly changes?

Grah. I got my current phone (a Nokia N95) back in January 2008. It was the first phone I ever paid for, and I don't expect that to change - The feature set I need from a phone is extremely basic, but I wanted to have a GPS unit on me, without needing to plan for it beforehand. As I recently blogged, I just got over the 5000Km mark (5269 as of last Friday), with almost daily usage since I registered. A couple of days ago (although please note I don't log in often to the web page, I just upload my , I was greeted by this gem:

IMPORTANT NEWS Thank you Nokia Sports Tracker beta users! After two and a half years of operation, the Nokia Sports Tracker beta service will close as of June 30, 2010. Thanks to all of you exercise enthusiasts who used the service and contributed valuable feedback to its improvement. We want you to stay active. You can keep tracking workouts through a new, non-Nokia service provided by Sports Tracking Technologies available for free in the Ovi Store. The new service is open for registrations at sports-tracker.com, and you'll even be able import your workout history from Nokia Sports Tracker until August 31, 2010.

In my case, instead of Thank you they could have just said Fuck off. They are directing me to a new application available in Nokia's Ovi store. Of course, this application is linked to a new 100% Flash-powered website. That would be a big turnoff for me, yes, although there are many ways around it (after all, I don't need the website functionality every day). However, I find the list of supported phones does not include the N95 (any version). This means, my two year old phone (which I bought as soon as the model arrived in Mexico) is obsolete now and they want me to (spend more money and) switch to a new one. Am I going to do that? Hell no! The N95 phone is quite good. The only gripes I have is with the (slow, buggy and memory leak and phone crash prone) Symbian S60 operating system. I will probably keep using the SportsTracker application, although I won't be able to link my tracks as easily as I do today. So... Dear lazyweb, do you know of any similar program? The feature set I use in Sports Tracker is:

1. Have different activity profiles. I use Running and Cycling, and very ocassionaly Walking.
2. Of course, GPS tracking, getting average and top speeds, altitude profile, etc.
3. Aggregation into something website-friendly
4. The ability to do my uploads post-factum (i.e. do not rely on a data plan, allow me to upload my excercising when I get home)

There are probably more things I'd want. But I am a geek, and I basically enjoy tracking myself, and sharing my tracks with $world. Thanks, dear lazyweb. I am confident you won't let me down.

The world around me seems to whirl these days. One week ago, I graduated from MIT. People I've known during the last four years have been dispelling to various parts of the globe one by one, day by day. California, Canada, Indonesia, Seattle. Some will be back again. Some will not, or if so only to visit. pika is a continuous bustle of activity as the summer has commenced and it has filled with creative and adventurous MIT students who've suddenly found themselves having free time. A hammock being built on the roofdeck. Thrice-weekly icecream forays. Common areas overflowing with people playing musical instruments, chatting, and messing around on laptops. Summer's warmth has arrived, bringing with it farmer's markets, strawberry picking, and swimming expeditions. While it's wonderful to get to meet so many new people living in a college environment, I can't help but feel sadness thinking about everyone who's left. There are always more friends to be made as new people arrive, but old ones moving away leave bittersweet memories, and the new relationships are always a bit different as the age discrepancy between me and others changes. Or the me-the-ephemeral-collection-of-thoughts-which-when-regarding-other-people-sometimes-involve-the-mentor/mentee-distinctions-caused-by-one-party-being-older-or-more-knowledgeable-than-the-other-at-least-in-certain-areas changes. The end of a semester always feels like this, but this year even more so as the people I started university with start down new paths. For me, that was going to involve staying on at MIT to complete a one-year master's program, the "M.Eng." in electrical engineering and computer science. That plan, too, has changed. I've deferred the degree and accepted a full-time engineering position at Ksplice, an exciting early-stage Linux startup here in Cambridge. I'd been working at Ksplice part-time since January before joining full-time immediately following graduation. Ksplice is the realization of ideas I saw being born on the whiteboard at SIPB when I was a freshman, and it's fun to see that play out in a small, ever-changing, low-bullshit company. All in all, there are many more exciting things down the road, and, working at an MIT startup, I haven't even escaped the MIT/Cambridge reality-distortion bubble yet. Still, it's tempting to resist change and let myself romanticize the good old days, hoping to catch every person I've ever enjoyed spending time with and hold them down here forever. That's not the way life works, though. Change happens.

I just packaged and uploaded Jan Vornberger s window manager bluetile to Debian. This very nice piece of software brings the benefits of a tiling window manager to users who prefer to use the mouse and who don t want to learn a new programing language to configure their window manager. Bluetile uses the xmonad libraries and extends them with an easy to use and discoverable user interface.

Something I've wanted to do for a while now, is somehow combine my blog with all of my projects/git repositories, and now I've done it! Welcome to my new site :) So, I got bored a few nights ago, and started trying to find some kind of project management type system to host my git repositories in. After searching for a while I stumbled across Gitorious. I'd heard of Gitorious before, but I'd never really looked at it, nor did I know that it's codebase was infact completely open. So I thought I'd give it a try to see what it was like Now I have to admit, my first impressions of it weren't exactly great. Firstly, it uses Ruby, which I can't say I'm much of a fan of (but that's a different discussion :p), nor am I a fan of installing all of the 'packages' it needed, via gem. Deploying it is an absolute nightmare, it has 4 different daemons, all of which have hideous init scripts that don't really work properly, and low and behold you try and put the git repositories and the site itself under different users, figuring out what needs to run as what and what needs what permissions is very much a game of trial and error. In Gitorious' defence, there /are/ numerous guide lurking around on the internet explaining how to do these things. But once I'd got over these niggles, and had actually deployed it, it worked quite nicely, and really did exactly what I wanted, so I decided to stick with it. It then occurred to me that it probably wouldn't be too hard to stick a blog onto it, after all, it's written using Rails, which is supposed to be easy, right? So with no knowledge of Ruby, let alone Rails, I got stuck in. Fortunately, it turns out Rails is in fact pretty easy to use, (at least once you've got used to it implicitly doing everything behind the scenes..), I'm even starting to think I prefer it to Django... So there we have it, a brand new site, and one that looks considerably better than my old one.. even if it is just Gitorious in disguise _ The only thing I'm disappointed Gitorious doesn't have, is some kind of bug tracker, but then, I'm not sure I'd use one given most of my projects just go through Debian.

Good morning everyone, after quite some time of organization, we got the MiniDebConf running in Berlin. Yesterday at 9.30, Zack had the pleasure of opening the event which seems to be reflected in the press already. After that, we had nice talks, one every hour from 10 to 6 with mine being the last one of the day at 5 p.m. Yes, I did it :) Slides, links, and all that stuff will follow in a later post, or more official announcements. Right now I m preparing myself to get back to the fairground. The second day starts with another bunch of talks at 10 and I m excited to meet different people again and hear more interesting talks. The BSP went well. :) Actually, we didn t really fix many bugs, I guess. A few maintainers stayed after the event as well as a couple of users completely new to packaging. Thus, it was more like a small social event bit it was really fun. And of course the HackLab is still open today, so we have every chance to fix more bugs This just as a small input from where we stand. More is to come some time later, probably after the weekend, we ll see

In response to my previous post about logging in directly as root [1] it was suggested that using sudo is the only way to log the commands that are entered as root. One reason for doing this is if you don t trust the people who are granted root access and you want to log all commands to a remote server that is run by different people. I wonder whether it is really possible to run systems with untrusted sysadmins, if someone can apply patches etc then they can surely install a trojan and then wait a while before activating it to make things more difficult for anyone who is analysing the logs. One of the many issues is that even the restricted version of vim permits the :r and :w commands, so one could start vim from sudo with an innocuous file as the target of the edit operation and then read and write some critical file such as /etc/shadow. I expect that someone has written an editor which has a restricted mode that doesn t allow reading/writing files other than the one specified on the command-line, and if not it surely wouldn t be difficult to patch vim (or your favorite editor) to have such a mode of operation. But there are always other programs that can access files other than the ones specified on their command-line. It seems that using the auditctl interface to log access to certain critical files (EG read access to /etc/shadow and write access to everything under /etc, /bin, /sbin, and /usr) would be a necessary part of an effective auditing strategy and that sudo would only comprise a small part of a useful configuration. There are other viable ways of logging everything that is done as root which offer benefits over sudo. Ways of Logging Shell Commands The Korn shell supports doing all the logging you might desire as part of a shell function [2]. Bash can have a similar shell function to do the logging, but when a command is entered the previous command is logged [3], this means that any single bash command that unsets this will never be logged. It might be possible to solve this if you know more about Bash than I do. I wonder if the Korn shell function has the same issue. This is still probably useful for some situations when you want to track what honorable sysadmins do, but of little benefit for tracking hostile sysadmins (tracking hostile sysadmins is actually possible). You can put code in a file such as /etc/bash.bash_logout to log the commands elsewhere, but even trivial things such as kill -9 $$ can defeat that so it s only useful when the sysadmin is trusted. The Sudoshell project exists to log all data that is entered in a shell [3]. One deficiency of this for the people who don t trust the root user is that it logs the data to files on disk, but it shouldn t be difficult to rewrite sudoscriptd to write directly to another machine over the network. Also one benefit of this for auditing is that it captures all the output of the commands as well (which can be a little inconvenient to decipher when curses programs are run. The web site also describes some of the problems with trying to use sudo directly for everything (such as pipelines). If you compile Bash version 4.1 with the SYSLOG_HISTORY macro enabled (which can be done by editing the file config-top.h) then it will log all commands to syslog. RootShell.be has a short post about this which mentions the security issues, some commands take passwords as parameters and these passwords could be exposed to the network [5]. Of course the best option is to just avoid such commands. Thanks to Chris Samuel for pointing out the Bash logging feature. Conclusion If you use sudo for auditing root access then you lose some shell functionality. Sudo also only logs the commands that are executed you don t get logging of output. It seems that depending on the exact needs either a modified version of Sudoshell or the logging that can be compiled in to Bash would be the way to go depending on the exact requirements. The main benefit of using sudo for logging would be that some distributions of Linux are configured that way by default but it seems unlikely that someone would go to the effort of running a separate logging server that the regular sysadmin team can t touch and then configure their servers in a default manner.

• [1] http://etbe.coker.com.au/2010/05/29/logging-in-as-root/
• [2] http://blogs.sun.com/chrisg/entry/logging_commands_in_korn_shell
• [3] http://blog.rootshell.be/2009/02/28/bash-history-to-syslog/
• [4] http://www.egbok.com/sudoscript/sudoshell.1.html
• [5] http://blog.rootshell.be/2009/12/28/bash-syslog-history-could-lead-to-data-leakage/

So, I spent an hour or two this afternoon following the iPhone 4 liveblog. It all looked fairly compelling (the screen!), right up until Steve's 'and one more thing': video calling.



HELLO I'M IN 2007, CAN YOU HEAR ME
(Photo of the Nokia N800, which shipped in January 2007, from rnair.)

The moral of the story? If you want to be four years ahead of the WWDC closing bombshell, email sales@collabora.co.uk. :)


PS: The 2010 'fuck it, we're going to fivefour blades' version; we also had a six-way video call going earlier today.

For work recently I've been doing some Django-related tasks that involve talking to an external API with POSTed forms. Django forms objects are declared by creating a class that inherits from django.forms.Form, with the fields of the form declared by declaring attributes of that class. Which works well and is clean and easy to remember unless the API you're working with requires a field with the same name as a Python keyword, such as return. You can't declare a field like this as an attribute; it will trigger a syntax error. I spent some time scratching my head over this, and came up with this as a workaround after source-diving to find out how Form objects actually work:

from django import forms
class ExampleForm(forms.Form):
    def __init__(self, data=None, files=None, auto_id='id_%s', prefix=None,
            initial=None, errorclass=ErrorList, label_suffix=':', empty_permitted=False, return_url=None):
        forms.Form.__init__(self, data, files, auto_id, prefix, initial,
            errorclass, label_suffix, empty_permitted)
        if return_url is not None:
            self.fields['return'] = forms.CharField(widget=forms.HiddenInput, initial=return_url)

It turns out that the attribute declaration is just syntactic sugar for creating a dictionary of key/value pairs, which is then stored in the fields attribute. So we can monkeypatch in extra values after the translation. Which is somewhat more awkward and ugly, but works in a pinch. Note that I haven't extensively tested what interactions this may cause with other forms code, so use with some caution.

It seems I ve missed quite a few weeks in my countdown to our Mini DebConf. Fortunately, it s not too late for you to join even if, right now, you don t know why you should! So what s the status? Well, we ve had out problems in the beginning to start the whole thing but we managed. There are a few minor issues left, I admit. If you happen to have a bit time on 10th or 11th to hold the camera for an hour or so, contact us! Or if there is *anything* you would wanna do to help us out a bit, don t be shy. You don t need to be a known Debian contributor, you don t need to know us. Arguably, you should know Debian but then, if you re able to spell the name correctly, you re probably in. :-) Apart from that, we really think it s gonna be awesome. Not only will there be a few Debian Developers from outside Germany, like our project leader zack who we have the honor of listening to for he has the first talk on Thursday. There will also be quite a few German faces that some of us haven t met yet have you met Sebastian, the chief of our beloved debianforum.de, yet? We are going to have a great time and you re a fool not to join. Thus, a last time for all of you who still don t know what I m talking about: Next week there ll be the famous LinuxTag in Berlin from Wednesday to Saturday. Right in the middle, on Thursday and Friday, we, the Debian community, will have a Mini DebConf right there. There will be talks about different Debian related things. And many active developers will be there happy to talk to you about whatever you always wanted to know or say about Debian. And then, not to forget, we ll have a hacklab where you can easily plug in your notebook and start hacking with us. Have an idea to improve Debian? Always wanted to know how to fix a bug? You want to make Debian Squeeze the best Debian release ever? Join us it seems we re fighting on the same side. See you there, guys! PS: The latest and hottest news about it.

After xen-tools had been removed from Debian Unstable by the upstream developer last December due to no more upstream development, I took over upstream development as well as Debian package maintainership in January. Since then a lot happened:

• Development moved from Mercurial to a set of git repositories at gitorious.org.
• Dmitry Nedospasov joined my effort to revive xen-tools.
• Via gitorious clone tracking we noticed that there were useful patches in clones of xen-tools and incorporated most of them back into our master branch.
• We created a second mailing list xen-tools-dev which is mainly thought for communication between the xen-tools developers.
• The xen-tools.org website and the xen-tools mailing lists moved from Steve s to my server. The website has been slightly overhauled and the mailing lists now use Mailman.
• Support for all recent Debian and Ubuntu releases including Squeeze and Lucid.
• Full support for pygrub and DomU distribution kernels installed in the DomU. Now you easily can install DomUs whose udev doesn t fit to your Dom0 kernel.
• Full support for cdebootstrap in addition to debootstrap.
• Tons of bugs fixed
• I m on the quest to find the holy grail of bug tracking systems. ;-) More on this in another blog posting.

But despite all the things which were to fix after Steve discontinued the xen-tools developement, Steve Kemp did a great job to bring xen-tools so far initially. I m really happy that this software exists and that s the reason why I won t let it die so fast. :-) xen-tools 4.2 beta 1 released Last weekend I released xen-tools 4.2 beta 1 which I also uploaded to Debian Unstable. And thanks to the active ftp-master team, xen-tools went through the NEW queue in less than 12 hours and is now available again in Debian Unstable. 4.2 beta 1 still has some known bugs, the worst ones are in xen-delete-image. But no fear, it doesn t delete more than wanted, just not enough or things which don t exist anyway. :-) There are also some smaller issues to fix for a final 4.2 release. We may also create an PPA for Ubuntu Lucid in the future to work around the lack of xen-tools in Ubuntu Lucid. What about Fedora/RedHat/CentOS support? Since rinse is out of date with regards to current Fedora releases, rpmstrap seems to be out of date for even longer (it has been removed from Debian before the Lenny release) and febootstrap neither seems to work out of the box nor seems to support bootstrapping to mount points (because it wants to delete them first and fails), bootstrapping Fedora ist currently not supported. The copy installation method should work though. rinse still is able to bootstrap CentOS 4 and 5, but the installation seems incomplete. We ll try to work around these issues for the final 4.2 release. We haven t looked at mach and mock so far, but according to Lucas, both are out of date, too. Sure, implementing Lucas howto in form of a script would be a possible way, but we think it would just create yet another not so well supported clone of rinse, rpmstrap, mach, mock or febootstrap. Because of that, Dmitry works on support for the installation of Xen DomUs via kickstart. Fate of xen-shell? Xen-shell has also been removed from Debian Unstable last December because Steve stopped its developement, too. But in comparison to xen-tools itself, we didn t resurrect xen-shell since none of us currently has use for it. So if you are currently using xen-shell and intend to continue to do so, you should hurry up if you want to see it in Debian Squeeze. We can also help you, but we surely can t and won t do it alone nor will we take the majority of the necessary workload.

Now your SQL statements look like:

SELECT "auth_user"."id", [..] WHERE "auth_user"."id" = 1 -- /login

This allows you to quickly go from a misbehaving query in your production environment to the view that is executing it.

I was asked by email whether SE Linux could implement traditional Unix users and groups. The Strictly Literal Answer to that Question The core of the SE Linux access control is the domain-type model where every process has a domain and every object that a process can access (including other processes) has a type. Domains are not strongly differentiated from types. It would be possible to create a SE Linux policy that created a domain for every combination of UID and GID that is valid for a user shell given that such combinations are chosen by the sysadmin who could limit them to some degree. There are about 2^32 possible UIDs and about 2^32 possible GIDs, as every domain is listed in kernel memory we obviously can t have 2^64 domains, but we could have enough to map a typical system that s in use. Of course the possible combinations of supplemental groups probably makes this impossible for even relatively small systems, but we can use a simpler model that doesn t emulate supplemental groups. For files there are more possible combinations because anyone who is a member of a group can create a SETGID directory and let other users create files in it. But in a typical system the number of groups is not much greater than the number of users the maximum number of groups is typically the number of users plus about 60. So if we had 100 users then the number of combinations of UID and GID would be something like 100*(100+60)=16,000 it should be possible to have that many domains in a SE Linux policy (but not desirable). Then all that would be needed is rules specifying that each domain (which is based on a combination of UID and GID) can have certain types of access to certain other types based on them having either the same UID or the same GID. Such a policy would be large, it would waste a lot of kernel memory, it would need to be regenerated whenever a user is added, and it s generally something you don t want to use. No-one has considered implementing such a policy, I merely describe it to illustrate why certain configuration options are not desirable. The rest of this post is about realistic things that you can do with SE Linux policy and how it will be implemented in Debian/Squeeze. My previous post titled Is SE Linux Unixish? addresses this issue at a more conceptual level and also considers MAC vs DAC [1]. The History of mapping Unix Accounts to SE Linux Access Control In the early releases of SE Linux (long before it was included in Fedora) every user who could login to a system needed to have their user-name compiled into the policy. The policy specified which roles the user could access, the roles specified which domains could be accessed, and therefore what the user could do. The identity of files on disk was used for two purposes, one was logging (you could see who created a file) and the other was a permission check for the SE Linux patched version of Vixie cron which would not execute any command on behalf of a user unless the identity on the crontab file in the cron spool matched the identity used to run it this is an analogy of the checks that Vixie cron makes on the Unix UID of the crontab file (some other cron daemons do fewer checks). Having to recompile policy source every time you added a user was annoying. So a later development was to allow arbitrary mappings between Unix account names and SE Linux Identities (which included a default identity) and another later development was to have a utility program semanage to map particular Unix account and group names to SE Linux identities. This was all done years ago. Fedora Core 5 which was released in 2006 had the modular policy which included these features (apart from mapping Unix groups to SE Linux identities which was more recent). Fedora Core 5 also introduced MCS which was comprised of a set of categories that a security context may have. The sysadmin would configure the set of categories that each account would have. A recent development has been a concept named UBAC (User Based Access Control) which basically means that a process running directly on behalf of a regular user (IE with a SE Linux identity that s not system_u) can only access files that have an identity of system_u or which have the same identity as the process. This means that you can only access your own files or system files not files of other users which may have inappropriate Unix permissions. So for example if a user with a SE Linux identity of john gives their home directory the Unix permission mode of 0777 then a user with a SE Linux identity of jane can t access their files. Of course this means that if you have a group of people working together on a project then they probably need to all have the same Identity and in practice you would probably end up with everyone having the same identity. I ve given up on the idea of using UBAC in Debian. The Current Plan for Users and SE Linux Access Control in Debian My plan is to have things work in Squeeze in much the same way as in Lenny. You have a SE Linux identity assigned to a login session and everything related to it (including cron jobs) based on the Unix account name or possibly the Unix group name (if there are login entries for both the user-name and the group-name then the user-name entry has precedence). The mapping between Unix accounts and SE Linux identities is configured by the sysadmin and SE Linux identities don t matter much for the Targeted configuration (which is what most people use). The identity determines which roles may be used and also has a limit on the MCS categories. The MCS categories are also specified in the login configuration which has to be a sub-set of the categories used by the identity record. So for example the following is the output of a couple of commands run on a Debian/Unstable system. They show that the test Unix account is assigned a SE Linux identity of staff_u and an MCS range of s0-s0:c1 (this means it creates files by default at level s0 and can also write to other files at that level, but can also have read/write access to files at the level s0:c1 ). The staff_u identity (as shown in the output of semanage user -l can be used with all categories in the set s0:c0.c1023 where the dot means the set of categories from c0 to c1023 inclusive) but in the case of the test user only one category will be used. The test group however (as expressed with %test ) is given the identity user_u and is not permitted to use any categories.

# semanage login -l
Login Name    SELinux User    MLS/MCS Range           
%test         user_u          s0                     
__default__   unconfined_u    s0-s0:c0.c1023         
root          unconfined_u    s0-s0:c0.c1023         
system_u      system_u        s0-s0:c0.c1023         
test          staff_u         s0-s0:c1               
# semanage user -l
             Labeling   MLS/       MLS/                         
SELinux User Prefix     MCS Level  MCS Range         SELinux Roles
root         sysadm     s0         s0-s0:c0.c1023    staff_r sysadm_r system_r
staff_u      staff      s0         s0-s0:c0.c1023    staff_r sysadm_r
sysadm_u     sysadm     s0         s0-s0:c0.c1023    sysadm_r
system_u     user       s0         s0-s0:c0.c1023    system_r
unconfined_u unconfined s0         s0-s0:c0.c1023    system_r unconfined_r
user_u       user       s0         s0                user_r

I hope to get the policy written to support multiple user roles in time for the release of Squeeze. If I don t make it then I will put such a policy on my own web site and try to get it included in an update. The policy currently basically works for a Targeted configuration where the users are not confined (apart from MCS). How MMCS Basically Works The vast majority of SE Linux users run with the MCS policy rather than the MLS policy. For Debian I have written a modified version of MCS that I call MMCS. MMCS is mandatory (you can t relabel files to escape it) and it prevents write-down. If a process has the range s0-s0:c1,c3 then it has full access to files labelled as s0, s0:c1, s0:c3, and s0:c1,c3 and any files it creates will be labeled as s0. If a process has the range s0:c1-s0:c1,c3 then it has read-only access to files labelled as s0 and s0:c3 and read-write access to files labelled as s0:c1 and s0:c1,c3. This means that any secret data it accesses that was labelled with category c1 can t be leaked down to a file that is not labelled with that category. Now MCS currently has no network access controls, so there s nothing stopping a user from using scp or other network utilities to transfer files. But that s the way with most usable systems. I don t think that this is necessarily a problem, the almost total lack of network access controls in a traditional Unix model doesn t seem to concern most people. Now to REALLY Answer that Question SE Linux is a Mandatory Access Control (MAC) system, this makes it inherently different to a Discretionary Access Control (DAC) system such as traditional Unix access controls. Unix permissions are based on each file having a UID, a GID, and a set of permissions and each process having a UID, a GID, and a set of supplementary GIDs. If a user runs a setuid or setgid program then the process will have extra privileges. It also has a lot of stuff that most people aren t aware of such as real vs effective UIDs, the tag bit, setgid directories, and lots more including some quite arbitrary things like making ports <1024 special. SE Linux is based on every object (process, file, socket, etc) having a single security label which includes an identity, a role, a type, and a sensitivity label (MCS categories or an MLS range). There is no support for an object to have more than one label. The SE Linux equivalent to setuid/setgid files is a label for a file which triggers a domain transition when it s executed. This differs from setuid files in that the domain transition is complete (the old privileges can t be restored) and the transition is generally not to a strict super-set of the access (usually a different sub-set of possible access and sometimes to lesser access). These differences are quite fundamental. So really SE Linux can t implement traditional Unix access control. What SE Linux is designed to do is to provide a second layer of defense and to also provide access controls that have different aims than that of Unix permissions such as being mandatory and implementing features such as MLS.

• [1] http://etbe.coker.com.au/2008/08/25/is-se-linux-unixish/

I'we just launched website (well it's quite pretentious name for few pages), which shows debts for politics when they were in government in Czech Republic. We're still not in situation like in Greece, but it can quite easily happen that we would be in few years and it's good to show some numbers describing what was happening in past. It took just few hours of coding yesterday evening using Django. Domain has been registered and it is online now. While doing such thing, I decided to make more fun for me and publish the link just on social networks (Identi.ca, Twitter, Facebook, LinkedIn) and follow if (and how) the link will spread. So no, you will not find link in this blog post :-).

Recently, I ve been working on an exciting project with JaNet(UK) on a project to bring federated authentication to non-web applications. I ve worked on authentication projects a lot, although this is the first federation project I ve worked on. The big difference appears to be an emphasis on credential independence: the subject (person trying to authenticate) and service will not share a common credential type. Within their organization, the subject and their identity provider share a credential. Then, the federation has some credential mechanism such that the user s organization and the service share some (probably completely different) type of credential. The other emphasis is on providing personalization. For web applications, there are a lot of options to achieve this: Information Card, Open ID, SAML, and OAuth all provide solutions in this space. However there are not good options for non-web applications. If you out-source your mail and chat infrastructure but want to use your own chat client or IMAP client, then you will not get the same federation benefits you can get with the web. If you re using usernames and passwords and don t mind the potential problems with your out-sourcing provider being able to impersonate all your users, you can simply synchronize usernames and passwords. Within an enterprise, you can do better using Kerberos. JaNet(UK) runs the UK Access Federation, which is a SAML-based web single-sign-on federation. In order to better meet the needs of their customers they d like to expand this offering to non-web applications. This demand is apparently shared across the European academic community. I suspect there is also some demand in the US academic community and in enterprise situations. With the web, it turns out that you have a convenient platform for interactions with the identity provider: you can simply direct a web browser to the identity provider and need not specify the user interaction with the web browser at all. This is seen as a significant branding and usability advantage. With other environments, it becomes necessary to specify the interaction with the identity provider. Consider an automated client that wishes to examine a mail box and provide advanced mail sorting or aggregation. That automated client cannot directly use a web browser. OAuth solves this issue with an enrollment step that does typically involve a web browser that produces a consumer key and an authentication step that does not. However for non-web clients it seems like avoiding reliance on the browser authentication will be important. It turns out we already have widely used technologies that do this: the Extensible Authentication Protocol (EAP) mediates the interaction between a subject and identity provider for obtaining network access. It also turns out that we have fairly good technologies for abstract security services within non-web applications: thanks to Kerberos and Active Director, many application protocols and a fair number of applications support GSS-API. JaNet(UK) proposes to combine these technologies with SAML in order to produce a solution for federation beyond the web. I prepared a feasibility analysis of this proposal. At a technical level, the proposal is sound. There s a lot of standardization and implementation work, but there appears to be sufficient motivation to form the seeds of a standards activity and put together a proof-of-concept implementation. However, the big question is Will anyone use it? In particular, to be useful beyond fairly small communities, support from client vendors and application framework vendors will be needed. It s taken massive money and around 20 years to get Kerberos support to a point where it is effective within an enterprise. Moonshot can leverage that work to a large extent, but moonshot may also have greater usability and penetration goals. It s interesting that I m advocating EAP for application layer authentication. When I was a Security AD, I made a strong statement that EAP must only be used for network access. I ve been fairly consistent about that since then. I think there are two huge problems with using EAP for application authentication. The first is that EAP only authenticates the home realm; it does not authenticate what service you re going to. So you might try to connect to your e-mail and end up giving something access to your stored files and pictures instead. That is, EAP has a phishing exposure in the federated context. If the only thing you can get by using EAP is network access, that exposure is only moderate. However in a fully federated environment that is a huge exposure. Moonshot will address this problem by using EAP channel binding and by doing the necessary work to make that a viable solution. The second concern is that interoperability is reduced when you have multiple authentication approaches for the same problem. If EAP is going to be used for application authentication, we need to understand how it relates to the rest of the application authentication metasystem. Moonshot proposes such a relationship, addressing my objection. Moonshot is designed to work well with the objectives of the Identity Metasystem and its laws of identity. It uses a different technology, but does have an approach for dealing with claims-based identity and hopefully will have a user experience very similar to the identity metasystem. It uses a different underlying technology. However one of the main beliefs behind the identity metasystem is that is the user experience and universal interoperability that is important, not any specific technology. In its domain, the technologies Moonshot selects will be a better fit than a web services stack. It s strange not to be working on Kerberos; Moonshot uses some Kerberos technology, but its core is definitely not today s Kerberos. In some ways it is fun to be working on something new. There s one aspect of Kerberos I really miss: Moonshot has nothing like tickets. There s no place to remember state or exchange to directly involve the client in what the server learns. My analysis talks about ways to make Moonshot more like Kerberos; there are some potential advantages, but so far, the tradeoffs do not justify changes. We re hoping to have a bar BOF at IETF 77 and a BOF in the summer at IETF 78.

Conference Management systems are web applications designed to make the lives of conference organisers easier. They usually include features such as registration, payment, paper submission and review, scheduling and publishing of announcements.

Since Wikipedia is apparently not a place for such a "spammy list," I figured I should at least post this here:

Name	Used by	License	Programming Language
A Conference Toolkit	YAPC::Eu	Artistic License	Perl
ConMan	Utah Open Source Conference, Texax Linux Fest	GPL	Python (Django)
Open Conference Systems	various academic conferences	GPL	PHP
OpenConferenceWare	Open Source Bridge	MIT	Ruby on Rails
Pentabarf	DebConf, FOSDEM, CCC	GPL	Ruby on Rails
SCALEreg	SCALE	GPL	Python (Django)
Zookeepr	linux.conf.au	GPL	Python (Pylons)

If you know of any other Open Source systems, please leave a comment!

I just updated the software at http://ddportfolio.debian.net/ it contains some new links (UbuntuDiff, piuparts, patch tracker, search on the Debian website and with search.debian.org). Thanks to Paul Wise for the ideas. If you want to have ddportfolio.debian.net in your language you may send me translated versions of the translation template in the git repository.

In an amazing but meaningless feat of synchronicity, Toxicore's post led me to check how much distance have I done while excercising I have mainly done this cycling, although some running also comes into play. Turns out I was just about to break the same mark he did. From my Nokia Sportstracker records: Before breaking the 5000 Km mark, yesterday at noon:
After breaking the 5000 Km mark, yesterday at night:
So, following Bubulle's style... If I have done 5000 Km since January 2008, when do you think I should reach 5500? 6000? 7000? Infinity and beyond?

One interesting feature in Active Directory, is the ability to create a new user with an expired password, and thus force the user to change the password on the first login attempt. I'm not quite sure how to do that with the LDAP setup in Debian Edu, but did some initial testing with a local account. The account and password aging information is available in /etc/shadow, but unfortunately, it is not possible to specify an expiration time for passwords, only a maximum age for passwords. A freshly created account (using adduser test) will have these settings in /etc/shadow:

root@tjener:~# chage -l test
Last password change                                    : May 02, 2010
Password expires                                        : never
Password inactive                                       : never
Account expires                                         : never
Minimum number of days between password change          : 0
Maximum number of days between password change          : 99999
Number of days of warning before password expires       : 7
root@tjener:~#

The only way I could come up with to create a user with an expired account, is to change the date of the last password change to the lowest value possible (January 1th 1970), and the maximum password age to the difference in days between that date and today. To make it simple, I went for 30 years (30 * 365 = 10950) and January 2th (to avoid testing if 0 is a valid value). After using these commands to set it up, it seem to work as intended:

root@tjener:~# chage -d 1 test; chage -M 10950 test
root@tjener:~# chage -l test
Last password change                                    : Jan 02, 1970
Password expires                                        : never
Password inactive                                       : never
Account expires                                         : never
Minimum number of days between password change          : 0
Maximum number of days between password change          : 10950
Number of days of warning before password expires       : 7
root@tjener:~#  

So far I have tested this with ssh and console, and kdm (in Squeeze) login, and all ask for a new password before login in the user (with ssh, I was thrown out and had to log in again). Perhaps we should set up something similar for Debian Edu, to make sure only the user itself have the account password? If you want to comment on or help out with implementing this for Debian Edu, please contact us on debian-edu@lists.debian.org. Update 2010-05-02 17:20: Paul T tterman tells me on IRC that the shadow(8) page in Debian/testing now state that setting the date of last password change to zero (0) will force the password to be changed on the first login. This was not mentioned in the manual in Lenny, so I did not notice this in my initial testing. I have tested it on Squeeze, and 'chage -d 0 username' do work there. I have not tested it on Lenny yet. Update 2010-05-02-19:05: Jim Paris tells me via email that an equivalent command to expire a password is 'passwd -e username', which insert zero into the date of the last password change.