Search Results: "pm"

8 June 2010

Matt Zimmerman: DevOps and Cloud

DevOps I first heard about DevOps from Lindsay Holmwood at linux.conf.au 2010. Since then, I ve been following the movement with interest. It seems to be about cross-functional involvement in software teams, specifically between software development and system administration (or operations). In many organizations, especially SaaS shops, these two groups are placed in opposition to each other: developers are driven to deliver new features to users, while system administrators are held accountable for the operation of the service. In the best case, they maintain a healthy balance by pushing in opposite directions, but more typically, they resent each other for getting in the way, as a result of this dichotomy:
Development Operations
is responsible for creating products offering services
is measured on delivery of new features high reliability
optimizes by increasing velocity controlling change
and so is perceived as reckless and irresponsible obstructing progress
Of course, both functions are essential to a viable service, and so DevOps aims to replace this opposition with cooperation. By removing this friction from the organization, we hope to improve efficiency, lower costs, and generally get more work done.
So, DevOps promotes the formation of cross-functional teams, where individuals still take on specialist development or operations roles, but work together toward the common goal of delivering a great experience to users. By working as teammates, rather than passing work over the wall , they can both contribute to development, deployment and maintenance according to their skills and expertise. The team becomes a devops team, and is responsible for the entire product life cycle. Particular tasks may be handled by specialists, but when there s a problem, it s the team s problem. Some take it a step further, and feel that what s needed is to combine the two disciplines, so that individuals contribute in both ways. Rather than thinking of themselves as developers or sysadmins , these folks consider themselves devops . They work to become proficient in both roles, and to synthesize new ways of working by drawing on both types of skills and experience. A common crossover activity is the development of sophisticated tools for automating deployment, monitoring, capacity management and failure resolution. DevOps meets Cloud Like DevOps, cloud is not a specific technology or method, but a reorganization of the model (as I ve written previously). It s about breaking down the problem in a different way, splitting and merging its parts, and creating a new representation which doesn t correspond piece-for-piece to the old one. DevOps drives cloud because it offers a richer toolkit for the way they work: fast, flexible, efficient. Tools like Amazon EC2 and Google App Engine solve the right sorts of problems. Cloud also drives DevOps because it calls into question the traditional way of organizing software teams. A development/operations division just doesn t fit cloud as well as a DevOps model. Deployment is a classic duty of system administrators. In many organizations, only the IT department can implement changes in the production environment. Reaping the benefits of an IaaS environment requires deploying through an API, and therefore deployment requires development. While it is already common practice for system administrators to develop tools for automating deployment, and tools like Puppet and Chef are gaining momentum, IaaS makes this a necessity, and raises the bar in terms of sophistication. Doing this well requires skills and knowledge from both sides of the fence between development and operations, and can accelerate development as well as promote stability in production.
This is exemplified by infrastructure service providers like Amazon Web Services, where customers pay by the hour for black box access to computing resources. How those resources are provisioned and maintained is entirely Amazon s problem, while its customers must decide how to deploy and manage their applications within Amazon s IaaS framework. In this scenario, some operations work has been explicitly outsourced to Amazon, but IaaS is not a substitute for system administration. Deployment, monitoring, failure recovery, performance management, OS maintenance, system configuration, and more are still needed. A development team which is lacking the experience or capacity for this type of work cannot simply switch to an IaaS model and expect these needs to be taken care of by their service provider.
With platform service providers, the boundaries are different. Developers, if they build their application on the appropriate platform, can effectively outsource (mostly) the management of the entire production environment to their service provider. The operating system is abstracted away, and its maintenance can be someone else s problem. For applications which can be built with the available facilities, this will be a very attractive option for many organizations. The customers of these services may be traditional developers, who have no need for operations expertise. PaaS providers, though, will require deep expertise in both disciplines in order to build and improve their platform and services, and will likely benefit from a DevOps approach. Technical architecture draws on both development and operations expertise, because design goals like performance and robustness are affected by all layers of the stack, from hardware, power and cooling all the way up to application code. DevOps itself promotes greater collaboration on architecture, by involving experts in both disciplines, but cloud is a great catalyst because cloud architecture can be described in code. Rather than talking to each other about their respective parts of the system, they can work together on the whole system at once. Developers, sysadmins and hybrids can all contribute to a unified source tree, containing both application code and a description of the production environment: how many virtual servers to deploy, their specifications, which components run on which servers, how they are configured, and so on. In this way, system and network architecture can evolve in lockstep with application architecture. Cloudy promises such as dynamic scaling and fault tolerance call for a DevOps approach in order to be realized in a real-world scenario. These systems involve dynamically manipulating production infrastructure in response to changing conditions, and the application must adapt to these changes. Whether this takes the form of an active, intelligent response or a passive crash-only approach, development and operational considerations need to be aligned. So what? DevOps and cloud will continue to reinforce each other and gain momentum. Both individuals and organizations will need to adapt in order to take advantage of the opportunities provided by these new models. Because they re complementary, it makes sense to adopt them together, so those with expertise in both will be at an advantage.

7 June 2010

Chris Lamb: Distributing locking with Python and Redis

This is already possible with Memcache, but if you are already drinking the Koolaid...
import time
class Lock(object):
    def __init__(self, key, expires=60, timeout=10):
        """
        Distributed locking using Redis SETNX and GETSET.
        Usage::
            with Lock('my_lock'):
                print "Critical section"
        :param  expires     We consider any existing lock older than
                             expires  seconds to be invalid in order to
                            detect crashed clients. This value must be higher
                            than it takes the critical section to execute.
        :param  timeout     If another client has already obtained the lock,
                            sleep for a maximum of  timeout  seconds before
                            giving up. A value of 0 means we never wait.
        """
        self.key = key
        self.timeout = timeout
        self.expires = expires
    def __enter__(self):
        timeout = self.timeout
        while timeout >= 0:
            expires = time.time() + self.expires + 1
            if redis.setnx(self.key, expires):
                # We gained the lock; enter critical section
                return
            current_value = redis.get(self.key)
            # We found an expired lock and nobody raced us to replacing it
            if current_value and float(current_value) < time.time() and \
                redis.getset(self.key, expires) == current_value:
                    return
            timeout -= 1
            time.sleep(1)
        raise LockTimeout("Timeout whilst waiting for lock")
    def __exit__(self, exc_type, exc_value, traceback):
        redis.delete(self.key)
class LockTimeout(BaseException):
    pass
One common use case for distributed locks in web applications is to prevent clients dog-piling onto an expensive cache key:
def cache_without_dogpiling(key, cb, cache_expiry=None, *args, **kwargs):
    val = cache.get(key)
    if val is not None:
        return val
    # Cache miss; gain the lock to prevent multiple clients calling cb()
    with Lock(key, *args, **kwargs):
        # Check cache again - another client may have set the cache
        val = cache.get(key)
        if val is None:
            val = cb()
            cache.set(key, val, cache_expiry)
        return val
def slow():
    print "Inside slow()"
    return 1 + 1 # Python is slow
>>> cache_without_dogpiling('my_key', slow, 60 * 10)
Inside slow()
2
>>> cache_without_dogpiling('my_key', slow, 60 * 10)
2
As a bonus, if you don't want your test or development environment to rely on Redis, you can replace it with a no-op lock:
import contextlib
@contextlib.contextmanager
def Lock(*args, **kwargs):
    yield

5 June 2010

Jos? Parrella: Ditching PCI cards in Asterisk for fun and profit

In the past, I ve used PCI cards providing telephony TDM for my Asterisk-based projects. I ve only worked with Digium s FXO/FXS and T1/E1 cards, both in PCI and PCI-x formats, until a client asked for a gateway solution with FXO, FXS and E1, where using the PCI cards in a 1U IPBX was simply out of the question. Using Ethernet gateways has always been a problem in those cases, since entry-to-medium level solutions lack the processing power for massive call handling, and signalling is poor, making the architecture unfit for reporting purposes and specialized apps such as call centers. But hey, that s my experience with gateways. Therefore, I looked into Xorcom s Astribank. They pack FXO, FXS and PRI/BRI ports inside a convenient 19 1U case, which can connect to an Asterisk-based IPBX server via USB and gets recognized as an integral DAHDI device in modern setups. So I ordered a composite 8xFXO, 8xFXS and 1xE1 PRI from Astribank, labelled XR0072. Getting the equipment from Arizona to Florida was easy, and since I worked with DHL, getting it to Guayaquil in Ecuador was a breeze (next-day delivery) but passing customs is, as always in Latin America, a time-consuming operation. After paying 12.5% taxes and then some for DHL operations, and 15 days waiting, I got the equipment in Quito. To my surprise, the Astribank was extremely light and small, slightly near 3 Kg in 1U form factor and under 1/4 depth in the rack. While it had a rack-mounting kit, which I used, it could also rest securely over my IPBX without rack screws. My Astribank uses a single USB cable to connect to the master PBX it could also be connected to a secondary PBX and provide a fault tolerant setup. It also depends on a DC supply, which they included. Since I was using a modern Debian, with a modern Asterisk with already-compiled modern DAHDI (formerly called Zapata/Zaptel) drivers, I just connected the equipment via USB to the IPBX, issued an invoke-rc.d dahdi restart and we were good to go. Note: you do need the fxload package in Debian for firmware loading to the Astribank. And for good to go I mean you can check the registered spans with dahdi_registration and, as soon as you connect the E1 cable that comes from the CPE to the CPE port (which in my XR0072 is the first on the lower row for PRI) you get an OK status with dahdi_tool. Also note that I use an E1/CPE port, which is the default for the Astribank, otherwise I d have to set the XPP_PRI_SETUP variable as indicated on the User Manual. DAHDI insists on configuring a crc4 parameter for the detected spans, which I needed to remove and restart DAHDI so the equipment could sync with the CPE (that is, the box that the telco provided, with the balun and the crossover RJ45 cable) and avoid some nasty noise issues on the line, that also occur with Digium s Wildcard for PRI.
usb:008/005          xpp_usb+     e4e4:1162 Astribank-modular FPGA-firmware
 LABEL=[usb:...]       CONNECTOR=@usb-0000:00:1d.7-1
	XBUS-00/XPD-00: E1       (31)  Span 1  DAHDI-SYNC
	XBUS-00/XPD-10: FXS      (8)   Span 2
	XBUS-00/XPD-20: FXO      (8)   Span 3
Using DAHDI s dahdi_genconf command writes an /etc/asterisk/dahdi-channels.conf file with the group and channel configuration as detected by DAHDI. It should be included by the chan_dahdi.conf file, which, in my case, is not necessary since I only wanted to enable one DAHDI group (DAHDI/g0) with my 31 E1 channels (1-15,17-31)
group=0
context=from-pstn
channel => 1-15,17-31
That means that when I want to use the FXO/FXS ports, it d probably be easier to include the dahdi-channels.conf (FXS ports, for example, belong to a different context, from-internal) so I don t have to type more than my laziness permits. I also found that, in my setup, some Asterisk parameters were superfluous, such as the switchtype (either national or euroisdn work, and the former is the default) or the echo-cancelling stuff, so I assume Astribank s doing a great job in negotiating that with the CPE. Do take note, however, that in my case the CPE needed to be cold-restarted so I could get an Up status in Asterisk (pri show spans) and that my only non-default, channel related configuration in chan_dahdi.conf is signalling=pri_cpe. So far, so good. I m really happy with the Astribank and I can fully understand how useful it is in terms of price (quite competitive in comparison with traditional cards, specially when bought from resellers -like me-), size/weight and functionality/ease of setup and administration. Kudos!

3 June 2010

Matthew Garrett: Poulsbo still makes me sad

Meego 1.0 was released last month. Predictably enough, it lacks any form of support for the GMA500 graphics chipset[1], which is reasonably in line with Intel's placing of US15W as an embedded product for MID-type devices rather than having anything to do with netbooks. Of course, the market pretty much ignored that detail, so we're left with the release of a netbook experience that's incompatible with a large number of netbooks.

There's been hints that there might be some sort of new driver coming, though, and Intel's rapidly heading towards the release of Moorestown - their new MID platform that's x86 but not a PC, that replaces Poulsbo (unless you want to run Windows) and which has the same SGX 3D core[2] as Poulsbo. Moorestown support has started landing in the mainline Linux kernel and Intel are talking about devices shipping with it this year, so we ought to see the graphics driver soon, right?

Well, kind of. Digging through the Meego kernel git tree reveals an updated driver with Moorestown support. The blob is too big for gitorious to display, so I've put up a copy here. Things to note - this isn't a new driver (it's clearly derived from the PSB one), it includes a huge blob of Imagination's platform-independent code, the front end of the driver is still pretty much the i915 driver with a small number of PSB-specfic changes and despite the cheerily optimistic "Patch-mainline: 2.6.35?" at the top it stands pretty much no chance whatsoever of going mainline given that it's even more offensive than the previous version and that one got rejected out of hand.

So it's not clear what Intel's doing here. If this is the driver that Intel are developing for upstream then there's been a pretty serious breakdown in communication over what their driver has to look like. If it's not, Intel have another and presumably better driver somewhere that they're developing entirely behind closed doors despite having shipped millions of units of hardware that people would dearly love to be able to run mainline kernels on. Neither case looks especially good, so the continuing perception would seem to be that only a subset of the company understands the Linux development model - we'll see how this ends up interacting with Meego as a whole.

(Huh. I was about to hit "Post" on this, and then found that another and entirely different driver was submitted for the Meego kernel last week. A copy of it's here. "IVI" in this context appears to stand for "In-vehicle interface" and is aimed at those contexts rather than generic ones. It includes much the same Imagination glue code as the Moorestown driver, appears to be derived from a driver that has support for other Intel chipsets, was written to be at least partially OS-independent given that it abstracts things like PCI and stands even less chance of going upstream than the other driver - as acknowledged by "Patch-mainline: Never". Hilariously, if we include IEGD, that makes three different drivers for the same hardware from the same company, each having no chance to go upstream. Thintel.)

[1] Weirdly, it includes the kernel DRM for Matrox cards. No X driver though.
[2] It's clocked higher, though

2 June 2010

Axel Beckert: xen-tools back in Debian Unstable

After xen-tools had been removed from Debian Unstable by the upstream developer last December due to no more upstream development, I took over upstream development as well as Debian package maintainership in January. Since then a lot happened: But despite all the things which were to fix after Steve discontinued the xen-tools developement, Steve Kemp did a great job to bring xen-tools so far initially. I m really happy that this software exists and that s the reason why I won t let it die so fast. :-) xen-tools 4.2 beta 1 released Last weekend I released xen-tools 4.2 beta 1 which I also uploaded to Debian Unstable. And thanks to the active ftp-master team, xen-tools went through the NEW queue in less than 12 hours and is now available again in Debian Unstable. 4.2 beta 1 still has some known bugs, the worst ones are in xen-delete-image. But no fear, it doesn t delete more than wanted, just not enough or things which don t exist anyway. :-) There are also some smaller issues to fix for a final 4.2 release. We may also create an PPA for Ubuntu Lucid in the future to work around the lack of xen-tools in Ubuntu Lucid. What about Fedora/RedHat/CentOS support? Since rinse is out of date with regards to current Fedora releases, rpmstrap seems to be out of date for even longer (it has been removed from Debian before the Lenny release) and febootstrap neither seems to work out of the box nor seems to support bootstrapping to mount points (because it wants to delete them first and fails), bootstrapping Fedora ist currently not supported. The copy installation method should work though. rinse still is able to bootstrap CentOS 4 and 5, but the installation seems incomplete. We ll try to work around these issues for the final 4.2 release. We haven t looked at mach and mock so far, but according to Lucas, both are out of date, too. Sure, implementing Lucas howto in form of a script would be a possible way, but we think it would just create yet another not so well supported clone of rinse, rpmstrap, mach, mock or febootstrap. Because of that, Dmitry works on support for the installation of Xen DomUs via kickstart. Fate of xen-shell? Xen-shell has also been removed from Debian Unstable last December because Steve stopped its developement, too. But in comparison to xen-tools itself, we didn t resurrect xen-shell since none of us currently has use for it. So if you are currently using xen-shell and intend to continue to do so, you should hurry up if you want to see it in Debian Squeeze. We can also help you, but we surely can t and won t do it alone nor will we take the majority of the necessary workload.

1 June 2010

Steve Langasek: bzr-git case study

Robert Collins did a very nice writeup not so long ago of creating a Debian package with bzr-builddeb, but he starts from the assumption that upstream also uses bzr. As we all know, there are many projects that have their upstream sources in a VCS, but don't use bzr. Can we still get reasonable results from bzr-builddeb, including the ability to merge new upstream releases direct from the VCS, if, say, upstream is using git? Yes, we can! Of course, if you're happy with using git as the VCS for your Debian packaging already, there's probably no compelling reason for you to switch to bzr. But if you prefer to use bzr and have been frustrated at having to choose between being able to use the VCS client of your choice for your packages and being able to use your VCS client to access upstream revision history, read on. Thanks to the fine work of Jelmer Vernooij and friends, bzr-git has been usable for a while if you just want to track the default git branch. But what if you care about tracking multiple upstream branches? Enter bzr git-import, which we'll use to set up a bzr repo from scratch for our cifs-utils package, with a full import of all the branches of upstream's git tree. First we do some work to set up a shared bzr repository. As long as we're going to the effort of mirroring all the git branches, we probably want to publish them for other people to use, so we make sure our bzr repository is reasonably configured for sharing data between branches:
$ bzr init-repo --no-trees cifs-utils.server
Shared repository (format: 2a)
Location:
  shared repository: cifs-utils.server
$ cd cifs-utils.server
Then we use bzr git-import, provided by the bzr-git plugin, to do a full import of the upstream branches into a cleverly named subdirectory:
$ bzr git-import git://git.samba.org/cifs-utils.git upstream
[/                   ] Counting objects: 181, done.
Now we create our own local 'trunk' for the package, by branching from the upstream tag matching the release version we're going to package.
$ bzr branch -r tag:cifs-utils-4.0rc1 upstream/HEAD trunk
Branched 42 revision(s).
Do a little more prep work for future branches...
$ mkdir branches
And that's it. Now we have a bzr repo locally that we can push out to our hosting server of choice (N.B.: we could do this all directly on the server, but alioth doesn't currently have bzr-git installed):
$ rsync -az . alioth.debian.org:/bzr/pkg-samba/cifs-utils/
$
Now that we have a bzr repository, it's time to get ourselves a working directory and do some packaging. We're probably going to work with multiple branches locally as well, so we create another shared repository for local use, and get a copy of the trunk branch we created before.
$ bzr init-repo cifs-utils
Shared repository with trees (format: 2a)
Location:
  shared repository: cifs-utils
$ cd cifs-utils
$ bzr co bzr+ssh://bzr.debian.org/bzr/pkg-samba/cifs-utils/trunk
Now we grab the upstream tarball, and whip up some quick packaging with (what else?) debhelper 7:
$ wget ftp://ftp.samba.org/pub/samba/cifs-utils/cifs-utils-4.0rc1.tar.bz2
$ ln -s cifs-utils-4.0rc1.tar.bz2 cifs-utils_4.0~rc1.orig.tar.bz2
$ cd trunk
$ mkdir -p debian/source
$ echo '3.0 (quilt)' > debian/source/format
$ echo 7 > debian/compat
$ cp /usr/share/doc/debhelper/examples/rules.tiny debian/rules
$ dch --package cifs-utils --versio 4.0~rc1-1 --create 'Initial package'
Create debian/control by hand, generate an initial source package, and import it into bzr with bzr-builddeb.
$ debuild -uc -us -S -i
$ rm -r debian
$ bzr import-dsc ../*.dsc
$
Since we want to continue tracking upstream development, we need to periodically sync our bzr import of the git tree.
$ bzr git-import git://git.samba.org/cifs-utils.git bzr+ssh://bzr.debian.org/bzr/pkg-samba/cifs-utils/upstream
$
And when we find a new upstream version in that import that we want to package, bzr-builddeb makes this a snap, too.
$ cd cifs-utils
$ wget ftp://ftp.samba.org/pub/samba/cifs-utils/cifs-utils-$version.tar.bz2
$ cd trunk
$ bzr merge-upstream --v3 --version $version_with_epoch \
    ../cifs-utils-$version.tar.bz2 -r tag:cifs-utils-$version \
    bzr+ssh://bzr.debian.org/bzr/pkg-samba/cifs-utils/upstream/HEAD
No distribution specified, and no changelog, assuming 'debian'
Committing to: /tmp/tmpbzOVMs/upstream/
Committed revision 2.
All changes applied successfully.
The new upstream version has been imported.
You should now review the changes and then commit.
$ bzr diff
<review changes>
$ bzr commit -m "merge upstream $version"
Committing to: bzr+ssh://bzr.debian.org/bzr/pkg-samba/cifs-utils/trunk
modified debian/changelog
Committed revision 50.
$
And that's it! It's exciting to see signs of real interoperability between DVCSes at last. Thanks to everyone who's helped make it possible!

Russell Coker: Can SE Linux Implement Traditional Unix Users and Groups?

I was asked by email whether SE Linux could implement traditional Unix users and groups. The Strictly Literal Answer to that Question The core of the SE Linux access control is the domain-type model where every process has a domain and every object that a process can access (including other processes) has a type. Domains are not strongly differentiated from types. It would be possible to create a SE Linux policy that created a domain for every combination of UID and GID that is valid for a user shell given that such combinations are chosen by the sysadmin who could limit them to some degree. There are about 2^32 possible UIDs and about 2^32 possible GIDs, as every domain is listed in kernel memory we obviously can t have 2^64 domains, but we could have enough to map a typical system that s in use. Of course the possible combinations of supplemental groups probably makes this impossible for even relatively small systems, but we can use a simpler model that doesn t emulate supplemental groups. For files there are more possible combinations because anyone who is a member of a group can create a SETGID directory and let other users create files in it. But in a typical system the number of groups is not much greater than the number of users the maximum number of groups is typically the number of users plus about 60. So if we had 100 users then the number of combinations of UID and GID would be something like 100*(100+60)=16,000 it should be possible to have that many domains in a SE Linux policy (but not desirable). Then all that would be needed is rules specifying that each domain (which is based on a combination of UID and GID) can have certain types of access to certain other types based on them having either the same UID or the same GID. Such a policy would be large, it would waste a lot of kernel memory, it would need to be regenerated whenever a user is added, and it s generally something you don t want to use. No-one has considered implementing such a policy, I merely describe it to illustrate why certain configuration options are not desirable. The rest of this post is about realistic things that you can do with SE Linux policy and how it will be implemented in Debian/Squeeze. My previous post titled Is SE Linux Unixish? addresses this issue at a more conceptual level and also considers MAC vs DAC [1]. The History of mapping Unix Accounts to SE Linux Access Control In the early releases of SE Linux (long before it was included in Fedora) every user who could login to a system needed to have their user-name compiled into the policy. The policy specified which roles the user could access, the roles specified which domains could be accessed, and therefore what the user could do. The identity of files on disk was used for two purposes, one was logging (you could see who created a file) and the other was a permission check for the SE Linux patched version of Vixie cron which would not execute any command on behalf of a user unless the identity on the crontab file in the cron spool matched the identity used to run it this is an analogy of the checks that Vixie cron makes on the Unix UID of the crontab file (some other cron daemons do fewer checks). Having to recompile policy source every time you added a user was annoying. So a later development was to allow arbitrary mappings between Unix account names and SE Linux Identities (which included a default identity) and another later development was to have a utility program semanage to map particular Unix account and group names to SE Linux identities. This was all done years ago. Fedora Core 5 which was released in 2006 had the modular policy which included these features (apart from mapping Unix groups to SE Linux identities which was more recent). Fedora Core 5 also introduced MCS which was comprised of a set of categories that a security context may have. The sysadmin would configure the set of categories that each account would have. A recent development has been a concept named UBAC (User Based Access Control) which basically means that a process running directly on behalf of a regular user (IE with a SE Linux identity that s not system_u) can only access files that have an identity of system_u or which have the same identity as the process. This means that you can only access your own files or system files not files of other users which may have inappropriate Unix permissions. So for example if a user with a SE Linux identity of john gives their home directory the Unix permission mode of 0777 then a user with a SE Linux identity of jane can t access their files. Of course this means that if you have a group of people working together on a project then they probably need to all have the same Identity and in practice you would probably end up with everyone having the same identity. I ve given up on the idea of using UBAC in Debian. The Current Plan for Users and SE Linux Access Control in Debian My plan is to have things work in Squeeze in much the same way as in Lenny. You have a SE Linux identity assigned to a login session and everything related to it (including cron jobs) based on the Unix account name or possibly the Unix group name (if there are login entries for both the user-name and the group-name then the user-name entry has precedence). The mapping between Unix accounts and SE Linux identities is configured by the sysadmin and SE Linux identities don t matter much for the Targeted configuration (which is what most people use). The identity determines which roles may be used and also has a limit on the MCS categories. The MCS categories are also specified in the login configuration which has to be a sub-set of the categories used by the identity record. So for example the following is the output of a couple of commands run on a Debian/Unstable system. They show that the test Unix account is assigned a SE Linux identity of staff_u and an MCS range of s0-s0:c1 (this means it creates files by default at level s0 and can also write to other files at that level, but can also have read/write access to files at the level s0:c1 ). The staff_u identity (as shown in the output of semanage user -l can be used with all categories in the set s0:c0.c1023 where the dot means the set of categories from c0 to c1023 inclusive) but in the case of the test user only one category will be used. The test group however (as expressed with %test ) is given the identity user_u and is not permitted to use any categories.
# semanage login -l
Login Name    SELinux User    MLS/MCS Range           
%test         user_u          s0                     
__default__   unconfined_u    s0-s0:c0.c1023         
root          unconfined_u    s0-s0:c0.c1023         
system_u      system_u        s0-s0:c0.c1023         
test          staff_u         s0-s0:c1               
# semanage user -l
             Labeling   MLS/       MLS/                         
SELinux User Prefix     MCS Level  MCS Range         SELinux Roles
root         sysadm     s0         s0-s0:c0.c1023    staff_r sysadm_r system_r
staff_u      staff      s0         s0-s0:c0.c1023    staff_r sysadm_r
sysadm_u     sysadm     s0         s0-s0:c0.c1023    sysadm_r
system_u     user       s0         s0-s0:c0.c1023    system_r
unconfined_u unconfined s0         s0-s0:c0.c1023    system_r unconfined_r
user_u       user       s0         s0                user_r
I hope to get the policy written to support multiple user roles in time for the release of Squeeze. If I don t make it then I will put such a policy on my own web site and try to get it included in an update. The policy currently basically works for a Targeted configuration where the users are not confined (apart from MCS). How MMCS Basically Works The vast majority of SE Linux users run with the MCS policy rather than the MLS policy. For Debian I have written a modified version of MCS that I call MMCS. MMCS is mandatory (you can t relabel files to escape it) and it prevents write-down. If a process has the range s0-s0:c1,c3 then it has full access to files labelled as s0, s0:c1, s0:c3, and s0:c1,c3 and any files it creates will be labeled as s0. If a process has the range s0:c1-s0:c1,c3 then it has read-only access to files labelled as s0 and s0:c3 and read-write access to files labelled as s0:c1 and s0:c1,c3. This means that any secret data it accesses that was labelled with category c1 can t be leaked down to a file that is not labelled with that category. Now MCS currently has no network access controls, so there s nothing stopping a user from using scp or other network utilities to transfer files. But that s the way with most usable systems. I don t think that this is necessarily a problem, the almost total lack of network access controls in a traditional Unix model doesn t seem to concern most people. Now to REALLY Answer that Question SE Linux is a Mandatory Access Control (MAC) system, this makes it inherently different to a Discretionary Access Control (DAC) system such as traditional Unix access controls. Unix permissions are based on each file having a UID, a GID, and a set of permissions and each process having a UID, a GID, and a set of supplementary GIDs. If a user runs a setuid or setgid program then the process will have extra privileges. It also has a lot of stuff that most people aren t aware of such as real vs effective UIDs, the tag bit, setgid directories, and lots more including some quite arbitrary things like making ports <1024 special. SE Linux is based on every object (process, file, socket, etc) having a single security label which includes an identity, a role, a type, and a sensitivity label (MCS categories or an MLS range). There is no support for an object to have more than one label. The SE Linux equivalent to setuid/setgid files is a label for a file which triggers a domain transition when it s executed. This differs from setuid files in that the domain transition is complete (the old privileges can t be restored) and the transition is generally not to a strict super-set of the access (usually a different sub-set of possible access and sometimes to lesser access). These differences are quite fundamental. So really SE Linux can t implement traditional Unix access control. What SE Linux is designed to do is to provide a second layer of defense and to also provide access controls that have different aims than that of Unix permissions such as being mandatory and implementing features such as MLS.

Julien Danjou: Thoughts and rambling on the X protocol

Two years ago, while working on awesome, I joined the Freedesktop initiative to work on XCB. I had to learn the arcane of the X11 protocol and all the mysterious and old world that goes with it. Now that I've swum all this months in this mud, I just feel like I need to share my thoughts about what become a mess over the decades. When I was unborn

the Toto band were releasing their song "Africa" and some smart guys were working on a windowing system: the X Window System (this is its full name) which therefore has a (too) long history. The latest version of its protocol, the 11th one, has been designed in the 80's. You can learn more about the history in the Wikipedia article about X. In 2010, we still listen disco music and we still use various protocols designed in the 80's and even before X. Music have evolved, protocols have evolved, and so did X11. The problem is that X11 did not evolve that well. The guys at MIT-and-some other-places-with-very-smart-people-in-it created X version 1 in 1984, and updated it until X version 11 (the one we're still using) in 1987. Eleven version in 3 years, that was following the "release early, release often" model. But I don't know why, it just stopped to happen for the last 23 years1. I don't know what changes have been made in the first 11 major versions of the X protocol, but I'm rather sure we should have deserve a couple of major version updates this last 2 decades. In my humble opinion, X11 was not designed to live 23 years. But hey, I'm not blaming anyone here: I was 4 years old and playing Lego when they released this latest version of the X protocol, so there is little chance I'd have done something better.

1. That's not totally true: they added (and then deprecated) many extensions. We won't fix. We'll work-around.

That is probably one of the guideline of the X protocol for the last years. And don't misread me: I'm not bashing anyone thereafter. Since the X11 protocol was aging, the X guys started to add extensions. They added tons of them over the years. This, in application of one of the early principles of X:

It is as important to decide what a system is not as to decide what it is. Do not serve all the world's needs; rather, make the system extensible so that additional needs can be met in an upwardly compatible fashion.

All of them with no exception were added because, bad luck, the X11 protocol did not anticipated the things that happened in the last 23 years, like video, OpenGL, multiple monitors, or the pleasure to draw oval windows. Some of this extensions are still in use, while some of them have been dropped. While this is not a bad thing to extends the protocol, it seems like a bad thing to try to fix the protocol with for example the XFixes extension, even with all the good intentions Keith Packard might have in his greatness. Actually it's even worst than you think

The X11 protocol (without extensions) defines about 120 types of requests: create a window, move a window, etc. Nowadays, there's at least 25 % of them which are useless: usage of server-side font, or the drawing of squares and polygon, are unused by any modern application or toolkit. All of this is superseded by requests from extensions, like the XRender one. The handling of multiple monitors displays has totally been screwed up. X11 has been designed to work in Zaphod mode (independent monitors). But Xinerama, and nowadays XRandR have replaced it up: recent X servers (released after ~2007) does not support Zaphod mode anymore, even if it's a core piece of the X11 protocol. Worst: on many requests, there's limitation or design flaws, like described in this document: Why X Is Not Our Ideal Window System by DEC researchers. We'll add more broken standard on top of that

Following its early principle, X does not define policies but only mechanisms, which seems like a good thing, Consequently, people started writing specifications to determine a number of stuff and dogmas: ICCCM. That was 22 years ago in 1988. It's useless to add that many things in this specification are now obsolete, useless, or that it misses many modern stuff. I was not the only one to think that. The people from what will be the major desktop environments, KDE and GNOME, saw that too in the 90's while I was learning to count. So they wrote EWMH, another standard that comes on top of ICCCM and extends it with nifty features like maximization, full screen mode, etc. The problem is that this standard has also been written by narrow-minded people who at that time, were working on GNOME or KDE (and maybe others). This desktop environments were having and still have some strong concepts of how should work a desktop: "it should have work-spaces", "a window is only on one workspace", "we only see a workspace at a time", "you do not have multiple screens", etc. Dude, we don't care: we have toolkits!

This vision of how the desktop should work have now been written in marble in all applications and libraries implementing EWMH, like GTK+ or Qt. Nowadays, everybody forgot about all of this standards. Toolkits have implemented this, circumvented the X11 protocol limitation and flaws, and nobody wants to look back. Like all standards, obviously some people implemented them badly. This had some side effects, like OpenOffice acting like a pager. We don't look back? Worst, we forgot where we came from!

With all these layers of bad designed standards, the desktop continued to evolve for more than a decade. They continued to add more standard, the more recent ones being based on D-Bus like the Desktop Notification Specification or the latest Status Notifier Specification developed by KDE. The Status Notifier is a new implementation of the good old system tray based on XEmbed, using D-Bus instead of the X11 mechanisms, and adding the possibility to show the system tray with something else than icons. This specification draft saw an important issue and design flaw raised by Wolfgang Draxinger in this thread on the XDG mailing-list. What Wolfgang points out, is that X is network-oriented, and D-Bus is not. Therefore, making the Status Notifier specification to use D-Bus to pass system tray messages around is a bad idea, since running a X application from host A on host B will draw the system tray on the wrong host! Apparently, reading the thread, this does not fear some of the KDE people:

of course this is a bizarre corner case not worth much thought. at least that's what you'll think until you actually run into it yourself (be it because you are testing something or because you are setting up some weird kiosk environment).

What Oswald describes as a corner case is an actual common use case for many of us. Of course, YMMV. From my point of view, this is a step back in the wrong direction. But we can conclude that the network part of X is now worthless, to at least KDE. I used to believe in XCB

When I joined Freedesktop, it was to work on XCB, the X C Binding. XCB is a nice, clean, 21st century technology based API to play with the X11 protocol. Its code is auto generated based on XML file describing the protocol. In comparison, Xlib is 80's obfuscated code with almost no comments and hard-coded things. Only a few people can understand some of its corner like its i18n or XKB implementations. And all its code is synchronous. For people not knowing it yet, X is a network protocol where you send request (like a GET in HTTP) and then get a response. Xlib forces the application to wait for the reply to its request, so the application is blocked until the X server sends the reply to the request. XCB on the other hand does not block and allows the application to send a batch of requests, do some other stuff in the mean time, and then gets the replies. It's like your Web browser would send one request at a time to a Web server, and would wait until you downloaded all the images one by one to display the page. In cases where X and all its clients are on the same host, the latency is small and not really visible, therefore the gain for XCB to be asynchronous is small. On slow network however, the gain can be huge, as proved in the rewrite of xlsclients with XCB by Peter Harris. One of the long standing goal of the XCB folks is to kick-out Xlib, to increase speed and hides latency in X11 applications. That requires to port many libraries, because almost none of them (Cairo being an exception) supports XCB. From where I stand, I don't really see if the work is worth it now. The desktop world is trusted by GNOME and KDE, meaning GTK+ and Qt. It seems none of this toolkits are interested to work on XCB, neither on the X protocol. They probably put hard effort in bypassing X limitation and flaws, and they now sit on top of crap of workarounds and broken-by-design-standard implementation. It seems to me they don't want to go back in the layers and improves things. They're too high to go back down and they don't see what the gain would be. Enlightenment with its EFL was the first toolkit to have an XCB back-end with the work of Vincent Torri. Unfortunately, the back-end is not maintained and nobody cares about it. Last time I tried it, it did not compile at all. X12 ?

There's a page called X12 on the Freedesktop wiki, listing all the things that should be fixed some days. Unfortunately, the list continues to grow up an no one talks about working on X12. On the other hand, there's a handset of people trying to work when they will have time on XKB2, the second version of the "let's-try-to-fix-up-the keyboard-part-of-the-protocol-we-wrote-23-years-ago-a-second-time" extension. To me, it does not seem X12 will happen in the next decade neither. Alternative ?

Do we got alternative to X ? There's Wayland, but it's far from being usable. There's DirectFB, but that's not very portable. None seems candidate to replace X some days to me. Anyhow, none of the main toolkits around support this alternative. GTK+ once supported DirectFB, but as far as I know, it is not supported nor works nowadays, as stated by Josselin Mouette. This is why recent versions of the Debian installer have migrated to X for the graphic part, thanks to Cyril Brulebois work. Conclusion

XCB has been around for more than half-a-decade, and very few people showed interested in it. As far as I can see, nobody is interested to use the X protocol and everybody tries to encapsulate it in some higher-level API as soon as possible to stop seeing it. This leads to poorly written application and toolkits, with a lot of ugly hack. All of that also means that starting to write applications and graphical toolkits based on XCB would be a very interesting project, but that would lead to spend too much time learning to circumvent the X protocol flaws, things that have been done in years by predecessors like Qt and GTK+. Major toolkits implementations have almost nothing to win in going back in the dark water of X. I guess most of their folks prefer to work on shiny 3D effects based on your GPS location, rather than redefining better basis for everyone. The manpower available in the X world is very small. Debian lacking of X maintainers is just the summit of that. There is very smart and very competent and skilled guys in the X world, as you can see by simply reading blog posts on Planet Freedesktop for example (me excluded). Unfortunately, there's not enough of them to cover all the things involved in X: input devices, graphics devices, new protocol extension specification and so on. The X server is really late, and it seems most of the developers prefers to work on the server itself than on the protocol behalf. Which is understandable. I'm curious to see where all of that will lead in the upcoming years. I've been walking in the X world hallways for about 3 years now, and I feel desktop alternatives to KDE and GNOME will all die sooner or later. The time were you could choose between a dozen "modern" window managers has passed away. After all, maybe that is simply Darwinism applied to computer software.

31 May 2010

Dirk Eddelbuettel: JPM Chase Corporate Challenge 2010

It's Memorial Day weekend so it was time for the Chicago's JP Morgan Chase Corporate Challenge on Thursday. The weather was glorious, the usual 20-some thousand runners participated and a good time was had. Work had arranged for a nice tent, food, music --- and a bunch of people showed up and enjoyed it. Nice one. This time we all got chip-timing via a small (rfid ?) strip tagged to back of the bib number. Which is handy as I managed to not stop my time by hand correctly. Given that I am still nursing a sore Achilles tendon and don't train well or much, the time of 23:51 (or 6:48 min/mile) was ok compared to the other seven previous times I have run this.

Matt Zimmerman: Summary of development plans for Ubuntu 10.10

With the 10.10 developer summit behind us, several teams have published engineering plans for the 10.10 release cycle, including:

30 May 2010

Erich Schubert: BPM tap toy update

I've updated my BPM tap toy, which started behaving strangely since some GTK update. I've identified two issues, and it is working again now.The BPM tap toy is a tiny Python script to obtain a BPM estimation by tapping on your mouse or keyboard. In contrast to many other such applications it does give you an error estimation and will visualize your precision in tapping. This way, you can judge how precise the value is.I have some ideas to further increase precision while adding another useful visualization to judge the result quality.

29 May 2010

Lucas Nussbaum: Ubuntu and Debian

I recently noticed the new Ubuntu and Debian link on every page of http://www.ubuntu.com/. That is very nice, and much better than the previous situation. Debian is also mentioned on the pages about Easy administration and Security. Canonical has clearly improved a lot in the way they communicate about how Ubuntu bases its development on Debian s.

28 May 2010

Ana Beatriz Guerrero Lopez: KOffice 2.2.0 packages for Debian

As some of you might have seen, KOffice 2.2.0 was finally released today. Quoting the release announcement: There are still areas in the user interface that we want to work on before stating that KOffice is fit for the general public. We are, however, at a stage where we think that KOffice can be used for real work by some users.
I agree with this statement and it is still to be decided if KOffice 2.2.0 will make it into Squeeze. I am inclined to do so but I would love some well argumented feedback. Take into consideration some applications are more mature than others. The packages will be uploaded to the official archive soon, but they still need some minor work and given they include new deb files (kformula shape and kexi), they need to go thru the NEW queue, it might take some time. Meanwhile, you can find packages for amd64 & i386 from the semi-official repository at: http://qt-kde.debian.net/.
You also have available the localization packages (see the list here) and braindump, an interesting application based in KOffice libraries. These packages are not in the Debian archive so please *don t* report bugs against them in the Debian Bug Tracking System. If you see any packaging problem, please report it in the debian-kde mailing list.
If you find a bug, miss a feature or have a wishlist, please file a report in the KDE bugzilla: https://bugs.kde.org/ Include all the information you can about your system (Debian experimental, KOffice 2.2.0) and all the details you are able to give. In the case of crashes, install the package koffice-dbg to get a backtrace. Read more at this techbase article.

27 May 2010

C.J. Adams-Collier: Building libvirt on CentOS5 (incomplete)

Much thanks to Brett for the pointers on rpm-fu. http://grantmcwilliams.com/index.php?option=com_content&view=article&id=229: $ sudo yum install \
xen-devel \
xhtml1-dtds \
hal-devel \
libpcap-devel \
cyrus-sasl-devel \
parted-devel \
numactl-devel \
avahi-devel \
slang-devel \
libvolume_id-devel \
openldap-devel # device-mapper-devel \
# xmlrpc-c-devel \ for pkg in \
libssh2-devel-1.2.5-1.el5.pp.x86_64.rpm \
libssh2-1.2.5-1.el5.pp.x86_64.rpm \
libssh-0.2.1-0.2.svn193.el5.pp.x86_64.rpm \
libssh-devel-0.2.1-0.2.svn193.el5.pp.x86_64.rpm
do
wget ftp://ftp.pbone.net/mirror/ftp.pramberger.at/systems/linux/contrib/rhel5/x86_64/$pkg
rpm -i $pkg
done wget http://www.fateyev.com/RPMS/RHEL5/x86_64/xmlrpc-c-devel-1.14.8-1.el5.x86_64.rpm
wget http://www.fateyev.com/RPMS/RHEL5/x86_64/xmlrpc-c-1.14.8-1.el5.x86_64.rpm
rpm -i xmlrpc-c-*1.14*el5*.rpm wget ftp://ftp.icm.edu.pl/vol/rzm1/linux-fedora-secondary/development/source/SRPMS/corosync-0.95-2.fc11.src.rpm
alien -t corosync-0.95-2.fc11.src.rpm
mv corosync-0.95.tgz /tmp
cd ~/rpm/SOURCES
tar xfz /tmp/corosync-0.95.tgz
patch -p0
--- corosync.spec.orig  2010-05-26 19:17:15.000000000 +0000
+++ corosync.spec       2010-05-26 19:21:39.000000000 +0000
@@ -39,11 +39,7 @@
 fi
 %endif
-% _configure   CFLAGS="$(echo '% optflags ')" \
-               --prefix=/usr \
-               --sysconfdir=/etc \
-               --localstatedir=/var \
-               --libdir=% _libdir 
+% configure    CFLAGS="$(echo '% optflags ')"
 %build
 make % _smp_mflags 
@@ -52,8 +48,8 @@
 rm -rf % buildroot 
 make install DESTDIR=% buildroot 
-install -d % buildroot % _initddir 
-install -m 755 init/redhat % buildroot % _initddir /corosync
+install -d % buildroot % _sysconfdir /init.d
+install -m 755 init/redhat % buildroot % _sysconfdir /init.d/corosync
 ## tree fixup
 # drop static libs
@@ -95,7 +91,7 @@
 % _sbindir /corosync-fplay
 % _sbindir /corosync-pload
 %config(noreplace) /etc/corosync.conf
-% _initddir /corosync
+% _sysconfdir /init.d/corosync
 %dir % _libexecdir /lcrso
 % _libexecdir /lcrso/coroparse.lcrso
 % _libexecdir /lcrso/objdb.lcrso
rpmbuild -bb corosync.spec
sudo rpm -i ~/rpm/RPMS/x86_64/corosynclib*.rpm wget ftp://ftp.icm.edu.pl/vol/rzm1/linux-fedora-secondary/development/source/SRPMS/openais-0.94-1.fc11.src.rpm
alien -t openais-0.94-1.fc11.src.rpm
mv openais-0.94.tgz /tmp
cd ~/rpm/SOURCES
tar xfz /tmp/openais-0.94.tgz
rpmbuild -bb openais.spec
sudo rpm -i ~/rpm/RPMS/x86_64/openaislib-*.rpm wget ftp://ftp.icm.edu.pl/vol/rzm1/linux-fedora-secondary/development/source/SRPMS/lvm2-2.02.45-4.fc11.src.rpm
alien -t lvm2-2.02.45-4.fc11.src.rpm
mv lvm2-2.02.45.tgz /tmp
cd ~/rpm/SOURCES
tar xfz /tmp/lvm2-2.02.45.tgz
rpmbuild -bb lvm2.spec

Matt Zimmerman: Rethinking the Ubuntu Developer Summit

This is a repost from the ubuntu-devel mailing list, where there is probably some discussion happening by now. After each UDS, the organizers evaluate the event and consider how it could be further improved in the future. As a result of this process, the format of UDS has evolved considerably, as it has grown from a smallish informal gathering to a highly structured matrix of hundreds of 45-to-60-minute sessions with sophisticated audiovisual facilities. If you participated in UDS 10.10 (locally or online), you have hopefully already completed the online survey, which is an important part of this evaluation process. A survey can t tell the whole story, though, so I would also like to start a more free-form discussion here among Ubuntu developers as well. I have some thoughts I d like to share, and I m interested in your perspectives as well. Purpose The core purpose of UDS has always been to help Ubuntu developers to explore, refine and share their plans for the subsequent release. It has expanded over the years to include all kinds of contributors, not only developers, but the principle remains the same. We arrive at UDS with goals, desires and ideas, and leave with a plan of action which guides our work for the rest of the cycle. The status quo UDS looks like this: This screenshot is only 1600 1200, so there are another 5 columns off the right edge of the screen for a total of 18 rooms. With 7 time slots per day over 5 days, there are over 500 blocks in the schedule grid. 9 tracks are scattered over the grid. We produce hundreds of blueprints representing projects we would like to work on. It is an impressive achievement to pull this event together every six months, and the organizers work very hard at it. We accomplish a great deal at every UDS, and should feel good about that. We must also constantly evaluate how well it is working, and make adjustments to accommodate growth and change in the project. How did we get here? (this is all from memory, but it should be sufficiently accurate to have this discussion) In the beginning, before it was even called UDS, we worked from a rough agenda, adding items as they came up, and ticking them off as we finished talking about them. Ad hoc methods worked pretty well at this scale. As the event grew, and we held more and more discussions in parallel, it was hard to keep track of who was where, and we started to run into contention. Ubuntu and Launchpad were planning their upcoming work together at the same time. One group would be discussing topic A, and find that they needed the participation of person X, who was already involved in another discussion on topic B. The A group would either block, or go ahead without the benefit of person X, neither of which was seen to be very effective. By the end of the week, everyone was mentally and physically exhausted, and many were ill. As a result, we decided to adopt a schedule grid, and ensure that nobody was expected to be in two places at once. Our productivity depended on getting precisely the right people face to face to tackle the technical challenges we faced. This meant deciding in advance who should be present in each session, and laying out the schedule to satisfy these constraints. New sessions were being added all the time, so the UDS organizers would stay up late at night during the event, creating the schedule grid for the next day. In the morning, over breakfast, everyone would tell them about errors, and request revisions to the schedule. Revisions to the schedule were painful, because we had to re-check all of the constraints by hand. So, in the geek spirit, we developed a program which would read in the constraints and generate an error-free schedule. The UDS organizers ran this at the end of each day during the event, checked it over, and posted it. In the morning, over breakfast, everyone would tell them about constraints they hadn t been aware of, and request revisions to the schedule. Revisions to the schedule were painful, because a single changed constraint would completely rearrange the schedule. People found themselves running all over the place to different rooms throughout the day, as they were scheduled into many different meetings back-to-back. At around this point, UDS had become too big, and had too many constraints, to plan on the fly (unconference style). We resolved to plan more in advance, and agree on the scheduling constraints ahead of time. We divided the event into tracks, and placed each track in its own room. Most participants could stay in one place throughout the day, taking part in a series of related meetings except where they were specifically needed in an adjacent track. We created the schedule through a combination of manual and automatic methods, so that scheduling constraints could be checked quickly, but a human could decide how to resolve conflicts. There was time to review the schedule before the start of the event, to identify and fix problems. Revisions to the schedule during the event were fewer and less painful. We added keynote presentations, to provide opportunities to communicate important information to everyone, and ease back into meetings after lunch. Everyone was still exhausted and/or ill, and tiredness took its toll on the quality of discussion, particularly toward the end of the week. Concerns were raised that people weren t participating enough, and might stay on in the same room passively when they might be better able to contribute to a different session happening elsewhere. As a result, the schedule was randomly rearranged so that related sessions would not be held in the same room, and everyone would get up and move at the end of each hour. This brings us roughly to where things stand today. Problems with the status quo
  1. UDS is big and complex. Creating and maintaining the schedule is a lot of work in itself, and this large format requires a large venue, which in turn requires more planning and logistical work (not to mention cost). This is only worthwhile if we get proportionally more benefit out of the event itself.
  2. UDS produces many more blueprints than we need for a cycle. While some of these represent an explicit decision not to pursue a project, most of them are set aside simply because we can t fit them in. We have the capacity to implement over 100 blueprints per cycle, but we have *thousands* of blueprints registered today. We finished less than half of the blueprints we registered for 10.04. This means that we re spending a lot of time at UDS talking about things which can t get done that cycle (and may never get done).
  3. UDS is (still) exhausting. While we should work hard, and a level of intensity helps to energize us, I think it s a bit too much. Sessions later in the week are substantially more sluggish than early on, and don t get the full benefit of the minds we ve brought together. I believe that such an intense format does not suit the type of work being done at the event, which should be more creative and energetic.
  4. The format of UDS is optimized for short discussions (as many as we can fit into the grid). This is good for many technical decisions, but does not lend itself as well to generating new ideas, deeply exploring a topic, building broad consensus or tackling big picture issues facing the project. These deeper problems sometimes require more time. They also benefit tremendously from face-to-face interaction, so UDS is our best opportunity to work on them, and we should take advantage of it.
  5. UDS sessions aim for the minimum level of participation necessary, so that we can carry on many sessions in parallel: we ask, who do we need in order to discuss this topic? This is appropriate for many meetings. However, some would benefit greatly from broader participation, especially from multiple teams. We don t always know in advance where a transformative idea will come from, and having more points of view represented would be valuable for many UDS topics.
  6. UDS only happens once per cycle, but design and planning need to continue throughout the cycle. We can t design everything up front, and there is a lot of information we don t have at the beginning. We should aim to use our time at UDS to greatest effect, but also make time to continue this work during the development cycle. design a little, build a little, test a little, fly a little
Proposals
  1. Concentrate on the projects we can complete in the upcoming cycle. If we aren t going to have time to implement something until the next cycle, the blueprint can usually be deferred to the next cycle as well. By producing only moderately more blueprints than we need, we can reduce the complexity of the event, avoid waste, prepare better, and put most of our energy into the blueprints we intend to use in the near future.
  2. Group related sessions into clusters, and work on them together, with a similar group of people. By switching context less often, we can more easily stay engaged, get less fatigued, and make meaningful connections between related topics.
  3. Organize for cross-team participation, rather than dividing teams into tracks. A given session may relate to a Desktop Edition feature, but depends on contributions from more than just the Desktop Team. There is a lot of design going on at UDS outside of the Design track. By working together to achieve common goals, we can more easily anticipate problems, benefit from diverse points of view, and help each other more throughout the cycle.
  4. Build in opportunities to work on deeper problems, during longer blocks of time. As a platform, Ubuntu exists within a complex ecosystem, and we need to spend time together understanding where we are and where we are going. As a project, we have grown rapidly, and need to regularly evaluate how we are working and how we can improve. This means considering more than just blueprints, and sometimes taking more than an hour to cover a topic.

26 May 2010

Michal &#268;iha&#345;: Great things on providing web based translation

Quite recently (just when migrating to gettext), it is possible to translate phpMyAdmin using web interface, which is powered by great software called Pootle. Thanks to this new interface, we've attracted quite a lot of translators and lot of languages are now much improved. Yesterday I also implemented way to backport these translations to old PHP based translation system which is still being used in 3.3 branch, so that even upcoming releases from this branch will benefit from this translation boost. Yet another great think which happened thanks to l10n.cihar.com, is availability of German version of Wammu website, where enormous amount of work has happened in last few days. Kudos to all people who are working there.

Russell Coker: UBAC and SE Linux in Debian

A recent development in SE Linux policy is the concept of UBAC (User Based Access Control) which prevents SE Linux users (identitied) from accessing each other s files. SE Linux user identities may map 1:1 to Unix users (as was required in the early versions of SE Linux), you might have unique identities for special users and a default identity for all the other users, or you might have an identity per group or use some other method of assigning identities to groups. The UBAC constraints in the upstream reference policy prevent a process with a SE Linux identity other than system_u from accessing any files with an identity other than system_u. So basically any regular user can access files from the system but not other users and system processes (daemons) can access files from all users. Of course this is just one layer of protection, so while the UBAC constraint doesn t prevent a user from accessing any system files the domain-type access controls may do so. If you used a unique SE Linux identity for each Unix account then UBAC would prevent any user from accessing a file created by another user. For my current policy that I am considering uploading to Debian/Unstable I have allowed the identity unconfined_u to access files owned by all identities. This means that unconfined_u is an identity for administrators, if I proceed on this path then I will grant the same rights to sysadm_u. UBAC was not enabled in Fedora last time I checked, so I m wondering whether there is any point in including it I don t feel obliged to copy everything that Fedora does, but there is some benefit in maintaining compatibility across distributions. For protecting users from each other it seems that MCS (which is Mandatory in the Debian policy) is adequate. MCS allows a much better level of access control. For example I could assign categories c0 to c10 to a set of different projects and allow the person who manages all the projects to be assigned all those categories when they login. That user could then use the command runcon -l s0:c1 bash to start a shell for the purpose of managing files from project 1 and any file or process created by that command would have the category c1 and be prevented from writing to a file with a different category. Of course the down-side to removing UBAC is that since RBAC was removed there is no other way of separating SE Linux users, while MCS is good for what it does it wasn t designed for the purpose of isolating different types of user. So I ll really want to get RBAC reinstalled before Squeeze is released if I remove UBAC. Regardless of this I will need to get RBAC working on Squeeze eventually anyway. I ve had a SE Linux Play Machine running with every release of SE Linux for the last 8 years and I don t plan to stop now.

25 May 2010

Russell Coker: GM Food and Vaccines

Michael Specter gave an interesting TED talk about the dangers of science-denial [1]. Most of his talk is about the people who oppose vaccines, such as the former Playboy model Jenny McCarthy who thinks that she knows more about medicine than people who do medical research. He notes that a doctor who advocates vaccination has been receiving threats from the anti-vaccine lobby, including threats to his children. An good new development is that Andrew Wakefield (the British ex-Doctor behind the discredited research linking Autism and Vaccination) has been barred from practicing by Britain s General Medical Council [2]. Michael also mentions the opposition to GM food which has the potential to save many lives in developing countries that have food shortages. This convinced me to reduce my opposition to GM food, it s really not GM food that I m opposed to but the poor testing, the bad features (such as the Terminator Gene), and the Intellectual Property controls which allow GM companies to sue farmers who accidentally have GM crops grow on their land due to wind-borne seeds. It s also a pity that there is no work being done on GM versions of any food crop which is only used for feeding poor people. Every GM plant is one that is used to provide food for rich people and is essentially a way for farmers in first-world countries to make more money. But GM versions of Cassava (with less of the toxic chemicals among other things) and Sorghum would improve the situation of many poor people. One interesting related development is that Craig Venter has just announced the creation of the first synthetic life [3]. This technical development could lead to dramatic changes in the production of basic foods, such as algae that produce proteins that have the ideal mixture of all the essential amino acids needed for humans as well as the semi-essential ones that children need. While feeding pond slime to children isn t going to be glamorous it would be a lot better than the current situation where a significant number of children in developing countries have their physical and mental development stunted due to malnutrition. Craig mentions the possibility of using his research to develop vaccines much faster, including perhaps the possibility of vaccinating people against fast evolving viruses such as the common cold!

22 May 2010

Petter Reinholdtsen: More flexible firmware handling in debian-installer

After a long break from debian-installer development, I finally found time today to return to the project. Having to spend less time working dependency based boot in debian, as it is almost complete now, definitely helped freeing some time. A while back, I ran into a problem while working on Debian Edu. We include some firmware packages on the Debian Edu CDs, those needed to get disk and network controllers working. Without having these firmware packages available during installation, it is impossible to install Debian Edu on the given machine, and because our target group are non-technical people, asking them to provide firmware packages on an external medium is a support pain. Initially, I expected it to be enough to include the firmware packages on the CD to get debian-installer to find and use them. This proved to be wrong. Next, I hoped it was enough to symlink the relevant firmware packages to some useful location on the CD (tried /cdrom/ and /cdrom/firmware/). This also proved to not work, and at this point I found time to look at the debian-installer code to figure out what was going to work. The firmware loading code is in the hw-detect package, and a closer look revealed that it would only look for firmware packages outside the installation media, so the CD was never checked for firmware packages. It would only check USB sticks, floppies and other "external" media devices. Today I changed it to also look in the /cdrom/firmware/ directory on the mounted CD or DVD, which should solve the problem I ran into with Debian edu. I also changed it to look in /firmware/, to make sure the installer also find firmware provided in the initrd when booting the installer via PXE, to allow us to provide the same feature in the PXE setup included in Debian Edu. To make sure firmware deb packages with a license questions are not activated without asking if the license is accepted, I extended hw-detect to look for preinst scripts in the firmware packages, and run these before activating the firmware during installation. The license question is asked using debconf in the preinst, so this should solve the issue for the firmware packages I have looked at so far. If you want to discuss the details of these features, please contact us on debian-boot@lists.debian.org.

20 May 2010

Kees Cook: openoffice presenter console

During the Lucid development sprint, Pete showed me the glory that is the OpenOffice presenter console. Since then I ve switched laptops, and I figured I need to document for myself, and maybe others, how to have your OOo presentation on one screen (e.g. the projector) and the presentation notes, clock, etc, on another (e.g. the main laptop display). First, install openoffice.org-presenter-console , and once you have your displays configured, launch OpenOffice and select Slide Show / Slide Show Settings . From here, you can configure the displays under Multiple displays . So nice. :) UPDATE: corrected my typo. thank you! :)

Next.

Previous.