Yesterday morning, Romain pushed inline release 0.3.5 to CRAN. This is some ways a continuation of the 0.3.4 release I had made in December. That release had opened the door for the wide use of inline in our Rcpp package. And just how Rcpp has grown, we now have needs beyond the initial change. See the post on Romain's blog for details, but in a nutshell we are now gaining

• cxxfunction which extends cfunction further for C++ use and, among other things, adds a plugin system we can use from RcppArmadillo to permit use of inline
• package.skeleton which makes it easy to carry a function that one has prototyped with inline over into its own package -- and how to do that was a question at my most recent Rcpp talk in Vienna), and
• getDynLib which Romain will use to great effect in the next version of Rcpp to provide something not unlike Boost::Python. Stay tuned!

Last but not least, our thanks to Oleg Sklyar for letting us extend his amazing inline package for use by Rcpp.

For work recently I've been doing some Django-related tasks that involve talking to an external API with POSTed forms. Django forms objects are declared by creating a class that inherits from django.forms.Form, with the fields of the form declared by declaring attributes of that class. Which works well and is clean and easy to remember unless the API you're working with requires a field with the same name as a Python keyword, such as return. You can't declare a field like this as an attribute; it will trigger a syntax error. I spent some time scratching my head over this, and came up with this as a workaround after source-diving to find out how Form objects actually work:

from django import forms
class ExampleForm(forms.Form):
    def __init__(self, data=None, files=None, auto_id='id_%s', prefix=None,
            initial=None, errorclass=ErrorList, label_suffix=':', empty_permitted=False, return_url=None):
        forms.Form.__init__(self, data, files, auto_id, prefix, initial,
            errorclass, label_suffix, empty_permitted)
        if return_url is not None:
            self.fields['return'] = forms.CharField(widget=forms.HiddenInput, initial=return_url)

It turns out that the attribute declaration is just syntactic sugar for creating a dictionary of key/value pairs, which is then stored in the fields attribute. So we can monkeypatch in extra values after the translation. Which is somewhat more awkward and ugly, but works in a pinch. Note that I haven't extensively tested what interactions this may cause with other forms code, so use with some caution.

Since the recent announce of http://ubuntudiff.debian.net, some people started using it and asked for a few features. Lately, I ve been working on it and tried to implement the following:

• Use a single page (instead of a small HTML page with a lot of JS) well, that was easy
• Show the debdiff between the Ubuntu package and the Debian one: that was simpler than I first thought. Right now, it s implemented and deployed [1]. When I first generated all patches, I realized how people can be crazy I had diffs larger than 60MB. So, I had to reduce that by keeping only diffs less than 200 lines per file. At the end, I had a little more than 2GB of patches (and html files) generated for ~2000 source packages (which is somehow reasonable). So now, below the changelog, there is a list of modified files and you can:
  • download a patch for each file, extracted from the debdiff,
  • or click on the file s name to see the diff (if not too large, i.e. less than 200 lines)
  Note that red colored filenames denote files with too large diffs.

I ll still have some details to fix like putting the download patch link on the left, fix the show/hide thing which is also activated when you click on download patch . Then, I ll consider it feature-complete (almost) and won t touch it again. My next game will be to write some tools to analyse or detect new transitions if you want to play, let me know As always, please test this new beta version of UbuntuDiff (and enjoy it :p) and maybe, you may also report errors/bugs, if any [1] http://ubuntudiff.debian.net/beta/

Would it be wrong to make a new blog entry just to test the blogging platform? If so I'm wrong. To pretend that isn't the case I'll briefly document some recent events:

Coding

I've been making changes to the blogging software, chronicle, to support RSS feeds on comments, and to allow outgoing Pings on new entries. I've also been flirting with an OpenID-backed image upload and sharing site, provisionally named picshare. Why? Partly for fun, partly because I like comments, and partly to flirt with RESTful design and API construction.

Photography

I've provisionally agreed to be the photographer for a special event at a local hair-dressers. This will either be fun, or all end badly. No money, but free food and the chance to experiment more. So all good.

There. Almost like a real entry, right? ObSubject: Cruel Intentions

submit your own dependency-resolution issue, a-la popcon [ repost from the original, with updates ] During the question time of my FOSDEM 2010 talk about cross-distro dependency resolution several interested people, mostly sysadms, asked how they could contribute upgrade scenarios to the Mancoosi project. The underlying idea is simple: we are working on improving dependency resolution techniques, algorithms, and tools; to that end we need a corpus of upgrade scenarios where shortcomings of state of the art tools can be observed. We already have quite some of such scenarios (partly provided by members of the project, partly generated in vitro, etc.), but we could use more. So, I'm hereby happy to announce the public availability of the mancoosi-contest package, for Debian-based distributions. mancoosi-contest essentially offers a wrapper, called dudf-save, that you can prepend to usual invocations of package managers such as apt-get and aptitude. What it does is to capture all package meta-data concerning your request, as well as the request itself and the package manager output. All these data are first compacted using snapshot.d.o hashes, then collected into a DUDF document, and finally (optionally) uploaded to a DUDF collector service. All in all it is very similar to the popularity-contest architecture, but instead of collecting installed packages it collects specific upgrade attempts. The similarity of course extends to privacy concerns, which we do take very seriously (censoring IP addresses and offering all the software we write and run available under FOSS licenses for scrutiny). Obviously, participation is completely optional and the mancoosi-contest package is currently being provided only unofficially by the Mancoosi project (even though it is signed by my key, in case you want a trust path from the Debian keyring). All that said, if, one day, you feel like having stumbled upon a challenging upgrade scenario, please consider contributing it using mancoosi-contest. We, for once, would be grateful , and you will be helping in improving the state of the art in dependency solving. Some of the most interesting upgrade scenarios already received are currently being used as material for the ongoing MISC 2010 competition, which we plan to repeat in the future. Technically, dudf-save is still quite hackish and while apt-get support is OK-ish, aptitude support is sub-optimal. In particular it will work properly only for "completely batch" aptitude sessions (or, better, it will not catch any intermediate solving attempt that aptitude goes through when interactively asking questions to the user). You can get mancoosi-contest from mancoosi.debian.net, i.e. simply doing apt-get install mancoosi-contest after having added the following APT repository to your sources.list:

    deb http://mancoosi.debian.net/debian/ unstable/

For the time being, bug reports can be sent to me.

It seems I ve missed quite a few weeks in my countdown to our Mini DebConf. Fortunately, it s not too late for you to join even if, right now, you don t know why you should! So what s the status? Well, we ve had out problems in the beginning to start the whole thing but we managed. There are a few minor issues left, I admit. If you happen to have a bit time on 10th or 11th to hold the camera for an hour or so, contact us! Or if there is *anything* you would wanna do to help us out a bit, don t be shy. You don t need to be a known Debian contributor, you don t need to know us. Arguably, you should know Debian but then, if you re able to spell the name correctly, you re probably in. :-) Apart from that, we really think it s gonna be awesome. Not only will there be a few Debian Developers from outside Germany, like our project leader zack who we have the honor of listening to for he has the first talk on Thursday. There will also be quite a few German faces that some of us haven t met yet have you met Sebastian, the chief of our beloved debianforum.de, yet? We are going to have a great time and you re a fool not to join. Thus, a last time for all of you who still don t know what I m talking about: Next week there ll be the famous LinuxTag in Berlin from Wednesday to Saturday. Right in the middle, on Thursday and Friday, we, the Debian community, will have a Mini DebConf right there. There will be talks about different Debian related things. And many active developers will be there happy to talk to you about whatever you always wanted to know or say about Debian. And then, not to forget, we ll have a hacklab where you can easily plug in your notebook and start hacking with us. Have an idea to improve Debian? Always wanted to know how to fix a bug? You want to make Debian Squeeze the best Debian release ever? Join us it seems we re fighting on the same side. See you there, guys! PS: The latest and hottest news about it.

After xen-tools had been removed from Debian Unstable by the upstream developer last December due to no more upstream development, I took over upstream development as well as Debian package maintainership in January. Since then a lot happened:

• Development moved from Mercurial to a set of git repositories at gitorious.org.
• Dmitry Nedospasov joined my effort to revive xen-tools.
• Via gitorious clone tracking we noticed that there were useful patches in clones of xen-tools and incorporated most of them back into our master branch.
• We created a second mailing list xen-tools-dev which is mainly thought for communication between the xen-tools developers.
• The xen-tools.org website and the xen-tools mailing lists moved from Steve s to my server. The website has been slightly overhauled and the mailing lists now use Mailman.
• Support for all recent Debian and Ubuntu releases including Squeeze and Lucid.
• Full support for pygrub and DomU distribution kernels installed in the DomU. Now you easily can install DomUs whose udev doesn t fit to your Dom0 kernel.
• Full support for cdebootstrap in addition to debootstrap.
• Tons of bugs fixed
• I m on the quest to find the holy grail of bug tracking systems. ;-) More on this in another blog posting.

But despite all the things which were to fix after Steve discontinued the xen-tools developement, Steve Kemp did a great job to bring xen-tools so far initially. I m really happy that this software exists and that s the reason why I won t let it die so fast. :-) xen-tools 4.2 beta 1 released Last weekend I released xen-tools 4.2 beta 1 which I also uploaded to Debian Unstable. And thanks to the active ftp-master team, xen-tools went through the NEW queue in less than 12 hours and is now available again in Debian Unstable. 4.2 beta 1 still has some known bugs, the worst ones are in xen-delete-image. But no fear, it doesn t delete more than wanted, just not enough or things which don t exist anyway. :-) There are also some smaller issues to fix for a final 4.2 release. We may also create an PPA for Ubuntu Lucid in the future to work around the lack of xen-tools in Ubuntu Lucid. What about Fedora/RedHat/CentOS support? Since rinse is out of date with regards to current Fedora releases, rpmstrap seems to be out of date for even longer (it has been removed from Debian before the Lenny release) and febootstrap neither seems to work out of the box nor seems to support bootstrapping to mount points (because it wants to delete them first and fails), bootstrapping Fedora ist currently not supported. The copy installation method should work though. rinse still is able to bootstrap CentOS 4 and 5, but the installation seems incomplete. We ll try to work around these issues for the final 4.2 release. We haven t looked at mach and mock so far, but according to Lucas, both are out of date, too. Sure, implementing Lucas howto in form of a script would be a possible way, but we think it would just create yet another not so well supported clone of rinse, rpmstrap, mach, mock or febootstrap. Because of that, Dmitry works on support for the installation of Xen DomUs via kickstart. Fate of xen-shell? Xen-shell has also been removed from Debian Unstable last December because Steve stopped its developement, too. But in comparison to xen-tools itself, we didn t resurrect xen-shell since none of us currently has use for it. So if you are currently using xen-shell and intend to continue to do so, you should hurry up if you want to see it in Debian Squeeze. We can also help you, but we surely can t and won t do it alone nor will we take the majority of the necessary workload.

I was asked by email whether SE Linux could implement traditional Unix users and groups. The Strictly Literal Answer to that Question The core of the SE Linux access control is the domain-type model where every process has a domain and every object that a process can access (including other processes) has a type. Domains are not strongly differentiated from types. It would be possible to create a SE Linux policy that created a domain for every combination of UID and GID that is valid for a user shell given that such combinations are chosen by the sysadmin who could limit them to some degree. There are about 2^32 possible UIDs and about 2^32 possible GIDs, as every domain is listed in kernel memory we obviously can t have 2^64 domains, but we could have enough to map a typical system that s in use. Of course the possible combinations of supplemental groups probably makes this impossible for even relatively small systems, but we can use a simpler model that doesn t emulate supplemental groups. For files there are more possible combinations because anyone who is a member of a group can create a SETGID directory and let other users create files in it. But in a typical system the number of groups is not much greater than the number of users the maximum number of groups is typically the number of users plus about 60. So if we had 100 users then the number of combinations of UID and GID would be something like 100*(100+60)=16,000 it should be possible to have that many domains in a SE Linux policy (but not desirable). Then all that would be needed is rules specifying that each domain (which is based on a combination of UID and GID) can have certain types of access to certain other types based on them having either the same UID or the same GID. Such a policy would be large, it would waste a lot of kernel memory, it would need to be regenerated whenever a user is added, and it s generally something you don t want to use. No-one has considered implementing such a policy, I merely describe it to illustrate why certain configuration options are not desirable. The rest of this post is about realistic things that you can do with SE Linux policy and how it will be implemented in Debian/Squeeze. My previous post titled Is SE Linux Unixish? addresses this issue at a more conceptual level and also considers MAC vs DAC [1]. The History of mapping Unix Accounts to SE Linux Access Control In the early releases of SE Linux (long before it was included in Fedora) every user who could login to a system needed to have their user-name compiled into the policy. The policy specified which roles the user could access, the roles specified which domains could be accessed, and therefore what the user could do. The identity of files on disk was used for two purposes, one was logging (you could see who created a file) and the other was a permission check for the SE Linux patched version of Vixie cron which would not execute any command on behalf of a user unless the identity on the crontab file in the cron spool matched the identity used to run it this is an analogy of the checks that Vixie cron makes on the Unix UID of the crontab file (some other cron daemons do fewer checks). Having to recompile policy source every time you added a user was annoying. So a later development was to allow arbitrary mappings between Unix account names and SE Linux Identities (which included a default identity) and another later development was to have a utility program semanage to map particular Unix account and group names to SE Linux identities. This was all done years ago. Fedora Core 5 which was released in 2006 had the modular policy which included these features (apart from mapping Unix groups to SE Linux identities which was more recent). Fedora Core 5 also introduced MCS which was comprised of a set of categories that a security context may have. The sysadmin would configure the set of categories that each account would have. A recent development has been a concept named UBAC (User Based Access Control) which basically means that a process running directly on behalf of a regular user (IE with a SE Linux identity that s not system_u) can only access files that have an identity of system_u or which have the same identity as the process. This means that you can only access your own files or system files not files of other users which may have inappropriate Unix permissions. So for example if a user with a SE Linux identity of john gives their home directory the Unix permission mode of 0777 then a user with a SE Linux identity of jane can t access their files. Of course this means that if you have a group of people working together on a project then they probably need to all have the same Identity and in practice you would probably end up with everyone having the same identity. I ve given up on the idea of using UBAC in Debian. The Current Plan for Users and SE Linux Access Control in Debian My plan is to have things work in Squeeze in much the same way as in Lenny. You have a SE Linux identity assigned to a login session and everything related to it (including cron jobs) based on the Unix account name or possibly the Unix group name (if there are login entries for both the user-name and the group-name then the user-name entry has precedence). The mapping between Unix accounts and SE Linux identities is configured by the sysadmin and SE Linux identities don t matter much for the Targeted configuration (which is what most people use). The identity determines which roles may be used and also has a limit on the MCS categories. The MCS categories are also specified in the login configuration which has to be a sub-set of the categories used by the identity record. So for example the following is the output of a couple of commands run on a Debian/Unstable system. They show that the test Unix account is assigned a SE Linux identity of staff_u and an MCS range of s0-s0:c1 (this means it creates files by default at level s0 and can also write to other files at that level, but can also have read/write access to files at the level s0:c1 ). The staff_u identity (as shown in the output of semanage user -l can be used with all categories in the set s0:c0.c1023 where the dot means the set of categories from c0 to c1023 inclusive) but in the case of the test user only one category will be used. The test group however (as expressed with %test ) is given the identity user_u and is not permitted to use any categories.

# semanage login -l
Login Name    SELinux User    MLS/MCS Range           
%test         user_u          s0                     
__default__   unconfined_u    s0-s0:c0.c1023         
root          unconfined_u    s0-s0:c0.c1023         
system_u      system_u        s0-s0:c0.c1023         
test          staff_u         s0-s0:c1               
# semanage user -l
             Labeling   MLS/       MLS/                         
SELinux User Prefix     MCS Level  MCS Range         SELinux Roles
root         sysadm     s0         s0-s0:c0.c1023    staff_r sysadm_r system_r
staff_u      staff      s0         s0-s0:c0.c1023    staff_r sysadm_r
sysadm_u     sysadm     s0         s0-s0:c0.c1023    sysadm_r
system_u     user       s0         s0-s0:c0.c1023    system_r
unconfined_u unconfined s0         s0-s0:c0.c1023    system_r unconfined_r
user_u       user       s0         s0                user_r

I hope to get the policy written to support multiple user roles in time for the release of Squeeze. If I don t make it then I will put such a policy on my own web site and try to get it included in an update. The policy currently basically works for a Targeted configuration where the users are not confined (apart from MCS). How MMCS Basically Works The vast majority of SE Linux users run with the MCS policy rather than the MLS policy. For Debian I have written a modified version of MCS that I call MMCS. MMCS is mandatory (you can t relabel files to escape it) and it prevents write-down. If a process has the range s0-s0:c1,c3 then it has full access to files labelled as s0, s0:c1, s0:c3, and s0:c1,c3 and any files it creates will be labeled as s0. If a process has the range s0:c1-s0:c1,c3 then it has read-only access to files labelled as s0 and s0:c3 and read-write access to files labelled as s0:c1 and s0:c1,c3. This means that any secret data it accesses that was labelled with category c1 can t be leaked down to a file that is not labelled with that category. Now MCS currently has no network access controls, so there s nothing stopping a user from using scp or other network utilities to transfer files. But that s the way with most usable systems. I don t think that this is necessarily a problem, the almost total lack of network access controls in a traditional Unix model doesn t seem to concern most people. Now to REALLY Answer that Question SE Linux is a Mandatory Access Control (MAC) system, this makes it inherently different to a Discretionary Access Control (DAC) system such as traditional Unix access controls. Unix permissions are based on each file having a UID, a GID, and a set of permissions and each process having a UID, a GID, and a set of supplementary GIDs. If a user runs a setuid or setgid program then the process will have extra privileges. It also has a lot of stuff that most people aren t aware of such as real vs effective UIDs, the tag bit, setgid directories, and lots more including some quite arbitrary things like making ports <1024 special. SE Linux is based on every object (process, file, socket, etc) having a single security label which includes an identity, a role, a type, and a sensitivity label (MCS categories or an MLS range). There is no support for an object to have more than one label. The SE Linux equivalent to setuid/setgid files is a label for a file which triggers a domain transition when it s executed. This differs from setuid files in that the domain transition is complete (the old privileges can t be restored) and the transition is generally not to a strict super-set of the access (usually a different sub-set of possible access and sometimes to lesser access). These differences are quite fundamental. So really SE Linux can t implement traditional Unix access control. What SE Linux is designed to do is to provide a second layer of defense and to also provide access controls that have different aims than that of Unix permissions such as being mandatory and implementing features such as MLS.

• [1] http://etbe.coker.com.au/2008/08/25/is-se-linux-unixish/

Moonshot is being discussed at the TERENA TNC 2010 conference. Our session started at 08:00 UTC (a few minutes ago), but will be going on for around the next hour or so. There is a presentation before Moonshot, but then Josh is up. See here for streaming and the Moonshot web site for our updated specifications. When the session is archived I ll post a pointer to the video stream.a

This should solve the yurt's bed problem. Over the past two years I've learned there are really two: It's not a fully dry environment so anything left on the floor will mildew; and the yurt makes no pretense of having a level floor. Thus this custom bed with independently levelable legs, custom sized to hold together a multipart matress. In keeping with the yurt thing, it's disassemblable and portable, but not too easily. :) Couldn't have been done without Mark's skilled ripping of the plywood. Anna cut the smaller boards and assisted, and we were very happy with what we managed to accomplish in a few hours. It's sturdy, and even level.

Two years ago, while working on awesome, I joined the Freedesktop initiative to work on XCB. I had to learn the arcane of the X11 protocol and all the mysterious and old world that goes with it. Now that I've swum all this months in this mud, I just feel like I need to share my thoughts about what become a mess over the decades. When I was unborn

the Toto band were releasing their song "Africa" and some smart guys were working on a windowing system: the X Window System (this is its full name) which therefore has a (too) long history. The latest version of its protocol, the 11th one, has been designed in the 80's. You can learn more about the history in the Wikipedia article about X. In 2010, we still listen disco music and we still use various protocols designed in the 80's and even before X. Music have evolved, protocols have evolved, and so did X11. The problem is that X11 did not evolve that well. The guys at MIT-and-some other-places-with-very-smart-people-in-it created X version 1 in 1984, and updated it until X version 11 (the one we're still using) in 1987. Eleven version in 3 years, that was following the "release early, release often" model. But I don't know why, it just stopped to happen for the last 23 years¹. I don't know what changes have been made in the first 11 major versions of the X protocol, but I'm rather sure we should have deserve a couple of major version updates this last 2 decades. In my humble opinion, X11 was not designed to live 23 years. But hey, I'm not blaming anyone here: I was 4 years old and playing Lego when they released this latest version of the X protocol, so there is little chance I'd have done something better.

1. That's not totally true: they added (and then deprecated) many extensions. We won't fix. We'll work-around.

That is probably one of the guideline of the X protocol for the last years. And don't misread me: I'm not bashing anyone thereafter. Since the X11 protocol was aging, the X guys started to add extensions. They added tons of them over the years. This, in application of one of the early principles of X:

It is as important to decide what a system is not as to decide what it is. Do not serve all the world's needs; rather, make the system extensible so that additional needs can be met in an upwardly compatible fashion.

All of them with no exception were added because, bad luck, the X11 protocol did not anticipated the things that happened in the last 23 years, like video, OpenGL, multiple monitors, or the pleasure to draw oval windows. Some of this extensions are still in use, while some of them have been dropped. While this is not a bad thing to extends the protocol, it seems like a bad thing to try to fix the protocol with for example the XFixes extension, even with all the good intentions Keith Packard might have in his greatness. Actually it's even worst than you think

The X11 protocol (without extensions) defines about 120 types of requests: create a window, move a window, etc. Nowadays, there's at least 25 % of them which are useless: usage of server-side font, or the drawing of squares and polygon, are unused by any modern application or toolkit. All of this is superseded by requests from extensions, like the XRender one. The handling of multiple monitors displays has totally been screwed up. X11 has been designed to work in Zaphod mode (independent monitors). But Xinerama, and nowadays XRandR have replaced it up: recent X servers (released after ~2007) does not support Zaphod mode anymore, even if it's a core piece of the X11 protocol. Worst: on many requests, there's limitation or design flaws, like described in this document: Why X Is Not Our Ideal Window System by DEC researchers. We'll add more broken standard on top of that

Following its early principle, X does not define policies but only mechanisms, which seems like a good thing, Consequently, people started writing specifications to determine a number of stuff and dogmas: ICCCM. That was 22 years ago in 1988. It's useless to add that many things in this specification are now obsolete, useless, or that it misses many modern stuff. I was not the only one to think that. The people from what will be the major desktop environments, KDE and GNOME, saw that too in the 90's while I was learning to count. So they wrote EWMH, another standard that comes on top of ICCCM and extends it with nifty features like maximization, full screen mode, etc. The problem is that this standard has also been written by narrow-minded people who at that time, were working on GNOME or KDE (and maybe others). This desktop environments were having and still have some strong concepts of how should work a desktop: "it should have work-spaces", "a window is only on one workspace", "we only see a workspace at a time", "you do not have multiple screens", etc. Dude, we don't care: we have toolkits!

This vision of how the desktop should work have now been written in marble in all applications and libraries implementing EWMH, like GTK+ or Qt. Nowadays, everybody forgot about all of this standards. Toolkits have implemented this, circumvented the X11 protocol limitation and flaws, and nobody wants to look back. Like all standards, obviously some people implemented them badly. This had some side effects, like OpenOffice acting like a pager. We don't look back? Worst, we forgot where we came from!

With all these layers of bad designed standards, the desktop continued to evolve for more than a decade. They continued to add more standard, the more recent ones being based on D-Bus like the Desktop Notification Specification or the latest Status Notifier Specification developed by KDE. The Status Notifier is a new implementation of the good old system tray based on XEmbed, using D-Bus instead of the X11 mechanisms, and adding the possibility to show the system tray with something else than icons. This specification draft saw an important issue and design flaw raised by Wolfgang Draxinger in this thread on the XDG mailing-list. What Wolfgang points out, is that X is network-oriented, and D-Bus is not. Therefore, making the Status Notifier specification to use D-Bus to pass system tray messages around is a bad idea, since running a X application from host A on host B will draw the system tray on the wrong host! Apparently, reading the thread, this does not fear some of the KDE people:

of course this is a bizarre corner case not worth much thought. at least that's what you'll think until you actually run into it yourself (be it because you are testing something or because you are setting up some weird kiosk environment).

What Oswald describes as a corner case is an actual common use case for many of us. Of course, YMMV. From my point of view, this is a step back in the wrong direction. But we can conclude that the network part of X is now worthless, to at least KDE. I used to believe in XCB

When I joined Freedesktop, it was to work on XCB, the X C Binding. XCB is a nice, clean, 21st century technology based API to play with the X11 protocol. Its code is auto generated based on XML file describing the protocol. In comparison, Xlib is 80's obfuscated code with almost no comments and hard-coded things. Only a few people can understand some of its corner like its i18n or XKB implementations. And all its code is synchronous. For people not knowing it yet, X is a network protocol where you send request (like a GET in HTTP) and then get a response. Xlib forces the application to wait for the reply to its request, so the application is blocked until the X server sends the reply to the request. XCB on the other hand does not block and allows the application to send a batch of requests, do some other stuff in the mean time, and then gets the replies. It's like your Web browser would send one request at a time to a Web server, and would wait until you downloaded all the images one by one to display the page. In cases where X and all its clients are on the same host, the latency is small and not really visible, therefore the gain for XCB to be asynchronous is small. On slow network however, the gain can be huge, as proved in the rewrite of xlsclients with XCB by Peter Harris. One of the long standing goal of the XCB folks is to kick-out Xlib, to increase speed and hides latency in X11 applications. That requires to port many libraries, because almost none of them (Cairo being an exception) supports XCB. From where I stand, I don't really see if the work is worth it now. The desktop world is trusted by GNOME and KDE, meaning GTK+ and Qt. It seems none of this toolkits are interested to work on XCB, neither on the X protocol. They probably put hard effort in bypassing X limitation and flaws, and they now sit on top of crap of workarounds and broken-by-design-standard implementation. It seems to me they don't want to go back in the layers and improves things. They're too high to go back down and they don't see what the gain would be. Enlightenment with its EFL was the first toolkit to have an XCB back-end with the work of Vincent Torri. Unfortunately, the back-end is not maintained and nobody cares about it. Last time I tried it, it did not compile at all. X12 ?

There's a page called X12 on the Freedesktop wiki, listing all the things that should be fixed some days. Unfortunately, the list continues to grow up an no one talks about working on X12. On the other hand, there's a handset of people trying to work when they will have time on XKB2, the second version of the "let's-try-to-fix-up-the keyboard-part-of-the-protocol-we-wrote-23-years-ago-a-second-time" extension. To me, it does not seem X12 will happen in the next decade neither. Alternative ?

Do we got alternative to X ? There's Wayland, but it's far from being usable. There's DirectFB, but that's not very portable. None seems candidate to replace X some days to me. Anyhow, none of the main toolkits around support this alternative. GTK+ once supported DirectFB, but as far as I know, it is not supported nor works nowadays, as stated by Josselin Mouette. This is why recent versions of the Debian installer have migrated to X for the graphic part, thanks to Cyril Brulebois work. Conclusion

XCB has been around for more than half-a-decade, and very few people showed interested in it. As far as I can see, nobody is interested to use the X protocol and everybody tries to encapsulate it in some higher-level API as soon as possible to stop seeing it. This leads to poorly written application and toolkits, with a lot of ugly hack. All of that also means that starting to write applications and graphical toolkits based on XCB would be a very interesting project, but that would lead to spend too much time learning to circumvent the X protocol flaws, things that have been done in years by predecessors like Qt and GTK+. Major toolkits implementations have almost nothing to win in going back in the dark water of X. I guess most of their folks prefer to work on shiny 3D effects based on your GPS location, rather than redefining better basis for everyone. The manpower available in the X world is very small. Debian lacking of X maintainers is just the summit of that. There is very smart and very competent and skilled guys in the X world, as you can see by simply reading blog posts on Planet Freedesktop for example (me excluded). Unfortunately, there's not enough of them to cover all the things involved in X: input devices, graphics devices, new protocol extension specification and so on. The X server is really late, and it seems most of the developers prefers to work on the server itself than on the protocol behalf. Which is understandable. I'm curious to see where all of that will lead in the upcoming years. I've been walking in the X world hallways for about 3 years now, and I feel desktop alternatives to KDE and GNOME will all die sooner or later. The time were you could choose between a dozen "modern" window managers has passed away. After all, maybe that is simply Darwinism applied to computer software.

multistrap is gaining some important functionality. When preparing a root filesystem for a new product, the sources used to generate the system need to be made available. So, multistrap has an option to retain the sources and 2.1.5 adds the functionality of downloading the source packages to go alongside the binaries that are downloaded to make the system itself, into a directory outside the final system.

Equally, 2.1.5 includes the idea (contributed by Rodolfo Giometti) of not listing deb-src entries inside the final filesystem - a useful idea to save space on systems where there is no likelihood of sources actually being downloaded "on-device".

Also in 2.1.5, is a handler for device tables. udev is one thing but when creating new systems from scratch, some devices still need to be created with mknod. MAKEDEV is too general, creating dozens of nodes when maybe only a handful are needed yet also omitting device nodes that actually are needed. So, /usr/share/multistrap/device-table.pl can parse a mini device table file (a tab-separated-value file), creating directories, symlinks and device nodes in a specified directory.

The aim is to combine the entire generation of a bootable root filesystem in a single call to multistrap - to which two further steps are necessary.

1. Packing up into a tarball at the end


2. Running a customised script outside the eventual filesystem, passing it the location and architecture of the filesystem.

Both are in 2.1.5 - the config script can be used to pre-seed debconf questions, write out custom configuration changes, run the device table parser and various other tasks.

As with earlier versions, multistrap supports creating package building chroots (including cross-building chroots) and, to go alongside this functionality, pdebuild-cross has now entered unstable. pdebuild-cross is a set of hooks for pbuilder to support cross-building inside a pbuilder chroot created by multistrap.

I'm hoping to be able to cover more about these issues at DebConf10.

If debootstrap is not doing what you need, multistrap is probably the answer - and if it doesn't quite do what you need yet, ask.

Last week was very busy for me. Unfortunately, I spent most of my time no my study duties. An exam session is fast approaching, so in next 3 weeks the situation will not be better. But as I promised, during second part of week, I am working full time on my project.

So, here are missing mockups of future Aptitude Qt GUI:

Changes Preview tab:

Only one "Preview Tab" can be opened at the same time. Changes will be live updated here.

Resolver:

Perform Changes tab:


This time I am not too innovative. I think that current Aptitude GTK design for this tabs is quite intuitive and good-looking, so I have added only minor tweaks.

To sum up mockups topic, below are links to all presented mockups (with some comments which raises during my recent private discussions). Now, I am open for any suggestions and comments about current design. At the end of next week, second iteration will be presented. Depending of the level of required changes there could be third iteration, before sending mockups to Piotr "Patpi" Pe zowski, who is the person behind Kadu Usability Project. I am glad that he agreed to spend his spare time and help me with this task :)

Packages tab:

This tab in second iteration will have only small changes. I was told that will be nice to select multiple categories at the same time. I agree, so categories list will support multi selection. There will be also an option in configuration window (There is no mockup now. Currently, I do not know what option will be configurable and it is not the most important thing at this stage. The window will be patterned upon KDE configuration windows) to move this list on the right side of this tab.

It will be possible to have multiple "Packages tabs" opened at the same time. It will not be allowed to close first "Package Tab". Ctrl + T shortcut, action in main menu and toolbar (I have forgotten about toolbar in this iteration - maybe it is not necessary?) will open new tab. There will be also an option that each text search will do this. I will try to add shortcuts wherever is it possible to allow using the GUI without mouse.

"Show changes" button will be change to "Resolve dependencies" when necessary.

Updates widget:

Most of the comments from "Packages tab" apply here too. In the table showing updates will be also "Current Version" column.


Package Informations tab:

Only one clarification: multiple "Packages Informations" tabs for the same package will not be allowed.

I am looking forward to read any comments regarding presented design. Please leave comment below this post or contact me directly (see About Me page).

On Monday/Tuesday I will write first weekly report.

Cheers.

Today, I have finished only one mockup. Designing "Package Details" tab was much harder than I had thought it would be. But here is it:

Package details:

It is not finished and requires more work (as all mockups). There are two main mockups left (Preview and Resolver tabs). They will have been finished till the end of Wednesday. Fortunately, there is still some time to finish and evaluate mockups. As I wrote before, firstly I will be coding low level classes.

Today (it is past midnight in my timezone ;) ) is also official start of coding period (It has not been announcement yet). For the next month, I will have been sharing my time between GSoC and my normal study duties. Due to that fact, most of my work will be done in second part of each week. I will write weakly updates on this blog.

So, please stay tuned and wish me good luck :)

Here are two mockups presenting my basic ideas about future aptitude-Qt GUI:

Packages tab:


Updates tab:


Mockups for Show Package and Preview tabs will be finished tomorrow.

Any comments and constructive criticism are much appreciated.

Hello all.

Last week I was very busy, due to my university duties. On Tuesday I had my first exam in this semester (in Numerical Analysis), which took me more time than I had thought it would be.
From this reason my project is slightly behind schedule, but now I will do my best to catch the plan.

On the other hand, there are also some good news for me. Some time ago, Daniel Burrows migrated aptitude repository from mercurial to GIT. I am git user for nearly year, so I strongly support this move ;) .

Here are some basic thoughts which functionalities good package manager must have (and which approach (from aptitude-gtk, synaptic and adept) I like the most):

• upgrade packeges with good conflict resolution - aptitude conflict resolver is the most advanced. Without a doubt, it has to be in Qt version
• fast searching and installing new packages - I prefer synaptic/yast approach with categories on the left and packages on the right, with single click behavior from KPackegeKit/adept (single click shows package description under selected package and with detailed informations available in new tab - as in aptitude-gtk)
• viewing detailed package informations
• downgrading, holding etc
• fetching packages list, cleaning cache etc

All of this will be elaborated in mockups. Am I forgetting something important?

The most important questions is: "What is the most common use case for package manager? Installing packages or upgrading packages?". I am not sure if this two actions require different views, or there should be only new category in install package view.

This is all now. More updates (and mockups) will be available today's midnight (or tomorrow morning). So please stay tuned :)

Hello Planet Debian readers!

I've never thought that such a thing can ever happen, but I've started blogging ;) So now it is time to introduce myself.

My name is Piotr Galiszewski and I am second year student of computer science at AGH - University of Science and Technology in Krakow (Poland). I have been GNU/Linux user for about 5 years (mostly Debian based distributions).

Thanks for Debian and Google, this summer I will be working on creating Qt-based user interface for aptitude as my Google Summer of Code project. I hope that my mentors Sune Vuorela and Daniel Barrows will be patient with me ;) I am sure I will learn a lot from them. Please look at abstract of my project made by Debian GSoC administrator Obey Arthur Liu:

Qt GUI for aptitude. Currently, KDE users need to use Aptitude via the console interface, or install the newly developed GTK frontend, which does not fit well into KDE desktop. Making Qt frontend to Aptitude would solve this problem and bring an advanced and fully Debian-compliant graphical package manager to KDE.

As I wrote in my proposal I will split my work into three main parts:

1. writing low level classes which will abstract aptitude signals and slots (which uses sigc++) into Qt slots and signals.
2. creating and evaluating GUI mockups
3. implementing GUI on top of classes from the first point

Point 1 and 2 will be made simultaneously and will take all May and half of June. Low level classes should implement all necessary functions for further use in GUI. This classes allow me to avoid direct usage of none Qt code in GUI classes and also give much more time to prepare completed and usability-wise mockups. Every mockups version will be presented and discussed on this blog. First version should be ready in next few days and updates will appear each week (or to weeks).
After finishing this two steps I will start coding GUI. With mockups and finished low level classes this should not be complicated (Yeah, I know that this is only my dream).

Full text of my proposal (including more precise time-line) can be found at Debian wiki.

Currently, project is slightly behind the schedule . It is caused by changes in my studies plan. The Juwanalia students' festival took place earlier, and yesterday it finished. But my first exam will take place one week later on 18 May, so I will have more time to catch up with time-line.

This project is my first direct contribution to Debian, but not first involvement in free software movement. I've been Kadu Instant Messenger developer for more then two year. In last two years I have been second most active developer with more than 700 commits in master branch. During GSoC period my Kadu activities will be limited. If time allows me to do this, I will be still contributing to Kadu. I still can be found at Kadu forum or #kadu channel on irc.freenode.net. I will also continue reviewing patches and fixing low time-consuming bugs.

My plans for the next few days:

• continue researching aptitude codebase
• discover all functionalities that should be implemented by Qt frontend
• review of other graphical package manager GUIs (and probably write about my thoughts on this blog)
• create first draft of my mockups

If you have any thoughts about this project, please add comment to this post or contact me directly. I will be glad to read all yours opinions

Cheers

PS. As you can see English is not my mother tongue, so please forgive me my mistakes

During the week-end, samba 3.5.3 hit the archive. It is still in experimental (and will remain there as Squeeze will be released with 3.4.* as finally decided during SambaXP). For lenny users, I also build backported packages. They are not on backports.org as the policy for these "as official as possible" backports is to have the same version than testing. So, they are provided "as is", for i386 and amd64, on the pkg-samba team's unofficial repository. The Release file on that repository is signed by my GPG key. Please note that these backports also need talloc from backports.org so you need to add entries for backports.org in your sources.list files to be able to use them. Since I last blogged about samba packages, the 3.4.8 version was also released. It entered testing without any problem and I also uploaded a backported version to backports.org. Finally, on May 5th, new versions of the 3.2.5 packages of lenny were updated to 2:3.2.5-4lenny11, currently in stable-proposed-updates. So, they will be part of the next point release of lenny. They fix two important bugs (#575951 for interdomain trust with w2k8 servers, and #538819 for memory leaks in some situations).

I've updated my BPM tap toy, which started behaving strangely since some GTK update. I've identified two issues, and it is working again now.The BPM tap toy is a tiny Python script to obtain a BPM estimation by tapping on your mouse or keyboard. In contrast to many other such applications it does give you an error estimation and will visualize your precision in tapping. This way, you can judge how precise the value is.I have some ideas to further increase precision while adding another useful visualization to judge the result quality.

Today I received a mail from the DebConf Travel Sponsorship Team and they informed me that I may get partially (or fully) sponsored. Ok that s a bit vague for now, but that also means that I may come to Debconf 10 yeah! I also applied for this Debconf Newbies thingy, as this will be the first DebConf for me, but unfortunately there where too many applicants and not enough money and they had to decline my request.