Search Results: "ron"

15 July 2013

Rob Bradford: Clutter & Cogl Wayland update

Lots of us over at Intel towers are working hard to bring you a great new Wayland powered experience. Cogl and Clutter have now been updated to work with the latest Wayland master. You need to compile master Cogl with: ./configure --enable-wayland-egl-platform=yes --enable-gl And then to run any applications using Cogl on Wayland you need to setup the environment variable COGL_RENDERER=egl_wayland then you can run cogl-crate! For Clutter you need to compile Clutter master with: ./configure --enable-wayland-backend You also need to set an environment variable to choose the Clutter backend too in this case it s CLUTTER_BACKEND=wayland (and you need to set COGL_RENDERER too) then you can run test-actors! Or just look at the video below :-) <iframe allowfullscreen="allowFullScreen" frameborder="0" height="300" mozallowfullscreen="mozallowfullscreen" src="http://player.vimeo.com/video/33734847?title=0&amp;byline=0&amp;portrait=0" webkitallowfullscreen="webkitAllowFullScreen" width="400"></iframe> Direct Vimeo link! A big kudos needs to go to Robert Bragg, Neil Roberts and Emmanuele Bassi for helping make this happen. The Clutter client side support is basically complete so i ve now moved onto working on GTK and MX watch this space for updates!

Daniel Pocock: Linux init-systems

Several value-added init systems have emerged in recent times. Most notably, Upstart has become the default on Ubuntu while systemd is now default in Fedora. Let's not forget the crusade to move beyond traditional init was given a big boost back in 2005 by Sun's introduction of SMF on Solaris 10 some years before the main Linux solutions gained prominence. Debian is currently debating whether to use one of these alternatives as a default. My own position on the matter is not decided, but I do feel that some of the debates are asking the wrong questions. Some of the most interesting issues here are likely to have ramifications for other distributions. My experience with systemd I've come across systemd in my efforts to package reSIProcate on Fedora. One of the challenges I came across was trying to both create a /var/run/repro directory and telling systemd to launch the process as a non-root user. I tried using the ExecStartPre directive in the systemd manifest to invoke mkdir. Unfortunately, it runs as the non-root user too and could not create the PID directory. The conventional init script doesn't suffer this problem. In the end, I have patched the repro binary to start as root and drop privileges itself, but that was only when it became apparent that we need to do that for WebRTC support (listening on port 443). While most daemons have the ability to drop privileges, there will be edge cases like this that need more than just a systemd manifest. Maintainers who encounter such issues may have to give feedback to the systemd and Upstart upstreams and give them the opportunity to provide generalized solutions. Then again, the most generalized solution would be another scripting language and then we'd be back where we started. The reason to change and the reason not to Some people have argued that Debian should change simply because other distributions did so. This is not a compelling technical argument alone. The fragmentation of init-alternatives is not something that makes upstreams excited either. It would be interesting to see if Upstart and systemd could agree on a common manifest format - maybe even a format that could be parsed by a wrapper from SysV init. Then Debian wouldn't really have to give up support for kFreeBSD and other non-Linux kernel efforts. A slight alternative to this approach would be using a system like cmake to read a single service XML manifest and spit out permutations readable by each of the different init schemes. It has also been well argued that the simple and well known solution of scripts is much less complicated to understand and maintain and is extremely stable. Wishlist items for all distributions While I don't know which solution is the best, one thing I do believe is that if there is a change and if it is going to be disruptive to maintainers and users then it should aim to bring in as many benefits as possible to offset the pain and encourage people to support the migration effort. Some of the potential road-map items are not only possible for Debian but could also become part of other platforms. Here are some of the issues that came to my mind:
  • Better support for starting multiple instances of a single process
  • Support for distributed solutions, for example, making sure that process A only starts on host 1 after process B on host 2
  • Support for clustering, for example, making sure that at least 2 instances of process A are running across 3 hosts.
  • Automated integration with monitoring and fault recovery systems: for example, can Nagios (or another tool) automatically detect whether a particular process should be running and whether it is running? Given that init-replacements typically need to start processes in the correct order, can they share that dependency information with Nagios to avoid manual/duplicated dependency mappings? Can Nagios restart the process without any manual scripting? Can this all be enabled for all processes on a given system in a plug-and-play manner?
  • Scheduler interaction: can and should any of the above, particularly the dependency mappings, be integrated with scheduling? For example, should some cron jobs be deferred when MySQL is down?
What other potential improvements do people feel to be worthwhile?

14 July 2013

Matt Zimmerman: Liberty and justice for all, but not in equal measure

200302020-001 As Americans we might like to believe that the US legal system is intended to protect all of our citizens. Unfortunately, it doesn t protect us all equally, and in fact disproportionately fails to protect the most vulnerable. We re surrounded by instances of injustice related to gender, race and other axes of social privilege, and the machinations of law are not exempt. The state of Florida has recently provided an especially stark example in the application of its self-defense laws in two cases: Marissa Alexander and George Zimmerman. This example is notable because although there were many similarities between the cases, the outcomes were very different. Alexander s case was tried in May 2012 , Zimmerman s in July 2013, both prosecuted by Florida state s attorney Angela Corey. Both cases involved the use of firearms which were legally purchased and carried, and their owners were trained in their use. Both prosecutions cast the defendant as the aggressor, who could have avoided the confrontation. Both of the encounters were with unarmed persons. Both defenses were based on Florida self-defense laws, which include stand your ground laws justifying the use of deadly force without the obligation to retreat. Both shooters admitted to firing a single shot with the intent of defending themselves. Beyond those similarities, each case had its own unique circumstances. The events of Alexander s case took place in her home. Her altercation was with her husband, Rico Gray Sr., who was under a restraining order following a conviction for domestic battery which put Alexander in the hospital. After Gray threatened to kill her, Alexander retrieved a handgun from her car, returned to confront him, firing once. She was arrested and charged the same day. She had had no prior criminal record. A jury deliberated for just 12 minutes before convicting her. A judge sentenced her to 20 years in prison, in accord with mandatory minimums specified by law. Gray, previously sentenced to probation for his earlier conviction, remains free. Zimmerman,_George_-_Seminole_County_MugZimmerman s shot was fired in his neighborhood, in an altercation with a teenager, Trayvon Martin, who was a guest in the community and walking by himself. The two were not acquainted. Zimmerman called police from his car, claiming that Martin appeared suspicious, and began to follow him. Some of the facts of their encounter remain in dispute, but that Zimmerman fired his gun is not in question. Afterward, Zimmerman was detained by police, questioned and released the same night without being arrested or charged. Following a public outcry, a new investigation was launched and two months later he was arrested and charged. He had been previously arrested and charged with assaulting a police officer, but the charges were later dropped. After 16 hours of deliberation, the jury found Zimmerman not guilty, and he is free today. The most striking difference between the two cases is where each defendant aimed their gun: George Zimmerman shot Trayvon Martin in the chest and killed him, while Marissa Alexander fired at a wall and injured no one. Alexander, a black woman, is in prison for scaring her abusive husband away, while Zimmerman, who killed a young black man, walks free. Alexander and Martin s families have lost a mother and a son. The outcomes for the people involved in these cases could not be more different. Regardless of the merits of the relevant laws themselves, their radically unequal application is deeply troubling. What does this tell us about the relative value of these human lives, as weighed by the judicial system?
The Florida criminal justice system has sent two clear messages today. One is that if women who are victims of domestic violence try to protect themselves, the Stand Your Ground Law will not apply to them. [...] The second message is that if you are black, the system will treat you differently. - U.S. Rep. Corrine Brown
References

Dirk Eddelbuettel: Go home, Feedly, you're drunk!

With the demise of Google Reader, many of us have scrambled to find alternatives which cover many/most of the features we were used to. For me, synchronised access from web and mobile counts quite high, and I want the (Android) mobile experience to be pretty pleasant too. So with that, and like many other folks, I had shifted over to Feedly. And before I carry on, let me say that Feedly actually does pretty well. They handled the onslaught of new users; they also listened and changed their UI a little to be more compact and Reader-alike And most importantly, it mostly just works. Until it doesn't. Around the time Feedly shifted folks to their own cloud-based backend, posts from my very own CRANberries aggregator of CRAN package changes for R started to show only truncated posts. See this screenshot from reading Feedly: Feedly truncates the CRANberries post and compare it with this one from TheOldReader: OldReader shows the full CRANberries post which contains the diffstat output, nicely marked up and all. Now, it so happens that I am the creator of the very RSS feed I am consuming here, and I wrote it so that I could read the very diffstat output that is now missing. And I know full well that neither the code, nor the hosting (on my own box), nor any other aspect changed. Before confirming this via TheOldReader, I looked at the RSS output, and I tried other frontend and apps --- and it became clear that the bug seems to be at the Feedly cloud storage level. And trying to be a good sport, I submitted a bug report / suggestion, but to no avail (apart from two other chiming in that they see this elsewhere too). Given that I am the coder behind the feed that is displayed in a truncated manner, I am aware that I am using a code stack that is stale (the original Blosxom static web generator) yet which has not posed another problem anywhere in the decade I used it. Nor does the CRANberries feed pose a problem when aggregated and viewed via the TheOldReader code path, or for that matter via Planet R which also carries (parts of) CRANberries. So for now, I have to either pivot out of the CRANberries RSS reading in Feedly and go directly to the webpage (fine, but cumbersome and in need of a working connection) or use a second subscription mechanism. I appreciate what the TheOldReader folks are doing---presumably with a minor fraction of the resources available to Feedly---and may hang with them for now. But it would be awfully nice if Feedly could sleep off its hangover and fix this. In which case I'd be much happier recommending its service.

Gregor Herrmann: RC bugs 2013/28

this week(end) I was more in the "RC bug fix flow" than in the last weeks, & here are the results:

Michael Stapelberg: How many packages does Debian Code Search contain?

The German computer magazine c't has covered Debsources in its most recent edition (c't 16/2013). In that article, they also state: 
Debsources integriert auch eine Code-Suche, allerdings werden lediglich die
Quellen des Unstable-Zweigs durchsucht, der zirka ein Drittel des Quellcodes
von Debsources ausmacht.
This loosely translates to: 
Debsources also integrates a code search engine, but it only searches the
sources of the unstable tree, which makes up for roughly one third of the
sources that Debsources covers.
I suspect the author of the article arrived at this conclusion because Debsources talks about 400 GiB of source code, whereas Debian Code Search talks about 130 GiB of source code. However, it still struck me as odd and giving the wrong impression. I thought that packages don t differ that much between stable, testing and unstable. So I fired up psql and pounded UDD until it revealed how many packages are different between stable, testing and unstable: Conclusion So, in conclusion, Debian Code Search covers 74% of wheezy, testing and unstable. Debsources also covers contrib and non-free, which explains the higher disk usage. In particular, there might be big blobs in non-free that account for a lot of disk space. Also, Debsources keeps sources around for a few weeks whereas Debian Code Search only keeps the most recent snapshot. Query 
SELECT
  COUNT(*)
FROM
  (SELECT
     migrations.source,
     sources.version AS stable_version,
     testing_version,
     unstable_version
   FROM sources
   LEFT JOIN migrations
   USING (source)
   WHERE sources.release = 'wheezy') AS x
WHERE
  regexp_replace(stable_version::text, E'-([0-9.]+)$', '') = regexp_replace(testing_version::text, E'-([0-9.]+)$', '');

Nicolas Dandrimont: Bootstrapping fedmsg for Debian

As you might (or might not) know, this summer, I have taken on mentoring of a GSoC project by Simon Chopin (a.k.a. laarmen) which goal is to bring fedmsg, the Fedora Infrastructure message bus, to Debian. Most of the work I ll be talking about here is Simon s work, please send all the praise towards him (I can take the blame, though). What is this about? As the project proposal states, the idea is to provide Debian with a unified, real-time, and open mechanism of communication between its services. This communication bus would allow anyone, anywhere, to start consuming messages and reacting to events happening in Debian s infrastructure: When we told upstream about our plan of adapting fedmsg to work on Debian, they were thrilled. And they have been very supportive of the project. How is the project going? Are you excited? I know I m excited. yep, he's excited too Well, the general idea was easy enough, but the task at hand is a challenge. First of all, fedmsg has a lot of (smallish) dependencies, most of them new to Debian. Thanks to Simon s work during the bonding period, and thanks to paultag s careful reviews, the first batch of packages (the first dependency level, comprising kitchen, bunch, m2ext, grapefruit, txws, txzmq and stomper) is currently sitting in the NEW queue. The four remaining packages (fabulous, moksha.common, moksha.hub and fedmsg proper) are mostly ready, waiting in the Debian Python Module Team SVN repository for a review and sponsorship. While we re waiting for the packages to trickle into Debian, Simon is not twiddling his thumbs. Work has taken place on a few fronts: fedmsging mentors.debian.net Package backports mentors.debian.net was chosen because I m an admin and could do the integration quickly. That involved backporting the eleven aforementioned packages, plus zeromq3 and python-zmq (that only have TCP_KEEPALIVE on recent versions), to wheezy, as that s what the mentors.d.n host is running. (Also, python-zmq needs a new-ish cython to build so I had to backport that too). Thankfully, those were no-changes backports, that were easily scripted, using a pbuilder hook to allow the packages to depend on previously built packages. I have made a wheezy package repository available here. It s signed with my GnuPG key, ID 0xB8E5087766475AAF, which should be fairly well connected. Code changes After Simon s initial setup of debexpo (which is not an easy task), the code changes have been fairly simple (yes, this is just a proof of concept). You can see them on top of the live branch on debexpo s sources. I finally had the time to make them live earlier this week, and mentors.debian.net has been sending messages on Debian s fedmsg bus ever since. Deployment mentors.d.n sends its messages on five endpoints, tcp://mentors.debian.net:3000 through tcp://mentors.debian.net:3004. That is one endpoint per WSGI worker, plus one for the importer process(es). You can tap in directly, by following the instructions below. debmessenger Debmessenger is the stop-gap email-to-fedmsg bridge that Simon is developing. The goal is to create some activity on the bus without disrupting or modifying any infrastructure service. It s written in hy, and it leverages the existing Debian-related python modules to do its work, using inotify to react when a mail gets dropped in a Maildir. Right now, it s supposed to understand changes mails (received from debian-devel-changes) and bugs mail (from debian-bugs-dist). I ll work on deploying an instance of debmessenger this weekend, to create some more traffic on the bus. Reliability of the bus I suggested using fedmsg as this was something that already existed, and that solved a problem identical to the one we wanted to tackle (open interconnection of a distribution s infrastructure services). Reusing a piece of infrastructure that already works in another distro means that we can share tools, share ideas, and come up with solutions that we might not have considered when working alone. The drawback is that we have to either adapt to the tool s idiosyncrasies, or to adapt the tool to our way of working. One of the main points raised by DSA when the idea of using fedmsg was brought up, was that of reliability. Debian s infrastructure is spread in datacenters (and basements :D ) all over the world, and thus faces different challenges than Fedora s infrastructure, which is more tightly integrated. Therefore, we have to ensure that a critical consumer (say, a buildd) doesn t miss any message it would need for its operation (say, that a package got accepted). There has been work upstream, to ensure that fedmsg doesn t lose messages, but we need to take extra steps to make sure that a given consumer can replay the messages it has missed, should the need arise. Simon has started a discussion on the upstream mailing list, and is working on a prototype replay mechanism. Obviously, we need to test scenarios of endpoints dropping off the grid, hence the work on getting some activity on the bus. How can I take a look? a.k.a. Another one rides the bus A parisian bus built in 1932 (Picture Yves-Laurent Allaert, CC-By-SA v2.5 / GFDL v1.2 license) So, the bus is pretty quiet right now, as only two kinds of events are triggering messages: a new upload to mentors.debian.net, and a new comment on a package there. Don t expect a lot of traffic. However, generating some traffic is easy enough: just login to mentors.d.n, pick a package of mine (not much choice there), or a real package you want to review, and leave a comment. poof, a message appears. For the lazy Join #debian-fedmsg on OFTC, and look for messages from the debmsg bot. Current example output: 
01:30:25 <debmsg> debexpo.voms-api-java.upload (unsigned) --
02:03:16 <debmsg> debexpo.ocamlbricks.comment (unsigned) --
(definitely needs some work, but it s a start) Listening in by yourself You need to setup fedmsg. I have a repository of wheezy packages and one of sid packages, signed with my GnuPG key, ID 0xB8E5087766475AAF. You can add them to a file in /etc/apt/sources.list.d like this: 
deb http://perso.crans.org/dandrimont/fedmsg-<sid wheezy>/ ./
Then, import my GnuPG key into apt (apt-key add), update your sources (apt-get update), and install fedmsg (apt-get install python-fedmsg). The versions are << to anything real, so you should get the real thing as soon as it hits the archive. Finally, in /etc/fedmsg.d/endpoints.py, you can comment-out the Fedora entries, and add a Debian entry like this: 
    "debian": [
        "tcp://fedmsg.olasd.eu:9940",
    ],
fedmsg.olasd.eu runs a fedmsg gateway connected to the mentors.d.n endpoints, and thus forwards all the mentors messages. It ll be connected to debmessenger as soon as it s running too. To actually see mesages, disable validate_signatures in /etc/fedmsg.d/ssl.py, setting it to False. The Debian messages aren t signed yet (it s on the roadmap), and we don t ship the Fedora certificates so we can t authenticate their messages either. Finally, you can run fedmsg-tail --really-pretty in a terminal. As soon as there s some activity, you should get that kind of output (color omitted): 
 
  "i": 1, 
  "msg":  
    "version": "2.0.9-1.1", 
    "uploader": "Emmanuel Bourg <ebourg@apache.org>"
   , 
  "topic": "org.debian.dev.debexpo.voms-api-java.upload", 
  "username": "expo", 
  "timestamp": 1373758221.491809
Enjoy real-time updates from your favorite piece of infrastructure! What s next? While Simon continues working on reliability, and gets started on message signing according to his schedule, I ll take a look at deploying the debmessenger bridge, and making the pretty-printer outputs useful for our topics. There will likely be some changes to the messages sent by debexpo, as we got some feedback from the upstream developers about making them work in the fedmsg tool ecosystem (datanommer and datagrepper come to mind). You can tune in to Simon s weekly reports on the soc-coordination list, and look at the discussions with upstream on the fedora messaging-sig list. You can also catch us on IRC, #debian-soc on OFTC. We re also hanging out on the upstream channel, #fedora-apps on freenode.

13 July 2013

Michael Stapelberg: Survey answers part 3: systemd is not portable and what this means for our ports

This blog post is the third of a series of posts dealing with the results of the Debian systemd survey. I intend to give a presentation at DebConf 2013, too, so you could either read my posts, or watch the talk, or both :-). The second-biggest concern in the survey results was that systemd is not portable to non-Linux systems, for example Debian GNU/kFreeBSD or Debian GNU/HURD. For convenience, I will from now on write kFreeBSD when I really mean all non-Linux ports. systemd not being portable is not an arbitrary decision Some people seem to think that the systemd upstream is just hostile to users of other operating systems when they hear that systemd is not portable. However, keep in mind that Lennart, Kay and other contributors have considerable experience with writing portable software such as Avahi and PulseAudio. The decision to only support Linux in systemd was thus not taken lightly. systemd s design requires many kernel features and certain semantics (e.g. procfs is not enough, /proc/$PID/exe needs to be supported), which are currently only available on Linux. Point 15 of Lennart s blog post 0pointer.de/blog/projects/the-biggest-myths.html contains an incomplete list of these features. Maintaining portable code increases complexity Since systemd is written in C, the canonical way to write portable code is by using conditional compilation, for example with ifdef statements. That makes the code harder to understand and reason about, but more importantly it blows up the test matrix. It also requires any new change to be tested on all supported systems, which is not feasible for most contributors. I think everybody agrees keeping complexity low is a good thing. What are the implications for our non-Linux ports? We, the Debian project, have two realistic options in my point of view:
  1. We stay with sysvinit, the least common denominator, forever.
  2. We use a modern init system such as systemd on Debian GNU/Linux.
In case we go with the first option, Debian will diverge more and more from virtually every other Linux distribution. This also means we are stuck with the limited features and capabilities that sysvinit has. A modern operating system needs to be able to adapt to a changing environment and the once static world in which sysvinit was developed has become much more dynamic. In our opinion, the only reasonable choice is the latter option: use systemd on Debian GNU/Linux. How will this work? For individual maintainers, this means they need to support two init systems. Luckily, systemd service files are usually really simple, but there still is additional maintenance work such as testing whether your service actually starts when using systemd. We think this maintenance overhead is justified due to the advantages a modern init system brings. Of course, not every maintainer can arrange it to install systemd and test his/her package. You are invited to contact us at pkg-systemd-maintainers@lists.alioth.debian.org at any time and we can help you out. Furthermore, automation can be introduced (and we have a proof of concept) to make it easier to spot mistakes and perform some simple tests, such as whether the service can be started. A concern that was voiced is that as sysvinit usage decreases, the init scripts would bit-rot and stop working at some point. If that happens, we rely on the users to file appropriate bug reports. This is no different from the situation today it is not feasible for maintainers to test every single combination of features all the time. Ports are different and that diversity is good The FreeBSD kernel and the Linux kernel are different, and each kernel provides distinct features that the other kernel does not have. As an example, Linux provides cgroups, which are heavily used by systemd. The FreeBSD kernel in turn offers the packet filter pf , which is not available on Linux. There certainly is value in having common infrastructure. But there also is value in providing the best features that each platform has to offer in case of Linux, that is clearly systemd as an init system IMO. Conclusion systemd is not portable because it relies on features only the Linux kernel provides an example is cgroups, which systemd uses to track processes in a reliable way. Not embracing these features and staying with sysvinit indefinitely is not a viable option if Debian wants to remain relevant for today s demands. In the short term, the migration to systemd will cause additional maintenance effort for individual package maintainers, but it will pay off in the long term.

12 July 2013

Justus Winter: Hi, I'm the one who wrote umount...

Everyone enjoys ASCII screenshots as much as I do, right? So here is the Hurd booting with my mtab-prototype bound to /run/mtab: 
INIT: version 2.88 booting
Using makefile-style concurrent boot in runlevel S.
Activating swap...done.
Checking root file system...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
/dev/hd0s1: clean, 44292/181056 files, 287759/723200 blocks
done.
Activating lvm and md swap...(default pager): Already paging to partition hd0s5!
done.
Checking file systems...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
done.
Cleaning up temporary files... /tmp.
Mounting local filesystems...done.
Activating swapfile swap...(default pager): Already paging to partition hd0s5!
done.
Cleaning up temporary files....
Configuring network interfaces...inetutils-ifconfig: invalid arguments
ifup: failed to open pid file /run/network/ifup-/dev/eth0.pid: No such file or directory
Internet Systems Consortium DHCP Client 4.2.2
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
Listening on Socket//dev/eth0
Sending on   Socket//dev/eth0
DHCPDISCOVER on /dev/eth0 to 255.255.255.255 port 67 interval 4
DHCPREQUEST on /dev/eth0 to 255.255.255.255 port 67
DHCPOFFER from 10.0.2.2
DHCPACK from 10.0.2.2
can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
bound to 10.0.2.15 -- renewal in 42550 seconds.
done.
Cleaning up temporary files....
Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix.
INIT: Entering runlevel: 2
Using makefile-style concurrent boot in runlevel 2.
Starting enhanced syslogd: rsyslogd.
Starting deferred execution scheduler: atd.
Starting periodic command scheduler: cron.
Starting system message bus: dbusFailed to set socket option"/var/run/dbus/system_bus_socket": Protocol not available.
Starting OpenBSD Secure Shell server: sshd.
GNU 0.3 (debian) (console)
login:
As you can see, the boot process looks quite clean. In particular, there are no issues left with Debians initscripts. The remaining noise comes from: This is accomplished by my mtab translator prototype. It's work in progress, but I have reworked and broken up my initial prototype. My initial prototype only patched libdiskfs, one of three libraries translator authors can use to write filesystem-like translators. I have made the necessary changes to the other two libraries and patched up any loose ends. This is the translator in action: 
% fsysopts /run/mtab
/hurd/mtab /
% cat /run/mtab
/dev/hd0s1 / ext2fs writable,no-inherit-dir-group,store-type=typed 0 0
none /dev/ttyp0 /hurd/term name,/dev/ptyp0,type,pty-master 0 0
none /run /hurd/tmpfs writable,no-suid,no-exec,no-inherit-dir-group,no-sync,size=80484K 0 0
none /run/lock /hurd/tmpfs writable,no-suid,no-exec,no-inherit-dir-group,no-sync,size=5M 0 0
none /run/shm /hurd/tmpfs writable,no-suid,no-exec,no-inherit-dir-group,no-sync,size=259480K 0 0
I also patched the sysvinit package. My version includes and switches to runsystem.sysv that uses /sbin/init to start the system. It also carries all the necessary workarounds for the remaining sysvinit related issues. The resulting Debian/Hurd system is very nice, it is getting more and more similar to what a Debian user would expect: I have rebuild the hurd and sysvinit package and uploaded them into an apt repository: 
deb http://teythoon.cryptobitch.de/gsoc/heap/debian unstable main
Please use unstable for now. Also make sure that you have a recovery plan for your Debian/Hurd installation if anything goes wrong. For your convenience there's a seed tarball containing packages with the appropriate sources.list.d snippets and the repository key: https://teythoon.cryptobitch.de/gsoc/heap/debian/seed.tar Currently /hurd/console is not started (I'll add a init script for that later), but for now you have to start a getty on the console. Make sure that your /etc/inittab contains a line like this: 
7:2345:respawn:/sbin/getty 38400 console
The patched initscripts package contains the runsystem.sysv file and uses the Debian alternatives system to replace the runsystem.gnu variant. It also switches to the appropriate halt and reboot utilities. If you install the package, you must use halt-hurd or reboot-hurd to halt or reboot the system. Running halt or reboot is not harmful though, it just doesn't work. So here I am. I've been coding for the Hurd for four weeks now, and suddenly I'm the guy who wrote umount. My point is, if you ever wanted to work on an operating system, you might want to consider working on the Hurd. It the Debian/Hurd port is very decent and (according to #debian-hurds "Debian GNU/Hurd Doomsday-o-meter") 78.44% of all Debian packages are available, so you've got a familiar environment. Also, if your solution is right and your code is reasonably clean, it is not that hard to get your changes accepted. The list of open issues is long and might contain something that interests you. You could be the one implementing a read-ahead solution to massively improve Hurds overall performance. Also since Hurd is just a Hird of Unix replacing daemons running on-top the GNU Mach microkernel, most of the functionality lies within userspace processes. This makes the whole system very extensible and quite easy to work with. For an overview over the Hurd, see this page. Next week I'll focus on improving my mtab prototype, improving the sysvinit patches and upstreaming the ones that are ready. As always, in case I run out of stuff to do, I'll pick something from my list of issues, most likely the network related issues. In completely unrelated news, my Debian/Linux workstation died with a kernel panic while I was writing this blog entry. It's a good thing Emacs honors "save early, save often". M-x insert-monolithic-kernel-rant-here. This is it for this week, thanks for your attention :)

Daniel Pocock: Practical VPNs with strongSwan, Shorewall, Linux firewalls and OpenWRT routers

There is intense interest in communications privacy at the moment thanks to the Snowden scandal. Open source software has offered credible solutions for privacy and encryption for many years. Sadly, making these solutions work together is not always plug-and-play. In fact, secure networking, like VoIP, has been plagued by problems with interoperability and firewall/filtering issues although now the solutions are starting to become apparent. Here I will look at some of them, such as the use of firewall zones to simplify management or the use of ECDSA certificates to avoid UDP fragmentation problems. I've drawn together a lot of essential tips from different documents and mailing list discussions to demonstrate how to solve a real-world networking problem. A typical network scenario and requirements Here is a diagram of the network that will be used to help us examine the capabilities of these open source solutions. Some comments about the diagram:
  • The names in square brackets are the zones for Shorewall, they are explained later.
  • The dotted lines are IPsec tunnels over the untrusted Internet.
  • The road-warrior users (mobiles and laptops) get virtual IP addresses from the VPN gateway. The branch offices/home routers do not use virtual IPs.
  • The mobile phones are largely untrusted: easily lost or stolen, many apps have malware, they can only tunnel to the central VPN and only access a very limited range of services.
  • The laptops are managed devices so they are trusted with access to more services. For efficiency they can connect directly to branch office/home VPNs as well as the central server.
  • Smart-phone user browsing habits are systematically monitored by mobile companies with insidious links into mass-media and advertising. Road-warriors sometimes plug-in at client sites or hotels where IT staff may monitor their browsing. Therefore, all these users want to tunnel all their browsing through the VPN.
  • The central VPN gateway/firewall is running strongSwan VPN and Shorewall firewall on Linux. It could be Debian, Fedora or Ubuntu. Other open source platforms such as OpenBSD are also very well respected for building firewall and VPN solutions, but Shorewall, which is one of the key ingredients in this recipe, only works on Linux at present.
  • The branch office/home network could be another strongSwan/Shorewall server or it could also be an OpenWRT router
  • The default configuration for most wifi routers creates a bridge joining wifi users with wired LAN users. Not here, it has been deliberately configured as an independent network. Road-warriors who attach to the wifi must use VPN tunnelling to access the local wired network. In OpenWRT, it is relatively easy to make the Wifi network an independent subnet. This is an essential security precaution because wifi passwords should not be considered secure: they are often transmitted to third parties, for example, by the cloud backup service in many newer smart phones.
Package mayhem The major components are packaged on all the major Linux distributions. Nonetheless, in every case, I found that it is necessary to re-compile fresh strongSwan packages from sources. It is not so hard to do but it is necessary and worth the effort. Here are related blog entries where I provide the details about how to re-build fresh versions of the official packages with all necessary features enabled: Using X.509 certificates as a standard feature of the VPN For convenience, many people building a point-to-point VPN start with passwords (sometimes referred to as pre-shared keys (PSK)) as a security mechanism. As the VPN grows, passwords become unmanageable. In this solution, we only look at how to build a VPN secured by X.509 certificates. The certificate concept is not hard. In this scenario, there are a few things that make it particularly easy to work with certificates:
  • strongSwan comes with a convenient command line tool, ipsec pki. Many of it's functions are equivalent to things you can do with OpenSSL or GnuTLS. However, the ipsec pki syntax is much more lightweight and simply provides a convenient way to do the things you need to do when maintaining a VPN.
  • Many of the routine activities involved in certificate maintainence can be scripted. My recent blog about using Android clients with strongSwan gives a sample script demonstrating how ipsec pki commands can be used to prepare a PKCS#12 (.p12) file that can be easily loaded into an Android device using the SD-card.
  • For building a VPN, there is no need to use a public certificate authority such as Verisign. Consequently, there is no need to fill in all their forms or make any payments for each device/certificate. Some larger organisations do choose to outsource their certificate management to such firms. For smaller organisations, an effective and sometimes better solution can be achieved by maintaining the process in-house with a private root CA.
UDP fragmentation during IPsec IKEv2 key exchange and ECDSA A common problem for IPsec VPNs using X.509 certificates is the fragmentation of key exchange datagrams during session setup. Sometimes it works, sometimes it doesn't. Various workarounds exist, such as keeping copies of all certificates from potential peers on every host. As the network grows, this becomes inconvenient to maintain and to some extent it eliminates the benefits of using PKI. Fortunately, there is a solution: Elliptic Curve Cryptography (ECC). Many people currently use RSA key-pairs. Best practice suggests using RSA keys of at least 2048 bits and often 4096 bits. Using ECC with a smaller 384-bit key is considered to be equivalent to a 7680 bit RSA key pair. Consequently, ECDSA certificates are much smaller than RSA certificates. Furthermore, at these key sizes, the key exchange packets are almost always smaller than the typical 1500 byte MTU. A further demand for ECDSA is arising due to the use of ECC within smart cards. Many smartcards don't support any RSA key larger than 2048 bits. The highly secure 384-bit ECC key is implemented in quite a few common smart cards. Smart card vendors have shown a preference for the ECC keys due to the US Government's preference for ECC and the lower computational overheads make them more suitable for constrained execution environments. Anyone who wants to use smart cards as part of their VPN or general IT security now, or in the future, needs to consider ECC/ECDSA. Making the network simple with Shorewall zones For this example, we are not just taking some easy point-to-point example. We have a real-world, multi-site, multi-device network with road warriors. Simplifying this architecture is important to help us understand and secure it. The solution? Each of these is abstracted to a "zone" in Shorewall. In the diagram above, the zone names are in square brackets. The purpose of each zone is described below:
Zone name Description
loc This is the private LAN and contains servers like databases, private source code respositories and NFS file servers
dmz This is the DMZ and contains web servers that are accessible from the public internet. Some of these servers talk to databases or message queues in the LAN network loc
vpn_a These are road warriors that are not very trustworthy, such as mobile devices. They are occasionally stolen and usually full of spyware (referred to by users as "apps"). They have limited access to ports on some DMZ servers, e.g. for sending and receiving mail using SMTP and IMAP (those ports are not exposed to the public Internet at large). They use the VPN tunnel for general internet access/browsing, to avoid surveillance by their mobile carrier.
vpn_b These are managed laptops that have a low probability of malware infection. They may well be using smart cards for access control. Consequently, they are more trusted than the vpn_a users and have access to some extra intranet pages and file servers. Like the smart-phone users, they use the VPN tunnel for general internet access/browsing, to avoid surveillance by third-party wifi hotspot operators.
vpn_c This firewall zone represents remote sites with managed hardware, such as branch offices or home networks with IPsec routers running OpenWRT.
cust These are servers hosted for third-parties or collaborative testing/development purposes. They have their own firewall arrangements if necessary.
net This zone represents traffic from the public Internet.
A practical Shorewall configuration Shorewall is chosen to manage the iptables and ip6tables firewall rules. Shorewall provides a level of abstraction that makes netfilter much more manageable than manual iptables scripting. The Shorewall concept of zones is very similar to the zones implemented in OpenWRT and this is an extremely useful paradigm for firewall management. Practical configuration of Shorewall is very well explained in the Shorewall quick start. The one thing that is not immediately obvious is a strategy for planning the contents of the /etc/shorewall/policy and /etc/shorewall/rules files. The exact details for making it work effectively with a modern IPsec VPN are not explained in a single document, so I've gathered those details below as well. An effective way to plan the Shorewall zone configuration is with a table like this:
Destination zone
loc dmz vpn_a vpn_b vpn_c cust net
Source zone loc \
dmz ? \ X X X
vpn_a ? ? \ X X
vpn_b ? ? X \ X
vpn_c X \
cust X ? X X X \
net X ? X X X \
The symbols in the table are defined:
Symbol Meaning
ACCEPT in policy file
X REJECT or DROP in policy file
? REJECT or DROP in policy file, but ACCEPT some specific ports in the rules file
Naturally, this modelling technique is valid for both IPv4 and IPv6 firewalling (with Shorewall6) Looking at the diagram in two dimensions, it is easy to spot patterns. Each pattern can be condensed into a single entry in the rules file. For example, it is clear from the first row that the loc zone can access all other zones. That can be expressed very concisely with a single line in the policy file: 
loc all ACCEPT
Specific Shorewall tips for use with IPsec VPNs and strongSwan Shorewall has several web pages dedicated to VPNs, including the IPsec specific documentation.. Personally, I found that I had to gather a few details from several of these pages to make an optimal solution. Here are those tips:
  • Ignore everything about the /etc/shorewall/tunnels file. It is not needed and not used any more
  • Name the VPN zones (we call them vpn_a, vpn_b and vpn_c) in the zones file but there is no need to put them in the /etc/shorewall/interfaces file.
  • The /etc/shorewall/hosts file is not just for hosts and can be used to specify network ranges, such as those associated with the VPN virtual IP addresses. The ranges you put in this file should match the rightsourceip pool assignments in strongSwan's /etc/ipsec.conf
  • One of the examples suggests using mss=1400 in the /etc/shorewall/zones file. I found that is too big and leads to packets being dropped in some situations. To start with, try a small value such as 1024 and then try larger values later after you prove everything works. Setting mss for IPsec appears to be essential.
  • Do not use the routefilter feature in the /etc/shorewall/interfaces file as it is not compatible with IPsec
Otherwise, just follow the typical examples from the Shorewall quick start guide and configure it to work the way you want. Here is an example /etc/shorewall/zones file: 
fw firewall
net ipv4
dmz ipv4
loc ipv4
cust ipv4
vpn_a ipsec mode=tunnel mss=1024
vpn_b ipsec mode=tunnel mss=1024
vpn_c ipsec mode=tunnel mss=1024
Here is an example /etc/shorewall/hosts file describing the VPN ranges from the diagram: 
vpn_a eth0:10.1.100.0/24 ipsec
vpn_b eth0:10.1.200.0/24 ipsec
vpn_c eth0:192.168.1.0/24 ipsec
Here is an example /etc/shorewall/policy file based on the table above: 
loc all ACCEPT
vpn_c all ACCEPT
cust net ACCEPT
net cust ACCEPT
all all REJECT
Here is an example /etc/shorewall/rules file based on the network: 
SECTION ALL
# allow connections to the firewall itself to start VPNs:
# Rule  source    dest    protocol/port details
ACCEPT   all       fw                ah
ACCEPT   all       fw                esp
ACCEPT   all       fw                udp 500
ACCEPT   all       fw                udp 4500
# allow access to HTTP servers in DMZ:
ACCEPT   all       dmz               tcp 80
# allow connections from HTTP servers to MySQL database in private LAN:
ACCEPT   dmz       loc:10.2.0.43     tcp 3306
# allow connections from all VPN users to IMAPS server in private LAN:
ACCEPT vpn_a,vpn_b,vpn_c loc:10.2.0.58 tcp 993
# allow VPN users (but not the smartphones in vpn_a) to the
# PostgresQL database for PostBooks accounting system:
ACCEPT vpn_b,vpn_c loc:10.2.0.48      tcp 5432
SECTION ESTABLISHED
ACCEPT   all       all
SECTION RELATED
ACCEPT   all       all
Once the files are created, Shorewall can be easily activated with: 
# shorewall compile && shorewall restart
strongSwan IPsec VPN setup Like Shorewall, strongSwan is also very well documented and I'm just going to focus on those specific areas that are relevant to this type of VPN project.
  • Allow the road-warriors to send all browsing traffic over the VPN means including leftsubnet=0.0.0.0/0 in the VPN server's /etc/ipsec.conf file. Be wary though: sometimes the road-warriors start sending the DHCP renewal over the tunnel instead of to their local DHCP server.
  • As we are using Shorewall zones for firewalling, you must set the options leftfirewall=no and lefthostaccess=no in ipsec.conf. Shorewall already knows about the remote networks as they are defined in the /etc/shorewall/hosts file and so firewall rules don't need to be manipulated each time a tunnel goes up or down.
  • As discussed above, X.509 certificates are used for peer authentication. In the certificate Distinguished Name (DN), store the zone name in the Organizational Unit (OU) component, for example, OU=vpn_c, CN=gw.branch1.example.org
  • In the ipsec.conf file, match the users to connections using wildcard specifications such as rightid="OU=vpn_a, CN=*"
  • Put a subjectAltName with hostname in every certificate. The --san option to the ipsec pki commands adds the subjectAltName.
  • Keep the certificate distinguished names (DN) short, this makes the certificate smaller and reduces the risk of fragmented authentication packets. Many examples show a long and verbose DN such as C=GB, O=Acme World Wide Widget Corporation, OU=Engineering, CN=laptop1.eng.acme.example.org. On a private VPN, it is rarely necessary to have all that in the DN, just include OU for the zone name and CN for the host name.
  • As discussed above, use the ECDSA scheme for keypairs (not RSA) to ensure that the key exchange datagrams don't suffer from fragmentation problems. For example, generate a keypair with the command ipsec pki --gen --type ecdsa --size 384 > user1Key.der
  • Road warriors should have leftid=%fromcert in their ipsec.conf file. This forces them to use the Distinguished Name and not the subjectAltName (SAN) to identify themselves.
  • For road warriors who require virtual tunnel IPs, configure them to request both IPv4 and IPv6 addresses (dual stack) with leftsourceip=%config4,%config6 and on the VPN gateway, configure the ranges as arguments to rightsourceip
  • To ensure that roadwarriors query the LAN DNS, add the DNS settings to strongswan.conf and make sure road warriors are using a more recent strongSwan version that can dynamically update /etc/resolv.conf. Protecting DNS traffic is important for privacy reasons. It also makes sure that the road-warriors can discover servers that are not advertised in public DNS records.
Central firewall/VPN gateway configuration Although they are not present in the diagram, IPv6 networks are also configured in these strongSwan examples. It is very easy to combine IPv4 and IPv6 into a single /etc/ipsec.conf file. As long as the road-warriors have leftsourceip=%config4,%config6 in their own configurations, they will operate dual-stack IPv4/IPv6 whenver they connect to the VPN. Here is an example /etc/ipsec.conf for the central VPN gateway: 
config setup
        charonstart=yes
        charondebug=all
        plutostart=no
conn %default
        ikelifetime=60m
        keylife=20m
        rekeymargin=3m
        keyingtries=1
        keyexchange=ikev2
conn branch1
        left=198.51.100.1
        leftsubnet=203.0.113.0/24,10.0.0.0/8,2001:DB8:12:80:/64
        leftcert=fw1Cert.der
        leftid=@fw1.example.org
        leftfirewall=no
        lefthostaccess=no
        right=%any
        rightid=@branch1-fw.example.org
        rightsubnet=192.168.1.0/24
        auto=add
conn rw_vpn_a
        left=198.51.100.1
        leftsubnet=0.0.0.0/0,::0/0
        leftcert=fw1Cert.der
        leftid=@fw1.example.org
        leftfirewall=no
        lefthostaccess=no
        right=%any
        rightid="OU=vpn_a, CN=*"
        rightsourceip=10.1.100.0/24,2001:DB8:1000:100::/64
        auto=add
conn rw_vpn_b
        left=198.51.100.1
        leftsubnet=0.0.0.0/0,::0/0
        leftcert=fw1Cert.der
        leftid=@fw1.example.org
        leftfirewall=no
        lefthostaccess=no
        right=%any
        rightid="OU=vpn_b, CN=*"
        rightsourceip=10.1.200.0/24,2001:DB8:1000:200::/64
        auto=add
Sample branch office or home router VPN configuration Here is an example /etc/ipsec.conf for the Linux server or OpenWRT VPN at the branch office or home: 
conn head_office
        left=%defaultroute
        leftid=@branch1-fw.example.org
        leftcert=branch1-fwCert.der
        leftsubnet=192.168.1.0/24,2001:DB8:12:80:/64
        leftfirewall=no
        lefthostaccess=no
        right=fw1.example.org
        rightid=@fw1.example.org
        rightsubnet=203.0.113.0/24,10.0.0.0/8,2001:DB8:1000::/52
        auto=start
# notice we only allow vpn_b users, not vpn_a
# these users are given virtual IPs from our own
# 192.168.1.0 subnet
conn rw_vpn_b
        left=branch1-fw.example.org
        leftsubnet=192.168.1.0/24,2001:DB8:12:80:/64
        leftcert=branch1-fwCert.der
        leftid=@branch1-fw.example.org
        leftfirewall=no
        lefthostaccess=no
        right=%any
        rightid="OU=vpn_b, CN=*"
        rightsourceip=192.168.1.160/27,2001:DB8:12:80::8000/116
        auto=add
Further topics Shorewall and Shorewall6 don't currently support a unified configuration. This can make it slightly tedious to duplicate rules between the two IP variations. However, the syntax for IPv4 and IPv6 configuration is virtually identical. Shorewall only currently supports Linux netfilter rules. In theory it could be extended to support other types of firewall API, such as pf used by OpenBSD and the related BSD family of systems. A more advanced architecture would split the single firewall into multiple firewall hosts, like the inner and outer walls of a large castle. The VPN gateway would also become a standalone host in the DMZ. This would require more complex routing table entries. Smart cards with PIN numbers provide an effective form of two-factor authentication that can protect certificates for remote users. Smart cards are documented well by strongSwan already so I haven't repeated any of that material in this article. Managing a private X.509 certificate authority in practice may require slightly more effort than I've described, especially when an organisation grows. Small networks and home users don't need to worry about these details too much, but for most deployments it is necessary to consider things like certificate revocation lists and special schemes to protect the root certificate's private key. EJBCA is one open source project that might help. Some users may want to consider ways to prevent the road-warriors from accidentally browsing the Internet when the IPsec tunnel is not active. Such policies could be implemented with some basic iptables firewalls rules in the road-warrior devices. Summary Using these strategies and configuration tips, planning and building a VPN will hopefully be much simpler. Please feel free to ask questions on the mailing lists for any of the projects discussed in this blog.

Keith Packard: Present

The Present Extension I ve finally gotten an initial implementation of the Present extension written and running and thought I should write up the current design and status. Present Design The current Present extension consists of two requests:
  1. PresentRegion. Puts new bits from a pixmap in a window.
  2. PresentSelectInput. Asks for Present events to be delivered.
PresentRegion This request takes a pile of arguments: 
 
    PresentRegion
    window: WINDOW
    pixmap: PIXMAP
    valid-area: REGION or None
    update-area: REGION or None
    x-off, y-off: INT16
    target-msc: CARD64
    divisor: CARD64
    remainder: CARD64
    idle-fence: FENCE
 
Errors: Drawable, Pixmap, Match
Provides new content for the specified window, to be made visible at the specified time (defined by target-msc , divisor and remainder ). update-area defines the subset of the window to be updated, or None if the whole window is to be updated. valid-area defines the portion of pixmap which contains valid window contents, or None if the pixmap contains valid contents for the whole window. PresentRegion may use any region of pixmap which contains update-area and which is contained by valid-area . In other words, areas inside update-area will be presented from pixmap , areas outside valid-area will not be presented from pixmap and areas inside valid-area but outside update-area may or may not be presented at the discretion of the X server. x-off and y-off define the location in the window where the 0,0 location of the pixmap will be presented. valid-area and update-area are relative to the pixmap. If target-msc is greater than the current msc for window , the presentation will occur at (or after) the target-msc field. Otherwise, the presentation will occur after the next field where msc % divisor == remainder . If divisor is zero, then the presentation will occur after the current field. idle-fence is triggered when pixmap is no longer in use. This may be at any time following the PresentRegion request, the contents may be immediately copied to another buffer, copied just in time for the vblank interrupt or the pixmap may be used directly for display, in which case it will be busy until some future PresentRegion operation. If window is destroyed before the presentation occurs, then the presentation action will not be completed. PresentRegion holds a reference to pixmap until the presentation occurs, so pixmap may be immediately freed after the request executes, even if that is before the presentation occurs. If idle-fence is destroyed before the presentation occurs, then idle-fence will not be signaled but the presentation will occur normally. PresentSelectInput 
 
    PresentSelectInput
    event-id: PRESENTEVENTID
    window: WINDOW
    eventMask: SETofPRESENTEVENT
 
Errors: Window, Value, Match, IDchoice, Access
Selects the set of Present events to be delivered for the specified window and event context. PresentSelectInput can create, modify or delete event contexts. An event context is associated with a specific window; using an existing event context with a different window generates a Match error. If eventContext specifies an existing event context, then if eventMask is empty, PresentSelectInput deletes the specified context, otherwise the specified event context is changed to select a different set of events. If eventContext is an unused XID, then if eventMask is empty no operation is performed. Otherwise, a new event context is created selecting the specified events. Present Extension Events There are three different events for the Present extension:
  1. PresentConfigureNotify
  2. PresentCompleteNotify
  3. PresentRedirectNotify
PresentConfigureNotify This event is moving from the DRI3 extension where it doesn t belong. 
 
    PresentConfigureNotify
    type: CARD8         XGE event type (35)
    extension: CARD8        Present extension request number
    length: CARD16          2
    evtype: CARD16          Present_ConfigureNotify
    eventID: PRESENTEVENTID
    window: WINDOW
    x: INT16
    y: INT16
    width: CARD16
    height: CARD16
    off_x: INT16
    off_y: INT16
    pixmap_width: CARD16
    pixmap_height: CARD16
    pixmap_flags: CARD32
PresentConfigureNotify events are sent when the window configuration changes if PresentSelectInput has requested it. PresentConfigureNotify events are XGE events and so do not have a unique event type. x and y are the parent-relative location of window . PresentCompleteNotify 
 
    PresentCompleteNotify
    type: CARD8         XGE event type (35)
    extension: CARD8        Present extension request number
    length: CARD16          2
    evtype: CARD16          Present_CompleteNotify
    eventID: PRESENTEVENTID
    window: WINDOW
    ust: CARD64
    msc: CARD64
    sbc: CARD64
Notify events are delivered when a PresentRegion operation has completed and the specified contents are being displayed. sbc, msc and ust indicate the swap count, frame count and system time of the related PresentRegion request. PresentRedirectNotify This one is not specified yet, but the intent is for it to contain sufficient information for the compositing manager to be able to construct a suitable screen update that includes an application window update. Finishing GLXOMLsync_control At this point, Mesa only exposes the old swapinterval configuration value, it doesn t provide any of the GLXOMLsynccontrol APIs. However, the Present protocol does have the bits necessary to support glXSwapBuffersMscOML in the PresentRegion request. Let s see how the remaining APIs in this GL extension will be supported. glXGetSyncValuesOML This one is easy; it only needs to return the UST/MSC/SBC values from the most recent SwapBuffers request. Those are returned in the PresentCompleteNotify event, so Mesa just needs to capture that event and save the values away for return to the application. We don t need any new Present protocol for this. glXGetMscRateOML This returns the refresh rate of the monitor associated with the specified drawable. RandR exposes all of the necessary data for each monitor, but the monitor each window is going to be synchronized against isn t exposed anywhere, and is effectively implementation-dependent. So, I think the easy thing to do here is to have a Present request which reports which RandR output a window will be synchronized with. glXSwapBuffersMscOML This is the one API which is already directly supported by the Present extension. glXWaitForMscOML This is effectively the same as glXSwapBuffersMscOML, except that it doesn t actually perform a swap. For this, I think we want a new request that generates an X event when the target values are reached, and then have the client block until that event is received. This will avoid blocking other threads using the same X connection. glXWaitForSbcOML Like glXWaitForMscOML, this just needs to trigger an event to be delivered at the right time. Presentation Redirection I m focusing on finishing the above stuff before I start writing the redirection spec and code, but I ve been thinking a bit about it. What we want is for applications that are presenting a new frame within a redirected window to have that presentation delivered directly to the compositing manager instead of having the bits copied to the redirected buffer and have the compositing manager only learn about this when the damage event is received from copying the bits. Once the compositing manager receives notification that there are new bits available for a portion of a window, it can then get those bits onto the screen in one of two ways:
  1. Copy them to the back buffer for the whole screen and then present that back buffer to the screen.
  2. Present them directly to the screen, bypassing the compositing manager s back buffer entirely.
The first option is what you d do if there were more updates than just a single window; it would update the whole screen at the same time. The second option is what you d do if there was only the one window update to act on, and this would automatically take advantage of page flipping to avoid copying data at all. No need to un-redirect windows for the page flip to work. In both cases, we need to inform the original application both when its pixmap is idle and when the swap actually hits the screen. And we need to make sure the final swap happens when the application requested. For the pixmap idle notification, the current PresentRegion idlefence argument should suffice; we just need to pass the idlefence XID along in the redirection event. Simple. For completion notification, we need to make sure the SBC value for the original window gets incremented and that a PresentCompleteNotify event is delivered. I think that means we want to append a chunk of data to the PresentRegion request so that suitable events can be delivered when the compositing manager PresentRegion occur. I think that just needs to be the window of that original window. To make sure the swap happens at the right time, we just need to have the targetmsc value provided to the compositing manager. I m hoping that having this all event driven is that when a later PresentRegion is redirected that has an *earlier* targetmsc, we can simply queue that as well and things should just work . Current Status I ve got the protocol definitions (both X server and XCB) done, and libxcb supporting the extension. I ve got the X server bits working, but the actual updates are not synchronized to the monitor; they re using OS timers and CopyArea for now, mostly so I can test things without also hacking drivers. Mesa is using the extension, but it only provides the swapinterval value and is not yet supporting the full GLXOMLsynccontrol extension. I ve also got a simple 2D core X application using the extension which is in the shmfd repository in my home directory on freedesktop.org. Availability As always, these bits are already published in my home directory on freedesktop.org in various git repositories.

11 July 2013

Johannes Schauer: Using botch to generate transition build orders

In October 2012, Joachim Breitner asked me whether botch (well, it didnt have a name back then) can also be used to calculate a build order for recompiling ~450 haskell-packages with a new ghc version (it was probably the 7.6.1 release) to upload them to experimental. What is still blocking this ability is the inability of botch to directly read *.dsc files instead of having to rely on Packages and Sources files. On the other hand (in case there exists a set of Packages and Sources files) it is now much easier to use botch for such a use case for which it was not originally designed. To demonstrate how botch can be used to calculate the build order for library transitions, I wrote the script create-transition-order.sh which executes the individual tools in the correct order with the correct arguments. To validate the correctness of botch, I compared the output to the order which ben produces for transitions. The create-transition-order.sh script is called with a ben query string and optionally a snapshot.debian.org timestamp as its arguments. The script relies on ben being installed because ben query strings cannot be translated into grep-dctrl query strings as ben query splits fields which contain comma separated values at the comma before searching for strings. Unfortunately the script also currently relies on a yet unreleased ben feature which allows ben download to create a ben.cache file. You can track the progress of this feature as bug#714703. Creating a ben.cache file is necessary because some queries rely on an association between binary and source packages which is not present in all packages in a Packages file. For example the haskell transition query makes use of this feature by including the .source field in its query. The result of these trials was, that botch produced nearly the same build order for nearly all transitions. The only differences are due to shortcomings of ben and botch. For example, ben is not able to create an order for transitions where involved packages form one or more cycles. A prominent example is the haskell transition where ghc itself is only built during step 15 after many other packages which would have needed ghc to be compiled. Botch solves this problem by reducing all strongly connected components in a cyclic graph to a single vertex before creating the order. This operation makes the graph acyclic and creating a build order trivial. The only remaining problems can then be found within the strongly connected components (for Haskell they are of maximum size two) but the overall build order is correct. On the other hand botch has no notion of packages which are affected by a transition and thus creates a build order which in some cases is longer than the one created by ben. When generating the interdependencies between packages, ben only considers those which are part of the transition. Botch on the other hand considers all dependency relationships. It would be simple to solve this issue in botch by removing unaffected packages from the dependency graph through edge contraction (an operation already used by botch for other tasks). This exercise also let me find another bug in dose3 where libdose would not associate a binary package with a binNMU version with its associated source package but instead report a version mismatch error. This problem was also reported in the process.

Daniel Pocock: Using debcheckout to build strongSwan 5.0 on Debian wheezy

Normally most people try to use packages from a stable Linux distribution because of their convenience and security support. Sometimes it's necessary to use a newer version, I recently had to do this with strongSwan and I'm sharing the procedure for other people to try. These days, it is actually very very easy to build a new version of a package from source and this procedure will work for many Debian packages. Debian wheezy ships with strongSwan version 4.5 packages. These are lacking support for a couple of features I needed, including dual-stack IPv4/IPv6 virtual IPs on a single user connection and maintaining the DNS server list on road-warriors Oddly enough, for a variety of reasons, I had to rebuild strongSwan on all the platforms where I wanted to use it (not just on Debian). I recently described how to build it with ECDSA support on Fedora and also how to build it with ECDSA support on OpenWRT. Following all of these procedures you will end up with strongSwan packages on all systems that can interoperate using dual stack IPv4, IPv6 and establish connections quickly, securely and reliably using ECC certificates. Get the build dependencies Many packages require extra tools and header files when building. These are packages that are not usually installed or required when running the package. However, they are very easy to get. For strongSwan: 
sudo apt-get update
sudo apt-get install devscripts fakeroot
sudo apt-get build-dep strongswan
This should download and install the necessary dependencies. Notice we also installed the devscripts package, it provides the very handy debcheckout utility. Obtain and build the package Now for the real work. Fortunately this is not too hard either: 
mkdir ~/ws
cd ~/ws
debcheckout strongswan
cd strongswan
dpkg-buildpackage -rfakeroot -uc -b
Look in the directory ~/ws and you will find the .deb package files. Problems? On my own system, I observed that I could upgrade from the existing packages to the v5.0.4-6 package without any change to the configuration files. It just worked. Hopefully, these packages will eventually end up in Debian's backports catalog where you can install them without compiling. Nonetheless, problems for these packages can still be submitted through Debian's bug tracker - just make sure you clearly indicate that you built the package from source and give the git commit number (output of the command cd ~/ws/strongswan && git show head -1) Before trying the package and definitely before reporting any problem, it is worth checking the bug tracker and the Package Tracking System for known issues

Russ Allbery: kadmin-remctl 3.4

kadmin-remctl is a wrapper in front of the Kerberos kadmin protocol that exports many operations via remctl and adds some additional synchronization functionality. The primary purpose of this release is to add disallow-svr to the default flags for all newly-created principals. This prohibits obtaining service tickets for the principal, which provides some hardening against brute force attacks. Since the create command is designed for creation of user principals, not service principals, and use of service tickets for user principals is quite obscure and rare in Kerberos, this seems like a better default. This release also relaxes the default allowed principal regex to allow two-character user principals. We mostly prohibit them, but we had some old, disabled ones and at least one of those users returned. This is just a default that can be overridden by other sites. This package, as I say most times when I release it, is badly in need of a complete overhaul, a switch to JSON for the attribute format, conversion to a saner Perl kadmin API, and various other improvements. One of these days.... You can get the latest version from the kadmin-remctl distribution page.

9 July 2013

Wouter Verhelst: Mark groet 's morgens de dingen

Or, Mark greeting the things in the morning
Dag ventje met de fiets op de vaas met de bloem Hi little guy with the bike on the vase with the flower
ploem ploem ploum ploum
Dag stoel naast de tafel Hi chair besides the table
Dag brood op de tafel Hi bread on the table
Dag visserke-vis met de pijp Hi fishery-man with the pipe
en and
Dag visserke-vis met de pet Hi fishery-man with the hat
goeiendag good day
Daa-ag vis Bye-ee fish
Dag lieve vis Bye dear fish
Dag klein visselijn mijn Bye my little fishery-fish
The above is one of Paul van Ostaijen's poems, and is, in my opinion, one of his finest. I tried translating it to English so that my non-Dutch-speaking readers could appreciate it (Dutch-speaking readers will most likely already know it anyway), but it's almost impossible to properly translate it. In this poem, van Ostaijen invents a few words that aren't part of the Dutch language proper. A good example is 'visserke-vis'. The first part, 'visserke', is (now) a dialect diminutive of 'visser', which translates to "fisherman" (though it is likely that in Van Ostaijen's days, 'visserke' wasn't considered to be a dialect word). The second part, 'vis', is just Dutch for 'fish'. A literal translation would thus be 'fishereman-fish' On the other hand, given the relative similarity of both words, it is conceivable that Van Ostaijen is just playing with the sound of those words here (he does that later in the poem, anyway), and as such, it's not entirely clear whether he's talking about a fish with a pipe and a had, or about a fisherman with a pipe and a hat. It could be both. His use of the word 'Dag' adds to confusion. In Dutch, the word 'Dag' can be used as a greeting when seeing people for the first time in the morning, but it can also be used as a greeting when leaving people in the evening. Since this is used near the end of the poem, I translated it by using its second meaning; but it's also possible that Van Ostaijen is just playing with us. Of that final line, I should add that the size of the letters is not my doing; Van Ostaijen's original version contained this. In fact, he is known for his expressionistic way of using letters, and some of his other poems go into extremes with that; his wikipedia entry has a nice example. The final verse of the poem sounds pretty well, but defies all grammar. In proper grammar, the verse would read "Dag mijn klein visje". Van Ostaijen turns 'visje' into 'visselijn' so that it rhymes on 'mijn', and puts 'mijn' at the end. Whereas it's still possible to figure out a meaning of the first part of 'visserke-vis', this certainly isn't possible with 'visselijn'. It just isn't a word; all you can make of it is that he's talking about a fish. This certainly contradicts the idea that he was talking about a fisherman earlier. All in all, "Mark groet 's morgens de dingen" is a very funny poem; it describes a man or a boy who does nothing else than say hello to objects when he gets up in the morning; and though it's necessary to explain the language if you don't speak Dutch to fully appreciate it, the idea that there's much more to be found behind the words is most likely wrong. This is just silly amusement.

Daniel Pocock: Enabling Elliptic Curve Cryptography in OpenWRT and strongSwan VPNs

OpenWRT currently ships an OpenSSL package with Elliptic Curve Cryptography (ECC) disabled. This is very inconvenient as ECC is now standard in other distributions like Debian and Ubuntu and it is necessary to solve certain problems such as making IPsec VPNs work reliably It is relatively easy to enable it, this also provides a useful opportunity to see how the OpenWRT build system works. Preparing the build environment The build was done on a Debian 7 (wheezy) workstation. It is necessary to install some packages that provide tools needed for the build: 
apt-get install gcc binutils flex make unzip gawk libz-dev linux-libc-dev
Rebuild the OpenSSL and StrongSwan packages This was tested with an ar71xx CPU-based router. You can easily change the config (run make menuconfig just before make defconfig) for other variations of router.
Running strongSwan IPsec on a router requires a router with plenty of flash, RAM and CPU power. Cheap routers with limited flash will not be able to install the packages at all. I tested on the Buffalo WZR-HP-AG300H-EU which has 32MB flash and 128MB RAM Here are the commands executed on the Debian build workstation: 
mkdir ~/ws
cd ~/ws
svn checkout svn://svn.openwrt.org/openwrt/trunk openwrt-trunk
cd openwrt-trunk
./scripts/feeds update -a
./scripts/feeds install -a
wget -O .config \
  http://danielpocock.com/sites/danielpocock.com/files/openwrt.config.txt
sed -i -e 's/no-ec//' package/libs/openssl/Makefile 
make defconfig
make -j 13
The "make" command will build all the packages. It takes some time. Copy the packages to a router The actual location of the packages in your source tree depends on the CPU architecture you chose to build for. You can find them with a command such as: 
find . -name '*.ipk'
Here we look at how to copy the packages for the ar71xx based router into the /tmp directory on the router. The router IP is 192.168.1.1 below: 
cd ./bin/ar71xx/packages/
scp libopenssl_1.0.1e-1_ar71xx.ipk strongswan*.ipk root@192.168.1.1:/tmp
Installing the packages It is relatively easy to install the packages using the opkg utility on OpenWRT. When using self-compiled packages, you are likely to get errors about the checksums. OpenWRT compares the checksum of the package to the checksum from the package catalog file. You can delete the catalog file (e.g. rm /var/opkg-lists/attitude_adjustment) and then opkg will install the package without complaining. The file is automatically placed there each time you run opkg update so there is no harm in deleting it. In my earlier blog, I looked at how to link Android clients to the strongSwan VPN. Unfortunately Android has a bug in ECC support too and must still be used with RSA certificates. Nonetheless, the OpenWRT VPN will be quite happy talking to Linux laptops and servers - you may need to rebuild OpenSSL and StrongSwan on any Fedora, Red Hat or CentOS systems that require ECC support too. As you migrate from RSA to ECC certificates it is quite feasible to have both types of certificates in use at the same time. strongSwan supports this type of hybrid environment. To make the migration to ECC go smoothly it is desirable to ensure all boxes have ECC support at the earliest opportunity.

6 July 2013

Yves-Alexis Perez: Xfce 4.10, final part

Someone recently asked me about the Debian Xfce 4.10 status, as apparently I forgot to update this post series. So, as you might have noticed, the full Xfce 4.10 desktop environment is currently in Debian Jessie (current name for testing). All in all, the transition went well and smooth. One of the most regular question I get about Xfce 4.10 is the panel behavior when it comes to the task bar expansion. In xfce4-panel 4.8, when people wanted to have a full side panel with a task bar plugin inside, they added a Windows buttons plugin and configured the panel to 100% length. Then the Windows buttons would expand to occupy all the free space on the panel. It was a special case plugins, as usually other plugins only used a fixed space. Now, in 4.10, this is not the case anymore. Windows button uses a fixed size. And the plugins are left-aligned, which means usually people end up with some space at the far right of the panel. To restore the previous behavior back (which is actually the pre-4.8 behavior, 4.8 was an exception by itself), one needs to add a Separator plugin, then configure it to expand (and optionnally select a transparent handle). Then move it next right to the Windows buttons plugin. Another thing which might be a bit surprising for upgrading users is the change in the Action buttons plugin, which people usually use to logout. In 4.8, by default, it's set to run the logout dialog. In 4.10, by default, it's set to logout directly (but with a confirmation dialog). If you prefer the previous behavior, you can just configure the Action buttons plugin and select the Log out item instead of the Log out one (I know, it's a bit confusing). If you have any question, don't hesitate to reach us by mail or on irc (#debian-xfce on Freenode). Other than that remember that Xfce really needs you help, both in Debian and upstream (and at that point, I'd say *especially* upstream). There's a lot of unmaintained project under the Xfce umbrella, some of them part of the core (like xfce4-power-manager), so if you have some spare time and some C/GTK+ knowledge, feel free to contact the Xfce team on the mailing list.

5 July 2013

Justus Winter: mount, umount and mtab

This week was all about getting mount into shape, addind umount and hacking up a prototype for the mtab issue. With all my patches to mount the boot log looks like this: 
INIT: version 2.88 booting
Using makefile-style concurrent boot in runlevel S.
Activating swap...done.
Checking root file system...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
ext2fs_check_if_mount: Can't check if filesystem is mounted due to missing mtab file while determining whether /dev/hd0s1 is mounted.
/dev/hd0s1: clean, 44101/181056 files, 283413/723200 blocks
done.
Cleaning up temporary files... /tmp.
/etc/rcS.d/S06mtab.sh: 48: /etc/rcS.d/S06mtab.sh: cannot open /proc/mounts: No such file
Activating lvm and md swap...(default pager): Already paging to partition hd0s5!
done.
Checking file systems...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
done.
Mounting local filesystems...done.
Activating swapfile swap...(default pager): Already paging to partition hd0s5!
done.
df: Warning: cannot read table of mounted file systems: No such file or directory
Cleaning up temporary files....
Configuring network interfaces...inetutils-ifconfig: invalid arguments
ifup: failed to open pid file /run/network/ifup-/dev/eth0.pid: No such file or directory
Internet Systems Consortium DHCP Client 4.2.2
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
Listening on Socket//dev/eth0
Sending on   Socket//dev/eth0
DHCPDISCOVER on /dev/eth0 to 255.255.255.255 port 67 interval 7
DHCPREQUEST on /dev/eth0 to 255.255.255.255 port 67
DHCPOFFER from 10.0.2.2
DHCPACK from 10.0.2.2
can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
bound to 10.0.2.15 -- renewal in 42107 seconds.
done.
Cleaning up temporary files....
Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix.
INIT: Entering runlevel: 2
Using makefile-style concurrent boot in runlevel 2.
Starting enhanced syslogd: rsyslogd.
Starting deferred execution scheduler: atd.
Starting periodic command scheduler: cron.
Starting system message bus: dbusFailed to set socket option"/var/run/dbus/system_bus_socket": Protocol not available.
Starting OpenBSD Secure Shell server: sshd.
GNU 0.3 (debian) (console)
login:
As you can see, mountall.sh works fine now (it's the line about mounting local filesystems). This is a list of issues I tackled this week: Note that even though I've implemented umount, it doesn't actually work yet. That is because umount needs a mtab file or /proc/mounts to do anything, because you can either invoke umount with the mount point or the device and umount needs an mtab style file to figure out whether the argument is referring to the former or the latter. So the next big issue is the mtab issue. There is some information about that in the Hurd wiki, it even is a gsoc project idea of its own: http://www.gnu.org/software/hurd/community/gsoc/project_ideas/mtab.html But I need this now, this cannot wait for another gsoc. The implementation outlined by that page consists roughly of: If this hypothetical mtab translator is bound to say /proc/mounts this would solve all our problems: 
% settrans -ca /proc/mounts /hurd/mtab /
As this involves adding a RPC procedure and keeping a list of passive and active translators, this involves some design decisions. As discussed in #hurd I hacked up a prototype and post it for review: http://lists.gnu.org/archive/html/bug-hurd/2013-07/msg00022.html As soon as I get the necessary feedback on this issue, I will get the implementation up to speed. Until then, I will pick another item from my status list, most probably the ifupdown issue. Speaking of feedback, all of my patches I sent last week were merged at the appropriate locations. That was really quick, many thanks for that :) While working on the mount issues, I frequently looked at the implementation of Linux' and FreeBSD's mount to see how --types is handled there. I must admit that I was not to keen about the idea of writing c again, but I was positively surprised by the code I've seen from Hurd so far. As Hurd is the operating system of the GNU system, its code is free to use any GNU extension to libc and gcc including nice stuff like nested functions. If you look at the different mount implementations, you'll find the Hurd version to be the most modern and clean one imho:
Linux
http://sources.debian.net/src/util-linux/2.20.1-5.4/mount/mount.c
FreeBSD
http://sources.debian.net/src/freebsd-utils/9.0%2Bds1-11/sbin/mount/mount.c
Hurd
http://sources.debian.net/src/hurd/20130620-1/utils/mount.c
See you next week :)

4 July 2013

Julien Danjou: OpenStack meets Lisp: cl-openstack-client

A month ago, a mail hit the OpenStack mailing list entitled "The OpenStack Community Welcomes Developers in All Programming Languages". You may know that OpenStack is essentially built using Python, and therefore it is the reference language for the client libraries implementations. As a Lisp and OpenStack practitioner, I used this excuse to build a challenge for myself: let's prove this point by bringing Lisp into OpenStack!
Welcome cl-openstack-client, the OpenStack client library for Common Lisp! The project is hosted on the classic OpenStack infrastructure for third party project, StackForge. It provides the continuous integration system based on Jenkins and the Gerrit infrastructure used to review contributions. How the tests works OpenStack projects ran a fabulous contribution workflow, which I already talked about, based on tools like Gerrit and Jenkins. OpenStack Python projects are used to run tox, to build a virtual environment and run test inside. We don't have such thing in Common Lisp as far as I know, so I had to build it myself. Fortunately, using Quicklisp, the fabulous equivalent of Python's PyPI, it has been a breeze to set this up. cl-openstack-client just includes a basic shell script that does the following: I just run the test using SBCL, though adding more compiler on the table would be a really good plan in the future, and should be straightforward. You can admire a log from a successful test run done when I proposed a patch via Gerrit, to check what it looks like. Implementation status For the curious, here's an example of how it works: 
* (require 'cl-openstack-client)
* (use-package 'cl-keystone-client)
* (defvar k (make-instance 'connection-v2 :username "demo" :password "somepassword" :tenant-name "demo" :url "http://devstack:5000"))
 
K
 
* (authenticate k)
 
((:ISSUED--AT . "2013-07-04T05:59:55.454226")
 (:EXPIRES . "2013-07-05T05:59:55Z")
 (:ID
  . "wNFQwNzo1OTo1NS40NTQyMthisisaverylongtokenwNFQwNzo1OTo1NS40NTQyM")
 (:TENANT (:DESCRIPTION) (:ENABLED . T)
  (:ID . "1774fd545df4400380eb2b4f4985b3be") (:NAME . "demo")))
 
* (connection-token-id k)
 
"wNFQwNzo1OTo1NS40NTQyMthisisaverylongtokenwNFQwNzo1OTo1NS40NTQyM"

Unfortunately, the implementation is far from being complete. It only implements for now the Keystone token retrieval. I've actually started this project to build an already working starting point. With this, future potential contributors will be able to spend efforts on writing code, and not on setting up the basic continuous integration system or module infrastructure. If you wish to help me and contribute, just follow the OpenStack Gerrit workflow howto or feel free to come by me and ask any question (I'm hanging out on #lisp on Freenode too). See you soon, hopping to bring more Lisp into OpenStack!

Julien Danjou: OpenStack meets Lisp: cl-openstack-client

A month ago, a mail hit the OpenStack mailing list entitled "The OpenStack Community Welcomes Developers in All Programming Languages". You may know that OpenStack is essentially built using Python, and therefore it is the reference language for the client libraries implementations. As a Lisp and OpenStack practitioner, I used this excuse to build a challenge for myself: let's prove this point by bringing Lisp into OpenStack!
Welcome cl-openstack-client, the OpenStack client library for Common Lisp! The project is hosted on the classic OpenStack infrastructure for third party project, StackForge. It provides the continuous integration system based on Jenkins and the Gerrit infrastructure used to review contributions. How the tests works OpenStack projects ran a fabulous contribution workflow, which I already talked about, based on tools like Gerrit and Jenkins. OpenStack Python projects are used to run tox, to build a virtual environment and run test inside. We don't have such thing in Common Lisp as far as I know, so I had to build it myself. Fortunately, using Quicklisp, the fabulous equivalent of Python's PyPI, it has been a breeze to set this up. cl-openstack-client just includes a basic shell script that does the following: I just run the test using SBCL, though adding more compiler on the table would be a really good plan in the future, and should be straightforward. You can admire a log from a successful test run done when I proposed a patch via Gerrit, to check what it looks like. Implementation status For the curious, here's an example of how it works: 
* (require 'cl-openstack-client)
* (use-package 'cl-keystone-client)
* (defvar k (make-instance 'connection-v2 :username "demo" :password "somepassword" :tenant-name "demo" :url "http://devstack:5000"))
 
K
 
* (authenticate k)
 
((:ISSUED--AT . "2013-07-04T05:59:55.454226")
 (:EXPIRES . "2013-07-05T05:59:55Z")
 (:ID
  . "wNFQwNzo1OTo1NS40NTQyMthisisaverylongtokenwNFQwNzo1OTo1NS40NTQyM")
 (:TENANT (:DESCRIPTION) (:ENABLED . T)
  (:ID . "1774fd545df4400380eb2b4f4985b3be") (:NAME . "demo")))
 
* (connection-token-id k)
 
"wNFQwNzo1OTo1NS40NTQyMthisisaverylongtokenwNFQwNzo1OTo1NS40NTQyM"

Unfortunately, the implementation is far from being complete. It only implements for now the Keystone token retrieval. I've actually started this project to build an already working starting point. With this, future potential contributors will be able to spend efforts on writing code, and not on setting up the basic continuous integration system or module infrastructure. If you wish to help me and contribute, just follow the OpenStack Gerrit workflow howto or feel free to come by me and ask any question (I'm hanging out on #lisp on Freenode too). See you soon, hopping to bring more Lisp into OpenStack!

Next.

Previous.