Earlier this year, I helped plan and run the Community Data Science Workshops: a series of three (and a half) day-long workshops designed to help people learn basic programming and tools for data science tools in order to ask and answer questions about online communities like Wikipedia and Twitter. You can read our initial announcement for more about the vision. The workshops were organized by myself, Jonathan Morgan from the Wikimedia Foundation, long-time Software Carpentry teacher Tommy Guy, and a group of 15 volunteer mentors who taught project-based afternoon sessions and worked one-on-one with more than 50 participants. With overwhelming interest, we were ultimately constrained by the number of mentors who volunteered. Unfortunately, this meant that we had to turn away most of the people who applied. Although it was not emphasized in recruiting or used as a selection criteria, a majority of the participants were women. The workshops were all free of charge and sponsored by the UW Department of Communication, who provided space, and the eScience Institute, who provided food. The curriculum for all four session session is online:

• Friday April 4th: Setup and Programming Practice
• Saturday April 5th: Introduction to Python
• Saturday May 3rd: Building data sets using web APIs
• Saturday May 31st: Data analysis and visualization

The workshops were designed for people with no previous programming experience. Although most our participants were from the University of Washington, we had non-UW participants from as far away as Vancouver, BC. Feedback we collected suggests that the sessions were a huge success, that participants learned enormously, and that the workshops filled a real need in the Seattle community. Between workshops, participants organized meet-ups to practice their programming skills. Most excitingly, just as we based our curriculum for the first session on the Boston Python Workshop s, others have been building off our curriculum. Elana Hashman, who was a mentor at the CDSW, is coordinating a set of Python Workshops for Beginners with a group at the University of Waterloo and with sponsorship from the Python Software Foundation using curriculum based on ours. I also know of two university classes that are tentatively being planned around the curriculum. Because a growing number of groups have been contacting us about running their own events based on the CDSW and because we are currently making plans to run another round of workshops in Seattle late this fall I coordinated with a number of other mentors to go over participant feedback and to put together a long write-up of our reflections in the form of a post-mortem. Although our emphasis is on things we might do differently, we provide a broad range of information that might be useful to people running a CDSW (e.g., our budget). Please let me know if you are planning to run an event so we can coordinate going forward.

The annual Debian developer meeting took place in Portland, Oregon, 23 to 31 August 2014. DebConf14 attendees participated in talks, discussions, workshops and programming sessions. Video teams captured a lot of the main talks and discussions for streaming for interactive attendees and for the Debian video archive. Between the video, presentations, and handouts the coverage came from the attendees in blogs, posts, and project updates. We ve gathered a few articles for your reading pleasure: Gregor Herrmann and a few members of the Debian Perl group had an informal unofficial pkg-perl micro-sprint and were very productive. Vincent Sanders shared an inspired gift in the form of a plaque given to Russ Allbery in thanks for his tireless work of keeping sanity in the Debian mailing lists. Pictures of the plaque and design scheme are linked in the post. Vincent also shared his experiences of the conference and hopes the organisers have recovered. Noah Meyerhans adventuring to Debian by train, (Inter)netted some interesting IPv6 data for future road and railwarriors. Hideki Yamane sent a gentle reminder for English speakers to speak more slowly. Daniel Pocock posted of GSoC talks at DebConf14, highlights include the Java Project Dependency Builder and the WebRTC JSCommunicator. Thomas Goirand gives us some insight into a working task list of accomplishments and projects he was able to complete at DebConf14, from the OpenStack discussion to tasksel talks, and completion of some things started last year at DebConf13. Antonio Terceiro blogged about debci and the Debian Continuous Integration project, Ruby, Redmine, and Noosfero. His post also shares the atmosphere of being able to interact directly with peers once a year. Stefano Zacchiroli blogged about a talk he did on debsources which now has its own HACKING file. Juliana Louback penned: DebConf 2014 and How I Became a Debian Contributor. Elizabeth Krumbach Joseph s in-depth summary of DebConf14 is a great read. She discussed Debian Validation & CI, debci and the Continuous Integration project, Automated Validation in Debian using LAVA, and Outsourcing webapp maintenance. Lucas Nussbaum by way of a blog post releases the very first version of Debian Trivia modelled after the TCP/IP Drinking Game. Fran ois Marier s shares additional information and further discussion on Outsourcing your webapp maintenance to Debian. Joachim Breitner gave a talk on Haskell and Debian, created a new tool for binNMUs for Haskell packages which runs via cron job. The output is available for Haskell and for OCaml, and he still had a small amount of time to go dancing. Jaldhar Harshad Vyas was not able to attend DebConf this year, but he did tune in to the videos made available by the video team and gives an insightful viewpoint to what was being seen. J r my Bobbio posted about Reproducible builds in Debian in his recap of DebConf14. One of the topics at hand involved defining a canonical path where packages must be built and a BOF discussion on reproducible builds from where the conversation moved to discussions in both Octave and Groff. New helpers dh_fixmtimes and dh_genbuildinfo were added to BTS. The .buildinfo format has been specified on the wiki and reviewed. Lots of work is being done in the project, interested parties can help with the TODO list or join the new IRC channel #debian-reproducible on irc.debian.org. Steve McIntyre posted a Summary from the d-i / debian-cd BoF at DC14, with some of the session video available online. Current jessie D-I needs some help with the testing on less common architectures and languages, and release scheduling could be improved. Future plans: Switching to a GUI by default for jessie, a default desktop and desktop choice, artwork, bug fixes and new architecture support. debian-cd: Things are working well. Improvement discussions are on selecting which images to make I.E. netinst, DVD, et al., debian-cd in progress with http download support, Regular live test builds, Other discussions and questions revolve around which ARM platforms to support, specially-designed images, multi-arch CDs, and cloud-init based images. There is also a call for help as the team needs help with testing, bug-handling, and translations. Holger Levsen reports on feedback about the feedback from his LTS talk at DebConf14. LTS has been perceived well, fits a demand, and people are expecting it to continue; however, this is not without a few issues as Holger explains in greater detail the lacking gatekeeper mechanisms, and how contributions are needed from finance to uploads. In other news the security-tracker is now fixed to know about old stable. Time is short for that fix as once jessie is released the tracker will need to support stable, oldstable which will be wheezy, and oldoldstable. Jonathan McDowell s summary of DebConf14 includes a fair perspective of the host city and the benefits of planning of a good DebConf14 location. He also talks about the need for facetime in the Debian project as it correlates with and improves everyone s ability to work together. DebConf14 also provided the chance to set up a hard time frame for removing older 1024 bit keys from Debian keyrings. Steve McIntyre posted a Summary from the State of the ARM BoF at DebConf14 with updates on the 3 current ports armel, armhf and arm64. armel which targets the ARM EABI soft-float ARMv4t processor may eventually be going away, while armhf which targets the ARM EABI hard-float ARMv7 is doing well as the cross-distro standard. Debian is has moved to a single armmp kernel flavour using Device Tree Blobs and should be able to run on a large range of ARMv7 hardware. The arm64 port recently entered the main archive and it is hoped to release with jessie with 2 official builds hosted at ARM. There is talk of laptop development with an arm64 CPU. Buildds and hardware are mentioned with acknowledgements for donated new machines, Banana Pi boards, and software by way of ARM s DS-5 Development Studio - free for all Debian Developers. Help is needed! Join #debian-arm on irc.debian.org and/or the debian-arm mailing list. There is an upcoming Mini-DebConf in November 2014 hosted by ARM in Cambridge, UK. Tianon Gravi posted about the atmosphere and contrast between an average conference and a DebConf. Joseph Bisch posted about meeting his GSOC mentors, attending and contributing to a keysigning event and did some work on debmetrics which is powering metrics.debian.net. Debmetrics provides a uniform interface for adding, updating, and viewing various metrics concerning Debian. Harlan Lieberman-Berg s DebConf Retrospective shared the feel of DebConf, and detailed some of the work on debugging a build failure, work with the pkg-perl team on a few uploads, and work on a javascript slowdown issue on codeeditor. Ana Guerrero L pez reflected on Ten years contributing to Debian.

AddThis:

Categories:

• Debian-Blog
• Computing
• Tools

Keywords:

• laptop-mode-tools

The upcoming version of Lintian will feature a new set of API that attempts to promote safer code. It is hardly a ground-breaking discovery , just a much needed feature. The primary reason for this API is that writing safe and correct code is simply too complicated that people get it wrong (including yours truly on occasion). The second reason is that I feel it is a waste having to repeat myself when reviewing patches for Lintian. Fortunately, the kind of issues this kind of mistake creates are usually minor information leaks, often with no chance of exploiting it remotely without the owner reviewing the output first[0]. Part of the complexity of writing correct code originates from the fact that Lintian must assume Debian packages to be hostile until otherwise proven[1]. Consider a simplified case where we want to read a file (e.g. the copyright file):

package Lintian::cpy_check;
use strict; use warnings; use autodie;
sub run  
  my ($pkg, undef, $info) = @_;
  my $filename = "usr/share/doc/$pkg/copyright";
  # BAD: This is an example of doing it wrong
  open(my $fd, '<', $info->unpacked($filename));
  ...;
  close($fd);
  return;
 

This has two trivial vulnerabilities[2].

1. Any part of the path (usr,usr/share, ) can be asymlink to somewhere else like /
   1. Problem: Access to potentially any file on the system with the credentials of the user running Lintian. But even then, Lintian generally never write to those files and the user has to (usually manually) disclose the report before any information leak can be completed.
2. The target path can point to a non-file.
   1. Problem: Minor inconvenience by DoS of Lintian. Examples include a named pipe, where Lintian will get stuck until a signal kills it.

Of course, we can do this right[3]:

package Lintian::cpy_check;
use strict; use warnings; use autodie;
use Lintian::Util qw(is_ancestor_of);
sub run  
  my ($pkg, undef, $info) = @_;
  my $filename = "usr/share/doc/$pkg/copyright";
  my $root = $info->unpacked
  my $path = $info->unpacked($filename);
  if ( -f $path and is_ancestor_of($root, $path))  
    open(my $fd, '<', $path);
    ...;
    close($fd);
   
  return;
 

Where is_ancestor_of is the only available utility to assist you currently. It hides away some 10-12 lines of code to resolve the two paths and correctly asserting that $path is (an ancestor of) $root. Prior to Lintian 2.5.12, you would have to do that ancestor check by hand in each and every check[4]. In the new version, the correct code would look something like this:

package Lintian::cpy_check;
use strict; use warnings; use autodie;
sub run  
  my ($pkg, undef, $info) = @_;
  my $filename = "usr/share/doc/$pkg/copyright";
  my $path = $info->index_resolved_path($filename);
  if ($path and $path->is_open_ok)  
    my $fd = $path->open;
    ...;
    close($fd);
   
  return;
 

Now, you may wonder how that promotes safer code. At first glance, the checking code is not a lot simpler than the previous correct example. However, the new code has the advantage of being safer even if you forget the checks. The reasons are:

1. The return value is entirely based on the file index of the package (think: tar vtf data.tar.gz). At no point does it use the file system to resolve the path. Whether your malicious package trigger an undef warning based on the return value of index_resolved_index leaks nothing about the host machine.
   1. However, it does take safe symlinks into account and resolves them for you. If you ask for foo/bar and foo is a symlink to baz and baz/bar exists in the package, you will get baz/bar . If baz/bar happens to be a symlink, then it is resolved as well.
   2. Bonus: You are much more likely to trigger the undef warning during regular testing, since it also happens if the file is simply missing.
2. If you attempt to call $path->open without calling $path->is_open_ok first, Lintian can now validate the call for you and stop it on unsafe actions.

It also has the advantage of centralising the code for asserting safe access, so bugs in it only needs to be fixed in one place. Of course, it is still possible to write unsafe code. But at least, the new API is safer by default and (hopefully) more convenient to use. [0] Lintian.debian.org being the primary exception here. [1] This is in contrast to e.g. piuparts, which very much trusts its input packages by handing the package root access (albeit chroot ed, but still). [2] And also a bug. Not all binary packages have a copyright instead some while have a symlink to another package. [3] The code is hand-typed into the blog without prior testing (not even compile testing it). The code may be subject to typos, brown-paper-bag bugs etc. which are all disclaimed (of course). [4] Fun fact, our documented example for doing it correctly prior to implementing is_ancestor_of was in fact not correct. It used the root path in a regex (without quoting the root path) fortunately, it just broke lintian when your TMPDIR / LINTIAN_LAB contained certain regex meta-characters (which is pretty rare).

The UDD bugs interface currently knows about the following release critical bugs:

• In Total: 1393
  • Affecting Jessie: 408 That's the number we need to get down to zero before the release. They can be split in two big categories:
    • Affecting Jessie and unstable: 360 Those need someone to find a fix, or to finish the work to upload a fix to unstable:
      • 50 bugs are tagged 'patch'. Please help by reviewing the patches, and (if you are a DD) by uploading them.
      • 20 bugs are marked as done, but still affect unstable. This can happen due to missing builds on some architectures, for example. Help investigate!
      • 290 bugs are neither tagged patch, nor marked done. Help make a first step towards resolution!
    • Affecting Jessie only: 48 Those are already fixed in unstable, but the fix still needs to migrate to Jessie. You can help by submitting unblock requests for fixed packages, by investigating why packages do not migrate, or by reviewing submitted unblock requests.
      • 0 bugs are in packages that are unblocked by the release team.
      • 48 bugs are in packages that are not unblocked.

Graphical overview of bug stats thanks to azhag:

$ zcat Sources.gz   grep -o -E 'debhelper [(]>= 9[.][0-9] ,7 ([^0-9)][^)]*)?[)]'   sort   uniq -c   sort -rn
    338 debhelper (>= 9.0.0)
     70 debhelper (>= 9.0)
     18 debhelper (>= 9.0.0~)
     10 debhelper (>= 9.0~)
      2 debhelper (>= 9.2)
      1 debhelper (>= 9.2~)
      1 debhelper (>= 9.0.50~)

Is it a way to protest against the current debhelper's version scheme?

Reproducible builds? I never did any - manually I've never done a reproducible build attempt of any package, manually, ever. But what I've done now is setting up reproducible builds on jenkins.debian.net which will build hundreds or thousands of packages, hopefully reproducibly, regularily in the future. Thanks to Lunar's and many other peoples work, this was actually rather easy. If you want to do this manually, it should take you just a few minutes to setup a suitable build environment. So three days ago when I wasn't exactly bored I decided that it was a good moment to implement some reproducible build jobs on jenkins.d.n, and so I gave it a try and two hours later the basic implementation was working, and then it was an evening and morning of fine tuning until I was mostly satisfied. Since then there has been some polishing, but the basic setup is done and has been working since. What's the result? One job, reproducible_setup will just create a suitable environment for pbuilding reproducible packages as documented so well on the Debian wiki. And as that job runs 3.5 minutes only (to debootstrap from scratch), it's run daily. And then there are currently 16 other jobs, which test reproducible builds in different areas: d-i, core, some six major desktops and some selected desktop applications, some security + privacy related packages, some build chains we have in Debian, libreoffice and X.org. Most of these jobs run several hours, but luckily not days. And they discover packages which still fail to build reproducibly, which already has caused some bugs to be filed, eg. #762732 "libdebian-installer: please do not write timestamps in Doxygen generated documentation". So this is the output from testing the reproducibilty of all debian-installer packages: 72 packages were successfully built reproducibly, while 6 packages failed to do so. I was quite impressed by these numbers as AFAIK noone tried to build d-i reproducibly before.

72 packages successfully built reproducibly: userdevfs user-setup usb-discover udpkg tzsetup rootskel rootskel-gtk rescue preseed pkgsel partman-xfs partman-target partman-partitioning partman-nbd partman-multipath partman-md partman-lvm partman-jfs partman-iscsi partman-ext3 partman-efi partman-crypto partman-btrfs partman-basicmethods partman-basicfilesystems partman-base partman-auto partman-auto-raid partman-auto-lvm partman-auto-crypto partconf os-prober oldsys-preseed nobootloader network-console netcfg net-retriever mountmedia mklibs media-retriever mdcfg main-menu lvmcfg lowmem localechooser live-installer lilo-installer kickseed kernel-wedge kbd-chooser iso-scan installation-report installation-locale hw-detect grub-installer finish-install efi-reader dh-di debian-installer-utils debian-installer-netboot-images debian-installer-launcher clock-setup choose-mirror cdrom-retriever cdrom-detect cdrom-checker cdebconf-terminal cdebconf-entropy bterm-unifont base-installer apt-setup anna 
6 packages failed to built reproducibly: win32-loader libdebian-installer debootstrap console-setup cdebconf busybox 

What's also impressive: all packages for the newly introduced Cinnamon Desktop build reproducibly from the start! The jenkins setup is configured via just three small files:

• job-cfg/reproducible.yaml - defines which jobs exist and which source packages are to be build by them.
• bin/reproducible_setup.sh - creates a base-reproducible.tgz for pbuilder with the (currently) five packages which need patches to support reproducible building.
• bin/reproducible_build.sh - tests whether given source packages from sid build reproducibly.

That's it and that's enough to keep several cores busy for days. But as each job only takes a few hours each is scheduled twice a month and more jobs and packages shall be added in future (with some heuristics to schedule known good packages less often...) I guess it's an appropriate opportunity to say "many thanks to Profitbricks", who have been donating the powerful virtual machine jenkins.debian.net is running on since October 2012. I also want to say "many many thanks to Helmut" (Grohne) who has recently joined me in maintaining this jenkins setup. And then I'd like to thank "the KGB trio" (Gregor, Tincho and Dam!) for providing those KGB bots on IRC, which are very helpful for providing notifications on IRC channels and last but not least thanks to everybody who contributed so that reproducible builds got this far! Keep up the jolly good work! And If you happen to know failing packages not included in job-cfg/reproducible.yaml I'd like to hear about those, so they'll get regularily tested and appear on the radar, until finally bugs are filed, fixed and migrated to stable. So one day all binary packages in Debian stable will be build reproducibly. An important step on this road is probably to have this defined as an release goal for Jessie+1. And then for jessie+1 hopefully the first 10k packages will build reproducibly? Or whooping 23k maybe? And maybe release jessie+2 with 100%?!? We will see! Even Jessie already has quite some packages (someone needs to count them...) which build reproducibly with just modified dpkg(-dev) and debhelper packages alone... So let's fix all the bugs! That said, an easier start for most of you is probably the list of useful things you (yes, you!) can do! Oh, and last but surely not least in my book: many thanks too to the nice people hosting me so friendly in the last days! Keep on rockin'!

The Debian Edu / Skolelinux project provide a Linux solution for schools, including a powerful desktop with education software, a central server providing web pages, user database, user home directories, central login and PXE boot of both clients without disk and the installation to install Debian Edu on machines with disk (and a few other services perhaps to small to mention here). We in the Debian Edu team are currently working on the Jessie based version, trying to get everything in shape before the freeze, to avoid having to maintain our own package repository in the future. The current status can be seen on the Debian wiki, and there is still heaps of work left. Some fatal problems block testing, breaking the installer, but it is possible to work around these to get anyway. Here is a recipe on how to get the installation limping along. First, download the test ISO via ftp, http or rsync (use ftp.skolelinux.org::cd-edu-testing-nolocal-netinst/debian-edu-amd64-i386-NETINST-1.iso). The ISO build was broken on Tuesday, so we do not get a new ISO every 12 hours or so, but thankfully the ISO we already got we are able to install with some tweaking. When you get to the Debian Edu profile question, go to tty2 (use Alt-Ctrl-F2), run

nano /usr/bin/edu-eatmydata-install

and add 'exit 0' as the second line, disabling the eatmydata optimization. Return to the installation, select the profile you want and continue. Without this change, exim4-config will fail to install due to a known bug in eatmydata. When you get the grub question at the end, answer /dev/sda (or if this do not work, figure out what your correct value would be. All my test machines need /dev/sda, so I have no advice if it do not fit your need. If you installed a profile including a graphical desktop, log in as root after the initial boot from hard drive, and install the education-desktop-XXX metapackage. XXX can be kde, gnome, lxde, xfce or mate. If you want several desktop options, install more than one metapackage. Once this is done, reboot and you should have a working graphical login screen. This workaround should no longer be needed once the education-tasks package version 1.801 enter testing in two days. I believe the ISO build will start working on two days when the new tasksel package enter testing and Steve McIntyre get a chance to update the debian-cd git repository. The eatmydata, grub and desktop issues are already fixed in unstable and testing, and should show up on the ISO as soon as the ISO build start working again. Well the eatmydata optimization is really just disabled. The proper fix require an upload by the eatmydata maintainer applying the patch provided in bug #702711. The rest have proper fixes in unstable. I hope this get you going with the installation testing, as we are quickly running out of time trying to get our Jessie based installation ready before the distribution freeze in a month.

Earlier this evening I gave a short talk about R and Docker at the September Meetup of the Docker Chicago group. Thanks to Karl Grzeszczak for setting the meeting, and for providing a pretty thorough intro talk regarding CoreOS and Docker. My slides are now up on my presentations page.

Much has already been written about the recent bash security problem, allocated the CVE identifier CVE-2014-6271, so I'm not even going to touch it. It did remind me to double-check my systems to make sure that I didn't have any packages installed that I didn't need though, because obviously having fewer packages installed and fewer services running reduces the potential attack surface. I had noticed in the past I had python installed and just though "Oh, yeah, I must have python utilities running". It turns out though that on 16 out of 19 servers I control I had python installed solely for the lsb_release script! So I hacked up a horrible replacement for lsb_release in pure shell, and then became cruel:

~ # dpkg --purge python python-minimal python2.7 python2.7-minimal lsb-release

That horrible replacement is horrible because it defers detection of all the names/numbers to the /etc/os-release which wasn't present in earlier versions of Debian. Happily all my Debian GNU/Linux hosts run Wheezy or later, so it all works out. So that left three hosts that had a legitimate use for Python:

• My mail-host runs offlineimap
  • So I purged it.
  • I replaced it with isync.
• My host-machine runs KVM guests, via qemu-kvm.
  • qemu-kvm depends on Python solely for the script /usr/bin/kvm_stat.
  • I'm not pleased about that but will tolerate it for now.
• The final host was my ex-mercurial host.
  • Since I've switched to git I just removed tha package.

So now 1/19 hosts has Python installed. I'm not averse to the language, but given that I don't personally develop in it very often (read "once or twice in the past year") and by accident I had no python-scripts installed I see no reason to keep it on the off-chance. My biggest surprise of the day was that now that we can use dash as our default shell we still can't purge bash. Since it is marked as Essential. Perhaps in the future.

Recently the discussions around how to distribute third party applications for "Linux" has become a new topic of the hour and for a good reason - Linux is becoming mainstream outside of free software world. While having each distribution have a perfectly packaged, version-controlled and natively compiled version of each application installable from a per-distribution repository in a simple and fully secured manner is a great solution for popular free software applications, this model is slightly less ideal for less popular apps and for non-free software applications. In these scenarios the developers of the software would want to do the packaging into some form, distribute that to end-users (either directly or trough some other channels, such as app stores) and have just one version that would work on any Linux distribution and keep working for a long while. For me the topic really hit at Debconf 14 where Linus voiced his frustrations with app distribution problems and also some of that was touched by Valve. Looking back we can see passionate discussions and interesting ideas on the subject from systemd developers (another) and Gnome developers (part2 and part3). After reading/watching all that I came away with the impression that I love many of the ideas expressed, but I am not as thrilled about the proposed solutions. The systemd managed zoo of btrfs volumes is something that I actually had a nightmare about. There are far simpler solutions with existing code that you can start working on right now. I would prefer basing Linux applications on Docker. Docker is a convenience layer on top of Linux cgroups and namespaces. Docker stores its images in a datastore that can be based on AUFS or btrfs or devicemapper or even plain files. It already has a semantic for defining images, creating them, running them, explicitly linking resources and controlling processes. Lets play a simple scenario on how third party applications should work on Linux. Third party application developer writes a new game for Linux. As his target he chooses one of the "application runtime" Docker images on Docker Hub. Let's say he chooses the latest Debian stable release. In that case he writes a simple Dockerfile that installs his build-dependencies and compiles his game in "debian-app-dev:wheezy" container. The output of that is a new folder containing all the compiled game resources and another Dockerfile - this one describes the runtime dependencies of the game. Now when a docker image is built from this compiled folder, it is based on "debian-app:wheezy" container that no longer has any development tools and is optimized for speed and size. After this build is complete the developer exports the Docker image into a file. This file can contain either the full system needed to run the new game or (after #8214 is implemented) just the filesystem layers with the actual game files and enough meta-data to reconstruct the full environment from public Docker repos. The developer can then distribute this file to the end user in the way that is comfortable for them. The end user would download the game file (either trough an app store app, app store website or in any other way) and import it into local Docker instance. For user convenience we would need to come with an file extension and create some GUIs to launch for double click, similar to GDebi. Here the user would be able to review what permissions the app needs to run (like GL access, PulseAudio, webcam, storage for save files, ...). Enough metainfo and cooperation would have to exist to allow desktop menu to detect installed "apps" in Docker and show shortcuts to launch them. When the user does so, a new Docker container is launched running the command provided by the developer inside the container. Other metadata would determine other docker run options, such as whether to link over a socket for talking to PulseAudio or whether to mount in a folder into the container to where the game would be able to save its save files. Or even if the application would be able to access X (or Wayland) at all. Behind the scenes the application is running from the contained and stable libraries, but talking to a limited and restricted set of system level services. Those would need to be kept backwards compatible once we start this process. On the sandboxing part, not only our third party application is running in a very limited environment, but also we can enhance our system services to recognize requests from such applications via cgroups. This can, for example, allow a window manager to mark all windows spawned by an application even if the are from a bunch of different processes. Also the window manager can now track all processes of a logical application from any of its windows. For updates the developer can simply create a new image and distribute the same size file as before, or, if the purchase is going via some kind of app-store application, the layers that actually changed can be rsynced over individually thus creating a much faster update experience. Images with the same base can share data, this would encourage creation of higher level base images, such as "debian-app-gamegl:wheezy" that all GL game developers could use thus getting a smaller installation package. After a while the question of updating abandonware will come up. Say there is is this cool game built on top of "debian-app-gamegl:wheezy", but now there was a security bug or some other issue that requires the base image to be updated, but that would not require a recompile or a change to the game itself. If this Docker proposal is realized, then either the end user or a redistributor can easily re-base the old Docker image of the game on a new base. Using this mechanism it would also be possible to handle incompatible changes to system services - ten years down the line AwesomeAudio replaces PulseAudio, so we create a new "debian-app-gamegl:wheezy.14" version that contains a replacement libpulse that actually talks to AwesomeAudio system service instead. There is no need to re-invent everything or push everything and now package management too into systemd or push non-distribution application management into distribution tools. Separating things into logical blocks does not hurt their interoperability, but it allows to recombine them in a different way for a different purpose or to replace some part to create a system with a radically different functionality. Or am I crazy and we should just go and sacrifice Docker, apt, dpkg, FHS and non-btrfs filesystems on the altar of systemd? P.S. You might get the impression that I dislike systemd. I love it! Like an init system. And I love the ideas and talent of the systemd developers. But I think that systemd should have nothing to do with application distribution or processes started by users. I am sometimes getting an uncomfortable feeling that systemd is morphing towards replacing the whole of System V jumping all the way to System D and rewriting, obsoleting or absorbing everything between the kernel and Gnome. In my opinion it would be far healthier for the community if all of these projects would developed and be usable separately from systemd, so that other solutions can compete on a level playing field. Or, maybe, we could just confess that what systemd is doing is creating a new Linux meta-distribution.

For an actual project we decided to use Redis for some reasons. As there is availability a critical part, we discovered that Redis Sentinel can monitor Redis and handle an automatic master failover to a available slave. Setting up the Redis replication was straight forward and even setting up Sentinel. Please keep in mind, if you configure Redis to require an authentication password, you even need to provide that for the replication process (masterauth) and for the Sentinel connection (auth-pass). The more interesting part is, how to migrate over the clients to the new master in case of a failover process. While Redis Sentinel could also be used as configuration provider, we decided not to use this feature, as the application needs to request the actual master node from Redis Sentinel much often, which will maybe a performance impact.
The first idea was to use some kind of VRRP, implemented into keepalived or something like this. The problem with such a solution is, you need to notify the VRRP process when a redis failover is in progress.
Well, Redis Sentinel has a configuration option called 'sentinel client-reconfig-script':

# When the master changed because of a failover a script can be called in
# order to perform application-specific tasks to notify the clients that the
# configuration has changed and the master is at a different address.
# 
# The following arguments are passed to the script:
#
# <master-name> <role> <state> <from-ip> <from-port> <to-ip> <to-port>
#
# <state> is currently always "failover"
# <role> is either "leader" or "observer"
# 
# The arguments from-ip, from-port, to-ip, to-port are used to communicate
# the old address of the master and the new address of the elected slave
# (now a master).
#
# This script should be resistant to multiple invocations.

This looks pretty good and as there is provided a <role>, I thought it would be a good idea to just call a script which evaluates this value and based on it's return, it adds the VIP to the local network interface, when we get 'leader' and removes it when we get 'observer'. It turned out that this was not working as <role> didn't returned 'leader' when the local redis instance got master and 'observer' when got slave in any case. This was pretty annoying and I was short before giving up.
Fortunately I stumpled upon a (maybe) chinese post about Redis Sentinal, where was done the same like I did. On the second look I recognized that the decision was made on $ 6 which is <to-ip>, nothing more then the new IP of the Redis master instance. So I rewrote my tiny shell script and after some other pitfalls this strategy worked out well. Some notes about convergence. Actually it takes round about 6-7 seconds to have the VIP migrated over to the new node after Redis Sentinel notifies a broken master. This is not the best performance, but as we expect this happen not so often, we need to design the application using our Redis setup to handle this (hopefully) rare scenario.

I am among the lucky people who got back home from DebConf with a brand new computer: a Banana Pi. Despite the name similarity, it is not affiliated with the very well known Raspberry Pi, although it is a very comparable (although much better) machine: A dual-core ARM A7 system with 1GB RAM, several more on-board connectors, and same form-factor. I have not yet been able to get it to boot, even from the images distributed on their site (although I cannot complain, I have not devoted more than a hour or so to the process!), but I do have a gripe on how the images are distributed. I downloaded some images to play with: Bananian, Raspbian, a Scratch distribution, and Lubuntu. I know I have a long way to learn in order to contribute to Debian's ARM port, but if I can learn by doing... So, what is my gripe? That the three images are downloaded as archive files: Now... that is quite an odd way to distribute image files! Specially when looking at their contents: And what is bad about them? That they force me to either have heaps of disk space available (2GB or 4GB for each image) or to spend valuable time extracting before recording the image each time. Why not just compressing the image file without archiving it? That is,
Now, wouldn't we need to decompress said files as well? Yes, but thanks to the magic of shell redirections, we can just do it on the fly. That is, instead of having 3 4GB+1 2GB files sitting on my hard drive, I just need to have several files ranging between 145M and I guess ~1GB. Then, it's as easy as doing: And the result should be the same: A fresh new card with Bananian ready to fly. Right, right, people using these files need to have xz installed on their systems, but... As it stands now, I can suppose current prospective users of a Banana Pi won't fret about facing a standard Unix tool! (Yes, I'll forward this rant to the Banana people, it's not just bashing on my blog :-P ) [update] Several people (thanks!) have contacted me stating that I use a bashism: The <( ) construct is specific to Bash. If you want to do this with any other shell, it can be done with a simple pipe: That allows for less piping to be done on the kernel, and is portable between different shells. Also, a possibility would be: Although that might not be desirable, as it avoids the block-by-block nature of dd. I'm not sure if it makes a realdifference, but it's worth saying :) And yes, some alternatives for not unarchiving the file Here in the blog, an anon commenter suggests (respectively, for zip and .tar.gz files): And a commenter by IRC suggests: Thanks!

I use the lsdvd tool to handle my fairly large DVD collection. It is a nice command line tool to get details about a DVD, like title, tracks, track length, etc, in XML, Perl or human readable format. But lsdvd have not seen any new development since 2006 and had a few irritating bugs affecting its use with some DVDs. Upstream seemed to be dead, and in January I sent a small probe asking for a version control repository for the project, without any reply. But I use it regularly and would like to get an updated version into Debian. So two weeks ago I tried harder to get in touch with the project admin, and after getting a reply from him explaining that he was no longer interested in the project, I asked if I could take over. And yesterday, I became project admin. I've been in touch with a Gentoo developer and the Debian maintainer interested in joining forces to maintain the upstream project, and I hope we can get a new release out fairly quickly, collecting the patches spread around on the internet into on place. I've added the relevant Debian patches to the freshly created git repository, and expect the Gentoo patches to make it too. If you got a DVD collection and care about command line tools, check out the git source and join the project mailing list. :)

Review: Turn the Ship Around!, by L. David Marquet

Publisher:	Portfolio
Copyright:	2012
ISBN:	1-101-62369-1
Format:	Kindle
Pages:	272

Turn the Ship Around! (yes, complete with the irritating exclamation point in the title) is marketed to the business and management non-fiction market, which is clogged with books claiming to provide simple techniques to be a great manager or fix an organization. If you're like me, this is a huge turn-off. The presentation of the books is usually just shy of the click-bait pablum of self-help books. Many of the books are written by famous managers best known for doing horrible things to their staff (*cough* Jack Welch). It's hard to get away from the feeling that this entire class of books is an ocean of bromides covering a small core of outright evil. This book is not like that, and Marquet is not one of those managers. It can seem that way at times: it is presented in the format that caters to short attention span, with summaries of primary points at the end of every short chapter and occasionally annoying questions sprinkled throughout. I'm capable of generalizing information to my own life without being prompted by study questions, thanks. But that's just form. The core of this book is a surprisingly compelling story of Marquet's attempt to introduce a novel management approach into one of the most conservative and top-down of organizations: a US Navy nuclear submarine. I read this book as an individual employee, and someone who has no desire to ever be a manager. But I recently changed jobs and significantly disrupted my life because of a sequence of really horrible management decisions, so I have strong opinions about, at least, the type of management that's bad for employees. A colleague at my former employer recommended this book to me while talking about the management errors that were going on around us. It did such a good job of reinforcing my personal biases that I feel like I should mention that as a disclaimer. When one agrees with a book this thoroughly, one may not have sufficient distance from it to see the places where its arguments are flawed. At the start of the book, Marquet is assigned to take over as captain of a nuclear submarine that's struggling. It had a below-par performance rating, poor morale, and the worst re-enlistment rate in the fleet, and was not advancing officers and crew to higher ranks at anywhere near the level of other submarines. Marquet brought to this assignment some long-standing discomfort with the normal top-down decision-making processes in the Navy, and decided to try something entirely different: a program of radical empowerment, bottom-up decision-making, and pushing responsibility as far down the chain of command as possible. The result (as you might expect given that you can read a book about it) was one of the best-performing submarines in the fleet, with retention and promotion rates well above average. There's a lot in here about delegated decision-making and individual empowerment, but Turn the Ship Around! isn't only about that. Those are old (if often ignored) rules of thumb about how to manage properly. I think the most valuable part of this book is where Marquet talks frankly about his own thought patterns, his own mistakes, and the places where he had to change his behavior and attitude in order to make his strategy successful. It's one thing to say that individuals should be empowered; it's quite another to stop empowering them (which is still a top-down strategy) and start allowing them to be responsible. To extend trust and relinquish control, even though you're the one who will ultimately be held responsible for the performance of the people reporting to you. One of the problems with books like this is that they focus on how easy the techniques presented in the book are. Marquet does a more honest job in showing how difficult they are. His approach was not complex, but it was emotionally quite difficult, even though he was already biased in favor of it. The control, hierarchy, and authority parts of the book are the most memorable, but Marquet also talks about, and shows through specific examples from his command, some accompanying principles that are equally important. If everyone in an organization can make decisions, everyone has to understand the basis for making those decisions and understand the shared goals, which requires considerable communication and open discussion (particularly compared to a Navy ideal of an expert and taciturn captain). It requires giving people the space to be wrong, and requires empowering people to correct each other without blame. (There's a nice bit in here about the power of deliberate action, and while Marquet's presentation is more directly applicable to the sorts of physical actions taken in a submarine, I was immediately reminded of code review.) Marquet also has some interesting things to say about the power of, for lack of a better term, esprit de corps, how to create it, and the surprising power of acting like you have it until you actually develop it. As mentioned, this book is very closely in alignment with my own biases, so I'm not exactly an impartial reviewer. But I found it fascinating the degree to which the management situation I left was the exact opposite of the techniques presented in this book in nearly every respect. I found it quite inspiring during my transition period, and there are bits of it that I want to read again to keep some of the techniques and approaches fresh in my mind. There is a fair bit of self-help-style packaging and layout here, some of which I found irritating. If, like me, you don't like that style of book, you'll have to wade through a bit of it. I would have much preferred a more traditional narrative story from which I could draw my own conclusions. But it's more of a narrative than most books of this sort, and Marquet is humble enough to show his own thought processes, tensions, and mistakes, which adds a great deal to the presentation. I'm not sure how directly helpful this would be for a manager, since I've never been in that role, but it gave me a lot to think about when analyzing successful and unsuccessful work environments. Rating: 8 out of 10

This small guide assumes that you know how to create a public repository with git (or other version control system). Maybe some projects use other VCS, Subversion or whatever; the process would be similar although the commands will be different of course. If you don t want to use any VCS, you can just download the corresponding file, translate it, and send it by email or to the BTS of the project, but the commands required are very easy and you ll see soon that using git (or any VCS) is quite comfortable and less scary than what it seems. So, you were going to recommend a nice app that you use or found in F-Droid to your friend, but she does not understand English. Why not translating the app for her? And for everybody? It s a job that can be done in 15 minutes or so (Android apps have very short strings, few menus, and so). Let s go! 1.- Search the app in the F-Droid website You can do it going to the URL: https://f-droid.org/repository/browse/?fdfilter=wordofappname Example: https://f-droid.org/repository/browse/?fdfilter=pomodoro Then, open the details of the app, and learn where s the source code. 2.- Clone the source code If you have an account in that forge, fork/clone the project into your account, and then, clone your fork/clone in local. If you haven t got an account in that forge, clone the project in local. git clone URLofTheProjectOrYourClone 3.- In local, create a new branch, and checkout to it. cd nameofrepo git checkout -b Spanish 4.- Then, copy the /res/values folder into res/values-XX folder (where XX is your language code) cp ./res/values /res/values-es -R 5.- Translate Edit the strings.xml file that is in the res/values-XX folder, and change the English strings to your language (respect the XML format). 6.- Translate other files, or delete them If there are more files in that folder (e.g. arrays.xml ), review them to know if they have translatable strings. If yes, translate them. If not, delete the files. 7.- Commit When you are finished, commit your changes: git add res/values-es/* git commit -a (Message can be Spanish translation or so) 8.- Push your changes to your public repo If you didn t create a public clone of the repo in your forge, create a public repo and push your local stuff into there. git push --all 9.- Request a merge to the original repo (Using the web interface of the forge, if it is the same for the original repo and your clone, or sending an email or creating an issue and providing the URL of your repo). For example, open a new issue in the project s BTS Title: Spanish translation available for merging Body: Hi everybody. Thanks for your work in "nameofapp". I have completed a Spanish translation, it's available for review/merge in the Spanish branch of my repo: https://urlofyourclone Best regards 10.- Congratulations! Translations are new features, and having a new feature in your app for free is a great thing, so probably the app developer(s) will merge your translation soon. Share your joy with your friends, so they begin to use the app you translated, and maybe become translators too! Comments? You can comment on this post in this pump.io thread.
Filed under: Tools, Writings (translations) Tagged: Android, Contributing to libre software, English, Free Software, libre software, translations

Patti Smith is right, more than any other US city I have visited, Portland feels different. Although living in Cambridge, which sometimes feels like where books were invented, might give me a warped sense of a place.

I have visited Portland a few times previously and I feel comfortable every time I arrive at PDX. Sure the place still suffers from the american obsession with the car but similar to New York you can rely on public transport to get about.

On this occasion my visit was for the Debian Conference which i was excited to attend having missed the previous one in Switzerland. This time the conference has changed its format to being 10 days long and mixing the developer time in with the more formal sessions.

The opening session gave Steve McIntyre and myself the opportunity to present a small token of our appreciation to Russ. The keynote speakers that afternoon were all very interesting both Stefano Zacchiroli and Gabriella Coleman giving food for thought on two very different subjects.

Several conferences in the past have experienced issues with sponsored accommodation and food, I am very pleased to report that both were very good this time. The room I was in had a small kitchen area, en-suite bathroom, desks and most importantly comfortable beds.

The food provision was in the form of a buffet in the Ondine facility. The menu was not greatly varied but catered to all requirements including vegetarian and gluten free diets.

Some of us went on a visit to the Evergreen air and space museum to look at some rare aircraft and rockets. I can thoroughly recommend a visit if you are in the area.

These are just the highlights of the week though, the time in the hack-labs was productive with several practical achievements Including:
- Uploading new packages reducing the bug count
- Sorting out getting an updated key into the Debian keyring.

Overall I had a thoroughly enjoyable time and got a lot out of the conference this year. The new format suited me surprisingly well and as usual the social side was as valuable as the practical.

I hope the organisers have recovered enough to appreciate just how good a job they did and not get hung up on the small number of things that went wrong when the majority of things went perfectly to plan.

The Request I was asked for advice about cheap 3G data plans. One of the people who asked me has a friend with no home Internet access, the friend wants access but doesn t want to pay too much. I don t know whether the person in question can t use ADSL/Cable (maybe they are about to move house) or whether they just don t want to pay for it. 3G data in urban areas in Australia is fast enough for most Internet use. But it s not good for online games or VOIP. It s also not very useful for Youtube and other online video. There is a variety of 3G speed testing apps for Android phones and there are presumably similar apps for the iPhone. Before signing up for 3G at home it s probably best to get a friend who s on the network in question to test Internet speed at your house, it would be annoying to sign up for an annual contract and then discover that your home is in a 3G dead spot. Cheapest Offers The best offer at the moment for moderate data use seems to be Amaysim with 10G for $99.90 and an expiry time of 365 days [1]. 10G in a year isn t a lot, but it s pre-paid so the user can buy another 10G of data whenever they want. At the moment $10 for 1G of data in a month and $20 for 2G of data in a month seem to be common offerings for 3G data in Australia. If you use exactly 1G per month then Amaysim isn t any better than a number of other telcos, but if your usage varies (as it does with most people) then spreading the data use over several months offers significant savings without the need to save big downloads for the last day of the month. For more serious Internet use Virgin has pre-paid offerings of 6G for $30 and 12G for $40 which has to be used in a month [2]. Anyone who uses an average of more than 3G per month will get better value from the Virgin offers. If anyone knows of cheaper options than Amaysim and Virgin then please let me know. Better Coverage Both Amaysim and Virgin use the Optus network which covers urban areas quite well. I used Virgin a few years ago (and presume that it has only improved since then) and my wife uses Amaysim now. I haven t had any great problems with either telco. If you need better coverage than the Optus network provides then Telstra is the only option. Telstra have a number of prepaid offers, the most interesting is $100 for 10G of data that expires in 90 days [3]. That Telstra offer is the same price as the Amaysim offer and only slightly more expensive than Virgin if you average 3.3G per month. It s a really good deal if you average 3.3G per month as you can expect it to be faster and have better coverage. Which One to Choose? I think that the best option for someone who is initially connecting their home via 3g is to start with Amaysim. Amaysim is the cheapest for small usage and they have an Amaysim Android app and web page for tracking usage. After using a few gig of data on Amaysim it should be possible to determine which plan is going to be most economical in the long term. Connecting to the Internet To get the best speed you need a 4G AKA LTE connection. But given that 3G speed is great enough to use expensive amounts of data it doesn t seem necessary to me. I ve done a lot of work over the Internet with 3G from Virgin, Kogan, Aldi, and Telechoice and haven t felt a need to pay for anything faster. I think that the best thing to do is to use an old phone running Android 2.3 or iOS 4.3 as a Wifi access point. The cost of a dedicated 3G Wifi AP is enough to significantly change the economics of such Internet access and most people have access to old smart phones.

• [1] http://www.amaysim.com.au/mobile-plans/amaysim-mobile-broadband.html
• [2] http://tinyurl.com/ldjqhm4
• [3] http://www.telstra.com.au/broadband/mobile-broadband/prepaid/

I had dinner with a friend this evening and ended up discussing the FSF's four freedoms. The fundamental premise of the discussion was that the freedoms guaranteed by free software are largely academic unless you fall into one of two categories - someone who is sufficiently skilled in the arts of software development to examine and modify software to meet their own needs, or someone who is sufficiently privileged[1] to be able to encourage developers to modify the software to meet their needs.

The problem is that most people don't fall into either of these categories, and so the benefits of free software are often largely theoretical to them. Concentrating on philosophical freedoms without considering whether these freedoms provide meaningful benefits to most users risks these freedoms being perceived as abstract ideals, divorced from the real world - nice to have, but fundamentally not important. How can we tie these freedoms to issues that affect users on a daily basis?

In the past the answer would probably have been along the lines of "Free software inherently respects users", but reality has pretty clearly disproven that. Unity is free software that is fundamentally designed to tie the user into services that provide financial benefit to Canonical, with user privacy as a secondary concern. Despite Android largely being free software, many users are left with phones that no longer receive security updates[2]. Textsecure is free software but the author requests that builds not be uploaded to third party app stores because there's no meaningful way for users to verify that the code has not been modified - and there's a direct incentive for hostile actors to modify the software in order to circumvent the security of messages sent via it.

We're left in an awkward situation. Free software is fundamental to providing user privacy. The ability for third parties to continue providing security updates is vital for ensuring user safety. But in the real world, we are failing to make this argument - the freedoms we provide are largely theoretical for most users. The nominal security and privacy benefits we provide frequently don't make it to the real world. If users do wish to take advantage of the four freedoms, they frequently do so at a potential cost of security and privacy. Our focus on the four freedoms may be coming at a cost to the pragmatic freedoms that our users desire - the freedom to be free of surveillance (be that government or corporate), the freedom to receive security updates without having to purchase new hardware on a regular basis, the freedom to choose to run free software without having to give up basic safety features.

That's why projects like the GNOME safety and privacy team are so important. This is an example of tying the four freedoms to real-world user benefits, demonstrating that free software can be written and managed in such a way that it actually makes life better for the average user. Designing code so that users are fundamentally in control of any privacy tradeoffs they make is critical to empowering users to make informed decisions. Committing to meaningful audits of all network transmissions to ensure they don't leak personal data is vital in demonstrating that developers fundamentally respect the rights of those users. Working on designing security measures that make it difficult for a user to be tricked into handing over access to private data is going to be a necessary precaution against hostile actors, and getting it wrong is going to ruin lives.

The four freedoms are only meaningful if they result in real-world benefits to the entire population, not a privileged minority. If your approach to releasing free software is merely to ensure that it has an approved license and throw it over the wall, you're doing it wrong. We need to design software from the ground up in such a way that those freedoms provide immediate and real benefits to our users. Anything else is a failure.

(title courtesy of My Feminism will be Intersectional or it will be Bullshit by Flavia Dzodan. While I'm less angry, I'm solidly convinced that free software that does nothing to respect or empower users is an absolute waste of time)

[1] Either in the sense of having enough money that you can simply pay, having enough background in the field that you can file meaningful bug reports or having enough followers on Twitter that simply complaining about something results in people fixing it for you

[2] The free software nature of Android often makes it possible for users to receive security updates from a third party, but this is not always the case. Free software makes this kind of support more likely, but it is in no way guaranteed.

comments

So Monty and Sean have recently blogged about about the structures (1, 2) they think may work better for OpenStack. I like the thrust of their thinking but had some mumblings of my own to add. Firstly, I very much like the focus on social structure and needs what our users and deployers need from us. That seems entirely right. And I very much like the getting away from TC picking winners and losers. That was never an enjoyable thing when I was on the TC, and I don t think it has made OpenStack better. However, the thing that picking winners and losers did was that it allowed users to pick an API and depend on it. Because it was the X API for OpenStack . If we don t pick winners, then there is no way to say that something is the X API for OpenStack , and that means that there is no forcing function for consistency between different deployer clouds. And so this appears to be why Ring 0 is needed: we think our users want consistency in being able to deploy their application to Rackspace or HP Helion. They want vendor neutrality, and by giving up winners-and-losers we give up vendor neutrality for our users. Thats the only explanation I can come up with for needing a Ring 0 because its still winners and losers (e.g. picking an arbitrary project) keystone, grandfathering it in, if you will. If we really want to get out of the role of selecting projects, I think we need to avoid this. And we need to avoid it without losing vendor neutrality (or we need to give up the idea of vendor neutrality). One might say that we must pick winners for the very core just by its, but I don t think thats true. If the core is small, many people will still want vendor neutrality higher up the stack. If the core is large, then we ll have a larger % of APIs covered and stable granting vendor neutrality. So a core with fixed APIs will be under constant pressure to expand: not just from developers of projects, but from users that want API X to be fixed and guaranteed available and working a particular way at [most] OpenStack clouds. Ring 0 also fulfils a quality aspect we can check that it all works together well in a realistic timeframe with our existing tooling. We are essentially proposing to pick functionality that we guarantee to users; and an API for that which they have everywhere, and the matching implementation we ve tested. To pull from Monty s post: What does a basic end user need to get a compute resource that works and seems like a computer? (end user facet) What does Nova need to count on existing so that it can provide that. He then goes on to list a bunch of things, but most of them are not needed for that: We need Nova (its the only compute API in the project today). We don t need keystone (Nova can run in noauth mode and deployers could just have e.g. Apache auth on top). We don t need Neutron (Nova can do that itself). We don t need cinder (use local volumes). We need Glance. We don t need Designate. We don t need a tonne of stuff that Nova has in it (e.g. quotas) end users kicking off a simple machine have -very- basic needs. Consider the things that used to be in Nova: Deploying containers. Neutron. Cinder. Glance. Ironic. We ve been slowly decomposing Nova (yay!!!) and if we keep doing so we can imagine getting to a point where there truly is a tightly focused code base that just does one thing well. I worry that we won t get there unless we can ensure there is no pressure to be inside Nova to win . So there s a choice between a relatively large set of APIs that make the guaranteed available APIs be comprehensive, or a small set that that will give users what they need just at the beginning but might not be broadly available and we ll be depending on some unspecified process for the deployers to agree and consolidate around what ones they make available consistently. In sort one of the big reasons we were picking winners and losers in the TC was to consolidate effort around a single API not implementation (keystone is already on its second implementation). All the angst about defcore and compatibility testing is going to be multiplied when there is lots of ecosystem choice around APIs above Ring 0, and the only reason that won t be a problem for Ring 0 is that we ll still be picking winners. How might we do this? One way would be to keep picking winners at the API definition level but not the implementation level, and make the competition be able to replace something entirely if they implement the existing API [and win hearts and minds of deployers]. That would open the door to everything being flexible and its happened before with Keystone. Another way would be to not even have a Ring 0. Instead have a project/program that is aimed at delivering the reference API feature-set built out of a single, flat Big Tent and allow that project/program to make localised decisions about what components to use (or not). Testing that all those things work together is not much different than the current approach, but we d have separated out as a single cohesive entity the building of a product (Ring 0 is clearly a product) from the projects that might go into it. Projects that have unstable APIs would clearly be rejected by this team; projects with stable APIs would be considered etc. This team wouldn t be the TC : they too would be subject to the TC s rulings. We could even run multiple such teams as hinted at by Dean Troyer one of the email thread posts. Running with that I d then be suggesting

• IaaS product: selects components from the tent to make OpenStack/IaaS
• PaaS product: selects components from the tent to make OpenStack/PaaS
• CaaS product (containers)
• SaaS product (storage)
• NaaS product (networking but things like NFV, not the basic Neutron we love today). Things where the thing you get is useful in its own right, not just as plumbing for a VM.

So OpenStack/NaaS would have an API or set of APIs, and they d be responsible for considering maturity, feature set, and so on, but wouldn t own Neutron, or Neutron incubator or any other component they would be a *cross project* team, focused at the product layer, rather than the component layer, which nearly all of our folk end up locked into today. Lastly Sean has also pointed out that we have large N N^2 communication issues I think I m proposing to drive the scope of any one project down to a minimum, which gives us more N, but shrinks the size within any project, so folk don t burn out as easily, *and* so that it is easier to predict the impact of changes clear contracts and APIs help a huge amount there.