Search Results: "stone"

25 November 2014

Dirk Eddelbuettel: Rcpp now used by 300 CRAN packages

max-heap image This morning, Rcpp reached another round milestone: 300 packages on CRAN now depend on it (as measured by Depends, Imports and LinkingTo declarations). The graph is on the left depicts the growth of Rcpp usage over time. There are 41 more on BioConductor (which is not included in the chart). The first and less detailed part uses manually save entries, the second half of the data set was generated semi-automatically via a short script appending updates to a small file-based backend. A list of user package is kept on this page. Also displayed in the graph is the relative proportion of CRAN packages using Rcpp. The four per-cent hurdle was cleared just before useR! 2014 where I showed a similar graph (as two distinct graphs) in my invited talk. We may well hit five per-cent before the end of the year. 300 is a pretty humbling and staggering number. Also interesting that we we cleared 200 only at the end of April, and 250 in early August. So from everybody behind Rcpp, a heartfelt Thank You! to all the users and of course other contributors.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

17 November 2014

Vincent Sanders: NetSurf Developer workshop IV

Michael Drake, John-Mark Bell, Daniel Silverstone, Rob Kendrick and Vincent Sanders at the Codethink manchester office
Over the weekend the NetSurf developers met to make a concentrated effort on improving the browser. This time we were kindly hosted by Codethink in their Manchester office in a pleasant environment with plenty of refreshments.

Five developers managed to attend in person from around the UK: Michael Drake, John-Mark Bell, Daniel Silverstone, Rob Kendrick and Vincent Sanders. We also had Chris Young providing some bug fixes remotely.

We started the weekend by discussing all the thorny core issues that had been put on the agenda and ensuring the outcomes were properly noted. We also held the society AGM which was minuted by Daniel.

The emphasis of this weekend was very much on planning and doing the disruptive changes we had been putting off until we were all together.

John-Mark and myself managed to change the core build system as used by all the libraries to using standard triplets to identify systems and use the gnu autoconf style of naming for parameters (i.e. HOST, BUILD and CC being used correctly).

This was accompanied by improvements and configuration changes to the CI system to accommodate the new usage.

Several issues from the bug tracker were addressed and we put ourselves in a stronger position to address numerous other usability problems in the future.

We managed to pack a great deal into the 20 hours of work on Saturday and Sunday although because we were concentrating much more on planning and infrastructure rather than a release the metrics of commits and files changed were lower than at previous events.

6 November 2014

Dirk Eddelbuettel: A Software Carpentry workshop at Northwestern

On Friday October 31, 2014, and Saturday November 1, 2014, around thirty-five graduate students and faculty members attended a Software Carpentry workshop. Attendees came primarily from the Economics department and the Kellogg School of Management, which was also the host and sponsor providing an excellent venue with the Allen Center on the (main) Evanston campus of Northwestern University. The focus of the two-day workshop was an introduction, and practical initiation, to working effectively at the shell, getting introduced and familiar with the git revision control system, as well as a thorough introduction to working and programming in R---from the basics all the way to advanced graphing as well as creating reproducible research documents. The idea of the workshop had come out of discussion during our R/Finance 2014 conference. Bob McDonald of Northwestern, one of this year's keynote speakers, and I were discussing various topic related to open source and good programming practice --- as well as the lack of a thorough introduction to either for graduate students and researcher. And once I mentioned and explained Software Carpentry, Bob was rather intrigued. And just a few months later we were hosting a workshop (along with outstanding support from Jackie Milhans from Research Computing at Northwestern). We were extremely fortunate in that Karthik Ram and Ramnath Vaidyanathan were able to come to Chicago and act as lead instructors, giving me an opportunity to get my feet wet. The workshop began with a session on shell and automation, which was followed by three session focusing on R: a core introduction, a session focused on function, and to end the day, a session on the split-apply-combine approach to data transformation and analysis. The second day started with two concentrated session on git and the git workflow. In the afternoon, one session on visualization with R as well as a capstone-alike session on reproducible research rounded out the second day.

Things that worked The experience of the instructors showed, as the material was presented and an effective manner. The selection of topics, as well as the pace were seen by most students to be appropriate and just right. Karthik and Ramnath both did an outstanding job. No students experienced any real difficulties installing software, or using the supplied information. Participants were roughly split between Windows and OS X laptops, and had generally no problem with bash, git, or R via RStudio. The overall Software Carpentry setup, the lesson layout, the focus on hands-on exercises along with instruction, the use of the electronic noteboard provided by etherpad and, of course, the tried-and-tested material worked very well.

Things that could have worked better Even more breaks for exercises could have been added. Students had difficulty staying on pace in some of the exercise: once someone fell behind even for a small error (maybe a typo) it was sometimes hard to catch up. That is a general problem for hands-on classes. I feel I could have done better with the scope of my two session. Even more cohesion among the topics could have been achieved via a single continually used example dataset and analysis.

Acknowledgments Robert McDonald from Kellogg, and Jackie Milhans from Research Computing IT, were superb hosts and organizers. Their help in preparing for the workshop was tremendous, and the pick of venue was excellent, and allowed for a stress-free two days of classes. We could not have done this without Karthik and Ramnath, so a very big Thank You to both of them. Last but not least the Software Carpentry 'head office' was always ready to help Bob, Jackie and myself during the earlier planning stage, so another big Thank You! to Greg and Arliss.

2 November 2014

Thomas Goirand: OpenStack packaging activity: October 2014

Wednesday 1:
Uploaded python-xstatic-jquery removing the .pth file from package.
Uploaded python-taskflow 0.4 to experimental, needed by Cinder Juno RC1
Uploaded Cinder Juno RC1 to experimental Thuesday 2:
Finally understood that the issue with murano-dashboard was that it doesn t build without django-nose >= 1.2. Opened new patch at: https://review.openstack.org/125565
Uploaded murano-dashboard to Experimental, now using django-nose from wheezy-backports in my jenkins setup, so murano-dashboard can be built for Wheezy.
Uploaded python-oslotest 1.1.0.0 (really is upstream 1.1.0)
Uploaded python-oslo.serialization 1.0.0-1 (needed by Ceilometer Juno RC1)
Uploaded Ceilometer Juno RC1
Uploaded Heat Juno RC1
Uploaded oslo.rootwrap 1.3.0.0
Uploaded oslo.db 1.0.2 (bugfix release)
Wrote a new system in openstack-pkg-tools to generate init scripts and. service files from a template, so we don t have to write N times the same thing. Friday 3:
Reworked openstack-pkg-tools to generate automatically sysv-rc init scripts, upstart jobs and systemd unit files, making the system more unified and consistent.
Applied the new system to all packages in Juno.
Uploaded Keystone 2014.1.3-1 to Sid
Uploaded Nova 2014.1.3-1 to Sid
Uploaded Glance 2014.1.3-1 to Sid
Uploaded Neutron 2014.1.3-1 to Sid
Uploaded Horizon 2014.1.3-1 to Sid
Uploaded Cinder 2014.1.3-1 to Sid
Uploaded Trove 2014.1.3-1 to Sid
Uploaded Ceilometer 2014.1.3-1 to Sid Saturday 4:
Uploaded Horizon Juno RC1 to Experimental
Uploaded oslotest 1.1.0.0 to Experimental
Uploaded Ironic Juno RC1 to Experimental
Uploaded Designate Juno RC1 to Experimental
Uploaded Nova Juno RC1 to Experimental
Uploaded Neutron Juno RC1 to Experimental
Uploaded openstack-meta-packages 0.10 to Sid
Uploaded openstack-pkg-tools 13 to Experimental
Uploaded murano-agent Juno RC1 to Experimental Sunday 5:
Uploaded Sahara Juno RC1 to Experimental (it s been approved by FTP masters)
Uploaded Murano Juno RC1 to Experimental (it s been approved by FTP masters)
Fixed all debian/watch file to understand ~b and ~rc releases (fixed applied on both Icehouse and Juno branches, though no upload yet, I ll wait until uploads are needed to have this in the archive ).
Uploaded Trove Juno RC1 to Experimental
Uploaded Sahara Juno RC1 to Experimental With this last upload, everything of Juno RC1 is in Debian Experimental! \o/ Monday 6:
Uploaded some fixes for Nova 2014.1.3-2 in Sid:
* Removed contrib/boto_v6/* in debian/copyright, replaced bin/nova-manage by nova/cmd/ baremetal_, manage.py.
* Mangling upstream rc and beta versions in watch file.
* Added 9990_update_german_programm_messages.patch, thanks to Helge Kreutzmann <debian@helgefjell.de>.
* Fixed correct de.po (Closes: #763682).
* Added nl.po initial Debconf translation, thanks to Frans Spiesschaert <Frans.Spiesschaert@yucom.be> (Closes: #764125).
* Standards-Version is now 3.9.6 (no change).
Upstreamed german translation of po file: https://review.openstack.org/126212
Uploaded Designate 2014.1-12 to Sid, added new de.po also to the Juno branch on alioth (but didn t upload the fix yet).
Uploaded sphinxcontrib-httpdomain new upstream 1.3.0 release, added Python 3.x support to the package, and transitionning to the correct namespaced python-sphinxcontrib.httpdomain package name.
Spent most of the day fixing python-xstatic issues:
o uploaded libjs-twitter-bootstrap-datepicker 1.3.1
o uploaded python-xstatic-bootstrap-datepicker requiring this libjs package
o fixed python-xstatic-jquery-ui package
Now Horizon Juno RC1 builds well, and can be installed again. \o/ Tuesday 7:
Backported python-libvirt 1.2.8 in Wheezy (for Nova Juno support )
Uploaded Ceilometer Juno RC1 with ceilometer-agent-ipmi added (the package will therefore go through the NEW queue).
Uploaded python-requestbuilder 0.2.2-1, needed by the maintainers of euca2ools.
Ported the unified generated init system scripts to Icehouse packages.
Uploaded to Sid updates for: openstack-pkg-tools, ceilometer, cinder, glance, keystone, cinder, nova. Wednesday 8:
Uploaded openstack-pkg-tools 16 to Sid
Uploaded murano-dashboard (with upstream fix to remove font-awesome, which was the reason for FTP master s rejection)
Uploaded ceilometer Juno RC1 with new IPMI agent package (needed for Ironic support).
Uploaded heat 2014.1.3 which I forgot.
Tested https://review.openstack.org/#/c/126777/ which solves the bug I sent to launchpad and approved the patch.
Uploaded python-requestbuilder 0.2.3 Thesday 9:
Worked on fixing Neutron Alembic migration with SQLite3.
Uploade Neutron 2014.2~rc1-3 with a fix for a patch that was destroying dhcp.py. This still doesn t include the Alembic migration fixes, which are still a WIP. Firday 10:
Finished fixing Neutron SQLite 3 Alembic migrations.
Uploaded neutron 2014.2~rc1-3 with the fixes.
Fixed Ceilometer wrong generation of sample config file, using upstream patch (after discussing with Julien Danjou so he wrote it).
Uploaded Ceilometer 2014.2~rc1-4 with the fix
Checked that all packages can be installed in non-interactive mode. This works well now! \o/ Saturday 11:
Uploaded new version of python-xstatic-angular-cookies (ie: 1.2.24.1-2) which allows a higher version of libjs-angularjs (otherwise the package is not installable in Sid/Jessie since last version of angularjs is uploaded). Sunday 12:
Uploaded factory-boy fix for FTBFS
Uploaded python-django-appconf FTBFS
Uploaded Horizon Juno RC2
Uploaded Heat Juno RC3
Uploaded Trove Juno RC2
Uploaded Glance Juno RC2
Uploaded Sahara Juno RC2
Uploaded Nova Juno RC2
Uploaded Neutron Juno RC2
Uploaded Cinder Juno RC2
Uploaded murano-dashboard Juno RC2 Monday 13:
Uploaded python-heatclient 0.2.12-1 to Experimental
Uploaded python-yaql with RC bugfix to Sid (missing dep on python3-ply). Thuesday 14:
Fixed arping newly added dependency in Neutron
Started testing install of all of openstack Juno at once Wednesday 15:
Fixed missing configuration files in Ceilometer (ceilometer-api couldn t start)
Upgraded to Ceilometer Juno RC3.
Backported python-setuptools, as keystone and others are broken due to the namespace of modules not working correctly with the old version of python-pkg-resources. With the new one, everything is back in order. Thesday 16:
Uploaded to Debian Experimental the final release of Juno (ie: 2014.2) for:
Sahara
Nova
Ceilometer
Cinder
Heat
Neutron
Glance
Keystone
Horizon (with fix for Django 1.7 in the wsgi file)
Uploaded to Sid:
Swift 2.2.0
Horizon 2014.1.3-3 with fix for Django 1.7 in the wsgi file that was crashing apache. OpenStack Juno packages are out!!! (ready the day of the upstream release ) Friday 17:
Investigated Trove RC bug #765348, couldn t reproduce, and therefore closed it.
Uploaded Ironic Juno final to Experimental
Uploaded Designate Juno final to Experimental
Uploaded a fix for python-jingo which failed to build with Django 1.7. Sent pull request upstream: https://github.com/jbalogh/jingo/pull/63
Uploaded CVE-2014-7230 & CVE-2014-7231 fixes for both Cinder and Nova in Debian Sid, as per OSSA 2014-036 patches. No need to upload a fix for Trove, as 2014.1.3 already has the fixes. Saturday 18:
Started building Trusty packages
Fixed oslo-config so that it never depends on python3-argparse, which doesn t exist (uploaded to Experimental)
Uploaded python-django-pyscss 1.0.3-2 with python-simplejson now as build-depends (it failed to build in my Trusty jenkins without it).
Uploaded a fix for stevedore and oslo-config to not depends on python3-argparse in Ubuntu (added debian/py3dist-overrides) Sunday 19:
Uploaded python-taskflow with ordereddict in debian/pydist-overrides.
Backported JS packages for Horizon and libvirt for Trusty (from Sid). My new Jenkin server is now producing a full set of Juno packages for Ubuntu trusty. And of course, it s updated on each git push, just like for the Wheezy backports. Monday 20:
Added FORCE_COULEUR=1 when running tests in python-couleur, so that it doesn t fail when running with git-buildpackage. Uploaded result in Sid.
Fixed python-mockito so that it never downloads distribute or nose on its clean target, which was annoying when running git-buildpackage. Uploaded to Sid.
Started to work again on automatic package deployment using openstack-deploy, from the openstack-meta-packages source package. Thuesday 21, Wednesday 22:
Worked on testing packages, did couples of minor fixes, reworked some of the default configuration files to match the install-guide, move configuration directive to the correct new section in nova.conf, etc. Thursday 23:
Patch the Neutron chapter in the install-guide to take into account the changes done on Thuesday 21, Wednesday 22, and simplify the install procedure in Debian. https://review.openstack.org/#/c/130501/ Friday 24:
Busy packing my stuff for moving to France Not much packaging work, except more auto-deploy stuff and some tests. Saturday 25:
Uploaded Nova, Neutron, Cinder and Horizon Icehouse in Sid, including some debconf translation updates, beating the Jessie freeze deadline in 10 days.
Fixed and uploaded openstack-debian-images in Sid: the login option wasn t modifying the default sudoers file, which always contained debian , instead of the custom login. Sunday 26:
Traveled to Moscow Monday 27 & Tuesday 28:
Fixed some murano & murano-dashboard stuff, thanks to the help of some murano team members in Moscow office. Uploaded fixes for murano & murano-dashboard. Tested that murano-dashboard works well, and now it does! :)
Uploaded version dependency fixes for python-xstatic-angular-cookies and python-xstatic-d3 which couldn t be installed in Sid/Jessie because of libjs-* updates. Wednesday 29:
Meeting with Saratov team
Updated sahara endpoints, but didn t upload the package yet to Debian. Thursday 30:
Uploaded ruby-raemon needed for Astute (part of Fuel web).
Packaged ruby-symboltable (not uploaded yet). Friday 31:
Wrote unit test runner for python-webpy (the current package doesn t have unit test runs).
Uploaded python-dbutils (needed by python-web.py unit tests) to Sid: now in NEW queue
Uploaded python-nose-parametrized & python-nose-timer to Sid: now in NEW queue
Uploaded sahara -2 fixing the API endpoint registration URL and service name.
Uploaded python-sphinxcontrib.plantuml to Sid: : now in NEW queue

25 October 2014

Miriam Ruiz: Video game players and Gamers are different things

Even though the Wikipedia defines gamer as someone who partakes in interactive gaming, such as (predominantly) video games or board games , this doesn t really gets close to that term means socially at the moment. Going back to Wikipedia, we find that the video game subculture is a form of new media subculture that has been influenced by video games , so it might be quite accurate to define gamers as members of that subculture. You will find that most of the uses of the term gamer in the social networks and in the blogosphere refer to that. Please notice that, even though it is quite likely that most of the gamers play video games, the other way round does not need to be true and, in fact, it isn t. Not everyone who plays video games belongs to the video game subculture, shares their point of view, their values and aesthetics, or even know about it. Kind of like what happens with the word hacker . Not everyone who hacks around with a computer belongs to the hacker subculture. Mostly everyone who has access to the technology plays video games now. From babies and kids to grandparents. And people play them in every possible technological system around, not only on video game consoles or personal computers, but alse on mobile phones, tablets, web browsers. And many of those people who use different kind of technologies to play video games are not gamers. Not in the sense of belonging to the video game subculture. It is important to acknowledge that: that the video game subculture does not have the monopoly over video games or the video game developing industry anymore. As you can imagine, all this rand doesn t come from nowhere. During the last months, we have been witnessing a fight between some conservative core members of the video game subculture and people who want to bring some fresh air into the sociocultural elements of that subculture. Namely, that women shouldn t be discriminated inside it. As every time that a women raises her voice to complain about anything in the Internet, they have been subjected to insults, attacks, rape and death threats, etc. I m talking about something called #GamerGate, and even though I m not going to get into it, I will provide some URLs in case someone might be interested. Please acknowledge that not all the points of view might be represented in this list (in fact, they are not, as I won t be promoting in my blog things that I severely disagree with), so search the web for more information if you want to get that. I ve never been a gamer myself, meaning part of the subculture I mentioned. At some point I was probably closer tho the core values they had then than I am now. In any case, video games have already consolidated themselves as an important part of current culture, entertainment, education and socialization, and are definitely here to stay. That will probably mean that the percentage of gamers (members of the video game subculture) will become smaller. as the number of non-gamer video game players keeps raising.

22 October 2014

Russ Allbery: Another haul post

I know I've been very quiet here lately. That's due to a variety of reasons, but mostly because settling in to a new job is taking nearly all of my attention and time. When that's combined with getting obsessed with watching the League of Legends world championships, it means no real time for writing things. I've had lots of time for reading things, though, and have a lot of book reviews that I need to write. So, of course, I felt like buying more books. Elizabeth Bear One-Eyed Jack (sff)
Steven Brust Hawk (sff)
Kenneth T. Jackson Crabgrass Frontier (non-fiction)
Ann Leckie Ancillary Sword (sff)
Scott Lynch Republic of Thieves (sff)
Randall Munroe What If? (non-fiction)
Sarah Tolmie The Stone Boatmen (sff)
Jeffrey Toobin The Oath (non-fiction) I'm pretty excited about everything in this shipment, but particularly the new Vlad Taltos novel from Brust and the sequel to Ancillary Justice (probably the best novel that I've read so far this year). And of course there's What If?.

18 October 2014

Erich Schubert: Beware of trolls - do not feed

A particularly annoying troll has been on his hate crusade against systemd for months now.
Unfortunately, he's particularly active on Debian mailing lists (but apparently also on Ubuntu and the Linux Kernel mailing list) and uses a tons of fake users he keeps on setting up. Our listmasters have a hard time blocking all his hate, sorry.
Obviously, this is also the same troll that has been attacking Lennart Poettering.
There is evidence that this troll used to go by the name "MikeeUSA", and has quite a reputation with anti-feminist hate for over 10 years now.
Please, do not feed this troll.
Here are some names he uses on YouTube: Gregory Smith, Matthew Bradshaw, Steve Stone.
Blacklisting is the best measure we have, unfortunately.
Even if you don't like the road systemd is taking or Lennart Poetting personall - the behaviour of that troll is unacceptable to say the least; and indicates some major psychological problems... also, I wouldn't be surprised if he is also involved in #GamerGate.
See this example (LKML) if you have any doubts. We seriously must not tolerate such poisonous people.
If you don't like systemd, the acceptable way of fighting it is to write good alternative software (and you should be able to continue using SysV init or openRC, unless there is a bug, in Debian - in this case, provide a bug fix). End of story.

10 October 2014

Martin Pitt: Running autopkgtests in the cloud

It s great to see more and more packages in Debian and Ubuntu getting an autopkgtest. We now have some 660, and soon we ll get another ~ 4000 from Perl and Ruby packages. Both Debian s and Ubuntu s autopkgtest runner machines are currently static manually maintained machines which ache under their load. They just don t scale, and at least Ubuntu s runners need quite a lot of handholding. This needs to stop. To quote Tim The Tool Man Taylor: We need more power!. This is a perfect scenario to be put into a cloud with ephemeral VMs to run tests in. They scale, there is no privacy problem, and maintenance of the hosts then becomes Somebody Else s Problem. I recently brushed up autopkgtest s ssh runner and the Nova setup script. Previous versions didn t support revert yet, tests that leaked processes caused eternal hangs due to the way ssh works, and image building wasn t yet supported well. autopkgtest 3.5.5 now gets along with all that and has a dozen other fixes. So let me introduce the Binford 6100 variable horsepower DEP-8 engine python-coated cloud test runner! While you can run adt-run from your home machine, it s probably better to do it from an autopkgtest controller cloud instance as well. Testing frequently requires copying files and built package trees between testbeds and controller, which can be quite slow from home and causes timeouts. The requirements on the controller node are quite low you either need the autopkgtest 3.5.5 package installed (possibly a backport to Debian Wheezy or Ubuntu 12.04 LTS), or run it from git ($checkout_dir/run-from-checkout), and other than that you only need python-novaclient and the usual $OS_* OpenStack environment variables. This controller can also stay running all the time and easily drive dozens of tests in parallel as all the real testing action is happening in the ephemeral testbed VMs. The most important preparation step to do for testing in the cloud is quite similar to testing in local VMs with adt-virt-qemu: You need to have suitable VM images. They should be generated every day so that the tests don t have to spend 15 minutes on dist-upgrading and rebooting, and they should be minimized. They should also be as similar as possible to local VM images that you get with vmdebootstrap or adt-buildvm-ubuntu-cloud, so that test failures can easily be reproduced by developers on their local machines. To address this, I refactored the entire knowledge how to turn a pristine default vmdebootstrap or cloud image into an autopkgtest environment into a single /usr/share/autopkgtest/adt-setup-vm script. adt-buildvm-ubuntu-cloud now uses this, you shold use it with vmdebootstrap --customize (see adt-virt-qemu(1) for details), and it s also easy to run for building custom cloud images: Essentially, you pick a suitable pristine image, nova boot an instance from it, run adt-setup-vm through ssh, then turn this into a new adt specific "daily" image with nova image-create. I wrote a little script create-nova-adt-image.sh to demonstrate and automate this, the only parameter that it gets is the name of the pristine image to base on. This was tested on Canonical's Bootstack cloud, so it might need some adjustments on other clouds. Thus something like this should be run daily (pick the base images from nova image-list):
  $ ./create-nova-adt-image.sh ubuntu-utopic-14.10-beta2-amd64-server-20140923-disk1.img
  $ ./create-nova-adt-image.sh ubuntu-utopic-14.10-beta2-i386-server-20140923-disk1.img
This will generate adt-utopic-i386 and adt-utopic-amd64. Now I picked 34 packages that have the "most demanding" tests, in terms of package size (libreoffice), kernel requirements (udisks2, network manager), reboot requirement (systemd), lots of brittle tests (glib2.0, mysql-5.5), or needing Xvfb (shotwell):
  $ cat pkglist
  apport
  apt
  aptdaemon
  apache2
  autopilot-gtk
  autopkgtest
  binutils
  chromium-browser
  cups
  dbus
  gem2deb
  glib-networking
  glib2.0
  gvfs
  kcalc
  keystone
  libnih
  libreoffice
  lintian
  lxc
  mysql-5.5
  network-manager
  nut
  ofono-phonesim
  php5
  postgresql-9.4
  python3.4
  sbuild
  shotwell
  systemd-shim
  ubiquity
  ubuntu-drivers-common
  udisks2
  upstart
Now I created a shell wrapper around adt-run to work with the parallel tool and to keep the invocation in a single place:
$ cat adt-run-nova
#!/bin/sh -e
adt-run "$1" -U -o "/tmp/adt-$1" --- ssh -s nova -- \
    --flavor m1.small --image adt-utopic-i386 \
    --net-id 415a0839-eb05-4e7a-907c-413c657f4bf5
Please see /usr/share/autopkgtest/ssh-setup/nova for details of the arguments. --image is the image name we built above, --flavor should use a suitable memory/disk size from nova flavor-list and --net-id is an "always need this constant to select a non-default network" option that is specific to Canonical Bootstack. Finally, let' run the packages from above with using ten VMs in parallel:
  parallel -j 10 ./adt-run-nova -- $(< pkglist)
After a few iterations of bug fixing there are now only two failures left which are due to flaky tests, the infrastructure now seems to hold up fairly well. Meanwhile, Vincent Ladeuil is working full steam to integrate this new stuff into the next-gen Ubuntu CI engine, so that we can soon deploy and run all this fully automatically in production. Happy testing!

5 October 2014

Thomas Goirand: OpenStack packaging activity: September 2014

I decided I d post this monthly. It may be a bit boring, sorry, but I think it s a nice thing to have this public. The log starts on the 6th, because on the 4th I was back from Debconf (after a day in San Francisco, plus 20 hours of traveling and 15 hours of time gap). It is to be noted that every time something is uploaded in Debian for Icehouse (in Sid), or for Juno (in Experimental), there s also a corresponding backport produced for Wheezy. Saturday 6th & Sunday 7th:
packaged libjs-twitter-bootstrap-wizard (in new queue)
Uploaded python-pint after reviewing the debian/copyright
Worked on updating python-eventlet in Experimental, and adding Python3 support. It seems Python3 support isn t ready yet, so I will probably remove that feature from the package update.
Tried to apply the Django 1.7 patches for python-django-bootstrap-form. They didn t work, but Raphael came back on Monday morning with new versions
of the patches, which should be good this time.
Helped the DSA (Debian System Administrators) with the Debian OpenStack cloud. It s looking good and working now (note: I helped them during Debconf 14).
Started a page about adding more tasksel tasks: https://wiki.debian.org/tasksel/MoreTasks. It s looking like Joey Hess is adding new tasks by default in Tasksel, with OpenStack compute node and OpenStack proxy node . It will be nice to have them in the default Debian installer! :)
Packaged and uploaded python-dib-utils, now in NEW queue. Monday 8th:
Uploaded fixed python-django-bootstrap-form with patch for Django 1.7.
Packaged and uploaded python-pysaml2.
Finilized and uploaded python-jingo which is needed for python-django-compressor unit tests
Finalized and uploaded python-coffin which is needed for python-django-compressor unit tests
Worked on running the unit tests for python-django-compressor, as I needed to know if it could work with Django 1.7. It was hard to find the correct way to run the unit tests, but finally, they all passed. I will add the unit tests once coffin and jingo will be accepted in Sid.
Applied patches in the Debian BTS for python-django-openstack-auth and Django 1.7. Uploaded the fixed package.
Fixed python-django-pyscss compat with Django 1.7, uploaded the result.
Updated keystone to Juno b3.
Built Wheezy backports of some JS libs needed for Horizon in Juno, which I already uploaded to Sid last summer:
o libjs-twitter-bootstrap-datepicker
o libjs-jquery.quicksearch
o libjs-spin.js
Upstreamed the Django 1.7 patch for python-django-openstack-auth: https://review.openstack.org/119972 Tuesday 9:
Updated and uploaded Swift 2.1.0. Added swift-object-expirer package to it, together with init script. Wednesday 10:
Basically, cleaned the Debian BTS of almost all issues today :P
Added it.po update to nova (Closes: #758305).
Backported libvirt 1.2.7 to Wheezy, to be able to close this bug: https://bugs.debian.org/757548 (eg: changed dependency from libvirt-bin to libvirt-daemon-system)
Uploaded the fixed nova package using libvirt-daemon-system
Upgraded python-trollius to 1.0.1
Fixed tuskar-ui to work with Django 1.7. Disabled pep8 tests during build. Added build-conflicts: python-unittest2.
Fixed python-django-compressor for Django 1.7, and now running unit tests with it, after python-coffin and python-jingo got approved in Sid by FTP masters.
Fixed python-xstatic wrong upstream URLs.
Added it.po debconf translation to Designate.
Added de.po debconf translation to Tuskar.
Fixed copyright holders in python-xstatic-rickshaw
Added python-passlib as dependency for python-cinder. Remaining 3 issues in the BTS: ceilometer FTBFS, Horizon unit test with Django 1.7, Designate fail to install. All of the 3 are harder to fix, and I may try to do so later this week. Thursday 11:
Fixed python-xstatic-angular and python-xstatic-angular-mock to deal with the new libjs-angularjs version (closes 2 Debian RC bugs: uninstallable).
Fixed ceilometer FTBFS (Closes rc bug) Friday 12:
Fixed wrong copyright file for libjs-twitter-bootstrap-wizard after the FTP masters told me, and reuploaded to Sid.
Reuploaded wrong upload of ceilometer (wrong hash for orig.tar.xz)
Packaged and uploaded python-xstatic-bootstrap-scss
Packaged and uploaded python-xstatic-font-awesome
Packaged and uploaded ntpstat Monday 15:
packaged and uploaded python-xstatic-jquery.bootstrap.wizard
Fixed python-xstatic-angular-cookies to use new libjs-angularjs version (fixed version dependencies)
Fixed Ceilometer FTBFS (Closes: #759967)
Backported all python-xtatic packages to Wheezy, including all dependencies. This includes backporting of a bunch of packages from nodejs which were needed as build-dependencies (around 70 packages ). Filed about 5 or 6 release critical bugs as some nodejs packages were not buildable as-is.
Fixed some too restrictive python-xstatic-angular* dependencies on the libjs-angularjs (the libjs-angularjs increased version). Tuesday 16:
Uploaded updates to Experimental:
o python-eventlet 0.15.2 (this one took a long time as it needed maintenance)
o oslo-config
o python-oslo.i18n
Uploaded to Sid:
o python-diskimage-builder 0.1.30-1
o python-django-pyscss 1.0.2-1
Fixed horizon libapache-mode-wsgi to be a dependency of openstack-dashboard-apache and not just openstack-dashboard (in both Icehouse & Juno).
Removed the last failing Django 1.7 unit test from Horizon. It doesn t seem relevant anyway.
Backported python-netaddr 0.7.12 to Wheezy (needed by oslo-config).
Started working on oslo.rootwrap, though it failed to build in Wheezy with a unit test failure. Wednesday 17:
To experimental:
o Uploaded oslo.rootwrap 1.3.0.0~a1. It needed a build-depends on iproute2 because of a new test.
o Uploaded python-oslo.utils 0.3.0
o Uploaded python-oslo.vmware 0.6.0, fixed sphinx-build conf.py and filed a bug about it: https://bugs.launchpad.net/oslo.vmware/+bug/1370370 plus emailed the commiter of the issue (which appeared 2 weeks ago).
o Uploaded python-pycadf 0.6.0
o Uploaded python-pyghmi 0.6.17
o Uploaded python-oslotest 1.1.0.0~a2, including patch for Wheezy, which I also submited upstream: https://review.openstack.org/122171/
o Uploaded glanceclient 0.14.0, added a patch to not use the embedded version of urllib3 in requests: https://review.openstack.org/122184
To Sid:
o Uploaded python-zake_0.1.6-1 Thesday 18:
Backported zeromq3-4.0.4+dfs, pyzmq-14.3.1, pyasn1-0.1.7, python-pyasn1-modules-0.0.5
Uploaded keystoneclient 0.10.1, fixed the saml2 unit tests which were broken using testtools >= 0.9.39. Filed bug, and warned code author: https://bugs.launchpad.net/python-keystoneclient/+bug/1371085
Uploade swiftclient 2.3.0 to experimental.
Uploaded ironicclient 0.2.1 to experimental.
Uploaded saharaclient, filed bug with saharaclient expecting an up and running keystone server: https://bugs.launchpad.net/python-saharaclient/+bug/1371177 Friday 19:
Uploaded keystone Juno b3, filed but about unit tests downloading with git, while no network access should be performed during package build (forbidden by
Debian policy)
Uploaded python-oslo.db 1.0.0 which I forgot in the dependency list, and which was needed for Neutron.
Uploaded nova 2014.2~b3-1 (added a new nova-serialproxy service daemon to the nova-consoleproxy) Saturday 20:
Uploaded Neutron Juno b3.
Uploaded python-retrying 1.2.3 (was missing from depends upload)
Uploaded Glance Juno b3.
Uploaded Cinder Juno b3.
Fixed python-xstatic-angular-mock which had a .pth packaged, as well as the data folder (uploaded debian release -3).
Fixed missing depends and build-conflicts in python-xstatic-jquery. Sunday 21:
Dropped python-pil & python-django-discover-runner from runtime Depends: of python-django-pyscss, as it s only needed for tests. It also created a conflicts, because python-django-discover-runner depends on python-unittest2 and horizon build-conflicts with it.
Forward-ported the Django 1.7 patches for Horizon. Opened new patch: https://review.openstack.org/122992 (since the old fix has gone away after a refactor of the unit test).
Uploaded Horizon Juno b3.
Applied https://review.openstack.org/#/c/122768/ to the keystone package, so that it doesn t do git clone of the keystoneclient during build.
Uploaded oslo.messaging 1.4.0.0 (which really is 1.4.0) to experimental
Uploaded oslo.messaging 1.4.0.0+really+1.3.1-1 to fix the issue in Sid/Jessie after the wrong upload (due to Zul wrong tagging of Keystone in the 2014.1.2 point release). Monday 22:
Uploaded ironic 2014.2~b3-1 to experimental
Uploaded heat 2014.2~b3-1 (with some fixes for sphinx doc build)
Uploaded ceilometer 2014.2~b3-1 to experimental
Uploaded openstack-doc-tools 0.19-1 to experimental
Uploaded openstack-trove 2014.2~b3-1 to experimental Tuesday 23:
Uploaded python-neutronclient with fixed version number for cliff and six. This missing requirement for cliff version produced an error in Trove, which I don t want to happen again.
Added fix for unit tests in Trove: https://review.openstack.org/#/c/123450/1,publish
Uploaded oslo.messaging 1.4.1 in Experimental, fixing the version conflicts with the one in Sid/Jessie. Thanks to Doug Hellman for doing the tagging. I will need to upload new versions of the following packages with the >= 1.4.1 depends:
> ceilometer
> ironic
> keystone
> neutron
> nova
> oslo-config
> oslo.rootwrap
> oslo.i18n
> python-pycadf
See http://lists.openstack.org/pipermail/openstack-dev/2014-September/046795.html for more explanation about the mess I m repairing
Uploaded designate Juno b3. Wednesday 24:
Uploaded oslosphinx 2.2.0.0
Uploaded update to django-openstack-auth (new last minute requirement for Horizon).
Uploaded final oslo-config package version 1.4.0.0 (really is 1.4.0)
Packaged and uploaded Sahara. This needs some tests by someone else as I don t even know how it works. Thuesday 25:
Uploaded python-keystonemiddleware 1.0.0-3, fixing CVE-2014-7144] TLS cert verification option not honoured in paste configs. https://bugs.debian.org/762748
Packaged and uploaded python-yaql, sent pull request for fixing print statements into Python3 compatible print function calls: https://github.com/ativelkov/yaql/pull/15
Packaged and uploaded python-muranoclient.
Started the packaging of Murano (not finished yet).
Uploaded python-keystoneclient 0.10.1-2 with the CVE-2014-7144 fix to Sid, with urgency=high. Uploaded 0.11.1-1 to Experimental.
Uploaded python-keystonemiddleware fix for CVE-2014-7144.
Uploaded openstack-trove 2014.2~b3-3 with last unit test fix from https://review.openstack.org/#/c/123450/ Friday 26:
Uploaded a fix for murano-agent, which makes it run as root.
Finished the packaging of Murano
Started packaging murano-dashboard, sent this patch to fix the wrong usage of the /usr/bin/coverage command: https://review.openstack.org/124444
Fixed wrong BASE_DIR in python-xstatic-angular and python-xstatic-angular-mock Saturday 27:
uploaded python-xstatic-boostrap-scss which I forgot to upload :(
uploaded python-pyscss 1.2.1 Sunday 28:
After a long investigation, I found out that the issue when installing the openstack-dasboard package was due to a wrong patch I did for Python 3.2 in Wheezy in python-pyscss. Corrected the patch from version 1.2.1-1, and uploaded version 1.2.1-2, the dashboard now installs correctly. \o/
Did a new version of an Horizon patch at https://review.openstack.org/122992/ to address Django 1.7 compat. Monday 29:
Uploaded new version of python-pyscss fixing the last issue with Python 3 (there was a release critical bug on it).
Uploaded fixup for python-django-openstack-auth fail to build in the Sid version, which was broken since the last upload of keystoneclient (which makes some of its API now as private).
Uploaded python-glance-store 0.1.8, including Ubuntu patch to fix unit tests.
Reviewed the packaging of python-strict-rfc3339 (see https://bugs.debian.org761152).
Uploaded Sheepdog with fix in the init script to start after corosync (Closes: #759216).
Uploaded pt_BR.po Brazilian Portuguese debconf templates translation for nova Icehouse in Sid (only commited it in Git for Juno).
Same for Glance. Tuesday 30:
Added Python3 support in python-django-appconf, uploaded to Sid
Upgraded to python-django-pyscss 1.0.3, and fixed broken unit tests with this new release under Django 1.7. Created pull request: https://github.com/fusionbox/django-pyscss/pull/22
Fixed designate requirements.txt in Sid (Icehouse) to allow SQLA 0.9.x. Uploaded resulting package to Sid.
Uploaded new Debian fix for python-tooz: kills memcached only if the package scripts started it (plus cleans .testrepository on clean).
Uploaded initial release of murano
Uploaded python-retrying with patch from Ubuntu to remove embedded copy of six.py code.
Uploaded python-oslo.i18n 1.0.0 to experimental (same as before, just bump of version #)
Uploaded python-oslo.utils 1.0.0 to experimental (same as before, just bump of version #)
Uploaded Keystone Juno RC1
Uploaded Glance Juno RC1

24 September 2014

Robert Collins: what-poles-for-the-tent

So Monty and Sean have recently blogged about about the structures (1, 2) they think may work better for OpenStack. I like the thrust of their thinking but had some mumblings of my own to add. Firstly, I very much like the focus on social structure and needs what our users and deployers need from us. That seems entirely right. And I very much like the getting away from TC picking winners and losers. That was never an enjoyable thing when I was on the TC, and I don t think it has made OpenStack better. However, the thing that picking winners and losers did was that it allowed users to pick an API and depend on it. Because it was the X API for OpenStack . If we don t pick winners, then there is no way to say that something is the X API for OpenStack , and that means that there is no forcing function for consistency between different deployer clouds. And so this appears to be why Ring 0 is needed: we think our users want consistency in being able to deploy their application to Rackspace or HP Helion. They want vendor neutrality, and by giving up winners-and-losers we give up vendor neutrality for our users. Thats the only explanation I can come up with for needing a Ring 0 because its still winners and losers (e.g. picking an arbitrary project) keystone, grandfathering it in, if you will. If we really want to get out of the role of selecting projects, I think we need to avoid this. And we need to avoid it without losing vendor neutrality (or we need to give up the idea of vendor neutrality). One might say that we must pick winners for the very core just by its, but I don t think thats true. If the core is small, many people will still want vendor neutrality higher up the stack. If the core is large, then we ll have a larger % of APIs covered and stable granting vendor neutrality. So a core with fixed APIs will be under constant pressure to expand: not just from developers of projects, but from users that want API X to be fixed and guaranteed available and working a particular way at [most] OpenStack clouds. Ring 0 also fulfils a quality aspect we can check that it all works together well in a realistic timeframe with our existing tooling. We are essentially proposing to pick functionality that we guarantee to users; and an API for that which they have everywhere, and the matching implementation we ve tested. To pull from Monty s post: What does a basic end user need to get a compute resource that works and seems like a computer? (end user facet) What does Nova need to count on existing so that it can provide that. He then goes on to list a bunch of things, but most of them are not needed for that: We need Nova (its the only compute API in the project today). We don t need keystone (Nova can run in noauth mode and deployers could just have e.g. Apache auth on top). We don t need Neutron (Nova can do that itself). We don t need cinder (use local volumes). We need Glance. We don t need Designate. We don t need a tonne of stuff that Nova has in it (e.g. quotas) end users kicking off a simple machine have -very- basic needs. Consider the things that used to be in Nova: Deploying containers. Neutron. Cinder. Glance. Ironic. We ve been slowly decomposing Nova (yay!!!) and if we keep doing so we can imagine getting to a point where there truly is a tightly focused code base that just does one thing well. I worry that we won t get there unless we can ensure there is no pressure to be inside Nova to win . So there s a choice between a relatively large set of APIs that make the guaranteed available APIs be comprehensive, or a small set that that will give users what they need just at the beginning but might not be broadly available and we ll be depending on some unspecified process for the deployers to agree and consolidate around what ones they make available consistently. In sort one of the big reasons we were picking winners and losers in the TC was to consolidate effort around a single API not implementation (keystone is already on its second implementation). All the angst about defcore and compatibility testing is going to be multiplied when there is lots of ecosystem choice around APIs above Ring 0, and the only reason that won t be a problem for Ring 0 is that we ll still be picking winners. How might we do this? One way would be to keep picking winners at the API definition level but not the implementation level, and make the competition be able to replace something entirely if they implement the existing API [and win hearts and minds of deployers]. That would open the door to everything being flexible and its happened before with Keystone. Another way would be to not even have a Ring 0. Instead have a project/program that is aimed at delivering the reference API feature-set built out of a single, flat Big Tent and allow that project/program to make localised decisions about what components to use (or not). Testing that all those things work together is not much different than the current approach, but we d have separated out as a single cohesive entity the building of a product (Ring 0 is clearly a product) from the projects that might go into it. Projects that have unstable APIs would clearly be rejected by this team; projects with stable APIs would be considered etc. This team wouldn t be the TC : they too would be subject to the TC s rulings. We could even run multiple such teams as hinted at by Dean Troyer one of the email thread posts. Running with that I d then be suggesting So OpenStack/NaaS would have an API or set of APIs, and they d be responsible for considering maturity, feature set, and so on, but wouldn t own Neutron, or Neutron incubator or any other component they would be a *cross project* team, focused at the product layer, rather than the component layer, which nearly all of our folk end up locked into today. Lastly Sean has also pointed out that we have large N N^2 communication issues I think I m proposing to drive the scope of any one project down to a minimum, which gives us more N, but shrinks the size within any project, so folk don t burn out as easily, *and* so that it is easier to predict the impact of changes clear contracts and APIs help a huge amount there.

17 September 2014

Steve Kemp: If this goes well I have a new blog engine

Assuming this post shows up then I'll have successfully migrated from Chronicle to a temporary replacement. Chronicle is awesome, and despite a lack of activity recently it is not dead. (No activity because it continued to do everything I needed for my blog.) Unfortunately though there is a problem with chronicle, it suffers from a bit of a performance problem which has gradually become more and more vexing as the nubmer of entries I have has grown. When chronicle runs it : In the general case you rebuild a blog because you've made a entry, or received a new comment. There is some code which tries to use memcached for caching, but in general chronicle just isn't fast and it is certainly memory-bound if you have a couple of thousand entries. Currently my test data-set contains 2000 entries and to rebuild that from a clean start takes around 4 minutes, which is pretty horrific. So what is the alternative? What if you could parse each post once, add it to an SQLite database, and then use that for writing your output pages? Instead of the complex data-structure in-RAM and the need to parse a zillion files you'd have a standard/simple SQL structure you could use to build a tag-cloud, an archive, & etc. If you store the contents of the parsed-blog, along with the mtime of the source file you can update it if the entry is changed in the future, as I sometimes make typos which I only spot once Ive run make steve on my blog sources. Not surprisingly the newer code is significantly faster if you have 2000+ posts. If you've imported the posts into SQLite the most recent entries are updated in 3 seconds. If you're starting cold, parsing each entry, inserting it into SQLite, and then generating the blog from scratch the build time is still less than 10 seconds. The downside is that I've removed features, obviously nothing that I use myself. Most notably the calendar view is gone, as is the ability to use date-based URLs. Less seriously there is only a single theme, which is what is used upon this site. In conclusion I've written something last night which is a stepping stone between the current chronicle and chronicle2 which will appear in due course.
PS. This entry was written in markdown, just because I wanted to be sure it worked.

15 September 2014

Cyril Brulebois: Freelance Debian consultant

I m not used to talking about my day job but here s an exception. Over the past few years I worked in two startups (3 years each). It was nice to spend time in different areas: one job was mostly about research and development in a Linux cluster environment; the other one was about maintaining a highly-customized, Linux-based operating system, managing a small support team, and performing technological surveillance in IT security. In the meanwhile I ve reached a milestone: 10 years with Debian. I had been wondering for a few months whether I could try my luck going freelance, becoming a Debian consultant. I finally decided to go ahead and started in August! The idea is to lend a hand for various Debian-related things like systems administration, development/debugging, packaging/repository maintenance, or Debian Installer support, be it one-shot or on a regular basis. I didn t think about trainings/workshops at first but sharing knowledge is something I ve always liked, even if I didn t become a teacher. For those interested, details can be found on my website: https://mraw.org/. Of course this doesn t mean I m going to put an end to my volunteer activities within Debian, especially as a Debian Installer release manager. Quite the contrary in fact! See the August and September debian-boot@ archives, which have been busy months. :)

Gergely Nagy: Looking ahead

A little more than a year ago, I started working as a syslog-ng OSE developer full-time. That was a tremendously important milestone in my career, as one of the goals I wanted to achieve in life - to work on free software for a living - became a reality. Rocket boots were fired up, and we accomplised quite a lot in the past year, and I'm very, very proud of the work we did - we, the whole community. I enjoyed every bit of it, but as it turns out, some of the other desires I wish to pursue, and new challenges I am looking for, will lead me in a different direction. At the end of August, I handed in my resignation, and the past friday was my last work day at BalaBit.There are a few questions that will likely be asked, and I'll try my best to answer them beforehand. Questions such as: What will happen to syslog-ng?, Who will be the new maintainer?, How does this affect your Debian work?, and so on, and so forth.syslog-ngWhat will happen to syslog-ng?After careful consideration, with the syslog-ng team at BalaBit, we decided that they will take over OSE-related responsibilities as well. They will do releases, they will engage the people on GitHub and the mailing list, they will take care of the @sngOSE twitter account.During the past few months, we've been working on pushing the team closer to the community: we moved issues to GitHub, the team submitted pull requests, and in general, we moved closer to each other in every possible way.While they may not have the open source maintainer experience I had, they are all capable folk, and will quickly learn on the job. The team taking over also has the advantage of having more manpower, and better collaboration between the premium edition and OSE.Who will be the syslog-ng maintainer?There will be no single bottle neck. This has both good and bad implications, but I believe the good ones outweigh the disadvantages.What about the roadmap? When will 3.6.1 happen?The roadmap is already laid out for 3.6, but it is the team's judgement when it will be released. The latest 3.6.0beta2 was released together with the team, too. I expect there will be delays (I planned 3.6.1 to be released on September 27th), but not much; a few weeks, perhaps.What will your involvement in syslog-ng development be?With changing jobs, my involvement will drastically decrease. I will remain a syslog-ng user, and I will continue packaging it for Debian and Ubuntu, and will keep my unofficial repository running. I do not see myself contributing much, perhaps bug reports, opinions and an occasional idea.Nevertheless, my expertise is still considerable, and I will help the team and the community in whatever way I can - but I will be severely time constrained.DebianWhile BalaBit were very free software friendly, and allowed me to work on Debian tasks from time to time, my other activities took up an increasingly big chunk of my paid time, and I had to cut back a little. At the new job, things will be a bit different: while I won't have as much time to do Debian work on the job, I will have a lot more free time, in which I hope to do more for Debian.I will - as promised above - continue maintaining syslog-ng, and all other packages I currently maintain. Apart from those, I wish to make myself more useful within the Clojure and Hy teams.MiscellaneousWhere to are you moving?This is something I will answer in due time. For now, I will have two weeks off, which I wish to spend my way, picking up projects I neglected for far too long.

10 September 2014

Ian Donnelly: New Release: Elektra 0.8.8

Hi Everybody! Great news! I am very happy to announce that we have reached a new milestone for Elektra and released a new version, 0.8.8! This release comes right on the tail of the 0.8.7 release and it might just be our biggest release yet! We already have a great article covering all the changes from the previous release on our News documentation on GitHub. I just wanted to focus on a few of those changes on this blog, especially the ones that pertain to my Google Summer of Code Project. First of all, Felix has worked to greatly improve the ini plug-in. This is the plug-in I used in my technical demo for mounting Samba s smb.conf file. It now works even better with complex ini files such as smb.conf which means the automatic merging of files like smb.conf is even better now! That really goes to show one of the greatest strengths of the design of Elektra. Just by improving plug-ins, all the functions of Elektra can improve as well. The merge code was not changed in this release, yet because of an updated plug-in, the merge has improved. Secondly, there have been some good improvements to the kdb command-line tool. Many of these improvements were used in my technical demo, but now they are actually a part of release (and a little more refined from then). We added a new command called kdb remount which allows a user (or script) to mount a file to the Elektra Key Database using an existing backend. An example of this command is:
kdb remount [new filename] [new path] [existing mountpoint]

This command mounts the new file to the new path in the Key Database using an existing backend. This works with the conffile merging by allowing us to mount the various versions of the conffile without having to specify which backend to use (it will use the same backend as the currently used conffile). Additionally, the umount command was updated to allow users to umount using the current mountpath (allowing commands such as kdb umount system/smb.conf) as opposed to backends. Moreover, we added an option to the kdb import command to specify a merge strategy using thing -s option. Now you can import a file into the Key Database and merge the content of that file with the current Keys in the Database. Thirdly, we added some new scripts to Elektra to help with the ucf integration. These scripts were used in my technical demo, but now they are part of the release. elektra-mount and elektra-umount are wrappers for the commands kdb mount and kdb umount respectively. They are designed to be used in debian package scripts and are adapted for easier use than the generic commands. For instance, running elektra-mount will check to see if a file is already mounted at that location in the Key Database. Similarly, elektra-umount will not produce an error if the file was already unmounted. This is because maintainer scripts can be run multiple times in a row and producing an error will stop dpkg even when it shouldn t. Additionally, we added a script called elektra-merge which can be used as a method for ucf to merge configuration files. This script acts as a liaison between ucf and elektra allowing automatic merges to be done by ucf using Elektra s merge features in a seamless manner. For information of how these scripts work, check out my tutorial on integrating elektra-merge into a debian package. The last bit of news I would like to share is the great progress of the Debian package. Thanks to Pino Toscano, version 0.8.7-4 of Elektra is now available in the Debian testing repo! This is great news as we are now that much closer to replacing the outdated Elektra 0.7 versions that are currently the latest versions of Elektra in the stable repo. Once the 0.8.X versions of Elektra make it to stable it will be much easier for us to keep the latest versions of Elektra in Debian, and that s key to allowing Elektra to help improve users lives. You can download the release from:
http://www.markus-raab.org/ftp/elektra/releases/elektra-0.8.8.tar.gz size: 1644441
md5sum: fe11c6704b0032bdde2d0c8fa5e1c7e3
sha1: 16e43c63cd6d62b9fce82cb0a33288c390e39d12
sha256: ae75873966f4b5b5300ef5e5de5816542af50f35809f602847136a8cb21104e2 And the API-Documentation can be found here: http://doc.libelektra.org/api/0.8.8/html/ Hope you enjoy the new release! Sincerely,
Ian S. Donnelly

6 September 2014

Russ Allbery: Accumulated hauls

I haven't made one of these in a long time, so I have some catching from random purchases to do, which includes a (repurposed) nice parting gift from my previous employer and a trip to Powell's since I was in the area for DebConf14. This also includes the contents of the Hugo voter's packet, which contained a wide variety of random stuff even if some of the novels were represented only by excerpts. John Joseph Adams (ed.) The Mad Scientist's Guide to World Domination (sff anthology)
Roger McBride Allen The Ring of Charon (sff)
Roger McBride Allen The Shattered Sphere (sff)
Iain M. Banks The Hydrogen Sonata (sff)
Julian Barnes The Sense of an Ending (mainstream)
M. David Blake (ed.) 2014 Campbellian Anthology (sff anthology)
Algis Budrys Benchmarks Continued (non-fiction)
Algis Budrys Benchmarks Revisited (non-fiction)
Algis Budrys Benchmarks Concluded (non-fiction)
Edgar Rice Burroughs Carson of Venus (sff)
Wesley Chu The Lives of Tao (sff)
Ernest Cline Ready Player One (sff)
Larry Correia Hard Magic (sff)
Larry Correia Spellbound (sff)
Larry Correia Warbound (sff)
Sigrid Ellis & Michael Damien Thomas (ed.) Queer Chicks Dig Time Lords (non-fiction)
Neil Gaiman The Ocean at the End of the Lane (sff)
Max Gladstone Three Parts Dead (sff)
Max Gladstone Two Serpents Rise (sff)
S.L. Huang Zero Sum Game (sff)
Robert Jordan & Brandon Sanderson The Wheel of Time (sff)
Drew Karpyshyn Mass Effect: Revelation (sff)
Justin Landon & Jared Shurin (ed.) Speculative Fiction 2012 (non-fiction)
John J. Lumpkin Through Struggle, the Stars (sff)
L. David Marquet Turn the Ship Around! (non-fiction)
George R.R. Martin & Raya Golden Meathouse Man (graphic novel)
Ramez Naam Nexus (sff)
Eiichiro Oda One Piece Volume 1 (manga)
Eiichiro Oda One Piece Volume 2 (manga)
Eiichiro Oda One Piece Volume 3 (manga)
Eiichiro Oda One Piece Volume 4 (manga)
Alexei Panshin New Celebrations (sff)
K.J. Parker Devices and Desires (sff)
K.J. Parker Evil for Evil (sff)
Sofia Samatar A Stranger in Olondria (sff)
John Scalzi The Human Division (sff)
Jonathan Straham (ed.) Fearsome Journeys (sff anthology)
Vernor Vinge The Children of the Sky (sff)
Brian Wood & Becky Cloonan Demo (graphic novel)
Charles Yu How to Live Safely in a Science Fictional Universe (sff) A whole bunch of this is from the Hugo voter's packet, and since the Hugos are over, much of that probably won't get prioritized. (I was very happy with the results of the voting, though.) Other than that, it's a very random collection of stuff, including a few things that I picked up based on James Nicoll's reviews. Now that I have a daily train commute, I should pick up the pace of reading, and as long as I can find enough time in my schedule to also write reviews, hopefully there will be more content in this blog shortly.

31 August 2014

Steve Kemp: A diversion - The National Health Service

Today we have a little diversion to talk about the National Health Service. The NHS is the publicly funded healthcare system in the UK.
Actually there are four such services in the UK, only one of which has this name:
  • The national health service (England)
  • Health and Social Care in Northern Ireland.
  • NHS Scotland.
  • NHS Wales.
In theory this doesn't matter, if you're in the UK and you break your leg you get carried to a hospital and you get treated. There are differences in policies because different rules apply, but the basic stuff "free health care" applies to all locations. (Differences? In Scotland you get eye-tests for free, in England you pay.)
My wife works as an accident & emergency doctor, and has recently changed jobs. Hearing her talk about her work is fascinating. The hospitals she's worked in (Dundee, Perth, Kirkcaldy, Edinburgh, Livingstone) are interesting places. During the week things are usually reasonably quiet, and during the weekend things get significantly more busy. (This might mean there are 20 doctors to hand, versus three at quieter times.) Weekends are busy largely because people fall down hills, get drunk and fight, and are at home rather than at work - where 90% of accidents occur. Of course even a "quiet" week can be busy, because folk will have heart-attacks round the clock, and somebody somewhere will always be playing with a power tool, a ladder, or both! So what was the point of this post? Well she's recently transferred to working for a childrens hospital (still in A&E) and the patiences are so very different. I expected the injuries/patients she'd see to differ. Few 10 year olds will arrive drunk (though it does happen), and few adults fall out of trees, or eat washing machine detergent, but talking to her about her day when she returns home is fascinating how many things are completely different from how I expected. Adults come to hospital mostly because they're sick, injured, or drunk. Children come to hospital mostly because their parents are paranoid. A child has a rash? Doctors are closed? Lets go to the emergency ward! A child has fallen out of a tree and has a bruise, a lump, or complains of pain? Doctors are closed? Lets go to the emergency ward! I've not kept statistics, though I wish I could, but it seems that she can go 3-5 days between seeing an actually injured or chronicly-sick child. It's the first-time-parents who bring kids in when they don't need to. Understandable, completely understandable, but at the same time I'm sure it is more than a little frustrating for all involved. Finally one thing I've learned, which seems completely stupid, is the NHS-Scotland approach to recruitment. You apply for a role, such as "A&E doctor" and after an interview, etc, you get told "You've been accepted - you will now work in Glasgow". In short you apply for a post, and then get told where it will be based afterward. There's no ability to say "I'd like to be a Doctor in city X - where I live", you apply, and get told where it is post-acceptance. If it is 100+ miles away you either choose to commute, or decline and go through the process again. This has lead to Kirsi working in hospitals with a radius of about 100km from the city we live in, and has meant she's had to turn down several posts. And that is all I have to say about the NHS for the moment, except for the implicit pity for people who have to pay (inflated and life-changing) prices for things in other countries.

29 August 2014

Antti-Juhani Kaijanaho: Licentiate Thesis is now publicly available

My recently accepted Licentiate Thesis, which I posted about a couple of days ago, is now available in JyX. Here is the abstract again for reference: Kaijanaho, Antti-Juhani
The extent of empirical evidence that could inform evidence-based design of programming languages. A systematic mapping study.
Jyv skyl : University of Jyv skyl , 2014, 243 p.
(Jyv skyl Licentiate Theses in Computing,
ISSN 1795-9713; 18)
ISBN 978-951-39-5790-2 (nid.)
ISBN 978-951-39-5791-9 (PDF)
Finnish summary Background: Programming language design is not usually informed by empirical studies. In other fields similar problems have inspired an evidence-based paradigm of practice. Central to it are secondary studies summarizing and consolidating the research literature. Aims: This systematic mapping study looks for empirical research that could inform evidence-based design of programming languages. Method: Manual and keyword-based searches were performed, as was a single round of snowballing. There were 2056 potentially relevant publications, of which 180 were selected for inclusion, because they reported empirical evidence on the efficacy of potential design decisions and were published on or before 2012. A thematic synthesis was created. Results: Included studies span four decades, but activity has been sparse until the last five years or so. The form of conditional statements and loops, as well as the choice between static and dynamic typing have all been studied empirically for efficacy in at least five studies each. Error proneness, programming comprehension, and human effort are the most common forms of efficacy studied. Experimenting with programmer participants is the most popular method. Conclusions: There clearly are language design decisions for which empirical evidence regarding efficacy exists; they may be of some use to language designers, and several of them may be ripe for systematic reviewing. There is concern that the lack of interest generated by studies in this topic area until the recent surge of activity may indicate serious issues in their research approach. Keywords: programming languages, programming language design, evidence-based paradigm, efficacy, research methods, systematic mapping study, thematic synthesis

23 August 2014

Antti-Juhani Kaijanaho: A milestone toward a doctorate

Yesterday I received my official diploma for the degree of Licentiate of Philosophy. The degree lies between a Master s degree and a doctorate, and is not required; it consists of the coursework required for a doctorate, and a Licentiate Thesis, in which the student demonstrates good conversance with the field of research and the capability of independently and critically applying scientific research methods (official translation of the Government decree on university degrees 794/2004, Section 23 Paragraph 2). The title and abstract of my Licentiate Thesis follow:
Kaijanaho, Antti-Juhani
The extent of empirical evidence that could inform evidence-based design of programming languages. A systematic mapping study.
Jyv skyl : University of Jyv skyl , 2014, 243 p.
(Jyv skyl Licentiate Theses in Computing,
ISSN 1795-9713; 18)
ISBN 978-951-39-5790-2 (nid.)
ISBN 978-951-39-5791-9 (PDF)
Finnish summary Background: Programming language design is not usually informed by empirical studies. In other fields similar problems have inspired an evidence-based paradigm of practice. Central to it are secondary studies summarizing and consolidating the research literature. Aims: This systematic mapping study looks for empirical research that could inform evidence-based design of programming languages. Method: Manual and keyword-based searches were performed, as was a single round of snowballing. There were 2056 potentially relevant publications, of which 180 were selected for inclusion, because they reported empirical evidence on the efficacy of potential design decisions and were published on or before 2012. A thematic synthesis was created. Results: Included studies span four decades, but activity has been sparse until the last five years or so. The form of conditional statements and loops, as well as the choice between static and dynamic typing have all been studied empirically for efficacy in at least five studies each. Error proneness, programming comprehension, and human effort are the most common forms of efficacy studied. Experimenting with programmer participants is the most popular method. Conclusions: There clearly are language design decisions for which empirical evidence regarding efficacy exists; they may be of some use to language designers, and several of them may be ripe for systematic reviewing. There is concern that the lack of interest generated by studies in this topic area until the recent surge of activity may indicate serious issues in their research approach. Keywords: programming languages, programming language design, evidence-based paradigm, efficacy, research methods, systematic mapping study, thematic synthesis
A Licentiate Thesis is assessed by two examiners, usually drawn from outside of the home university; they write (either jointly or separately) a substantiated statement about the thesis, in which they suggest a grade. The final grade is almost always the one suggested by the examiners. I was very fortunate to have such prominent scientists as Dr. Stefan Hanenberg and Prof. Stein Krogdahl as the examiners of my thesis. They recommended, and I received, the grade very good (4 on a scale of 1 5). The thesis has been accepted for publication published in our faculty s licentiate thesis series and will in due course appear has appeared in our university s electronic database (along with a very small number of printed copies). In the mean time, if anyone wants an electronic preprint, send me email at antti-juhani.kaijanaho@jyu.fi.
Figure 1 of the thesis: an overview of the mapping processFigure 1 of the thesis: an overview of the mapping process
As you can imagine, the last couple of months in the spring were very stressful for me, as I pressed on to submit this thesis. After submission, it took me nearly two months to recover (which certain people who emailed me on Planet Haskell business during that period certainly noticed). It represents the fruit of almost four years of work (way more than normally is taken to complete a Licentiate Thesis, but never mind that), as I designed this study in Fall 2010.
Figure 8 of the thesis: Core studies per publication yearFigure 8 of the thesis: Core studies per publication year
Recently, I have been writing in my blog a series of posts in which I have been trying to clear my head about certain foundational issues that irritated me during the writing of the thesis. The thesis contains some of that, but that part of it is not very strong, as my examiners put it, for various reasons. The posts have been a deliberately non-academic attempt to shape the thoughts into words, to see what they look like fixed into a tangible form. (If you go read them, be warned: many of them are deliberately provocative, and many of them are intended as tentative in fact if not in phrasing; the series also is very incomplete at this time.) I closed my previous post, the latest post in that series, as follows:
In fact, the whole of 20th Century philosophy of science is a big pile of failed attempts to explain science; not one explanation is fully satisfactory. [...] Most scientists enjoy not pondering it, for it s a bit like being a cartoon character: so long as you don t look down, you can walk on air.
I wrote my Master s Thesis (PDF) in 2002. It was about the formal method called B ; but I took a lot of time and pages to examine the history and content of formal logic. My supervisor was, understandably, exasperated, but I did receive the highest possible grade for it (which I never have fully accepted I deserved). The main reason for that digression: I looked down, and I just had to go poke the bridge I was standing on to make sure I was not, in fact, walking on air. In the many years since, I ve taken a lot of time to study foundations, first of mathematics, and more recently of science. It is one reason it took me about eight years to come up with a doable doctoral project (and I am still amazed that my department kept employing me; but I suppose they like my teaching, as do I). The other reason was, it took me that long to realize how to study the design of programming languages without going where everyone has gone before. Debian people, if any are still reading, may find it interesting that I found significant use for the dctrl-tools toolset I have been writing for Debian for about fifteen years: I stored my data collection as a big pile of dctrl-format files. I ended up making some changes to the existing tools (I should upload the new version soon, I suppose), and I wrote another toolset (unfortunately one that is not general purpose, like the dctrl-tools are) in the process. For the Haskell people, I mainly have an apology for not attending to Planet Haskell duties in the summer; but I am back in business now. I also note, somewhat to my regret, that I found very few studies dealing with Haskell. I just checked; I mention Haskell several times in the background chapter, but it is not mentioned in the results chapter (because there were not studies worthy of special notice). I am already working on extending this work into a doctoral thesis. I expect, and hope, to complete that one faster.

Tim Retout: Packaging pump.io for Debian

I intend to intend to package pump.io for Debian. It's going to take a long time, but I don't know whether that's weeks or years yet. The world needs decentralized social networking. I discovered the tools that let me create this wiki summary of the progress in pump.io packaging. There are at least 35 dependencies that need uploading, so this would go a lot faster if it weren't a solo effort - if anyone else has some time, please let me know! But meanwhile I'm hoping to build some momentum. I think it's important to keep the quality of the packaging as high as possible, even while working through so many. It would cost a lot of time later if I had to go back and fix bugs in everything. I really want to be running the test suites in these builds, but it's not always easy. One of the milestones along the way might be packaging nodeunit. Nodeunit is a Nodejs unit testing framework (duh), used by node-bcrypt (and, unrelatedly, statsd, which would be pretty cool to have in Debian too). Last night I filed eight pull requests to try and fix up copyright/licensing issues in dependencies of nodeunit. Missing copyright statements are one of the few things I can't fix by myself as a packager. All I can do is wait, and package other dependencies in the meantime. Fortunately there are plenty of those. And I have not seen so many issues in direct dependencies of pump.io itself - or at least they've been fixed in git.

Tim Retout: Packaging pump.io for Debian

I intend to intend to package pump.io for Debian. It's going to take a long time, but I don't know whether that's weeks or years yet. The world needs decentralized social networking. I discovered the tools that let me create this wiki summary of the progress in pump.io packaging. There are at least 35 dependencies that need uploading, so this would go a lot faster if it weren't a solo effort - if anyone else has some time, please let me know! But meanwhile I'm hoping to build some momentum. I think it's important to keep the quality of the packaging as high as possible, even while working through so many. It would cost a lot of time later if I had to go back and fix bugs in everything. I really want to be running the test suites in these builds, but it's not always easy. One of the milestones along the way might be packaging nodeunit. Nodeunit is a Nodejs unit testing framework (duh), used by node-bcrypt (and, unrelatedly, statsd, which would be pretty cool to have in Debian too). Last night I filed eight pull requests to try and fix up copyright/licensing issues in dependencies of nodeunit. Missing copyright statements are one of the few things I can't fix by myself as a packager. All I can do is wait, and package other dependencies in the meantime. Fortunately there are plenty of those. And I have not seen so many issues in direct dependencies of pump.io itself - or at least they've been fixed in git.

Next.

Previous.