Search Results: "nate"

20 February 2014

Andrew Pollock: [life] Day 23, Playgroup and swimming and some more taxes

Today was a pretty good day. I've found that I quite enjoyed it. Zoe woke up a bit early, around 5:30am. She decided that despite the ridiculous heat, she wanted porridge for breakfast, with cherries in it, so we pitted the cherries and threw it all into the Thermomix to cook while she got dressed. After breakfast and the obligatory bit of TV, we biked over to Playgroup. Zoe's got a few toys that she's taken a liking to, and immediately sought them out. She also seems to really enjoy just running the length of the hall after Playgroup is all done and everything has been packed away and most of the kids are gone. Today the craft activity was making paper crowns, and she wanted to that immediately upon arrival. After we got home, while I was preparing lunch, she blinged it up a bit more with some stuff from home. She took a 2 hour nap after lunch, and I did some more work on my taxes, and then, because it was still stupidly hot, we biked over to Colmslie pool for a swim. Zoe's swimming progress continues in leaps and bounds. She's had one lesson at her new level, and she's started with a kickboard and is doing strokes, so I borrowed a kickboard from the pool, and she did really well with it. We also just fooled around with the kickboard a bit. I took it to the bottom of the pool and launched it out of the water with its own buoyancy, and Zoe tried standing on it under water like a skateboard. The sinkies were also popular again. We grabbed a few things for dinner from the Hawthorne Garage on the way home, and I managed to get Zoe to bed a little bit early. I thought I was going to have a smooth bedtime, but unfortunately, as is a common occurrence at the moment, after lights out, she started crying for her mother. It took about 15 minutes to get her from near-hysterical to calm and drowsy. I'm starting to think I might need to either shorten or eliminate the afternoon naps. One of her complaints recently has been that she's had trouble sleeping at bedtime. That said, she still seems to get to sleep relatively quickly once she gets into the right frame of mind.

18 February 2014

Christoph Egger

Writing this because there seems to be no correct documentation on the relevant google websites and it turns out to be non-trivial. Our goal here is to unsubscribe from a ordinary google group. Mails from the google group contain the quoted footer:
-- 
You received this message because you are subscribed to the Google
Groups "FOO" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to FOO+unsubscribe@googlegroups.com.
Visit this group at http://groups.google.com/group/FOO
For more options, visit https://groups.google.com/groups/opt_out.
Seems easy enough, so let's send a Mail to this FOO+unsubscribe address. Back comes a E-Mail:
From: FOO <FOO+unsubconfirm@googlegroups.com>
Subject: Unsubscribe request for FOO [ EJzZjpgFhDHd9seTdRA0 ]
To: Christoph Egger <christoph@example.com>
Date: Tue, 18 Feb 2014 18:55:24 +0000 (38 minutes, 53 seconds ago)
 [Leave This Group]
Visit Go 
[Start] your own group, [visit] the help center, or [report]
abuse.
So click on the [Leave This Group] link and be done? Unfortunately not. Looking at the link you notice it's called http://groups.google.com/group/FOO/subscribe -- no token and "subscribe"? I actually want to unsubscribe! And indeed, clicking gets an Interface that offers to "Enter the email address to subscribe:" + Captcha. And whatever it does, it -- of course -- doesn't unsubscribe. (My guess is, it would actually work if you had a real google account associated with that email address and were logged in to that account but there's no way of verifying this as already the first condition is false in this case) Now if you disable HTML completely for the email, a totally different content emerges:
Hello christoph@example.com,
We have received your request to unsubscribe from FOO. In order for us to complete the request, please reply to this email or visit the following confirmation URL:
http://groups.google.com/group/FOO/subscribe
If you have questions related to this or any other Google Group, visit the Help Center at http://groups.google.com/support/.
Thanks,
Google Groups
Still the non-functional link, however it also mentions a different solution: "please reply to this email" which was not present in the HTML mail at all. And it works.

16 February 2014

Aurelien Jarno: On configure systems

I will never understand the point of using autotools, cmake or whatever configure system, when later the code uses an hardcoded list of architectures to determine the size of a pointer Unfortunately for porters this pattern is quite common. Update: As people keep asking, the way to check for the size of a given type is explained in the autoconf manual. To check for the size of the pointer, the following entry has to be added to configure.ac: AC_CHECK_SIZEOF(void *) On a 64-bit system, this will lead to the following entry in config.h:

/* The size of void *', as computed by sizeof. */
#define SIZEOF_VOID_P 8

Mark Brown: Human factors

An issue which I always find depressing but sadly unsurprising in discussions of process with software is the frequent disregard for human elements; indeed often the goal people have in creating process is to try to control and eliminate human elements. Little thought is given to what is going to motivate people to do what s asked and if they are going to follow it at all, or in the spirit it was intended. One example I ve seen several times is the idea that some engineers work on irrelevant things and that this can be fixed by requiring every commit to be tied to the project plan or a bug so off project work is obvious. This isn t really attacking the problem so much as putting a roadblock in place to try to avert it; the real problem is normally people not communicating about what they re doing and what s important but those problems are really hard to address. Sadly what tends to happen is that people work around the roadblock and cause some collateral damage; for example devaluing the bug tracker by referencing irrelevant bugs or creating meaningless bugs solely to allow commits, while continuing to behave like they did originally. This is often worse than the original situation. This sort of issue is one of the things I appreciate most about working on Linux there s quite a bit of process but because of the way it has been evolved the incentives are usually right to make sure they are followed in the spirit in which they were intended. For example the patch submission process is essentially just best practices for making sure changes go to people who care in a form which makes it easy for them to work with it; there is a bunch of tooling around it which built on those practices (and in turn influenced the practices) but it all comes back to that basic thing of getting attention for changes and making them easy to work with. As a result people mostly do the right thing (or close enough) so everything runs smoothly. The trick is to remember to standardize and write things down when it s needed notice the good practice and spread its adoption. The key difference here seems to be if process is viewed as something to solve problems for people or if it is viewed as a way to solve problems with people. If the problem is with the people then the view starts off negative and it s perhaps unsurprising that there is little consideration of how they will react. Sometimes it s not so much that a problem is seen with people as that the people doing don t have any real interest in the process (write only timesheets often bear no relation to reality for example) but there is an assumption that they are going to just do what they re asked (but instead they for example don t fill in their time sheets or don t provide accurate information). The end result is the same, their needs don t get considered and their actions end up being counterproductive. Starting to avoid these problems is fairly straightforward take a step back and consider why the people expected to carry out the process are going to want to do so. How does it help them? How will they see it helping others? Generally what s in it for them? If these are difficult questions to answer then there may be a problem it is likely people will ignore the process or do it badly without a lot of active enforcement, but perhaps that s OK for the situation. Beware of using metrics to provide incentives, metrics are gameable and that game can cause problems without care. But do think about people; making software is all about people.

15 February 2014

Thomas Goirand: OpenStack 2013.2.2 uploaded

This is the 2nd point release of OpenStack Havana (this is the name of the current stable release of OpenStack). It was out on Thursday (US time), and I uploaded it on Friday (Chinese time). Unfortunately, I realized that the latest python-keystoneclient didn t support the token and endpoint command line options, effectively breaking Keystone itself, and all automated endpoint registration in all core packages. So after fixing Keystone and openstack-pkg-tools, I had to re-upload a 2nd Debian release. However, after this glitch, the packages passed successfully our CI testing suite: my friends at eNovance and I run the tempest functional test suite each time there s this kind of major update, so that we can validate the packages are working as expected. And all went well after the new keystone command line options were fixed in the postinst scripts. As a side note, I fail to see consistency on deprecating these interfaces. While we still have the old glance index thing in the Glance client (this is really old), keystoneclient just broke backward compatibility, when keystone is quite recent. So, it looks to me that it wasn t a great idea to just remove token and endpoint this way (though I can understand the need of consistency). Hopefully, this will all be fixed anyway, when we move to the unified python-openstackclient command lines instead of per-project clients. I realized today that the pkg-openstack team now maintains more than 100 packages. This of course includes general purpose Python modules, which probably will move to the Python module team if it one day supports Git (life is too short: I do not wish to learn such a deprecated technology as SVN). Though this is still a lot of packages, and some more are coming: as per the OpenStack TODO list wiki page, Ironic (a replacement for nova-baremetal), TripleO (OpenStack on OpenStack), Tuskar (contols where things are deployed with TripleO) and Designate (DNS as a Service) are already packaged, and I am waiting for the Icehouse (this is the name of the next stable release) beta3 to be out to upload them to Experimental. This is scheduled for the 6th of March. I believe I am currently nearly up-to-date with the current global-requirements.txt (eg: python dependencies of OpenStack), pending a few Python module upgrades, so hopefully, packaging OpenStack Icehouse beta3 will smooth.

13 February 2014

Craig Small: Google doesn t get SPF

Someone has decided to use my email address for a spam source. They have even used google to relay it which, given Googles current policies seems like a winning idea. I keep getting emails from Google s servers with header lines like this:
X-Original-Authentication-Results: mx.google.com; spf=hardfail (google.com: domain of csmall@small.dropbear.id.au does not designate 66.80.26.66 as permitted sender)
You don t say? You mean even though my SPF records do not include some dodgy server in California, even though Google knows I don t include this in my SPF records well we will let the email go through anyhow. SPF records mean that s where my email comes from. If the record has a -all at the end of it, like mine do, then it means don t accept it from anywhere else. The hardfail means Google sees the -all and still does nothing about it.

12 February 2014

Dirk Eddelbuettel: RInside 0.2.11

A new release 0.2.11 of RInside is now on CRAN. RInside provides a set of convenience classes which facilitate embedding of R inside of C++ applications and programs, using the classes and functions provided by the Rcpp R and C++ integration package. This release, the first in fourteen months, fixes one important initialization issue created by the recent Rcpp 0.11.0 release, adds a few other changes related to that release and improves a number of small points such as new or improved examples. The NEWS extract below has more details.
Changes in RInside version 0.2.11 (2014-02-11)
  • Updated for Rcpp 0.11.0:
    • Updated initialization by assigning global environment via pointer only after R itself has been initialized with special thanks to Kevin Ushey for the fix
    • Updated DESCRIPTION with Imports: instead of Depends:
    • Added correspondiing importFrom(Rcpp, evalCpp) to NAMESPACE
    • Noted in all inst/examples/*/Makefile that Rcpp no longer requires a library argument, but left code for backwards compatibility in case 0.11.0 is not yet installed.
  • Added --vanilla --slave to default arguments for R initialization
  • Added a few more explicit #include statements in the qt example which Qt 5.1 now appears to require with thanks to Spencer Behling for the patch
  • Added new MPI example with worker functions and RInside instance, kindly contributed by Nicholas Pezolano and Martin Morgan
CRANberries also provides a short report with changes from the previous release. More information is on the RInside page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

11 February 2014

Jonathan McDowell: Choosing a new laptop

Recently I've been thinking about getting a new laptop. I have this rule that a laptop should last me at least 3 years (ideally more) and my old laptop was bought in September 2010. So for the past few months I've been trying to work out if there's something suitable on the market that is a good replacement (last time I didn't manage to find something that ticked all the boxes, but did pretty well for the price I paid). To start with I decided to track my laptops over time - largely because one of my concerns was about the size of a replacement, because I have a significant leaning towards subnotebooks. In the end the reason I decided to upgrade was for some extra CPU grunt; my old machine had a tendency to get pretty hot under any sort of load.
DateModelCPUScreenRAMStorageW (mm)H (mm)D (mm)WeightCost
1991Amstrad PPC 640DNEC V30 8MHz9" 640x200 non-backlit green LCD640k2 x 3.5" FDD45023010010kg???
August 1997Compaq Aero 4/33c486sx337.8" 640x480 CSTN LCD4MB80MB260190431.6kg???
July 2002Compaq Evo N200P-III 700MHz10.4" 1024x768 TFT192MB20GB251198201.1kg 939.99
August 2005Toshiba Portege R200Pentium M 753 1.2GHz12.1" 1024x768 TFT1280MB60GB286229201.29kg 1313.58
September 2008Asus EEE 901Atom N270 1.6GHz8.9" 1024x600 TFT2GB4GB + 16GB SSD248175231.1kg 299.99
September 2010Acer Aspire 1830TCore-i5 470UM 1.33GHz11.6" 1366x768 TFT8GB500GB284203281.4kg$699.99 (~ 480)
The EEE didn't actually replace the Toshiba, but I mention it for completeness. It was actually the only machine I moved to the US with, but after about a month of it as my primary machine I realized it wasn't an option for day to day use - though it was fantastic as a machine to throw in an overnight bag, especially when coupled with a 3G dongle. I wasn't keen on significantly increasing the size of my laptop. There are a number of decent 13" Ultrabook options out there, and I looked at a few of them, but nothing grabbed me as being worth the increase in size. Also I wanted something better than the Acer - one of the major problems was finding something smaller than 13" that had 8G RAM, let alone more. There's a significant trend towards everything soldered in for the smaller/slimmer notebooks, which makes some sense but means that the base spec had better be right. Much to my surprise the Microsoft Surface Pro 2 looked like an option. It comes with an i5-4300U processor (at least since around Christmas 2013), and the 256/512G SSD models have 8G RAM. Screen resolution is an attractive true HD (1920x1080) and the 10" display means it's smaller than the Aspire. Unfortunately the keyboard lets it down. It's fine given a flat surface, but not great if you want to support the whole thing on your lap. Which is something I tend to do with my laptop, whether that's on the sofa, or in bed, or on a bus/train. Another option was the Sony Vaio Pro 11. This is a pretty sweet laptop (I managed to get to play with one at a Sony store in the US). Super slim and light. 8GB RAM. True HD screen. However I have bad memories of the build quality of the older Vaios and the fact that there was /no/ user replaceable parts put me off - it's a safe bet that a laptop battery is going to need replaced in a 3 year lifespan. What I managed to find, and purchase, was a Dell Latitude E7240. I admit that the Dell brand made me wary - while I've not had any issue with their desktops I didn't associated their laptops as being particularly high quality. Mind you, I could say the same for Acer and I've been very pleased with the Aspire (if they'd had a more up to date model I'd have bought it). I bought the E7240 with the Core-i5 4300U (so the same as the Surface Pro 2) and True HD touch screen. It has a replaceable battery, expandable RAM (up to 16G) and the storage is an mSATA SSD. It also came with a built in 3G card. At 12.5" it's a little bigger than my old machine, but I decided that was a reasonable idea given the higher resolution. I'm typing this article on it now, having finally completed the setup and migration of the data from the old laptop to it this evening. More details once I've been using it for a little bit I think.

Dirk Eddelbuettel: RcppSMC 0.1.2

Late last week, and just before leaving to participate in this crazy thing, I managed to get a new version of RcppSMC onto CRAN. RcppSMC combines the SMCTC template classes for Sequential Monte Carlo and Particle Filters (Johansen, 2009, JSS) with the Rcpp package for R/C++ Integration (Eddelbuettel and Francois, 2011, JSS) and thereby allows for easier and more direct access from R to the computational core of the Sequential Monte Carlo algorithm. This release regroups a few minor changes we accumulated over the last few months, but was triggered by the Rcpp 0.11.0 release last week. The NEWS entry is below:
Changes in RcppSMC version 0.1.2 (2014-02-06)
  • Updated for Rcpp 0.11.0 with explicit importFrom in NAMESPACE and corresponding versioned Imports: in DESCRIPTION; also removed linking instruction from src/Makevars as no it is longer needed with this new Rcpp version
  • Addded GitHub / Travis CI support
  • Use more portable dev.new() rather than x11() in pfLinearBS.R
  • Applied some corrections to pfNonlinBS.R example
  • Converted NEWS to NEWS.Rd
Courtesy of CRANberries, there is also a diffstat report relative to the previous release. As always, more detailed information is on the RcppSMC page,

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

10 February 2014

Andrew Pollock: [life] Day 13, early start, Kindergarten and messing around with a hot glue gun

I had an early start today. I set my alarm for 5:30am, and finally got going a bit after 6am, and knocked out a 10km run. It was my first 10km in about a month. It wasn't pretty, but I was looking for accomplishment, not achievement. Sarah's doing a work training course that had an early start, so she asked me if she could drop Zoe to me early and have me take care of Kindergarten drop off for her. Zoe arrived at around 8am, and I put some sunscreen on her and we jumped on the bike. I had one of the smoothest drop offs I've had since we've been back in Australia. Megan was already there, and after we got settled, the two girls started building with some large wooden blocks, and Zoe pretty much gave me a hug and a kiss and sent me on my way. She didn't require any handing off to a teacher, wasn't clingy or anything. It was wonderful to see. Today was supposed to be my Debian day, but I've got something like 3 tax returns to do for two different countries, so I got stuck into collating records instead. After lunch I allowed myself to get sidetracked with trying to figure out the Google Drive API, because I'm trying to directly scan documents to Google Drive. Then it was time to do the Kindergarten pick up. Both girls were fast asleep when I got there. I tried rousing Zoe without much success, so I went to try and wake up Megan. She was also pretty heavily sleeping, so I left her with one of the teachers to try and wake her up and returned to Zoe. She took a lot of work to wake up, but woke up happy. I was expecting a couple of packages today, and the first one was being delivered by Australia Post as we got home, so that was pretty good timing. The second one was delivered while we were home, so that worked out pretty well. For this afternoon's activity with the girls, I thought we'd make peg dolls. I've been stockpiling craft materials for almost as long as I've been back in Australia. I bought the big IKEA EXPEDIT 5 x 5 shelf, and have the bottom two rows set up as Zoe-accessible craft materials, reminiscent of the "bin wall" that the TechShop had. I've been getting stuff from Riot!, but I've also found that there's lots of good stuff to be had for cheap from Overflow. So today's activity I think came from Riot! It was your bog standard wooden clothes pin, but the kit included a conical wooden "sleeve" that I guess was supposed to be a "dress". I thought I'd take things to the next level by hot-gluing a styrofoam ball on the top of the peg, and then hot-gluing eyes and a pom-pom nose on. The girls decided on their own to glue orange cellophane for hair, and got me to draw on eyebrows and a mouth. I had a couple of sheets of felt, so I cut holes in the middle of them and used the conical wooden sleeve to hold the felt dress in place. Pipe cleaners made the arms and they were finished. The girls seemed to enjoy making and playing with them. It was the first time I've used the hot glue gun, and I wasn't sure how it was going to go. I explained to the girls that the tip was hot and to stay away from it, and let them use it under close supervision. Unfortunately Zoe learned the hard way that the glue is also very hot after it's been applied to the surface that it's going to be used on. Hot glue was definitely the way to go for a quick project where we didn't want to be sitting around waiting for PVA glue to dry. After that, the girls watched a DVD for a little while and then Megan's Dad arrived to pick her up. I started dinner a bit late, I should have started it while the girls were watching TV, so we didn't sit down to eat until later than usual. Bedtime went pretty smoothly, but also ran a bit late. Here's hoping Zoe sleeps well tonight.

9 February 2014

Thomas Goirand: Removal of XCP / Xen-API from Jessie and Sid

This is sad, but no choice, due to the lack of upstream support: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=738322 I just hope xenserver-core will work soon in Debian, so we can fix this unfortunate situation.

8 February 2014

Mario Lang: Uploading Into and Downloading From The Universe

Have you ever wondered how new ideas, concepts, artworks etc. get into the Superconscious Collective of the universe, so that they then can be downloaded by exactly the right person who is meant to do something with them? yep, I, too, think that many of these ideas, concepts etc. come from the Great Unknown, where the seeds of limitless ideas and possibilities reside. But a lot of them also come from our own minds, from people who upload these things into the Superconscious Collective of our universe. Such ideas and concepts can be uploaded in several ways. Some of them are simply uploaded by a person who sources them, connects to the Superconscious Collective of the universe and streams them up. Others are maybe even uploaded by somebody virtually typing an article with their fingertips into the spread out hands or soles of the feet of their loved one, playfully teasing him/her and thereby streaming information into him/her, which s/he then again uploads, mainly subconsciously, into the Superconscious Collective of the universe. Once uploaded, all these limitless ideas and concepts become available to everyone and every living being, on Earth and throughout the whole universe. The thing is, that most people on Earth don't know how to consciously access information from there. Some do unconsciously and it again shows up in them as entirely new concepts and ideas, which they then spice with the flavor of their own essence and then stream them out into the world, be it as music, as software, as hardware, as artwork, as science and in limitless other ways. But it is also accessed by living beings throughout the whole universe, who also add their own ideas and concepts to the Great Superconscious Collective . Wanna know how to access this limitless information, ideas, concepts and/or possibilities from the Superconscious Collective? Just breathe, relax, expand ... expand and get into your biggest, greatest, awesomest being of you. Become all that you are, connected to everything. Now, consciously plug into the Great Superconscious Collective, the Great Cosmic Internet. Allow the ideas, information, possibilities stream into you and through you and look which ones you resonate with. Call them, feel them, let them land in you, and ask them what they want you to do with them, as you are the one who can express them in the highest possible way, otherwise they wouldn't have come to you in the first place. Then do one small thing to make them real in the world and watch them unfold. Or do whatever feels right for you to do, and create the most amazing new thing in the world.

Mario Lang: boost::python and boost::variant

I have unsuccessfully tried to find a solution for the following problem on the internet several times. Now that I have come at least closer to a usable approach, I thought I'd document what I have found so that others trying to achieve a similar thing can use this as a starting point. Boost.Python offers a very nice and flexible way to interface C++ data types with Python. With just a few lines of code, and the proper linker flags, you get a Python importable shared object from your C++ compiler. This can be very productive. However, there is one aspect of C++ data types that I couldn't figure out how to interface with Python, which are C++ discriminated unions, or more specifically, heterogeneous containers. While Python has no problems with containers containing objects of different types, C++ does not make this very easy by default. Usually the problem is solved with a container of pointers to a base class, and various subclasses with virtual functions. However, this approach is not always practical, especially if the different types of objects in an heterogeneous container dont have many things in common. This is where discriminated unions come to the rescue. They basically behave like a normal union in C, but have an additional field which indicates the type of object currently stored in the union. Boost.Variant does exactly that, with a nice visitor interface added on top of it.
Heterogeneous containers in C++ If we put the boost::variant<> template inside a STL container like std::vector<>, the result is a heterogeneous container. For the purpose of illustration, lets implement such a container. The example below is deliberately simple. In reality, the various types allowed in your variant will probably have more fields then just one.
#include <boost/variant.hpp>
#include <vector>
struct a   int x;  ;
struct b   std::string y;  ;
typedef boost::variant<a, b> variant;
typedef std::vector<variant> vector;
To ease creation of these two types of objects, we are going to write a few factory functions. We are going to wrap them in Python later on.
variant make_variant()   return variant();  
vector make_vector()   return vector a(), b(), a() ;  
Boost.Python Now lets create a Python module which exports the above functionality to Python.
#include <boost/python/class.hpp>
#include <boost/python/def.hpp>
#include <boost/python/implicit.hpp>
#include <boost/python/init.hpp>
#include <boost/python/module.hpp>
#include <boost/python/object.hpp>
#include <boost/python/suite/indexing/vector_indexing_suite.hpp>
vector_indexing_suite apparently needs operator== defined on the value_type of the container. In our case, this is our boost::variant<a, b> type. Luckily, boost::variant<> already provides operator==. However, that operator== relies on operator== being defined for the underlying types. Since equality comparison is probably useful for other things as well, lets just create operator== for our two classes a and b.
bool operator==(a const &lhs, a const &rhs)   return lhs.x == rhs.x;  
bool operator==(b const &lhs, b const &rhs)   return lhs.y == rhs.y;  
Convert a boost::variant<> to PyObject * Boost.Python needs a way to convert our discriminated union to a Python object. This code relies on Python class definitions being present for all underlying variant types. We will define them later.
struct variant_to_object : boost::static_visitor<PyObject *>  
  static result_type convert(variant const &v)  
    return apply_visitor(variant_to_object(), v);
   
  template<typename T>
  result_type operator()(T const &t) const  
    return boost::python::incref(boost::python::object(t).ptr());
   
 ;
And finally, lets create our Python module.
BOOST_PYTHON_MODULE(bpv)  
  using namespace boost::python;
  class_<a>("a", init<a>()).def(init<>()).def_readwrite("x", &a::x);
  class_<b>("b", init<b>()).def(init<>()).def_readwrite("y", &b::y);
  to_python_converter<variant, variant_to_object>();
  implicitly_convertible<a, variant>();
  implicitly_convertible<b, variant>();
  def("make_variant", make_variant);
  class_<vector>("vector").def(vector_indexing_suite<vector, true>());
  def("make_vector", make_vector);
 
Compiling Lets create a shared object for Python.
$ g++ -std=c++11 -fPIC -shared $(python-config --includes) -o bpv.so file.cpp -lboost_python
Running We can load the module into Python and see what it does.
>>> import bpv
>>> variant=bpv.make_variant()
>>> variant
<bpv.a object at 0x7f06bb2130c0>
>>> variant.x
0
>>> variant.x=2
>>> variant.x
2
Nice. We can access the underlying type, and even modify it. Lets see how our heterogeneous container wrapping code behaves.
>>> vector=bpv.make_vector()
>>> vector
<bpv.vector object at 0x7f20693289d0>
>>> len(vector)
3
>>> list(vector)
[<bpv.a object at 0x7f20693190c0>, <bpv.b object at 0x7f20693193d0>, <bpv.a object at 0x7f2069319440>]
So far, so good. This will at least make it possible to convert heterogeneous containers from C++ to Python, which was my initial goal. Unfortunately, contained objects are not treated as references. Whenever retrieved, we get a copy. So in-place modification does not work.
>>> vector[0].x
0
>>> vector[0].x=2
>>> vector[0].x
0
However, we can override an existing element with a modified copy.
>>> e0=vector[0]
>>> type(e0)
<class 'bpv.a'>
>>> e0.x = 2
>>> vector[0] = e0
>>> vector[0].x
2
And we can also use the append and extend methods of Python containers.
>>> len(vector)
3
>>> vector.extend(vector)
>>> vector.append(bpv.a())
>>> len(vector)
7
>>> len(filter(lambda x: type(x)==bpv.b, vector))
2
>>> len(filter(lambda x: type(x)==bpv.a, vector))
5
>>> map(lambda x: x.x, filter(lambda x: type(x)==bpv.a, vector))
[2, 0, 2, 0, 0]
All that is missing for a perfect world is reference semantics for container elements. If anyone has a hint on how to achieve this, please let me know.

7 February 2014

John Goerzen: VirtFS isn t quite ready

Despite claims to the contrary [PDF], VirtFS the 9P-based virtio KVM/QEMU layer designed to pass through a host s filesystem to the guest is quite slow. I have yet to get it to perform at even 1/10 the speed of the virtual block device (VBD). That s unfortunate, because in theory it should be significantly faster. At this rate, I suspect even NFS will be significantly faster. Beyond that, it seems impossible to use VirtFS as the root filesystem in a VM, at least with Debian; initramfs-tools doesn t know how to build an initrd in that situation, and the support is just not there. It would make a great combination with btrfs or zfs, but unfortunately looks to be just not ready yet.

6 February 2014

Johannes Schauer: botch updates

My last update about ongoing development of botch, the bootstrap/build ordering tool chain, was three months ago with the announcement of bootstrap.debian.net. Since then a number of things happened, so I thought an update was due.

New graphs for port metrics By default, a dependency graph is created by arbitrarily choosing an installation set for binary package installation or source package compilation. Installation set vertices and source vertices are connected according to this arbitrary selection. Niels Thykier approached me at Debconf13 about the possibility of using this graph to create a metric which would be able to tell for each source package, how many other source packages would become uncompilable or how many binary packages would become uninstallable, if that source package was removed from the archive. This could help deciding about the importance of packages. More about this can be found at the thread on debian-devel. For botch, this meant that two new graph graphs can now be generated. Instead of picking an arbitrary installation set for compiling a source package or installing a binary package, botch can now create a minimum graph which is created by letting dose3 calculate strong dependencies and a maximum graph by using the dependency closure.

Build profile syntax in dpkg With dpkg 1.17.2 we now have experimental build profile support in unstable. The syntax which ended up being added was:
Build-Depends: large (>= 1.0), small <!profile.stage1>
But until packages with that syntax can hit the archive, a few more tools need to understand the syntax. The patch we have for sbuild is very simple because sbuild relies on libdpkg for dependency parsing. We have a patch for apt too, but we have to rebase it for the current apt version and have to adapt it so that it works exactly like the functionality dpkg implements. But before we can do that we have to decide how to handle mixed positive and negative qualifiers or whether to remove this feature altogether because it causes too much confusion. The relevant thread on debian-dpkg starts here.

Update to latest dose3 git Botch heavily depends on libdose3 and unfortunately requires features which are only available in the current git HEAD. The latest version packaged in Debian is 3.1.3 from October 2012. Unfortunately the current dose3 git HEAD also relies on unreleased features from libcudf. On top of that, the GraphML output of the latest ocamlgraph version (1.8.3) is also broken and only fixed in the git. For now everything is set up as git submodules but this is the major blocker preventing any packaging of botch as a Debian package. Hopefully new releases will be done soon for all involved components.

Writing and reading GraphML Botch is a collection of several utilities which are connected together in a shell script. The advantage of this is, that one does not need to understand or hack the OCaml code to use botch for different purposes. In theory it also allows to insert 3rd party tools into a pipe to further modify the data. Until recently this ability was seriously hampered by the fact that many botch tools communicated with each other through marshaled OCaml binary files which prevent everything which is not written in OCaml from modifying them. The data that was passed around like this were the dependency graphs and I initially implemented it like that because I didnt want to write a GraphML parser. I now ended up writing an xmlm based GraphML parser so as of now, botch only reads and writes ASCII text files in XML (for the graphs) and in rfc822 packages format (for Packages and Sources files) which can both easily be modified by 3rd party tools. The ./tools directory contains many Python scripts using the networkx module to modify GraphML and the apt_pkg module to modify rfc822 files.

Splitting of tools To further increase the ability to modify program execution without having to know OCaml, I split up some big tools into multiple smaller ones. Some of the smaller tools are now even written in Python which is probably much more hackable for the general crowd. I converted those tools to Python which did not need any dose3 functionality and which were simple enough so that writing them didnt take much time. I could convert more tools but that might introduce bugs and takes time which I currently dont have much of (who does?).

Gzip instead of bz2 Since around January 14, snapshot.debian.org doesnt offer bzip2 compressed Packages and Sources files anymore but uses xz instead. This is awesome for must purposes but unfortunately I had to discover that there exist no OCaml bindings for libxz. Thus, botch is now using gzip instead of bz2 until either myself or anybody else finds some time to write a libxz OCaml binding.

Self hosting Fedora Paul Wise made me aware of Harald Hoyer's attempts to bootstrap Fedora. I reproduced his steps for the Debian dependency graph and it turns out that they are a little bit bigger. I'm exchanging emails with Harald Hoyer because it might not be too hard to use botch for rpm based distributions as well because dose3 supports rpm. The article also made me aware of the tred tool which is part of graphviz and allows to calculate the transitive reduction of a graph. This can help making horrible situations much better.

Dose3 bugs I planned to generate such simplified graphs for the neighborhood of each source package on bootstrap.debian.net but then binutils stopped building binutils-gold and instead provided binutils-gold while libc6-dev breaks binutils-gold (<< 2.20.1-11). This unfortunately triggered a dose3 bug and thus bootstrap.debian.net will not generate any new results until this is fixed in dose3. Another dose3 bug affects packages which Conflicts/Replaces/Provides:bar while bar is fully virtual and are Multi-Arch:same. Binaries of different architecture with this property can currently not be co-installed with dose3. Unfortunately linux-libc-dev has this property and thus botch cannot be used to analyze cross builds until that bug is fixed in dose3. I hope I get some free time soon to be able to look at these dose3 issues myself.

More documentation Since I started to like the current set of tools and how they work together I ended up writing over 2600 words of documentation in the past few days. You can start setting up and running botch by reading the first steps and get more detailed information by reading about the 28 tools that botch makes use of as of now. All existing articles, thesis and talks are linked from the wiki home.

Andrew Pollock: [life] Day 10, Playgroup again

I was bracing myself for a rough night, but Zoe slept through until almost 6am. Today was playgroup again, so we just took a slow start to the day and headed out on the bike a bit after 9am. There was a mix of new faces and people from last week. I packed a lunchbox this time instead of relying on the snack collection in my "Dad Bag" backpack. We worked on trying to approach other girls to get them to play. Unfortunately they're all that little bit younger and more shy, so we haven't had a huge success. Zoe's been pretty happy just playing with the various toys and playing with me. Parachute time was a big hit again. I think there were about 25 adults there (I was the only Dad this week), with many with multiple children. The youngest baby I saw was a 7 week old. Zoe had been asking me what a honeydew was, so after Playgroup we dropped by the Hawthorne Garage and grabbed half a honeydew and a couple of tomatoes and I made some fritters for lunch with leftover corned beef from dinner the other night. Zoe wasn't too impressed with them. I think she didn't like the diced tomato. She declared that the honeydew was just like rockmelon. We read a story in the hammock and she went down pretty easily for nap after that. I'd just started doing the dishes when my mate Steve gave me a call. He was in the area, and he dropped in. That worked out well, because I'd been meaning to pick his brain about my little electronics project. We geeked it up for a bit. Zoe's mini lab coat arrived in the mail, and when she finally woke up after 3pm, I asked her if she wanted to go out to Bunnings to get a pair of kids-sized safety glasses, but she said she wanted to do something "fun" instead. When I asked her what she wanted to do, she initially wanted to go to Wet and Wild, but that's an all day outing and we're going next Wednesday, so she settled on going to Colmslie pool again instead. So we quickly got organised and jumped on the bike and biked over to the pool. We started out in the kid's play area, which had a couple of small slides and some fountains and whatnot, and then we gravitated over to the 50 metre pool, where Zoe practised her diving for a bit and generally quite confidently navigated the 1.5m deep end of the pool. It was good, because at least I could just stand up instead of tread water while she practised diving in off the blocks. We had a post-swim ice cream and biked home, and then I whipped up dinner and it was bathtime and bedtime. I'm going to have to start tracking my weekly kilometres with Zoe on the bike, because I think it'll be a reasonable distance.

5 February 2014

Andrew Shadura: Clearlooks-Ph nix update

Once upon a time, GTK+ 3.0 was released. That release brought at least one Bad Thing : incompatibility with GTK+ 2.x themes. At the same time, previously popular Clearlooks theme hasn t been ported. Many people didn t like that, but only one decided to DTRT to do the Right Thing. Jean-Philippe Fleury wrote Clearlooks-Ph nix (originally, Clearwaita), a GTK+ 3 theme which was supposed to have a look and feel as close as possible to the original Clearlooks. He based his work on an engine of a new GTK+ default, Adwaita theme. Quite soon, however, GTK+ 3 theme API has changed, and it became easily possible to rewrite the theme without using any additional theming engines, with just plain GTK+ stuff involved. Then GTK+ 3.6 came and broke the API once more. Jean-Philippe fixed stuff to work with newer GTK+ again, and everyone was happy again. But Empire striked back: GTK+ 3.8 brought more disruptive changes, rendering menus as ugly as never before. Unfortunately, Jean-Philippe was unable to cope with changes alone, he set up a mailing list to collectively develop Clearlooks-Ph nix, but that didn t help, and no fix has been released during more than half a year. As a maintainer and a user of Clearlooks-Ph nix, I had to do something with this, as more and more things needed recent GTK+ 3 to work. After spending some time studying CSS code and playing with its knobs, I found a solution which proved to be quite simple, and uploaded an updated package. Unfortunately, the changes aren t in the upstream package yet. Now, with GTK+ 3.10, stuff is not quite working again. Some changes in GTK+ internals lead to menu bars not being coloured properly. At the moment of this writing, an updated package is already available in unstable.

Ricardo Mones: Fixing partridge eggs with industrial duct tape

Human nature is hard to change. Very hard. We can talk about it for ages, but mistakes repeat again and again. In the end it's mostly by mistakes how we learn, so I doubt this could ever be changed without losing our own nature. One of these is trying to fix some social issue with a technical measure. Unfortunately, given the technical orientation of most of the developers, this appears from time to time in our Debian private mailing list, and yesterday I realized it's our own version of Godwin's law:

As a social problem discussion grows longer in debian-private the probability of some developer proposing a technical solution aproaches one.

Not discussing about this problems in debian-private would be a good start, but of course that would only change the name of the list in the above sentence ;-).

3 February 2014

Wouter Verhelst: Waking up

Today is the monday after FOSDEM 2014, and I'm slowly waking up. Being in charge of the video team, and after having spent a whole week preparing followed by a whole weekend stressing about, today is a day of slowly waking up from hibernation and returning gear to various rental companies. At FOSDEM 2013, we had recorded five rooms. Even though we did lose a few talks, I felt pretty good about the whole thing, and I think we did pretty well then. It had been my plan to increase the number of rooms to be recorded from 5 to a reasonably higher number say, 10 or so but someone convinced me to do all of them. I'll readily admit I was afraid. Scratch that, I was terrified. Petrified, that we wouldn't be able to pull it off. For previous editions, we'd kept the number conservative, since video work is a high-risk business: if something goes wrong with volunteers work (say, the heralding volunteer doesn't show up), you find a replacement, or you have a staff member introduce a speaker, and the show will go on. Not so with video work: if something minor goes wrong, you will most likely lose content. If something minor goes wrong in one devroom, I'll deal with it. If something minor goes wrong in 20% of devrooms, that's 4 to 5 devrooms, and I can't deal with it. Moreover, finding enough volunteers to manage the video for five rooms was a challenge last year; I couldn't even begin to imagine how to do so for 22 of them. We did work out a way that didn't seem too far-fetched, and that might work: why not look for volunteers from the devrooms themselves? But then, they couldn't be expected to have the required experience, so we'd have to do a lot of handholding. The week before FOSDEM, four of us sat down in an office in Mechelen, and started preparing. We had a lot to do; and when you do, more jobs keep being added to the pile; TODO lists tend to grow longer, not shorter. Installing servers, testing cameras, setting twinpact dipswitches, configuring laptops. Changing a few tests in the config management system, and having to go over the laptops again. Buying extra gear. Buying yet more extra gear, several times. Buying cardboard boxes when it appeared that the plastic foldable ones we'd ordered wouldn't make it in time, and unloading the pallet of them when it did, after all, so we could load them into the van with all of the other gear. Laminating instruction sheets. Receiving shipments of rented gear. Calling another supplier last minute when one of our other suppliers had contacted us to let us know one of their cameras that they'd promised us had broken down, and couldn't be repaired in time. Calling that last-minute supplier again when we found out that some of the cameras we'd rented could do HDV only (and not DV), and we needed to find more than the ones we'd already asked for. Discovering that his cameras were of the same problematic model and so also couldn't be used. The feeling of relief when he called me a few hours later and told me he'd managed to scrounge up enough cameras to help us out after all. And then it was time to go to the ULB and start setting up everything. On friday, we had a little bit of help, but did have to do most of it ourselves. After setting up everything in every room and testing as much as possible within the time constraints that we had, we picked up the most expensive bits of the gear, brought it down to a safe place, and went home. On saturday and sunday, for me it was mostly a matter of handing out gear to various devrooms, running around to fix various issues, and then in the evening receiving boxes back, using a checklist to ensure everything expected was in the box (and that things not needed would remain in the room, or be sent back). It was exhausting, to the extent that on saturday afternoon, I had to take a bit of time off so I could go and rest. I think it's safe to say that something like this hasn't been done before. FOSDEM is a huge conference; most conferences with multiple rooms don't have more somewhere between 5 and 10 of them. The fact that we have so many more, and the fact that we record all of them, puts FOSDEM video work in a class of its own, to the extent that the professional cameramen I talked to in order to get the right cameras would incredulously ask "what do you need that for?" when I gave them the numbers we needed. As such, I didn't expect, nor was aiming for, perfection. Unfortunately, with video work it is close to impossible to attain perfection; I have told multiple people during the event that I would be happy if we reached 85% of talks recorded at acceptable quality; and though it is much too soon to say for certain, my gut feeling tells m that we've probably achieved that. One thing we hadn't planned for was streaming. During the past three years, FOSDEM enlisted the help and sponsorship of flumotion to get streaming going. Unfortunately, that did not work out this year; and since we already had far too much on our plates with everything else, we decided to forego streaming and focus on recording, only. A few days before the event, however, Steinar Gunderson took it upon himself to fix that. While we couldn't support him a great deal, we could give him access to the secondary laptops (which weren't otherwise doing much anyway) on which he could then do his thing. This was mostly transparent to us; we did communicate to some extent (e.g., when a machine that had been down had fixed and was working again), but he mostly did his thing while we did ours. Full details, for those who want it, in the linked blog post. Today, then, I spent most of my time handling hardware: waiting for other members of FOSDEM to drive the van up to my office; waiting for the laptop rental company to retrieve most of the laptops; waiting for two of the camera rental companies to retrieve their cameras and tripods; driving over to the final camera rental company to return their cameras and tripods; and finally, driving up to the ULB to retrieve the last laptops who had been copying files all night. As I write this, some video files have already been uploaded to the FOSDEM video archive. However, these are extremely low-quality renditions of video snippets that need to be reviewed; they are not ready yet for public consumption. When those files exist, expect an announcement through the main FOSDEM website.

Dirk Eddelbuettel: Rcpp 0.11.0

A new major release 0.11.0 of Rcpp is now on the CRAN network for GNU R and binaries for Debian have been uploaded as well. Before you read on, please note that this release will most likely require a reinstallation of all your packages using Rcpp as it now works without a user-facing shared library. The biggest change in this version is that it is now effectively headers-only. Before you wonder, there is still compiled code provided by Rcpp for use by packages. But this uses the function registration mechanism of GNU R instead via instantiation at package startup. This does make package building easier, will remove the need to query GNU R for the Rcpp library file in src/Makevars when compiling, and should generally avoid building issues such as the dreaded fails on paths with spaces still annoying users of a certain OS. There are a number of other fixes, upgrades and other extensions detailed in NEWS file extract below, in the ChangeLog file in the package and on the Rcpp Changelog page as well as in a release announcement I'll post later.
Changes in Rcpp version 0.11.0 (2014-02-02)
  • Changes in Rcpp API:
    • Functions provided/used by Rcpp are now registered with R and instantiated by client package alleviating the need for explicit linking against libRcpp which is therefore no longer created.
    • Updated the Rcpp.package.skeleton() function accordingly.
    • New class StretchyList for pair lists with fast addition of elements at the front and back. This abstracts the 3 functions NewList, GrowList and Insert used in various packages and in parsers in R.
    • The function dnt, pnt, qnt sugar functions were incorrectly expanding to the no-degree-of-freedoms variant.
    • Unit tests for pnt were added.
    • The sugar table function did not handle NAs and NaNs properly for numeric vectors. Fixed and tests added.
    • The internal coercion mechanism mapping numerics to strings has been updated to better match R (specifically with Inf, -Inf, and NaN.)
    • Applied two bug fixes to Vector sort() and RObject definition spotted and corrected by Kevin Ushey
    • New checkUserInterrupt() function that provides a C++ friendly implementation of R_CheckUserInterrupt.
  • Changes in Rcpp attributes:
    • Embedded R code chunks in sourceCpp are now executed within the working directory of the C++ source file.
    • Embedded R code chunks in sourceCpp can now be disabled.
  • Changes in Rcpp documentation:
    • The Rcpp-FAQ and Rcpp-package vignettes have been updated and expanded.
    • Vignettes are now typeset with grey background for code boxes.
    • The bibtex reference file has been update to reflexct current package versions.
  • Changes in Rcpp unit tests:
    • The file tests/doRUnit.R was rewritten following the pattern deployed in RProtoBuf which is due to Murray Stokely
    • The function test() was rewritten; it provides an easy entry point to running unit tests of the installed package
Thanks to CRANberries, you can also look at a diff to the previous release 0.10.6. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Next.

Previous.