Search Results: "ag"

11 February 2014

Jonathan McDowell: Choosing a new laptop

Recently I've been thinking about getting a new laptop. I have this rule that a laptop should last me at least 3 years (ideally more) and my old laptop was bought in September 2010. So for the past few months I've been trying to work out if there's something suitable on the market that is a good replacement (last time I didn't manage to find something that ticked all the boxes, but did pretty well for the price I paid). To start with I decided to track my laptops over time - largely because one of my concerns was about the size of a replacement, because I have a significant leaning towards subnotebooks. In the end the reason I decided to upgrade was for some extra CPU grunt; my old machine had a tendency to get pretty hot under any sort of load.
DateModelCPUScreenRAMStorageW (mm)H (mm)D (mm)WeightCost
1991Amstrad PPC 640DNEC V30 8MHz9" 640x200 non-backlit green LCD640k2 x 3.5" FDD45023010010kg???
August 1997Compaq Aero 4/33c486sx337.8" 640x480 CSTN LCD4MB80MB260190431.6kg???
July 2002Compaq Evo N200P-III 700MHz10.4" 1024x768 TFT192MB20GB251198201.1kg 939.99
August 2005Toshiba Portege R200Pentium M 753 1.2GHz12.1" 1024x768 TFT1280MB60GB286229201.29kg 1313.58
September 2008Asus EEE 901Atom N270 1.6GHz8.9" 1024x600 TFT2GB4GB + 16GB SSD248175231.1kg 299.99
September 2010Acer Aspire 1830TCore-i5 470UM 1.33GHz11.6" 1366x768 TFT8GB500GB284203281.4kg$699.99 (~ 480)
The EEE didn't actually replace the Toshiba, but I mention it for completeness. It was actually the only machine I moved to the US with, but after about a month of it as my primary machine I realized it wasn't an option for day to day use - though it was fantastic as a machine to throw in an overnight bag, especially when coupled with a 3G dongle. I wasn't keen on significantly increasing the size of my laptop. There are a number of decent 13" Ultrabook options out there, and I looked at a few of them, but nothing grabbed me as being worth the increase in size. Also I wanted something better than the Acer - one of the major problems was finding something smaller than 13" that had 8G RAM, let alone more. There's a significant trend towards everything soldered in for the smaller/slimmer notebooks, which makes some sense but means that the base spec had better be right. Much to my surprise the Microsoft Surface Pro 2 looked like an option. It comes with an i5-4300U processor (at least since around Christmas 2013), and the 256/512G SSD models have 8G RAM. Screen resolution is an attractive true HD (1920x1080) and the 10" display means it's smaller than the Aspire. Unfortunately the keyboard lets it down. It's fine given a flat surface, but not great if you want to support the whole thing on your lap. Which is something I tend to do with my laptop, whether that's on the sofa, or in bed, or on a bus/train. Another option was the Sony Vaio Pro 11. This is a pretty sweet laptop (I managed to get to play with one at a Sony store in the US). Super slim and light. 8GB RAM. True HD screen. However I have bad memories of the build quality of the older Vaios and the fact that there was /no/ user replaceable parts put me off - it's a safe bet that a laptop battery is going to need replaced in a 3 year lifespan. What I managed to find, and purchase, was a Dell Latitude E7240. I admit that the Dell brand made me wary - while I've not had any issue with their desktops I didn't associated their laptops as being particularly high quality. Mind you, I could say the same for Acer and I've been very pleased with the Aspire (if they'd had a more up to date model I'd have bought it). I bought the E7240 with the Core-i5 4300U (so the same as the Surface Pro 2) and True HD touch screen. It has a replaceable battery, expandable RAM (up to 16G) and the storage is an mSATA SSD. It also came with a built in 3G card. At 12.5" it's a little bigger than my old machine, but I decided that was a reasonable idea given the higher resolution. I'm typing this article on it now, having finally completed the setup and migration of the data from the old laptop to it this evening. More details once I've been using it for a little bit I think.

Erich Schubert: Debian chooses systemd as default init

It has been all over the place. The Debian CTTE has chosen systemd over upstart as default init system by chairman call. This decision was overdue, as it was pretty clear that the tie will not change, and thus it will be up to chairman. There were no new positions presented, and nobody was being convinced of a different preference. The whole discussion had been escalating, and had started to harm Debian. Some people may not want to hear this, but another 10 ballots and options would not have changed this outcome. Repating essentially the same decision (systemd, upstart, or "other things nobody prefers") will do no good, but turn out the same result, a tie. Every vote counting I saw happen would turn out this tie. Half of the CTTE members prefer systemd, the other half upstart. The main problems, however, are these: So please, everybody get back to work now. If there ever is enough reason to overturn this decision, there are formal ways to do this, and there are people in the CTTE (half of the CTTE, actually) that will take care of this. Until then, live with the result of a 0.1 votes lead for systemd. Instead of pursuing a destructive hate campaign, why not improve your favorite init system instead. Oh, and don't forget about the need to spell out a policy for init system support requirements of packages.

Sylvestre Ledru: Hello Planet Mozilla

Hello Planet Mozilla I started as a new employee a few weeks ago. I am working as Release Manager for the next releases of Firefox (both on desktop + android).
As a start, Lukas and I will be working on Firefox 29 (no pressure at all to start with Australis + Firefox account).
I grew up with the first versions of Netscape, followed the Phoenix/Firebird adventures and I have been a supporter of Firefox since then.
I am glad to join such an important project, especially in the perspective of the recent privacy scandals. I think that Mozilla is in the best position to communicate to the public, to denounce certain activities and provides technologies to bypass them. Besides that, let me introduce myself. I have been in free software for almost a decade.
My previous work as working on Scilab (from Inria to Scilab Enterprises) where I stayed for almost 8 years. Over the years, I worked on various development (C, C++, Java, Python, Scilab, etc), user experience, roadmap definition, release management, R and D / customer project management, management, board member, etc. During this period, I learnt tons of things from coding, release and product management, QA, etc. In parallel, I worked part time at IRILL, a free software research center (hosted by Inria). I was involved as community management ie organizing event like GNU Hacker meeting, FOSDEM, Euro LLVM, etc and recording them. I had the chance to meet plenty of people from the FLOSS world (including folks from Mozilla who suggested me to join the adventure). I am also involved in two communities: Debian and LLVM. The first one is Debian. I started to contribute in Debian to get Scilab properly packaged. 7 years
after, I am now maintaining
important packages
like LLVM, Clang, libedit, emscripten (!) or BLAS/LAPACK, treasurer of the French representative association and organizing the French mini Debian Conference (Debconf) / Meetup. For the last couple of years, I contributed to LLVM, Clang and LLDB. I participated to the release 3.3 and 3.4. My technical contributions were mainly driven by the packaging, the nightly Debian/Ubuntu packages and also by the rebuild of the Debian archive using Clang (instead of gcc). And when I am not behind my laptop, I am teaching scuba diving with my girlfriend or hiking, mountaineering, ski touring, climbing, etc. Yeh, I am a bit hyperactive.

Andrew Pollock: [life] Day 14, Kindergarten and a make up swim class

Zoe woke up pretty early today, at around 5:30am. I also officially became a Xoogler at some point overnight. I used this transition as the opportunity to move from my Nexus 4 to my Nexus 5, so I could have a clean start on a phone that had never had work credentials on it. I started that process before breakfast, and mostly had a usable phone by the time we biked to Kindergarten. Zoe was a bit more like her usual clingy self at drop off today, so I spent a bit longer getting her settled. I got home and pretty much spent the rest of the day sorting out my phone. I can say many good things about Android, but seamless device migration isn't one of them. I'll probably factory reset my Nexus 4 tomorrow. I biked back to Kindergarten to pick up Zoe, and sure enough, she was fast asleep again. It turned out that she'd gone down about 30 minutes later today, so she was deeply asleep, and didn't take too kindly to being woken up. She was very sad, and it took forever just to leave the Kindergarten. We were well and truly the last ones out. Then she saw Megan having a trial tennis lesson at the tennis courts on the way to the bike, and wanted to do that too, and we burned some more time, while I tried to talk her out of that. Zoe had a make up swim class today to cover the one she'd missed on Saturday when she went down to Canberra with Sarah. That was at 3:15pm. We were a tad late for it. Weekday swim classes are much quieter than the weekend ones, so I used the opportunity to talk to one of the swim school staff about progressing Zoe to the next level. They conducted an assessment on the spot, and graduated her! She's moving from "Platypus" to "Octopus", starting next week. The class time options were pretty crappy though, something like 11:45am on Saturday, or a late afternoon weekday. We've opted for a weekday class after Kindergarten for the rest of this quarter and will try and get an earlier Saturday class next quarter. There's a reasonable gap between the end of Kindergarten and the start of swim class, so I'll have to figure out how best to use that time. I might make our library visit in between, even though it means a bit of back and forth. The upside is Saturday mornings are now wide open. We had a celebratory ice cream when we got home from swim class, and went out for a babyccino after dinner. I think it's important to celebrate achievements. I managed to get Zoe to bed slightly early tonight in light of her reduced nap time, but she just woke up briefly at 10:30pm. She seems to be particularly sensitive to mosquitoes, and has a bite on her forehead, right at the hairline, and it was giving her some grief. I gave her some antihistamine and she's gone back to bed fairly easily. Fingers crossed she sleeps okay for the rest of the night.

Dirk Eddelbuettel: RcppSMC 0.1.2

Late last week, and just before leaving to participate in this crazy thing, I managed to get a new version of RcppSMC onto CRAN. RcppSMC combines the SMCTC template classes for Sequential Monte Carlo and Particle Filters (Johansen, 2009, JSS) with the Rcpp package for R/C++ Integration (Eddelbuettel and Francois, 2011, JSS) and thereby allows for easier and more direct access from R to the computational core of the Sequential Monte Carlo algorithm. This release regroups a few minor changes we accumulated over the last few months, but was triggered by the Rcpp 0.11.0 release last week. The NEWS entry is below:
Changes in RcppSMC version 0.1.2 (2014-02-06)
  • Updated for Rcpp 0.11.0 with explicit importFrom in NAMESPACE and corresponding versioned Imports: in DESCRIPTION; also removed linking instruction from src/Makevars as no it is longer needed with this new Rcpp version
  • Addded GitHub / Travis CI support
  • Use more portable dev.new() rather than x11() in pfLinearBS.R
  • Applied some corrections to pfNonlinBS.R example
  • Converted NEWS to NEWS.Rd
Courtesy of CRANberries, there is also a diffstat report relative to the previous release. As always, more detailed information is on the RcppSMC page,

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

10 February 2014

Bits from Debian: skyrocketing how-can-i-help popcon count

This is a repost from Stefano Zacchiroli's post how-can-i-help by Lucas Nussbaum is one of the best things that happened in the area of attracting contributions to Debian in quite a while. It can be used both as a standalone tool to list opportunities for contributing to Debian which are related to your installed packages, and as an APT hook (which is also the default configuration) that at each upgrade will inform you of new contribution opportunities. how-can-i-help is great for newbies who are looking for ways to give back to Debian which are a good match for their skills: among other things, how-can-i-help shows bugs tagged "gift" related to packages you use. how-can-i-help is also great for experienced developers, as it allows them to find out, in a timely manner, that packages they use are in dire need of help: RC bugs, pending removals, adoptions needed, requests for sponsor, etc. (As highly unscientific evidence: I've noticed a rather quick turnover of RFA/O/ITA bugs on packages installed on my machine. I suspect how-can-i-help is somehow responsible for that, due to the fact that it increases awareness of ongoing package issues directly with the people using them.) So, if you haven't yet, please apt-get install how-can-i-help RIGHT NOW. I daresay that we should aim at installing how-can-i-help by default on all Debian machines, but that might be an ambitious initial goal. In the meantime I'll settle for making how-can-i-help's popcon count skyrocket. As of today, it looks like this: Alt how-can-i-help popularity contest graph 10/02/2014 which is definitely too low for my taste. Please spread the word about how-can-i-help. And let's see what we can collectively do to that graph. how-can-i-help is just a tiny teeny helper, but I'm convinced it can do wonders in liberating dormant contributions to the Debian Project.

Matthew Garrett: IBM's remote firmware configuration protocol

I spent last week looking into the firmware configuration protocol used on current IBM system X servers. IBM provide a tool called ASU for configuring firmware settings, either in-band (ie, running on the machine you want to reconfigure) or out of band (ie, running on a remote computer and communicating with the baseboard management controller - IMM in IBM-speak). I'm not a fan of using vendor binaries for this kind of thing. They tend to be large (ASU is a 20MB executable) and difficult to script, so I'm gradually reimplementing them in Python. Doing that was fairly straightforward for Dell and Cisco, both of whom document their configuration protocol. IBM, on the other hand, don't. My first plan was to just look at the wire protocol, but it turns out that it's using IPMI 2.0 to communicate and that means that the traffic is encrypted. So, obviously, time to spend a while in gdb.

The most important lesson I learned last time I did this was "Check whether the vendor tool has a debug option". ASU didn't mention one in its help text and there was no sign of a getopt string, so this took a little longer - but nm helpfully showed a function called DebugLog and setting a breakpoint on that in gdb showed it being called a bunch of the time, so woo. Stepping through the function revealed that it was checking the value of a variable, and looking for other references to that variable in objdump revealed a -l argument. Setting that to 255 gave me rather a lot of output. Nearby was a reference to --showsptraffic, and passing that as well gave me output like:

BMC Sent: 2e 90 4d 4f 00 06 63 6f 6e 66 69 67 2e 65 66 69
BMC Recv: 00 4d 4f 00 21 9e 00 00

which was extremely promising. 2e is IPMI-speak for an OEM specific command, which seemed pretty plausible. 4d 4f 00 is 20301 in decimal, which is the IANA enterprise number for IBM's system X division (this is required by the IPMI spec, it wasn't an inspired piece of deduction on my behalf). 63 6f 6e 66 69 67 2e 65 66 69 is ASCII for config.xml. That left 90 and 06 to figure out. 90 appeared in all of the traces, so it appears to be the command to indicate that the remaining data is destined for the IMM. The prior debug output indicated that we were in the QuerySize function, so 06 presumably means query the size. And this is supported by the response - 00 (a status code), 4d 4f 00 (the IANA enterprise number again, to indicate that the responding device is speaking the same protocol as you) and 21 9e 00 00 - or, in rational endianness, 40481, an entirely plausible size.

Once I'd got that far the rest started falling into place fairly quickly. 01 rather than 06 indicated a file open request, returning a four byte file handle of some description. 02 was read, 03 was write and 05 was close. Hacking this together with pyghmi meant I could open a file and read it. Success!

Well, kind of. I was getting back a large blob of binary. The debug trace showed calls to an EfiDecompress function, so on a whim I tried just decompressing it using the standard UEFI compression format. Shockingly, it worked and now I had a 345K XML blob and presumably many more problems than I'd previously had. Parsing the XML was fairly straightforward, and now I could retrieve the full set of config options, along with the default, current and possible values for them.

Making changes was pretty much just the reverse of this. A small XML blob containing the new values is compressed and written to asu_update.efi. One of the elements is a random identifier, which will then appear in another file called config_log along with a status. Just keep re-reading this until the value changes to CM_DONE and we're good.

The in-band configuration appears to be identical, but rather than sending commands over the wire they're send through the system's IPMI controller directly. Yes, this does mean that it's sending compressed XML through a single io port. Yes, I'd rather be drinking.

I've documented the protocol here and hope to be able to release this code before too long - right now I'm trying to nail down the interface a little, but it's broadly working.

comment count unavailable comments

Mario Lang: Neurofunkcasts

I have always loved Drum and Bass. In 2013 I rediscovered my love for Darkstep and Neurofunk, and found that these genres have developed quite a lot in the recent years. Some labels like Black Sun Empire and Evol Intent produce mixes/sets on a regular basis as podcasts these days. This article aggregates some neurofunk podcasts I like a lot, most recent first. Enjoy 33 hours and 57 minutes of fun with dark and energizing beats. Thanks to BSE Contrax and Evol Intent for providing such high quality sets. You can also see the Python source for the program that was used to generate this page.

Stefano Zacchiroli: apt-get install how-can-i-help

skyrocketing how-can-i-help popcon count how-can-i-help by Lucas Nussbaum is one of the best things that happened in the area of attracting contributions to Debian in quite a while. It can be used both as a standalone tool to list opportunities for contributing to Debian which are related to your installed packages, and as an APT hook (which is also the default configuration) that at each upgrade will inform you of new contribution opportunities. how-can-i-help is great for newbies who are looking for ways to give back to Debian which are a good match for their skills: among other things, how-can-i-help shows bugs tagged "gift" related to packages you use. how-can-i-help is also great for experienced developers, as it allows them to find out, in a timely manner, that packages they use are in dire need of help: RC bugs, pending removals, adoptions needed, requests for sponsor, etc. (As highly unscientific evidence: I've noticed a rather quick turnover of RFA/O/ITA bugs on packages installed on my machine. I suspect how-can-i-help is somehow responsible for that, due to the fact that it increases awareness of ongoing package issues directly with the people using them.) So, if you haven't yet, please apt-get install how-can-i-help RIGHT NOW. I daresay that we should aim at installing how-can-i-help by default on all Debian machines, but that might be an ambitious initial goal. In the meantime I'll settle for making how-can-i-help's popcon count skyrocket. As of today, it looks like this:
how-can-i-help popularity contest graph, 10/02/2014
which is definitely too low for my taste. Please spread the word about how-can-i-help. And let's see what we can collectively do to that graph. how-can-i-help is just a tiny teeny helper, but I'm convinced it can do wonders in liberating dormant contributions to the Debian Project.

Russell Coker: Fingerprints and Authentication

Dustin Kirkland wrote an interesting post about fingerprint authentication [1]. He suggests using fingerprints for identifying users (NOT authentication) and gives an example of a married couple sharing a tablet and using fingerprints to determine who s apps are loaded. In response Tollef Fog Heen suggests using fingerprints for lightweight authentication, such as resuming a session after a toilet break [2]. I think that one of the best comments on the issue of authentication for different tasks is in XKCD comic 1200 [3]. It seems obvious that the division between administrator (who installs new device drivers etc) and user (who does everything from playing games to online banking with the same privileges) isn t working, and never could work well particularly when the user in question installs their own software. I think that one thing which is worth considering is the uses of a signature. A signature can be easily forged in many ways and they often aren t checked well. It seems that there are two broad cases of using a signature, one is to enter into legally binding serious contract such as a mortgage (where wanting to sign is the relevant issue) and the other is cases where the issue doesn t matter so much (EG signing off on a credit card purchase where the parties at risk can afford to lose money on occasion for efficient transactions). Signing is relatively easy but that s because it either doesn t matter much or because it s just a legal issue which isn t connected to authentication. The possibility of serious damage (sending life savings or incriminating pictures to criminals in another jurisdiction) being done instantly never applied to signatures. It seems to me that in many ways signatures are comparable to fingerprints and both of them aren t particularly good for authentication to a computer. In regard to Tollef s ideas about lightweight authentication I think that the first thing that would be required is direct user control over the authentication required to unlock a system. I have read about some Microsoft research into a computer monitoring the office environment to better facilitate the user s requests, an obvious extension to such research would be to have greater unlock requirements if there are more unknown people in the area or if the device is in a known unsafe location. But apart from that sort of future development it seems that having the user request a greater or lesser authentication check either at the time they lock their session or by policy would make sense. Generally users have a reasonable idea about the risk of another user trying to login with their terminal so user should be able to decide that a toilet break when at home only requires a fingerprint (enough to keep out other family members) while a toilet break at the office requires greater authentication. Mobile devices could use GPS location to determine unlock requirements, GPS can be forged, but if your attacker is willing and able to do that then you have a greater risk than most users. Some users turn off authentication on their phone because it s too inconvenient. If they had the option of using a fingerprint most of the time and a password for the times when a fingerprint can t be read then it would give an overall increase in security. Finally it should be possible to unlock only certain applications. Recent versions of Android support widgets on the lock screen so you can perform basic tasks such as checking the weather forecast without unlocking your phone. But it should be possible to have different authentication requirements for various applications. Using a fingerprint scan to allow playing games or reading email in the mailing list folder would be more than adequate security. But reading the important email and using SMS probably needs greater authentication. This takes us back to the XKCD cartoon.

Andrew Pollock: [life] Day 13, early start, Kindergarten and messing around with a hot glue gun

I had an early start today. I set my alarm for 5:30am, and finally got going a bit after 6am, and knocked out a 10km run. It was my first 10km in about a month. It wasn't pretty, but I was looking for accomplishment, not achievement. Sarah's doing a work training course that had an early start, so she asked me if she could drop Zoe to me early and have me take care of Kindergarten drop off for her. Zoe arrived at around 8am, and I put some sunscreen on her and we jumped on the bike. I had one of the smoothest drop offs I've had since we've been back in Australia. Megan was already there, and after we got settled, the two girls started building with some large wooden blocks, and Zoe pretty much gave me a hug and a kiss and sent me on my way. She didn't require any handing off to a teacher, wasn't clingy or anything. It was wonderful to see. Today was supposed to be my Debian day, but I've got something like 3 tax returns to do for two different countries, so I got stuck into collating records instead. After lunch I allowed myself to get sidetracked with trying to figure out the Google Drive API, because I'm trying to directly scan documents to Google Drive. Then it was time to do the Kindergarten pick up. Both girls were fast asleep when I got there. I tried rousing Zoe without much success, so I went to try and wake up Megan. She was also pretty heavily sleeping, so I left her with one of the teachers to try and wake her up and returned to Zoe. She took a lot of work to wake up, but woke up happy. I was expecting a couple of packages today, and the first one was being delivered by Australia Post as we got home, so that was pretty good timing. The second one was delivered while we were home, so that worked out pretty well. For this afternoon's activity with the girls, I thought we'd make peg dolls. I've been stockpiling craft materials for almost as long as I've been back in Australia. I bought the big IKEA EXPEDIT 5 x 5 shelf, and have the bottom two rows set up as Zoe-accessible craft materials, reminiscent of the "bin wall" that the TechShop had. I've been getting stuff from Riot!, but I've also found that there's lots of good stuff to be had for cheap from Overflow. So today's activity I think came from Riot! It was your bog standard wooden clothes pin, but the kit included a conical wooden "sleeve" that I guess was supposed to be a "dress". I thought I'd take things to the next level by hot-gluing a styrofoam ball on the top of the peg, and then hot-gluing eyes and a pom-pom nose on. The girls decided on their own to glue orange cellophane for hair, and got me to draw on eyebrows and a mouth. I had a couple of sheets of felt, so I cut holes in the middle of them and used the conical wooden sleeve to hold the felt dress in place. Pipe cleaners made the arms and they were finished. The girls seemed to enjoy making and playing with them. It was the first time I've used the hot glue gun, and I wasn't sure how it was going to go. I explained to the girls that the tip was hot and to stay away from it, and let them use it under close supervision. Unfortunately Zoe learned the hard way that the glue is also very hot after it's been applied to the surface that it's going to be used on. Hot glue was definitely the way to go for a quick project where we didn't want to be sitting around waiting for PVA glue to dry. After that, the girls watched a DVD for a little while and then Megan's Dad arrived to pick her up. I started dinner a bit late, I should have started it while the girls were watching TV, so we didn't sit down to eat until later than usual. Bedtime went pretty smoothly, but also ran a bit late. Here's hoping Zoe sleeps well tonight.

9 February 2014

Jan Dittberner: Going to Chemnitzer Linux-Tage 2014

This year I take care of organizing of the Debian booth at Chemnitzer Linux-Tage 2014 which has been approved a few days ago. The CLT is a yearly (mostly) german speaking Free Software community event which takes place on the weekend of 15th/16th march in Chemnitz (Germany). On the Linux-Live pages you find a lot of projects that will have a booth there and the talk schedule contains many interesting topics. There will also be a key signing event for which you can register until 11th of march. The Wiki page for the Event is already in a good shape. Many things are already organized, but we still have some items left. A lot of people from the Debian community have already told me that they will be there. We will have a Debian Wheezy BabelBox demonstration running on a VirtualBox host that Jan Wagner will provide as well as merchandising (Thanks to Alexander Wirt). Two talks from people on our Wiki page have been accepted by the CLT organizers too: I am happy to meet many nice people from the Free Software community in Chemnitz soon.

Erich Schubert: Minimizing usage of debian-multimedia packages

How to reduce your usage of debian-multimedia packages: As you might be aware, the "deb-multimedia" packages have seen their share of chaos. Originally, they were named "debian-multimedia", but it was then decided that they should better be called "deb-multimedia" as they are not an official part of Debian. The old domain was then grabbed by cybersquatters when it expired. While a number of packages remain indistributable for Debian due to legal issues - such as decrypting DVDs - and thus "DMO" remains useful to many desktop users, please note that for many of the packages, a reasonable version exists within Debian "main". So here is a way to prevent automatically upgrading to DMO versions of packages that also exist in Debian main: We will use a configuration option known as apt pinning. We will modify the priority of DMO package to below 100, which means they will not be automatically upgraded to; but they can be installed when e.g. no version of this package exists within Debian main. I.e. the packages can be easily installed, but it will prefer using the official versions. For this we need to create a file I named /etc/apt/preferences.d/unprefer-dmo with the contents:
Package: *
Pin: release o=Unofficial Multimedia Packages
Pin-Priority: 123
As long as the DMO archive doesn't rename itself, this pin will work; it will continue to work if you use a mirror of the archive, as it is not tied to the URL. It will not downgrade packages for you. If you want to downgrade, this will be a trickier process. You can use aptitude for this, and start with a filter (l key, as in "limit") of ?narrow(~i,~Vdmo). This will only show packages installed where the version number contains dmo. You can now patrol this list, enter the detail view of each package, and check the version list at the end for a non-dmo version. I cannot claim this will be an easy process. You'll probably have a lot of conflicts to clear up, due to different libavcodec* API versions. If I recall correctly, I opted to uninstall some dmo packages such as ffmpeg that I do not really need. In other cases, the naming is slightly different: Debian main has handbrake, while dmo has handbrake-gtk. A simple approach is to consider uninstalling all of these packages. Then reinstall as needed; since installing Debian packages is trivial, it does not hurt to deinstall something, does it? When reinstalling, Debian packages will be preferred over DMO packages. I prefer to use DFSG-free software wherever possible. And as long as I can watch my favorite series (Tatort, started in 1970 and airing episode 900 this month) and the occasional DVD movie, I have all I need. An even more powerful aptitude filter to review installed non-Debian software is ?narrow(~i,!~ODebian), or equivalently ?narrow(?installed,?not(?origin(Debian))). This will list all package versions, which cannot be installed from Debian sources. In particular, this includes versions that are no longer available (they may have been removed because of security issues!), software that has been installed manually from .deb files, or any other non-Debian source such as Google or DMO. (You should check the output of aptitude policy that no source claims to be o=Debian that isn't Debian though). This filter is a good health check for your system. Debian packages receive a good amount of attention with respect to packaging quality, maintainability, security and all of these aspects. Third party packages usually failed the Debian quality check in at least one respect. For example, Sage isn't in Debian yet; mostly because it has so many dependencies, that making a reliable and upgradeable installation is a pain. Similarly, there is no Hadoop in Debian yet. If you look at Hadoop packaging efforts such as Apache Bigtop, they do not live up to Debian quality yet. In particular, the packages have high redundancy, and re-include various .jar copies instead of sharing access to an independently packaged dependency. If a security issue arises with any of these jars, all the Hadoop packages will need to be rebuilt. As you can see, it is usually not because of Debian being too strict or too conservative about free software when software is not packaged. More often than not, it's because the software in question itself currently does not live up to Debian packaging quality requirements. And of course sometimes because there is too little community backing the software in question, that would improve the packaging. If you want your software to be included in Debian, try to make it packaging friendly. Don't force-include copies of other software, for example. Allow the use of system-installed libraries where possible, and provide upgrade paths and backwards compatibility. Make dependencies optional, so that your software can be pacakged even if a dependency is not yet available - and does not have to be removed if the dependency becomes a security risk. Maybe learn how to package yourself, too. This will make it easier for you to push new features and bug fixes to your userbase: instead of requiring your users to manually upgrade, make your software packaging friendly. Then your users will be auto-upgraded by their operating system to the latest version. Update: If you are using APT::Default-Release or other pins, these may override above pin. You may need to use apt-cache policy to find out if the pins are in effect, and rewrite your default-release pin into e.g.
Package: *
Pin: release a=testing,o=Debian
Pin-Priority: 912
for debugging, use a different priority for each pin, so you can see which one is in effect. Notice the o=Debian in this pin, which makes it apply only to Debian repositories.

Neil Williams: Debian ARMMP in LAVA using iMX53

The home lab is my LAVA development environment and I ve recently got two iMX53 Quick Start Boards working with a typical LAVA master image based on a Linaro build of Linux 3.1.0 for mx5 with a Ubuntu Oneiric Ocelot 11.10 rootfs:
 3.1.0-1002-linaro-lt-mx5 (buildd@hubbard) (gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3) ) #13-Ubuntu PREEMPT Fri Dec 16 01:21:07 UTC 2011
As part of my Debian work, it was clearly time to look at a current, Debian, kernel and rootfs and as I m developing and testing on Debian unstable, this would necessarily mean testing the Debian ARMMP (multi-platform) kernel which replaces the mx5 build used in Wheezy.
Linux version 3.12-1-armmp (debian-kernel@lists.debian.org) (gcc version 4.8.2 (Debian 4.8.2-10) ) #1 SMP Debian 3.12.9-1 (2014-02-01)
I will be scripting the creation of a suitable image for these tests and there are other changes planned in LAVA to make it easier to build suitable images, but it is useful to document just how I worked out how to build the first image. Manual build steps First, I ve already discovered that the u-boot on the iMX53 doesn t like ext4, so the first step was to prepare an ext3 filesystem for the image. With a SATA drive attached, it was also much better to use that than the SD card, at least for generating the image. I m also doing this natively, so I am working inside the booted master image. This is fine as the master is designed to manipulate test images, so the only package I needed to install on the LAVA master image was debootstrap. I had an empty SATA drive to play with for these tests, so first prepare an ext3 filesystem:
# mkfs.ext3 /dev/sda1
# mkdir /mnt/sata
# mount /dev/sda1 /mnt/sata
# mkdir /mnt/sata/chroots/
Start the debootstrap:
# apt-get update
# apt-get install debootstrap
# debootstrap --arch=armhf --include=linux-image-armmp \
 --verbose unstable \
 /mnt/sata/chroots/unstable-armhf http://ftp.uk.debian.org/debian
Various actions in this chroot will need proc, so mount it here:
# chroot /mnt/sata/chroots/unstable-armhf
# mount proc -t proc /proc
# mount devpts -t devpts /dev/pts
# exit
You may also have to edit the apt sources the LAVA master image doesn t have editors installed, so either use echo or download an edited file. I m using:
deb http://ftp.uk.debian.org/debian sid main
flash-kernel needs changes For the initial tests, I ve got to get this image to boot directly from u-boot, so flash-kernel is going to be needed inside the chroot and to get the iMX53 to work with the ARMMP kernel and Device Tree, flash-kernel will need an update which will mean a patch:
# chroot /mnt/sata/chroots/unstable-armhf
# apt-get update
# apt-get install patch flash-kernel
# cp /usr/share/flash-kernel/db/all.db /home
# cd /home
# patch -p2
The patch itself goes through a couple of iterations. Initially, it is enough to use:
 Machine: Freescale MX53 LOCO Board
-Kernel-Flavors: mx5
+Kernel-Flavors: armmp
+DTB-Id: imx53-qsb.dtb
+DTB-Append: yes
Later, once the device has booted with the ARMMP kernel, the Machine Id can be updated to distinguish it from the mx5 flavour (from /proc/cpuinfo) and use the model name from the Device Tree (/proc/device-tree/model):
diff --git a/db/all.db b/db/all.db
index fab3407..41f6c78 100644
--- a/db/all.db
+++ b/db/all.db
@@ -62,6 +62,18 @@ U-Boot-Initrd-Address: 0x00800000
 Required-Packages: u-boot-tools
 Bootloader-Sets-Incorrect-Root: yes
 
+Machine: Freescale i.MX53 Quick Start Board
+Kernel-Flavors: armmp
+DTB-Id: imx53-qsb.dtb
+DTB-Append-From: 3.12
+Boot-DTB-Path: /boot/dtb
+U-Boot-Kernel-Address: 0x70008000
+U-Boot-Initrd-Address: 0x0
+Boot-Kernel-Path: /boot/uImage
+Boot-Initrd-Path: /boot/uInitrd
+Required-Packages: u-boot-tools
+Bootloader-Sets-Incorrect-Root: no
+
 Machine: Freescale MX53 LOCO Board
 Kernel-Flavors: mx5
 U-Boot-Kernel-Address: 0x70008000
(I will be filing this patch in a bug report against flash-kernel soon.) With that patched, update and run flash-kernel:
# mv all.db /usr/share/flash-kernel/db/all.db
# flash-kernel
flash-kernel: installing version 3.12-1-armmp
Generating kernel u-boot image... done.
Taking backup of uImage.
Installing new uImage.
Generating initramfs u-boot image... done.
Taking backup of uInitrd.
Installing new uInitrd.
Installing new dtb.
# exit
LAVA overlays This will be a LAVA test image and it needs an overlay if you want a vanilla image, set up a passwd inside the chroot instead.
# cd /mnt/sata/chroots/unstable-armhf/
# wget --no-check-certificate https://launchpad.net/~linaro-maintainers/+archive/overlay/+files/linaro-overlay-minimal_1112.2_all.deb
# wget --no-check-certificate https://launchpad.net/~linaro-maintainers/+archive/overlay/+files/linaro-overlay_1112.2_all.deb
# chroot /mnt/sata/chroots/unstable-armhf/
# dpkg -i linaro-overlay-minimal_1112.2_all.deb linaro-overlay_1112.2_all.deb
# rm linaro-overlay-minimal_1112.2_all.deb linaro-overlay_1112.2_all.deb
# exit
Changes to allow the chroot to boot Now the chroot needs setting up as a boot image:
# chroot /mnt/sata/chroots/unstable-armhf/
# echo T0:23:respawn:/sbin/getty -L ttymxc0 115200 vt102 >> ./etc/inittab
# echo auto lo eth0 > ./etc/network/interfaces.d/mx53loco
# echo iface lo inet loopback >> ./etc/network/interfaces.d/mx53loco
# echo iface eth0 inet dhcp >> ./etc/network/interfaces.d/mx53loco
# apt-get clean
# umount /dev/pts
# umount /proc
# exit
Partitioning as a LAVA test image This would be enough as a single partition test image but, currently, LAVA expects a much more hard-wired image. Depending on the history of the device and the need for LAVA to be able to put fresh kernel builds together with a known rootfs, LAVA has used a separate /boot and / partition in the test image for nearly all boards. Standard LAVA test images for many boards (like the iMX53) also have a small unallocated space at the start of the SD card, so until I can get LAVA upstream to handle test images of arbitrary design, I m adapting the image to suit the expectations inside LAVA. Yes, I know but it s better to get something working before spending time fixing things to make it work better. It will be fixed, in time. So I needed to separate out the /boot contents from the rest of the rootfs whilst keeping the chroot itself in a state which I can easily update and use to create other images.
# cd /mnt/sata/chroots/unstable-armhf/boot/
# tar -czf ../../boot.tar.gz ./*
# cd ..
# tar -czf ../root.tar.gz ./*
Now the test image file and its partitions:
# dd if=/dev/zero of=/mnt/sata/images/empty/debian.img bs=1M count=1024
# cp /mnt/sata/images/empty/debian.img /mnt/sata/images/debian-unstable-armhf-armmp.img
# losetup /dev/loop0 /mnt/sata/images/debian-unstable-armhf-armmp.img
# parted /dev/loop0 -s unit mb mktable msdos
# parted /dev/loop0 -s unit mb mkpart primary 1 10
# parted /dev/loop0 -s unit mb mkpart primary 11 110
# parted /dev/loop0 -s unit mb mkpart primary 111 1024
I did make the mistake of using kpartx at this stage but there are many areas of confusion when translating the kpartx output to offsets for mount when parted is easier:
# parted /dev/loop0 unit B -s print
Model:  (file)
Disk /dev/loop0: 1073741824B
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start       End          Size        Type     File system  Flags
 1      1048576B    10485759B    9437184B    primary
 2      10485760B   110100479B   99614720B   primary
 3      110100480B  1024458751B  914358272B  primary
Use the Start numbers and use losetup to create the loop devices for each
partition
# losetup -o 10485760 /dev/loop1 /dev/loop0
# losetup -o 110100480 /dev/loop2 /dev/loop0
# mkfs.vfat /dev/loop1
# mkfs.ext3 /dev/loop2
Now clean up the loop mountpoints:
# losetup -d /dev/loop2
# losetup -d /dev/loop1
# losetup -d /dev/loop0
# losetup -a
losetup -a should return nothing. If it doesn t, investigate the contents of /dev/mapper and use dmsetup remove until losetup -a does report empty. Otherwise the subsequent stages will fail. Now deploy the /boot contents into the empty image:
# mount -oloop,offset=10485760 /mnt/sata/images/debian-unstable-armhf-armmp.img /mnt/boot/
# pushd /mnt/boot/
# tar -xzf /mnt/sata/chroots/boot.tar.gz
# popd
# sync
# umount /mnt/boot/
and the / contents (removing the duplicate ./boot contents)
# mount -oloop,offset=110100480 /mnt/sata/images/debian-unstable-armhf-armmp.img /mnt/root/
# pushd /mnt/root
# tar -xzf /mnt/sata/chroots/root.tar.gz
# rm ./boot/*
# popd
# sync
# umount /mnt/root
Check the image
# mount -oloop,offset=10485760 debian-unstable-armhf-armmp.img /mnt/boot
# ls /mnt/boot
config-3.12-1-armmp  initrd.img-3.12-1-armmp  System.map-3.12-1-armmp  uImage  uInitrd  vmlinuz-3.12-1-armmp
# umount /mnt/boot
# mount -oloop,offset=110100480 debian-unstable-armhf-armmp.img /mnt/root
# ls /mnt/root
bin  boot  dev  etc  home  initrd.img  lib  lost+found  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var  vmlinuz
# ls /mnt/root/bin/auto-serial-console
/mnt/root/bin/auto-serial-console
# umount /mnt/root
# md5sum debian-unstable-armhf-armmp.img
Downloads Yes, I have put that file online, if you are interested. Do read the readme first though. Getting the image off the device
# ip a
# python -m SimpleHTTPServer 80 2>/dev/null
then wget (just http://, IP address / and the filename), md5sum , finish with Ctrl-C. Depending on setup, it may be quicker to transfer the uncompressed
file over a LAN than to let the device compress it. Your main machine
will do the compression much faster, even with a larger download. (It also helps to not compress the image on the device, you can
test the mount offsets more easily and do check that the image
can be mounted with the offsets from parted.) Results
# cat /proc/cpuinfo
processor	: 0
model name	: ARMv7 Processor rev 5 (v7l)
Features	: swp half thumb fastmult vfp edsp thumbee neon vfpv3 tls vfpd32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x2
CPU part	: 0xc08
CPU revision	: 5
Hardware	: Freescale i.MX53 (Device Tree Support)
Revision	: 0000
Serial		: 0000000000000000
# uname -a
Linux imx53-02 3.12-1-armmp #1 SMP Debian 3.12.9-1 (2014-02-01) armv7l GNU/Linux
# lsb_release -a
No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux unstable (sid)
Release:	unstable
Codename:	sid
Next! Yes, there is a lot to do from here on. Then there is the whole issue of actually making this multi-platform. After all, it is far from ideal that a multi-platform kernel package has to have platform-specific operations using flash-kernel at each update. So Grub on ARM is going to be on the list of things to investigate .

8 February 2014

Paul Tagliamonte: Upcoming Talks

Hello, World! I ve got two upcoming talks - one at LibrePlanet 2014 about OpenPGP, and one at PyCon US 2014 (in Canada) on Hy. Slides will be posted as I give the talks. Hope to see folks there!

Sam Hartman: IETF Takes on Challenge of Internet Monitoring

At IETF 88, we held a plenary discussion of how we could harden the Internet against ongoing monitoring and survalence. There were no significant surprises in what people said about monitoring. So, we had an opportunity to focus on what the IETF as a standards organization responsible for technical standards that drive the Internet can do about this problem. I think we made amazing progress. The IETF works by consensus. We discuss issues, and see if there s some position that can gain a rough consensus of the participants in the discussion. After a couple of hours of discussion, Russ Housley asked several consensus questions. The sense of the room is that we should treat these incidents of monitoring as an attack and include them in the threats we try and counter with security features in our protocols. The room believes that encryption, even without authentication has value in fighting these attacks. There was support for the idea of end-to-end encryption is valuable even when there are middle boxes. IETF decisions made in meetings are confirmed on public mailing lists, so the sense of the room is not final. Also, note that I did not capture the exact wording of the questions that were posed. This is huge. There is very strong organizational agreement that we re going to take work in this space as seriously. Now that we ve decided pervasive monitoring is an attack, anyone can ask how a proposed protocol (or change to a protocol) counters that attack. If it doesn t handle the attack and there is a way to address the attack, then we will be in a stronger position arguing the threat could be addressed. In addition, the commitment to encryption providing value without authentication will be useful in providing privacy and minimizing fingerprinting by passive attackers. The IETF is only one part of the solution. We can work on the standards that describe how systems interact. However, implementations, policy makers, operators and users will also play a role in securing the Internet against pervasive attacks.

Enrico Zini: original-sin

Original sin I feel that I was somewhat born innocent, an animal with sound, primal instincts. But if I remained that way, I wouldn't be able to function in a complex society, so I got education. Education taught me what is expected from me in order to be accepted by my peers[1]. Education taught me more than that: it also taught me to enjoy what a complex society can give me: art, science, history, philosophy, adding depth of meaning and correlations to my perceptions and memories. Education wasn't perfect, though. Some of my educators obsessed about some of the expectations, and gave me rules to follow that aren't really needed to interact with a society. Sit down for hours in silence without complaining. Don't talk back to figures of authority, even when they are abusing me. Don't ever feel that my efforts in a task have been enough, because there is always something more that I could have done. Do what people expect me to do, regardless of what I wish to do. Do what I need to do, not what I want to do. So I grew up with a set of arbitrary expectations that weren't needed to function in a society, and weren't in any way meeting any of my needs, yet I still felt them as a part of me, putting every effort into meeting them that I would put in making myself happy. I like to call this set of learned arbitrary, unneeded expectations "neurosis". I like this as an interpretation of the "original sin" myth: in order to go from an innocent animal to a member of a complex society, I acquired a set of neuroses that make me behave in a meaningless way. Or, rephrased going along with the myth, that rob me of my innocence. In some environments like BDSM, many have a name for an unnegotiated practice forced on a person: they call it "abuse". It made sense to accept that I have been abused many times while being educated, because my educators also had their own baggage of parasite expectations, of neuroses. And since I don't currently know how I can ever be sure that I freed myself from all my neuroses, I feel like I should accept that it is possible that I do and will abuse others; by accepting it as a possibility, I hope at least to be able to realise as soon as possible that I am doing it, and try to stop. I have now realised that my life is far simpler and more rewarding when expectations are negotiated between all the parties involved. I have recently spent a substantial amount of my energy in recognising and renegotiating many of the expectations and the neuroses that I had learnt while growing up, and, who knows, perhaps this effort will continue throughout my life. So there you go, an agnostic atheist freeing himself of his original sin, and regaining his paradise lost one step at a time, alive, on earth. And enjoying every moment of it. [1] I suspect that is why people educated in a high class society tend to be accepted more easily by a high class society: they conform to the right set of expectations. But I digress.

Mario Lang: Uploading Into and Downloading From The Universe

Have you ever wondered how new ideas, concepts, artworks etc. get into the Superconscious Collective of the universe, so that they then can be downloaded by exactly the right person who is meant to do something with them? yep, I, too, think that many of these ideas, concepts etc. come from the Great Unknown, where the seeds of limitless ideas and possibilities reside. But a lot of them also come from our own minds, from people who upload these things into the Superconscious Collective of our universe. Such ideas and concepts can be uploaded in several ways. Some of them are simply uploaded by a person who sources them, connects to the Superconscious Collective of the universe and streams them up. Others are maybe even uploaded by somebody virtually typing an article with their fingertips into the spread out hands or soles of the feet of their loved one, playfully teasing him/her and thereby streaming information into him/her, which s/he then again uploads, mainly subconsciously, into the Superconscious Collective of the universe. Once uploaded, all these limitless ideas and concepts become available to everyone and every living being, on Earth and throughout the whole universe. The thing is, that most people on Earth don't know how to consciously access information from there. Some do unconsciously and it again shows up in them as entirely new concepts and ideas, which they then spice with the flavor of their own essence and then stream them out into the world, be it as music, as software, as hardware, as artwork, as science and in limitless other ways. But it is also accessed by living beings throughout the whole universe, who also add their own ideas and concepts to the Great Superconscious Collective . Wanna know how to access this limitless information, ideas, concepts and/or possibilities from the Superconscious Collective? Just breathe, relax, expand ... expand and get into your biggest, greatest, awesomest being of you. Become all that you are, connected to everything. Now, consciously plug into the Great Superconscious Collective, the Great Cosmic Internet. Allow the ideas, information, possibilities stream into you and through you and look which ones you resonate with. Call them, feel them, let them land in you, and ask them what they want you to do with them, as you are the one who can express them in the highest possible way, otherwise they wouldn't have come to you in the first place. Then do one small thing to make them real in the world and watch them unfold. Or do whatever feels right for you to do, and create the most amazing new thing in the world.

Vasudev Kamath: Friendica instance on my VPS is down

I started running Friendica instance on my VPS. With help of Jonas Smedegaard I managed to run Friendica in a uWSGI container. The site was running at samsargika.copyninja.info and is no longer accessible. Since VPS itself was running Debian Wheezy I couldn't run uWSGI with PHP support on it. (support for PHP in uWSGI landed after Wheezy). But Jonas was kind enough for me to provide a backported version. Recently Wheezy got a security update for PHP and thats where all the problem started. The backported uwsgi-plugin-php was not recompiled to use security updated PHP and I couldn't upgrade the things. After few days I noticed first freeze on my VPS and I had to reboot the VPS to get it online again. The fact I noticed was uWSGI being killed due to a OOM in syslog but I didn't explore much and consulted Jonas to get a updated uWSGI. But that didn't happen as Jonas himself is facing some problem with uWSGI builds. While again checking with aptitude for upgrade I accidentally confirmed removal of uwsgi-plugin-php for getting security updates :-/. But nothing happened to my running service as upgrade of libs in Debian doesn't cause the restart of all services which are using that lib (desired effect is restart of service but I don't know the side affects involved). Second freeze happened yesterday and on the reboot uwsgi-plugin-php was missing there by taking my Friendica instance down. More closer investigation showed the same OOM but this time I noticed each OOM occurred just after cronjob was running poller.php a script which is actually causing all federation in Friendica. So it was clear there is something wrong either in poller.php or in my setup which was making it eat memory and freeze my VPS. But I also found some stupidity I did during configuration which Jonas also pointed me out.
  1. Installing cronjob inside crontab rather than cron.d
  2. Installing poller.php crontab for root user :-/
I basically violated the basic rule by running a unsafe script as root user, good that script didn't do some crazy stuff. So even though my instance went down I learnt my lessons
  1. Don't ever ever ever run a unsafe script as root that too through cron
  2. Sandbox the unsafe script so it can be killed on time rather than it taking the whole system down.
  3. PHP is not really secure, if it was secure there wouldn't be security updates and atleast my site would be still running :-D
So I now need to wait till Jonas get me new shiny backported uWSGI linked against new PHP on Wheezy and till that time I need to explore on how I can sandbox the poller.php script.

Mario Lang: boost::python and boost::variant

I have unsuccessfully tried to find a solution for the following problem on the internet several times. Now that I have come at least closer to a usable approach, I thought I'd document what I have found so that others trying to achieve a similar thing can use this as a starting point. Boost.Python offers a very nice and flexible way to interface C++ data types with Python. With just a few lines of code, and the proper linker flags, you get a Python importable shared object from your C++ compiler. This can be very productive. However, there is one aspect of C++ data types that I couldn't figure out how to interface with Python, which are C++ discriminated unions, or more specifically, heterogeneous containers. While Python has no problems with containers containing objects of different types, C++ does not make this very easy by default. Usually the problem is solved with a container of pointers to a base class, and various subclasses with virtual functions. However, this approach is not always practical, especially if the different types of objects in an heterogeneous container dont have many things in common. This is where discriminated unions come to the rescue. They basically behave like a normal union in C, but have an additional field which indicates the type of object currently stored in the union. Boost.Variant does exactly that, with a nice visitor interface added on top of it.
Heterogeneous containers in C++ If we put the boost::variant<> template inside a STL container like std::vector<>, the result is a heterogeneous container. For the purpose of illustration, lets implement such a container. The example below is deliberately simple. In reality, the various types allowed in your variant will probably have more fields then just one.
#include <boost/variant.hpp>
#include <vector>
struct a   int x;  ;
struct b   std::string y;  ;
typedef boost::variant<a, b> variant;
typedef std::vector<variant> vector;
To ease creation of these two types of objects, we are going to write a few factory functions. We are going to wrap them in Python later on.
variant make_variant()   return variant();  
vector make_vector()   return vector a(), b(), a() ;  
Boost.Python Now lets create a Python module which exports the above functionality to Python.
#include <boost/python/class.hpp>
#include <boost/python/def.hpp>
#include <boost/python/implicit.hpp>
#include <boost/python/init.hpp>
#include <boost/python/module.hpp>
#include <boost/python/object.hpp>
#include <boost/python/suite/indexing/vector_indexing_suite.hpp>
vector_indexing_suite apparently needs operator== defined on the value_type of the container. In our case, this is our boost::variant<a, b> type. Luckily, boost::variant<> already provides operator==. However, that operator== relies on operator== being defined for the underlying types. Since equality comparison is probably useful for other things as well, lets just create operator== for our two classes a and b.
bool operator==(a const &lhs, a const &rhs)   return lhs.x == rhs.x;  
bool operator==(b const &lhs, b const &rhs)   return lhs.y == rhs.y;  
Convert a boost::variant<> to PyObject * Boost.Python needs a way to convert our discriminated union to a Python object. This code relies on Python class definitions being present for all underlying variant types. We will define them later.
struct variant_to_object : boost::static_visitor<PyObject *>  
  static result_type convert(variant const &v)  
    return apply_visitor(variant_to_object(), v);
   
  template<typename T>
  result_type operator()(T const &t) const  
    return boost::python::incref(boost::python::object(t).ptr());
   
 ;
And finally, lets create our Python module.
BOOST_PYTHON_MODULE(bpv)  
  using namespace boost::python;
  class_<a>("a", init<a>()).def(init<>()).def_readwrite("x", &a::x);
  class_<b>("b", init<b>()).def(init<>()).def_readwrite("y", &b::y);
  to_python_converter<variant, variant_to_object>();
  implicitly_convertible<a, variant>();
  implicitly_convertible<b, variant>();
  def("make_variant", make_variant);
  class_<vector>("vector").def(vector_indexing_suite<vector, true>());
  def("make_vector", make_vector);
 
Compiling Lets create a shared object for Python.
$ g++ -std=c++11 -fPIC -shared $(python-config --includes) -o bpv.so file.cpp -lboost_python
Running We can load the module into Python and see what it does.
>>> import bpv
>>> variant=bpv.make_variant()
>>> variant
<bpv.a object at 0x7f06bb2130c0>
>>> variant.x
0
>>> variant.x=2
>>> variant.x
2
Nice. We can access the underlying type, and even modify it. Lets see how our heterogeneous container wrapping code behaves.
>>> vector=bpv.make_vector()
>>> vector
<bpv.vector object at 0x7f20693289d0>
>>> len(vector)
3
>>> list(vector)
[<bpv.a object at 0x7f20693190c0>, <bpv.b object at 0x7f20693193d0>, <bpv.a object at 0x7f2069319440>]
So far, so good. This will at least make it possible to convert heterogeneous containers from C++ to Python, which was my initial goal. Unfortunately, contained objects are not treated as references. Whenever retrieved, we get a copy. So in-place modification does not work.
>>> vector[0].x
0
>>> vector[0].x=2
>>> vector[0].x
0
However, we can override an existing element with a modified copy.
>>> e0=vector[0]
>>> type(e0)
<class 'bpv.a'>
>>> e0.x = 2
>>> vector[0] = e0
>>> vector[0].x
2
And we can also use the append and extend methods of Python containers.
>>> len(vector)
3
>>> vector.extend(vector)
>>> vector.append(bpv.a())
>>> len(vector)
7
>>> len(filter(lambda x: type(x)==bpv.b, vector))
2
>>> len(filter(lambda x: type(x)==bpv.a, vector))
5
>>> map(lambda x: x.x, filter(lambda x: type(x)==bpv.a, vector))
[2, 0, 2, 0, 0]
All that is missing for a perfect world is reference semantics for container elements. If anyone has a hint on how to achieve this, please let me know.

Next.

Previous.