A new release 0.1.0 of the RPushbullet package (interfacing the neat Pushbullet service) landed on CRAN today.
It brings a number of goodies relative to the first release 0.0.2 of a few months ago:
~/.rpushbullet.json file or optionsrecpients which can be indices, device names or, if missing entirely, the (new) default deviceemail is supported as another recipient option in which case the Pushbullet service will send an email to the give addresspbGetDevices() now returns a proper S3 object with corresponding print() and summary() methodsThis post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
This doesn t look like we share resources yet, right? The dependencies are still bundled with the application requiring them. The trick lies in the installation step: While the application above can be executed right away without installing it, there will also be an option to install it. For the user, this will mean that the application shows up in GNOME-Shell s overview or KDEs Plasma launcher, gets properly registered with mimetypes and is if installed for all users available system-wide.
Technically, this will mean that the application s main bundle is extracted and moved to a special location on the file system, so are the dependency-bundles. If bundles already exist, they will not be installed again, and the new application will simply use the existing software. Since the bundles contain information about their dependencies, the system is able to determine which software is needed and which can simply be deleted from the installation directories.
If the application is started now, the bundles are combined and mounted, so the application can see the libraries it depends on.
Additionally, this concept allows secure updates of applications and shared resources. The bundle metadata contains an URL which points to a bundle repository. If new versions are released, the system s auto-updater can automatically pick these up and install them this means e.g. the Qt bundle will receive security updates, even if the developer who shipped it with his/her app didn t think of updating it.
Conclusion
So far, no productive code exists for this I just have a proof-of-concept here. But I pretty much like the idea, and I am thinking about going further in that direction, since it allows deploying applications on the Linux desktop as well as deploying software on servers in a way which plays nice with the native package manager, and which does not duplicate much code (less risk of having not-updated libraries with security flaws around).
However, there might be issues I haven t thought about yet. Also, it makes sense to look at GNOME to see how the whole 3rd-party app deployment issue develops. In case I go further with Listaller-NEXT, it is highly likely that it will make use of the ideas sketched above (comments and feedback are more than welcome!).
I've wanted to ride to Paris for a few months now but was put off by the hassle of taking a bicycle on the Eurostar, as well having a somewhat philosophical and aesthetic objection to taking a bike on a train in the first place. After all, if one already is possession of a mode of transport...
My itinerary was straightforward:
Tower Bridge.
Fanny's Farm Shop, Merstham, Surrey.
Plumpton, East Sussex.
West Pier, Newhaven.
Leaving Newhaven on the 23h00 ferry.
Beauvoir-en-Lyons, Haute-Normandie.
S rifontaine, Picardie.
La tour Eiffel, Paris.
Champ de Mars, Paris.
Pont de Grenelle, Paris.
Cormeilles-en-Vexin, le-de-France.
Gisors, Haute-Normandie.
Paris-Brest, Gisors, Haute-Normandie.
Gournay-en-Bray, Haute-Normandie.
D but de l'Avenue Verte, Forges-les-Eaux, Haute-Normandie.
Mesni res-en-Bray, Haute-Normandie.
Dieppe, Haute-Normandie.
La Mancha .
Peacehaven, East Sussex.
Highbrook, West Sussex.
London weather.
git merge-file. I am glad to say that Elektra outperforms git s merge functionality in the same ways it outperforms diff3 when applied to configuration files. Obviously, git s merge functionality does a much better job with source-code as that is not the goal of elektra. In that previous example, I showed that because diff3 is lined based, where Elektra is not (unless you mount a file using the line plugin). The example I used before, and I will go over again, is using smb.conf and in-line comments.
Many of our storage plug-ins understand the difference between comments and actual configuration data. So if a configuration file has an inline comment like so:
max log size = 10000 ; Controls the size of the log file (in KiB)
we can compare the actual Keys, value pairs between versions max log size = 10000 and deal with the comments separately.
As a result, if we have a base:
max log size = 1000 ; Size in KiB
max log size = 10000 ; Size in KiB
max log size = 1000 ; Controls the size of the log file (in KiB)
elektra-merge would be:
max log size = 10000 ; Controls the size of the log file (in KiB)
Just like diff3, git merge-file can only compare these lines as lines, and thus there is a conflict. When running git merge-file smb.conf.ours smb.conf.base smb.conf.theirs we get the following output showing a conflict:
<<<<<<< smb.conf.ours
max log size = 2000 ; Size in KiB
=======
max log size = 1000 ; Controls the size of the log file (in KiB)
>>>>>>> smb.conf.theirs
This really shows the strength of Elektra s plugin system and why it makes merge an obvious use-case of Elektra. I hope this example makes it clear why using Elektra s merge functionality is advantageous over git s merge functionality. Once again I would like to stress the importance of quality storage plug-ins for Elektra. The more quality plugins we have the more powerful Elektra can be. If you are interested in plugins and would like to help us by adding functionality to Elektra by creating a new plug-in be sure to read my basic tutorial on how to do so.
Sincerely,
I just installed, for work, Hanno B ck s bashcheck utility on our monitoring system, and watched all systems go blue.
All but two. One is not executing remote scripts from the monitoring for security reasons, the other is my desktop which runs Debian sid (unstable).
(Update, 2014-10-20: jessie, precise, trusty are also green now.)
This means that all those distributions still have unfixed #shellshock bugs.
root@debian-wheezy:~ # env 'BASH_FUNC_foo()=() x() _; ; x() _; <<'"$(perl -e ' print "A"x1000 '); " bash -c : Segmentation fault 139 root@debian-wheezy:~ # dmesg tail -1 [3121102.362274] bash[1699]: segfault at dfdfdfdf ip 00000000f766df36 sp 00000000ffe90b34 error 4 in libc-2.13.so[f75ee000+15d000]
After 10 years I finally got around to re-decorating my website. One reason was ICFP, where just too many people told me that I don t look like on my old website any more (which is very true). Another reason was that I was visting my brother, who is very good at web design (check out his portfolio), who could help me a bit.
I wanted something practical and maybe a bit staid, so I drew inspiration from typical Latex typography, and also from Edward Z. Yang s blog: A serif font (Utopia) for the main body, justified and hyphenated text. Large section headers in a knobbly bold sans-serif font (Latin Modern Sans, which reasonably resembles Computer Modern). To intensify that impression, I put the main text on a white box that lies like a paper on the background. As a special gimmic the per-page navigation (or, in the case of the blog, the list of categories) is marked up like a figure in a paper.
Of course this would be very dire without a suitable background. I really like the procedural art by Jared Tarbell, espcially substrate and interAggregate. Both have been turned into screensavers shipped with xscreensaver, so I hacked the substrate code to generate a seamless tile and took a screenshot of the result. I could not make up my mind yet how dense it has to be to look good, so I for every page I randomly pick one of six variants randomly for now.
I simplified the navigation a bit. The old News section has been removed recently already. The Links section is gone I guess link lists on homepages are so 90s. The section Contact and About me are merged and awaiting some cleanup. The link to the satire news Heisse News is demoted to a mention on the Contents section.
This hopefully helps to make the site navigatable on mobile devices (the old homepage was unusable). CSS media queries adjust the layout slightly on narrow screens, and separately for print devices.
Being the nostaltic I am, I still keep the old design, as well as the two designs before that, around and commented their history.
As preannounced here the GNOME Infrastructure switched to a new Account Management System which is reachable at https://account.gnome.org. All the details will follow.
Introduction
It s been a while since someone actually touched the underlaying authentication infrastructure that powers the GNOME machines. The very first setup was originally configured by Jonathan Blandford (jrb) who configured an OpenLDAP istance with several customized schemas. (pServer fields in the old CVS days, pubAuthorizedKeys and GNOME modules related fields in recent times)
While OpenLDAP-server was living on the GNOME machine called clipboard (aka ldap.gnome.org) the clients were configured to synchronize users, groups, passwords through the nslcd daemon. After several years Jeff Schroeder joined the Sysadmin Team and during one cold evening (date is Tue, February 1st 2011) spent some time configuring SSSD to replace the nslcd daemon which was missing one of the most important SSSD features: caching. What surely convinced Jeff to adopt SSSD (a very new but promising sofware at that time as the first release happened right before 2010 s Christmas) and as the commit log also states ( New sssd module for ldap information caching ) was SSSD s caching feature.
It was enough for a certain user to log in once and the /var/lib/sss/db directory was populated with its login information preventing the LDAP daemon in charge of picking up login details (from the LDAP server) to query the LDAP server itself every single time a request was made against it. This feature has definitely helped in many occasions especially when the LDAP server was down for a particular reason and sysadmins needed to access a specific machine or service: without SSSD this wasn t ever going to work and sysadmins were probably going to be locked out from the machines they were used to manage. (except if you still had /etc/passwd , /etc/group and /etc/shadow entries as fallback)
Things were working just fine except for a few downsides that appeared later on:
ssh -l yourgnomeuserid account.gnome.orgThe command will start an SSH connection between you and account.gnome.org, once authenticated (with the SSH key you previously had registered on our Infrastructure) you will trigger a command that will directly send your brand new password on the e-mail registered for your account. From my tests seems GMail sees the e-mail as a phishing attempt probably because the body contains the word password twice. That said if the e-mail won t appear on your INBOX, please double-check your Spam folder. Now that Mango is gone how can I request a new account? With Mango we used to have a form that automatically e-mailed the maintainer of the selected GNOME module which was then going to approve / reject the request. From there and in the case of a positive vote from the maintainer the Accounts Team was going to create the account itself. With the recent introduction of a commit robot directly on l10n.gnome.org the number of account requests reduced its numbers. In addition to that users will now be able to perform pretty much all the needed maintenance on their accounts themselves. That said and while we will probably work on building a form in the future we feel that requesting accounts can definitely be achieved directly by mailing the Accounts Team itself which will mail the maintainer of the respective module and create the account. As just said the number of account creations has become very low and the queue is currently clear. The documentation has been updated to reflect these changes at: https://wiki.gnome.org/AccountsTeam
As preannounced here the GNOME Infrastructure switched to a new Account Management System which is reachable at https://account.gnome.org. All the details will follow.
Introduction
It s been a while since someone actually touched the underlying authentication infrastructure that powers the GNOME machines. The very first setup was originally configured by Jonathan Blandford (jrb) who configured an OpenLDAP istance with several customized schemas. (pServer fields in the old CVS days, pubAuthorizedKeys and GNOME modules related fields in recent times)
While OpenLDAP-server was living on the GNOME machine called clipboard (aka ldap.gnome.org) the clients were configured to synchronize users, groups, passwords through the nslcd daemon. After several years Jeff Schroeder joined the Sysadmin Team and during one cold evening (date is Tue, February 1st 2011) spent some time configuring SSSD to replace the nslcd daemon which was missing one of the most important SSSD features: caching. What surely convinced Jeff to adopt SSSD (a very new but promising sofware at that time as the first release happened right before 2010 s Christmas) and as the commit log also states ( New sssd module for ldap information caching ) was SSSD s caching feature.
It was enough for a certain user to log in once and the /var/lib/sss/db directory was populated with its login information preventing the LDAP daemon in charge of picking up login details (from the LDAP server) to query the LDAP server itself every single time a request was made against it. This feature has definitely helped in many occasions especially when the LDAP server was down for a particular reason and sysadmins needed to access a specific machine or service: without SSSD this wasn t ever going to work and sysadmins were probably going to be locked out from the machines they were used to manage. (except if you still had /etc/passwd , /etc/group and /etc/shadow entries as fallback)
Things were working just fine except for a few downsides that appeared later on:
ssh -l yourgnomeuserid account.gnome.orgThe command will start an SSH connection between you and account.gnome.org, once authenticated (with the SSH key you previously had registered on our Infrastructure) you will trigger a command that will directly send your brand new password on the e-mail registered for your account. From my tests seems GMail sees the e-mail as a phishing attempt probably because the body contains the word password twice. That said if the e-mail won t appear on your INBOX, please double-check your Spam folder. Now that Mango is gone how can I request a new account? With Mango we used to have a form that automatically e-mailed the maintainer of the selected GNOME module which was then going to approve / reject the request. From there and in the case of a positive vote from the maintainer the Accounts Team was going to create the account itself. With the recent introduction of a commit robot directly on l10n.gnome.org the number of account requests reduced its numbers. In addition to that users will now be able to perform pretty much all the needed maintenance on their accounts themselves. That said and while we will probably work on building a form in the future we feel that requesting accounts can definitely be achieved directly by mailing the Accounts Team itself which will mail the maintainer of the respective module and create the account. As just said the number of account creations has become very low and the queue is currently clear. The documentation has been updated to reflect these changes at: https://wiki.gnome.org/AccountsTeam
The next version of upstream MIT Kerberos will have support for fetching a ticket via ports 443 and marshalling the request over HTTPS. We ll need to run a proxy on the server side, but we should be able to make it work: Read up here
http://adam.younglogic.com/2014/06/kerberos-firewalls
It s been more than 2 years since my last post about my smartphone. In the time after that post I upgraded my much loved Windows Phone 7 device to Windows Phone 8 (which I got rid of within months, for sucking), briefly used Firefox OS, then eventually used a Nexus 4 for at least a year.
After years of terrible service provision and pricing, I decided I would not stay with my network Orange a moment longer and in getting a new contract, I would get a new phone too. So on Friday, I signed up to a new 15 per month contract with Three, including 200 minutes, unlimited data, and 25GB of data roaming in the USA and other countries (a saving of 200,000 per month versus Orange). Giffgaff is similarly competitive for data, but not roaming. No other network in the UK is competitive.
For the phone, I had a shortlist of three: Apple iPhone 6, Sony Xperia Z3 Compact, and Samsung Galaxy Alpha. These are all small phones by 2014 standards, with a screen about the same size as the Nexus 4. I didn t consider any Windows Phone devices because they still haven t shipped a functional music player app on Windows Phone 8. Other more fringe OSes weren t considered, as I insist on trying out a real device in person before purchase, and no other comparable devices are testable on the high street.
iPhone 6
This was the weakest offering, for me. 120 more than the Samsung, and almost 200 more than the Sony, a much lower hardware specification, physically larger, less attractive, and worst of all mandatory use of iTunes for Windows for music syncing.
The only real selling point for me would be for access to iPhone apps. And, I guess, decreased chance of mockery by co-workers.
Galaxy Alpha
Now on to the real choices. I ve long felt that Samsung s phones are ugly plasticy tat the Galaxy S5 is popular, well-marketed, but looks and feels cheap compared to HTC s unibody aluminium One. They ve also committed the cardinal sin of gimping the specifications of their mini (normal-sized) phones, compared to the normal (gargantuan) versions. The newly released S5 Mini is about the same spec as early 2012 s S3, the S4 Mini was mostly an S2 internally, and so on.
However, whilst HTC have continued along these lines, Samsung have finally released a proper phone under 5 , in the Alpha.
The Alpha combines a 4.7 AMOLED screen, a plastic back, metal edges, 8-core big.LITTLE processor, and 2GB RAM. It is a PRETTY device the screen really dazzles (as is the nature of OLED). It feels like a mix of design cues from an iPhone and Samsung s own, keeping the angular feel of iPhone 4->5S rather than the curved edges on the iPhone 6.
The Galaxy Alpha was one of the two devices I seriously considered.
Xperia Z3 Compact
The other Android device I considered was the Compact version of Sony s new Xperia Z3. Unlike other Android vendors, Sony decided that mini shouldn t mean low end when they released the Z1 compact earlier this year. The Z3 follows suit, where the same CPU and storage are found on both the big and little versions.
The Z3C has a similar construction to the Nexus 4, with glass front and back, and plastic rim. The specification is similar to the Galaxy Alpha (with a quadcore 2.5GHz Qualcomm processor about 15% faster than the big.LITTLE Exynos in the Galaxy Alpha). It differs in a few places LCD rather than AMOLED (bad); a non-removable (bad) 2600 mAh battery (good) compared to the removable 1860 mAh in the Samsung; waterproofing (good); A less hateful Android shell (Xperia on Android vs Samsung Touchwiz).
For those considering a Nexus-4-replacement class device (yes, rjek, that means you), both the Samsung and the Sony are worth a look. They both have good points and bad points. In the end, both need to be tested to form a proper opinion. But for me, the chunky battery and tasteful green were enough to swing it for the Sony. So let s see where I stand in a few months time. Every phone I ve owned, I ve ended up hating it for one reason or another. My usual measure for whether a phone is good or not is how long it takes me to hit the I can t use this limit. The Nokia N900 took me about 30 minutes, the Lumia 800 lasted months. How will the Z3 Compact do? Time will tell.
It's taken me a while to get sufficiently riled up about Australia's current
Islamaphobia outbreak, but it's been brewing in me for a couple of weeks.
For the record, I'm an Atheist, but I'll defend your right to practise your
religion, just don't go pushing it on me, thank you very much. I'm also not a
huge fan of Islam, because it does seem to lend itself to more violent
extremism than other religions, and ISIS/ISIL/IS (whatever you want to call
them) aren't doing Islam any favours at the moment. I'm against extremism of
any stripes though. The Westboro Baptists are Christian extremists. They just
don't go around killing people. I'm also not a big fan of the burqa,
but again, I'll defend a Muslim woman's right to choose to wear one. They key
point here is choice.
I got my carpets cleaned yesterday by an ethnic couple. I like accents, and I
was trying to pick theirs. I thought they may have been Turkish. It turned out
they were Kurdish. Whenever I hear "Kurd" I habitually stick "Bosnian" in front
of it after the Bosnian War that happened in my childhood. Turns out I wasn't
listening properly, and that was actually "Serb". Now I feel dumb, but I
digress.
I got chatting with the lady while her husband did the work. I got a refresher
on where most Kurds are/were (Northern Iraq) and we talked about Sunni versus
Shia Islam, and how they differed. I learned a bit yesterday, and I'll have
to have a proper read of the Wikipedia article I just linked to, because I
suspect I'll learn a lot more.
We briefly talked about burqas, and she said that because they were Sunni, they
were given the choice, and they chose not to wear it. That's the sort of Islam
that I support. I suspect a lot of the women running around in burqas don't get
a lot of say in it, but I don't think banning it outright is the right
solution to that. Those
women need to feel empowered enough to be able to cast off their burqas if
that's what they want to do.
I completely agree that a woman in a burqa entering a secure place (for
example Parliament House) needs to be identifiable (assuming that
identification is verified for all entrants to Parliament House). If it's
not, and they're worried about security, that's what the metal detectors are
for. I've been to Dubai. I've seen how they handle women in burqas at
passport control. This is an easily solvable problem. You don't have to
treat burqa-clad women as second class citizens and stick them in a glass
box. Or exclude them entirely.
A lot of the kernel work I've ended up doing has involved dealing with bugs on Intel-based systems - figuring out interactions between their hardware and firmware, reverse engineering features that they refuse to document, improving their power management support, handling platform integration stuff for their GPUs and so on. Some of this I've been paid for, but a bunch has been unpaid work in my spare time[1].
Chromium (the browser) and DRI3
I got a note on IRC a week ago that Chromium was crashing with
DRI3.
The Google team working on Chromium eventually sent me a link to the
bug report.
That's secret Google stuff, so you won't be able to
follow the link, even though it's a bug in a free software application
when running on free software drivers.
There's a
bug report in the freedesktop bugzilla
which looks the same to me.
In both cases, the recommended fix was to switch from DRI3 back to
DRI2. That's not exactly a great plan, given that DRI3 offers better
security between GPU-using applications, which seems like a pretty
nice thing to have when you're running random GL applications from the
web.
Chromium Sandboxing
I'm not entirely sure how it works, but Chromium creates a process
separate from the main browser engine to talk to the GPU. That process
has very limited access to the operating system via some fancy library
adventures. Presumably, the hope is that security bugs in the GL
driver would be harder to leverage into a remote system exploit.
Debugging in this environment is a bit tricky as you can't simply run
chromium under gdb and expect to be able to set breakpoints in the GL
driver. Instead, you have to run chromium with a magic flag which
causes the GPU process to pause before loading the driver so you can
connect to it with gdb and debug from there, along with a flag that lets you
see crashes within the gpu process and the usual flag that
causes chromium to ignore the GPU black list which seems to always
include the Intel driver for one reason or another:
$ chromium --gpu-startup-dialog --disable-gpu-watchdog --ignore-gpu-blacklist
Once Chromium starts up, it will print out a message telling you to
attach gdb to the GPU process and send that process a SIGUSR1 to
continue it. Now you can happily debug and get a stack trace when the
crash occurs.
Locating the Bug
The bug manifested with a segfault at the first access to a
DRI3-allocated buffer within the application. We've seen this problem
in the past; whenever buffer allocation fails for some reason, the
driver ignores the problem and attempts to de-reference through the
(NULL) buffer pointer, causing a segfault. In this case, Chromium
called glClear, which tried (and failed) to allocate a back buffer
causing the i965 driver to subsequently segfault.
We should probably go fix the i965 driver to not segfault when
buffer allocation fails, but that wouldn't provide a lot of additional
information. What I have done is add some error messages in the DRI3
buffer allocation path which at least tell you why the buffer
allocation failed. That patch has been merged to Mesa master, and
should also get merged to the Mesa stable branch for the next stable
release.
Once I had added the error messages, it was pretty easy to see what
happened:
$ chromium --ignore-gpu-blacklist
[10618:10643:0930/200525:ERROR:nss_util.cc(856)] After loading Root Certs, loaded==false: NSS error code: -8018
libGL: pci id for fd 12: 8086:0a16, driver i965
libGL: OpenDriver: trying /local-miki/src/mesa/mesa/lib/i965_dri.so
libGL: Can't open configuration file /home/keithp/.drirc: Operation not permitted.
libGL: Can't open configuration file /home/keithp/.drirc: Operation not permitted.
libGL error: DRI3 Fence object allocation failure Operation not permitted
The first two errors were just the sandbox preventing Mesa from using
my GL configuration file. I'm not sure how that's a security problem,
but it shouldn't harm the driver much.
The last error is where the problem lies. In Mesa, the DRI3
implementation uses a chunk of shared memory to hold a fence object
that lets Mesa know when buffers are idle without using the X
connection. That shared memory segment is allocated by creating a
temporary file using the O_TMPFILE flag:
fd = open("/dev/shm", O_TMPFILE O_RDWR O_CLOEXEC O_EXCL, 0666);
This call cannot fail as /dev/shm is used by glibc for shared memory
objects, and must therefore be world writable on any glibc
system. However, with the Chromium sandbox enabled, it returns
EPERM.
Running Without a Sandbox
Now that the bug appears to be in the sandboxing code, we can re-test
with the GPU sandbox disabled:
$ chromium --ignore-gpu-blacklist --disable-gpu-sandbox
And, indeed, without the sandbox getting in the way of allocating a
shared memory segment, Chromium appears happy to use the Intel driver
with DRI3.
Final Thoughts
I looked briefly at the Chromium sandbox code. It looks like it needs
to know intimate details of the OpenGL implementation for every
possible driver it runs on; it seems to contain a fixed list of all
possible files and modes that the driver will pass to open(2). That
seems incredibly fragile to me, especially when used in a general
Linux desktop environment. Minor changes in how the GL driver operates
can easily cause the browser to stop working.
This was obviously causing some issues for the ARM Debian ports which Steve wanted to rectify. After talking the problem through for a while we came to the conclusion we could use much larger 60mm fans to blow air directly through the top of the case onto the cpu heatsink.
Thanks to the sponsorship of multiple companies, I have been paid to work 11 hours on Debian LTS this month.
I started by doing lots of triage in the security tracker (if you want to help, instructions are here) because I noticed that the dla-needed.txt list (which contains the list of packages that must be taken care of via an LTS security update) was missing quite a few packages that had open vulnerabilities in oldstable.
In the end, I pushed 23 commits to the security tracker. I won t list the details each time but for once, it s interesting to let you know the kind of things that this work entailed:
No comment Liked this article? Click here. My blog is Flattr-enabled.
For a little toy project of mine (a wikipedia XML dump word counter)
I wrote a little C++11 helper class to distribute work to all
available CPU cores. It took me many years to overcome
my fear of threading: In the past, whenever I toyed with
threaded code, I ended up having a lot of deadlocks, and generally being
confused. It appears that I finally have understood enough of this
crazyness to be able to come up with the small helper class below.
int main()
typedef std::string item_type;
distributor<item_type> process([](item_type &item)
// do work
);
while (/* input */) process(std::move(/* item */));
return 0;
#include <condition_variable>
#include <mutex>
#include <queue>
#include <stdexcept>
#include <thread>
#include <vector>
template <typename Type, typename Queue = std::queue<Type>>
class distributor: Queue, std::mutex, std::condition_variable
typename Queue::size_type capacity;
bool done = false;
std::vector<std::thread> threads;
public:
template<typename Function>
distributor( Function function
, unsigned int concurrency = std::thread::hardware_concurrency()
, typename Queue::size_type max_items_per_thread = 1
)
: capacity concurrency * max_items_per_thread
if (not concurrency)
throw std::invalid_argument("Concurrency must be non-zero");
if (not max_items_per_thread)
throw std::invalid_argument("Max items per thread must be non-zero");
for (unsigned int count 0 ; count < concurrency; count += 1)
threads.emplace_back(static_cast<void (distributor::*)(Function)>
(&distributor::consume), this, function);
distributor(distributor &&) = default;
distributor &operator=(distributor &&) = delete;
~distributor()
std::lock_guard<std::mutex> guard(*this);
done = true;
notify_all();
for (auto &&thread: threads) thread.join();
void operator()(Type &&value)
std::unique_lock<std::mutex> lock(*this);
while (Queue::size() == capacity) wait(lock);
Queue::push(std::forward<Type>(value));
notify_one();
private:
template <typename Function>
void consume(Function process)
std::unique_lock<std::mutex> lock(*this);
while (true)
if (not Queue::empty())
Type item std::move(Queue::front()) ;
Queue::pop();
notify_one();
lock.unlock();
process(item);
lock.lock();
else if (done)
break;
else
wait(lock);
;
A new release 0.11.3 of Rcpp is now on the CRAN network for GNU R, and an updated Debian package has been uploaded too.
Rcpp has become the most popular way of enhancing GNU R with C++ code. As of today, 273 packages on CRAN depend on Rcpp for making analyses go faster and further.
This release brings a fairly large number of continued enhancements, fixes and polishing to Rcpp. These were provided by a total of seven different contributors---which is a new record as well.
See below for a detailed list of changes extracted from the NEWS file, but some highlights included in this release are
InternalFunction interface, and new Timer functionality.Rcpp Attributes as well as a new dryRun option.DESCRIPTION file.Rcpp.package.skeleton() can now deploy functionality from pkgKitten to create Rcpp packages that purr.Changes in Rcpp version 0.11.3 (2014-09-27)Thanks to CRANberries, you can also look at a diff to the previous release As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads page, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.
- Changes in Rcpp API:
- The deprecation of
RCPP_FUNCTION_*which was announced with release 0.10.5 last year is proceeding as planned, and the filemacros/preprocessor_generated.hhas been removed.Timerno longer records time between steps, but times from the origin. It also gains aget_timers(int)methods that creates a vector ofTimerthat have the same origin. This is modelled on theRcpp11implementation and is more useful for situations where we use timers in several threads.Timeralso gains a constructor taking ananotime_tto use as its origin, and aoriginmethod. This can be useful for situations where the number of threads is not known in advance but we still want to track what goes on in each thread.- A cast to
boolwas removed in the vector proxy code as inconsistent behaviour between clang and g++ compilations was noticed.- A missing
update(SEXP)method was added thanks to pull request by Omar Andres Zapata Mesa.- A proxy for
DimNameswas added.- A
no_initoption was added for Matrices and Vectors.- The
InternalFunctionclass was updated to work withstd::function(provided a suitable C++11 compiler is available) via a pull request by Christian Authmann.- A
new_env()function was added toEnvironment.h- The return value of range eraser for Vectors was fixed in a pull request by Yixuan Qiu.
- Changes in Rcpp Sugar:
- In
ifelse(), the returnedNAtype was corrected foroperator[].- Changes in Rcpp Attributes:
- Include LinkingTo in DESCRIPTION fields scanned to confirm that C++ dependencies are referenced by package.
- Add
dryRunparameter tosourceCpp.- Corrected issue with relative path and R chunk use for
sourceCpp.- Changes in Rcpp Documentation:
- The
Rcpp-FAQvignette was updated with respect to OS X issues.- A new entry in the
Rcpp-FAQclarifies the use of licenses.- Vignettes build results no longer copied to
/tmpto please CRAN.- The Description in
DESCRIPTIONhas been shortened.- Changes in Rcpp support functions:
- The
Rcpp.package.skeleton()function will now use pkgKitten package, if available, to create a package which passesR CMD checkwithout warnings. A newSuggests:has been added for pkgKitten.- The
modules=TRUEcase forRcpp.package.skeleton()has been improved and now runs without complaints fromR CMD checkas well.- Changes in Rcpp unit test functions:
- Functions from the RUnit package are now prefixed with
RUnit::- The
testRcppModuleandtestRcppClasssample packages now passR CMD check --as-crancleanly with NOTES or WARNINGS
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
A few months ago I was showing my friend Bruno Souza the work I had been doing with my childhood friend and robotics genius, David Hanson. I had been watching what David was going through in his process of creating life-like robots with the limited industrial software available for motor control. I had suggested to David that binding motors to Blender control structures was a genuinely viable possibility. David talked with his forward looking CEO, Jong Lee, and they were gracious enough to invite me to Hong Kong to make this exciting idea a reality. Working closely the HRI team (Vytas, Gabrielos, Fabien and Davide) with David s friend and collaborators at OpenCog (Ben Goertzel, Mandeep, David, Jamie, Alex and Samuel) a month long creative hack-fest yielded pretty amazing results.
Bruno is an avid puppeteer, a global organizer of java user groups and creator of Juggy the Java Finch, mascot of Java users and user groups everywhere. We started talking about how cool it would be to have a robot version of Juggy. When I was in China I had spent a little time playing with Mark Tilden s RSMedia and various versions of David s hobby servo based emotive heads. Bruno and I did a little research into the ROS Java bindings for the Robot Operating System and decided that if we could make that part of the picture we had a great and fun idea for a JavaOne talk.
Hunting and gathering
I tracked down a fairly priced RSMedia in Alaska, Bruno put a pair of rubber Juggy puppet heads in the mail and we were on our way.sudo dd bs=4M if=/dev/sdb gzip > /home/your_username/image date +%d%m%y .gzGetting Java in the mix Once you have your Pi all set up with minimal Raspbian and ROS you are going to want a Java VM. The Pi runs the ARM CPU so you need the corresponding version of Java. I tried getting things going initially with OpenJDK and I had some issues with that. I will work on resolving that in the future because I would like to have a 100% Free Software kit for this but since this was for JavaOne I also wanted JDK8, which isn t available in Debian yet. So, I downloaded the Oracle JDK8 package for ARM. Java 8 JDK for ARM At this point you are ready to start installing the ROS Java packages. I m pretty sure the way I did this initially is wrong but I was trying to reconcile the two install procedures for ROS Java and ROS Hydro for Raspberry Pi. I started by following these directions for ROS Java but with a few exceptions (you have to click the install from source link in the page to see the right stuff: Installing ROS Java on Hydro Now these instructions are good but this is a Pi running Debian and not an Ubuntu install. You won t run the apt-get package commands because those tools were already installed in your earlier steps. Also, this creates its own workspace and we really want these packages all in one workspace. You can apparently chain workspaces in ROS but I didn t understand this well enough to get it working so what I did was this:
> mkdir -p ~/rosjava > wstool init -j4 ~/rosjava/src https://raw.github.com/rosjava/rosjava/hydro/rosjava.rosinstall > source ~/ros_catkin_ws/install_isolated/setup.bash > cd ~/rosjava # Make sure we've got all rosdeps and msg packages. > rosdep update > rosdep install --from-paths src -i -yand then copied the sources installed into ~/rosjava/src into my main ~/ros_catkin_ws/src. Once those were copied over I was able to run a standard build.
> catkin_make_isolated --installLike the main ROS install this process will take a little while. The Java gradle builds take an especially long time. One thing I would recommend to speed up your workflow is to have an x86 Debian install (native desktop, QEMU instance, docker, whatever) and do these same build from source installs there. This will let you try your steps out on a much faster system before you try them out in the Pi. That can be a big time saver. Putting together the pieces Around this time my RSMedia had finally showed up from Alaska. At first I thought I had a broken unit because it would power up, complain about not passing system tests and then shut back down. It turns out that if you just put the D batteries in and miss the four AAs that it will kind of pretend to be working so watch for that mistake. Here is a picture of the RSMedia when it first came out of the box:
Other parts were starting to roll in as well. The rubber puppet heads had made their way through Brazilian customs and my Pololu Mini Maestro 24 had also shown up as well as the my servo motors and pan and tilt camera rig. I had previously bought a set of 10 motors for goofing around so I bought the pan and tilt rig by itself for about $5(!) but you can buy a complete set for around $25 from a number of EBay stores.
Complete pan and tilt rig with motors for $25
A bit more about the Pololu. This astonishing little motor controller costs about $25 and gives you control of 24 motors with an easy to use and high level serial API. It is probably also possible to control these servos directly from the Pi and eliminate this board but that will be genuinely difficult because of the real-time timing issues. For $25 this thing is a real gem and you won t regret buying it.
Now it was time to start dissecting the RSMedia and getting control of its brain. Unfortunately a lot of great information about the RSMedia has floated away since it was in its heyday 5 years ago but there is still some solid information out there that we need to round up and preserve. A great resource is the SourceForge based website here at http://rsmediadevkit.sourceforge.net.
That site has links to a number of useful sites. You will definitely want to check out their wiki. To disassemble the RSMedia I followed their instructions. I will say, it would be smart to take more pictures as you are going because they don t take as many as they should. I took pictures of each board and its associated connections as dismantled the unit and that helped me get things back together later. Another important note is that if all you want to do is solder onto the control board and not replace the head then its feasible to solder the board in place without completely disassembling the unit. Here are some photos of the dis-assembly:
Now I also had to start adjusting the puppet head, building an armature for the motors to control it and hooking it into the robot. I need to take some more photos of the actual armature. I like to use cardboard for this kind of stuff because it is so fast to work with and relatively strong. One trick I have also learned about cardboard is that if you get something going with it and you need it to be a little more production strength you can paint it down with fiberglass resin from your local auto store. Once it dries it becomes incredibly tough because it soaks through the fibers of the cardboard and hardens around them. You will want to do this in a well ventilated area but its a great way to build super tough prototypes.
Another prototyping trick I can suggest is using a combination of Velcro and zipties to hook things together. The result is surprisingly strong and still easy to take apart if things aren t working out. Velcro self-adhesive pads stick to rubber like magic and that is actually how I hooked the jaw servo onto the mask. You can see me torturing its first initial connection here:
Since the puppet head had come all the way from Brazil I decided to cook some chicken hearts in the churrascaria style while I worked on them in the garage. This may sound gross but I m telling you, you need to try it! I soaked mine in soy sauce, Sriracha and chinese cooking wine. Delicious but I digress.
As I was eating my chicken hearts I was also connecting the pan and tilt armature onto the puppet s jaw and eye assembly. It took me most of the evening to get all this going but by about one in the morning things were starting to look good!
I only had a few days left to hack things together before JavaOne and things were starting to get tight. I had so much to do and had also started to run into some nasty surprises with the ROS Java control software. It turns out that ROS Java is less than friendly with ROS message structures that are not built in . I had tried to follow the provided instructions but was not (and still have not) been able to get that working.
Using unofficial messages with ROS Java
I still needed to get control of the RSMedia. Doing that required the delicate operation of soldering to its control board. On the board there are a set of pins that provide a serial interface to the ARM based embedded Linux computer that controls the robot. To do that I followed these excellent instructions:
Connecting to the RSMedia Linux Console Port
After some sweaty time bent over a magnifying glass I had success:
I had previously purchased the USB-TTL232 accessory described in the article from Dallas awesome Tanner Electronics store in Dallas. If you are a geek I would recommend that you go there and say hi to its proprietor (and walking encyclopedia of electronics knowledge) Jim Tanner.
It was very gratifying when I started a copy of minicom, set it to 115200, N, 8, 1, plugged in the serial widget to the RSMedia and booted it up. I was greeted with a clearly recognizable Linux startup and console prompt. At first I thought I had done something wrong because I couldn t get it to respond to commands but I quickly realized I had flow control turned on. Once turned off I was able to navigate around the file system, execute commands and have some fun. A little research and I found this useful resource which let me get all kinds of body movements going:
A collection of useful commands for the RSMedia
At this point, I had a usable set of controls for the body as well as the neck armature. I had a controller running the industry s latest and greatest robotics framework that could run on the RSMedia without being tethered to power and I had most of a connection to Java going. Now I just had to get all those pieces working together. The only problem is that time was running out and I only had a couple of days until my talk and still had to pack and square things away at work.
The last day was spent doing things that I wouldn t be able to do on the road. My brother Erik (and fantastic artist) came over to help paint up the juggy head and fix the eyeball armature. He used a mix of oil paint, rubber cement which stuck to the mask beautifully.
I bought battery packs for the USB Pi power and the 6v motor control and integrated them into a box that could sit below the neck armature. I fixed up a cloth neck sleeve that could cover everything. Luckily during all this my beautiful and ever so supportive girlfriend Becca had helped me get packed or I probably wouldn t have made it out the door.
Welcome to San Francisco
THIS ARTICLE IS STILL BEING WRITTEN
Next.