Search Results: "Dirk Eddelbuettel"

28 April 2016

Dirk Eddelbuettel: RcppRedis 0.1.7

A new release of RcppRedis arrived on CRAN today. And just like for the previous release, Russell Pierce contributed a lot of changes via several pull requests which make for more robust operations. In addition, we have started to add support for MessagePack by relying on our recently-announced RcppMsgPack package.
Changes in version 0.1.7 (2016-04-27)
  • Added support for timeout constructor argument (PR #14 by Russell Pierce)
  • Added new commands exists, ltrim, expire and pexpire along with unit tests (PR #16 by Russell Pierce)
  • Return NULL for empty keys in serialized get for consistency with lpop and rpop (also PR #16 by Russell Pierce)
  • Minor corrections to get code and hget and hset documentation (also PR #16 by Russell Pierce)
  • Error conditions are now properly forwarded as R errors (PR #22 by Russell Pierce)
  • Results from Redis commands are now checked for NULL (PR #23 by Russell Pierce)
  • MessagePack encoding can now be used which requires MessagePackage headers of version 1.0 or later; the (optional) RcppMsgPack package can be used.
Courtesy of CRANberries, there is also a diffstat report for the most recent release. More information is on the RcppRedis page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

26 April 2016

Dirk Eddelbuettel: RcppMgsPack 0.1.0

Over the last few months, I have been working casually on a new package to integrate MessagePack with R. What is MessagePack, you ask? To quote its website, "It's like JSON, but fast and small." Or in more extended terms:
MessagePack is an efficient binary serialization format. It lets you exchange data among multiple languages like JSON. But it's faster and smaller. Small integers are encoded into a single byte, and typical short strings require only one extra byte in addition to the strings themselves.
Now, serialization formats are a dime a dozen: JSON, BSON, Protocol Buffers, CaptnProto, Flatbuffer. The list goes on and on. So why another? In a nutshell: "software ecosystems". I happen to like working with Redis, and within the world of Redis, MessagePack is a first-class citizen supported by things close to the core like the embedded Lua interpreter, as well as fancy external add-ons such as the Redis Desktop Manager GUI. So nothing overly fundamentalist here, but a fairly pragmatic choice based on what happens to fit my needs. Plus, having worked on and off with Protocol Buffers for close to a decade, the chance of working with something not requiring a friggin' schema compiler seemed appealing for a chance. So far, we have been encoding a bunch of data streams at work via MessagePack into Redis (and of course back). It works really well---header-only C++11 libraries for the win. I'll provide an updated RcppRedis which uses this (if present) in due course. For now and the foreseeable future, this RcppMsgPack package will live only on the ghrr drat repository. To make RcppMsgPack work, I currently have to include the MessagePack 1.4.0 headers. A matching package for this version of the headers is in Debian but so far only in experimental. Once this hits the mainline repository I can depend on it, and upload a (lighter, smaller) RcppMsgPack to CRAN. Until then, please just do
## install drat if not present
if (!require(drat)) install.packages("drat")
## use drat to select ghrr repo
drat::addRepo("ghrr")
## install RcppMsgPack
install.packages("RcppMsgPack")
More details, issue tickets etc are at the GitHub repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

24 April 2016

Dirk Eddelbuettel: Brad Mehldau at the CSO, again

Almost seven years since the last time we saw him here, Brad Mehldau returned to the CSO for a concert on Friday eve in his standard trio setup with Larry Grenadier on bass and Jeff Ballard on drums. The material mostly (all?) new and drawn from the upcoming album Blues and Ballads. The morning of the concert---which happened to be the final one in their tour---he retweeted a bit from this review in the Boston Globe
[Brad Mehldau] flashed facets of his renowned pianism: crystalline touch, deep lyricism, harmonic sophistication, adroit use of space, and the otherworldly independence of his right and left hands.
I cannot really describe his style any better than this. If you get a chance to see him, go!

18 April 2016

Dirk Eddelbuettel: RcppCCTZ 0.0.4

A few days ago a new upstream version "2.0" of CCTZ was released. See here for the corresponding post on the Google OpenSource Blog. CCTZ is a C++ library for translating between absolute and civil times using the rules of a time zone. It requires only a proper C++11 compiler and the standard IANA time zone data base which standard Unix, Linux, OS X, ... computers tend to have in /usr/share/zoneinfo. RcppCCTZ connects this library to R by relying on Rcpp. This new version adds more support to the notion of civil time representation -- i.e. independent of timezones -- which can then be mapped to different time zone representations. Changes in this version are summarized here:
Changes in version 0.0.4 (2016-04-17)
  • Synchronized with CCTZ v2 upstream.
  • Updated examples.cpp accordingly
A quick example is provided here where we look at the time when Neil Armstrong first stepped on the moon as an absolute ("civil") time and two local representations:
// from examples/hello.cc
// 
// [[Rcpp::export]]
int helloMoon()  
    cctz::time_zone syd;
    if (!cctz::load_time_zone("Australia/Sydney", &syd)) return -1;
    // Neil Armstrong first walks on the moon
    const auto tp1 = cctz::convert(cctz::civil_second(1969, 7, 21, 12, 56, 0), syd);
    const std::string s = cctz::format("%F %T %z", tp1, syd);
    Rcpp::Rcout << s << "\n";
    cctz::time_zone nyc;
    cctz::load_time_zone("America/New_York", &nyc);
    const auto tp2 = cctz::convert(cctz::civil_second(1969, 7, 20, 22, 56, 0), nyc);
    return tp2 == tp1 ? 0 : 1;
 
We can call this from R, and get the expected result (of equivalence between the dates):
R> library(RcppCCTZ)
R> helloMoon()
1969-07-21 12:56:00 +1000
[1] 0
R>
We also have a diff to the previous version thanks to CRANberries. More details, issue tickets etc at the GitHub repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

12 April 2016

Reproducible builds folks: Reproducible builds: week 48 in Stretch cycle

What happened in the reproducible builds effort between March 20th and March 26th: Toolchain fixes Daniel Kahn Gillmor worked on removing build path from build symbols submitting a patch adding -fdebug-prefix-map to clang to match GCC, another patch against gcc-5 to backport the removal of -fdebug-prefix-map from DW_AT_producer, and finally by proposing the addition of a normalizedebugpath to the reproducible feature set of dpkg-buildflags that would use -fdebug-prefix-map to replace the current directory with . using -fdebug-prefix-map. Sergey Poznyakoff merged the --clamp-mtime option so that it will be featured in the next Tar release. This option is likely to be used by dpkg-deb to implement deterministic mtimes for packaged files. Packages fixed The following packages have become reproducible due to changes in their build dependencies: augeas, gmtkbabel, ktikz, octave-control, octave-general, octave-image, octave-ltfat, octave-miscellaneous, octave-mpi, octave-nurbs, octave-octcdf, octave-sockets, octave-strings, openlayers, python-structlog, signond. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet: tests.reproducible-builds.org i386 build nodes have been setup by converting 2 of the 4 amd64 nodes to i386. (h01ger) Package reviews 92 reviews have been removed, 66 added and 31 updated in the previous week. New issues: timestamps_generated_by_xbean_spring, timestamps_generated_by_mangosdk_spiprocessor. Chris Lamb filed 7 FTBFS bugs. Misc. On March 20th, Chris Lamb gave a talk at FOSSASIA 2016 in Singapore. The very same day, but a few timezones apart, h01ger did a presentation at LibrePlanet 2016 in Cambridge, Massachusetts. Seven GSoC/Outreachy applications were made by potential interns to work on various aspects of the reproducible builds effort. On top of interacting with several applicants, prospective mentors gathered to review the applications.

7 April 2016

Dirk Eddelbuettel: RcppArmadillo 0.6.700.3.0

armadillo image A new Armadillo release 6.700.3 is out, and we uploaded RcppArmadillo 0.6.700.3.0 to CRAN and Debian. This followed the usual thorough reverse-dependecy checking of by now 216 packages using. Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. Changes in this release are as follows:
Changes in RcppArmadillo version 0.6.700.3.0 (2016-04-05)
  • Upgraded to Armadillo 6.700.3 (Catabolic Amalgamator Deluxe)
    • added logmat() for calcuating the matrix logarithm
    • added regspace() for generating vectors with regularly spaced elements
    • added logspace() for generating vectors with logarithmically spaced elements
    • added approx_equal() for determining approximate equality
    • added trapz() for numerical integration
    • expanded .save() and .load() with hdf5_binary_trans file type, to save/load data with columns transposed to rows
Courtesy of CRANberries, there is also a diffstat report for this release. As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

6 April 2016

Dirk Eddelbuettel: RcppAPT 0.0.2

A new version of RcppAPT -- our interface from R to the C++ library behind the awesome apt, apt-get, apt-cache, ... commands and their cache -- is now on CRAN. It adds three new commands to the package. Two are relatively simple: showSrc() and dumpPackage() displays information about a given package, similar to what apt-cache showsrc and apt-cache showpkg, respectively, would reveal. Lastly, the buildDepends() function retrieves all build-dependencies for a packages. This should come in handy for automated tested for which I have some plans. A bit more information is on the package page here as well as as the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

27 March 2016

Lunar: Reproducible builds: week 48 in Stretch cycle

What happened in the reproducible builds effort between March 20th and March 26th:

Toolchain fixes
  • Sebastian Ramacher uploaded breathe/4.2.0-1 which makes its output deterministic. Original patch by Chris Lamb, merged uptream.
  • Rafael Laboissiere uploaded octave/4.0.1-1 which allows packages to be built in place and avoid unreproducible builds due to temporary build directories appearing in the .oct files.
Daniel Kahn Gillmor worked on removing build path from build symbols submitting a patch adding -fdebug-prefix-map to clang to match GCC, another patch against gcc-5 to backport the removal of -fdebug-prefix-map from DW_AT_producer, and finally by proposing the addition of a normalizedebugpath to the reproducible feature set of dpkg-buildflags that would use -fdebug-prefix-map to replace the current directory with . using -fdebug-prefix-map. As succesful result of lobbying at LibrePlanet 2016, the --clamp-mtime option will be featured in the next Tar release. This option is likely to be used by dpkg-deb to implement deterministic mtimes for packaged files.

Packages fixed The following packages have become reproducible due to changes in their build dependencies: augeas, gmtkbabel, ktikz, octave-control, octave-general, octave-image, octave-ltfat, octave-miscellaneous, octave-mpi, octave-nurbs, octave-octcdf, octave-sockets, octave-strings, openlayers, python-structlog, signond. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet:
  • #818742 on milkytracker by Reiner Herrmann: sorts the list of source files.
  • #818752 on tcl8.4 by Reiner Herrmann: sort source files using C locale.
  • #818753 on tk8.6 by Reiner Herrmann: sort source files using C locale.
  • #818754 on tk8.5 by Reiner Herrmann: sort source files using C locale.
  • #818755 on tk8.4 by Reiner Herrmann: sort source files using C locale.
  • #818952 on marionnet by ceridwen: dummy out build date and uname to make build reproducible.
  • #819334 on avahi by Reiner Herrmann: ship upstream changelog instead of the one generated by gettextize (although duplicate of #804141 by Santiago Vila).

tests.reproducible-builds.org i386 build nodes have been setup by converting 2 of the 4 amd64 nodes to i386. (h01ger)

Package reviews 92 reviews have been removed, 66 added and 31 updated in the previous week. New issues: timestamps_generated_by_xbean_spring, timestamps_generated_by_mangosdk_spiprocessor. Chris Lamb filed 7 FTBFS bugs.

Misc. On March 20th, Chris Lamb gave a talk at FOSSASIA 2016 in Singapore. The very same day, but a few timezones apart, h01ger did a presentation at LibrePlanet 2016 in Cambridge, Massachusetts. Seven GSoC/Outreachy applications were made by potential interns to work on various aspects of the reproducible builds effort. On top of interacting with several applicants, prospective mentors gathered to review the applications. Huge thanks to Linda Naeun Lee for the new hackergotchi visible on Planet Debian.

26 March 2016

Dirk Eddelbuettel: Rcpp 0.12.4: And another one

The fourth update in the 0.12.* series of Rcpp has now arrived on the CRAN network for GNU R, and has just been pushed to Debian as well. This follows four days of idleness in the incoming/ directory: Word is that the tireless CRAN maintainers were traveling and only covering simpler packages. The 0.12.4 release follows the 0.12.0 release from late July, the 0.12.1 release in September, the 0.12.2 release in November, and the 0.12.3 release in January -- making it the eight release at the steady bi-montly release frequency. As before, this release is more of a maintenance release addressing a number of small bugs, nuisances or documentation issues without adding any major new features. Rcpp has become the most popular way of enhancing GNU R with C or C++ code. As of today, 615 packages on CRAN depend on Rcpp for making analytical code go faster and further. That is up by more than sixty packages from the last release in January! As during the last few releases, we have new first-time contributors. Kirill Mueller extended Nullable<> to const objects. James "coatless" Balamuta helped with a much-needed update to the Rcpp FAQ concerning recommendations for OS X installations. Colin Gillespie corrected another (small) vignette issue. Contributions from the rest of the gang as well as by-now regular contributors such as Nathan or Dan are detailed below.
Changes in Rcpp version 0.12.4 (2016-03-22)
  • Changes in Rcpp API:
    • New accessors as() and clone() were added to the Nullable class (Dan in PR #423 closing #421)
    • The Nullable<>::operator SEXP() and Nullable<>::get() now also work for const objects (Kirill Mueller in PR #417).
    • A subsetting error was fixed (Qiang via #432 closing #431).
  • Changes in Rcpp Sugar:
    • Added new Sugar function median() (Nathan in PR #425 closing #424)
    • Added new Sugar function cbind() (Nathan in PR #447 closing #407)
  • Changes in Rcpp Attributes:
    • A plugin for C++14 was added (Dan in PR #427)
  • Changes in Rcpp Documentation:
    • An entry was added to the Rcpp-FAQ vignette describing the required packages for vignette building (#422).
    • Use on OS X was further detailed (James Balamuta in #433 with further review by Bob Rudis).
    • An entry was added concerning the hard-code limit of arguments to some constructor and function (cf #435).
    • The Rcpp-FAQ vignette now contains a table of content.
    • Typos and indentation were corrected in the Rcpp Sugar vignette (#445 by Colin Gillespie).
Thanks to CRANberries, you can also look at a diff to the previous release. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads page, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

17 March 2016

Dirk Eddelbuettel: RcppArmadillo 0.6.600.4.0

armadillo image Conrad announced a new Armadillo release 6.600.4 yesterday. This followed some work we had done for thorough pre-release checking with reverse dependencies tests for the over 200 CRAN packages using it and resulted in one more squashed bug. This releasehas been folded into RcppArmadillo 0.6.600.4.0 which arrived on CRAN and Debian earlier today. Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. Changes in this release are as follows:
Changes in RcppArmadillo version 0.6.600.4.0 (2016-03-15)
  • Upgraded to Armadillo 6.600.4 (Catabolic Amalgamator)
    • expanded sum(), mean(), min(), max() to handle cubes
    • expanded Cube class to handle arbitrarily sized empty cubes (eg. 0x5x2)
    • added shift() for circular shifts of elements
    • added sqrtmat() for finding the square root of a matrix
    • fix for gmm_diag when using Mahalanobis distance
  • The configure script now reflects the full LAPACK fallback offered by R 3.3.0 or later (PR #81)
Courtesy of CRANberries, there is also a diffstat report for this release. As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

16 March 2016

Dirk Eddelbuettel: Rblpapi 0.3.3

A new release of Rblpapi is now on CRAN. It provides a direct interface between R and the Bloomberg Terminal via the C++ API provided by Bloomberg Labs (but note that a valid Bloomberg license and installation is required). This marks the fourth release since the package first appeared on CRAN last year. Thanks to multiple contributors, it packs a number of changes (including two new functions bsrch() and fieldInfo()), numerous internal as well as build improvements, additional documentation and last but not least use of unit testing (provided a Bloomberg connection is available, ie not at CRAN or on Travis CI). See the NEWS.Rd extract for more details:
Changes in Rblpapi version 0.3.3 (2016-03-14)
  • configure adds a missing method="libcurl" options (PR #109 by Martin Bel).
  • New function bsrch() adds basic search functionality (PR #111 by Morgan Williams fixing ticket #82).
  • The licensing status of the Rblpapi source package was clarified (PR #119).
  • The bds() help page now shows an example using an overrides argument (PR #121).
  • A new function fieldInfo was added, and field information is used in bdh and bdp (PRs #122, #123, #125, and #127).
  • The bdp function now checks the order (PR #126).
  • Unit testing was added (PRs #129 and #131).
  • Data retrieval is now smarter about the types returned from Bloomberg (PR #132 and #133; and #141 and #142).
  • The bdh and bds functions now support an option verbose (PRs #138).
Courtesy of CRANberries, there is also a diffstat report for the this release. As always, more detailed information is on the Rblpapi page. Questions, comments etc should go to the issue tickets system at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

15 March 2016

Dirk Eddelbuettel: x13binary 0.1.2

A new version of x13binary is now on CRAN. It updates the underlying X-13ARIMA-SEATS binary to build 1.1.26 which was released recently by the US Census Bureau. Our x13binary package takes the pain out of installing this, and provides it to the seasonal package upon which other packages such as gunsales or ggseas build their functionality. Things just work: Depend on x13binary and on all relevant OSs supported by R, you should have an X-13ARIMA-SEATS binary installed which will be called seamlessly by the higher-level packages. See this announcement blog post describing the initial 0.1.0 release; we had since followed up with a 0.1.1 release tightening up behaviour on two edge case OSs. Courtesy of CRANberries, there is also a diffstat report for the most recent release as well as the preceding bugfix release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

11 March 2016

Dirk Eddelbuettel: Rcpp now used by 600 CRAN packages

600 Rcpp packages Earlier this morning, Rcpp reached another milestone: 600 packages on CRAN now depend on it (as measured by Depends, Imports and LinkingTo declarations). The graph is on the left depicts the growth of Rcpp usage over time. Rcpp cleared 300 packages in November 2014. It passed 400 packages in June of last year (when I only tweeted about it) and then 500 packages in late October. The chart now extends to the very beginning as I manually compiled some more data from the CRANberries checked with crandb. Then next part uses manually saved entries, and the final and largest part of the data set was generated semi-automatically via a short script appending updates to a small file-based backend. A list of user package is kept on this page. Also displayed in the graph is the relative proportion of CRAN packages using Rcpp. The four per-cent hurdle was cleared just before useR! 2014 where I showed a similar graph (as two distinct graphs) in my invited talk. We passed five percent in December of 2014, six percent last July, seven percent just before Christmas and now stands at 7.31 percent, or a little less than one in thirteen R packages. 600 user packages is truly humbling, a staggering number and a big responsibility. We continue to try our best try to keep Rcpp as performant and reliable as it has been so that the next set of packages can rely on it---just like these 600 do. So with that a very big Thank You! to all users and contributors of Rcpp for help, suggestions, bug reports, documentation or, of course, code.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

8 March 2016

Dirk Eddelbuettel: gtrendsR 1.3.3

A very nice new update to the gtrendsR package by Philippe and myself is now avilable via CRAN. I had only blogged about the initial 1.3.0 release, and we have added a whole slew of new features and fixes. Philippe rewrote a lot of the parsing to make it more robust to different encodings, and to add other features. So in no particular order, we can now sub-group by regions more finely, withstand various misfeatures in returned data sets, generally do better on connections, and more --- and also allow for intra-day, daily and weekly queries! That last part is pretty fun. Here is the code I ran last Saturday to look at the query for Donald Drumpf, a name brought to us via a beautiful John Oliver episode worth watching which ran about nine days ago. So last Saturday, when we were still within the seven day window, I ran
library(gtrendsR)
dp <- gtrends("Donald Drumpf", res="7d")
plot(dp) + ggplot2::ggtitle("The Drumpf") + ggplot2::theme(legend.position="none")
which resulted in the following chart Donald Drump query which highlights another nice feature: the ggplot2 object created by the plotting function is returned, so we can locally modify and tune it. Here we set a title and suppress the default legend. As I had not blogged about the interim bug-fix releases 1.3.1 and 1.3.2, here is the set of NEWS entries for the last three releases:

gtrendsR 1.3.3
  • A ggplot2 object can now be returned for further customization. plot(gtrends("NHL")) + ggtitle("NHL trend") + theme(legend.position="none")
  • Support for hourly and daily data (#67). For example, it is now possible to have hourly data for the last seven days with gtrends("nhl", geo = "CA", res = "7d"). Use ?gtrends for more information about the time resolution supported by the package.
  • Support for categorties (#46). Ex.: gtrends("NHL", geo = "US", cat = "0-20") will search only in the sport category.
  • Some countries (ex: Hong Kong) were missing from the list (#69).
  • Various typos and documentation work.

gtrendsR 1.3.2
  • Added support for sub-countries (#25). Ex.: gtrends("NHL", geo = "CA-QC") will return trends data for Qu bec province in Canada. The list of supported sub-countries can be obtained via data(countries).
  • Data parsing should work for any data returned by Google Trends (i.e. countries independent).
  • Better support for queries using keywords in different languages (#50, #57). Ex.: gtrends(" ", geo = "TW")
  • Now able to specify up to five countries (#53) via gtrends("NHL", geo = c("CA", "US"))
  • Fixing issue #51 allowing UK-based queries via geo = "GB"

gtrendsR 1.3.1
  • Fixing issue #34 where connection verification was not done properly.
  • Now able to use more latin character in query. For example: gtrends("montr al").
  • Can now deal with data returned other than in English language.

Courtesy of CRANberries, there is also a diffstat report for the this release. As always, more detailed information is on the gtrendsR repo where questions, comments etc should go via the issue tickets system.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Edited 2016-03-08: Corrected code snipped and one grammar instance

7 March 2016

Dirk Eddelbuettel: Rblpapi 'Release Candidate' 0.3.2.5

We have a made numerous changes to Rblpapi since the previous release 0.3.2 in early December. You can see the commits, or look at the ChangeLog or NEWS.Rd to get an indea of the changes. We have new functions, improved internals, bug fixes and more. In order to facilitate more widespread testing, I have just placed source and (Windows) binaries in the ghrr drat repo from which you can install the new version simply via
drat:::add("ghrr")          # if you have drat installed
install.packages("Rblpapi") # uses the drat version b/c higher version number than release
Alternatively you can also do
drat:::add("ghrr")          # if you have drat installed
update.packages()           # refresh all packages against all repos
We would appreciate wider testing, and feedback / bug reports / ... via the issue tracker. PRs with unit test suggestions would also be most welcome -- we now use RUnit and run the tests if a file ~/.R/rblpapiOptions.R exists which sets the options() values for automatic connection (see help(blpConnect)) as well as the blpUnitTests=TRUE option. This is needed to 'opt-in' as standard test setups at [Travis])(https://travis-ci.org/) or CRAN will not have access to a Bloomberg terminal.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

5 March 2016

Dirk Eddelbuettel: gunsales 0.1.1

A minor update to our recently released gunsales package is now on the CRAN network. The changes are completely internal and deal with automagic discovery of the X-13ARIMA-SEATS program via our associated x13binary package. More precisely, we let Solaris and the Windows old-release builds fail more gracefully. Others, ie current Windows builds, OS X and Linux, are entirely unaffected in their functionality. For simplicity, all examples now have an \dontrun wrapper, and the vignette makes its code conditional on the platform it is running on. Courtesy of CRANberries, there is also a diffstat report for the most recent release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

2 March 2016

Dirk Eddelbuettel: RcppEigen 0.3.2.8.1

A good week after the update to Eigen 3.2.8 in our RcppEigen package 0.3.2.8.0, we have another (local) update with RcppEigen 0.3.2.8.1 which should fix another UBSAN issue. This new version in now on CRAN and in Debian. Big thanks once again to Yixuan Qiu for liasing with Eigen upstream, reporting the bug and preparing a PR with the fix. The NEWS file entry follows.
Changes in RcppEigen version 0.3.2.8.1 (2016-02-29)
  • Applied another upstream UBSAN fix (PR #30 by Yixuan)
Courtesy of CRANberries, there is also a diffstat report for the most recent release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

29 February 2016

Dirk Eddelbuettel: New CRAN package gunsales

This is based on joint work with Gregor Aisch and Josh Keller of the New York Times. A new package gunsales is now on the CRAN network for R. It is based the NYTimes/gunsales repository underlying the excellent New York Times visualizations, first published first in December 2015 and updated with more recent data since. The analysis takes public government data on gun sales from the National Instant Criminal Background Check System (NICS). The original data is scraped from the pdf, included in the package, and analysed in a cross-section and time-series manner. The standard US Census tool X-13ARIMA-SEATS is used to deseasonalize the timeseries at the national or state level. (Note that Buzzfeed also published data and (Python) code in another GitHub repo.) As an aside, it was the use of X-13ARIMA-SEATS here -- and its somewhat awkward and manual installation also seen in the initial versions of the code in the NYTimes/gunsales repo -- which lead to the recent work by Christoph Sax and myself. We now provide a new package x13binary on CRAN so that Christoph's excellent seasonal package can simply depend upon it and have a working binary provided and installed ready to use; see the recent blog post for more. The net result is that a package like this new gunsales project can simply depend upon seasonal and also be assurred that x13binary "just works". As Martha would say, "A Good Thing". Back to the gunsales project. Following the initial publication of the repository with the data and R code in a simple script, I felt compelled to reorganize it as a package. Packages for R, as we teach our students, colleagues, or anybody else who wants to listen are really the best way to bundle code, data, documentation (i.e. vignettes) and tests. All that exists now in the gunsales package. The package now has one main function, analysis(), which returns a single dataframe object. This dataframe object can then be fed to two plotting functions. The first, plot_gunsales(), will then recreate all the (base R) plots from the original code base. The second, ggplot_gunsales(), does the same but via ggplot2. This should give anybody the ability to look at the data, study the transformations done, form and maybe test new hypotheses and visualize in manner comparable to the original publication. As an amuse gueule, here are the key plots also shown in the main README.md at GitHub: Total Estimated Gun Sales Total Estimated Gun Sales, Seasonally Adjusted Total Estimated Gun Sales, Population-Growth Adjusted Handguns vs Longguns Six States DC We look forward to more remixes and analysis of this data. The plan of the GitHub repository is to keep the data set updated as new data points are published.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

24 February 2016

Dirk Eddelbuettel: RcppEigen 0.3.2.8.0

Another minor release of RcppEigen is on CRAN and getting into Debian. The main focus is an upgrade to the recent 3.2.7 release of Eigen which should address another UBSAN issue. And once again Yixuan Qiu did all the heavy lifting. The NEWS file entry follows.
Changes in RcppEigen version 0.3.2.8.0 (2016-02-23)
  • Updated to version 3.2.8 of Eigen (PR #29 by Yixuan)
Courtesy of CRANberries, there is also a diffstat report for the most recent release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

2 February 2016

Dirk Eddelbuettel: Like peanut butter and jelly: x13binary and seasonal

This post was written by Dirk Eddelbuettel and Christoph Sax and will be posted on both author's respective blogs. The seasonal package by Christoph Sax brings a very featureful and expressive interface for working with seasonal data to the R environment. It uses the standard tool of the trade: X-13ARIMA-SEATS. This powerful program is provided by the statisticians of the US Census Bureau based on their earlier work (named X-11 and X-12-ARIMA) as well as the TRAMO/SEATS program by the Bank of Spain. X-13ARIMA-SEATS is probably the best known tool for de-seasonalization of timeseries, and used by statistical offices around the world. Sadly, it also has a steep learning curve. One interacts with a basic command-line tool which users have to download, install and properly reference (by environment variables or related means). Each model specification has to be prepared in a special 'spec' file that uses its own, cumbersome syntax. As seasonal provides all the required functionality to use X-13ARIMA-SEATS from R --- see the very nice seasonal demo site --- it still required the user to manually deal with the X-13ARIMA-SEATS installation. So we decided to do something about this. A pair of GitHub repositories provide both the underlying binary in a per-operating system form (see x13prebuilt) as well as a ready-to- use R package (see x13binary) which uses the former to provide binaries for R. And the latter is now on CRAN as package x13binary ready to be used on Windows, OS-X or Linux. And the seasonal package (in version 1.2.0 -- now on CRAN -- or later) automatically makes use of it. Installing seasaonal and x13binary in R is now as easy as:

install.packages("seasonal")
which opens the door for effortless deployment of powerful deasonalization. By default, the principal function of the package employs a number of automated techniques that work well in most circumstances. For example, the following code produces a seasonal adjustment of the latest data of US retail sales (by the Census Bureau) downloaded from Quandl:
library(seasonal) 
library(Quandl)   ## not needed for seasonal but has some niceties for Quandl data
rs <- Quandl(code="USCENSUS/BI_MARTS_44000_SM", type="ts")/1e3
m1 <- seas(rs)
plot(m1, main = "Retail Trade: U.S. Total Sales", ylab = "USD (in Billions)")
This tests for log-transformation, performs an automated ARIMA model search, applies outlier detection, tests and adjusts for trading day and easter effects, and invokes the SEATS method to perform seasonal adjustment. And this is how the adjusted series looks like:
Of course, you can access all available options of X-13ARIMA-SEATS as well. Here is an example where we adjust the latest data for Chinese exports (as tallied by the US FED), taking into account the different effects of Chinese New Year before, during and after the holiday:
xp <- Quandl(code="FRED/VALEXPCNM052N", type="ts")/1e9
m2 <- seas(window(xp, start = 2000),
          xreg = cbind(genhol(cny, start = -7, end = -1, center = "calendar"), 
                       genhol(cny, start = 0, end = 7, center = "calendar"), 
                       genhol(cny, start = 8, end = 21, center = "calendar")
          ),
          regression.aictest = c("td", "user"),
          regression.usertype = "holiday")
plot(m2, main = "Goods, Value of Exports for China", ylab = "USD (in Billions)")
which generates the following chart demonstrating a recent flattening in export activity measured in USD.
We hope this simple examples illustrates both how powerful a tool X-13ARIMA-SEATS is, but also just how easy it is to use X-13ARIMA-SEATS from R now that we provide the x13binary package automating its installation.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Next.

Previous.