Search Results: "edd"

27 January 2017

Dirk Eddelbuettel: digest 0.6.12

A new release, now at version 0.6.12, of the digest package is now on CRAN and in Debian. The digest creates hash digests of arbitrary R objects (using the 'md5', 'sha-1', 'sha-256', 'crc32', 'xxhash' and 'murmurhash' algorithms) permitting easy comparison of R language objects. This release extends sha1 digest methods to even more types, thanks to another contribution by Thierry Onkelinx. CRANberries provides the usual summary of changes to the previous version. For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

24 January 2017

Shirish Agarwal: Budget and Economics 101

The Budget The story which I wanted to share is there are few friends (from Debian as well as elsewhere) who shared that they didn t get the whole demonetisation play or what the Government is/was trying to do. As budget is just round the corner (India will be presenting its yearly budget on 1st of February), thought it is prudent to share at least some basics, ideas and theories of what goals the Finance Minister would be looking at when presenting his budget. I would NOT talk of Inflation targeting or some such exotica as those topics would require their own blog-posts altogether. I would mainly be talking a bit about Taxation and in that Personal Income Tax. I would also not use words like Receivables and like which thought bit more accurate are not used in everyday language. Just like Private Companies and increasingly public utilities, The Government of the day has two-three different aims when it is presenting a budget a. The first is to give an update about how things went last year. Did all the incomes that were projected, did it happen or was there a short-fall ? If there was a short-fall what were the reasons for the shortfall. Similarly, did all the budgeted expenditure earmarked for the year was spent and were it spent under the heads they were supposed to ? If not what went wrong there ? There is usually a tussle between Planned and Unplanned expenditure and one of the hallmarks of good governance is that unplanned expenditure is kept at minimum, while planned expenditure and projects completion or/and assets coming on-line were within the estimated time-frames. So these updates are given to the Parliament and hence public at large. The second and the more interesting part are the plans for the immediate future, 1 year down-the-line. Based on the performance last year, a bit of crystal-ball gazing of external and internal conditions of the country, the Finance Minister along with her/his colleagues of Finance Ministry. Trivia There hasn t been a female finance minister till date in India. The Finance Ministry as a whole also holds consultations with most sections of the society before sharing/putting his Fiscal Policy (Planned Expenditure) for debate and passage in form of the Budget. While the budget itself is a technical exercise, it is also a Political exercise as both the budget and the finance bill (which contains the taxation proposals) need to be passed in Lok Sabha (Lower house). After passing scrutiny of Lok Sabha (Lower House having people s representatives directly elected) and Rajya Sabha (Upper House, indirectly elected), the taxation proposals becomes the law. It isn t that simple but for our understanding, keeping it simple. This Political model of governance with two houses is modeled under the British (Westminister) model since 1947. The Government, just like any other Organization gives a similar Profit and Loss Account and a Balance Sheet.
How A country's budget is made.

How A country s budget is made. A representational and simplified version of how things flow was made using Graphviz. Click on it to see image in detail.

I am a newbie to graphviz. The graph was made like this
graph Budget
subgraph tier1
node [color="limegreen",style="filled",group="tier1"]
Country_Budget

......
Country_Budget -- Profit_and_Loss_Account [type=s];
It might be possible to make the graph much better than it is currently . The Profit and Loss Account of the Government tells what Incomes it is projected to earn in the upcoming year and whatever Expenditures it hopes to do this year. The Income and Expenditure independently can be bifurcated into two, Revenue Income and Capital Income and Revenue Expenditure and Capital Expenditure.
Indian Railways EMU local train

Indian Railways EMU local train

The simplest example of such planned expenditure which comes to my mind is the Indian Railways Budget which is all planned expenditure. As can be seen even with ample funds Railways were able to spent only 50% of the total amount disbursed last year. Similarly income generation for Railways was far below the target. Examples of Revenue Income include taxes of all sorts, while Capital Income are rare, like divestment/stake sale of a company owned by the Government. These are usually one-off events. Examples of Capital Expenditure is when the Government makes a road, makes a bridge etc. Usually large expenditures come under Capital Expenditure while salaries to Government employees and routine expenditures are known as Revenue Expenditure. There was a statement by the present Government that the last 6-7 years the budgets has been more or less static as far as numbers are concerned. This hampers Government s ability to take up any new work. The Revenue income earned by the Government can again be bifurcated primarily into two Direct Taxes and Indirect Taxes.
INR 2000 Rupees

INR 2000 Rupees

Direct Taxes are those which the Government earns through Personal Income Tax and Corporate Tax. As only 1 percent of Indians pay Personal income tax, the rest Government tries to raise by Hence the Government of the day is in fix. It needs to have more money if it wants to invest into infrastructure, defence spending, social spending such as health and education and so on and so forth. It cannot Another point is that unlike China which is a Large State-backed Enterprises Export-led Economy which has its own problems, India s economy is much more consumption-based, hence any large tinkering upwards may possibly stall whatever little spending the middle-class does, similar to the stall in consumer durables which has been happening over the last few years. There are a couple of short-term solutions that the Government may do While both seem to be attractive ways, but both have their disadvantages also, both have costs associated for them. In the first one, like any other scheme, when any scheme is launched, it needs to be underwritten by GOI which means even if it s not a success they would have to service all and any obligations towards investors. Also they have to be careful how much they are borrowing as excessive borrowing for today could lead to a Greece-like meltdown situation, whether internal or external borrowers. With external borrowers they also usually like to have a guarantee that the Rupee will not slide beyond a point otherwise the Government will have to pay all and any losses but this is going beyond what I wanted to share. Printing excess money in the system could lead to loss in the value of the money itself as well as leading to inflationary pressures which leads to more problems for the poor and greater inequality between the classes among other things. So while the Government may use all of the above ways in varying degrees, the present Government had the idea that if we were to reduce black money or hidden economy (AFAIK no country can claim to completely eliminate it) we would be able to raise the finance we need without a major cost associated to it. For instance, I was reading that even in Canada, it is expected that 20% of black money/shadow economy works and that assessment is by their own taxation authorities. So While doing demonetization, it came out with an equivalent Black Money Declaration Scheme (IDS). The idea is simple, even if 1 percent of the population comes in the traditional tax net the Government of the day would be able to enhance budgets to various expenditure. Now while the idea is good in theory, implementation has been the Achilees heel. While the Government s expected something like 15% of the whole economy was black money or shadow money, almost 95% of the money in circulation came back in Banks during demonetization ( These are unofficial figures, Finance Ministry/RBI would be disclosing the real figures on 1st of February 2017 so we will know). It is suspected that 10% of money in Banks is black money. There are considerable costs to search analyze, prove in the court of law that it is so. There are and would be considerable costs to train new officers as existing Income Tax Officers are already burdened with Advance Tax being paid by Corporates and small business-man paying round the year (every 3 months), The existing Income Tax Officers already have their hands full. Also till Governments don t fix up realty sector/real-estate sector and other places where the black money/shadow economy may prevail. Hence all the training, salary, buildings where new Income Tax Officers could work, infrastructure, new buildings where suspect cases have to be tried and lawyers for those. As have shared a few times on this blog, India has almost 29 million court cases pending in the lower judiciary alone. Unless any such cases are not successfully tried within time by the Government, it would be a waste. Now whether the Government knew of these issues or not would probably be never known. Lastly, there is a voluntary part that the Government hopes, that they will by themselves join the mainstream tax-paying public. This might happen but any such happening will happen over years. People make their own choices. And unless there are not any stick and carrot approach to the Government s Policies people will tend to go back to their old ways. I would share an example from the demonetisation process which would help prove my point During demonetization, there was a great push towards doing digital transaction either via smartphones or greater usage of debit and credit cards etc. For the first 60 days till 31st December 2016, you could do digital transactions without paying any transaction fee. During that period, I used my Debit card to shop, to eat at restaurants or/and even small shops. But come 1st January 2017, the charges for digital transactions are anything between 1.5% to 3% of transactions. Naturally, I stopped using them and use them very sparingly where cash won t work. So at the end, while the Government made the whole demonetization drive to drive out shadow economy, terror financing etc. While terror financing has been hurt quite a bit, the same cannot be said of the shadow/black economy. It seems that the Government would need to close many more doors and windows before people join the mainstream. While Politically it was risky, socially it was also a bit risky move as it was uncertain how and where things will move. Venezuela tried the same thing and fell flat on its face. All said and done, if and when people become part of the tax-paying class/people, The most optimistic idea that the Government has that everybody will go cashless and it would be far easier to find out who s not paying taxes. As shared before, I don t think this will happen unless the charges for cashless is at 0.05% or something similar. Even IF people do join the mainstream, it is very much possible that the present Govt. will not enjoy fruits of this labour as fruits might come in 2018/19 or even later even if they do come. So whether the decision had the right affect or not, we may never come to know. Governments tend to tinker around with the figures as well. But I hope some idea of how things happen is known now.
Filed under: Miscellenous Tagged: #demonetization, #Government Budget, #graphviz, #Limitations, #Profit and Loss Account, #Taxation

22 January 2017

Shirish Agarwal: Debian contributions and World History

Beware, this would be slightly longish. Debian Contributions In the last couple of weeks, was lucky to put up a patch against debian-policy which had been bothering me for a long-long time. The problem statement is simple, man-pages historically were made by software engineers for software-engineers. The idea, probably then was you give the user some idea of what the software does and the rest the software engineer would garner from reading the source-code. But over period of time, the audience has changed. While there are still software engineers who use GNU/Linux for the technical excellence, the man-pages have not kept up with this new audience who perhaps are either not technically so sound that or they do not want to take the trouble to reading the source-code to understand how things flow. An example or examples in a man-page gives us (the lesser mortals) some insight as how the command works, how the logic flows. A good example of a man-page is the ufw man-page
EXAMPLES
Deny all access to port 53: ufw deny 53 Allow all access to tcp port 80: ufw allow 80/tcp Allow all access from RFC1918 networks to this host: ufw allow from 10.0.0.0/8
ufw allow from 172.16.0.0/12
ufw allow from 192.168.0.0/16 Deny access to udp port 514 from host 1.2.3.4: ufw deny proto udp from 1.2.3.4 to any port 514 Allow access to udp 1.2.3.4 port 5469 from 1.2.3.5 port 5469: ufw allow proto udp from 1.2.3.5 port 5469 to 1.2.3.4 port 5469
Now if we had man-pages like the above which give examples, then the user at least can try to accomplish whatever s/he is trying to do. I truly believe not having examples in a man-page kills 50% of your audience and people who could potentially use your tool. Personal wishlist The only thing (and this might be my failure) is we need a good way to search through a man-page. The only way I know is using / and try to give a pattern. Lots of times it fails because I, the user doesn t know the exact keyword which the documenter was using. What would be nice, great if we do have some sort of parser where I tell it, $this is what I m looking for and the parser tries the pattern + all its synonyms and whatever seems to be most relevant passage from the content, in this case a manpage it tells me. It would make my life a lot easier while at the same time force people to document more and more. I dunno if there has been any research or study of the relationship between good documentation and popularity of a program. I know there are lots of different tiny bits which make or break a program, one of which would definitely be documentation and in that a man-page IF it s a command-line tool. A query on Quora gives some indication https://www.quora.com/How-comprehensible-do-you-find-Unix-Linux-Man-pages although the low response rate tells its own story. there have been projects like man2html and man2pdf and others which try to make the content more accessible to people who are not used to the man-page interface but till you don t have Examples the other things can work only so far. Also if anybody talks about X project which claims to solve this problem they will have to fight manpages who have been around like forever. As can be seen in the patch, did some rookie mistakes as can be seen. I also filed a lintian bug at the same time. Hope the patch does get merged at some point in debian-policy and then a check introduced in lintian in some future release. I do agree with anarcat s assertion that it should be at the level of the manpage missing level. I am no coder but finding 14,000 binary packages without a manpage left me both shocked and surprised. I came to know about manpage-alert from the devscripts package to know which all binary packages that have been installed but not have man-pages. I hope to contribute a manpage or two if I across a package I m somewhat comfortable with. I have made a beginning of sorts by running manpage alert and putting the output in a .txt file which I would grep through manually and see if something interesting jumps at me. The learning garnered from putting the patch to the debian package resulted in another patch but this time for an upstream project altogether. As can be seen all are just baby-steps that even a non-coder can take. Another couple of bugs I filed which were fixed were of a sim called unknown-horizons . A 2D realtime strategy simulation. I had filed three bugs, two of which were fixed in 2 days, the 3rd I hope is also fixed soonish. Lastly, I spent most of the week-end poring over packages who have left files in /etc/bash_completion.d/ . I spent almost 4-5 odd hours as each package in question as well as entries found in /etc/bash-completion.d/$filename I had to find which package it belonged to first

[$] dpkg -S /etc/bash_completion.d/git-prompt
git: /etc/bash_completion.d/git-prompt
I know that dpkg-query also does the same
[$] dpkg-query -S /etc/bash_completion.d/git-prompt
git: /etc/bash_completion.d/git-prompt
But I am used to plain dpkg although do know that dpkg-query can do lot more intimate searching in various ways than dpkg can. Once the package name was established, first simulate the purge [$] sudo aptitude -s purge git

[sudo] password for shirish:
The following packages will be REMOVED:
git p
0 packages upgraded, 0 newly installed, 1 to remove and 14 not upgraded.
Need to get 0 B of archives. After unpacking 29.5 MB will be freed.
The following packages have unmet dependencies:
libgit-wrapper-perl : Depends: git but it is not going to be installed
git-extras : Depends: git (>= 1.7.0) but it is not going to be installed
bup : Depends: git but it is not going to be installed
git-remote-gcrypt : Depends: git but it is not going to be installed
git-svn : Depends: git (> 1:2.11.0) but it is not going to be installed
Depends: git ( 1:2.11.0) but it is not going to be installed
Depends: git (= 1:1.8.1) but it is not going to be installed
git-core : Depends: git (> 1:1.7.0.2) but it is not going to be installed
The following actions will resolve these dependencies:
Remove the following packages:
1) bup [0.29-2 (now, testing, unstable)]
2) fdroidserver [0.7.0-1 (now, testing, unstable)]
3) git-annex [6.20161012-1 (now, testing)]
4) git-core [1:2.11.0-2 (now, testing, unstable)]
5) git-extras [4.2.0-1 (now, testing, unstable)]
6) git-remote-gcrypt [1.0.1-1 (now, testing, unstable)]
7) git-repair [1.20151215-1 (now, unstable)]
8) git-svn [1:2.11.0-2 (now, testing, unstable)]
9) gitk [1:2.11.0-2 (now, testing, unstable)]
10) libgit-wrapper-perl [0.047-1 (now, testing, unstable)]
11) python3-git [2.1.0-1 (now, testing, unstable)]
12) svn2git [2.4.0-1 (now, testing, unstable)] Leave the following dependencies unresolved:
13) devscripts recommends libgit-wrapper-perl
14) dh-make-perl recommends git
15) fdroidserver recommends git
16) git-annex recommends git-remote-gcrypt (>= 0.20130908-6)
17) gplaycli recommends fdroidserver
18) python-rope recommends git-core
Accept this solution? [Y/n/q/?] q
Abandoning all efforts to resolve these dependencies.
Abort.
Then I made a note of all the packages being affected, saw purging all of them wouldn t call others (the Package dependency Hell), made the purge and then reinstalled anew. The reason I did this is that many a times during upgrade, either during update/upgrade sometimes the correct action doesn t happen. To take the git s example itself, there were two files git-extras and git-prompt which were in /etc/bash_completion.d/ both of which were showing their source as git. Purging git and installing git afresh removed git-extras file and git-prompt is the only one remaining. While blogging about the package, did try to grep through changelog.Debian.gz and changelog.gz in git

[shirish@debian] - [/usr/share/doc/git] - [10046]
[$] zless changelog.gz
and similarly

[shirish@debian] - [/usr/share/doc/git] - [10046]
[$] zless changelog.Debian.gz
But failed to find any mention of the now gone git-extras. Doing this with all the packages took considerable time as didn t want to deal with any potential fallout later on. For instance, ufw (uncomplicated firewall) also had an entry in /etc/bash_completion.d/, hence before purging ufw, took backup of all the rules I have made, did a successful simulation

[$] sudo aptitude -s purge ufw gufw


The following packages will be REMOVED:
gufw p ufw p
0 packages upgraded, 0 newly installed, 2 to remove and 14 not upgraded.
Need to get 0 B of archives. After unpacking 4,224 kB will be freed.
Note: Using 'Simulate' mode.
Do you want to continue? [Y/n/?] y
Would download/install/remove packages.
purged the packages, reinstalled it and then re-added all the rules. Doing it all for various sundry packages, had to do it manually as there is no one size fits all solution. A sensitive one was grub which still has an entry in /etc/bash_completion.d/grub. Doing it wrong could have resulted in a non-bootable situation. There are workarounds for that, but it would have taken quite a bit of time, energy, notes and bit of recall factor what I did the last time something like that happened. Doing it manually, being present meant I could do it rightly the first time. So, was it worth it It would be if the package maintainers do the needful and the remaining entries are moved out of /etc/bash_completion.d/ to /usr/share/bash-completions and some to my favourite /usr/share/zsh/vendor-completions/ for instance

[shirish@debian] - [/usr/share/zsh/vendor-completions] - [10064]
[$] ll -h _youtube-dl
-rw-r--r-- 1 root root 3.2K 2016-12-01 08:48 _youtube-dl But trying to get all or even major packages to use zsh-completions would be hard work and would take oodles of time and this concerns upstream stuff, also very much outside what I was sharing. World History Before, during and even after South-African experience, I was left wondering why India and South Africa, two countries who had similar histories at least the last couple of hundred years or more, the final result of Independence was so different for both the countries. It took me quite sometime to articulate that in a form of question , while the answers were interesting, from what little I know of India itself, if I were an Englishman I would never leave Hindustan . What the people answering failed to take into account was that in that era it was Hindustan or un-divided India.
Pre-partition map of India

Pre-partition map of India

This map can be found at https://commons.wikimedia.org/wiki/File:British_Indian_Empire_1909_Imperial_Gazetteer_of_India.jpg and is part of quite a few Indian articles. I would urge people to look at the map in-depth. Except for the Central India Agency and Central India Provinces, most of the other regions were quite comfortable weather-wise. Hence I can t help but feel the assertion that Britishers didn t like India (as to live here) slightly revolting. See the excerpt/take on Dale Kennedy s The Magic Mountains: Hill Stations and the British Raj. Berkeley: University of California Press, 1996. xv + 264 pp. . A look at the list of hill stations of divided India is enough to tell that there were lot of places which either were founded by the Britishers or they chose to live there. And this is not all, there are supposed to lot of beautiful places even in Pakistan, especially in North East Frontier, Swat for instance. While today it s infamous for Taliban and Islamic Terrorism, there was a time it was known for its beauty.
The second most difficult mountain in the world - K2, Pakistan

The second most difficult mountain in the world K2, Pakistan

Trivia After Everest, K2 is the smaller one although whatever I have read of people s accounts, most people who ascended all 14 8,000 metre peaks say K2 is technically more tougher than Everest and after Everest has the highest casualty rate. Also places like the disputed North half of Pakistan Occupied Kashmir, Gilgit Baltistan, Extreme northern Punjab of Pakistan , Northern half of Khyber-Pakhtunkhawa province and Northern Balochistan all of these places would have been more than conducive to the Britishers as it is near to the British climate (snow and pleasant weather all year round). It really is a pity that Pakistan chose to become a terrorist state where it could have become one of the more toured places of Asia. I really feel nauseous and sad at the multiple chances that Pakistan frittered away, it could have been something else.
Filed under: Miscellenous Tagged: #bash-completion, #British Raj, #contributions, #debian, #debian-policy, #Debian-QA, #Hill Stations, #India Independance Movement, #lintian, #obsolete-conffiles, #unknown-horizons, #weather, #World History, adequate, Pakistan, tourism

21 January 2017

Dirk Eddelbuettel: Updated Example Repo for RMarkdown and Metropolis/Mtheme

During useR! 2016, Nick Tierney had asked on Twitter about rmarkdown and metropolis and whether folks had used RMarkdown-driven LaTeX Beamer presentations. My firm hell yeah answer, based on having used mtheme outright or in local mods for quite some time (see my talks page), lead to this blog post of mine describing this GitHub repo I had quickly set up during breaks at useR! 2016. The corresponding blog post and the repo have some more details on how I do this, in particular about local packages (also with sources on GitHub) for the non-standard fonts I use. This week I got around to updating the repo / example a little by making the default colours (in my example) a little less awful, and a page on blocks and, most importantly, turning the example into the animated gif below:
And thanks to the beautiful tint package -- see its repo and CRAN package --- I now know how to create a template package. So if there is interest (and spare time), we could build a template package for RStudio too. With that, may I ask a personal favour of anybody still reading the post? Please do not hit my twitter handle with questions for support. All my code is an GitHub, and issue tickets there are much preferred. Larger projects like Rcpp also have their own mailing lists, and it is much better to use those. And if you like neither, maybe ask on StackOverflow. But please don't spam my Twitter handle. Thank you.

18 January 2017

Reproducible builds folks: Reproducible Builds: week 90 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday January 8 and Saturday January 14 2017: Upcoming Events Reproducible work in other projects Reproducible Builds have been mentioned in the FSF high-priority project list. The F-Droid Verification Server has been launched. It rebuilds apps from source that were built by f-droid.org and checks that the results match. Bernhard M. Wiedemann did some more work on reproducibility for openSUSE. Bootstrappable.org (unfortunately no HTTPS yet) was launched after the initial work was started at our recent summit in Berlin. This is another topic related to reproducible builds and both will be needed in order to perform "Diverse Double Compilation" in practice in the future. Toolchain development and fixes Ximin Luo researched data formats for SOURCE_PREFIX_MAP and explored different options for encoding a map data structure in a single environment variable. He also continued to talk with the rustc team on the topic. Daniel Shahaf filed #851225 ('udd: patches: index by DEP-3 "Forwarded" status') to make it easier to track our patches. Chris Lamb forwarded #849972 upstream to yard, a Ruby documentation generator. Upstream has fixed the issue as of release 0.9.6. Alexander Couzens (lynxis) has made mksquashfs reproducible and is looking for testers. It compiles on BSD systems such as FreeBSD, OpenBSD and NetBSD. Bugs filed Chris Lamb: Lucas Nussbaum: Nicola Corna: Reviews of unreproducible packages 13 package reviews have been added and 13 have been removed in this week, adding to our knowledge about identified issues. 1 issue type has been added: Weekly QA work During our reproducibility testing, the following FTBFS bugs have been detected and reported by: diffoscope development Bugs in diffoscope in the last year Many bugs were opened in diffoscope during the past few weeks, which probably is a good sign as it shows that diffoscope is much more widely used than a year ago. We have been working hard to squash many of them in time for Debian stable, though we will see how that goes in the end reproducible-website development tests.reproducible-builds.org Misc. This week's edition was written by Ximin Luo, Chris Lamb and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Dirk Eddelbuettel: RProtoBuf 0.4.8: Windows support for proto3

Issue ticket #20 demonstrated that we had not yet set up Windows for version 3 of Google Protocol Buffers ("Protobuf") -- while the other platforms support it. So I made the change, and there is release 0.4.8. RProtoBuf provides R bindings for the Google Protocol Buffers ("Protobuf") data encoding and serialization library used and released by Google, and deployed as a language and operating-system agnostic protocol by numerous projects. The NEWS file summarises the release as follows:

Changes in RProtoBuf version 0.4.8 (2017-01-17)
  • Windows builds now use the proto3 library as well (PR #21 fixing #20)

CRANberries also provides a diff to the previous release. The RProtoBuf page has an older package vignette, a 'quick' overview vignette, a unit test summary vignette, and the pre-print for the JSS paper. Questions, comments etc should go to the GitHub issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

15 January 2017

Dirk Eddelbuettel: Rcpp 0.12.9: Next round

Yesterday afternoon, the nineth update in the 0.12.* series of Rcpp made it to the CRAN network for GNU R. Windows binaries have by now been generated; and the package was updated in Debian too. This 0.12.9 release follows the 0.12.0 release from late July, the 0.12.1 release in September, the 0.12.2 release in November, the 0.12.3 release in January, the 0.12.4 release in March, the 0.12.5 release in May, the 0.12.6 release in July, the 0.12.7 release in September, and the 0.12.8 release in November --- making it the thirteenth release at the steady bi-montly release frequency. Rcpp has become the most popular way of enhancing GNU R with C or C++ code. As of today, 906 packages on CRAN depend on Rcpp for making analytical code go faster and further. That is up by sixthythree packages over the two months since the last release -- or about a package a day! Some of the changes in this release are smaller and detail-oriented. We did squash one annoying bug (stemming from the improved exception handling) in Rcpp::stop() that hit a few people. Nathan Russell added a sample() function (similar to the optional one in RcppArmadillo; this required a minor cleanup by for small number of other packages which used both namespaces 'opened'. Date and Datetime objects now have format() methods and << output support. We now have coverage reports via covr as well. Last but not least James "coatless" Balamuta was once more tireless on documentation and API consistency --- see below for more details.

Changes in Rcpp version 0.12.9 (2017-01-14)
  • Changes in Rcpp API:
    • The exception stack message is now correctly demangled on all compiler versions (Jim Hester in #598)
    • Date and Datetime object and vector now have format methods and operator<< support (#599).
    • The size operator in Matrix is explicitly referenced avoiding a g++-6 issues (#607 fixing #605).
    • The underlying date calculation code was updated (#621, #623).
    • Addressed improper diagonal fill for non-symmetric matrices (James Balamuta in #622 addressing #619)
  • Changes in Rcpp Sugar:
    • Added new Sugar function sample() (Nathan Russell in #610 and #616).
    • Added new Sugar function Arg() (James Balamuta in #626 addressing #625).
  • Changes in Rcpp unit tests
    • Added Environment::find unit tests and an Environment::get(Symbol) test (James Balamuta in #595 addressing issue #594).
    • Added diagonal matrix fill tests (James Balamuta in #622 addressing #619)
  • Changes in Rcpp Documentation:
    • Exposed pointers macros were included in the Rcpp Extending vignette (MathurinD; James Balamuta in #592 addressing #418).
    • The file Rcpp.bib move to directory bib which is guaranteed to be present (#631).
  • Changes in Rcpp build system
    • Travis CI now also calls covr for coverage analysis (Jim Hester in PR #591)

Thanks to CRANberries, you can also look at a diff to the previous release. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads page, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

14 January 2017

Russ Allbery: Review: Enchanters' End Game

Review: Enchanters' End Game, by David Eddings
Series: The Belgariad #5
Publisher: Del Rey
Copyright: December 1984
Printing: February 1990
ISBN: 0-345-33871-5
Format: Mass market
Pages: 372
And, finally, the conclusion towards which everything has been heading, and the events for which Castle of Wizardry was the preparation. (This is therefore obviously not the place to start with this series.) Does it live up to all the foreshadowing and provide a satisfactory conclusion? I'd say mostly. The theology is a bit thin, but Eddings does a solid job of bringing all the plot threads together and giving each of the large cast a moment to shine. Enchanters' End Game (I have always been weirdly annoyed by that clunky apostrophe) starts with more of Garion and Belgarath, and, similar to the end of Castle of Wizardry, this feels like them rolling on the random encounter table. There is a fairly important bit with Nadraks at the start, but the remaining detour to the north is a mostly unrelated bit of world-building. Before this re-read, I didn't remember how extensive the Nadrak parts of this story were; in retrospect, I realize a lot of what I was remembering is in the Mallorean instead. I'll therefore save my commentary on Nadrak gender roles for an eventual Mallorean re-read, since there's quite a lot to dig through and much of it is based on information not available here. After this section, though, the story leaves Garion, Belgarath, and Silk for nearly the entire book, returning to them only for the climax. Most of this book is about Ce'Nedra, the queens and kings of the west, and what they're doing while Garion and his small party are carrying the Ring into Mordor er, you know what I mean. And this long section is surprisingly good. We first get to see the various queens of the west doing extremely well managing the kingdoms while the kings are away (see my previous note about how Eddings does examine his stereotypes), albeit partly by mercilessly exploiting the sexism of their societies. The story then picks up with Ce'Nedra and company, including all of the rest of Garion's band, being their snarky and varied selves. There are some fairly satisfying set pieces, some battle tactics, some magical tactics, and a good bit of snarking and interplay between characters who feel like old friends by this point (mostly because of Eddings's simple, broad-strokes characterization). And Ce'Nedra is surprisingly good here. I would say that she's grown up after the events of the last book, but sadly she reverts to being awful in the aftermath. But for the main section of the book, partly because she's busy with other things, she's a reasonable character who experiences some actual consequences and some real remorse from one bad decision she makes. She's even admirable in how she handles events leading up to the climax of the book. Eddings does a good job showing every character in their best light, putting quite a lot of suspense (and some dramatic rescues) into this final volume, and providing a final battle that's moderately interesting. I'm not sure I entirely bought the theological ramifications of the conclusion (the bits with Polgara do not support thinking about too deeply), but the voice in Garion's head continues to be one of the better characters of the series. And Errand is a delight. After the climax, the aftermath sadly returns to Eddings's weird war between the sexes presentation of all gender relationships in this series, and it left me with a bit of a bad taste in my mouth. (There is absolutely no way that some of these relationships would survive in reality.) Eddings portrays nearly every woman as a manipulative schemer, sometimes for good and sometimes for evil, and there is just so much gender stereotyping throughout this book for both women and men. You can tell he's trying with the queens, but women are still only allowed to be successful at politics and war within a very specific frame. Even Polgara gets a bit of the gender stereotyping, although she remains mostly an exception (and one aspect of the ending is much better than it could have been). Ah well. One does not (or at least probably should not) read this series without being aware that it has some flaws. But it has a strange charm as well, mostly from its irreverence. The dry wise-cracking of these characters rings more true to me than the epic seriousness of a lot of fantasy. This is how people behave under stress, and this is how quirky people who know each other extremely well interact. It also keeps one turning the pages quite effectively. I stayed up for several late nights finishing it, and was never tempted to put it down and stop reading. This is not great literature, but it's still fun. It wouldn't sustain regular re-reading for me, but a re-read after twenty years or so was pretty much exactly the experience I was hoping for: an unchallenging, optimistic story with amusing characters and a guaranteed happy ending. There's a place for that. Followed, in a series sense, by the Mallorean, the first book of which is The Guardians of the West. But this is a strictly optional continuation; the Belgariad comes to a definite end here. Rating: 7 out of 10

11 January 2017

Dirk Eddelbuettel: R / Finance 2017 Call for Papers

Last week, Josh sent the call for papers to the R-SIG-Finance list making everyone aware that we will have our nineth annual R/Finance conference in Chicago in May. Please see the call for paper (at the link, below, or at the website) and consider submitting a paper. We are once again very excited about our conference, thrilled about upcoming keynotes and hope that many R / Finance users will not only join us in Chicago in May 2017 -- but also submit an exciting proposal. We also overhauled the website, so please see R/Finance. It should render well and fast on devices of all sizes: phones, tablets, desktops with browsers in different resolutions. The program and registration details still correspond to last year's conference and will be updated in due course. So read on below, and see you in Chicago in May!

Call for Papers R/Finance 2017: Applied Finance with R
May 19 and 20, 2017
University of Illinois at Chicago, IL, USA The ninth annual R/Finance conference for applied finance using R will be held on May 19 and 20, 2017 in Chicago, IL, USA at the University of Illinois at Chicago. The conference will cover topics including portfolio management, time series analysis, advanced risk tools, high-performance computing, market microstructure, and econometrics. All will be discussed within the context of using R as a primary tool for financial risk management, portfolio construction, and trading. Over the past eight years, R/Finance has included attendees from around the world. It has featured presentations from prominent academics and practitioners, and we anticipate another exciting line-up for 2017. We invite you to submit complete papers in pdf format for consideration. We will also consider one-page abstracts (in txt or pdf format) although more complete papers are preferred. We welcome submissions for both full talks and abbreviated "lightning talks." Both academic and practitioner proposals related to R are encouraged. All slides will be made publicly available at conference time. Presenters are strongly encouraged to provide working R code to accompany the slides. Data sets should also be made public for the purposes of reproducibility (though we realize this may be limited due to contracts with data vendors). Preference may be given to presenters who have released R packages. Financial assistance for travel and accommodation may be available to presenters, however requests must be made at the time of submission. Assistance will be granted at the discretion of the conference committee. Please submit proposals online at http://go.uic.edu/rfinsubmit. Submissions will be reviewed and accepted on a rolling basis with a final deadline of February 28, 2017. Submitters will be notified via email by March 31, 2017 of acceptance, presentation length, and financial assistance (if requested). Additional details will be announced via the conference website as they become available. Information on previous years' presenters and their presentations are also at the conference website. We will make a separate announcement when registration opens. For the program committee:
Gib Bassett, Peter Carl, Dirk Eddelbuettel, Brian Peterson,
Dale Rosenthal, Jeffrey Ryan, Joshua Ulrich

Dirk Eddelbuettel: nanotime 0.1.0: Now on Windows

Last month, we released nanotime, a package to work with nanosecond timestamps. See the initial release announcement for some background material and a few first examples. nanotime relies on the RcppCCTZ package for high(er) resolution time parsing and formatting: R itself stops a little short of a microsecond. And it uses the bit64 package for the actual arithmetic: time at this granularity is commonly represented at (integer) increments (at nanosecond resolution) relative to an offset, for which the standard epoch of Januar 1, 1970 is used. int64 types are a perfect match here, and bit64 gives us an integer64. Naysayers will point out some technical limitations with R's S3 classes, but it works pretty much as needed here. The one thing we did not have was Windows support. RcppCCTZ and the CCTZ library it uses need real C++11 support, and the g++-4.9 compiler used on Windows falls a little short lacking inter alia a suitable std::get_time() implementation. Enter Dan Dillon who ported this from LLVM's libc++ which lead to Sunday's RcppCCTZ 0.2.0 release. And now we have all our ducks in a row: everything works on Windows too. The next paragraph summarizes the changes for both this release as well as the initial one last month:

Changes in version 0.1.0 (2017-01-10)
  • Added Windows support thanks to expanded RcppCCTZ (closes #6)
  • Added "mocked up" demo with nanosecond delay networking analysis
  • Added 'fmt' and 'tz' options to output functions, expanded format.nanotime (closing #2 and #3)
  • Added data.frame support
  • Expanded tests

Changes in version 0.0.1 (2016-12-15)
  • Initial CRAN upload.
  • Package is functional and provides examples.

We also have a diff to the previous version thanks to CRANberries. More details and examples are at the nanotime page; code, issue tickets etc at the GitHub repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

9 January 2017

Shirish Agarwal: The Great Indian Digital Tamasha

Indian Railways This is an extension to last month s article/sharing where I had shared the changes that had transpired in the last 2-3 months. Now am in a position to share the kind of issues a user can go through in case he is looking for support from IRCTC to help him/her go cashless. If you a new user to use IRCTC services you wouldn t go through this trouble. For those who might have TL;DR issues it s about how hard it can become to get digital credentials fixed in IRCTC (Indian Railway Catering and Tourism Corporation) a. 2 months back Indian Prime Minister gave a call incentivizing people to use digital means to do any commercial activities. One of the big organizations which took/takes part is IRCTC which handles the responsibility for e-ticketing millions of Rail tickets for common people. In India, a massive percentage moves by train as it s cheaper than going by Air. A typical fare from say Pune Delhi (capital of India) by second class sleeper would be INR 645/- for a distance of roughly 1600 odd kms and these are monopoly rates, there are no private trains and I m not suggesting anything of that sort, just making sure that people know. An economy class ticket by Air for the same distance would be anywhere between INR 2500-3500/- for a 2 hour flight between different airlines. Last I checked there are around 8 mainstream airlines including flag-carrier Air India. About 30% of the population live on less than a dollar and a half a day which would come around INR 100/-. There was a comment some six months back on getting more people out of the poverty line. But as there are lots of manipulations in numbers for who and what denotes above poor and below poor in India and lot of it has to do with politics it s not something which would be easily fixable. There are lots to be said in that arena but this article is not an appropriate blog-post for that. All in all, it s only 3-5% of the population at the most who can travel via Air if situation demands and around 1-2% who might be frequent, business or leisure travellers. Now while I can thankfully afford an Air Ticket if the situation so demands, my mother gets motion sickness so while together we can only travel by train. b. With the above background, I had registered with IRCTC few years ago with another number (dual-SIM) I had purchased and was thinking that I would be using this long-term (seems to my first big mistake, hindsight 50:50) . This was somewhere in 2006/2007. c. Few months later I found that the other service provider wasn t giving good service or was not upto mark. I was using IDEA (the main mobile operator) throughout those times. d. As I didn t need the service that much, didn t think to inform them that I want to change to another service provider at that point in time (possibly the biggest mistake, hindsight 50:50) e. In July 2016 itself IRCTC cut service fees, f. This was shared as a NEW news item/policy decision at November-end 2016 . g. While I have done all that has been asked by irctc-care haven t still got the issues resolved  IRCTC s e-mail id care@irctc.co.in Now in detail This is my first e-mail sent to IRCTC in June 2016
Dear Customer care, I had applied and got username and password sometime back . The
number I had used to register with IRCTC was xxxxxxxxxx (BSNL mobile number not used anymore) . My mobile was lost and along with that the number was also lost. I had filed a complaint with the police and stopped that number as well. Now I have an another mobile number but have forgotten both the password and the security answer that I had given when I had registered . I do have all the conversations I had both with the ticketadmn@irctc.co.in as well as care@irctc.co.in if needed to prove my identity. The new number I want to tie it with is xxxxxxxxxx (IDEA number in-use for last 10 years) I see two options :- a. Tie the other number with my e-mail address b. Take out the e-mail address from the database so that I can fill in
as a new applicant. Looking forward to hear from you.
There was lot of back and forth with various individuals on IRCTC and after a lot of back and forth, this is the final e-mail I got from them somewhere in August 2016, he writes
Dear Customer, We request you to send mobile bill of your mobile number if it is post paid or if it is prepaid then contact to your service provider and they will give you valid proof of your mobile number or they will give you in written on company head letter so that we may update your mobile number to update so that you may reset your password through mobile OTP.
and Kindly inform you that you can update your profile by yourself also. 1.login on IRCTC website
2.after login successfully move courser on my profile tab.
3.then click on update profile
4.re-enter your password then you can update your profile
5.click on user-profile then email id.
6. click on update. Still you face any problem related to update profile please revert to us with the screen shots of error message which you will get at the time of update profile . Thanks & Regards Parivesh Patel
Executive, Customer Care
care@irctc.co.in
http://www.irctc.co.in
[#3730034]
IRCTC s response seemed responsible, valid and thought it would be a cake-walk as private providers are supposed to be much more efficient than public ones. The experience proved how wrong was I trust them with doing the right thing 1. First I tried the twitter handle to see how IDEA uses their twitter handle. 2. The idea customer care twitter handle was mild in its response. 3. After sometime I realized that the only way out of this quagmire would perhaps be to go to a brick-mortar shop and get it resolved face-to-face. I went twice or thrice but each time something or the other would happen. On the fourth and final time, I was able to get to the big Official shop only to be told they can t do anything about this and I would have to the appellate body to get the reply. The e-mail address which they shared (and I found it later) was wrong. I sent a somewhat longish e-mail sharing all the details and got bounce-backs. The correct e-mail address for the IDEA Maharashtra appellate body is appellette.mh@idea.aditybirla.com I searched online and after a bit of hit and miss finally got the relevant address. Then finally on 30th December, 2016 wrote a short email to the service provider as follows
Dear Sir,
I have been using prepaid mobile connection number xxxxxxx taken from IDEA for last 10 odd years. I want to register myself with IRCTC for online railway booking using
my IDEA mobile number. Earlier, I was having a BSNL connection which I discontinued 4 years back, For re-registering myself with IRCTC, I have to fulfill their latest
requirements as shown in the email below . It is requested that I please be issued a letter confirming my
credentials with your esteemed firm. I contacted your local office at corner of Law College Road and
Bhandarkar Road, Pune (reference number Q1 84786060793) who
refused to provide me any letter and have advised me to contact on the
above e-mail address, hence this request is being forwarded to you. Please do the needful at your earliest.
Few days later I got this short e-mail from them
Dear Customer, Greetings for the day! This is with reference to your email regarding services. Please accept our apologies for the inconvenience caused to you and delay in response. We regret to inform you that we are unable to provide demographic details from our end as provision for same is not available with us. Should you need any further assistance, please call our Customer Service help line number 9822012345 or email us at customercare@idea.adityabirla.com by mentioning ten digit Idea mobile number in subject line. Thanks & Regards, Javed Khan Customer Service Team IDEA Cellular Limited- Maharashtra & Goa Circle.
Now I was at almost my wit s end. Few days before, I had re-affirmed my e-mail address to IDEA . I went to the IDEA care site, registered with my credentials. While the https connection to the page is weak, but let s not dwell on that atm. I logged into the site, I went through all the drop-down menus and came across My Account > Raise a request link which I clicked on . This came to a page where I could raise requests for various things. One of the options given there was Bill Delivery. As I wasn t a postpaid user but a prepaid user didn t know if that would work or not I still clicked on it. It said it would take 4 days for that to happen. I absently filed it away as I was somewhat sure that nothing would happen from my previous experience with IDEA. But this time the IDEA support staff came through and shared a toll-free SMS number and message format that I could use to generate call details from the last 6 months. The toll-free number from IDEA is 12345 and the message format is EBILL MON (short-form for month so if it s January would be jan, so on and so forth). After gathering all the required credentials, sent my last mail to IRCTC about a week, 10 days back
Dear Mr. Parivesh Patel, I was out-of-town and couldn t do the needful so sorry for the delay.
Now that I m back in town, I have been able to put together my prepaid
bills of last 6 months which should make it easy to establish my
identity. As had shared before, I don t remember my old password and the old
mobile number (BSNL number) is no longer accessible so can t go
through that route. Please let me know the next steps in correcting the existing IRCTC
account (which I haven t operated ever) so I can start using it to
book my tickets. Look forward to hearing from you.
Haven t heard anything them from them, apart from a generated token number, each time you send a reply happens. This time it was #4763548 The whole sequence of events throws a lot of troubling questions a. Could IRCTC done a better job of articulating their need to me instead of the run-around I was given ? b. Shouldn t there be a time limit to accounts from which no transactions have been done ? I hadn t done a single transaction since registering. When cell service providers including BSNL takes number out after a year of not using a number, why is that account active for so long ? c. As that account didn t have OTP at registration, dunno if it s being used for illegal activities or something. Update This doesn t seem to be a unique thing at all. Just sampling some of the tweets by people at @IRCTC_LTD https://twitter.com/praveen4al/status/775614978258718721 https://twitter.com/vis_nov25/status/786062572390932480 https://twitter.com/ShubhamDevadiya/status/794241443950948352 https://twitter.com/rajeshhindustan/status/798028633759584256 https://twitter.com/ameetsangita/status/810081624343908352 https://twitter.com/grkisback/status/813733835213078528 https://twitter.com/gbalaji_/status/804230235625394177 https://twitter.com/chandhu_nr/status/800675627384721409 , all of this just goes to show how un-unique the situation really is.
Filed under: Miscellenous Tagged: #customer-service, #demonetization, #IDEA-aditya birla, #IRCTC, #web-services, rant

Dirk Eddelbuettel: RcppCCTZ 0.2.0

A new version, now at 0.2.0, of RcppCCTZ is now on CRAN. And it brings a significant change: windows builds! Thanks to Dan Dillon who dug deep enough into the libc++ sources from LLVM to port the std::get_time() function that is missing from the 4.* series of g++. And with Rtools being fixed at g++-4.9.3 this was missing for us here. Now we can parse dates for use by RcppCCTZ on Windows as well. That is important not only for RcppCCTZ but also particularly for the one package (so far) depending on it: nanotime. CCTZ is a C++ library for translating between absolute and civil times using the rules of a time zone. In fact, it is two libraries. One for dealing with civil time: human-readable dates and times, and one for converting between between absolute and civil times via time zones. It requires only a proper C++11 compiler and the standard IANA time zone data base which standard Unix, Linux, OS X, ... computers tend to have in /usr/share/zoneinfo -- and for which R on Windows ships its own copy we can use. RcppCCTZ connects this library to R by relying on Rcpp. The RcppCCTZ page has a few usage examples, as does the post announcing the previous release. The changes in this version are summarized here:

Changes in version 0.2.0 (2017-01-08)
  • Windows compilation was enabled by defining OFFSET() and ABBR() for MinGW (#10 partially addressing #9)
  • Windows use completed with backport of std::get_time from LLVM's libc++ to enable strptime semantics (Dan Dillon in #11 completing #9)
  • Timezone information on Windows is supplied via R's own copy of zoneinfo with TZDIR set (also #10)
  • The interface to formatDouble was cleaned up

We also have a diff to the previous version thanks to CRANberries. More details are at the RcppCCTZ page; code, issue tickets etc at the GitHub repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

8 January 2017

Keith Packard: embedded-arm-libc

Finding a Libc for tiny embedded ARM systems You'd think this problem would have been solved a long time ago. All I wanted was a C library to use in small embedded systems -- those with a few kB of flash and even fewer kB of RAM. Small system requirements A small embedded system has a different balance of needs: Available small C libraries I've looked at: Current AltOS C library We've been using pdclib for a couple of years. It was easy to get running, but it really doesn't match what we need. In particular, it uses a lot of stack space in the stdio implementation as there's an additional layer of abstraction that isn't necessary. In addition, pdclib doesn't include a math library, so I've had to 'borrow' code from other places where necessary. I've wanted to switch for a while, but there didn't seem to be a great alternative. What's wrong with newlib? The "obvious" embedded C library is newlib. Designed for embedded systems with a nice way to avoid needing a 'real' kernel underneath, newlib has a lot going for it. Most of the functions have a good balance between speed and size, and many of them even offer two implementations depending on what trade-off you need. Plus, the build system 'just works' on multi-lib targets like the family of cortex-m parts. The big problem with newlib is the stdio code. It absolutely requires dynamic memory allocation and the amount of code necessary for 'printf' is larger than the flash space on many of our devices. I was able to get a cortex-m3 application compiled in 41kB of code, and that used a smattering of string/memory functions and printf. How about avr libc? The Atmel world has it pretty good -- avr-libc is small and highly optimized for atmel's 8-bit avr processors. I've used this library with success in a number of projects, although nothing we've ever sold through Altus Metrum. In particular, the stdio implementation is quite nice -- a 'FILE' is effectively a struct containing pointers to putc/getc functions. The library does no buffering at all. And it's tiny -- the printf code lacks a lot of the fancy new stuff, which saves a pile of space. However, much of the places where performance is critical are written in assembly language, making it pretty darn hard to port to another processor. Mixing code together for fun and profit! Today, I decided to try an experiment to see what would happen if I used the avr-libc stdio bits within the newlib environment. There were only three functions written in assembly language, two of them were just stubs while the third was a simple ultoa function with a weird interface. With those coded up in C, I managed to get them wedged into newlib. Figuring out the newlib build system was the only real challenge; it's pretty awful having generated files in the repository and a mix of autoconf 2.64 and 2.68 version dependencies. The result is pretty usable though; my STM 32L discovery board demo application is only 14kB of flash while the original newlib stdio bits needed 42kB and that was still missing all of the 'syscalls', like read, write and sbrk. Here's gitweb pointing at the top of the tiny-stdio tree: gitweb And, of course you can check out the whole thing
git clone git://keithp.com/git/newlib
'master' remains a plain upstream tree, although I do have a fix on that branch. The new code is all on the tiny-stdio branch. I'll post a note on the newlib mailing list once I've managed to subscribe and see if there is interest in making this option available in the upstream newlib releases. If so, I'll see what might make sense for the Debian libnewlib-arm-none-eabi packages.

Dirk Eddelbuettel: Rcpp now used by 900 CRAN packages

800 Rcpp packages Today, Rcpp passed another milestone as 900 packages on CRAN now depend on it (as measured by Depends, Imports and LinkingTo declarations). The graph is on the left depicts the growth of Rcpp usage over time. The easiest way to compute this is to use the reverse_dependencies_with_maintainers() function from a helper scripts file on CRAN. This still gets one or two false positives of packages declaring a dependency but not actually containing C++ code and the like. There is also a helper function revdep() in the devtools package but it includes Suggests: which does not firmly imply usage, and hence inflates the count. I have always opted for a tighter count with corrections. Rcpp cleared 300 packages in November 2014. It passed 400 packages in June 2015 (when I only tweeted about it), 500 packages in late October 2015, 600 packages last March, 700 packages last July and 800 packages last October. The chart extends to the very beginning via manually compiled data from CRANberries and checked with crandb. The next part uses manually saved entries. The core (and by far largest) part of the data set was generated semi-automatically via a short script appending updates to a small file-based backend. A list of packages using Rcpp is kept on this page. Also displayed in the graph is the relative proportion of CRAN packages using Rcpp. The four per-cent hurdle was cleared just before useR! 2014 where I showed a similar graph (as two distinct graphs) in my invited talk. We passed five percent in December of 2014, six percent July of last year, seven percent just before Christmas eight percent this summer, and nine percent mid-December. 900 user packages is a really large number. This puts more than some responsibility on us in the Rcpp team as we continue to keep Rcpp as performant and reliable as it has been. At the rate things are going, the big 1000 may be hit some time in April. And with that a very big Thank You! to all users and contributors of Rcpp for help, suggestions, bug reports, documentation or, of course, code.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

6 January 2017

Jonathan McDowell: 2016 in 50 Words

Idea via Roger. Roughly chronological order. Some things were obvious inclusions but it was interesting to go back and look at the year to get to the full 50 words. Speaking at BelFOSS. Earthlings birthday. ATtiny hacking. Speaking at ISCTSJ. Dublin Anomaly. Co-habiting. DebConf. Peak Lion. Laura s wedding. Christmas + picnic. Engagement. Car accident. Car write off. Tennent s Vital. Dissertation. OMGWTFBBQ. BSides. New job. Rachel s wedding. Digital Privacy talk. Graduation. All The Christmas Dinners. IMDB Top 250. Shay leaving drinks. (This also serves as a test to see if I ve correctly updated Planet Debian to use https and my new Hackergotchi that at least looks a bit more like I currently do.)

Dirk Eddelbuettel: RcppTOML 0.1.0

Big news: RcppTOML now works on Windows too! This package had an uneventful 2016 without a single update. Release 0.0.5 had come out in late 2015 and we had no bugs or issues to fix. We use the package daily in production: a key part of our parameterisation is in TOML files In the summer, I took one brief stab at building on Windows now that R sports itself a proper C++11 compiler on Windows too. I got stuck over the not-uncommon problem of incomplete POSIX and/or C++11 support with MinGW and g++-4.9. And sadly ... I appears I wasn't quite awake enough to realize that the missing functionality was right there exposed by Rcpp! Having updated that date / datetime functionality very recently, I was in a better position to realize this when Devin Pastoor asked two days ago. I was able to make a quick suggestion which he tested, which I then refined ... here we are: RcppTOML on Windows too! (For the impatient: CRAN has reported that it has built the Windows binaries, they should hit mirrors such as this CRAN package for RcppTOML shortly.) So what is this TOML thing, you ask? A file format, very suitable for configurations, meant to be edited by humans but read by computers. It emphasizes strong readability for humans while at the same time supporting strong typing as well as immediate and clear error reports. On small typos you get parse errors, rather than silently corrupted garbage. Much preferable to any and all of XML, JSON or YAML -- though sadly these may be too ubiquitous now. But TOML is making good inroads with newer and more flexible projects. The Hugo static blog compiler is one example; the Cargo system of Crates (aka "packages") for the Rust language is another example. The new release updates the included cpptoml template header by Chase Geigle, brings the aforementioned Windows support and updates the Travis configuration. We also added a NEWS file for the first time so here are all changes so far:

Changes in version 0.1.0 (2017-01-05)
  • Added Windows support by relying on Rcpp::mktime00() (#6 and #8 closing #5 and #3)
  • Synchronized with cpptoml upstream (#9)
  • Updated Travis CI support via newer run.sh

Changes in version 0.0.5 (2015-12-19)
  • Synchronized with cpptoml upstream (#4)
  • Improved and extended examples

Changes in version 0.0.4 (2015-07-16)
  • Minor update of upstream cpptoml.h
  • More explicit call of utils::str()
  • Properly cope with empty lists (#2)

Changes in version 0.0.3 (2015-04-27)
  • First CRAN release after four weeks of initial development

Courtesy of CRANberries, there is a diffstat report for this release. More information and examples are on the RcppTOML page. Issues and bugreports should go to the GitHub issue tracker.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

4 January 2017

Dirk Eddelbuettel: digest 0.6.11

A new minor release with version number 0.6.11 of the digest package is now on CRAN and in Debian. It is mostly a maintenance release. Sometime last spring we were asked to consider changing the license GPL-2 to GPL (>= 2). Having gotten agreement of all copyright holders, this could finally happen. But it so happens that the last yay just after the last release, so it took another cycle. In other changes, I also made the makeRaw function fully generic and added documentation. The pull request by Jim Hester to add covr support was finally folded in, leading as always to some gaming and improvement of the coverage metrics. A polite request by Radford Neal to support a nosharing option in base::serialize was also honoured; this should help for use with his interesting pqR variant of R. CRANberries provides the usual summary of changes to the previous version. For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

3 January 2017

Russ Allbery: Review: Castle of Wizardry

Review: Castle of Wizardry, by David Eddings
Series: The Belgariad #4
Publisher: Del Rey
Copyright: May 1984
Printing: September 1991
ISBN: 0-345-33570-8
Format: Mass market
Pages: 373
Castle of Wizardry is the fourth book of the Belgariad and very much the middle of the story. Despite coming after an intermediate climax, this isn't the sort of series you can start in the middle. The problem with intermediate climaxes in a long series is that the next book can be a bit of a let-down as the characters do the necessary regrouping and reorienting and determine next steps. I think that hurts Castle of Wizardry quite a lot. The best bits are at the beginning, as the party escapes the consequences of Magician's Gambit, collects one more party member, shows us a lot more of Errand (who is always delightful), and confounds Relg's life and world view considerably. (Although there's a good bit of authorial fiat in the last.) This builds into a major story event, which would normally help avoid the let-down after the climax, but it's the major story event that is so frequently and obviously foreshadowed that you'd have to be as dumb as, well, Garion to not know what's coming. That gives a certain "yes, yes, we know already" tone to proceedings that robs it of its ability to rebuild tension. That said, the appeal of this series continues to be in the small details. While the first major event of this book goes pretty much as expected (including Ce'Nedra's reaction, which is just as irritating as you might be expecting), my favorite part was the endless, bubbly enthusiasm of the incredibly powerful artifact that features heavily. Usually epic fantasy will treat such world-breaking objects with seriousness and awe, as treasures to be admired and sacred (or terrifying) great works. See, for instance, the ur-example of Tolkien's rings, both the One Ring and the elven rings of power. Eddings manages a mix of awe and bemusement that doesn't undermine their power but that adds a delightful human element. This series pulls off treating a powerful magical artifact like an over-enthusiastic puppy without making it feel any less dangerous. It's a very neat, and I think underappreciated, trick to pull off. Another part of this book I liked, if a more stock one, is Garion's reactions after the big story event. This isn't the first book to portray basic decency and thoughtfulness as a major feature in people from humble backgrounds elevated to great power, but I always enjoy seeing that. Garion stops whining (mostly) and starts acting like a decent, level-headed person who doesn't assume he has the right to arrange other people's lives, and is rewarded for it. Real life is often not that fair or ethical, but that's why one reads wish-fulfillment fantasy like this: for a world in which being a good person is rewarded. However, Eddings does have some structural issues here. The narrative arc of this book, as a stand-alone entity, is odd. Its most dramatic event is in the middle, and then has a long traveling section that's, by comparison, much less exciting. The events of that section feel more like random encounters than a coherent part of the story, and are preceded by the most utterly ridiculous temper tantrums. I think the tantrums were meant to be pure humor, but my reaction was primarily eye-rolling. I have a hard time reconciling a screaming fit and breaking furniture with the long life experience and thoughtful planning of the character in question. And then there is the Ce'Nedra section that closes this book, and Ce'Nedra in this book more generally. To be fair, Castle of Wizardry is clearly intended to be Ce'Nedra's moment to grow as a person and stop being a childish brat. This does happen somewhat, and there are moments in the last section of the book where she does admirable things. But I couldn't quite believe in the mechanism, and it doesn't help that it's one of the most ham-handed bits of pre-ordained success in a book that has a tendency towards them. That undermines the real attempts Eddings makes to ground that success in Ce'Nedra's actual skills. Also undermining this is that those skills are manipulating people shamelessly, which Eddings seems to think is charming and attractive and I... don't. But the real problem is that I flatly disbelieve in Ce'Nedra as a character, or, given the apparent existence of such a creature, the level of tolerance that other characters show her. If I'd been Polgara, within fifteen minutes of meeting her I would have been seriously debating whether the destruction of the world might be a small price to pay for the satisfaction of dumping her down the nearest well. And not only is she awful by herself but Garion also descends to the same level whenever he's around her, until both of them are behaving like blithering idiots. I suspect part of my issue is that, to the extent that she is realistic at all, Ce'Nedra is the sort of intensely high-drama person who I have some amount of life experience with, and that life experience says "do not let this person anywhere near your life." Red flags all over everything. Garion needs to nope the hell out, because this will not end well. (Except, of course, it will, because it's that sort of series and the power of the author is strong.) I want female characters with real agency in my fantasy, and I want a female protagonist who is doing things of equal importance as the male protagonist (not that Eddings attempts to go that far). But Ce'Nedra reads like a fictional character written by someone who had never met a woman, but has extensively studied female supporting characters in books about junior-high social cliques and then tried to reconcile that research with the stereotype of women as manipulative seductresses. Yes, this series is full of stereotypes and characters painted in broad strokes, but Ce'Nedra is several tiers below every other supporting character in the book in both believability and in my desire to read about her. It's not that Eddings doesn't know how to write women at all. Polgara still falls into a few stereotyped categories, but she's sensible, opinionated, and has clear agency throughout the story. Taiba is delightful, if minor here. Poledra is absolutely wonderful whenever she appears. Some of the queens are obviously practical and sensible. And this book features a surprisingly good resolution to the subplot around Barak's wife, although the mechanism is a bit eye-rollingly cliched. Ce'Nedra's character is unusual for the series and almost certainly a deliberate authorial choice, and this book is supposed to be her coming of age. But I am baffled by that choice, and there's very little about it that I enjoyed reading. One more minor complaint: Silk gets a "tragic secret" in this book, and I really wish he hadn't. More time with Silk is always a feature, and I still love the character, but his oddities were already adequately explained by both his innate character and his way of dealing with a particularly awkward court situation. (One that ties into Eddings's habit of using some bad relationship stereotypes, but that's a rant for another day.) I think this additional tragic secret was gratuitous and really unnecessary, not to mention weirdly implausible and oddly cruel towards the other character involved. I was hoping that Magician's Gambit had turned a corner for the series, but Castle of Wizardry, despite having some neat moments, has some serious flaws. One more book to go, in which we learn that some of the eastern races have redeeming qualities! Followed by Enchanter's End Game. Rating: 6 out of 10

2 January 2017

Shirish Agarwal: India Tourism, E-Visa and Hong Kong

A Safe and Happy New Year to all. While Debconf India is still a pipe-dream as of now, did see that India has been gradually doing it easier for tourists and casual business visitors to come visit India. This I take as very positive development for India itself. The 1st condition is itself good for anybody visiting India
Eligibility International Travellers whose sole objective of visiting India is recreation , sight-seeing , casual visit to meet friends or relatives, short duration medical treatment or casual business visit.
https://indianvisaonline.gov.in/visa/tvoa.html That this facility is being given to 130 odd countries is better still
Albania, Andorra, Anguilla, Antigua & Barbuda, Argentina, Armenia, Aruba, Australia, Austria, Bahamas, Barbados, Belgium, Belize, Bolivia, Bosnia & Herzegovina, Botswana, Brazil, Brunei, Bulgaria, Cambodia, Canada, Cape Verde, Cayman Island, Chile, China, China- SAR Hong-Kong, China- SAR Macau, Colombia, Comoros, Cook Islands, Costa Rica, Cote d lvoire, Croatia, Cuba, Czech Republic, Denmark, Djibouti, Dominica, Dominican Republic, East Timor, Ecuador, El Salvador, Eritrea, Estonia, Fiji, Finland, France, Gabon, Gambia, Georgia, Germany, Ghana, Greece, Grenada, Guatemala, Guinea, Guyana, Haiti, Honduras, Hungary, Iceland, Indonesia, Ireland, Israel, Jamaica, Japan, Jordan, Kenya, Kiribati, Laos, Latvia, Lesotho, Liberia, Liechtenstein, Lithuania, Luxembourg, Madagascar, Malawi, Malaysia, Malta, Marshall Islands, Mauritius, Mexico, Micronesia, Moldova, Monaco, Mongolia, Montenegro, Montserrat, Mozambique, Myanmar, Namibia, Nauru, Netherlands, New Zealand, Nicaragua, Niue Island, Norway, Oman, Palau, Palestine, Panama, Papua New Guinea, Paraguay, Peru, Philippines, Poland, Portugal, Republic of Korea, Republic of Macedonia, Romania, Russia, Saint Christopher and Nevis, Saint Lucia, Saint Vincent & the Grenadines, Samoa, San Marino, Senegal, Serbia, Seychelles, Singapore, Slovakia, Slovenia, Solomon Islands, South Africa, Spain, Sri Lanka, Suriname, Swaziland, Sweden, Switzerland, Taiwan, Tajikistan, Tanzania, Thailand, Tonga, Trinidad & Tobago, Turks & Caicos Island, Tuvalu, UAE, Ukraine, United Kingdom, Uruguay, USA, Vanuatu, Vatican City-Holy See, Venezuela, Vietnam, Zambia and Zimbabwe.
This should make it somewhat easier for any Indian organizer as well as any participants from any of the member countries shared. There is possibility that this list would even get longer, provided we are able to scale our airports and all and any necessary infrastructure that would be needed for International Visitors to have a good experience. What has been particularly interesting is to know which ports of call are being used by International Visitors as well as overall growth rate
The Percentage share of Foreign Tourist Arrivals (FTAs) in India during November, 2016 among the top 15 source countries was highest from USA (15.53%) followed by UK (11.21%), Bangladesh (10.72%), Canada (4.66%), Russian Fed (4.53%), Australia (4.04%), Malaysia (3.65%), Germany (3.53%), China (3.14%), France (2.88%), Sri Lanka (2.49%), Japan (2.49%), Singapore (2.16%), Nepal (1.46%) and Thailand (1.37%).
And port of call
The Percentage share of Foreign Tourist Arrivals (FTAs) in India during November 2016 among the top 15 ports was highest at Delhi Airport (32.71%) followed by Mumbai Airport (18.51%), Chennai Airport (6.83%), Bengaluru Airport (5.89%), Haridaspur Land check post (5.87%), Goa Airport (5.63%), Kolkata Airport (3.90%), Cochin Airport (3.29%), Hyderabad Airport (3.14%), Ahmadabad Airport (2.76%), Trivandrum Airport (1.54%), Trichy Airport (1.53%), Gede Rail (1.16%), Amritsar Airport (1.15%), and Ghojadanga land check post (0.82%) .
The Ghojadanga land check post seems to be between West Bengal, India and Bangladesh. Gede Railway Station is also in West Bengal as well. So all and any overlanders could take any of those ways.Even Hardispur Land Check post comes in the Bengal-Bangladesh border only. In the airports, Delhi Airport seems to be attracting lot more business than the Mumbai Airport. Part of the reason I *think* is the direct link of Delhi Airport to NDLS via the Delhi Airport Express Line . The same when it will happen in Mumbai should be a game-changer for city too. Now if you are wondering why I have been suddenly talking about visas and airports in India, it came because Hong Kong is going to Withdraw Visa Free Entry Facility For Indians. Although, as rightly pointed out in the article doesn t make sense from economic POV and seems to be somewhat politically motivated. Not that I or anybody else can do anything about that. Seeing that, I thought it was a good opportunity to see how good/Bad our Government is and it seems to be on the right path. Although the hawks (Intelligence and Counter-Terrorist Agencies) will probably become a bit more paranoid , their work becomes tougher.
Filed under: Miscellenous Tagged: #Airport Metro Line 3, #CSIA, #Incredible India, #India, #International Tourism

31 December 2016

Russ Allbery: Review: Magician's Gambit

Review: Magician's Gambit, by David Eddings
Series: The Belgariad #3
Publisher: Del Rey
Copyright: June 1983
Printing: February 1990
ISBN: 0-345-33545-7
Format: Mass market
Pages: 305
Magician's Gambit is the third book of the Belgariad, and although it's the best, you probably can't start here. Too much setup has already happened by this point of the story. In retrospect, it's my memories of this book that led me to hang on to this series all these years and pick it up for a re-read. Garion has finally gotten past his whining (mostly), the party puts Ce'Nedra in a hole and leaves her there for a bunch of the book, and the characters finally do something concrete (and significant) in advancement of the plot. Eddings does a good job of avoiding the series problem of saving up all the climactic moments for the end of the story and instead adds a very good middle climax to the story that wraps up the main plot line of the first three books. I think this is also the book that best displays the interestingly quirky ways that Eddings approaches the epic fantasy genre. (Now to figure out how to talk about them without spoiling the story....) One of my favorite bits in this series is the dry voice in the back of Garion's head. I don't want to say too much about it, since discovering the nature of that voice slowly over the series is part of the fun. But this is the book where most of that mystery is revealed, and it's a rather different spin on the wise advisor trope. Garion has more than the usual number of mentor figures, and like many books of this type they function partly as plot devices to force the story down the correct path. One of the things that makes plot devices irritating is that they feel forced on the story from outside. Eddings here pulls a trick to embed that device inside the story, so it feels less like the hand of the author and more like an aspect of the mythology. I'm not sure it would work for everyone, and the execution isn't that sophisticated, but it's entertained me every time I read this series. It helps that the voice is an interesting character in its own right, and one of the few characters in the book who takes Garion seriously and explains things to him. By comparison, Belgarath and Polgara are still annoyingly high-handed and uncommunicative (particularly Polgara). Another highlight of this book is a lovely interlude in Belgarath's home, where we meet some of his colleagues (Beldin is a delight) and Garion finally starts experimenting with his own power. I mostly read this series for the supporting characters, not for Garion, who spends far too much time whining and most of the rest of the time being just neutrally there. But in this book Eddings finally gives him some moments of real awe and discovery, using one of my favorite approaches to the powerful chosen one character: making normally-hard things easy but normally-easy things unintuitive because they think about magic differently than anyone else. Most of the moments when Garion does something that shouldn't be possible are highlights. We also get nearly the final characters for Garion's band, and they're good ones. Relg is mostly amusing in some of his later reactions to specific characters and is a bit tedious in this book, but Errand is a unique idea and one of my favorite characters of the series. He fits well with the tendency of this series to not take itself, or power, or mythology, too seriously. Eddings avoids the trap of making everything Significant and Fraught; characters snark and do things on a whim, power has funny side effects, and the most powerful object in the story is treated in a way that has one both laughing and flinching at the same time. (It's a nice twist on and homage to some of the trappings around the One Ring, while making it far more light-hearted.) There are a few things I could complain about (the gender stereotyping between Garion and Ce'Nedra is so strong, for instance), but they're more minor here than in previous books. I enjoy reading about characters doing things they're very good at, and there's quite a lot of that here. (That said, I do feel like I should mention somewhere that one of the characters is very good at killing Murgos, who are supposedly human. I suppose it wouldn't be epic fantasy without a bit of casual genocide, and to be fair the Murgos in question are mostly trying to track down and kill the party, but the way that's played partly for laughs gets distasteful if one thinks about it too much.) This series is at its best when it balances dramatic prophecy with irreverent poking, gives the protagonists moments of odd and very human joy (Garion's colt!), and celebrates the unlikely and unexpected collection of people (or fable characters) who are drawn together by the story. I think this book hits those notes well, and although the flaws of the previous books are still there, they're muted and don't get as much screen time. Instead, we get some fun foreshadowing, a sense of power and growth, a few moments of delight and awe, and a lot of the sort of exasperated wise-cracking common to experienced people who trust each other and are working together on something hard. Magician's Gambit, and this whole series, will not be to everyone's taste, but it's the first book of the series I can comfortably recommend if you like this sort of thing. Followed by Castle of Wizardry. Rating: 8 out of 10

Next.

Previous.