Search Results: "edd"

21 August 2016

Dirk Eddelbuettel: RcppEigen 0.3.2.9.0

A new upstream release 3.2.9 of Eigen is now reflected in a new RcppEigen release 0.3.2.9.0 which got onto CRAN late yesterday and is now going into Debian. Once again, Yixuan Qiu did the heavy lifting of merging upstream (and two local twists we need to keep around). Another change is by James "coatless" Balamuta who added a row exporter. The NEWS file entry follows.
Changes in RcppEigen version 0.3.2.9.0 (2016-08-20)
  • Updated to version 3.2.9 of Eigen (PR #37 by Yixuan closing #36 from Bob Carpenter of the Stan team)
  • An exporter for RowVectorX was added (thanks to PR #32 by James Balamuta)
Courtesy of CRANberries, there is also a diffstat report for the most recent release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

19 August 2016

Dirk Eddelbuettel: RQuantLib 0.4.3: Lots of new Fixed Income functions

A release of RQuantLib is now on CRAN and in Debian. It contains a lot of new code contributed by Terry Leitch over a number of pull requests. See below for full details but the changes focus on Fixed Income and Fixed Income Derivatives, and cover swap, discount curves, swaptions and more. In the blog post for the previous release 0.4.2, we noted that a volunteer was needed for a new Windows library build of QuantLib for Windows to replace the outdated version 1.6 used there. Josh Ulrich stepped up, and built them. Josh and I tried for several month to get the win-builder to install these, but sadly other things took priority and we were unsuccessful. So this release will not have Windows binaries on CRAN as QuantLib 1.8 is not available there. Instead, you can use the ghrr drat and do
if (!require("drat")) install.packages("drat")
drat::addRepo("ghrr")
install.packages("RQuantLib")
to fetch prebuilt Windows binaries from the ghrr drat. Everybody else gets sources from CRAN. The full changes are detailed below.
Changes in RQuantLib version 0.4.3 (2016-08-19)
  • Changes in RQuantLib code:
    • Discount curve creation has been made more general by allowing additional arguments for day counter and fixed and floating frequency (contributed by Terry Leitch in #31, plus some work by Dirk in #32).
    • Swap leg parameters are now in combined variable and allow textual description (Terry Leitch in #34 and #35)
    • BermudanSwaption has been modfied to take option expiration and swap tenors in order to enable more general swaption structure pricing; a more general search for the swaptions was developed to accomodate this. Also, a DiscountCurve is allowed as an alternative to market quotes to reduce computation time for a portfolio on a given valuation date (Terry Leitch in #42 closing issue #41).
    • A new AffineSwaption model was added with similar interface to BermudanSwaption but allowing for valuation of a European exercise swaption utlizing the same affine methods available in BermudanSwaption. AffineSwaption will also value a Bermudan swaption, but does not take rate market quotes to build a term structure and a DiscountCurve object is required (Terry Leitch in #43).
    • Swap tenors can now be defined up to 100 years (Terry Leitch in #48 fising issue #46).
    • Additional (shorter term) swap tenors are now defined (Guillaume Horel in #49, #54, #55).
    • New SABR swaption pricer (Terry Leitch in #60 and #64, small follow-up by Dirk in #65).
    • Use of Travis CI has been updated and switch to maintained fork of deprecated mainline.
Courtesy of CRANberries, there is also a diffstat report for the this release. As always, more detailed information is on the RQuantLib page. Questions, comments etc should go to the rquantlib-devel mailing list off the R-Forge page. Issue tickets can be filed at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

15 August 2016

Shirish Agarwal: The road to TOR

Happy Independence Day to all. I had been looking forward to this day so I can use to share with my brothers and sisters what little I know about TOR . Independence means so many things to many people. For me, it means having freedom, valuing it and using it to benefit not just to ourselves but to people at large. And for that to happen, at least on the web, it has to rise above censorship if we are to get there at all. I am 40 years old, and if I can t read whatever I want to read without asking the state-military-Corporate trinity than be damned with that. Debconf was instrumental as I was able to understand and share many of the privacy concerns that we all have. This blog post is partly a tribute to being part of a community and being part of Debconf16. So, in that search for privacy couple of years ago, I came across TOR . TOR stands for The Onion Router project. Explaining tor is simple. Let us take the standard way in which we approach the website using a browser or any other means. a. We type out a site name, say debian.org in the URL/URI bar .
b. Now the first thing the browser would do is look into its DNS Cache to see if the name/URL has been used before. If it is something like debian.org which has been used before and is *fresh* and there is content already it would serve the content from the cache there itself.
c. In case, if it s not or the content is stale or something, it would generate a DNS lookup through the various routing tables till the DNS IP Address is found and information relayed to the browser.
d. The browser takes the IP Address and opens a TCP connection to the server, you have the handshake happen and after that it s business as usual.
e. In case if it doesn t work, you could get errors like Could not connect to server xyz or some special errors with error codes. This is a much simplified version of what happens or goes through normally with most/all of the browsers. One good way to see how the whole thing happens is to use traceroute and use the whois service. For e.g. [$] traceroute debian.org and then [$] whois 5.153.231.4 grep inetnum
inetnum: 5.153.231.0 - 5.153.231.255
Just using whois IP Address gives much more. I just shared a short version because I find it interesting that Debian has booked all 255 possible IP Addresses but speculating on that would be probably be a job for a different day. Now the difference when using TOR are two things a. The conversation is encrypted (somewhat like using https but encrypted through the relays)
b. The conversation is relayed over 2-3 relays and it will give a somewhat different identification to the DNS server at the other end.
c. It is only at the end-points that the conversation will be in plain text. For e.g. the TOR connection I m using atm is from me France (relay) Switzerland (relay) Germany (relay) WordPress.com . So wordpress thinks that all the connection is happening via Germany while I m here in India. It would also tells that I m running MS-Windows some version and a different browser while I m from somewhere in India, on Debian, using another browser altogether There are various motivations for doing that. For myself, I m just a private person and do not need or want that any other person/s or even the State should be looking over my shoulder as to what I m doing. And the argument that we need to spy on citizens because Terrorists are there doesn t hold water over me. There are many ways in which they can pass messages even without tor or web. The Government-Corporate-Military just get more powerful if and when they know what common people think, do, eat etc. So the question is how does you install tor if you a private sort of person . If you are on a Debian machine, you are one step closer to doing that. So the first thing that you need to do is install the following $ sudo aptitude install ooniprobe python-certifi tor tor-geoipdb torsocks torbrowser-launcher Once the above is done, then run torbrowser-launcher. This is how it would work out the first time it is run [$] torbrowser-launcher Tor Browser Launcher
By Micah Lee, licensed under MIT
version 0.2.6
https://github.com/micahflee/torbrowser-launcher
Creating GnuPG homedir /home/shirish/.local/share/torbrowser/gnupg_homedir
Downloading and installing Tor Browser for the first time.
Downloading https://dist.torproject.org/torbrowser/update_2/release/Linux_x86_64-gcc3/x/en-US
Latest version: 6.0.3
Downloading https://dist.torproject.org/torbrowser/6.0.3/tor-browser-linux64-6.0.3_en-US.tar.xz.asc
Downloading https://dist.torproject.org/torbrowser/6.0.3/tor-browser-linux64-6.0.3_en-US.tar.xz
Verifying signature
Extracting tor-browser-linux64-6.0.3_en-US.tar.xz
Running /home/shirish/.local/share/torbrowser/tbb/x86_64/tor-browser_en-US/start-tor-browser.desktop
Launching './Browser/start-tor-browser --detach'...
As can be seen above, you basically download the tor browser remotely from the website. Obviously, for this port 80 needs to be opened. One of the more interesting things is that it tells you where it installs the browser. /home/shirish/.local/share/torbrowser/tbb/x86_64/tor-browser_en-US/Browser/start-tor-browser and then detaches. The first time the TOR browser actually runs it looks something similar to this
Torbrowser picture

Torbrowser picture

Additionally it would give you 4 choices. Depending on your need for safety, security and convenience you make a choice and live with it. Now the only thing remaining to do is have an alias for your torbrowser. So I made [$] alias tor tor=/home/shirish/.local/share/torbrowser/tbb/x86_64/tor-browser_en-US/Browser/start-tor-browser It is suggested that you do not use the same usernames on the onion network. Also apart from the regular URL addresses such as flossexperiences.wordpress.com you will also see sites such as https://www.abc12defgh3ijkl.onion.to (fictional address) Now there would be others who would want to use the same/similar settings say as there are in their Mozilla Firefox installation. To do that do the following steps a. First close down both Torbrowser and Mozilla Firefox .
b. Open your file browser and go to where your mozilla profile details are. In typical Debian installations it is at ~/.mozilla/firefox/5r7t1r92.default In the next tab, navigate to ~/.local/share/torbrowser/tbb/x86_64/tor-browser_en-US/Browser/TorBrowser/Data/Browser/profile.default c. Now copy the following files over from your mozilla profile to your tor browser profile and you can resume where you left off.
    cert8.db
    chromeappsstore.sqlite
    content-prefs.sqlite
    cookies.sqlite
    formhistory.sqlite
    key3.db
    logins.json (Firefox 32 and above)
    mimeTypes.rdf
    permissions.sqlite
    persdict.dat
    places.sqlite
    signons3.txt (if exists)
    webappsstore.sqlite
and the following folders/directories
    bookmarkbackups
    chrome (if it exists)
    searchplugins (if it exists)
Once the above is done, fire up your torbrowser with the alias shared. This is usually put it in your .bashrc file or depending on whatever terminal interpreter you use, wherever the config file will be. Welcome to the world of TOR. Now, after a time if you benefit from tor and would like to give back to the tor community, you should look up tor bridges and relay. As the blog post has become long enough, I would end it now and hopefully we can talk about tor bridges and relay some other day.
Filed under: Miscellenous Tagged: #anonymity, #Debconf16, #debian, #tor, #torbrowser, GNU, Linux, Privacy

14 August 2016

Steinar H. Gunderson: Linear interpolation, source alignment, and Debian's embedding policy

At some point back when the dinosaurs roamed the Earth and I was in high school, I borrowed my first digital signal processing book from a friend. I later went on to an engineering education and master's thesis about DSP, but the very basics of DSP never stop to fascinate me. Today, I wanted to write something about one of them and how it affects audio processing in Nageru (and finally, how Debian's policies put me in a bit of a bind on this issue). DSP texts tend to obscure profound truths with large amounts of maths, so I'll try to present a somewhat less general result that doesn't require going into the mathematical details. That rule is: Adding a signal to weighted, delayed copies of itself is a filtering operation. (It's simple, but ignoring it will have sinister effects, as we'll see later.) Let's see exactly what that means with a motivating example. Let's say that I have a signal where I want to get rid of (or rather, reduce) high frequencies. The simplest way I can think of is to add every neighboring sample; that is, set yn = xn + xn-1. For each sample, we add the previous sample, ie., the signal as it was one sample ago. (We ignore what happens at the edges; the common convention is to assume signals extend out to infinity with zeros.) What effect will this have? We can figure it out with some trigonometry, but let's just demonstrate it by plotting instead: We assume 48 kHz sample rate (which means that our one-sample delay is 20.83 s) and a 22 kHz note (definitely treble!), and plot the signal with one-sample delay (the x axis is sample number): Filtered 22 kHz As you can see, the resulting signal is a new signal of the same frequency (which is always true; linear filtering can never create new frequencies, just boost or dampen existing ones), but with much lower amplitude. The signal and the delayed version of it end up cancelling each other mostly out. Also note that there signal has changed phase; the resulting signal has been a bit delayed compared to the original. Now let's look at a 50 Hz signal (turn on your bass face). We need to zoom out a bit to see full 50 Hz cycles: Filtered 50 Hz The original signal and the delayed one overlap almost exactly! For a lower frequency, the one-sample delay means almost nothing (since the waveform is varying so slowly), and thus, in this case, the resulting signal is amplified, not dampened. (The signal has changed phase here, too actually exactly as much in terms of real time but we don't really see it, because we've zoomed out.) Real signals are not pure sines, but they can be seen as sums of many sines (another fundamental DSP result), and since filtering is a linear operation, it affects those sines independently. In other words, we now have a very simple filter that will amplify low frequencies and dampen high frequencies (and delay the entire signal a little bit). We can do this for all frequencies from 0 to 24000 Hz; let's ask Octave to do it for us: Frequency plot for simple filter (Of course, in a real filter, we'd probably multiply the result with 0.5 to leave the bass untouched instead of boosting it, but it doesn't really change anything. A real filter would have a lot more coefficients, though, and they wouldn't all be the same!) Let's now turn to a problem that will at first seem different: Combining audio from multiple different time sources. For instance, when mixing video, you could have input from two different cameras or sounds card and would want to combine them (say, a source playing music and then some audience sound from a camera). However, unless you are lucky enough to have a professional-grade setup where everything runs off the same clock (and separate clock source cables run to every device), they won't be in sync; sample clocks are good, but they are not perfect, and they have e.g. some temperature variance. Say we have really good clocks and they only differ by 0.01%; this means that after an hour of streaming, we have 360 ms delay, completely ruining lip sync! This means we'll need to resample at least one of the sources to match the other; that is, play one of them faster or slower than it came in originally. There are two problems here: How do you determine how much to resample the signals, and how do we resample them? The former is a difficult problem in its own right; about every algorithm not backed in solid control theory is doomed to fail in one way or another, and when they fail, it's extremely annoying to listen to. Nageru follows a 2012 paper by Fons Adriaensen; GStreamer does well, something else. It fails pretty badly in a number of cases; see e.g. this 2015 master's thesis that tries to patch it up. However, let's ignore this part of the problem for now and focus on the resampling. So let's look at the case where we've determined we have a signal and need to play it 0.01% faster (or slower); in a real situation, this number would vary a bit (clocks are not even consistently wrong). This means that at some point, we want to output sample number 3000 and that corresponds to input sample number 3000.3, ie., we need to figure out what's between two input samples. As with so many other things, there's a way to do this that's simple, obvious and wrong, namely linear interpolation. The basis of linear interpolation is to look at the two neighboring samples and weigh them according to the position we want. If we need sample 3000.3, we calculate y = 0.7 x3000 + 0.3 x3001 (don't switch the two coefficients!), or, if we want to save one multiplication and get better numerical behavior, we can use the equivalent y = x3000 + 0.3 (x3001 - x3000). And if we need sample 5000.5, we take y = 0.5 x5000 + 0.5 x5001. And after a while, we'll be back on integer samples; output sample 10001 corresponds to x10000 exactly. By now, I guess it should be obvious what's going on: We're creating a filter! Linear interpolation will inevitably result in high frequencies being dampened; and even worse, we are creating a time-varying filter, which means that the amount of dampening will vary over time. This manifests itself as a kind of high-frequency flutter , where the amount of flutter depends on the relative resampling frequencies. There's also cubic resampling (which can mean any of several different algorithms), but it only really reduces the problem, it doesn't really solve it. The proper way of interpolating depends a lot on exactly what you want (e.g., whether you intend to change the rate quickly or not); this paper lays out a bunch of them, and was the paper that originally made me understand why linear interpolation is so bad. Nageru outsources this problem to zita-resampler, again by Fons Adriaensen; it yields extremely high-quality resampling under controlled delay, through a relatively common technique known as polyphase filters. Unfortunately, doing this kind of calculations takes CPU. Not a lot of CPU, but Nageru runs in rather CPU-challenged environments (ultraportable laptops where the GPU wants most of the TDP, and the CPU has to go down to the lowest frequency), and it is moving in a direction where it needs to resample many more channels (more on the later), so every bit of CPU helps. So I coded up an SSE optimization of the inner loop for a particular common case (stereo signals) and sent it in for upstream inclusion. (It made the code 2.7 times as fast without any structural changes or reducing precision, which is pretty much what you can expect from SSE.) Unfortunately, after a productive discussion, suddenly upstream went silent. I tried pinging, pinging again, and after half a year pinging again, but to no avail. I filed the patch in Debian's BTS, but the maintainer understandably is reluctant to carry a delta against upstream. I also can't embed a copy; Debian policy would dictate that I build against the system's zita-resampler. I could work around it by rewriting zita-resampler until it looks nothing like the original, which might be a good idea anyway if I wanted to squeeze out the last drops of speed; there are AVX optimizations to be had in addition to SSE, and the structure as-is isn't ideal for SSE optimizations (although some of the changes I have in mind would have to be offset against increased L1 cache footprint, so careful benchmarking would be needed). But in a sense, it feels like just working around a policy that's there for good reason. So like I said, I'm in a bit of a bind. Maybe I should just buy a faster laptop. Oh, and how does GStreamer solve this? Well, it doesn't use linear interpolation. It does something even worse it uses nearest neighbor. Gah. Update: I was asked to clarify that this is about the audio resampling done by the GStreamer audio sink to sync signals, not in the audioresample element, which solves a related but different problem (static sample rate conversion). The audioresample element supports a number of different resampling methods; I haven't evaluated them.

Dirk Eddelbuettel: rfoaas 1.0.0

rfoaas greed example The big 1.0.0 is here! Following the unsurpassed lead of the FOAAS project, we have arrived at a milestone: Release 1.0.0 is now on CRAN. The rfoaas package provides an interface for R to the most excellent FOAAS service--which itself provides a modern, scalable and RESTful web service for the frequent need to tell someone to f$#@ off. Release 1.0.0 brings fourteen (!!) new access points: back(), bm(), gfy(), think(), keep(), single_(), look(), looking(), no(), give(), zero(), pulp(), sake(), and anyway(). All with documentation and testing. Even more neatly, thanks to a very pull request by Tyler Hunt, we can now issue random FOs via the getRandomFO() function! As usual, CRANberries provides a diff to the previous CRAN release. Questions, comments etc should go to the GitHub issue tracker. More background information is on the project page as well as on the github repo

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

9 August 2016

Shirish Agarwal: Doha and the Supreme Court of DFSG Free

Hi, I am in two minds of what to write about Doha. My job has been vastly simplified by a friend when he shared with me https://www.youtube.com/watch?v=LdrAd-44LW0 . That video is more relevant and more closer to the truth than whatever I can share. As can be seen it is funny but more sad the way Qatarians are trying to figure out how things will be and as can be seen it seems to heading towards a real estate bubble . They would have to let go of the Sharia if they are thinking of wealthy westerners coming to stay put. I am just sad to know that many of my country-men are stuck there and although I hope the best for them, I dread it may turn out the way it has turned out for many people of Indians, and especially from Kerala in Saudi Arabia. I would touch about the Kerala situation probably in another blog post as this time is exclusively for legal aspects which were discussed in Debconf. A bit of backgrounder here, one part of my family is lawyers which means I have somewhat notion of law as practiced in our land. As probably everybody knows, India was ruled by the British for around 150 odd years. One of the things that they gave while leaving was/is the IPC (Indian Penal Code) and is practiced with the common law concept. The concept means precedence of any judgement goes quite some way in framing rulings and law of the land as time goes on besides the lobbying and the politics which happens in any democracy. Free software would not have been there without the GPL The General Public License. And the license is as much a legal document as it s something that the developers can work without becoming deranged, as it is one of the more simpler licenses to work with. My own understanding of the legal, ethical and moral issues around me were framed by two-three different TV shows, books (fiction and non-fiction alike) apart from what little news I heard in family. One was M*A*S*H* (with Alan Alda and his frailness, anarchism, humanism, civil rights), the Practise and Boston Legal which does lay bare the many grey areas that lawyers have to deal with ( The Practice also influenced a lot of civil rights understanding and First amendment, but as it is a TV show, how much of it is actually practiced for lawyers and how much moral dilemma they are can only be guessed at.) . In books it is artists like John Grisham, Michael Connelly as well as Perry Mason Agatha Christie. In non-fiction look at the treasures under bombayhighcourt e-books corner and series of Hamlyn Lectures. I would have to warn that all of the above are major time-sinks but rewarding in their own way. Also haven t read all of them as time and interests are constrained but do know they are good for understanding bit of our history. I do crave for a meetup kind of scenario when non-lawyers can read and discuss about facets of law . All that understanding was vastly amplified by Groklaw.net which made non-lawyers at the very least be able to decipher and understand what is going on in the free software world. After PJ (Pamela Jones) closed it in 2013 due to total surveillance by the Free World (i.e. the United States of America, NSA) we have been thirsty. We do get occasionally somewhat mildly interesting articles in lwn.net or arstechnica.net but nowhere the sheer brilliance of groklaw. So, it was a sheer stroke of luck that I met Mr. Bradley M. Kuhn who works with Karen Sandler on Software Conservancy. While I wanted to be there for his presentation, it was just one of those days which doesn t go as planned. However, as we met socially and over e-mail there were two basic questions which I asked him which also imbibes why we need to fight for software freedom in the court of law. Below is a re-wording of what he shared . Q1. why do people think that GPL still needs to be challenged in the court of law while there are gpl-violations which has been more or less successfully defended in the court of law ? Bradley Kuhn the GPL violations is basically a violation of one or more clauses of the GPL license and not the GPL license as a whole and my effort during my lifetime would be to make/have such precedents that the GPL is held as a valid license in the court of law. Q2. Let s say IF GPL is held to be valid in the court of law, would FSF benefit monetarily, at least to my mind it might be so, as more people and comapnies could be convinced to use strong copyleft licenses such as GPLv3 or AGPLv3 . Bradley Kuhn It may or may not. It is possible that even after winning, that people and especially companies may go for weak copyleft licenses if it suits them. The only benefit would probably would be to those people who are already using GPLv3 as the law could be used to protect them as well. Although we would want and welcome companies who would use strong copyleft license such as the GPL, the future is in future and hence uncertain. Both possibilities co-exist. While Bradley didn t say it, I would add further here it probably would mean also moving from being a more offensive mode (which GPL-violations is based upon where a violation occurs and somebody either from the victim s side or a by-stander notices the violation, brings it to the notice of the victim and the GPL-volations team.) to perhaps it being defended by the DMCA people themselves, once GPL is held as a valid license in the eyes of law. Although should you use the DMCA or not is a matter of choice, personal belief system as well as your legal recourses. I have to share that the FSF and the GPL-violations team are probably very discerning when they take up the fight as most of the work done by them is pro-bono (i.e. they don t make a single penny/paisa from the work done therein.) and hence in view of scarce resources, it makes sense to go only for the biggest violators in the hopes that you can either make them agree to compensate and agree to the terms of license of any software/hardware combination or sue them and take a bigger share of the reward/compensation awarded by the Court to help the defendant and maybe some of the proceeds donated by the defendant and people like you and me to make sure that Conservancy and the GPL-violations team is still around to help the next time something similar happens.
Bradley Kuhn presenting at #Debconf 16

Bradley Kuhn presenting at #Debconf 16

Now, as far as his presentation is concerned, whose video can be seen at http://meetings-archive.debian.net/pub/debian-meetings/2016/debconf16/The_Supreme_Court_of_DFSGFree.webm , I thought it was tame. While he talked about gaming the system in some sense, he was sharing that the system debian-legal works (most-of-the-time). The list actually works because many far more brilliant people than me take time to understand the intricacies of various licenses and how they should be interpreted through the excellently written Debian Free Software Guidelines and whether the license under discussion contravenes the DFSG or is part of it. I do agree with his point though that the ftp-master/s and the team may not be the right person to judge the license in adherence to the DFSG, or her/is not giving a reason for rejecting a package to not entering into the package archive. I actually asked the same question on debian-legal and while I had guessed, it seems there is enough review of the licenses per-se as answer from Paul Wise shows. Charles Pessley also shared an idea he has documented which probably didn t get much traction as involves more work on DD s without any benefit to show for it. All in all I hope it sheds some light on why there is need to be more aware of law in software freedom. Two Organizations which work on software freedom from legal standpoint are SFLC (Delhi) headed by the charming Mr. Eben Moglen and ALF (Bangalore). I do hope more people, especially developers take a bit more interest in some of the resources mentioned above.
Filed under: Miscellenous Tagged: #Alternative Law Forum, #bombayhighcourt e-library, #Common Law, #Debconf16, #Fiction, #Hewlyn lectures, #India, #Jurispudence, #legal fiction, #real estate bubble, #SFLC.in, #Software Freedom, #timesink, Doha, Law

7 August 2016

Dirk Eddelbuettel: drat 0.1.1: Updates schmupdates!

One year ago (tomorrow) drat 0.1.0 was released. It held up rather well, but a number of small fixes and enhancements piled up, along with somewhat-finished to still-raw additions to the examples/ sections. With that, we are happy to announce drat release 0.1.1 which arrived on CRAN earlier today. drat stands for drat R Archive Template, and helps with easy-to-create and easy-to-use repositories for R packages. Since its inception in early 2015 it has found reasonably widespread adoption among R users because repositories is what we use. In other words, friends don't let friends use install_github(). Just kidding. Maybe. Or not. This version 0.1.1 builds on the previous release from one year ago. Several users sent in nicely focused pull request, and I added a bit of spit and polish here and there. The NEWS file (added belatedly in this release) summarises the release as follows:
Changes in drat version 0.1.1 (2016-08-07)
  • Changes in drat functionality
    • Use dir.exists, leading to versioned Depends on R (>= 3.2.0)
    • Optionally pull remote before insert (Mark in PR #38)
    • Fix support for dots (Jan G. in PR #40)
    • Accept dots in package names (Antonio in PR #48)
    • Switch to htpps URLs at GitHub (Colin in PR #50)
    • Support additional fields in PACKAGE file (Jan G. in PR #54)
  • Changes in drat documentation
    • Further improvements and clarifications to vignettes
    • Travis script switched to run.sh from our fork
    • This NEWS file was (belatedly) added
Courtesy of CRANberries, there is a comparison to the previous release. More detailed information is on the drat page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Dirk Eddelbuettel: littler 0.3.1

max-heap image The second release of littler as a CRAN package is now available, following in the now more than ten-year history as a package started by Jeff in the summer of 2006, and joined by me a few weeks later. littler is the first command-line interface for R and predates Rscript. It is still faster, and in my very biased eyes better as it allows for piping as well shebang scripting via #!, uses command-line arguments more consistently and still starts faster. It prefers to live on Linux and Unix, has its difficulties on the OS X due yet-another-braindeadedness there (who ever thought case-insensitive filesystems where a good idea?) and simply does not exist on Windows (yet -- the build system could be extended -- see RInside for an existence proof, and volunteers welcome!). This release brings us fixes and enhancements from three other contributors, a couple new example scripts, more robust builds, extended documentation and more -- see below for details. The NEWS file entry is below.
Changes in littler version 0.3.1 (2016-08-06)
  • Changes in examples
    • install2.r now passes on extra options past -- to R CMD INSTALL (PR #37 by Steven Pav)
    • Added rcc.r to run rcmdcheck::rcmdcheck()
    • Added (still simple) render.r to render (R)markdown
    • Several examples now support the -x or --usage flag to show extended help.
  • Changes in build system
    • The AM_LDFLAGS variable is now set and used too (PR #38 by Mattias Ellert)
    • Three more directories, used when an explicit installation directory is set, are excluded (also #38 by Mattias)
    • Travis CI is now driven via run.sh from our fork, and deploys all packages as .deb binaries using our PPA where needed
  • Changes in package
    • SystemRequirements now mentions the need for libR, i.e. an R built with a shared library so that we can embed R.
    • The docopt and rcmdcheck packages are now suggested, and added to the Travis installation.
    • A new helper function r() is now provided and exported so that the package can be imported (closes #40).
    • URL and BugReports links were added to DESCRIPTION.
  • Changes in documentation
    • The help output for installGithub.r was corrected (PR #39 by Brandon Bertelsen)
Full details for the littler release are provided as usual at the ChangeLog page. The code is available via the GitHub repo, from tarballs off my littler page and the local directory here -- and now of course all from its CRAN page and via install.packages("littler"). Binary packages are available directly in Debian as well as soon via Ubuntu binaries at CRAN thanks to the tireless Michael Rutter. will probably have new Comments and suggestions are welcome at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

6 August 2016

Robert Edmonds: Cable modems: Arris SB6190 vs. Netgear CM600

Recently I activated new cable ISP service at my home and needed to purchase a cable modem. There were only a few candidate devices that supported at least 24 downstream channels (preferably 32), and did not contain an integrated router or access point. The first modem I tried was the Arris SB6190, which supports 32 downstream channels. It is based on the Intel Puma 6 SoC, and looking at an older release of the SB6190 firmware source reveals that the device runs Linux. This device, running the latest 9.1.93N firmware, goes into a failure mode after several days of uptime which causes it to drop 1-2% of packets. Here is a SmokePing graph that measures latency to my ISP's recursive DNS server, showing the transition into the degraded mode: SmokePing Arris SB6190 Firmware 9.1.93N It didn't drop packets at random, though. Some traffic would be deterministically dropped, such as the parallel A/AAAA DNS lookups generated by the glibc DNS stub resolver. For instance, in the following tcpdump output:
[1] 17:31:46.989073 IP [My IP].50775 > 75.75.75.75.53: 53571+ A? www.comcast6.net. (34)
[2] 17:31:46.989102 IP [My IP].50775 > 75.75.75.75.53: 14987+ AAAA? www.comcast6.net. (34)
[3] 17:31:47.020423 IP 75.75.75.75.53 > [My IP].50775: 53571 2/0/0 CNAME comcast6.g.comcast.net., [ ]
[4] 17:31:51.993680 IP [My IP].50775 > 75.75.75.75.53: 53571+ A? www.comcast6.net. (34)
[5] 17:31:52.025138 IP 75.75.75.75.53 > [My IP].50775: 53571 2/0/0 CNAME comcast6.g.comcast.net., [ ]
[6] 17:31:52.025282 IP [My IP].50775 > 75.75.75.75.53: 14987+ AAAA? www.comcast6.net. (34)
[7] 17:31:52.056550 IP 75.75.75.75.53 > [My IP].50775: 14987 2/0/0 CNAME comcast6.g.comcast.net., [ ]
Packets [1] and [2] are the A and AAAA queries being initiated in parallel. Note that they both use the same 4-tuple of (Source IP, Destination IP, Source Port, Destination Port), but with different DNS IDs. Packet [3] is the response to packet [1]. The response to packet [2] never arrives, and five seconds later, the glibc stub resolver times out and retries in single-request mode, which performs the A and AAAA queries sequentially. Packets [4] and [5] are the type A query and response, and packets [6] and [7] are the AAAA query and response. The Arris SB6190 running firmware 9.1.93N would consistently interfere with these parallel DNS requests, but only when operating in its degraded mode. It also didn't matter whether glibc was configured to use an IPv4 or IPv6 nameserver, or which nameserver was being used. Power cycling the modem would fix the issue for a few days. My ISP offered to downgrade the firmware on the Arris SB6190 to version 9.1.93K. This firmware version doesn't go into a degraded mode after a few days, but it does exhibit higher latency, and more jitter: SmokePing Arris SB6190 Firmware 9.1.93K It seemed unlikely that Arris would fix the firmware issues in the SB6190 before the end of my 30-day return window, so I returned the SB6190 and purchased a Netgear CM600. This modem appears to be based on the Broadcom BCM3384 and looking at an older release of the CM600 firmware source reveals that the device runs the open source eCos embedded operating system. The Netgear CM600 so far hasn't exhibited any of the issues I found with the Arris SB6190 modem. Here is a SmokePing graph for the CM600, which shows median latency about 1 ms lower than the Arris modem: SmokePing Netgear CM600 It's not clear which company is to blame for the problems in the Arris modem. Looking at the DOCSIS drivers in the SB6190 firmware source reveals copyright statements from ARRIS Group, Texas Instruments, and Intel. However, I would recommend avoiding cable modems produced by Arris in particular, and cable modems based on the Intel Puma SoC in general.

Dirk Eddelbuettel: RcppStreams 0.1.1

Streamulus A maintenance release of RcppStreams is now on CRAN. RcppStreams brings the excellent Streamulus C++ template library for event stream processing to R. Streamulus, written by Irit Katriel, uses very clever template meta-programming (via Boost Fusion) to implement an embedded domain-specific event language created specifically for event stream processing. This release updates the compilation standard to C++11 per CRAN's request as this helps with both current and upcoming compiler variants. A few edits were made to DESCRIPTION and README.md, the Travis driver file was updated, but no new code was added. The NEWS file entries follows below:
Changes in version 0.1.1 (2016-08-05)
  • Compilation is now done using C++11 standards per request of CRAN to help with an array of newer (pre-release) and older compilers
  • The Travis CI script was updated to use run.sh from our fork; it now also installs all dependencies as binary .deb files.
  • The README.md was updated with additional badges.
  • The DESCRIPTION file now has URL and BugReports entries.
Courtesy of CRANberries, there is also a copy of the DESCRIPTION file for this initial release. More detailed information is on the RcppStreams page page and of course on the Streamulus page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

3 August 2016

Elena 'valhalla' Grandi: The Cat Model of Package Ownership

The Cat Model of Package Ownership

Debian has been moving away from strong ownership of packages by package maintainers and towards encouraging group maintainership, for very good reasons: single maintainers have a bad bus factor and a number of other disadvantages.

When single maintainership is changed into maintainership by a small , open group of people who can easily communicate and sync with each other, everything is just better: there is an easy way to gradually replace people who want to leave, but there is also no duplication of efforts (because communication is easy), there are means to always have somebody available for emergency work and generally package quality can only gain from it.

Unfortunately, having such group of maintainers for every package would require more people than are available and willing to work on it, and while I think it's worth doing efforts to have big and important packages managed that way, it may not be so for the myriad of small ones that make up the long tail of a distribution.

Many of those packages may end up being maintained in a big team such as the language-based ones, which is probably better than remaining with a single maintainer, but can lead to some problems.

My experience with the old OpenEmbedded, back when it was still using monotone instead of git and everybody was maintaining everything, however, leads me to think that this model has a big danger of turning into nobody maintains anything, because when something needs to be done everybody is thinking that somebody else will do it.

As a way to prevent that, I have been thinking in the general direction of a Cat Model of Package Ownership, which may or may not be a way to prevent some risks of both personal maintainership and big teams.

The basic idea is that the my in my packages is not the my in my toys , but the my in my Cat, to whom I am a servant .

As in the case of a cat, if my package needs a visit to the vet, it's my duty to do so. Other people may point me to the need of such a visit, e.g. by telling me that they have seen the cat leaving unhealty stools, that there is a bug in the package, or even that upstream released a new version a week ago, did you notice?, but the actual putting the package in a cat carrier and bringing it to the vet falls on me.

Whether you're allowed to play with or pet the cat is her decision, not mine, and giving her food or doing changes to the package is usually fine, but please ask first: a few cats have medical issues that require a special diet.

And like cats, sometimes the cat may decide that I'm not doing a good enough job of serving her, and move away to another maintainer; just remember that there is a difference between a lost cat who wants to go back to her old home and a cat that is looking for a new one. When in doubt, packages usually wear a collar with contact informations, trying to ping those is probably a good idea.

This is mostly a summer afternoon idea and will probably require some refinement, but I think that the basic idea can have some value. Comments are appreciated on the federated social networks where this post is being published, via email (valid addresses are on my website http://www.trueelena.org/computers/articles/the_cat_model_of_package_ownership.html and on my GPG key http://www.trueelena.org/about/gpg.html) or with a post on a blog that appears on planet debian http://planet.debian.org/.

how small is small depends a lot on the size of the package, the amount of work it requires, how easy it is to parallelize it and how good are the people involved at communicating, so it would be quite hard to put a precise number here.

I've moved away from it because the boards I was using could run plain Debian, but I've heard that after the move to git there have been a number of workflow changes (of which I've seen the start) and everything now works much better.

Dirk Eddelbuettel: digest 0.6.10

A new release, now at version number 0.6.10, of the digest package is now on CRAN. I also just prepared the Debian upload. This release, just like the previous one, is once again the work mostly of external contributors. Michel Lang added length checks to sha1(); Thierry Onkelinx extended sha1() support and added more tests, Viliam Simko also extended sha1() to more types, and Henrik Bengtsson improved intervals and fixed a bug with file usage. CRANberries provides the usual summary of changes to the previous version.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

2 August 2016

Dirk Eddelbuettel: RcppGetconf 0.0.2

A first update for the recent RcppGetconf package for reading system configuration --- not unlike getconf from the libc library --- is now out. Almost immediately after I tweeted / blogged asking for help with OS X builds, fellow Rcpp hacker Qiang Kou created a clever pull request allowing for exactly that. So now we cover two POSIX systems that matter most these days --- Linux and OS X --- but as there are more out there, please do try, test and send those pull requests. We also added a new function getConfig() retrieving a single (given) value complementing the earlier catch-all function getAll(). You can find out more about RcppGetconf from the local RcppGetconf page and the GitHub repo. Changes in this release are as follows:
Changes in inline version 0.0.2 (2016-08-01)
  • A new function getConfig for single values was added.
  • The struct vars is now defined more portable allowing compilation on OS X (PR #1 by Qiang Kou).
Courtesy of CRANberries, there is a diffstat report. More about the package is at the local RcppGetconf page and the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

1 August 2016

Shirish Agarwal: Doha and the past year in APT

A week has gone by and another small sharing about Doha and one package that quite a few of us use everyday but don t think much of them. Let s start with Doha with these two pictures which tells/shares a bit about what the Doha of today is like qatar-1 qatar-2 While I have more than a dozen snapshots of Doha, all of them show same thing, all are huge skyscrapers and overall Doha seems to be aping Dubai and is in a frenzy as World Cup 2022 is around the corner. We did see a few of the older places but these seemed to be more done for tourists rather than the real thing. We saw stuff like wooden_ship This was a picture taken by Ritesh Raj Saraff, a friend and a DD whom I met while I was going to Debconf. The place where this picture has been taken is known as a Souk or what we know as market-place. This was a place where you could get spices. Quite a few of the spices that we get and use in India were bought from Middle-East in the olden times. In fact, it has been argued that the whole Mughlai food that is part of Indian culture was imported from Middle-East when we were trading them before India or Akhand Bharat was invaded. What was interesting to both of us is that we could perceive that most of the buildings had a sort of fakeness to it, they tried to show that it had a lot of detailed work on the buildings but we could see it was all done recently so not that old as being led to believe. One of the other interesting bits that we came to know throughout our stay in Qatar that almost 80-90% of the staff we met inside Qatar airport as well as in the Souk were people from Asian sub-continent and more specifically from South India. I had few interesting conversations with some of the people who were managing the shops were that almost of them were just employees while the owners were Qataris . I could understand this as the distance and flight between Qatar and India is hardly 3 hours. It seemed very similar to how Mexicans look for work in United States. The most expensive thing there was water as it s desert other than housing and most workers seemed to have shared accommodation from anywhere between 5-15 people in one room. It s only the relative strength of the Qatari Rial which probably compels them to be there. The temperature was around 45 degrees with a bit of humidity as it s next to the Ocean. For all the money in the world, I wouldn t work there. It is true that you know your city s worth only when you go outside:) I do have some more stories about Qatar but that would have to wait for another day now. Also, I really don t want to talk much about this part as it s part depressing but probably would explore it a bit in a further blog-post. One of the more interesting topics that I attended was the apt talk . There are 3-4 tools in the Debian world i.e. apt, aptitude, apt-get, dpkg and dselect. More often than not people know aptitude and apt-get whereas the rest of the packages are not thought so much about. What I somewhat suspected about the history of apt was revealed to be true today, courtesy David K. julias-andreas-klose-year-in-apt You can see the talk/video about apt at http://meetings-archive.debian.net/pub/debian-meetings/2016/debconf16/The_past_year_in_APT.webm. I had been curious about apt,libapt,dpkg and the entire tool-chain which goes into updating packages and like. I had a couple of conversations here in India before on mail, in person and IRC as well as couple of conversations in South Africa as well before the APT talk where it was told that packages are not signed or it s not easy to figure out the integrity. Being a Debian fan-boy I could not believe this to be true. Hence I asked and to my dismay found it to be true . I also then asked the same with a bit more background on the mailing list as well and got to know that this has been a concern since 2005. As I do not have the requisite skills and the person would require probably knowledge of dpkg internals as well as have probably good social skills to have at least 1-2 DD s help her/im to work on it and have probably some server space where even some partial archive is re-built using debian packages which use dpkg-sig . I also had some concerns that even if somebody did do the work, it might come in the way of the reproducible builds concept where Neils shared ways in which it could be overcome. Having said the above, it is totally doable if somebody has the will, skills and the patience to do it. Just look at the amazing work done by the team which re-built almost all the archive using clang. See clang.debian.net for the amazing work that they have done. Now, one of the issues in India which comes in popularizing Debian or in fact any free software distribution in India is the bandwidth issue or rather the lack of it or how expensive it is. The situation for better lack of term is pathetic . While nothing can be done till the time the Govt. gives limited term oligopoly licenses to telecom operators and they have a cabal (cabal closed team where decisions and policies are made without any knowledge of and to other stakeholders.) we need to find ways to make the best of the situation. Anyways, while there are some ideas to tackle that but that s a long-term goal and I will share some aspects of it in probably another blog post. In the interim somethings can definitely be made better. Now one of the issues that is there for most people is getting the package updates. Before updating the packages, the package index needs to be updated. Now, both in home and work environments most people are cautious to update the package index. But many times, either due to bandwidth issues or some other issue which is outside your control, your package index is corrupted. I have put both the possible reasons of why and how the package index corruption takes place and a probable work-around of in the deity mail post. I do hope to put in a more coherent state by probably making smaller bug issues so they could be tackled or answered one by one. Any improvements would be better for stability of debian infrastructure only. If anybody does do the required work and need a guinea pig for testing, count me in. Just holler and share you will be working on this aspect and at least one of my workstations would definitely take part in seeing if its better or not. Even if you are able to just provide a way to make a copy of /var/lib/apt/lists after every successful update and do the comparison with time-stamp on next run and only change the copy when a successful update occurs, that will be a huge help in itself. Look forward to hearing form one and all.
Filed under: Miscellenous Tagged: #Debconf16, #feature-request, #Julian Andreas Klose, #shell-script ?, apt, aptitude

Ben Armstrong: Retiring as a Debian developer

This is a repost and update of my retirement letter sent privately to Debian last month, July 10, 2016. At that time I received many notes of appreciation and good wishes which I treasure. Now, I d like to say goodbye to the broader Debian community and, as well, indicate which of the cleanup items have since been addressed in strikethrough style and with annotations. Also, I d like to stay in touch with many of you, so I have added some comments oriented towards those of you who are interested in doing that after the letter.
When in 1995, on a tip from a friend, I installed Debian on my 386 at work and was enthralled with the results, I could not have foreseen that two years later, friends I had made on channel #debian would nudge me to become a Debian developer. Nor when that happened did I have any idea that twenty years later, I d consider Debian to be like family, the greatest free software community in the world, and would still be promoting it and helping people with it whenever I could. Debian quietly, unexpectedly became a part of what defines me. My priorities in life have changed over that time, though. I have shifted my attention to things that are more important to me in life, such as my family, my health and well-being physically and spiritually, and bringing all I can to bear on the task of preserving our local wilderness areas and trails. In the latter area, I m now bringing all of what Debian has helped shaped me to be to the table, launching some ambitious projects I hope will bear fruit in the coming years, and make a measurable contribution to help us hang onto our precious natural preserves where I live. Unfortunately, as I ve poured more time and energy into these things, I ve increasingly not been giving my packages the care they need. Nor do I have any roles or goals now for any of the Debian projects I was previously involved in. So, after much careful deliberation, and as much as it pains me to say it, it s time to retire as a Debian developer. It has been a great privilege to work with you, and to meet many of you in New York at Debconf 10. I plan to be around online, and will continue to take an interest in Debian, lending a hand when I can. Thanks for all of the fun times, for all that I ve learned, and for the privilege to make awesome things with you. I ll treasure this forever. So much for the soppy bits.  Now, business. These things remain to clean up upon my departure, and I d appreciate help from QA, and anyone else who can lend a hand. My packages are effectively orphaned, but I haven t the time to do any of the cleanup myself, so please speak up if you can help.
  1. Debian Jr.
    • O: junior-doc. The junior-doc package has been awaiting an overhaul by whoever revives the project since I gave it up years ago. I m still listed as maintainer and that should be changed to Debian Junior Maintainers <debianjr-maintainers@lists.alioth.debian.org> if they want it. Otherwise, it is orphaned.
    • I should also be dropped from Uploaders from debian-junior, the metapackages source. Fixed in git.
  2. Tux Paint. This is a very special package that deserves to go to someone who will love it and care for it well. There are three source packages in all:
    • O: tuxpaint
    • O: tuxpaint-config
    • O: tuxpaint-stamps
  3. O: xletters. This is a cute little typing practice game and needs a new maintainer.
  4. XPilot is co-maintained by Phil Brooke <pjb@debian.org>, so he should replace me as Maintainer. Phil said he ll pick up xpilot-ng and will also look at xpilot-extra.
    • xpilot-ng
    • O: xpilot-extra (recently removed from testing due to my neglect, and not co-maintained by Phil; it s unclear if anyone really uses this anymore)
  5. GTypist is co-maintained by Daniel Leidert <dleidert@debian.org> and should replace me as Maintainer.
  6. My ruby packages. A group of packages that I brought into Debian as dependencies of taskwarrior-web, which I never completed. Maybe they ll be useful in and of themselves, and maybe not. In any case, they are maintained by pkg-ruby-extras-maintainers, but I m the sole developer in Uploaders and should be removed: Fixed in git.
    • ruby-blockenspiel
    • ruby-parseconfig
    • ruby-rack-flash3
    • ruby-simple-navigation
    • ruby-sinatra-simple-navigation
    • ruby-term-ansicolor
    • ruby-versionomy
  7. Debian Live stuff: I am listed in Uploaders for live-manual (fixed in git) and debian-installer-launcher (fixed in git) and need to be removed.
  8. O: eeepc-acpi-scripts. The defunct Debian EeePC project has just this one package. Recently, the mailing list was asked about its status, and it was recently NMU d. To my knowledge, nobody from the original team remains to take care of it, so it needs a new maintainer. I should be removed from Uploaders, and since the Debian Eee PC Team no longer exists, it should be removed as maintainer. It is effectively orphaned unless someone speaks up.
There are also some Alioth projects / lists that are defunct that I ll need to talk to the Alioth admins about cleaning up in the coming days. One of these is <debian-eeepc-devel@lists.alioth.debian.org> and since it is still listed as the maintainer of eeepc-acpi-scripts, that needs to be sorted out before the list can be closed. Thanks again, and see you around!
Ben

Stay in touch For those of you who would like to stay in touch, here are some ways to do that:
  • Follow my blog: http://syn.theti.ca
    If you already do that, great! If not, welcome to my blog! For the past couple of years you may have noticed a decrease in technical content and increase in local trails and conservation oriented posts. You can expect more of the latter.
  • Say hi to me on irc: SynrG (also SynrGy) on irc.oftc.net (irc.debian.org) or irc.freenode.net.
    I still intend to hang out and offer support when I can, just no longer as a developer. Channel #debian-offtopic on either network is a good place to catch up with me socially.
  • Follow me on Facebook: https://www.facebook.com/SynrG
    For better or worse, a lot of the trails and conservation folks hang out here, and many of you in the Debian community are already my Facebook friends.
  • Look for my Bluff Trail posts on their site: https://wrweo.ca
    Providing tech support to this organization is where much of my time and energy is going these days. I post here once in a while, but do most of my work behind the scenes as a volunteer and, newly this year, as a board member.
Facebooktwittergoogle_plusredditpinterestlinkedinmail

29 July 2016

Dirk Eddelbuettel: RcppGetconf 0.0.1

A new package RcppGetconf to read system configuration --- not unlike getconf from the libc library. Now R can read what system calls sysconf, pathconf and confstr have to say. The package is still pretty green, and now on CRAN in a very first version, corresponding to a very first (and single !) commit. Right now, the CRAN just has one function getAll() similar to getconf -a. A first example shows how it provides all values which can be retried -- currently 320 on my systems.
R> res <- getAll()
R> head(res)
               key value type
1         LINK_MAX 65000 path
2  _POSIX_LINK_MAX 65000 path
3        MAX_CANON   255 path
4 _POSIX_MAX_CANON   255 path
5        MAX_INPUT   255 path
6 _POSIX_MAX_INPUT   255 path
R> tail(res)
                      key  value type
315    LEVEL4_CACHE_ASSOC      0  sys
316 LEVEL4_CACHE_LINESIZE      0  sys
317                  IPV6 200809  sys
318           RAW_SOCKETS 200809  sys
319           _POSIX_IPV6 200809  sys
320    _POSIX_RAW_SOCKETS 200809  sys
R> 
Earlier this evening I added a second function to the GitHub repo which can access indivial values. But right now, the biggest need is really for someone with some systems skills---and an OS X machine---to look at the code, and maybe the getconf.c from the C library, in order to make this build OS X. If you can help, please get in touch. More about the package is at the local RcppGetconf page and the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

28 July 2016

Gunnar Wolf: Subtitling DebConf talks Come and join!

As I have said here a couple of times already, I am teaching a diploma course on embedded Linux at UNAM, and one of the modules I'm teaching (with Sandino Araico) is the boot process. We focus on ARM for obvious reasons, and while I have done my reading on the topic, I am very far from considering myself an expert. So, after attending Martin Michlmayr's Debian on ARM devices talk, I decided to do its subtitles as part of my teaching job. This talk gives a great panorama on what actually has to happen in order to get an ARM machine to boot, and how support for new ARM devices comes around to Linux in general and to Debian in particular Perfect for our topic! But my students are not always very fluent in English, so giving a hand is always most welcome. In case any of you dear readers didn't know, we have a DebConf subtitling team. Yes, our work takes much longer to reach the public, and we have no hopes whatsoever in getting it completed, but every person lending a hand and subtitling a talk that they thought was interesting helps a lot to improve our talks' usability. Even if you don't have enough time to do the whole talk (we are talking about some 6hr per 45 minute session), adding a bit of work is very very very welcome. So... Enjoy And thanks in advance for your work!

24 July 2016

Dirk Eddelbuettel: RcppArmadillo 0.7.200.2.0

armadillo image The second Armadillo release of the 7.* series came out a few weeks ago: version 7.200.2. And RcppArmadillo version 0.7.200.2.0 is now on CRAN and uploaded to Debian. This followed the usual thorough reverse-dependecy checking of by now over 240 packages using it. For once, I let it simmer a little preparing only a package update via the GitHub repo without preparing a CRAN upload to lower the update frequency a little. Seeing that Conrad has started to release 7.300.0 tarballs, the time for a (final) 7.200.2 upload was now right. Just like the previous, it now requires a recent enough compiler. As g++ is so common, we explicitly test for version 4.6 or newer. So if you happen to be on an older RHEL or CentOS release, you may need to get yourself a more modern compiler. R on Windows is now at 4.9.3 which is decent (yet stable) choice; the 4.8 series of g++ will also do. For reference, the current LTS of Ubuntu is at 5.4.0, and we have g++ 6.1 available in Debian testing. This new upstream release adds new indexing helpers, additional return codes on some matrix transformations, increased speed for compound expressions via vectorise, corrects some LAPACK feature detections (affecting principally complex number use under OS X), and a rewritten sample() function thanks to James Balamuta. Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. Changes in this release (and the preceding GitHub-only release 0.7.200.1.0 are as follows:
Changes in RcppArmadillo version 0.7.200.2.0 (2016-07-22)
  • Upgraded to Armadillo release 7.200.2
  • The sampling extension was rewritten to use Armadillo vector types instead of Rcpp types (PR #101 by James Balamuta)
Changes in RcppArmadillo version 0.7.200.1.0 (2016-06-06)
  • Upgraded to Armadillo release 7.200.1
    • added .index_min() and .index_max()
    • expanded ind2sub() to handle vectors of indices
    • expanded sub2ind() to handle matrix of subscripts
    • expanded expmat(), logmat() and sqrtmat() to optionally return a bool indicating success
    • faster handling of compound expressions by vectorise()
  • The configure code now (once again) sets the values for the LAPACK feature #define correctly.
Courtesy of CRANberries, there is a diffstat report. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

23 July 2016

Steve Kemp: A final post about the lua-editor.

I recently mentioned that I'd forked Antirez's editor and added lua to it. I've been working on it, on and off, for the past week or two now. It's finally reached a point where I'm content: Most of the obvious things I use in Emacs are present, such as the ability to customize the status-bar (right now it shows the cursor position, the number of characters, the number of words, etc, etc). Anyway I'll stop talking about it now :)

22 July 2016

Dirk Eddelbuettel: RcppCCTZ 0.0.5

Version 0.0.5 of RcppCCTZ arrived on CRAN a couple of days ago. It reflects an upstream fixed made a few weeks ago. CRAN tests revealed that g++-6 was tripping over one missing #define; this was added upstream and I subsequently synchronized with upstream. At the same time the set of examples was extended (see below). Somehow useR! 2016 got in the way and while working on the then-incomplete examples during the traveling I forgot to release this until CRAN reminded me that their tests still failed. I promptly prepared the 0.0.5 release but somehow failed to update NEWS files etc. They are correct in the repo but not in the shipped package. Oh well. CCTZ is a C++ library for translating between absolute and civil times using the rules of a time zone. In fact, it is two libraries. One for dealing with civil time: human-readable dates and times, and one for converting between between absolute and civil times via time zones. It requires only a proper C++11 compiler and the standard IANA time zone data base which standard Unix, Linux, OS X, ... computers tend to have in /usr/share/zoneinfo. RcppCCTZ connects this library to R by relying on Rcpp. Two good examples are now included, and shown here. The first one tabulates the time difference between New York and London (at a weekly level for compactness):
R> example(tzDiff)
tzDiffR> # simple call: difference now
tzDiffR> tzDiff("America/New_York", "Europe/London", Sys.time())
[1] 5
tzDiffR> # tabulate difference for every week of the year
tzDiffR> table(sapply(0:52, function(d) tzDiff("America/New_York", "Europe/London",
tzDiff+                                       as.POSIXct(as.Date("2016-01-01") + d*7))))
 4  5 
 3 50 
R> 
Because the two continents happen to spring forward and fall backwards between regular and daylight savings times, there are, respectively, two and one week periods where the difference is one hour less than usual. A second example shifts the time to a different time zone:
R> example(toTz)
toTzR> toTz(Sys.time(), "America/New_York", "Europe/London")
[1] "2016-07-14 10:28:39.91740 CDT"
R> 
Note that because we return a POSIXct object, it is printed by R with the default (local) TZ attribute (for "America/Chicago" in my case). A more direct example asks what time it is in my time zone when it is midnight in Tokyo:
R> toTz(ISOdatetime(2016,7,15,0,0,0), "Japan", "America/Chicago")
[1] "2016-07-14 15:00:00 CDT"
R>
More changes will come in 0.0.6 as soon as I find time to translate the nice time_tool (command-line) example into an R function. Changes in this version are summarized here:
Changes in version 0.0.5 (2016-07-09)
  • New utility example functions toTz() and tzDiff
  • Synchronized with small upstream change for additional #ifdef for compiler differentiation
We also have a diff to the previous version thanks to CRANberries. More details are at the RcppCCTZ page; code, issue tickets etc at the GitHub repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Next.

Previous.