Search Results: "nate"

3 August 2016

Dirk Eddelbuettel: digest 0.6.10

A new release, now at version number 0.6.10, of the digest package is now on CRAN. I also just prepared the Debian upload. This release, just like the previous one, is once again the work mostly of external contributors. Michel Lang added length checks to sha1(); Thierry Onkelinx extended sha1() support and added more tests, Viliam Simko also extended sha1() to more types, and Henrik Bengtsson improved intervals and fixed a bug with file usage. CRANberries provides the usual summary of changes to the previous version.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

2 August 2016

Dirk Eddelbuettel: RcppGetconf 0.0.2

A first update for the recent RcppGetconf package for reading system configuration --- not unlike getconf from the libc library --- is now out. Almost immediately after I tweeted / blogged asking for help with OS X builds, fellow Rcpp hacker Qiang Kou created a clever pull request allowing for exactly that. So now we cover two POSIX systems that matter most these days --- Linux and OS X --- but as there are more out there, please do try, test and send those pull requests. We also added a new function getConfig() retrieving a single (given) value complementing the earlier catch-all function getAll(). You can find out more about RcppGetconf from the local RcppGetconf page and the GitHub repo. Changes in this release are as follows:
Changes in inline version 0.0.2 (2016-08-01)
  • A new function getConfig for single values was added.
  • The struct vars is now defined more portable allowing compilation on OS X (PR #1 by Qiang Kou).
Courtesy of CRANberries, there is a diffstat report. More about the package is at the local RcppGetconf page and the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

1 August 2016

Ben Armstrong: Retiring as a Debian developer

This is a repost and update of my retirement letter sent privately to Debian last month, July 10, 2016. At that time I received many notes of appreciation and good wishes which I treasure. Now, I d like to say goodbye to the broader Debian community and, as well, indicate which of the cleanup items have since been addressed in strikethrough style and with annotations. Also, I d like to stay in touch with many of you, so I have added some comments oriented towards those of you who are interested in doing that after the letter.
When in 1995, on a tip from a friend, I installed Debian on my 386 at work and was enthralled with the results, I could not have foreseen that two years later, friends I had made on channel #debian would nudge me to become a Debian developer. Nor when that happened did I have any idea that twenty years later, I d consider Debian to be like family, the greatest free software community in the world, and would still be promoting it and helping people with it whenever I could. Debian quietly, unexpectedly became a part of what defines me. My priorities in life have changed over that time, though. I have shifted my attention to things that are more important to me in life, such as my family, my health and well-being physically and spiritually, and bringing all I can to bear on the task of preserving our local wilderness areas and trails. In the latter area, I m now bringing all of what Debian has helped shaped me to be to the table, launching some ambitious projects I hope will bear fruit in the coming years, and make a measurable contribution to help us hang onto our precious natural preserves where I live. Unfortunately, as I ve poured more time and energy into these things, I ve increasingly not been giving my packages the care they need. Nor do I have any roles or goals now for any of the Debian projects I was previously involved in. So, after much careful deliberation, and as much as it pains me to say it, it s time to retire as a Debian developer. It has been a great privilege to work with you, and to meet many of you in New York at Debconf 10. I plan to be around online, and will continue to take an interest in Debian, lending a hand when I can. Thanks for all of the fun times, for all that I ve learned, and for the privilege to make awesome things with you. I ll treasure this forever. So much for the soppy bits.  Now, business. These things remain to clean up upon my departure, and I d appreciate help from QA, and anyone else who can lend a hand. My packages are effectively orphaned, but I haven t the time to do any of the cleanup myself, so please speak up if you can help.
  1. Debian Jr.
    • O: junior-doc. The junior-doc package has been awaiting an overhaul by whoever revives the project since I gave it up years ago. I m still listed as maintainer and that should be changed to Debian Junior Maintainers <debianjr-maintainers@lists.alioth.debian.org> if they want it. Otherwise, it is orphaned.
    • I should also be dropped from Uploaders from debian-junior, the metapackages source. Fixed in git.
  2. Tux Paint. This is a very special package that deserves to go to someone who will love it and care for it well. There are three source packages in all:
    • O: tuxpaint
    • O: tuxpaint-config
    • O: tuxpaint-stamps
  3. O: xletters. This is a cute little typing practice game and needs a new maintainer.
  4. XPilot is co-maintained by Phil Brooke <pjb@debian.org>, so he should replace me as Maintainer. Phil said he ll pick up xpilot-ng and will also look at xpilot-extra.
    • xpilot-ng
    • O: xpilot-extra (recently removed from testing due to my neglect, and not co-maintained by Phil; it s unclear if anyone really uses this anymore)
  5. GTypist is co-maintained by Daniel Leidert <dleidert@debian.org> and should replace me as Maintainer.
  6. My ruby packages. A group of packages that I brought into Debian as dependencies of taskwarrior-web, which I never completed. Maybe they ll be useful in and of themselves, and maybe not. In any case, they are maintained by pkg-ruby-extras-maintainers, but I m the sole developer in Uploaders and should be removed: Fixed in git.
    • ruby-blockenspiel
    • ruby-parseconfig
    • ruby-rack-flash3
    • ruby-simple-navigation
    • ruby-sinatra-simple-navigation
    • ruby-term-ansicolor
    • ruby-versionomy
  7. Debian Live stuff: I am listed in Uploaders for live-manual (fixed in git) and debian-installer-launcher (fixed in git) and need to be removed.
  8. O: eeepc-acpi-scripts. The defunct Debian EeePC project has just this one package. Recently, the mailing list was asked about its status, and it was recently NMU d. To my knowledge, nobody from the original team remains to take care of it, so it needs a new maintainer. I should be removed from Uploaders, and since the Debian Eee PC Team no longer exists, it should be removed as maintainer. It is effectively orphaned unless someone speaks up.
There are also some Alioth projects / lists that are defunct that I ll need to talk to the Alioth admins about cleaning up in the coming days. One of these is <debian-eeepc-devel@lists.alioth.debian.org> and since it is still listed as the maintainer of eeepc-acpi-scripts, that needs to be sorted out before the list can be closed. Thanks again, and see you around!
Ben

Stay in touch For those of you who would like to stay in touch, here are some ways to do that:
  • Follow my blog: http://syn.theti.ca
    If you already do that, great! If not, welcome to my blog! For the past couple of years you may have noticed a decrease in technical content and increase in local trails and conservation oriented posts. You can expect more of the latter.
  • Say hi to me on irc: SynrG (also SynrGy) on irc.oftc.net (irc.debian.org) or irc.freenode.net.
    I still intend to hang out and offer support when I can, just no longer as a developer. Channel #debian-offtopic on either network is a good place to catch up with me socially.
  • Follow me on Facebook: https://www.facebook.com/SynrG
    For better or worse, a lot of the trails and conservation folks hang out here, and many of you in the Debian community are already my Facebook friends.
  • Look for my Bluff Trail posts on their site: https://wrweo.ca
    Providing tech support to this organization is where much of my time and energy is going these days. I post here once in a while, but do most of my work behind the scenes as a volunteer and, newly this year, as a board member.
Facebooktwittergoogle_plusredditpinterestlinkedinmail

Chris Lamb: Free software activities in July 2016

Here is my monthly update covering a large part of what I have been doing in the free software world (previously):



Debian
  • Created a proof-of-concept wrapper for pymysql to reduce the diff between Ubuntu and Debian's packaging of python-django. (tree)
  • Improved the NEW queue HTML report to display absolute timestamps when placing the cursor over relative times as well as to tidy the underlying HTML generation.
  • Tidied and pushed for the adoption of a patch against dak to also send mails to the signer of an uploaded package on security-master. (#796784)

LTS

This month I have been paid to work 14 hours on Debian Long Term Support (LTS). In that time I did the following:
  • "Frontdesk" duties, triaging CVEs, etc.
  • Improved the bin/lts-cve-triage.py script to ignore packages that have been marked as unsupported.
  • Improved the bin/contact-maintainers script to print a nicer error message if you mistype the package name.
  • Issued the following advisories:
    • DLA 541-1 for libvirt making the password policy consistent across the QEMU and VNC backends with respect to empty passwords.
    • DLA 574-1 for graphicsmagick fixing two denial-of-service vulnerabilities.
    • DLA 548-1 and DLA 550-1 for drupal7 fixing an open HTTP redirect vulnerability and a privilege escalation issue respectfully.
    • DLA 557-1 for dietlibc removing the current directory from the current path.
    • DLA 577-1 for redis preventing the redis-cli tool creating world-readable history files.

Uploads
  • redis:
    • 3.2.1-2 Avoiding race conditions in upstream test suite.
    • 3.2.1-3 Correcting world_readable ~/.rediscli_history files.
    • 3.2.1-4 Preventing a race condition in the previous upload's patch.
    • 3.2.2-1 New upstream release.
    • 3.2.1-4~bpo8+1 Backport to jessie-backports.
  • strip-nondeterminism:
    • 0.020-1 Improved the PNG handler to not blindly trust chunk sizes, rewriting most of the existing code.
    • 0.021-1 Correcting a regression in the PNG handler where it would leave temporary files in the generated binaries.
    • 0.022-1 Correcting a further regression in the PNG handler with respect to IEND chunk detection.
  • python-redis (2.10.5-1~bpo8+1) Backport to jessie-backports.
  • reprotest (0.2) Sponsored upload.

Patches contributed


I submitted patches to fix faulty initscripts in lm-sensors, rsync, sane-backends & vsftpd.

In addition, I submitted 7 patches to fix typos in debian/rules against cme:, gnugk: incorrect reference to dh_install_init, php-sql-formatter, python-django-crispy-forms, libhook-lexwrap-perl, mknbi & ruby-unf-ext.

I also submitted 6 patches to fix reproducible toolchain issues (ie. ensuring the output is reproducible rather than the package itself) against libextutils-parsexs-perl: Please make the output reproducible, perl, naturaldocs, python-docutils, ruby-ronn & txt2tags.

Lastly, I submitted 65 patches to fix specific reproducibility issues in amanda, boolector, borgbackup, cc1111, cfingerd, check-all-the-things, cobbler, ctop, cvs2svn, eb, eurephia, ezstream, feh, fonts-noto, fspy, ftplib, fvwm, gearmand, gngb, golang-github-miekg-pkcs11, gpick, gretl, hibernate, hmmer, hocr, idjc, ifmail, ironic, irsim, lacheck, libmemcached-libmemcached-perl, libmongoc, libwebsockets, minidlna, mknbi, nbc, neat, nfstrace, nmh, ntopng, pagekite, pavuk, proftpd-dfsg, pxlib, pysal, python-kinterbasdb, python-mkdocs, sa-exim, speech-tools, stressapptest, tcpflow, tcpreen, ui-auto, uisp, uswsusp, vtun, vtwm, why3, wit, wordgrinder, xloadimage, xmlcopyeditor, xorp, xserver-xorg-video-openchrome & yersinia.

RC bugs

I also filed 68 RC bugs for packages that access the internet during build against betamax, curl, django-localflavor, django-polymorphic, dnspython, docker-registry, elasticsearch-curator, elib.intl, elib.intl, elib.intl, fabulous, flask-restful, flask-restful, flask-restful, foolscap, gnucash-docs, golang-github-azure-go-autorest, golang-github-fluent-fluent-logger-golang, golang-github-franela-goreq, golang-github-mesos-mesos-go, golang-github-shopify-sarama, golang-github-unknwon-com, golang-github-xeipuuv-gojsonschema, htsjdk, lemonldap-ng, libanyevent-http-perl, libcommons-codec-java, libfurl-perl, libgravatar-url-perl, libgravatar-url-perl, libgravatar-url-perl, libgravatar-url-perl, libgravatar-url-perl, libhttp-async-perl, libhttp-oai-perl, libhttp-proxy-perl, libpoe-component-client-http-perl, libuv, libuv1, licenseutils, licenseutils, licenseutils, musicbrainzngs, node-oauth, node-redis, nodejs, pycurl, pytest, python-aiohttp, python-asyncssh, python-future, python-guacamole, python-latexcodec, python-pysnmp4, python-qtawesome, python-simpy, python-social-auth, python-structlog, python-sunlight, python-webob, python-werkzeug, python-ws4py, testpath, traitlets, urlgrabber, varnish-modules, webtest & zurl.


Finally, I filed 100 FTBFS bugs against abind, backup-manager, boot, bzr-git, cfengine3, chron, cloud-sptheme, cookiecutter, date, django-uwsgi, djangorestframework, docker-swarm, ekg2, evil-el, fasianoptions, fassets, fastinfoset, fest-assert, fimport, ftrading, gdnsd, ghc-testsuite, golang-github-magiconair-properties, golang-github-mattn-go-shellwords, golang-github-mitchellh-go-homedir, gplots, gregmisc, highlight.js, influxdb, jersey1, jflex, jhdf, kimwitu, libapache-htpasswd-perl, libconfig-model-itself-perl, libhtml-tidy-perl, liblinux-prctl-perl, libmoox-options-perl, libmousex-getopt-perl, libparanamer-java, librevenge, libvirt-python, license-reconcile, louie, mako, mate-indicator-applet, maven-compiler-plugin, mgt, mgt, mgt, misc3d, mnormt, nbd, ngetty, node-xmpp, nomad, perforate, pyoperators, pyqi, python-activipy, python-bioblend, python-cement, python-gevent, python-pydot-ng, python-requests-toolbelt, python-ruffus, python-scrapy, r-cran-digest, r-cran-getopt, r-cran-lpsolve, r-cran-rms, r-cran-timedate, resteasy, ruby-berkshelf-api-client, ruby-fog-libvirt, ruby-grape-msgpack, ruby-jquery-rails, ruby-kramdown-rfc2629, ruby-moneta, ruby-parser, ruby-puppet-forge, ruby-rbvmomi, ruby-redis-actionpack, ruby-unindent, ruby-web-console, scalapack-doc, scannotation, snow, sorl-thumbnail, svgwrite, systemd-docker, tiles-request, torcs, utf8proc, vagrant-libvirt, voms-api-java, wcwidth, xdffileio, xmlgraphics-commons & yorick.

FTP Team

As a Debian FTP assistant I ACCEPTed 114 packages: apertium-isl-eng, apertium-mk-bg, apertium-urd-hin, apprecommender, auto-apt-proxy, beast-mcmc, caffe, caffe-contrib, debian-edu, dh-make-perl, django-notification, dpkg-cross, elisp-slime-nav, evil-el, fig2dev, file, flightgear-phi, friendly-recovery, fwupd, gcc-5-cross, gdbm, gnustep-gui, golang-github-cznic-lldb, golang-github-dghubble-sling, golang-github-docker-leadership, golang-github-rogpeppe-fastuuid, golang-github-skarademir-naturalsort, golang-glide, gtk+2.0, gtranscribe, kdepim4, kitchen, lepton, libcgi-github-webhook-perl, libcypher-parser, libimporter-perl, liblist-someutils-perl, liblouis, liblouisutdml, libneo4j-client, libosinfo, libsys-cpuaffinity-perl, libtest2-suite-perl, linux, linux-grsec, lua-basexx, lua-compat53, lua-fifo, lua-http, lua-lpeg-patterns, lua-mmdb, lua-openssl, mash, mysql-5.7, node-quickselect, nsntrace, nvidia-graphics-drivers, nvidia-graphics-drivers-legacy-304xx, nvidia-graphics-drivers-legacy-340xx, openorienteering-mapper, oslo-sphinx, p4est, patator, petsc, php-mailparse, php-yaml, pykdtree, pypass, python-bioblend, python-cotyledon, python-jack-client, python-mido, python-openid-cla, python-os-api-ref, python-pydotplus, python-qtconsole, python-repoze.sphinx.autointerface, python-vispy, python-zenoss, r-cran-bbmle, r-cran-corpcor, r-cran-ellipse, r-cran-minpack.lm, r-cran-rglwidget, r-cran-rngtools, r-cran-scatterd3, r-cran-shinybs, r-cran-tibble, reproject, retext, ring, ruby-github-api, ruby-rails-assets-jquery-ui, ruby-swd, ruby-url-safe-base64, ruby-vmstat, ruby-webfinger, rustc, shadowsocks-libev, slepc, staticsite, steam, straight.plugin, svgwrite, tasksh, u-msgpack-python, ufo2otf, user-mode-linux, utf8proc, vizigrep, volk, wchartype, websockify & wireguard.

31 July 2016

Joey Hess: PocketCHIP quick review

PocketCHIP is the pocket sized Linux terminal I always used to want. Which is to say, it runs (nearly) stock Debian, X, etc, it has a physical keyboard, and the hardware and software is (nearly) non-proprietary and very hackable. Best of all, it's fun and it encourages playful learning. It's also clunky and flawed and constructed out of cheap components. This keeps it from being something I'd actually carry around in my pocket and use regularly. The smart thing they've done though is embrace these limitations, targeting it at the hobbiest, and not trying to compete with smart phones. The PocketCHIP is its own little device in its own little niche. PocketCHIP front, side, and back views Unless you're into hardware hacking and want to hook wires up to the GPIO pins, the best hardware feature is the complete keyboard, with even Escape and Control and arrow keys. You can ssh around and run vi on it, run your favorite REPL (I use ghci) to do quick programming, etc. The keyboard is small and a little strange, but you get used to it quickly; your QWERTY muscle memory is transferrable to it. I had fun installing nethack on it and handing it to my sister who had never played nethack before, to watch her learn to play. The screen resolution is 480x272, which is pretty tiny. And, it's a cheap resistive touchscreen, with a bezil around it. This makes it very hard to use scroll bars and icons near the edge of the screen. The customized interface that ships with it avoids these problems, and so I've been using that for now. When I have time, I plan to put a fullscreen window manager on it, and write a pdmenu menu configuration for it, so everything can be driven using the keyboard. I also have not installed Debian from scratch on it yet. This would be tricky because it uses a somewhat patched kernel (to support the display and wifi). The shipped distribution is sadly not entirely free software. There are some nonfree drivers and firmwares. And, they included a non-free gaming environment on it (a very nice one for part of the audience, that allows editing the games, but non-free nevertheless). They did do a good job of packaging up all the custom software they include on it, although they don't seem to have published source packages for everything. (They might be infringing my GPL copyright of flash-kernel by distributing a modified version without source. I say "might" because flash-kernel is a pile of shell scripts, so you could probably extract the (probably trivial) modifications. Still.. Also, they seem to have patched network-manager in some way and I wasn't able to find the corresponding source.) The battery life is around 5 hours. Unfortunately the "sleep" mode only turns off the backlight and maybe wifi, and leaves the rest of the system running. This and the slightly awkward form factor too big to really comfortably fit in a pocket limit the use of PocketCHIP quite a bit. Perhaps the sleeping will get sorted out, and perhaps I'll delete the GPIO breakout board from the top of mine to make it more pocket sized.

29 July 2016

Norbert Preining: TUG 2016 Day 4 Books, ooh Books (and Boats)

Talks have been finished, and as a special present to the participants, Pavneet has organized an excursion that probably was one of the best I ever had. First we visited the Toronto Reference Library where we were treated to a delicious collection of rare books (not to mention all the other books and architecture), and then a trip through the Ismaili Centre Toronto and the Aga Khan Museum.
Page from "A Dream of John Ball", Kelmscott Press Edition, 1892.
(Kelmscott press edition from 1892 of William Morris A Dream of John Ball.) All these places were great pieces of architecture with excellent samples of the writing and printing art. And after all that and not to be mentioned, the conference dinner evening cruise! Our first stop was the Toronto Reference Library. Designed by Raymond Moriyama, it features a large open atrium with skylights, and it gives the library an open and welcoming feeling. We were told that it resembles a tea cup that needs to be filled with knowledge.
The Toronto Reference Library's atrium
The library also features running water at several places the architect had the idea that natural ambient noise is more natural for a library than the eclectic silence that anyway never happens. Originally there were lots of greens hanging down into the Atrium, resembling the Hanging Gardens, but they have been scrapped due to financial reasons. But there are still green oasis like this beautiful green wall in a corner of the library.
Wall of Green in the middle of the library
We were guided first to the fifth floor where the special collection is housed. And what a special collection. The librarian in charge has laid out about 20 exquisite books starting from early illuminated manuscripts over incunabula to high pieces of printing art from the 18th and 19th century. Here we have a illuminated script in Carolingian minuscule.
Illuminated script in Carolingian minuscule
What was really surprising for all of us in this special collection that all these books were simply laid out in front of us, that the librarian touched and used it without gloves, and above all, that he told us that if one wants it is common practice to check out these books for study sessions and enjoy them on the spot in the reading room. I don t know any other library that allows you to actually handle these rare and beauty specimens! The library not only featured lots of great books, it also had some art installation like these light rods.
Art Light installation in the Toronto Reference Library
In one of the books I found by chance a map of my hometown of Vienna. Looking at this map from very old times, the place where I grew up is still uninhabited somewhere in the far upper right corner of the map. Times have changed.
Map op Vienna found in the Toronto Reference Library
After we left this open and welcoming treasure house of beautiful books, we moved to the Aga Khan Museum and Ismaili Centre Toronto, which are standing face-to-face separated by some water ponds in the Aga Khan park a bit outside of central Toronto. Here we see the Ismaili Centre from the Aga Khan Museum entrance. The big glass dome is the central prayer room, and is illuminated at night. Just one detail one can see in the outer wall one part that looks like glass, too. This is the prayer alcove in the back of the prayer hall, and is made from huge slabs of Onyx that are also lit up in the night.
View onto the Ismaili Centre's Prayer Hall formed by a glass dome
The Ismaili Centre, designed by Charles Correa combines modern functional and simple style with the wonderful ornamental art of the Islam heritage. The inside of the Ismaili Centre features many pieces of exquisite art calligraphy, murals, stone work, etc here is a medallion made from precious stone and set onto a hand-carved wall.
Medaillon made of precious stones in hand carved wall, Ismaili Centre Toronto
A calligraphy on the wall in the Ismaili Centre
Wall Calligraphy in the Ismaili Centre Toronto
Following the Ismaili Centre we turned to the Aga Khan museum which documents Islamic art, science, and history with an extensive collection. We didn t have much time, and in addition I had to do some fire-fighting over the phone, but the short trip through the permanent collection with samples of excellent calligraphy was amazing.
Koran Calligraphy, Aga Khan Museum Toronto
After returning from this lovely excursion and a short break, we set off for the last stop for tonight, the dinner cruise. After a short bus ride we could board our ship and off we go. Although the beer selection was not on par with what we are used from carft breweries, the perfectly sized boat with two decks and lots of places to hang around invited us to many discussions and chitchats. And finally I could enjoy also the skyline of Toronto.
View onto Toronto from the boat
After the dinner we had some sweets, one of which was a specially made cake with the TUG 2016 logo on it. I have to say, it was not only this cake but the whole excellent and overboarding food we had during all these days, that will make me go on diet when I am back in Japan. Pavneet organized for the lunch breaks three different style of kitchens (Thai, Indian, Italian), then the excursions to local brewers and and and If it wouldn t be for TeX, I would call it a Mastkur .
TUG 2016 cake
During the cruise we also had a little ceremony thanking Jim for his work as president of the TUG, but above all Pavneet for this incredible well organized conference. I think everyone agreed that this was the best TUG conference since long.
Sunset near Toronto
pensDuring the ceremony, Pavneet also announced the winners of the TUG 2016 fountain pen auction. These pens have a long history/travel behind them, see details on the linked page, and were presented to the special guests of the conference. Two remaining pens were auctioned with funds going to the TUG. The first one was handed over to Steve Grathwohl, and to my utter surprise the second one to myself. So now I am a happy owner of a TUG 2016 fountain pen. What a special feature! Just one more detail about these pens: They are traditional style, so without ink capsules, but one needs to insert the ink with a syringe. I guess I need to stack up a bit at home, and more importantly, train my really ugly hand-writing, otherwise it would be a shame to use this exquisite tool. We returned to the harbor around 10pm, and back to the hotel, where there was much greeting and thanking, as many people will return the following day.
Heading back to Toronto
I will also leave on Friday morning to meet with friends, thus I will not be participating in (and not reporting on) the last excursion of the TUG 2016. I will leave Toronto and the TUG 2016 with (nearly) exclusively good memories of excellent talks, great presentations, wonderful excursions, and lots of things I have learned. I hope to see all of the participants on next year s TUG meeting and I hope I will be able to attend it. Thanks a lot to Pavneet, you have done an incredible job. And last but not least, thanks to your lovely wife for letting you do all this, I know how much time we did steal from her. A few more photos can be found at the album Day 4 Books, ooh books.

Dirk Eddelbuettel: RcppGetconf 0.0.1

A new package RcppGetconf to read system configuration --- not unlike getconf from the libc library. Now R can read what system calls sysconf, pathconf and confstr have to say. The package is still pretty green, and now on CRAN in a very first version, corresponding to a very first (and single !) commit. Right now, the CRAN just has one function getAll() similar to getconf -a. A first example shows how it provides all values which can be retried -- currently 320 on my systems.
R> res <- getAll()
R> head(res)
               key value type
1         LINK_MAX 65000 path
2  _POSIX_LINK_MAX 65000 path
3        MAX_CANON   255 path
4 _POSIX_MAX_CANON   255 path
5        MAX_INPUT   255 path
6 _POSIX_MAX_INPUT   255 path
R> tail(res)
                      key  value type
315    LEVEL4_CACHE_ASSOC      0  sys
316 LEVEL4_CACHE_LINESIZE      0  sys
317                  IPV6 200809  sys
318           RAW_SOCKETS 200809  sys
319           _POSIX_IPV6 200809  sys
320    _POSIX_RAW_SOCKETS 200809  sys
R> 
Earlier this evening I added a second function to the GitHub repo which can access indivial values. But right now, the biggest need is really for someone with some systems skills---and an OS X machine---to look at the code, and maybe the getconf.c from the C library, in order to make this build OS X. If you can help, please get in touch. More about the package is at the local RcppGetconf page and the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

28 July 2016

Norbert Preining: TUG 2016 Day 3 Stories and Histories

The last day of TUG 2016, or at least the last day of talks, brought four one-hour talks from special guests, and several others, where many talks told us personal stories and various histories.
tug2016-biglow A great finish of a great conference. Jennifer Claudio The case for justified text Due to a strange timezone bug in my calendar program, I completely overslept a morning meeting and breakfast, as well as the first talk, so unfortunately I don t have anything to report about this surely intersting talk comparing justification in various word processors and TeX. Leyla Akhmadeeva, Rinat Gizatullin, Boris Veytsman Are justification and hyphenation good or bad for the reader? Still half dizzy and without coffee, I couldn t really follow this talk, and only woke up till the end when there was a lot of interesting discussion about speed reading and its non-existence (because it is simply skimming over text), and improvements on reading comprehension. Charles Bigelow Looking for legibility Another special guest, Charles Bigelow, presented a huge pool of research and work into readability, and how attitude and usage of fonts change over time. A very involving and well laid out talk, full of interesting background images and personal opinions and thoughts. Charles also touched onto topics of readability on modern devices like e-readers and mobiles. He compared the recent developments in font design for mobile devices with their work on Lucida 20+ years ago, and concluded that both arrived at the same solutions. A very educating and amusing talk packed full with information on readability. I surely will revisit the recording in a study session. David Walden Some notes on the history of digital typography David touches on many topics of the history of digital typography which he has experienced himself over the years: First the development of newspaper production and printing, then the evolvement of editors from simple text editors over word processors to full-fledged DTP programs. Finally he touches on various algorithmic problems that appear in the publishing business. Tim Inkster The beginning of my career Tim, our fanatastic guide through his print shop the Procupine s Quill on the second excursion day, talked about his private ups and downs in the printing business, all filled with an infinite flow of funny stories and surprising anecdotes. Without slides and anything but his voice and his stories, he kept us hanging on his lips without a break. I recommend watching the recording of his talk because one cannot convey the funny comments and great stories he shared with us in this simple blog. Joe Clark Type and tiles on the TTC Joe unveils the history of rise and fall of the underground types and
tiles in Toronto. It is surprising to me that a small metro network as in Toronto can have such a long history of changes of design, layout, presentation. Some of the photos completely stymed me how can anyone put up signs like that? I was thinking. To quote Joe (hopefully I remember correctly):
You see what happens without adult supervision.
Abdelouahad Bayar Towards an operational (La)TeX package supporting optical scaling of dynamic mathematical symbols A technical talk about a trial in providing optical scaling of mathematical symbols. As far as I understand it tries to improve on the TeX way of doing extensible math symbols by glueing things together. It seems to be highly involved and technically interesting project, but I couldn t completely grasp the aim of it. Michael Cohen, Blanca Mancilla, John Plaice Zebrackets: A score of years and delimiters John introduced us to Zebrackets, stripped parentheses and brackets, to help us keep track of pairing of those beasts. But as we know, Zebras are very elusive animals, and so we saw lots of stripped brackets around. The idea of better markup of matching parentheses is definitely worth developing. Charles Bigelow Probably approximately not quite correct: Revise, repeat The second talk of Charles, this time on the history of the Lucida fonts, from the early beginnings drawn on graph paper to recent developments using FontLab producing OpenType fonts. A truly unique crash course through the development of one of the very big families of fonts, and one of the first outside Computer Modern that had also support for proper math typesetting in TeX.
Aggressively legible!
One of the key phrases that popped up again and again was aggressively legible, mostly in negative connotations as far to fat symbols or far to big Arabic letters. But for me this font family is still close to my heart. I purchased it back than from Y&Y for my PhD thesis, and since then have upgraded to the TUG version including the OpenType fonts, and I use them for most of my presentation. Maybe I like the aggressive legibility! Chuck slided in lots of nice comments about Kris Holmes, the development practice in their cooperation, stories of business contacts, and many more, making this talk a very lively and amusing, and at the same time very educating talk.
This concluded the TUG conference talks, and we thanked Pavneet for his excellent organization. But since we still have up to two days of excursions, many people dispersed quickly, just to meet again for a optional Type and Tile Tour 3-5 subway stops with discussion of typesetting This guided tour through the underground of Toronto, guided by Joe Clark who spoke in the morning about this topic, was attended by far too many participants. I think there were around 25 when we left. I thought that this will not work out properly, and so decided to leave the group and wander around alone. The last program point for today was dinner with a blues music concert in the near by Jazz Bistro:
jazz-bistro Excellent life music in a bit schick/sophisticated atmosphere was a good finish for this excellent day. With Herb from MacTeX and his wife we killed two bottles of red wine, before slowly tingling back to the hotel. A great finishing day of talks.

27 July 2016

Norbert Preining: TUG 2016 Day 2 Figures to Fonts

The second day of TUG 2016 was again full of interesting talks spanning from user experiences to highly technical details about astrological chart drawing, and graphical user interfaces to TikZ to the invited talk by Robert Bringhurst on the Palatino family of fonts. tug2016-bringhurst With all these interesting things there is only one thing to compain I cannot get out of the dark basement and enjoy the city After a evening full of sake and a good night s sleep we were ready to dive into the second day of TUG. Kaveh Bazargan A graphical user interface for TikZ The opening speaker of Day 2 was Kaveh. He first gave us a quick run-down on what he is doing for business and what challenges publishers are facing in these times. After that he introduced us to his new development of a command line graphical user interface for TikZ. I wrote command line on purpose, because the editing operations are short commands issued on a kind of command line, which will give an immediate graphical feedback. Basic of the technique is a simplified TikZ-like meta language that is not only easy to write, but also easy to parse. While the amount of supported commands and features of TikZ is still quite small, I think the basic idea is a good one, and there is a good potential in it. Matthew Skala Astrological charts with horoscop and starfont Next up was Matthew who introduced us to the involved task of typesetting astrological charts. He included comparisons with various commercial and open source solutions, where Matthew of course, but me too, felt that his charts came of quite well! As an extra bonus we got some charts of famous singers, as well as the TUG 2016 horoscope. David Tulett Development of an e-textbook using LaTeX and PStricks David reported on his project to develop an e-textbook on decision modeling (lots of math!) using LaTeX and PStricks. His e-book is of course a PDF. There were a lot of very welcoming feedback free (CC-BY-NC-ND) textbooks for sciences are rare and we need more of them. Christian Gagn An Emacs-based writing workflow inspired by TeX and WEB, targeting the Web Christian s talk turned around editing and publishing using org-mode of Emacs and the various levels of macros one can use in this setup. He finished with a largely incomprehensible vision of a future equational logic based notation mode. I have used equational logic in my day-in-day-out job, and I am not completely convinced that this is a good approach for typesetting and publishing but who knows, I am looking forward to a more logic-based approach! Barbara Beeton, Frank Mittelbach In memoriam: Sebastian Rahtz (1955-2016) Frank recalled Sebastian s many contribution to a huge variety of fields, and recalled our much missed colleague with many photos and anecdotes. Jim Hefferon A LaTeX reference manual Jim reported about the current state of a LaTeX reference manual, which tries to provide a documentation orthogonally to the many introduction and user guides available, by providing a straight down-to-earth reference manual with all the technical bells and whistles necessary. As I had to write myself a reference manual for a computer language, it was very interested to see how they dealt with many of the same problems I am facing. Arthur Reutenauer, Mojca Miklavec Hyphenation past and future: hyph-utf8 and patgen Arthur reports about the current statue of the hyphenation pattern project, and in particular the license and usage hell they recently came into with large cooperations simply grabbing the patterns without proper attribution. In a second part he gave a rough sketch of his shot at a reimplementation of patgen. Unfortunately he wrote in rather unreadable hand-writing on a flip-chart, which made only the first line audience to actually see what he was writing. Federico Garcia-De Castro TeXcel? As an artist organizing large festivals Federico has to fight with financial planning and reports. He seemed not content with the abilities of the usual suspects, so he developed a way to do Excel like book-keeping in TeX. Nice idea, I hope I can use that system for the next conference I have to organize! Jennifer Claudio A brief reflection on TeX and end-user needs Last speaker in the morning session was Jennifer who gave us a new and end-user s view onto the TeX environment, and the respective needs. These kind of talks are a very much welcomed contrast to technical talks and hopefully all of us developers take home some of her suggestions. Sungmin Kim, Jaeyoung Choi, Geunho Jeong MFCONFIG: Metafont plug-in module for the Freetype rasterizer Jaeyoung reported about an impressive project to make Metafont fonts available to fontconfig and thus windowing systems. He also explained their development of a new font format Stemfont, which is a Metafont-like system that can work also for CJK fonts, and which they envisage to be built into all kind of mobile devices. Michael Sharpe New font offerings Cochineal, Nimbus15 and LibertinusT1Math Michael reports about his last font projects. The first two being extensions of the half-made half-butchered rereleased URW fonts, as well as his first (?) math font project. I talked to him over lunch one day, and asked him how many man-days he need for these fonts, and his answer was speaking a lot: For the really messed up new URW fonts, like Cochineal, he guessed about 5 man-months of work, while other fonts only needed a few days. I think we all can be deeply thankful to all the work he is investing into all these font projects. Robert Bringhurst The evolution of the Palatino tribe The second invited talk was Robert Bringhurst, famous for his wide contributions to typpography, book culture in general, as well as poetry. He gave a quick historic overview on the development of the Palatino tribe of fonts, with lots of beautiful photos. I was really looking forward to Robert s talk, and my expectations were extremely high. And unfortunately I must say I was quite disappointed. Maybe it is his style of presentation, but the feeling he transfered to me (the audience?) was that he was going through a necessary medical check, not much enjoying the presentation. Also, the content itself was not really full of his own ideas or thoughts, but a rather superficial listing of historical facts. Of course, a person like Robert Bringhurst is so full of anecdotes and background knowledge still was a great pleasure to listen and lots of things to learn, I only hoped for a bit more enthusiasm. TUG Annual General Meeting The afternoon session finished with the TUG Annual General Meeting, reports will be sent out soon to all TUG members. Herbert Schulz Optional workshop: TeXShop tips & tricks After the AGM, Herbert from MacTeX and TeXShop gave an on-the-spot workshop on TeXShop. Since I am not a Mac user, I skipped on that.
Another late afternoon program consisted of an excursion to Eliot s bookshop, where many of us stacked up on great books. This time again I skipped and took a nap. In the evening we had a rather interesting informal dinner in the food court of some building, where only two shops were open and all of us lined up in front of the Japanese Curry shop, and then gulped down from plastic boxes. Hmm, not my style I have to say, not even for informal dinner. But at least I could meet up with a colleague from Debian and get some gpg key signing done. And of course, talking to all kind of people around. The last step for me was in the pub opposite the hotel, with beer and whiskey/scotch selected by specialists in the field.

26 July 2016

Norbert Preining: TUG 2016 Day 1 Routers and Reading

The first day of the real conference started with an excellent overview of what one can do with TeX, spanning from traditional scientific journal styles to generating router configuration for cruising ships.
tug2016-color All this was crowned with an invited talk my Kevin Larson from Microsoft s typography department on how to support reading comprehension. Pavneet Aurora Opening: Passport to the TeX canvas Pavneet, our never-sleeping host and master of organization, opened the conference with a very philosophical introduction, touching upon a wide range of topics ranging from Microsoft, Twitter to the beauty of books, pages, and type. I think at some point he even mentioned TeX, but I can t remember for sure. His words put up a very nice and all-inclusive stage, a community that is open to all kind of influences with any disregard or prejudice. Let us hope that is reflects reality. Thanks Pavneet. Geoffrey Poore Advances in PythonTeX Our first regular talk was by Geoffrey reporting on recent advances in PythonTeX, a package that allows including python code in your TeX document. Starting with an introduction to PythonTeX, Geoggrey reports about an improved verbatim environment, fvextra, which patches fancyvrb, and improved interaction between tikz and PythonTeX. As I am a heavy user of listings for my teaching on algebraic specification languages, I will surely take a look at this package and see how it compares to listings. Stefan Kottwitz TeX in industry I: Programming Cisco network switches using TeX Next was Stefan from Lufthansa Industry Solutions, who reported first about his working environment, Cruise Ships with a very demanding IT infrastructure he has to design and implement. Then he introduced us to his way of generating IP configurations for all the devices using TeX. The reason he chose this method is that it allows him to generate at the same time proper documentation. It was surprising for me to hear that by using TeX he could far more efficiently and quicker produce well designed and easily accessible documentation, which helped both the company as well as made the clients happy! Stefan Kottwitz TeX in industry II: Designing converged network solutions After a coffee break, Stefan continued his exploration into industrial usage of TeX, this time about using tikz to generate graphics representing the network topology on the ships. Boris Veytsman Making ACM LaTeX styles Next up was Boris which brought us back to traditional realms of TeX when he guided us into the abyss of ACM LaTeX styles he tried to maintain for some time, until he plunged into a complete rewrite of the styles. Frank Mittelbach Alice goes floating global optimized pagination including picture placements The last talk before lunch (probably a strategic placement, otherwise Frank would continue for hours and hours) was Frank on global optimization of page breaks. Frank showed us what can and can not be done with current LaTeX, and how to play around with global optimization of pagination, using Alice in Wonderland as running example. We can only hope that his package is soon available in an easily consumable version to play around. Thai lunch Pavneet has organized three different lunch-styles for the three days of the conference, today s was Thai with spring rools, fried noodles, one kind of very interesting orange noodles, and chicken something. Michael Doob baseball rules summary After lunch Michael gave us an accessible explanation of the most arcane rules a game can have the baseball rules by using pseudo code. I think the total number of loc needed to explain the overall rules would fill more pages than the New York phonebook, so I am deeply impressed by all those who can understand these rules. Some of us even wandered off in the late afternoon to see a match with life explanations of Michael. Amartyo Banerjee, S.K. Venkatesan A Telegram bot for printing LaTeX files Next up was Amartyo who showed a Telegram (as in messenger application) bot running on a Raspberry Pi, that receives (La)TeX files and sends back compiled PDF files. While it is not ready for consumption (If you sneeze the bot will crash!), it looks like a promising application. Furthermore, it is nice to see how open APIs (like Telegram) can spur development of useful tools, while closed APIs (including threatening users, like WhatApp) hinders it. Norbert Preining Security improvements in the TeX Live Manager and installer Next up was my own talk about beefing up the security of TeX Live by providing integrity and authenticity checks via GnuPG, a feature that has been introduced with the recent release of TeX Live 2016. The following discussion gave me several good idea on how to further improve security and usability. Arthur Reutenauer -The TeX Live M sub-project (and open discussion) Arthur presented the TeX Live M (where the M stands supposedly for Mojca, who couldn t attend unfortunately) project: Their aim is to provide a curated and quality verified sub-part of TeX Live that is sufficiently complete for many applications, and easier for distributors and packagers. We had a lively discussion after Arthur s short presentation, mostly about why TeX Live does not have a on-the-fly installation like MikTeX. I insisted that this is already possible, using the tex-on-the-fly package which uses the mktextex infrastructure, but also caution against using it by default due to delays induced by repeatably reading the TeX Live database. I think this is a worth-while project for someone interested in learning the internals of TeX Live, but I am not sure whether I want to invest time into this feature. Another discussion point was about a testing infrastructure, which I am currently working on. This is in fact high on my list, to have some automatic minimal functionality testing a LaTeX package should at least load! Kevin Larson Reading between the lines: Improving comprehension for students Having a guest from Microsoft is rather rare in our quite Unix-centered environment, so big thanks to Pavneet again for setting up this contact, and big thanks to Kevin for coming. Kevin gave us a profound introduction to reading disabilities and how to improve reading comprehension. Starting with an excursion into what makes a font readable and how Microsoft develops optimally readable fonts, he than turned to reading disabilities like dyslexia, and how markup of text can increase students comprehension rate. He also toppled my long-term believe that dyslexia is connected to the similar shape of letters which are somehow visually malprocessed this was the scientific status from the 1920ies till the 70ies, but since then all researchers have abandoned this interpretation and dyslexia is now linked to problems linking shape to phonems. Kevin did an excellent job with a slightly difficult audience some people picking about grammer differences between British and US English and permanently derailing the discussion, and even more the high percentage of typographically somehow specially tasted participants. After the talk I had a lengthy discussion with Kevin about if/how this research can be carried over to non-Roman writing systems, in particular Kanji/Hanzi based writing systems, where dyslexia shows itself probably in different context. Kevin also mentioned that they want to add interword space to Chinese to help learners of Chinese (children, foreigners) to better parse, and studies showed that this helps a lot in comprehension. On a meta-level, this talk bracketed with the morning introduction by Pavneet, describing an open environment with stimulus back and forth in all directions. I am very happy that Kevin took the pain to come in his tight schedule, and I hope that the future will bring better cooperation at the end we are all working somehow on the same front only the the tools differ.
izakaya-sake-partyAfter the closing of the session, one part of our group went off to the baseball match, while another group dived into a Japanese-style Izakaya where we managed to kill huge amounts of sake and quite an amount of food. The photo shows me after the first bottle of sake, while just seeping on an intermediate small amount of genshu (kind of strong undiluted sake) before continuing to the next bottle. An interesting and stimulating first day of TUG, and I am sure that everyone was looking forward to day 2.

25 July 2016

Norbert Preining: TUG 2016 Day 0 Books and Beers

The second pre-conference day was dedicated to books and beers, with a visit to an exquisite print studio, and a beer tasting session at one of the craft breweries in Canada. In addition we could grab a view into the Canadian lifestyle by visiting Pavneet s beautiful house in the countryside, as well as enjoying traditional style pastries from a bakery.
Heidelberg printing machine at Porcupine's quilt
In short, a perfect combination for us typography and beer savvy freaks! This morning we had somehow an early start from the hotel. Soon the bus left downtown Toronto and entered countryside Ontario, large landscapes filled with huge (for my Japanese feeling) estates and houses, separated by fields, forests and wild landscape. Very beautiful and inviting to live there. On our way to the printing workshop we stopped at Pavneet s house for a very short visit of the exterior, which includes mathematics in the bricking. According to Pavneet, his kids hate to see math on the wall I would be proud to have it.
Pavneet's house is hiding some mathematics
A bit further on we entered into Erin, where the Porcupine s Quill is located. A small building along the street, one could easily oversee that rare jewel! In addition considering that according to the owners, Google Maps has a bad error which would lead you to a completely different location. This printing workshop, led by Tim and Elke Inkster, is producing books in a traditional style using an old Heidelberg Offset printing machine.
Entrance to the Porcupine's Quill, a local bookshop doing excellent printing
Elke introduced us to the sewing of folded signatures together with a lovely old sewing machine. It was the first time I actually saw one in action.
Sewn signatures
Tim, the head master of the printing shop, first entertained us with stories about Chinese publishers visiting them in the old cold-war times, before diving into explanations of the actual machines around, like the Heidelberg offset printing machine.
Master Tim is showing us offset technique
In the back of the basement of the little studio there is also a huge folding machine, which cuts up the big signatures of 16 pages and folds them into bundles. An impressive example of tricky engineering.
The folding machine creates from a printed signature 16 pages in proper order.
Due to the small size of the printing studio, our groups were actually split into two groups, and while the other group got its guided tour, we grabbed coffee and traditional cookies and pastries from the nearby Holtom s bakery. Loads of nice pastries with various filling, my favorite being the slightly salty cherry pie, and above all the rhubarb-raspberry pie.
Nearby old-style bakery, selling Viennese-style "Kaisersemmel", calling them "Kaiser buns". There must be an Austrian hiding somewhere around.
To my absolute astonishment I also found there a Viennese Kaisersemmel , called Kaiser bun here, but keeping the shape and the idea (but unfortunately not the crispy cracky quality of the original in Vienna). Of course I got two of them, together with a nice jam from the region, and enjoyed these Viennese breakfast the next day morning.
Viennese breakfast from the Bakery near Porcupine's Quill
Leaving the Quill we headed for a lunch in a nice pizzeria (I got Pizza Toscana) which also served excellent local beer how would I like to have something like this over in Japan! Our last stop on this excursion was Stone Hammer Brewery, ne of the most famous craft breweries in Canada.
One of the top craft breweries in Canada, the Stone Hammer
keep-calmAlthough they wont win a prize for typography (besides one page of a coaster there that carried a nice pun), their beers are exquisite. We got five different beers to taste, plus extensive explanations on brewing methods and differences. Now I finally understand why most of the new craft breweries in Japan are making Ales: Ales don t need a long process and are ready for sale in rather short time, compared to e.g. lagers.)
Explanations of the the secrets of beer brewing
Nothing to add to this poster found in the Stone Hammer Brewery!
Also at the Stone Hammer Brewery I spotted this very nice poster on the wall of the toilet. And I cannot agree more, everything can easily be discussed over a good beer it calms down aversions, makes even the worst enemies friends, and is healthy for both the mind and body. Filled with excellent beer, some of us (notably an unnamed US TeXnician and politician), stacked up on beers to carry home. I was very tempted to get a huge batch, but putting cans into an airplane seems to be not the optimal idea. Since we are talking cans, I was surprised to hear that many craft beer brewers nowadays prefer cans due to their better protection of the beer from light and oxygen, both killers of good beer. Before leaving we took a last look at the Periodic Table of Beer Types, which left me in awe about how much I don t know and I probably never will know. In particular, I heard the first time of a Vienna style beer Vienna is not really famous for beer, better to say, it is infamous. So maybe this is a different Vienna than my home town that is meant here.
Lots to study here, the Periodic Table of Beers
Another two hour bus ride brought us back to Toronto, where we met with other participants at the reception in a restaurant of Mediterranean cuisine, where I could enjoy for the first time in years a good Tahina and Humus. All around another excellent day, now I only would like to have two days of holidays, guess I need to relax in the lectures starting from tomorrow.

24 July 2016

Dirk Eddelbuettel: RcppArmadillo 0.7.200.2.0

armadillo image The second Armadillo release of the 7.* series came out a few weeks ago: version 7.200.2. And RcppArmadillo version 0.7.200.2.0 is now on CRAN and uploaded to Debian. This followed the usual thorough reverse-dependecy checking of by now over 240 packages using it. For once, I let it simmer a little preparing only a package update via the GitHub repo without preparing a CRAN upload to lower the update frequency a little. Seeing that Conrad has started to release 7.300.0 tarballs, the time for a (final) 7.200.2 upload was now right. Just like the previous, it now requires a recent enough compiler. As g++ is so common, we explicitly test for version 4.6 or newer. So if you happen to be on an older RHEL or CentOS release, you may need to get yourself a more modern compiler. R on Windows is now at 4.9.3 which is decent (yet stable) choice; the 4.8 series of g++ will also do. For reference, the current LTS of Ubuntu is at 5.4.0, and we have g++ 6.1 available in Debian testing. This new upstream release adds new indexing helpers, additional return codes on some matrix transformations, increased speed for compound expressions via vectorise, corrects some LAPACK feature detections (affecting principally complex number use under OS X), and a rewritten sample() function thanks to James Balamuta. Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. Changes in this release (and the preceding GitHub-only release 0.7.200.1.0 are as follows:
Changes in RcppArmadillo version 0.7.200.2.0 (2016-07-22)
  • Upgraded to Armadillo release 7.200.2
  • The sampling extension was rewritten to use Armadillo vector types instead of Rcpp types (PR #101 by James Balamuta)
Changes in RcppArmadillo version 0.7.200.1.0 (2016-06-06)
  • Upgraded to Armadillo release 7.200.1
    • added .index_min() and .index_max()
    • expanded ind2sub() to handle vectors of indices
    • expanded sub2ind() to handle matrix of subscripts
    • expanded expmat(), logmat() and sqrtmat() to optionally return a bool indicating success
    • faster handling of compound expressions by vectorise()
  • The configure code now (once again) sets the values for the LAPACK feature #define correctly.
Courtesy of CRANberries, there is a diffstat report. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Russ Allbery: Review: The Run of His Life

Review: The Run of His Life, by Jeffrey Toobin
Publisher: Random House
Copyright: 1996, 1997
Printing: 2015
ISBN: 0-307-82916-2
Format: Kindle
Pages: 498
The O.J. Simpson trial needs little introduction to anyone who lived through it in the United States, but a brief summary for those who didn't. O.J. Simpson is a Hall of Fame football player and one of the best running backs to ever play the game. He's also black, which is very relevant much of what later happened. After he retired from professional play, he became a television football commentator and a spokesperson for various companies (particularly Hertz, a car rental business). In 1994, he was arrested for the murder of two people: his ex-wife, Nicole Brown Simpson, and Ron Goldman (a friend of Nicole's). The arrest happened after a bizarre low-speed police chase across Los Angeles in a white Bronco that was broadcast live on network television. The media turned the resulting criminal trial into a reality TV show, with live cable television broadcasts of all of the court proceedings. After nearly a full year of trial (with the jury sequestered for nine months more on that later), a mostly black jury returned a verdict of not guilty after a mere four hours of deliberation. Following the criminal trial, in an extremely unusual legal proceeding, Simpson was found civilly liable for Ron Goldman's death in a lawsuit brought by his family. Bizarre events surrounding the case continued long afterwards. A book titled If I Did It (with "if" in very tiny letters on the cover) was published, ghost-written but allegedly with Simpson's input and cooperation, and was widely considered a confession. Another legal judgment let the Goldman family get all the profits from that book's publication. In an unrelated (but also bizarre) incident in Las Vegas, Simpson was later arrested for kidnapping and armed robbery and is currently in prison until at least 2017. It is almost impossible to have lived through the O.J. Simpson trial in the United States and not have formed some opinion on it. I was in college and without a TV set at the time, and even I watched some of the live trial coverage. Reactions to the trial were extremely racially polarized, as you might have guessed. A lot of black people believed at the time that Simpson was innocent (probably fewer now, given subsequent events). A lot of white people thought he was obviously guilty and was let off by a black jury for racial reasons. My personal opinion, prior to reading this book, was a common "third way" among white liberals: Simpson almost certainly committed the murders, but the racist Los Angeles police department decided to frame him for a crime that he did commit by trying to make the evidence stronger. That's a legitimate reason in the US justice system for finding someone innocent: the state has an obligation to follow correct procedure and treat the defendant fairly in order to get a conviction. I have a strong bias towards trusting juries; frequently, it seems that the media second-guesses the outcome of a case that makes perfect sense as soon as you see all the information the jury had (or didn't have). Toobin's book changed my mind. Perhaps because I wasn't watching all of the coverage, I was greatly underestimating the level of incompetence and bad decision-making by everyone involved: the prosecution, the defense, the police, the jury, and the judge. This court case was a disaster from start to finish; no one involved comes away looking good. Simpson was clearly guilty given the evidence presented, but the case was so badly mishandled that it gave the jury room to reach the wrong verdict. (It's telling that, in the far better managed subsequent civil case, the jury had no trouble reaching a guilty verdict.) The Run of His Life is a very detailed examination of the entire Simpson case, from the night of the murder through the end of the trial and (in an epilogue) the civil case. Toobin was himself involved in the media firestorm, breaking some early news of the defense's decision to focus on race in The New Yorker and then involved throughout the trial as a legal analyst, and he makes it clear when he becomes part of the story. But despite that, this book felt objective to me. There are tons of direct quotes, lots of clear description of the evidence, underlying interviews with many of the people involved to source statements in the book, and a comprehensive approach to the facts. I think Toobin is a bit baffled by the black reaction to the case, and that felt like a gap in the comprehensiveness and the one place where he might be accused of falling back on stereotypes and easy judgments. But other than hole, Toobin applies his criticism even-handedly and devastatingly to all parties. I won't go into all the details of how Toobin changed my mind. It was a cumulative effect across the whole book, and if you're curious, I do recommend reading it. A lot was the detailed discussion of the forensic evidence, which was undermined for the jury at trial but looks very solid outside the hothouse of the case. But there is one critical piece that I would hope would be handled differently today, twenty years later, than it was by the prosecutors in that case: Simpson's history of domestic violence against Nicole. With what we now know about patterns of domestic abuse, the escalation to murder looks entirely unsurprising. And that history of domestic abuse was exceedingly well-documented: multiple external witnesses, police reports, and one actual prior conviction for spousal abuse (for which Simpson did "community service" that was basically a joke). The prosecution did a very poor job of establishing this history and the jury discounted it. That was a huge mistake by both parties. I'll mention one other critical collection of facts that Toobin explains well and that contradicted my previous impression of the case: the relationship between Simpson and the police. Today, in the era of Black Lives Matter, the routine abuse of black Americans by the police is more widely known. At the time of the murders, it was less recognized among white Americans, although black Americans certainly knew about it. But even in 1994, the Los Angeles police department was notorious as one of the most corrupt and racist big-city police departments in the United States. This is the police department that beat Rodney King. Mark Fuhrman, one of the police officers involved in the case (although not that significantly, despite his role at the trial), was clearly racist and had no business being a police officer. It was therefore entirely believable that these people would have decided to frame a black man for a murder he actually committed. What Toobin argues, quite persuasively and with quite a lot of evidence, is that this analysis may make sense given the racial tensions in Los Angeles but ignores another critical characteristic of Los Angeles politics, namely a deference to celebrity. Prior to this trial, O.J. Simpson largely followed the path of many black athletes who become broadly popular in white America: underplaying race and focusing on his personal celebrity and connections. (Toobin records a quote from Simpson earlier in his life that perfectly captures this approach: "I'm not black, I'm O.J.") Simpson spent more time with white businessmen than the black inhabitants of central Los Angeles. And, more to the point, the police treated him as a celebrity, not as a black man. Toobin takes some time to chronicle the remarkable history of deference and familiarity that the police showed Simpson. He regularly invited police officers to his house for parties. The police had a long history of largely ignoring or downplaying his abuse of his wife, including not arresting him in situations that clearly seemed to call for that, showing a remarkable amount of deference to his side of the story, not pursuing clear violations of the court judgment after his one conviction for spousal abuse, and never showing much inclination to believe or protect Nicole. Even on the night of the murder, they started following a standard playbook for giving a celebrity advance warning of investigations that might involve them before the news media found out about them. It seems clear, given the evidence that Toobin collected, that the racist Los Angeles police didn't focus that animus at Simpson, a wealthy celebrity living in Brentwood. He wasn't a black man in their eyes; he was a rich Hall of Fame football player and a friend. This obviously raises the question of how the jury could return an innocent verdict. Toobin provides plenty of material to analyze that question from multiple angles in his detailed account of the case, but I can tell you my conclusion: Judge Lance Ito did a horrifically incompetent job of managing the case. He let the lawyers wander all over the case, interacted bizarrely with the media coverage (and was part of letting the media turn it into a daytime drama), was not crisp or clear about his standards of evidence and admissibility, and, perhaps worst of all, let the case meander on at incredible length. With a fully sequestered jury allowed only brief conjugal visits and no media contact (not even bookstore shopping!). Quite a lot of anger was focused on the jury after the acquittal, and I do think they reached the wrong conclusion and had all the information they would have needed to reach the correct one. But Toobin touches on something that I think would be very hard to comprehend without having lived through it. The jury and alternate pool essentially lived in prison for nine months, with guards and very strict rules about contact with the outside world, in a country where compensation for jury duty is almost nonexistent. There were a lot of other factors behind their decision, including racial tensions and the sheer pressure from judging a celebrity case about which everyone has an opinion, but I think it's nearly impossible to underestimate the psychological tension and stress from being locked up with random other people under armed guard for three quarters of a year. It's hard for jury members to do an exhaustive and careful deliberation in a typical trial that takes a week and doesn't involve sequestration. People want to get back to their lives and families. I can only imagine the state I would be in after nine months of this, or how poor psychological shape I would be in to make a careful and considered decision. Similarly, for those who condemned the jury for profiting via books and media appearances after the trial, the current compensation for jurors is $15 per day (not hour). I believe at the time it was around $5 per day. There are a few employers who will pay full salary for the entire jury service, but only a few; many cap the length at a few weeks, and some employers treat all jury duty as unpaid leave. The only legal requirement for employers in the United States is that employees that serve on a jury have their job held for them to return to, but compensation is pathetic, not tied to minimum wage, and employers do not have to supplement it. I'm much less inclined to blame the jurors than the system that badly mistreated them. As you can probably tell from the length of this review, I found The Run of His Life fascinating. If I had followed the whole saga more closely at the time, this may have been old material, but I think my vague impressions and patchwork of assumptions were more typical than not among people who were around for the trial but didn't invest a lot of effort into following it. If you are like me, and you have any interest in the case or the details of how the US criminal justice system works, this is a fascinating case study, and Toobin does a great job with it. Recommended. Rating: 8 out of 10

22 July 2016

Dirk Eddelbuettel: RcppCCTZ 0.0.5

Version 0.0.5 of RcppCCTZ arrived on CRAN a couple of days ago. It reflects an upstream fixed made a few weeks ago. CRAN tests revealed that g++-6 was tripping over one missing #define; this was added upstream and I subsequently synchronized with upstream. At the same time the set of examples was extended (see below). Somehow useR! 2016 got in the way and while working on the then-incomplete examples during the traveling I forgot to release this until CRAN reminded me that their tests still failed. I promptly prepared the 0.0.5 release but somehow failed to update NEWS files etc. They are correct in the repo but not in the shipped package. Oh well. CCTZ is a C++ library for translating between absolute and civil times using the rules of a time zone. In fact, it is two libraries. One for dealing with civil time: human-readable dates and times, and one for converting between between absolute and civil times via time zones. It requires only a proper C++11 compiler and the standard IANA time zone data base which standard Unix, Linux, OS X, ... computers tend to have in /usr/share/zoneinfo. RcppCCTZ connects this library to R by relying on Rcpp. Two good examples are now included, and shown here. The first one tabulates the time difference between New York and London (at a weekly level for compactness):
R> example(tzDiff)
tzDiffR> # simple call: difference now
tzDiffR> tzDiff("America/New_York", "Europe/London", Sys.time())
[1] 5
tzDiffR> # tabulate difference for every week of the year
tzDiffR> table(sapply(0:52, function(d) tzDiff("America/New_York", "Europe/London",
tzDiff+                                       as.POSIXct(as.Date("2016-01-01") + d*7))))
 4  5 
 3 50 
R> 
Because the two continents happen to spring forward and fall backwards between regular and daylight savings times, there are, respectively, two and one week periods where the difference is one hour less than usual. A second example shifts the time to a different time zone:
R> example(toTz)
toTzR> toTz(Sys.time(), "America/New_York", "Europe/London")
[1] "2016-07-14 10:28:39.91740 CDT"
R> 
Note that because we return a POSIXct object, it is printed by R with the default (local) TZ attribute (for "America/Chicago" in my case). A more direct example asks what time it is in my time zone when it is midnight in Tokyo:
R> toTz(ISOdatetime(2016,7,15,0,0,0), "Japan", "America/Chicago")
[1] "2016-07-14 15:00:00 CDT"
R>
More changes will come in 0.0.6 as soon as I find time to translate the nice time_tool (command-line) example into an R function. Changes in this version are summarized here:
Changes in version 0.0.5 (2016-07-09)
  • New utility example functions toTz() and tzDiff
  • Synchronized with small upstream change for additional #ifdef for compiler differentiation
We also have a diff to the previous version thanks to CRANberries. More details are at the RcppCCTZ page; code, issue tickets etc at the GitHub repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Martin Michlmayr: Debian on Seagate Personal Cloud and Seagate NAS

The majority of NAS devices supported in Debian are based on Marvell's Kirkwood platform. This platform is quite dated now and can only run Debian's armel port. Debian now supports the Seagate Personal Cloud and Seagate NAS devices. They are based on Marvell's Armada 370, a platform which can run Debian's armhf port. Unfortunately, even the Armada 370 is a bit dated now, so I would not recommend these devices for new purchases. If you have one already, however, you now have the option to run native Debian. There are some features I like about the Seagate NAS devices: If you have a Seagate Personal Cloud and Seagate NAS, you can follow the instructions on the Debian wiki. If Seagate releases more NAS devices on Marvell's Armada platform, I intend to add Debian support.

20 July 2016

Daniel Pocock: How many mobile phone accounts will be hijacked this summer?

Summer vacations have been getting tougher in recent years. Airlines cut into your precious vacation time with their online check-in procedures and a dozen reminder messages, there is growing concern about airport security and Brexit has already put one large travel firm into liquidation leaving holidaymakers in limbo. If that wasn't all bad enough, now there is a new threat: while you are relaxing in the sun, scammers fool your phone company into issuing a replacement SIM card or transferring your mobile number to a new provider and then proceed to use it to take over all your email, social media, Paypal and bank accounts. The same scam has been appearing around the globe, from Britain to Australia and everywhere in between. Many of these scams were predicted in my earlier blog SMS logins: an illusion of security (April 2014) but they are only starting to get publicity now as more aspects of our lives are at risk, scammers are ramping up their exploits and phone companies are floundering under the onslaught. With the vast majority of Internet users struggling to keep their passwords out of the wrong hands, many organizations have started offering their customers the option of receiving two-factor authentication codes on their mobile phone during login. Rather than making people safer, this has simply given scammers an incentive to seize control of telephones, usually by tricking the phone company to issue a replacement SIM or port the number. It also provides a fresh incentive for criminals to steal phones while cybercriminals have been embedding code into many "free" apps to surreptitiously re-route the text messages and gather other data they need for an identity theft sting. Sadly, telephone networks were never designed for secure transactions. Telecoms experts have made this clear numerous times. Some of the largest scams in the history of financial services exploited phone verification protocols as the weakest link in the chain, including a $150 million heist reminiscent of Ocean's 11. For phone companies, SMS messaging came as a side-effect of digital communications for mobile handsets. It is less than one percent of their business. SMS authentication is less than one percent of that. Phone companies lose little or nothing when SMS messages are hijacked so there is little incentive for them to secure it. Nonetheless, like insects riding on an elephant, numerous companies have popped up with a business model that involves linking websites to the wholesale telephone network and dressing it up as a "security" solution. These companies are able to make eye-watering profits by "purchasing" text messages for $0.01 and selling them for $0.02 (one hundred percent gross profit), but they also have nothing to lose when SIM cards are hijacked and therefore minimal incentive to take any responsibility. Companies like Google, Facebook and Twitter have thrown more fuel on the fire by encouraging and sometimes even demanding users provide mobile phone numbers to "prove they are human" or "protect" their accounts. Through these antics, these high profile companies have given a vast percentage of the population a false sense of confidence in codes delivered by mobile phone, yet the real motivation for these companies does not appear to be security at all: they have worked out that the mobile phone number is the holy grail in cross-referencing vast databases of users and customers from different sources for all sorts of creepy purposes. As most of their services don't involve any financial activity, they have little to lose if accounts are compromised and everything to gain by accurately gathering mobile phone numbers from as many users as possible.
Can you escape your mobile phone while on vacation? Just how hard is it to get a replacement SIM card or transfer/port a user's phone number while they are on vacation? Many phone companies will accept instructions through a web form or a phone call. Scammers need little more than a user's full name, home address and date of birth: vast lists of these private details are circulating on the black market, sourced from social media, data breaches (99% of which are never detected or made public), marketing companies and even the web sites that encourage your friends to send you free online birthday cards. Every time a company has asked me to use mobile phone authentication so far, I've opted out and I'll continue to do so. Even if somebody does hijack my phone account while I'm on vacation, the consequences for me are minimal as it will not give them access to any other account or service, can you and your family members say the same thing? What can be done?
  • Opt-out of mobile phone authentication schemes.
  • Never give the mobile phone number to web sites unless there is a real and pressing need for them to call you.
  • Tell firms you don't have a mobile phone or that you share your phone with your family and can't use it for private authentication.
  • If you need to use two-factor authentication, only use technical solutions such as smart cards or security tokens that have been engineered exclusively for computer security. Leave them in a locked drawer or safe while on vacation. Be wary of anybody who insists on SMS and doesn't offer these other options.
  • Rather than seeking to "protect" accounts, simply close some or all social media accounts to reduce your exposure and eliminate the effort of keeping them "secure" and updating "privacy" settings.
  • If your bank provides a relationship manager or other personal contact, this
    can also provide a higher level of security as they get to know you.
Previous blogs on SMS messaging, security and two factor authentication, including my earlier blog SMS Logins: an illusion of security.

Shirish Agarwal: Debconf 16 and My Experience with Debian

It has been often said that you should continually try new things in life so that a. Unlike the fish you do not mistake the pond to be the sea. b. You see other people, other types and ways of living and being which you normally won t in your day-to-day existence. With both of those as mantras I decided to take a leap into the unknown. I was unsure both about the visa process as well as the travel bit as I was traveling to an unknown place and although I had done some research about the place I was unsure about the authenticity of whatever is/was shared on the web. During the whole journey both to and fro, I couldn t sleep a wink. The Doha airport is huge. There are 5 Concourses, A, B , C, D, E and around 30+ gates in each Concourse. The ambition of the small state is something to be reckoned with. Almost 95% of the blue workers in the entire airport were of Asian sub-continent. While the Qatari Rial is 19 times stronger to us, the workers I suspect are worse-off than people doing similar things back home. Add to that the sharia law, even for all the money in the world, I wouldn t want to settle therein. Anyways, during the journey, a small surprise awaited me, Ritesh Raj Saraff, a DD was also traveling to Debconf. We bumped into each other while going to see the Doha City, courtesy Al-Hamad International Airport. I would probably share a bit more about Doha and my experiences with the city in upcoming posts. Cut to Cape Town, South Africa, we landed in the city half an hour after our scheduled time and then we sped along to University of Cape Town (UCT) which was to become our home for the next 13 odd days. The first few days were a whirlwind as there were new people to meet, old people whom I knew only as an e-mail id or an IRC nickname turned out to be real people and you have to try to articulate yourself in English, which is not a native language of mine. During Debcamp I was fortunate to be able visit some of the places and the wiki page had a lot of places which I knew I wouldn t be able to complete unless I had 15 days unlimited time and money to go around so didn t even try. I had gone with few goals in mind :- a. Do some documentation of the event In this I failed completely as just the walk from the venue to where the talks were energy-draining for me. Apart from that, you get swept in meeting new people and talking about one of million topics in Debian which interest you or the other person and while they are fulfilling, it is and was both physically and emotionally draining for me (in a good way). Bernelle (one of the organizers) had warned us of this phenomenon but you disregard it as you know you have a limited time-frame in which to meet and greet people and it is all a over-whelming experience. b. Another goal was to meet my Indian brethren who had left the country around 60~100 years mostly as slaves of East India company In this I was partially successful. I met a couple of beautiful ladies who had either a father or a mother who was Indian while the other was of African heritage. It seemed in them a yearning to know the culture but from what little they had, only Bollywood and Indian cuisine was what they could make of Indian culture. One of the girls, ummm women to be more truer, shared a somewhat grim tale. She had both an African boyfriend as well as Indian boyfriend in her life and in both cases, she was rejected by the boy s parents because she wasn t pure enough. This was deja vu all over again as the same thing can be seen here happening in casteism so there wasn t any advice I could give but just nod in empathy. What was sort of relevation was when their parents or grandparents came, the name and surnames were thrown off and the surname was just the place from where they belong. From the discussions it emerged that there were also lot of cases of forced conversions to Christianity during that era as well as temptations of a better life. As shared, this goal succeeded partially, as I was actually interested in their parents or grand-parents to know the events that shaped the Indian diaspora over there. While the children know only of today, yester-years could only be known by those people who made the unwilling perilous journey to Africa. I had also wanted to know more about Gandhiji s role in that era but alas, that part of history would have to wait for another day as I guess, both those goals would only have met had I visited Durban but that was not to be. I had applied for one talk My Experience with Debian and one workshop for Installation of Debian on systems. The My Experience with Debian was aimed at newbies and I had thought of using show-and-tell to share the differences between proprietary Operating Systems and a FOSS distribution such as Debian. I was going to take simple things such as changelogs, apt-listbugs, real-time knowledge of updates and upgrades as well as /etc/apt/sources.list to share both the versatility of the Debian desktop and real improvements than what proprietary Operating Systems had to offer. But I found myself engaging with Debian Developers (DD s) rather than the newbies so had to change the orientation and fundamentals of the talk on the fly. I knew or suspected rather that the old idea would not work as it would just be repeating to the choir. With that in the back of mind, and the idea that perhaps they would not be so aware of the politics and events which happened in India over the last couple of decades, I tried to share what little I was able to recollect what little I was able to remember about those times. Apart from that, I was also highly conscious that I had been given just the before lunch slot aka You are in the way of my lunch slot. So I knew I had to speak my piece as quickly as possible being as clear as can be. Later, I did get feedback that I was fast and seeing it through couple of times, do agree that I could have done a better job. What s done is done and the only thing I could do to salvage it a bit is to make a presentation which I am sharing as below. my_experience_with_debian Would be nice if somebody could come up with a lighter template for presentations. For reference the template I have taken it from is shared at https://wiki.debian.org/Presentations . Some pictures from the presentation. vlcsnap-00004 me_sharing my_experience_with_debian You can find the video at http://meetings-archive.debian.net/pub/debian-meetings/2016/debconf16/My_Experience_with_Debian.webm This is by no means the end of the Debconf16 experience, but actually the starting. I hope to share more of my thoughts, ideas and get as much feedback from all the wonderful people I met during Debconf.
Filed under: Miscellenous Tagged: #Debconf16, Doha, My talk, Qatar

Steinar H. Gunderson: Solskogen 2016 videos

I just published the videos from Solskogen 2016 on Youtube; you can find them all in this playlist. The are basically exactly what was being sent out on the live stream, frame for frame, except that the audio for the live shader compos has been remastered, and of course a lot of dead time has been cut out (the stream was sending over several days, but most of the time, only the information loop from the bigscreen). YouTube doesn't really support the variable 50/60 Hz frame rate we've been using well as far as I can tell, but mostly it seems to go to some 60 Hz upconversion, which is okay enough, because the rest of your setup most likely isn't free-framerate anyway. Solskogen is interesting in that we're trying to do a high-quality stream with essentially zero money allocated to it; where something like Debconf can use 2500 for renting and transporting equipment (granted, for two or three rooms and not our single stream), we're largely dependent on personal equipment as well as borrowing things here and there. (I think we borrowed stuff from more or less ten distinct places.) Furthermore, we're nowhere near the situation of two cameras, a laptop, perhaps a few microphones ; not only do you expect to run full 1080p60 to the bigscreen and switch between that and information slides for each production, but an Amiga 500 doesn't really have an HDMI port, and Commodore 64 delivers an infamously broken 50.12 Hz signal that you really need to deal with carefully if you want it to not look like crap. These two factors together lead to a rather eclectic setup; here, visualized beautifully from my ASCII art by ditaa: Solskogen 2016 A/V setup diagram Of course, for me, the really interesting part here is near the end of the chain, with Nageru, my live video mixer, doing the stream mixing and encoding. (There's also Cubemap, the video reflector, but honestly, I never worry about that anymore. Serving 150 simultaneous clients is just not something to write home about anymore; the only adjustment I would want to make would probably be some WebSockets support to be able to deal with iOS without having to use a secondary HLS stream.) Of course, to make things even more complicated, the live shader compo needs two different inputs (the two coders' laptops) live on the bigscreen, which was done with two video capture cards, text chroma-keyed on top from Chroma, and OBS, because the guy controlling the bigscreen has different preferences from me. I would take his screen in as a dirty feed and then put my own stuff around it, like this: Solskogen 2016 shader compo screenshot (Unfortunately, I forgot to take a screenshot of Nageru itself during this run.) Solskogen was the first time I'd really used Nageru in production, and despite super-extensive testing, there's always something that can go wrong. And indeed there was: First of all, we discovered that the local Internet line was reduced from 30/10 to 5/0.5 (which is, frankly, unusable for streaming video), and after we'd half-way fixed that (we got it to 25/4 or so by prodding the ISP, of which we could reserve about 2 for video demoscene content is really hard to encode, so I'd prefer a lot more) Nageru started crashing. It wasn't even crashes I understood anything of. Generally it seemed like the NVIDIA drivers were returning GL_OUT_OF_MEMORY on things like creating mipmaps; it's logical that they'd be allocating memory, but we had 6 GB of GPU memory and 16 GB of CPU memory, and lots of it was free. (The PC we used for encoding was much, much faster than what you need to run Nageru smoothly, so we had plenty of CPU power left to run x264 in, although you can of course always want more.) It seemed to be mostly related to zoom transitions, so I generally avoided those and ran that night's compos in a more static fashion. It wasn't until later that night (or morning, if you will) that I actually understood the bug (through the godsend of the NVX_gpu_memory_info extension, which gave me enough information about the GPU memory state that I understood I wasn't leaking GPU memory at all); I had set Nageru to lock all of its memory used in RAM, so that it would never ever get swapped out and lose frames for that reason. I had set the limit for lockable RAM based on my test setup, with 4 GB of RAM, but this setup had much more RAM, a 1080p60 input (which uses more RAM, of course) and a second camera, all of which I hadn't been able to test before, since I simply didn't have the hardware available. So I wasn't hitting the available RAM, but I was hitting the amount of RAM that Linux was willing to lock into memory for me, and at that point, it'd rather return errors on memory allocations (including the allocations the driver needed to make for its texture memory backings) than to violate the never swap contract. Once I fixed this (by simply increasing the amount of lockable memory in limits.conf), everything was rock-stable, just like it should be, and I could turn my attention to the actual production. Often during compos, I don't really need the mixing power of Nageru (it just shows a single input, albeit scaled using high-quality Lanczos3 scaling on the GPU to get it down from 1080p60 to 720p60), but since entries come in using different sound levels (I wanted the stream to conform to EBU R128, which it generally did) and different platforms expect different audio work (e.g., you wouldn't put a compressor on an MP3 track that was already mastered, but we did that on e.g. SID tracks since they have nearly zero ability to control the overall volume), there was a fair bit of manual audio tweaking during some of the compos. That, and of course, the live 50/60 Hz switches were a lot of fun: If an Amiga entry was coming up, we'd 1. fade to a camera, 2. fade in an overlay saying we were switching to 50 Hz so have patience, 3. set the camera as master clock (because the bigscreen's clock is going to go away soon), 4. change the scaler from 60 Hz to 50 Hz (takes two clicks and a bit of waiting), 5. change the scaler input in Nageru from 1080p60 to 1080p50, 6. steps 3,2,1 in reverse. Next time, I'll try to make that slightly smoother, especially as the lack of audio during the switch (it comes in on the bigscreen SDI feed) tended to confuse viewers. So, well, that was a lot of fun, and it certainly validated that you can do a pretty complicated real-life stream with Nageru. I have a long list of small tweaks I want to make, though; nothing beats actual experience when it comes to improving processes. :-)

19 July 2016

Dirk Eddelbuettel: Rcpp 0.12.6: Rolling on

The sixth update in the 0.12.* series of Rcpp has arrived on the CRAN network for GNU R a few hours ago, and was just pushed to Debian. This 0.12.6 release follows the 0.12.0 release from late July, the 0.12.1 release in September, the 0.12.2 release in November, the 0.12.3 release in January, the 0.12.4 release in March, and the 0.12.5 release in May --- making it the tenth release at the steady bi-montly release frequency. Just like the previous release, this one is once again more of a refining maintenance release which addresses small bugs, nuisances or documentation issues without adding any major new features. That said, some nice features (such as caching support for sourceCpp() and friends) were added. Rcpp has become the most popular way of enhancing GNU R with C or C++ code. As of today, 703 packages on CRAN depend on Rcpp for making analytical code go faster and further. That is up by about fourty packages from the last release in May! Similar to the previous releases, we have contributions from first-time committers. Artem Klevtsov made na_omit run faster on vectors without NA values. Otherwise, we had many contributions from "regulars" like Kirill Mueller, James "coatless" Balamuta and Dan Dillon as well as from fellow Rcpp Core contributors. Some noteworthy highlights are encoding and string fixes, generally more robust builds, a new iterator-based approach for vectorized programming, the aforementioned caching for sourceCpp(), and several documentation enhancements. More details are below.
Changes in Rcpp version 0.12.6 (2016-07-18)
  • Changes in Rcpp API:
    • The long long data type is used only if it is available, to avoid compiler warnings (Kirill M ller in #488).
    • The compiler is made aware that stop() never returns, to improve code path analysis (Kirill M ller in #487 addressing issue #486).
    • String replacement was corrected (Qiang in #479 following mailing list bug report by Masaki Tsuda)
    • Allow for UTF-8 encoding in error messages via RCPP_USING_UTF8_ERROR_STRING macro (Qin Wenfeng in #493)
    • The R function Rf_warningcall is now provided as well (as usual without leading Rf_) (#497 fixing #495)
  • Changes in Rcpp Sugar:
    • Const-ness of min and max functions has been corrected. (Dan Dillon in PR #478 fixing issue #477).
    • Ambiguities for matrix/vector and scalar operations have been fixed (Dan Dillon in PR #476 fixing issue #475).
    • New algorithm header using iterator-based approach for vectorized functions (Dan in PR #481 revisiting PR #428 and addressing issue #426, with futher work by Kirill in PR #488 and Nathan in #503 fixing issue #502).
    • The na_omit() function is now faster for vectors without NA values (Artem Klevtsov in PR #492)
  • Changes in Rcpp Attributes:
    • Add cacheDir argument to sourceCpp() to enable caching of shared libraries across R sessions (JJ in #504).
    • Code generation now deals correctly which packages containing a dot in their name (Qiang in #501 fixing #500).
  • Changes in Rcpp Documentation:
    • A section on default parameters was added to the Rcpp FAQ vignette (James Balamuta in #505 fixing #418).
    • The Rcpp-attributes vignette is now mentioned more prominently in question one of the Rcpp FAQ vignette.
    • The Rcpp Quick Reference vignette received a facelift with new sections on Rcpp attributes and plugins begin added. (James Balamuta in #509 fixing #484).
    • The bib file was updated with respect to the recent JSS publication for RProtoBuf.
Thanks to CRANberries, you can also look at a diff to the previous release. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads page, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

14 July 2016

Andrew Cater: Who wrote Hello world

Who wrote "Hello, world" ?
Rereading Kernighan and Ritchie's classic book on C - https://en.wikipedia.org/wiki/The_C_Programming_Language - almost the first thing you find is the listing for hello world. The comments make it clear that this is a commonplace - the sort of program that every programmer writes as a first test - the new computer works, the compiler / interpreter produces useful output and so on. It' s the classic, canonical thing to do.

A long time back, I got asked whether programming was an art or a science: it's both, but most of all it's only good insofar as it's shared and built on. I used hello world as an example: you can write hello world. You decide to add different text - a greeting (Hej! / ni hao / Bonjour tout le monde! )for friends.

You discover at / cron / anacron - now you can schedule reminders "It's midnight - do you know where your code is?" "Go to bed, you have school tomorrow"

You can discover how to code for a graphical environment: how to build a test framework around it to check that it _only_ prints hello world and doesn't corrupt other memory ... the uses are endless if it sparks creativity.

If you feel like it, you can share your version - and learn from others. Write it in different languages - there's the analogous 99 bottles of beer site showing how to count and use different languages at www.99-bottles-of-beer.net

Not everyone will get it: not everyone will see it but everyone needs the opportunity

Everyone needs the chance to share and make use of the commons, needs to be able to feel part of this

Needs to be included: needs to feel that this is part of common heritage.
If you work for an employer: get them to contribute code / money / resources - even if it's as a charitable donation or to offset against taxes

If you work for a government: get them to use Free/Libre/Open Source products

If you work for a hosting company / ISP - get them to donate bandwidth for schools/coding clubs.

Give your time, effort, expertise to help: you gained from others, help others gain

If you work for an IT manufacturer - get them to think of FLOSS as the norm, not the exception




Next.

Previous.