Welcome to the seventh post in the rarely relevant R ramblings series, or R4 for short.
We took a short break as several conferences and other events interfered during the month of May, keeping us busy and away from this series. But we are back now with a short and useful hack I came up with this weekend.
The topic is C++14, i.e. the newest formally approved language standard for C++, and its support in R and on Travis CI. With release R 3.4.0 of a few weeks ago, R now formally supports C++14. Which is great.
But there be devils. A little known fact is that R hangs on to its configuration settings from its own compile time. That matters in cases such as the one we are looking at here: Travis CI. Travis is a tremendously useful and widely-deployed service, most commonly connected to GitHub driving "continuous integration" (the 'CI') testing after each commit. But Travis CI, for as useful as it is, is also maddingly conservative still forcing everybody to live and die by [Ubuntu 14.04]http://releases.ubuntu.com/14.04/). So while we all benefit from the fine work by Michael who faithfully provides Ubuntu binaries for distribution via CRAN (based on the Debian builds provided by yours truly), we are stuck with Ubuntu 14.04. Which means that while Michael can provide us with current R 3.4.0 it will be built on ancient Ubuntu 14.04.
Why does this matter, you ask? Well, if you just try to turn the very C++14 support added to R 3.4.0 on in the binary running on Travis, you get this error:
** libs
Error in .shlib_internal(args) :
C++14 standard requested but CXX14 is not defined
And you get it whether or not you define CXX14 in the session.
So R (in version 3.4.0) may want to use C++14 (because a package we submitted requests it), but having been built on the dreaded Ubuntu 14.04, it just can't oblige. Even when we supply a newer compiler. Because R hangs on to its compile-time settings rather than current environment variables. And that means no C++14 as its compile-time compiler was too ancient. Trust me, I tried: adding not only g++-6 (from a suitable repo) but also adding C++14 as the value for CXX_STD. Alas, no mas.
The trick to overcome this is twofold, and fairly straightforward. First off, we just rely on the fact that g++ version 6 defaults to C++14. So by supplying g++-6, we are in the green. We have C++14 by default without requiring extra options. Sweet.
The remainder is to tell R to not try to enable C++14 even though we are using it. How? By removing CXX_STD=C++14 on the fly and just for Travis. And this can be done easily with a small script configure which conditions on being on Travis by checking two environment variables:
#!/bin/bash
## Travis can let us run R 3.4.0 (from CRAN and the PPAs) but this R version
## does not know about C++14. Even though we can select CXX_STD = C++14, R
## will fail as the version we use there was built in too old an environment,
## namely Ubuntu "trusty" 14.04.
##
## So we install g++-6 from another repo and rely on the fact that is
## defaults to C++14. Sadly, we need R to not fail and hence, just on
## Travis, remove the C++14 instruction
if [[ "$ CI " == "true" ]]; then
if [[ "$ TRAVIS " == "true" ]]; then
echo "** Overriding src/Makevars and removing C++14 on Travis only"
sed -i 's CXX_STD = CXX14 ' src/Makevars
fi
fiSystemRequirements: in the file DESCRIPTION, you need to modify it here.
So to sum up, there it is: C++14 with R 3.4.0 on Travis. Only takes a quick Travis-only modification.
tar --sort=name we need to compile tar before downloading everythingCONFIG_AUTOREMOVE to reduce required spaceopenwrt_get_banner(): use staging_dir instead of build_dir because the former is persistent among the two builds.node_cleanup_tmpdirsgit reset/clean like Jenkins doesWORKSPACE dir names, as WORKSPACE cannot be generated from $0 as it's a temporary name.scheduling_args.message.
Like each month, here comes a report about the work of paid contributors to Debian LTS.
Individual reports
In May, about 182 work hours have been dispatched among 11 paid contributors. Their reports are available:
No comment Liked this article? Click here. My blog is Flattr-enabled.
Some days ago, I blogged asking for pointers to get started with the translation of Made with Creative Commons. Thank you all for your pointers and ideas! To the people that answered via private mail, via IRC, via comments on the blog. We have made quite a bit of progress so far; I want to test some things before actually sending a call for help. What do we have?
$ pandoc -f odt MadewithCreativeCommonsmostup-to-dateversion.odt -t markdown > MadewithCreativeCommonsmostup-to-dateversion.md .po filepo4a-gettextize -f text -m MadewithCreativeCommonsmostup-to-dateversion.md -p MadewithCreativeCommonsmostup-to-dateversion.po -M utf-8poedit, and it works... Very nicely!
A new package! Or at least new on CRAN as the very initial version 0.1.0 had been available via the ghrr drat for over a year. But now we have version 0.1.1 to announce as a CRAN package.
RcppMspPack provides R with MessagePack header files for use via C++ (or C, if you must) packages such as RcppRedis.
MessagePack itself is an efficient binary serialization format. It lets you exchange data among multiple languages like JSON. But it is faster and smaller. Small integers are encoded into a single byte, and typical short strings require only one extra byte in addition to the strings themselves.
MessagePack is used by Redis and many other projects, and has bindings to just about any language.
To use this package, simply add it to the LinkingTo: field in the DESCRIPTION field of your R package---and the R package infrastructure tools will then know how to set include flags correctly on all architectures supported by R.
More information may be on the RcppMsgPack page. Issues and bugreports should go to the GitHub issue tracker.
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
domain=int.foobar.example
dhcp-range=192.168.0.240,192.168.0.242,255.255.255.0,1h
dhcp-boot=bootnetx64.efi
pxe-service=X86-64_EFI, "Boot UEFI PXE-64", bootnetx64.efi
enable-tftp
tftp-root=/usr/lib/debian-installer/images/8/amd64/text
dhcp-option=3,192.168.0.1
dhcp-host=00:c0:ff:ee:00:01,192.168.0.123,foobar-01
Now you've to link /usr/lib/debian-installer/images/8/amd64/text/bootnetx64.efi
to /usr/lib/debian-installer/images/8/amd64/text/debian-installer/amd64/bootnetx64.efi.
That got us of the ground and we had a working UEFI PXE boot that got
us into debian-installer.
Feeding d-i the preseed file
Next we added some grub.cfg settings and parameterized some basic stuff to
be handed over to d-i via the kernel command line. You'll find the correct
grub.cfg in
/usr/lib/debian-installer/images/8/amd64/text/debian-installer/amd64/grub/grub.cfg.
We added the following two lines to automate the start of the installer:
set default="0"
set timeout=5
and our kernel command line looks like this:
linux /debian-installer/amd64/linux vga=788 --- auto=true interface=eth1 netcfg/dhcp_timeout=60 netcfg/choose_interface=eth1 priority=critical preseed/url=tftp://192.168.0.2/preseed.cfg quiet
Important points:
# auto method must be lvm
d-i partman-auto/method string lvm
d-i partman-lvm/device_remove_lvm boolean true
d-i partman-md/device_remove_md boolean true
d-i partman-lvm/confirm boolean true
d-i partman-lvm/confirm_nooverwrite boolean true
d-i partman-basicfilesystems/no_swap boolean false
# Keep that one set to true so we end up with a UEFI enabled
# system. If set to false, /var/lib/partman/uefi_ignore will be touched
d-i partman-efi/non_efi_system boolean true
# enforce usage of GPT - a must have to use EFI!
d-i partman-basicfilesystems/choose_label string gpt
d-i partman-basicfilesystems/default_label string gpt
d-i partman-partitioning/choose_label string gpt
d-i partman-partitioning/default_label string gpt
d-i partman/choose_label string gpt
d-i partman/default_label string gpt
d-i partman-auto/choose_recipe select boot-root-all
d-i partman-auto/expert_recipe string \
boot-root-all :: \
538 538 1075 free \
$iflabel gpt \
$reusemethod \
method efi \
format \
. \
128 512 256 ext2 \
$defaultignore \
method format format \
use_filesystem filesystem ext2 \
mountpoint /boot \
. \
1024 4096 15360 ext4 \
$lvmok \
method format format \
use_filesystem filesystem ext4 \
mountpoint / \
. \
1024 4096 15360 ext4 \
$lvmok \
method format format \
use_filesystem filesystem ext4 \
mountpoint /var \
. \
1024 1024 -1 ext4 \
$lvmok \
method format format \
use_filesystem filesystem ext4 \
mountpoint /var/lib \
.
# This makes partman automatically partition without confirmation, provided
# that you told it what to do using one of the methods above.
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman-md/confirm boolean true
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true
# This is fairly safe to set, it makes grub install automatically to the MBR
# if no other operating system is detected on the machine.
d-i grub-installer/only_debian boolean true
d-i grub-installer/with_other_os boolean true
d-i grub-installer/bootdev string /dev/sda
I hope that helps to ease the processes to setup automated UEFI PXE installations
for some other people out there still dealing with bare metal systems. Some
settings took us some time to figure out, for example
d-i partman-efi/non_efi_system boolean true required some searching on
codesearch.d.n (amazing ressource
if you're writing preseed files and need to find the correct templates) and
reading scripts on git.d.o where you'll
find the source for partman-* and grub-installer.
Kudos
Thanks especially to P.P. and M.K. to figure out all those details.
| Title / language | Quantity | ||
|---|---|---|---|
| 2016 jan-jun | 2016 jul-dec | 2017 jan-may | |
| Culture Libre / French | 3 | 6 | 15 |
| Fri kultur / Norwegian | 7 | 1 | 0 |
| Free Culture / English | 14 | 27 | 16 |
| Total | 24 | 34 | 31 |
pslog 26475Finding psmisc psmisc will be available in your usual distributions shortly. The Debian packages are about to be uploaded and will be in the sid distribution soon. Other distributions I imagine will follow. For the source code, look in the GitLab repository or the Sourceforge file location.
Pid no 26475:
Log path: /opt/observium/logs/error_log
Log path: /var/log/apache2/other_vhosts_access.log
Log path: /opt/observium/logs/access_log
Well, I made this map using data from http://db.debian.org. As an example, currently, there are 27 Brazilian DDs. However, there are 23 DDs living in Brazil.
I recently finished a paper that presents a novel social computing system called the Wikipedia Adventure. The system was a gamified tutorial for new Wikipedia editors. Working with the tutorial creators, we conducted both a survey of its users and a randomized field experiment testing its effectiveness in encouraging subsequent contributions. We found that although users loved it, it did not affect subsequent participation rates.
Start screen for the Wikipedia Adventure.
The number of active, registered editors ( 5 edits per month) to Wikipedia over time. From Halfaker, Geiger, and Morgan 2012.The Wikipedia Adventure was designed to address issues that new editors frequently encountered while learning how to contribute to Wikipedia. It is structured into different missions that guide users through various aspects of participation on Wikipedia, including how to communicate with other editors, how to cite sources, and how to ensure that edits present a neutral point of view. The sequence of the missions gives newbies an overview of what they need to know instead of having to figure everything out themselves. Additionally, the theme and tone of the tutorial sought to engage new users, rather than just redirecting them to the troves of policy pages.
Those who play the tutorial receive automated badges on their user page for every mission they complete. This signals to veteran editors that the user is acting in good-faith by attempting to learn the norms of Wikipedia.
An example of a badge that a user receives after demonstrating the skills to communicate with other users on Wikipedia.
Survey responses about how users felt about TWA.
Survey responses about what users learned through TWA.
This blog post, and the open access paper that it describes, is a collaborative project with Sneha Narayan, Jake Orlowitz, Jonathan Morgan, and Aaron Shaw. Financial support came from the US National Science Foundation (grants IIS-1617129 and IIS-1617468), Northwestern University, and the University of Washington. We also published all the data and code necessary to reproduce our analysis in a repository in the Harvard Dataverse. Sneha posted the material in this blog post over on the Community Data Science Collective Blog.
While upgrading Libravatar to a more recent
version of Django, I ran into a
mysterious 400 error.
In debug mode, my site was working fine, but with DEBUG = False, I would
only a page containing this error:
Bad Request (400)
with no extra details in the web server logs.
settings.py:
LOGGING =
'version': 1,
'disable_existing_loggers': False,
'handlers':
'file':
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': '/tmp/debug.log',
,
,
'loggers':
'django':
'handlers': ['file'],
'level': 'DEBUG',
'propagate': True,
,
,
Then I got the following error message:
Invalid HTTP_HOST header: 'www.example.com'. You may need to add u'www.example.com' to ALLOWED_HOSTS.
settings.py would make it work outside of debug mode:
ALLOWED_HOSTS = ['*']
which means that there's a mismatch between the HTTP_HOST from Apache and
the one that Django expects.
ALLOWED_HOSTS setting.
I had this:
ALLOWED_HOSTS = 'www.example.com'
instead of:
ALLOWED_HOSTS = ['www.example.com']
I stumbled across some information about pathological behaviour of Western
Digital Green (and some Red) hard drives relating to drive-head parking when
the device is idle.
In some circumstances, a particular pattern of drive activity can result in the
drive head being repeatedly parked and un-parked in short intervals, possibly*
resulting in excess wear on the drive. Apparently* the drive head parking is
recorded in the S.M.A.R.T. "Load Cycle Count" attribute.
I have two WD Red drives in my NAS, one for live data and one
for backup. The latter drive is basically unused most of the day until scheduled
backup jobs kick in and those jobs are all clustered together. I already unmount
the backup filesystems when the jobs are not active (I wrote about this in
mount-on-demand backups).
Inspecting the S.M.A.R.T. attributes was surprising:
| drive | power on hours | load cycle count |
|---|---|---|
| regular | 12143 | 348 |
| backup | 12191 | 13043 |
hdparm.
( The problem with these kind of issues is there is precious little in the way
of reliable documentation as to the real issue, real drive behaviour, etc. I've
marked a few sections of this blog post with * asterisks to indicate where we
are having to make informed guesses. )
I have updated my Calibre Debian repository to include packages of the upcoming version 3, currently version 2.99.10. As with the previous packages, I kept RAR support in to allow me to read comic books.
The repository location hasn t changed, see below. Please report your experiences to the Calibre beta support channel at MobileRead.
deb http://www.preining.info/debian/ calibre main
deb-src http://www.preining.info/debian/ calibre main
The releases are signed with my Debian key 0x6CACA448860CDC13
Enjoy
Sometimes I needed a short break from work, and usually I go a bit running. Normally I run between five and ten km, just around our flat. But yesterday I somehow thought it might be a good idea to run through the hills near by. I usually drive up there to a climbing spot or a cafe, so it will for sure not be that long. Well, my estimation was wrong
Turned out to be 20km and quite a trip up and down, in fact 560mH running up wasn t what I could actually do, so I walked the steeper parts. Due to the permanent rain the views weren t that great, but one gets a feeling how far it is back to town.
I used my new fenix 5x for the run (was helpful with the built-in map, report later), so I got very detailed data from Garmin Connect. Not that I really understand what it is all for, but Garmin told me that I am overtraining and that I am not a good runner (due to vertical oscillation etc). I guess there is a whole lot to learn and improve.
Anyway, I made it back alive, and the first thing I needed was loads of water to drink
The OpenVAS
OpenVAS is a framework of several services and tools offering a comprehensive and powerful
vulnerability scanning and vulnerability management solution. The framework is part of Greenbone Networks' commercial vulnerability management solution from which developments are contributed to the Open Source community since 2009.
OpenVAS is composed of some elements, as OpenVAS-Cli, Greenbone Security Assistant, OpenVAS Scanner and OpenVAS Manager.
The official OpenVAS homepage is http://www.openvas.org.
From Kali Linux 2017.1 to Debian 9
Ok, this is a temporary solution. Now (June 2017), Debian 9 wasn't released yet and OpenVAS 9 is not available in Debian in good conditions (it is in Experimental but a bit problematic). I think that we will have OpenVAS in backports soon.
The OpenVAS 9 from Kali is working perfect for Debian 9. So, to take advantage of this, adopt the following procedures:
1. Add a line to end of /etc/apt/sources.list file:
deb http://http.kali.org/kali kali-rolling main
2. Run:
# apt-get update # apt-get install -t kali-rolling openvas
(if you want to simulate before install, add a -s option before -t)
3. Rermove or comment the previous line added to /etc/apt/sources.list file to avoid future problems in your Debian.
4. Run the following command to configure the OpenVAS and to download the initial database:
# openvas-setup
This step may take some time. Note that the initial password for user admin will be created and shown.
5. Finally, open a web browser and access the address https://127.0.0.1:9392 (use https!!!). Alternatively, you can call http://127.0.0.1 to be automagically redirected to https://127.0.0.1:9392.
The default user is admin and the password was shown in the last step. You will can change this password inside OpenVAS (menu Administration > Users > Edit User, which is an blue icon).
If your password was lost, you will be able to create a new administrative user via shell command. See the section "Some tips" below.
6. To see the packages installed from Kali Linux, use the command:
$ dpkg -l grep kaliSome tips To create a new administrative user called test:
# openvasmd --create-user test --role AdminTo update the database (NVTs):
# openvasmd --update # openvasmd --rebuild # service openvas-scanner restartTo solve the message "Login failed. Waiting for OMP service to become available":
# openvas-startTo make OpenVAS listen all network interfaces, allowing one to call the service in a remote web browser (http://remote-server, port 80 only), edit the /lib/systemd/system/greenbone-security-assistant.service and change:
ExecStart=/usr/sbin/gsad --foreground --listen=127.0.0.1 --port=9392 --mlisten=127.0.0.1 --mport=9390
to
ExecStart=/usr/sbin/gsad --foreground --listen=0.0.0.0 --port=9392 --mlisten=127.0.0.1 --mport=9390After the change, reload the configuration and restart the service:
# systemctl daemon-reload # systemctl restart greenbone-security-assistantQuick start for beginners If OpenVAS isn't familiar for you, I have a suggestion for a quick test. Firstly, download Ubuntu Server 8.04 from this page and install it inside a virtual machine, selecting all services as LAMP, SSH etc. I used the i386 version and VirtualBox. After this, use OpenVAS to scan the Ubuntu (menu Scans > Tasks > a pink icon with a magic wand in the upper left corner). After the scan, click over the blue bar where we will can see the word "Done". Enjoy!
Next.