The other day I wanted to add support for tests of javascript functionality in a (dockerized) rails application using rspec-rails.
Since rails 5.1 includes system tests with niceties like automatically taking a screenshot on failed tests, I hoped for a way to benefit from this
features without changing to another test framework. Lucky me only recently the authors of rspec-rails added support for so-called system specs. There is not much documentation so far (but there are a lot of useful information in the corresponding bug report
#1838 and a friendly guy named Thomas Walpole (
@twalpole) is helpfully answering to questions in that issue.
To make things a little bit more complicated: the application in question is usually running in a docker container and thus the tests of the application are also run in a docker container. I didn t want to change this, so here is what it took me to get this running.
Overview
Let s see what we want to achieve exactly: From a technical point of view, we will have the application under test (AUT) and the tests in one container (let s call it: web) and we will need another container running a javascript-capable browser (let s call it browser).
Thus we need the tests to drive a remote running browser (at least when running in a docker environment) which needs to access the application under a different address than usually. Namely an address reachable by the
chrome-container, since it will not be reachable via
127.0.0.1 as is (rightfully) assumed by default.
If we want Warden authentication stubbing to work (as we do, since our application uses Devise) and transactional fixtures as well (e.g. rails handling database cleanup between tests without database_cleaner gem) we also need to ensure that the application server is being started by the tests and the tests are actually run against that server. Otherwise we might run into problems.
Getting the containers ready
Assuming you already have a container setup (and are using docker-compose like we do) there is not that much to change on the docker front. Basically you need to add a new service called
chrome and point it to an appropriate image and add a link to it in your existing
web-container.
I ve decided to use standalone-chrome for the browser part, for which there are docker images provided by the selenium project (they also have images for other browsers). Kudos for that.
...
services:
chrome:
image: selenium/standalone-chrome
web:
links:
- chrome
The link ensures that the chrome instance is available before we run the tests and that the the
web-container is able to resolve the name of this container. Unfortunately this is not true for the other way round, so we need some magic in our test code to find out the ip-address of the
web-container. More to this later.
Other than that, you probably want to configure a volume for you to be able to access the screenshots, which get saved to
tmp/screenshots
in the application directory.
Preparing the application for running system tests
There is a bit more to do on the application side. The steps are roughly:
- Add necessary depends / version constraints
- Register a driver for the remote chrome
- Configure capybara to use the appropriate host for your tests (and your configured driver)
- Add actual tests with
type: :system and js: true
Let s walk them through.
Add necessary depends
What we need is the following:
- rspec-rails version >= 3.7.1
- rails itself > 5.1.4 (unreleased at time of writing)
- capybara and capybara-selenium
The required features are already part of 3.7.0, but this version is the version I used and it contains a bugfix, which may or may not be relevant.
One comment about the rails version: for the tests to properly work it s viable to have puma use certain settings. In rails 5.1.4 (the version released, at time of writing this) uses the settings from
config/puma.rb which most likely collides with the necessary
settings. You can ensure these settings yourself or use rails from branch
5-1-stable which includes
this change. I decided for the latter and pinned my
Gemfile to the then current commit.
Register a driver for the remote chrome
To register the required driver, you ll have to add some lines to your
rails_helper.rb:
if ENV['DOCKER']
selenium_url =
"http://chrome:4444/wd/hub"
Capybara.register_driver :selenium_remote do app
Capybara::Selenium::Driver.new(app,
:url => selenium_url, :browser => :remote, desired_capabilities: :chrome )
end
end
Note that I added those lines conditionally (since I still want to be able to use a local chrome via chromedriver) if an environment variable
DOCKER is set.
We defined that environment variable in our
Dockerfile and thus you might need to adapt this to your case.
Also note that the selenium_url is hard-coded. You could very well take a different approach, e.g. using an externally specified SELENIUM_URL, but ultimately the requirement is that the driver needs to know that the chrome instance is running on host
chrome, port 4444 (the containers default).
Configure capybara to use the appropriate host and driver
The next step is to ensure that javascript-requiring system tests are actually run with the given driver and use the right host. To achieve that we need to add a before-hook to the corresponding tests or we can configure rspec accordingly to always include such a hook by modifying the rspec-configuration in
rails_helper.rb like this:
RSpec.configure do config
...
config.before(:each, type: :system, js: true) do
if ENV['DOCKER']
driven_by :selenium_remote
ip = Socket.ip_address_list.detect addr addr.ipv4_private? .ip_address
host! "http://# ip :# Capybara.server_port "
else
driven_by :headless_chrome
end
end
Note the part with the ip-address: it tries to find an IPv4 private address for the
web-container (the container running the tests) to ensure the
chrome-container uses this address to access the application. The
Capybara.server_port is important here, since it will
correspond to the puma instance launched by the tests.
That heuristic (first ipv4 private address) works for us at the moment, but it might not work for you. It is basically a workaround to the fact that I couldn t get
web resolvable for the chrome container which may be fixable on the docker side, but I was to lazy to further investigate that.
If you change it: Just make sure the
host! method uses an URI pointing to an address of the
web-container that is reachable to the
chrome-container.
Define tests with
type: :system and
js: true
Last but certainly not least, you need actual tests of the required type and with or without
js: true. This can be achieved by creating tests files starting like this:
RSpec.feature "Foobar", type: :system, js: true do
Since the new rspec-style system tests are based around the feature-specs which used to be around previously, the rest of the tests is exactly like it is
described for feature specs.
Run the tests
To run the tests a commandline like the following should do:
docker-compose run web rspec
It won t make a big noise about running the tests against chrome, unless something fails. In that case you ll see a message telling you where the screenshot has been placed.
Troubleshooting
Below I add some hints about problems I ve seen during configuring that:
Test failing, screenshot shows login screen
In that case puma might be configured wrongly or you are not using transactional fixtures. See the hints above about the rails version to use which also includes some pointers to helpful explanations.
Note that rspec-rails by default does not output the puma startup output as it clutters the tests. For debugging purposes it might be helpful to change that by adding the following line to your tests:
ActionDispatch::SystemTesting::Server.silence_puma = false
Error message: Unable to find chromedriver
This indicates that your driver is not configured properly, because the default for system tests is to be
driven_by selenium, which tries to spawn an own chrome instance and is suitable for non-dockerized tests.
Check if your tests are marked as js: true (if you followed the instructions above) and that you properly added the before-hook to your rspec-configuration.
Collisions with VCR
If you happen to have tests that make use of the
vcr gem you might see it complaining about not knowing what to do with the requests between the driver and the chrome instance. You can fix this, by telling VCR to ignore that requests, by adding a line where you configured VCR:
VCR.configure do config
# required so we don't collide with capybara tests
config.ignore_hosts 'chrome'
...
Today this blog post
turned up on Hacker News, titled Obscurity is
a Valid Security Layer . It makes some excellent points on the distinction
between good and bad obscurity and it gives an example of good obscurity with
SSH.
From the post:
For many years now, I have been driving a diesel based VW Passat Estate. It
has served me very well and been as reliable as I might have hoped given how
awful I am with cars. Sadly Gunther was reaching the point where it was
going to cost more per year to maintain than the car was worth, and also I've
been being more and more irked by not having a car from the future.
I spent many months doing spreadsheets, trying to convince myself I could
somehow afford a Tesla of some variety. Sadly I never quite managed it. As
such I set my sights on the more viable BEVs such as the Nissan Leaf. For a
while, nothing I saw was something I wanted. I am quite unusual it seems, in
that I don't want a car which is a "Look at me, I'm driving an electric car"
fashion statement. I felt like I'd never get something which looked like a
normal car, but happened to be a BEV.
Then along came the Hyundai Ioniq. Hybrid, Plug-in Hybrid, and BEV all looking
basically the same, and not in-your-face-special. I began to covet.
Eventually I caved and arranged a test drive of an Ioniq plug-in hybrid because
the BEV was basically on 9 month lead. I enjoyed the drive and was instantly
very sad because I didn't want a plug-in hybrid, I wanted a BEV. Despondent,
I left the dealership and went home.
I went online and found a small number of second-hand Ioniq BEVs but as I
scrolled through the list, none seemed to be of the right trim level. Then,
just as I was ready to give up hope, I saw a new listing, no photo, of the
right thing. One snag, it was 200 miles away. No matter, I rang the place,
confirmed it was available, and agreed to sleep on the decision.
The following morning, I hadn't decided to not buy, so I called them up, put
down a deposit to hold the car until I could test drive it, and then began the
long and awkward process of working out how I would charge the car given I have
no off-street parking so I can't charge at home. (Yeah yeah, you'd think I'd
have checked that first, but no I'm just not that sensible). Over the week
I convinced myself I could do it, I ordered RFID cards for various schemes,
signed up with a number of services, and then, on Friday last week, I drove
down to a hotel near the dealership and had a fitful night's sleep.
I rocked up to the dealership exactly as they opened for business, shook the
hand of the very helpful salesman who had gone through the purchase process
with me over the phone during the week, and got to see the car. Instant want
coursed through me as I sat in it and decided "Yes, this somehow feels right".
I took the car for about a 45 minute test drive just to see how it felt
relative to the plug-in hybrid I'd driven the week before and it was like night
and day. The BEV felt so much better to drive. I was hooked. Back to the
dealership and we began the paperwork. Emptying Gunther of all of the bits
and bobs scattered throughout his nooks and crannies took a while and gave me a
chance to say goodbye to a car which, on reflection, had actually been a
pleasure to own, even when its expensive things went wrong, more than once.
But once I'd closed the Passat for the last time, and handed the keys over, it
was quite a bittersweet moment as the salesman drove off in what still felt
like my car, despite (by this point) it not being so.
Sitting in the Ioniq though, I headed off for the 200 mile journey back home.
With about 90% charge left after the test drive, I had two stops planned at
rapid chargers and I headed toward my first.
Unfortunately disaster struck, the rapid (50KW) charger refused to initialise,
and I ended up with my car on the slower (7KW) charger to get enough juice into
it to drive on to the next rapid charger enabled service station. When I got
the message that my maximum charge period (45m) had elapsed, I headed back to
the car to discover I couldn't persuade it to unlock from the car. Much hassle
later, and an AA man came and together we learned that it takes 2 to tango, one
to pull the emergency release in the boot, the other to then unplug the cable.
Armed with this knowledge, I headed on my way to a rapid charger I'd found on
the map which wasn't run by the same company. Vainly hoping that this would
work better, I plugged the car in, set the charger going, and headed into the
adjacent shop for a rest break. I went back to the car about 20 minutes later
to see the charger wasn't running. Horror of horrors. I imagined maybe some
nasty little oik had pressed 'stop' so I started the charger up again, and sat
in the car to read my book. After about 3 minutes, the charge stopped. Turns
out that charger was slightly iffy and couldn't cope with the charge current
and kept emergency-stopping as a result. The lovely lady I spoke to about it
directed me to a nearby (12 miles or so, easily done with the charge I had)
charger in the grounds of a gorgeous chateau hotel. That one worked perfectly
and I filled up. I drove on to my second planned stop and that charge went
perfectly too. In fact, every charge since has gone flawlessly. So perhaps my
baptism of failed charges has hardened me to the problems with owning a BEV.
I've spent the past few days trying different charge points around Manchester
enjoying my free charge capability, and trying different names for the car
before I finally settled on which is a reasonable Korean boy's name
(since the car is Korean I even got to set that as the bluetooth ID) and it's
roughly pronounced sock/gin which are two wonderful things in life.
I'm currently sat in a pub, eating a burger, enjoying myself while suckles
on the teat of "free" electricity to make up for the fact that I've developed a
non-trivial habit of leaving Audi drivers in the dust at traffic lights.
Further updates may happen as I play with Android Auto and other toys in an
attempt to eventually be able to ask the car to please "freeze my buttocks"
(a feature it has in the form of air-conditioned seats.)
Reproducible Builds Summit 3 next week in Berlin
Next week from Tuesday, the 31st of October until Thursday, November 2nd, we'll have the 
Today it s my first day working at the 
Narabu is a new intraframe video codec. You probably want to read
Here s are some of the bugs against the Debian Policy Manual. In
particular, there really are quite a few patches needing seconds from
DDs.
Consensus has been reached and help is needed to write a patch:
Wishing all Debian people a prosperous and auspicious Gujarati new year (V.S. 2074 called Saumya.)
This year fireworks became legal for the first time in New Jersey. Not that it ever stopped us before but it is nice to see the government stop meddling for no reason. (
Although you can only see sparklers in the picture above, we got enough armament to make 
Like 
