Motivation
At the very beginning, HTML was among the first and few technologies enabling creation of web documents. Not long, JavaScript was introduced, sprinkled here and there, drop-down menus and simple animations started to appear. Eventually, the web advanced from simple static websites to highly-interactive asynchronous web pages.
The web has evolved and so did Ruby on Rails. The framework now provides improved JavaScript tooling, including integrations for both yarn package manager and webpack module bundler - de facto choice in the JavaScript community. Out of the box support for these tools facilitates creation of rich client applications. We can now easier than ever build Rails-backed applications with modern tools like Vue, React and Angular.
However, as we develop our applications with modern JavaScript libraries, a new concern emerges. Feature specs that once brought confidence are now becoming increasingly brittle. Specs that once steadily navigated through almost static web pages now stumble and fall in an asynchronous world of JavaScript.
Hence, similarly to how webpack and yarn were previously introduced to build modern applications, we further introduce Cypress to test them. Particularly, in this post, we look at Cypress as an alternative tool for writing and executing feature specs. We explore the key points that make it stand out among other web automation tools. Finally, we demonstrate how to integrate Cypress into a Ruby on Rails development cycle by setting up tests in both local and CI environments.
Cypress
Cypress is a JavaScript end-to-end testing framework. It is an all-in-one solution that enables one to both write and execute tests that run in the browser. In a Rails project, Cypress takes place of both Capybara and the underlying web driver.
Cypress vs Selenium
Without doubt, Capybara is the most popular and widely adopted solution for integration testing in Rails. It provides a robust interface to communicate with the browser through a web driver. Although Capybara is agnostic of the installed driver, Selenium WebDriver is the prevailing choice among Rails developers. As Selenium is the most widespread solution also beyond Rails, it is therefore important to understand the difference between Cypress and the combination of Capybara/Selenium.
The key difference between Cypress and Selenium lies in their ways of communication with the browser, i.e. browser automation. With Selenium, all communications are done through an external to the browser driver (the WebDriver). To simulate user actions, Selenium queries the browser by sending remote calls through the network and then extracting the desired information from the responses. Cypress, on the other hand, doesn't involve any network communication. It lives directly in the browser and runs the tests in the same run-loop as the client application. That is, Cypress tests are right there - in the browser, sharing the same context with your application.
As an outside process, Selenium is not able to react to events in the same run-loop. Events are thus processed asynchronously, can be overlooked and network delays may be introduced. This may lead to test flakiness and longer execution times.
In contrast, by sharing the same JavaScript event-loop with your application, Cypress is synchronously notified of all events as they occur leading to more reliable test execution. Removing the network layer allows Cypress to execute tests as fast as the browser is capable of rendering.
The browser-integrated architecture of Cypress accounts for it's quick and reliable test execution. Sounds promising. But let's see it for ourselves. Next move: install Cypress in a Rails project. You can find the source code for the demo project here.
Cypress on Rails: hands-on experience
Application setup
We want to quickly bootstrap a Rails application with some logic, data, and views already in place so that we can focus on the testing part. A fresh installation of Solidus will do the job. Solidus is a highly customizable ecommerce platform. By adding Solidus gem to our project we get a default store implementation right out of the box. Bellow is an outline of all required application setup steps:
- Create a new Rails project, use Postgresql for the database and skip the test files:
$ rails new cypress-store -d postgresql -T
- Add Solidus to the project by installing two new gems and running their included generators:
# Gemfile
gem 'solidus'
gem 'solidus_auth_devise'
bundle exec rails g spree:install
bundle exec rails g solidus:auth:install
bundle exec rake railties:install:migrations
- Run the rails server and navigate to the index page, you should be greeted with the default store landing page:
bundle exec rails s
At this step, we have a full-fledged online store. One might want to tweak some styles before sending it to production, but for our testing purposes, it's just what we need. We are now ready to setup Cypress.
Cypress setup
Similarly to how react-rails gem is used to integrate React components with Rails views, we will use cypress-on-rails gem to integrate Cypress with Rails tests. The new gem is a light wrapper around Cypress that augments its functionality allowing us to prepare application state using factories or custom Ruby code. Following the gem's getting started guide, add the cypress-on-rails gem to the Gemfile and run the generator:
# Gemfile
group :test, :development do
gem 'cypress-on-rails', '~> 1.0'
end
bin/rails g cypress_on_rails:install
This will both add the cypress npm package and setup the Ruby integration. To ensure that Cypress is ready to run tests against our application, run the application server in a test environment on port 5002 and in a separate window run Cypress:
# start rails
bundle exec rails server -e test -p 5002
# in separate window start cypress
yarn cypress open --project ./spec
You might also need to precompile assets before running Cypress. The command cypress open
will launch a test runner window with some sample tests scaffolded for us by Cypress.
We will move on to substitute the sample tests with our own. Now, notice how simple
it is to add Cypress to a Rails project, a single gem and no external dependencies
like servers or drivers required.
However, let's throw in a few more gems to help us with arranging the application state. We want to empty the database before each spec and then create the data required by each test scenario.
As usually, we delegate the database cleaning job to the database_cleaner gem. Add the gem to the project and then key in the bellow scripts:
# spec/cypress/cypress_helper.rb
require 'database_cleaner'
# spec/cypress/app_commands/clean.rb
DatabaseCleaner.strategy = :truncation
DatabaseCleaner.clean
# spec/cypress/support/index.js
import './commands'
import './on-rails'
beforeEach(() => {
cy.app('clean');
});
The above scripts configure the gem and then run it (cy.app('clean')) before
each integration test. Now lets prepare the test data with FactoryBot. Add the
factory_bot_rails gem to the project together with the following code:
# spec/cypress/cypress_helper.rb
require 'database_cleaner'
require "factory_bot"
require "spree/testing_support/factories"
require "carmen"
Dir[Rails.root.join("spec", "support", "factories", "**", "*.rb")].each { |f| require f }
FactoryBot.find_definitions
We now have access to both the FactoryBot library and all the factories provided by Solidus (e.g. users, orders and products).
Add Cypress tests
Some online stores require their customers to sign in prior to the checkout. Solidus comes with user session management functionality built-in and for us it's a great test subject to begin with.
Our first test will verify that only registered (persisted) users can sign in. Users that cannot be found in our database will be presented with the corresponding sign in failure message.
Following the Arrange-Act-Assert (AAA) testing pattern, we first arrange the application state for our planned scenario. We will execute our tests against an application with one existing user. We will use a user factory imported from Solidus. Key in the following:
# spec/cypress/app_commands/scenarios/sign_in.rb
FactoryBot.create(:user, email: '[email protected]', password: 'test123')
With the state arranged, we move on to describe user actions and assert the desired outcome. Note that Cypress leverages other well-known libraries to write and organize tests. Specifically, Mocha provides TDD-like syntax to structure the tests (think decribe() - beforeEach() - it()). Likewise, Cypress wraps and extends Chai for writing assertions (think should). Moreover, with cypress-on-rails gem we are able to run our factory defined above by calling cy.scenario('sign_in'). Type in the following:
// spec/cypress/integration/user/sign_in.spec.js
describe('Sign in', () => {
beforeEach(() => {
cy.scenario('sign_in');
});
});
Great, with above we declared that we want our sign_in state preparation script
to run before any of the following tests. Now, let's describe a scenario in which
an existing user attempts to login:
// spec/cypress/integration/user/sign_in.spec.js
describe('Sign in', () => {
beforeEach(() => {
cy.scenario('sign_in');
cy.visit('/');
});
it('signs in user with valid credentials', () => {
cy.get('#link-to-login').click();
cy.url().should('include', '/login');
cy.get('[name="spree_user[email]"]').type('[email protected]');
cy.get('[name="spree_user[password]"]').type('test123');
cy.get('input.button')
.contains('Login')
.click();
cy.get('.flash.success')
.contains('Logged in')
.should('be.visible');
});
});
The test code is self-explanatory. The user visits the index page and clicks on the login button, which redirects him to the login page. On the login page, the user enters his valid credentials and clicks "Login" button. As the credentials are valid, we expect a login success message to be displayed. Now let's verify that a user with invalid credentials cannot sign in:
// spec/cypress/integration/user/sign_in.spec.js
describe('Sign in', () => {
beforeEach(() => {
cy.scenario('sign_in');
cy.visit('/');
});
it('signs in user with valid credentials', () => {
...
});
it('does not sign in user with invalid credentials', () => {
cy.get('#link-to-login').click();
cy.url().should('include', '/login');
cy.get('[name="spree_user[email]"]').type('[email protected]');
cy.get('[name="spree_user[password]"]').type('test123');
cy.get('input.button')
.contains('Login')
.click();
cy.get('.flash.error')
.contains('Invalid email or password.')
.should('be.visible');
});
});
It's not ideal, the duplicate code is screaming at us. We will return to it, but for now, we focus on executing the tests. Thus, we run the server and the test runner as we did previously. Running the tests we should see the Cypress test runner in action:
Great! Our tests are successfully executed: Cypress test runner opens a new browser window and mimics the user actions as described in our test scenario.
Notice how the browser window is divided into two parts with the command log on
the left and our application on the right. The command log displays all the executed
commands together with application data such as network requests, page loads, etc.
Notably, with each performed action Cypress takes a new DOM snapshot. This series
of snapshots enables us to navigate the command log and see the state of our application
back in time. Furthermore, remember that we are in the browser. Open the DevTools,
inspect the DOM or click on any command from the log and see it's full description.
One can also put a debugger in the application's code or within a test
scenario (cy.debug()) and inspect the state with the browser console. Are you
working with a Redux store? Plug into that too. Overall,
the interactive test runner enables us to better understand our application and
quickly reason about our tests, which makes testing a more enjoyable experience.
Finally, let's refactor our user sign in spec to remove duplicate code. One possible solution is to use Cypress custom commands, which allow one to group together common user patterns. We extract the repeating user sign in logic into a custom command:
// spec/cypress/support/commands.js
Cypress.Commands.add('login', (email, password) => {
cy.get('#link-to-login').click()
cy.url().should('include', '/login')
cy.get('[name="spree_user[email]"]')
.type(email)
cy.get('[name="spree_user[password]"]')
.type(password)
cy.get('input.button')
.contains('Login')
.click()
})
Cypress will load the new command for us so we can start using it right away:
// spec/cypress/integration/user/sign_in.spec.js
describe('Sign in', () => {
beforeEach(() => {
cy.scenario('sign_in');
cy.visit('/');
});
it('signs in user with valid credentials', () => {
cy.login('[email protected]', 'test123');
cy.get('.flash.success')
.contains('Logged in')
.should('be.visible');
});
it('does not sign in user with invalid credentials', () => {
cy.login('[email protected]', 'test123');
cy.get('.flash.error')
.contains('Invalid email or password.')
.should('be.visible');
});
});
Add Cypress to CI
Nearly any software product involves an engineering team continuously integrating small code changes into a shared repository. Without special measures, such frequent updates by different team members can lead to accumulation of bugs. We want to prevent that. We will catch the bugs early by running our Cypress integration tests on each commit to our shared remote repository.
In addition to Cypress we will also run RSpec to demonstrate a real scenario with various test suites running in parallel. Furthermore, we will also run in parallel the individual tests within each test suit to decrease the overall build time. Specifically, we will run Cypress test files in different containers, in parallel. We will highlight the interesting configuration parts, but please find the full configuration in the project repository.
First, we declare that our build_rspec_cypress workflow consists of two jobs
that we wish to run in parallel:
# .circleci/config.yml
workflows:
version: 2
build_rspec_cypress:
jobs:
- rspec
- cypress
The cypress job will run the required general steps like database preparation and asset compilation after which it will run the Rails server in the background:
# .circleci/config.yml
- run:
name: Run rails server in background
command: bundle exec rails server -e test -p 5002
background: true
Yes, just like we did on our local machine. However, we will need to wait for the server to start before executing integration tests against it:
- run:
name: Wait for server
command: |
until $(curl --retry 10 --output /dev/null --silent --head --fail http://127.0.0.1:5002/admin); do
printf '.'
sleep 5
done
Finally, the server is up and we can run our Cypress tests:
- run:
name: Executes Cypress end-to-end tests
command: |
export TEST_FILES="$(circleci tests glob "spec/cypress/**/*.spec.js" | circleci tests split --split-by=timings)"
export TEST_FILES="$(ruby -rshellwords -e'print ENV["TEST_FILES"].shellsplit.join(",")')"
echo yarn run cypress run --project ./spec --spec $TEST_FILES
yarn run cypress run --project ./spec --spec $TEST_FILES
- store_artifacts:
path: spec/cypress/videos
- store_artifacts:
path: spec/cypress/screenshots
The test files will be split to run in parallel in multiple containers. Change parallelism: 2
in the configuration file to the desired number of containers, but make sure it is
lower than the number of test files. Finally, configure CPU and RAM according to
your needs with the resource_class entry.
Summary
In this article, we covered a lot of ground. We started with a discussion of the new Cypress browser-integrated architecture. From there we initiliazed a new Rails application to see Cypress in action. We defined our test scenarios with JavaScript and prepared application state with Ruby. Finally, we setup parallel execution of Cypress tests as part of our continuous integration process.
