I'd have never expected A/B testing would be so easy with Ruby on Rails. At the very beginning I didn't even have an idea on what A/B tests were but after a little investigation I got over it using Split: a simple gem that rocks.
What is A/B testing?
Wikipedia says:
In marketing and business intelligence, A/B testing is a term for a randomized
experiment with two variants, A and B, which are the control and variation in
the controlled experiment.
In online settings, such as web design (especially user experience design), the
goal of A/B testing is to identify changes to web pages that increase or
maximize an outcome of interest (e.g., click-through rate for a banner
advertisement). A/B testing is a way to compare two versions of a single
variable typically by testing a subject's response to variable A against
variable B, and determining which of the two variables is more effective.
Requirements
In this article I'll take for granted you have already installed Ruby on Rails and Redis.
Let's start
Choose your tests
It may seem to be a foregone step but take some time to choose exactly what to test on your application. As a programmer this step may result the most complex one, so, if you are trying to improve your site effectiveness, you may consider to ask an expert for help (e.g. a web strategist or a UX designer).
Set up your Gemfile
Ok, now you know what to do so let's start by installing the gem. I chose Split for its ease of use and customization. Add the gem to your Gemfile
gem 'split', '~> 2.2.0', require: 'split/dashboard'
I'll explain later why the require parameter is needed. I suggest you to
control the version number with the ~> so you'll be sure to have 2.2.*
versions and be safe from breaking upgrades.
Start in your controller
Split allows you to create several tests at once. However, in my opinion, the best way is to start the test in your controller assigning the experiment value to an instance variable and then use it to create different pages. I found it very difficult to nest more than two tests at once and try to understand their results. The simplest way is to do as follows:
class YourController < ApplicationController
def index
@experiment = ab_test(:my_page, ["simple", "kind", "clickbait"])
end
end
By doing that your controller will start a different test for each user. You don't need to save data and remember each visit, Split will handle all the session details to make sure that every time users request the page they'll get the same experiment variant.
Warning
Running multiple experiments at once on your application is totally fine but avoid creating more than one experiment per page. You'll probably run into false positive tests, that's not A/B testing and if you need this kind of feedback you should probably check out multivariate testing. These tests are more complex than A/B tests and you'll need more visitors in order to get consistent results.
Edit related views
Now the @experiment variable will contain the experiment variant value so
let's start using it. The simplest way to do that is by using if/elsif
statements and switch your view code as you wish. Here's a simple example on how
to do that:
<% if @experiment == 'simple' %>
<p>That's a simple tutorial</p>
<% elsif @experiment == 'kind' %>
<p>Hi, can you please read this tutorial?</p>
<% elsif @experiment == 'clickbait' %>
<p>This tutorial is so cool, check it out!</p>
<% end %>
While running in the development environment you can ask Split to return a
particular experiment just by adding: ?ab_test[my_page]=simple in the URL
params. This will allow you to check if views are rendered properly.
Here's an example took from our client Floyd.
Choose wisely your trigger
Don't aim too high when you're choosing your trigger to decide if an experiment is a success. If you're thinking about directly measuring a product sale I'm sorry but that's not the correct one: we are trying to find a perceptible change. You may test your product page visits, this will probably be the most efficient trigger. After this tough decision you can configure the conversion tracking method in your controller following this pattern:
def show
# ...
ab_finished(:my_page)
# ...
end
Now every visit on the show action will be measured and saved to Redis.
Activate the dashboard
In order to check your results you need to configure Split's dashboard. But
here's the surprise: you've already done half the work. Do you remember your
Gemfile? Yes that require is needed to add the dashboard, it will include a
simple Sinatra front end application that does the trick.
The missing part is the route configuration. I suggest to put the dashboard
on the admin area so users can't see it:
Notice: I'm using Devise in this example!
namespace :admin do
authenticate :admin_user, ->(u) { u.admin? } do
mount Split::Dashboard, at: '/split'
end
end
Following this example you'll prevent curious users from visiting the dashboard
and discover your secret plans.
As you can see you'll find the dashboard on the /admin/split path.
All collected data is displayed in a simple and straightforward way:
Let Split do its job
At this point you'll probably be eager for results, you just have to wait for users to visit your website. The more you let your experiments run, the more they will be accurate and reward you with the most effective alternative.
Collect your results
When you're done testing your visitors like a mad scientist you can choose a
variant directly from the dashboard and make it default. It's a good practice
to remove completed tests and replace view code with the winning one.
Don't forget to remove the ab_test method inside your controller because
the experiment will be reinitialized on the next controller action call.
We barely scratched the surface here but it should be more than enough to get
you started with Ruby on Rails A/B testing.
