A/B Split Testing with Google Analytics

Successful search engine optimization campaigns are great for attracting website traffic to your website or blog. But without effective tracking in place, you’ll constantly spin your wheels trying to convert more traffic into leads or sales.

When visitors come to your website, what factors are impacting their willingness to begin a relationship with your business? These decisions are largely based around your site’s design, the messaging and your call-to-action, your content, and even your layout. But as you update your site, how can you be sure that an update or change is actually improving your website user experience, and increasing your conversion rates?

A/B Split Tests is a method for measuring the decisions that you have made for your website. They allow you to scientifically test and validate hypotheses around your goals.

Google Analytics’ “Experiments” tool makes it simple to setup your A/B Split Tests, and in this post, we’ll walk through each of the steps.

1. Define your Goal(s)

Some visitors’ actions tied directly to your organization’s website goals are important to setup as Goals in Google Analytics. Church and Non-profit websites should track supporter engagement and fund raising. Small Business websites should track leads and sales of their products and services.

2. Form your A/B Test Hypotheses

With your goal of increasing conversions, or leads, or donations established, it’s time to form your hypotheses: what design, messaging, content, or layout decisions do you think will help achieve your goal more effectively?

With Google Analytics Experiments, you’re not limited to just two, A and B, hypotheses. You can setup as many as you’d like to help improve your company digital marketing results. The algorithm Google uses is called a multi-armed bandit experiment. The statistics behind it are pretty complex, but the idea is not. The gist is, as time goes on and hypotheses start showing promise Google will send more visitors to the leaders. It has two big advantages over a traditional A/B test that distributes traffic evenly until completion.

First, by sending more traffic to winning variations while the experiment is running, you start to benefit from the leaders earlier. Fewer visitors are sent to poor performers before the experiment ends. A side effect of this is the other key benefit: deciding the best of multiple good choices happens more quickly.

Each hypothesis must be setup as its own page, with a unique URL, on your website. How this is done will depend on the content management system (CMS) your website uses. With a developer-friendly CMS, like WordPress, this is a simple task of cloning a page and changing its content.

3. Setup your Google Analytics A/B Split Test Experiment

With a goal and hypotheses in place, it’s time to setup your experiment in Google Analytics. This is the easy part!

Start by Creating an Experiment
You can reach your experiments in Google Analytics from the reporting sidebar. Select Behavior, then Experiments. This is how you’ll get to your experiments after you’ve set them up. Select Create Experiment.

Experiments only require a Name and the Goal you’re evaluating success with. There are other options, too, like how much of your traffic should be experimented with, e-mail notifications, the confidence level with which a winner is determined, and so on. For your first experiments, you can safely use the defaults, but consider enabling e-mail notifications to keep yourself updated often.

Drop-in Original URL and Hypothesis Variation URLs

The next step is to add all of your hypotheses to the experiment. This is just a matter of copy and pasting the URLs you setup in. If you are testing changes to a page that already exists, use it as the Original. For entirely new content, pick the URL you will eventually want to use for the winner. The Original Page is where the code snippet will be installed in step 3c.

Insert Your Experiment Testing Code
Once your hypotheses URLs are setup, the next step is to select “Manually insert the code” and install the experiment code snippet just inside the tag of your Original Page. It will look like this:

After you have saved the template, click Next Step and Google Analytics will verify that your experiment code is properly in place.

Deciding the Winner

The winning hypothesis will be decided statistically over a period of time, usually around 2 to 4 weeks. This period will vary in length depending on the volume of traffic to your site, the relative performance of hypotheses, the rate at which the leading hypotheses converts, and so on.

For experiments that end with a clear winner, if it is not the original, there are just a few more steps to take. First, replace the original’s content or template with the winning variation’s. Second, remove the experiment code from the template.

Google Analytics Experiments is a free A/B Split testing tool you can easily setup to help you make more confident decisions about your website’s design, messaging, content, and layout. It is a powerful tool you can use to make your website better at achieving your organization’s goals, and will in turn improve your return on investment.