Our home page, www.optimizely.com, has a red call-to-action button letting people test their website in the Optimizely editor.  

The Optimizely.com homepage with the winning variation, “Test it out!”.

We wanted to see what effect the button call-to-action had on conversion rate, so we ran an experiment changing the button text to the following three variations:

The Original Page used “Get Started!”

The first variation used “Try it out!”

The second variation used “Test it out!”

The third variation used “Give it a try!”

For this experiment, we’ll look at the get_started goal, a tracked element we created in the editor to track when a visitor clicks the button.

Here’s what the results page looks like, with the dates (x-axis) redacted for confidentiality:

The Experiment Created goal on the results page.

Let’s break this down.

First, note that we’re looking at the get_started goal.  There are a number of other goals we were measuring on the results page, but this is the one we want to focus on for this discussion.

Winner

In the top right corner, you can see we know there’s a winner.  It says that the “Test it out!” variation is beating the original page by 29.6%.  This means that the conversion rate of  this variation is 29.6% higher than the conversion rate  of the original page.

Test it out is the winning variation.

Conversion Rates

The statistics tell us exactly how many visitors converted per variation and the total number of visitors who saw each variation.  Calculating converted visitors / total visitors gives the conversion rate , in the second column.

Conversion rates calculated from visitor data.

Error Bars

The error barsshow you a visual representation of the original page’s observed conversion rate. The narrower the error bar the more likely the conversion rate falls within the range. The wider the error bar the more uncertain of the conversion rate. 

The fourth variation beats the original page, even when taking in to account statistical error.

As you can see, both “Test it out!” and “Give it a try!” are winners.  Their conversion rates  are higher than the original page’s conversion rate, even taking in to account the statistical error.  Visually, you can see this by the fact that the green box (representing the variation) does not overlap the gray box (representing the original page).  Additionally, the rows of winners are styled green.  (Technically, we style a row green whenever the chance to beat original is greater than 95%)

Graph

Finally, the graph shows us the conversion rate for each variation over time.  You can click on the graph to change the scale, as well as toggle which goals are shown by clicking on the line in the legend.

The graph shows you your conversion rates over time.

We hope that you like the new results page!  And if you’re new to Optimizely, well, why don’t you test it out?