Optimizely’s 100,000th Experiment: Total Site Redesign vs. Original Site

Some of the initial thoughts were that our existing site design had been successful with a lot of early adopters, but as we grow, so should our approach to engaging new visitors to Optimizely.

Our previous site design made a clear and simple statement of our initial offering, an easy-to-use product for website A/B testing which we captured with our tagline: “A/B Testing You’ll Actually Use.” We discussed the possible opportunity we had to re-imagine a design that would maintain our core message and brand while providing additional benefits to visitors to our homepage and more broadly capture the value of A/B testing.

One other area of focus with the new site was providing a more comprehensive approach to engaging different visitor types. We have continued to learn over time how different customer types use Optimizely to achieve a wide range of different goals. In thinking about a redesign, we determined we could explore an opportunity to provide different visitors with experiences that more closely speak to their goals, problems, and the benefits that Optimizely can provide them.

Applying the approach of ‘explore before you refine’, our design team took to developing several concepts that could possibly achieve a new look and feel that could take our site experience to the next level in several areas. At the end of the day we all agreed that the most important thing a potential tester can do on our site is enter a url and experience the WYSIWYG editor first hand. You may recall a previous experiment we ran on our site exploring different calls to action for entering the editor – the objective of the new homepage design was to accomplish all of the above without sacrificing the number of people who experience setting up a test in the editor.

After settling on a design, we were ready to put the redesigned site to the test.

Targeting

For the test, we determined we would use Optimizely targeting features to run the test on only new visitors to Optimizely. It was vital to limit this test to just new visitors in order to gather clean results that reflected each site’s actual performance. Setting up a redirect test, we launched the test, pitting both sites in their entirety against each other.

Variation Screenshots:

Homepage:

Old site New Site

About Page:

Old site New Site

Goals and Results

For the new test, we set up and measured several different goals across both sites. It’s important to track a wide spread of goals to get a holistic sense of how the new site design performs. Below are the highlights and results.

Accounts Created Successfully:The percentage of visitors who completed the ‘create an account’ sign-up form.

Uses Editor: The percentage of visitors who entered a URL on the homepage and used the Optimizely editor.

Engagement: This goal is included in every experiment created with Optimizely by default. It measures the percentage of visitors who click anywhere on the experiment page.

Visits Pricing Page:The percentage of visitors who visited the Optimizely pricing page.

Submits Free Trial Form: The percentage of visitors who submitted the free trial sign up form on the homepage.

Experiments Started Successfully: The percentage of visitors who landed on the experiment creation success confirmation page.

Conclusion

In just about every metric we measured, the new site was a clear winner against the old one. After running the test on new visitors for just over a month, we were confidently able to declare a winner and push the new site live to all visitors. Most of our thoughts about what our original site lacked were proven true. Such a large experiment bears many takeaways – we will follow up with more in a subsequent post. With a new design live, we can dive into a range of new tests already lined up as we refine from here – there is still plenty of great work to be done. Let us know what you think. We’d love your ideas, thoughts, comments and suggestions. Always be testing; we certainly will be.