Download our FREE ebook of 43 A/B testing case studies from the world's leading companies for test ideas and inspiration. Download Now

Optimizely Blog

Tips & Tricks for Building Your Experimentation Program

It’s not even Thanksgiving yet but Christmas decorations are already starting to pop up on shelves and banner ads everywhere. For online retailers, the oncoming holiday time means a steady increase in unique visitors and a huge opportunity to quickly iterate on website design that will make it as easy as possible for those visitors to buy something.

In August, WiderFunnel CEO and conversion rate optimization expert, Chris Goward, and Optimizely’s testing specialist, Ryan Lillis co-presented a live webinar about the A/B tests online retailers can run when online shopping peaks—Christmas time.

Watch the Recorded Webinar

In this high traffic time, Chris Goward suggests online retailers test 5 lucrative pages and offers concrete case studies of all five of them in the presentation:

  1. Your home page because it’s the most over-scrutinized and under optimized page.

  2. Your landing pages because they are the best value proposition test point.

  3. Your category pages because they are the most overlooked pages.

  4. Your product detail pages because they are the decision point.

  5. Your primary calls to action because they are the site-wide checkout entry point.

Between now and January, online retailers have a chance to gather lots of data on website design very quickly using A/B testing tools. To gather actionable data that will continue to be useful after the holidays are over, Ryan Lillis suggests running tests that will help answer a broad question instead of a very specific one. Do not ask: which image will increase conversions, this arrangement of products, or that arrangement of products? Instead ask: which image will increase conversions, an arrangement of just products, or products with models?

He goes through examples of questions to ask in order to form hypotheses that will lead to the most actionable lessons about your website.

  • Do not ask: What specific variations are we running?
  • Ask: What broad question are we trying to answer?
  • Do not ask: Will style outperform simplicity?
  • Ask: How much does prominence affect performance?

Check out the slides on slideshare to learn about the other questions not to ask or watch the recorded webinar.

Optimizely X