Download our FREE ebook of 43 A/B testing case studies from the world's leading companies for test ideas and inspiration. Download Now

Optimizely Blog

Tips & Tricks for Building Your Experimentation Program

X

Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!

Get It Now



As coronavirus continues to wreak unprecedented havoc across the globe, the retail and ecommerce industry finds themselves in unchartered waters. Even though physical stores are closing (or closed), and companies will incur losses, online business remains vital for so many consumers. Online grocery purchasing is through the roof, and staples are outpacing apparel in this changing landscape. In this reshaped economic marketplace, the many new users and their new behaviors offer challenges and opportunities for your experimentation programs. .

What can you do to keep your experimentation program up and running?

Utilize the new users

As stores have closed and most individuals are advised and in some cases ordered to stay at home, traditional offline shoppers are now discovering and switching to online. This trend is being felt across the industry but in particular amongst grocery chains. These new users are a great audience to run experiments for as they have no previous knowledge about your online experience. Here you can run tests like:

  • Providing users more guidance throughout the funnel, such as creating guiding CTAs in the checkout funnel
  • Optimizing the display, positioning and content of your USPs
  • Focusing on security and reducing uncertainty by showcasing return policies, payment methods, reviews, and others
  • Utilizing urgency metrics such as stock levels or number of users viewing the product

Get the most out of your higher traffic

At this time you may be experiencing higher traffic than ever, especially if you have closed your physical stores. This presents a good opportunity to test right now as experiments are more likely to reach statistical significance. Utilize this time to run experiments on low traffic sites or smaller impact tests.

Users have more time to spend → help fight the boredom!

We all feel it, working from home means you have more time. This additional time might allow your users to get distracted more easily and spend more time on your site than during a busy office / business travel day. This additional time can be used in several ways:

  • Increase the usage of your recommendation feature. Sections such as “you might also like” or “popular items” might be getting more traction now. Play around with placement, pages displayed on, number of those elements and more design aspects of it such as copy and title
  • (Re-)test your auto-load / infinite scroll on category pages
  • Pre-empt the needs of your users (especially as a grocery store) and display these products / categories on the top, with more prominence, or as a banner
  • Create more inspirational content, let the user browse the site and explore new products. Explore the impact of pushing different types of content such as less popular categories

Creating Urgency through Gating

If your site is seeing a large influx of visitors and struggling to cope with it, an option to explore through experimentation is gating your site. This could be in the form of creating a landing page where people need to sign up and will be informed when they will have access.It could be a live ticker on the site telling them which position they are in (like a queue in the store).

Delivery Times & Shipment Expectations

Even though the influx of traffic and conversion may be a positive byproduct of the COVID-19 virus, you might be challenged with longer delivery times due to lack of available staff at your provider.

  • Try different communication styles to convey to your users that delivery times may vary in times like these.
  • Potentially explore the communication of your more lenient return policies or emphasize your current policies through banners or pop-ups.

Make your Customer Service the stars

If your customer service team is still able to work with full power, maybe it is worth testing how you can get users to engage more with the customer service team. For example, display the customer service number prominently on the page in the header or a banner at the top.
On the other hand, if your customer service team is overloaded with work, you could explore the opposite. In this scenario you could put more emphasis on your self-serve section, chat widgets, FAQ sections or try to preempt what the customer may want to do / need help with (for example, changing flights and making this section the first one upon landing).

If you should have questions about certain ideas mentioned here, please reach out to your Customer Success Manager. They will be happy to assist you.

Optimizely X