Download our FREE ebook of 43 A/B testing case studies from the world's leading companies for test ideas and inspiration. Download Now

Optimizely Blog

Tips & Tricks for Building Your Experimentation Program


Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!

Get It Now

Expression emoticons

It’s that crazy time of year again. Black Friday and Cyber Monday are in the rear view, and the web is still buzzing with holiday shopping. For many online retailers, this means heavy site traffic and record sales. It also means a flood of additional customer data.

Earlier this week a client of ours commented on the overwhelming pile of site metrics they were evaluating. They noted that although they look at key metrics closely related to sales (e.g. revenue, add to cart clicks, check outs etc), they sometimes forget to look deeper into other key metrics and analytics for customer insights and behavioral understanding (e.g. traffic sources, time on site, click paths, product views etc.).

I agreed that sifting through all that data to unearth ideas for optimization strategies is a real challenge. It’s easy to get laser focused on specific metrics and overlook the fact that behind all those graphs and charts is a customer with feelings and emotions.

Below are some thoughts that can help you avoid “metric tunnel vision” and keep your optimization efforts human through the holidays, or any other time of high site traffic.

Celebrate diversity 

Even the most targeted websites reach a relatively diverse audience. Be sure to identify and define shopper segments within your primary audiences. To keep track of all the things you learn about each sub-group through data analysis, try creating a pin board of images, descriptions, attributes, behaviors and motivations. This collage of information and visual aides will help you identify each segment and remember their differences, preferences and motivations. Once you form a persona, test your learnings with A/B tests targeted to specific audiences.

Pin board for identifying buyer persona segments

Click to view in new window.

This pin board was used to help identify a targeted sub-audience of aging males. It helped provide understanding around what language was most effective for communicating with the segment. Testing confirmed the demographic responds strongly to savings-related messaging, providing a powerful tool to drive conversions.

TIP: Over the holidays, expand this practice to include specific sub-segments surrounding gift giving: early season shoppers, last minute shoppers, industry-familiar shoppers, industry-novice shoppers, etc.

Related Blog Post: 7 Essential Customer Segments for E-Commerce Sites

Speak their language

Tone is set by the copy and language used on your site. It can have a significant impact on engagement and conversion. Use language that not only aligns with your brand but also relates authentically to the target audience. Speaking your customers’ language will give them comfort and confidence to convert.Earn messaging vs. invite a friend test

While testing calls to action for an outdoor gear retailer’s refer-a-friend program, we found active outdoor enthusiasts are significantly more motivated by ‘Earn Rewards’ than ‘Get Rewards.” This finding is understandable considering the audience tends to be competitive among peers, lives the ‘no pain no gain’ mantra, and wants to ‘earn their turns’ on the mountain. In the case above, “Earn $25. Invite Friends.” increased program engagement from the baseline “Invite a Friend. Get $25.” by 60%.

TIP: Over the holidays, everyone is down for a great deal so experiment with how you message deals to each targeted sub-group. Consider exploring variations in levels of urgency, humor, holiday cheer etc.

Related Blog Post: Test Ideas for Holiday Season Hero Messaging

Plan for mood swings  

It’s important to recognize a single customer can behave very differently depending on time-of-day, current site offerings, their buying decision process, and even the weather. The holidays are a prime time to learn more about these differences, as many users are on an accelerated conversion paths as they discover, learn, share and convert quickly. Obvious behavioral shifts take place on iconic shopping days, such as Black Friday and Cyber Monday, but what about shopping behaviors on more typical days?

Optimization of large flash sale websites reveals two notably distinct shopping patterns. We see the impulsive ‘get-in-and-get-out’ flash sale deal shopper (triggered by morning emails) often become a more casual browsing shopper in the evenings.

Flash sale app view

Revenue results for a/b test

Click to view in new window.

In the test above, the addition of brand specific navigation (the “View All” tab on the product tile below) was highly effective in the afternoon and evening for browsing moods, but was detrimental to sales in the morning for flash sale ‘deal’ shoppers. The learnings around these two shopping moods, has proved a powerful tool in driving time-based traffic conversions.

TIP: Be sure to consider customer tendencies based on exterior factors that may alter their buying behavior (even on the same day). Experiment and optimize for all of your customer ‘moods’.

Related Blog Post: 12 Obvious and Not-So-Obvious Test Ideas for Online Retailers

So remember, while geeking out on the numbers, charts and graphs is great, don’t forget that there are people behind all that data and those people want to have a meaningful relationship with your website and your brand.

While optimizing over the coming months, be sure to:

  • Celebrate Diversity—Break down data by audiences sub-segments to further inform optimization decisions
  • Speak Their Language—Experiment with tone relevant to the sub-group motivations, seasonal incentives and varying degrees of urgency
  • Plan for Mood Swings—Don’t forget a single user can exhibit a variety of different behavioral characteristics depending on external factors and offers

Take the time and step back from all those numbers, and think about the faces behind them, and this just may be the most lucrative holiday season yet. Happy selling!

Optimizely X