Download our FREE ebook of 43 A/B testing case studies from the world's leading companies for test ideas and inspiration. Download Now

Optimizely Blog

Tips & Tricks for Building Your Experimentation Program

X

Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!

Get It Now



The global Coronavirus outbreak has had an enormous impact on the travel industry. Flight destinations have had to be changed. Trips have been canceled. Hotels are being shuttered. Not to mention a global scramble by expatriates and students to get across international borders before they close. 

The word that is continuously in use right now is ‘unprecedented’ which makes running your business even more challenging. In this light here is the second of our series on how to manage and make use of Optimizely in these times.

What can you do to keep your experimentation program up and running?

  1. Focus on what your users need right now
    The focus for most visitors on your site will be to understand what parts of their trip they can change, get a refund for or move to another date. The “My Trip” section has never gotten this much traffic.

    1. Make the “My Trip” section very prominent, easy to find, and ensure it works.
    2. Provide your users with the required information and new policies in effect, and clearly dated. Give them a feeling of security and of being in good hands.
    3. Ensure that users understand how to change their trip → you could run a quick test on this?
    4. Use this high traffic to run tests that you previously thought would never reach statistical significance due to low traffic.
  2. The opportunistic user
    While it is unclear when and how international travel will resume it is also true that flight prices are currently at an all-time low for many destinations. There are a select number of opportunistic users that are assessing their options and looking to book.

    1. Utilize your low prices to the extent possible to fill seats or rooms. Ensure these users see offers prominently on your site. Test your offer section.
    2. Flexible cancellation policy probably has never gotten more (or better) marketing than now. Understand how you can leverage this for example by testing how and where to display upgrades to flexible cancellation policies.
  3. Think positive, think ahead
    Travelers’ lives have been put on pause, but the appetite and desire to travel continues to grow. Once the threat is better understood, pent-up demand may cause users to book rapidly to travel frequently and for extended periods. However, while the threat remains, we should focus on inspiring travelers’ imaginations and driving awareness for new locations or experiences.

    1. Help your users to plan their next trip. Do not turn the whole website into a COVID-19 site with no ability to be inspired for the next vacation. Run experiments on your user flows and your inspiration pages. How can you help your users “dream” of their next adventure? How can you explore ways to help users with easy planning features? Focus on the “dream” and “plan” phase of your user journey.
  4. From High to Low Traffic
    Potentially you had a high traffic page with many thousands of users a day and experimentation and reaching statistical significance was not a challenge for you. But what now? Traffic is down and stats sig has stalled?

    1. There are various options and techniques to mitigate challenges with low traffic sites. Some can be found here.
    2. See the section below around Stats Accelerator & Multi-Armed Bandit testing in the previous blog.
  5. Make your Customer Service the stars
    If your customer service team is still able to work, maybe it is worth testing how you can get users to engage more with the customer service team. For example, display the customer service number prominently on the page in the header or a banner at the top.

    On the other hand, if your customer service team is overloaded with work, you could explore the opposite. In this scenario, you could put more emphasis on your self-serve section, chat widgets, FAQs or try to preempt what the customer may be seeking, such as changing flights and make this section the first one upon landing.

If you should have questions about certain ideas mentioned here, please reach out to your Customer Success Manager. They will be happy to assist you.

Optimizely X