Download our FREE ebook of 43 A/B testing case studies from the world's leading companies for test ideas and inspiration. Download Now

Optimizely Blog

Tips & Tricks for Building Your Experimentation Program

If you have a website today, chances are you’re using search engine marketing (SEM) to drive more visitors to it. SEM is a powerful way to drive highly-qualified traffic– searchers who are looking for Liftopia Logosomething specifically-related to the good or service you provide – to your site. But once they get there, then what?

Website optimization can make the dollars you spend on SEM campaigns  more effective by driving more clicks, purchases, and engagement from these visitors. It’s insanely simple and extremely powerful. Here’s how Liftopia did it.

The Challenge: Maximizing Revenue from SEM Campaigns

Liftopia uses SEM ads to drive potential customers to ski deals on its site. SEM drives highly-qualified traffic – in this case, searchers with a specific resort in mind – to a landing page midway through Liftopia’s purchase funnel. From there, Liftopia’s goal is to provide the best user experience possible in order to drive more purchases and revenue from its site.

“If you’re searching for a Squaw Valley deal, our goal is to bring you into our site, remove all distractions from other resorts’ deals and offers, and keep you moving down our funnel.” – Dave Nuffer, Product Manager, Liftopia

In the past, this presented a hurdle – pages in Liftopia’s purchase funnel offered numerous options for cross-sell via a sidebar column and a search widget. When visitors landed on a page from SEM, they had the option to either buy the results, search for new results, or click directly on deals to other resorts. Liftopia wanted to determine whether tailoring their funnel pages around the resort promoted in the SEM advertisement would drive more visitors down the funnel to purchase the package.

The Hypothesis: Tailored = Better?

Since they knew what SEM-traffic was searching for already, the team hypothesized that showing a tailored experience with exactly what visitors had searched for – instead of options they didn’t want in the first place – would increase purchase conversions and decrease bounce rates.

The Test: A Multi-Page Experiment

Liftopia tested different experiences within the SEM purchase funnel and measured which converted more visitors into customers. They set up a multi-page experiment to ensure a unified experience throughout the flow. To execute the test, they created two variations across their funnel pages.

The original flow (below) included the search widget and other cross-sell options throughout the funnel.

Liftopia's original funnel pages had multiple options for cross-sell.

Liftopia’s original funnel pages had multiple options for cross-sell.

The variation flow removed the search widget and additional offer links altogether. Screen Shot 2013-05-08 at 10.19.41 AM

The Liftopia team targeted the test to apply solely to SEM traffic — this gave them the  ability to provide a highly customized experience to visitors entering their site from search.

The Results: 24% More Revenue

After just seven days, a statistically significant result emerged: the variation flow increased conversions by 23.7%.

Screen Shot 2013-05-07 at 11.17.51 PM

Dave and his team used Optimizely’s A/B testing tool to push 100% of SEM traffic to the winning variation while their engineers worked to implement the improved experience into their site code. Using Optimizely’s traffic allocation feature, the team could instantly realize the increase in revenue from their SEM ads.

The Moral of the Story: 

A/B testing makes money spent on other marketing initiatives — such as SEM campaigns — more effective. Testing different purchase funnel variations helped Liftopia better understand how SEM-driven traffic behaves and ultimately convert more visitors into customers. The Liftopia team quickly learned that the more qualified the traffic, the more targeted a page layout can be.

Optimizely X