Case Studies

A/B testing and conversion rate optimization case studies from the world’s largest brands and companies.

How Smarter Travel Doubled Revenue by Optimizing a Booking Funnel Landing Page

travel icon

One of the most important components of a successful optimization program is velocity – the ability to build a test, run it, collect data, and put that data into action quickly. Velocity helps optimization programs gain traction and garner executive support. The faster you get up and running, and the faster you can build and execute tests, the faster you can start proving value with results. BONUS: It can also make a whole lotta cash money. Take Smarter Travel’s story, for instance.

A Mobile Website Optimization Case Study: Hotwire’s Cars Page

android phone

This is the last post in a three-part series about the optimization program at Hotwire, an Expedia, Inc. company.

At Hotwire, mobile apps and mobile web experiences are a core part of how travelers book flights, hotels, and car rentals. Travel customers’ research and booking needs are uniquely mobile, and their on-the-go experience should help them meet their goals quickly and seamlessly.

When Hotwire’s Lead Product Manager for Site Optimization started working with the Cars line of business to optimize their conversion funnels, the Cars mobile website experience hadn’t been updated for several years. This is the story of how they optimized it.

A Method for Prioritizing A/B Test Ideas That Won’t Hurt Feelings

organizing backlog of experiments

Prioritization is the million-dollar question in optimization programs. It gets at the heart of our most frequently-asked question at Optimizely: “What should I test?” A strong prioritization process arms a growth marketer with the answer: “Here’s what we’ll test, in what order, and here’s why.”

Last week, we met Pauline Marol, the Lead Product Manager for optimization at Hotwire. Pauline’s prioritization process is a framework that keeps her optimization engine running smoothly with hundreds of ideas in the backlog and helps the very best ideas surface so they can be tested quickly.

As Pauline describes it, her prioritization “removes the emotion from A/B testing.”

Organized Ideation: How Hotwire Runs 120+ Experiments Per Year

Hotwire internal testing promotion poster

Curious what optimization looks like at one of the internet’s top travel companies? In this series of three posts, we’ll dive into the optimized testing process of Pauline Marol, the Lead Product Manager for Site Optimization at Hotwire, a leading travel booking site owned by Expedia, Inc. In this first installment, we’ll explore how Pauline structures brainstorming by ‘conversion veins’. In the future, stay tuned for a deep dive into her prioritization framework for experiment ideas and a big mobile experiment win for her team.

Creating Excellent Hypotheses with Website Analytics and User Research

How to generate A/B split test ideas

For an optimization team to be successful in reaching an aggressive revenue goal, both quantity and quality of experiments must be high. To support a focus on testing velocity, a strong backlog of experiment ideas is essential.

In this third installment of our spotlight on the optimization team at Move. Inc (the operator of, we look at a couple of the thoroughly vetted experiment ideas that the team has tested.

French Girls Loves Optimization

Scotch Mornington

How many times have you seen Titanic? Enough to remember the moment Rose tells Jack to “draw me like one of your French girls”? Well, a group of iOS developers from Scranton, PA remember… and they created an app inspired by it.

The app has risen in popularity over the last year, surpassing 1 million downloads in July 2014. With A/B testing, French Girls’ lean team is turning the majority of those downloads into actively engaged, activated users. Here’s how they’re doing it, lessons they’re learning along the way, and why they named the app French Girls.

Unconventional Places to Find Ideas for Your Next A/B Test

Investigator searching for data with magnifying glass

Before you can come up with meaningful A/B tests for your e-commerce store, you need to understand what your customers value when making purchases and what turns them away. The traditional method of collecting customer feedback via surveys has drawbacks.

Other, more unconventional, sources of information exist that can be extremely helpful in generating ideas for A/B tests. They are your customer service records, public forums, competitor data, and internal search keywords.

The Impact of Symmetry in Online Marketing


You may have heard that keeping your ad consistent with your landing page is good. But is that a myth? And if it isn’t, how can you quantify the impact of this symmetry? In early 2013, we performed an experiment on our Pay Per Click landing page aimed at answering these questions.

Our goal was more to answer an interesting question than to find a conversion lift, but we were pleasantly surprised to discover that making our landing page mirror our ads led to a tremendous lift in conversions.

2 Controversial Site Redesigns That Should Inspire You to A/B Test

Original version of Netflix page.

When it comes to making better data-driven decisions, the sooner the better. Often the temptation is to wait until after the redesign is done or the feature is rolled out. “Oh, we’re doing a redesign; we’ll do the A/B testing afterwards.” The fact is you actually want to A/B test the redesign. One story from Digg and another from Netflix show why.

Around 2010, Optimizely co-founders Dan Siroker and Pete Koomen were introduced to the folks at Digg by their new VP of Product Keval Desai to talk about using Optimizely. Their response was, “We are busy working on a complete overhaul of our site. After we do that, then we’ll do A/B testing.”