Case Studies

One of the best way to come up with A/B testing ideas is to learn from the experience of others. On this page you will find A/B testing case studies and conversion rate optimization case studies from the world’s largest brands and companies. We hope these stories will inspire you and help generate test ideas you can implement on your own website.

How JustGiving is Raising Millions More Online with Simple Experiments

The product team at JustGiving, has done a lot of work over the years to increase social sharing: we’ve added more social sharing functionality and made it easier to share.

In January this year, we decided to run our first test to see if we could increase post-donation sharing by appealing to different motivations. So we designed a simple A/B test on the copy to see if a different message would increase the proportion of people who shared news about their donation with their social network. Given the volume of donations we process, a small change could have a massive impact.

13 E-commerce Conversion Optimization Case Studies

shopping carts ecommerce conversion optimization

Ideas for A/B tests can come from the darnedest places. But there’s nothing like case studies from other similar companies to get the hypothesis juices flowing. Here are 13 e-commerce conversion optimization case studies from Optimizely customers. Each one has a clear takeaway you can use as inspiration for your own test hypotheses. Scroll forth and grow that list of experiment ideas!

How Secret Escapes Increased LTV and Doubled Signup on Mobile

Secret Escapes iOS app

Secret Escapes is a flash-sale luxury travel company that offers discounted deals on four and five-star hotels around the world. The business is growing quickly with 300% YoY growth, expanding to markets in Germany and the U.S. with a base of 10 million+ members on their iOS/Android apps and website.

As a high-growth business in the extremely competitive online travel market, Secret Escapes cannot afford to make changes to any of their digital properties that might throw off this momentum. That’s why Secret Escapes runs all app and website changes—no matter how small—as Optimizely experiments first before rolling them out live.

How Testing Everything Transformed Blu Dot’s Retail Business

Blu Dot tests everything. Really. I know what you’re thinking. Is testing everything possible? Is it even practical? The answer is yes on both counts. This is the very true story of a company that, within two years, evolved from never having conducted a split test to an organization that is continuously developing, prioritizing and validating hypotheses—big and small, broad and targeted.

A Method for Prioritizing A/B Test Ideas That Won’t Hurt Feelings

organizing backlog of experiments

Prioritization is the million-dollar question in optimization programs. It gets at the heart of our most frequently-asked question at Optimizely: “What should I test?” A strong prioritization process arms a growth marketer with the answer: “Here’s what we’ll test, in what order, and here’s why.”

Last week, we met Pauline Marol, the Lead Product Manager for optimization at Hotwire. Pauline’s prioritization process is a framework that keeps her optimization engine running smoothly with hundreds of ideas in the backlog and helps the very best ideas surface so they can be tested quickly.

As Pauline describes it, her prioritization “removes the emotion from A/B testing.”

Organized Ideation: How Hotwire Runs 120+ Experiments Per Year

Hotwire internal testing promotion poster

Curious what optimization looks like at one of the internet’s top travel companies? In this series of three posts, we’ll dive into the optimized testing process of Pauline Marol, the Lead Product Manager for Site Optimization at Hotwire, a leading travel booking site owned by Expedia, Inc. In this first installment, we’ll explore how Pauline structures brainstorming by ‘conversion veins’. In the future, stay tuned for a deep dive into her prioritization framework for experiment ideas and a big mobile experiment win for her team.

Creating Excellent Hypotheses with Website Analytics and User Research

How to generate A/B split test ideas

For an optimization team to be successful in reaching an aggressive revenue goal, both quantity and quality of experiments must be high. To support a focus on testing velocity, a strong backlog of experiment ideas is essential.

In this third installment of our spotlight on the optimization team at Move. Inc (the operator of, we look at a couple of the thoroughly vetted experiment ideas that the team has tested.

To Make Optimization Stick, Focus on a Quantifiable Goal


As we discussed in the first installment of the Move story, Investing in Optimization – The First Steps Towards Building a Team, a key component of the success of the optimization program was their ability to communicate. For the core team, this enabled them to seamlessly do cross-functional work, remove blockers, and execute their experiments at a rapid pace.

In this post, we’ll talk about why communication was an essential for another part of the company’s testing culture: setting and communicating a primary goal to raise awareness and understanding of optimization internally.

French Girls Loves Optimization

Scotch Mornington

How many times have you seen Titanic? Enough to remember the moment Rose tells Jack to “draw me like one of your French girls”? Well, a group of iOS developers from Scranton, PA remember… and they created an app inspired by it.

The app has risen in popularity over the last year, surpassing 1 million downloads in July 2014. With A/B testing, French Girls’ lean team is turning the majority of those downloads into actively engaged, activated users. Here’s how they’re doing it, lessons they’re learning along the way, and why they named the app French Girls.

Optimizing Content: How Kevy Writes More Without Writing Worse

Workspace station

Brooke Beach has a challenge common amongst many: producing a lot of content with limited resources without sacrificing quality.

Sound familiar?

Her marketing team has come up with a system that combines data from website analytics, marketing automation, and live chat to help create the right content for the right audiences. Intrigued as to how live chat contributes to this optimization equation, I talked to Brooke about how they go about it, and the impact it’s had on the business.