Download our FREE ebook of 43 A/B testing case studies from the world's leading companies for test ideas and inspiration. Download Now

Optimizely Blog

Tips & Tricks for Building Your Experimentation Program

X

Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!

Get It Now

A/B Testing Case Studies

One of the best way to come up with A/B testing ideas is to learn from the experience of others. On this page you will find A/B testing case studies and conversion rate optimization case studies from the world’s largest brands and companies. We hope these stories will inspire you and help generate test ideas you can implement on your own website.

A Method for Prioritizing A/B Test Ideas That Won’t Hurt Feelings

A Method for Prioritizing A/B Test Ideas That Won’t Hurt Feelings

Prioritization is the million-dollar question in optimization programs. It gets at the heart of our most frequently-asked question at Optimizely: “What should I test?” A strong prioritization process arms a growth marketer with the answer: “Here’s what we’ll test, in what order, and here’s why.”

Last week, we met Pauline Marol, the Lead Product Manager for optimization at Hotwire. Pauline’s prioritization process is a framework that keeps her optimization engine running smoothly with hundreds of ideas in the backlog and helps the very best ideas surface so they can be tested quickly.

As Pauline describes it, her prioritization “removes the emotion from A/B testing.”

Organized Ideation: How Hotwire Runs 120+ Experiments Per Year

Organized Ideation: How Hotwire Runs 120+ Experiments Per Year

Curious what optimization looks like at one of the internet’s top travel companies? In this series of three posts, we’ll dive into the optimized testing process of Pauline Marol, the Lead Product Manager for Site Optimization at Hotwire, a leading travel booking site owned by Expedia, Inc. In this first installment, we’ll explore how Pauline structures brainstorming by ‘conversion veins’. In the future, stay tuned for a deep dive into her prioritization framework for experiment ideas and a big mobile experiment win for her team.

Creating Excellent Hypotheses with Website Analytics and User Research

Creating Excellent Hypotheses with Website Analytics and User Research

For an optimization team to be successful in reaching an aggressive revenue goal, both quantity and quality of experiments must be high. To support a focus on testing velocity, a strong backlog of experiment ideas is essential.

In this third installment of our spotlight on the optimization team at Move. Inc (the operator of realtor.com), we look at a couple of the thoroughly vetted experiment ideas that the team has tested.

To Make Optimization Stick, Focus on a Quantifiable Goal

To Make Optimization Stick, Focus on a Quantifiable Goal

As we discussed in the first installment of the Move story, Investing in Optimization – The First Steps Towards Building a Team, a key component of the success of the optimization program was their ability to communicate. For the core team, this enabled them to seamlessly do cross-functional work, remove blockers, and execute their experiments at a rapid pace.

In this post, we’ll talk about why communication was an essential for another part of the company’s testing culture: setting and communicating a primary goal to raise awareness and understanding of optimization internally.

French Girls Loves Optimization

French Girls Loves Optimization

How many times have you seen Titanic? Enough to remember the moment Rose tells Jack to “draw me like one of your French girls”? Well, a group of iOS developers from Scranton, PA remember… and they created an app inspired by it.

The app has risen in popularity over the last year, surpassing 1 million downloads in July 2014. With A/B testing, French Girls’ lean team is turning the majority of those downloads into actively engaged, activated users. Here’s how they’re doing it, lessons they’re learning along the way, and why they named the app French Girls.

Optimizing Content: How Kevy Writes More Without Writing Worse

Optimizing Content: How Kevy Writes More Without Writing Worse

Brooke Beach has a challenge common amongst many: producing a lot of content with limited resources without sacrificing quality.

Sound familiar?

Her marketing team has come up with a system that combines data from website analytics, marketing automation, and live chat to help create the right content for the right audiences. Intrigued as to how live chat contributes to this optimization equation, I talked to Brooke about how they go about it, and the impact it’s had on the business.

How HubSpot Personalizes Every Customer Touchpoint

How HubSpot Personalizes Every Customer Touchpoint

At HubSpot, optimization is something we focus on every single day. It helps us deliver more engaging experiences to our customers and see greater returns on the content we push across our marketing channels.

We test and personalize just about everything. Here are a few examples of experiments we’ve run that have measurably improved our marketing programs.

The Impact of Symmetry in Online Marketing

The Impact of Symmetry in Online Marketing

You may have heard that keeping your ad consistent with your landing page is good. But is that a myth? And if it isn’t, how can you quantify the impact of this symmetry? In early 2013, we performed an experiment on our Pay Per Click landing page aimed at answering these questions.

Our goal was more to answer an interesting question than to find a conversion lift, but we were pleasantly surprised to discover that making our landing page mirror our ads led to a tremendous lift in conversions.

NY Public Library Has 100 Years Worth of Content to Optimize [Video]

NY Public Library Has 100 Years Worth of Content to Optimize [Video]

With 100 years of existence, 88 different lending libraries, four research facilities throughout Manhattan, Staten Island and the Bronx, the New York Public Library has a lot, a lot, of content. Informing people that this content collected over the last century exists is a major goal for the library. One way they achieve this goal is with an opt-in email newsletter.

The Experience Optimization Playbook: Get More From Your Digital Marketing [eBook]

The Experience Optimization Playbook: Get More From Your Digital Marketing [eBook]

Figuring out the right experiences to deliver customers is one of the most valuable questions any marketer can answer. Learn how to convert more visitors, more often with our free guide.

Inside, you’ll find infographics, worksheets, killer stats, and winning optimization strategies from companies like Google, Upworthy, HubSpot, Stanley Black & Decker, Movember, and more.