Case Studies

A/B testing and conversion rate optimization case studies from the world’s largest brands and companies.

Creating Excellent Hypotheses with Website Analytics and User Research


How to generate A/B split test ideas

For an optimization team to be successful in reaching an aggressive revenue goal, both quantity and quality of experiments must be high. To support a focus on testing velocity, a strong backlog of experiment ideas is essential.

In this third installment of our spotlight on the optimization team at Move. Inc (the operator of realtor.com), we look at a couple of the thoroughly vetted experiment ideas that the team has tested.

French Girls Loves Optimization


Scotch Mornington

How many times have you seen Titanic? Enough to remember the moment Rose tells Jack to “draw me like one of your French girls”? Well, a group of iOS developers from Scranton, PA remember… and they created an app inspired by it.

The app has risen in popularity over the last year, surpassing 1 million downloads in July 2014. With A/B testing, French Girls’ lean team is turning the majority of those downloads into actively engaged, activated users. Here’s how they’re doing it, lessons they’re learning along the way, and why they named the app French Girls.

Unconventional Places to Find Ideas for Your Next A/B Test

Investigator searching for data with magnifying glass

Before you can come up with meaningful A/B tests for your e-commerce store, you need to understand what your customers value when making purchases and what turns them away. The traditional method of collecting customer feedback via surveys has drawbacks.

Other, more unconventional, sources of information exist that can be extremely helpful in generating ideas for A/B tests. They are your customer service records, public forums, competitor data, and internal search keywords.

The Impact of Symmetry in Online Marketing


symmetry

You may have heard that keeping your ad consistent with your landing page is good. But is that a myth? And if it isn’t, how can you quantify the impact of this symmetry? In early 2013, we performed an experiment on our Pay Per Click landing page aimed at answering these questions.

Our goal was more to answer an interesting question than to find a conversion lift, but we were pleasantly surprised to discover that making our landing page mirror our ads led to a tremendous lift in conversions.

2 Controversial Site Redesigns That Should Inspire You to A/B Test

Original version of Netflix page.

When it comes to making better data-driven decisions, the sooner the better. Often the temptation is to wait until after the redesign is done or the feature is rolled out. “Oh, we’re doing a redesign; we’ll do the A/B testing afterwards.” The fact is you actually want to A/B test the redesign. One story from Digg and another from Netflix show why.

Around 2010, Optimizely co-founders Dan Siroker and Pete Koomen were introduced to the folks at Digg by their new VP of Product Keval Desai to talk about using Optimizely. Their response was, “We are busy working on a complete overhaul of our site. After we do that, then we’ll do A/B testing.”

NY Public Library Has 100 Years Worth of Content to Optimize [Video]


NYPL_logo

With 100 years of existence, 88 different lending libraries, four research facilities throughout Manhattan, Staten Island and the Bronx, the New York Public Library has a lot, a lot, of content. Informing people that this content collected over the last century exists is a major goal for the library. One way they achieve this goal is with an opt-in email newsletter.

Movember Drives 28,000 More Mo’ Bros with an A/B Test

Movember Logo

Take an imaginary trip with me to the date November 1st. If your office is like many in the U.S., it will probably be buzzing with stories from yesterday’s Halloween festivities. But if your office is like mine, then a new buzz will have taken hold and be growing rapidly. I don’t mean any gossip on best/worst costumes. I mean hair on the upper lip. That’s right ladies and gentlemen, it’s Movember.