Download our FREE ebook of 43 A/B testing case studies from the world's leading companies for test ideas and inspiration. Download Now

Optimizely Blog

Tips & Tricks for Building Your Experimentation Program

X

Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!

Get It Now

A/B Testing

A/B testing or split testing is the practice of showing two different versions of a page to users to determine which one performs better. Learn about A/B testing tips and best practices that companies have used to improve conversions on their websites and mobile apps.

42 Mobile A/B Testing Terms You Need to Know


42 Mobile A/B Testing Terms You Need to Know

As mobile continues to eat the world, the number of A/B tests running on mobile applications continues to grow. While A/B testing isn’t new, the vocabulary to talk about A/B testing mobile apps is new for many people. Succeeding in mobile A/B testing starts with the right vocabulary. Following in the footsteps of the 14 CRO Terms You Need to Know article, we have created one for a mobile lexicon. This alphabetized glossary of 42 terms and examples will help you communicate about mobile optimization.

Lessons We Learned from Email A/B Tests in 2014

Lessons We Learned from Email A/B Tests in 2014

This year, the Marketing Automation team at Optimizely got serious about tracking our email A/B tests. In 2015, Optimizely will be taking experience optimization to the next level, and continue to rigorously test campaigns in order to provide the best experience for our customers.

Looking back at some A/B tests we ran this year, here are some lessons learned, along with where I hope to take our email experiments in 2015.

5 Actionable Lessons Learned From Real Heat Map Case Studies

5 Actionable Lessons Learned From Real Heat Map Case Studies

I was more than excited when I first heard about the concept of heat maps a couple of years ago. “So you’re telling me I can track all my visitors and then see exactly where they click and how often?!” – is what I was thinking.

What’s truly great about heat maps is that they give you a clear visual representation of the raw data you’d otherwise get from Google Analytics.

So today, I’d like to share with you some of the things we’ve learned from more than 1 million heat map clicks.

Experiments to Drive More App Store Rankings & Reviews


Experiments to Drive More App Store Rankings & Reviews

Being the taco-fiend I am, I had to download the brand new, highly anticipated, Taco Bell app to check it out. Between the bright images and ease of use, I was very impressed (and more hungry).

My experience ordering a Burrito Supreme was delightful for the most part. Mid-order, a text message interrupted me (rude) and when I foregrounded the app to continue ordering, it asked me, “Wanna Rate the App?”

This experience inspired a question: when could Taco Bell ask me for a review to increase my chances of actually doing it?

5 Ways Phased Rollouts Will Protect Your iOS App


5 Ways Phased Rollouts Will Protect Your iOS App

Have you ever deleted an iPhone app you paid for? I have, albeit very begrudgingly. The app was Rise, an alarm clock that made waking up beautiful. I used it all the time, until they released an update that made it crash every time.

The buggy-release-causes-backlash-of-reviews-leads-to-better-release-to-fix-bugs scenario is pretty common. The point here is that these kinds of releases don’t have to be so common…thanks to phased rollouts.

The Problem With A/B Testing Success Stories

The Problem With A/B Testing Success Stories

Like many, I learned about the principles and practicalities of A/B testing from online articles and resources. From the time I was first introduced to split testing, I’ve made conversion optimization my full-time profession.

A year later, I noticed something alarming about the stories I was once so excited to read: success stories about A/B testing are bad for you.

Continue reading to learn why they’re bad, how to reverse their damage, and how to multiply the value of your tests without the help of success stories…

Unconventional Places to Find Ideas for Your Next A/B Test

Unconventional Places to Find Ideas for Your Next A/B Test

Before you can come up with meaningful A/B tests for your e-commerce store, you need to understand what your customers value when making purchases and what turns them away. The traditional method of collecting customer feedback via surveys has drawbacks.

Other, more unconventional, sources of information exist that can be extremely helpful in generating ideas for A/B tests. They are your customer service records, public forums, competitor data, and internal search keywords.