They say a picture is worth a thousand words. But what is the potential value of a single word?
Today, we are thrilled to share a guest post from Optimizely user Vitals, whose recent testing success story may shed a little light on the matter. Every day, a wide variety of A/B Testing experiments are run through our platform, ranging in complexity. We are delighted to introduce this post as the first in an on-going series intended to bring you insights and ideas from real world A/B testers of all types. Without further ado, we turn it over to Irving Weiss, Vice President, Web Marketing at Vitals.
Background
Vitals is a leading website committed to helping patients find and connect with quality doctors, hospitals and other health specialists. Every doctor on our site has a summary page that we traditionally refer to as the “Snapshot” page, and a more in-depth page that we refer to as the “Full report.” The purpose of this dual page design is to protect our in-depth data, and to quickly show users all of our data capabilities. It’s very important to us that the user proceeds from the “Snapshot” page to the “Full report.”
Experiment
One of our employees suggested we run an Optimizely test to rename the “Full report” tab to “Full profile.” The theory behind this test was that people are more apt to associate the word “profile” with a person and the word “report” with an object, making it less obvious that the “Full report” is about a doctor.
Most people on our team found it hard to believe that a single word could really have a meaningful impact on user behavior, so this experiment was definitely met with resistance. Prior to Optimizely, testing usually meant more people needed to be involved, which made for a longer process, and didn’t typically allow for a quick and easy test.
Once the experiment got the approval, it didn’t take more than a day to discover that the naysayers were totally wrong; the new treatment brought about a 65% increase to the CTR!
The original treatment: ”Full report” tab gets a 5.5% CTR
The new treatment: “Full profile” tab gets a 9.1% CTR
This Optimizely result screenshot shows the whole story:
Lessons Learned
- Even small modifications can make a whopping difference. In this particular case, not only did we get a much better CTR on the tab, but this change has lowered our bounce rate and allowed us to present our full offering to many more people.
- Listen to the ideas of all your employees. Sometimes the old timers are so stuck in their ways that they can’t think of doing things differently.
- In high traffic areas of the website, you can get a great test result in as little as a day. We did let the test run for a week just to be sure it wasn’t an anomaly.
About the Author:
Irving Weiss joined Vitals from its launch in January 2008 and was single-handedly responsible for building its visitor base. Currently, he oversees a traffic team that helps draw over 200,000 daily visitors and millions of visitors to Vitals monthly.
How did they do it?
Watch the quick video below to see how easily you can set up a text change experiment using Optimizely:
Comments, questions, or want to share an A/B testing story of your own? Let us know.
