I’m loving the fearlessness happening in the world of testing. In Optiverse this week, there was a great post from Russell Lee at Maplesoft titled “Winning Tests: How I Learned to Stop Worrying (About Data) and Love the Bomb.” The title caught me right away, but the content inspired me.

Radical hypothesis

He shared a success story about about how his radical hypothesis for a test turned out to be a success. He hypothesized that a drastically simplified design would create a winning landing page—even if it meant completely dismantling the previous design. If you’ve ever worked on a project like this, you know that it is incredibly difficult to radically change a piece of content that is “working.” It takes investment and prioritization within design, marketing and development to get this change through the approval process, and it’s even harder if the page has high visibility and traffic.

For Russell, the change was extreme, and he achieved a radical result with the new variation—a 611.1% improvement in conversion with a 98% confidence level.

Baseline:

Variation:

What Russell said about the results really struck a chord with me:

I’m sure some people would say, ‘of course your new variation worked, look how much more attractive it looks’, or that my reasoning and approach are sitting on the edge of extremist reductionism. Well, I guess if I was being honest about it—yeah, those people would be 100% correct. A/B testing gives us the ability to be data-driven in our approach, but too often we forget that it also empowers us to take greater risks and liberties in our designs. I’m not saying that we should completely push data aside in favor of instinctual design—I’m just trying to say that a better balance can exist between the times you need to leverage your analytics, and the times you need to take action directly to your design.

Moving out of the comfort zone

Another fearless story was from Alix George who works at Holiday Extras Ltd. titled “One of our winning tests using Optimizely 🙂”  (She had me from the happy face).  She shared a test that brought Holiday Extras outside of their “comfort zones.”  Their German side of the business wanted to drive bookings across their German sites, but also wanted to make sure they didn’t strip away any key messaging as German customers prefer to research products and pricing before completing their bookings.

They decided to create a variation where the majority of the SEO content, navigation bar, an external links were removed. The end statistically significant result was a 49.19% increased booking rate and a 89.1% increased search-through rate.

Baseline:

Variation:

What’s great about this story is that Alix was fearless in her pursuits, and now has the confidence to make continue running experiments that are outside of the comfort zone.  In her own words:

“Running this kind of test meant that we didn’t need to invest any developer time before we had proof of concept—I could jump in, make the changes, and go live within an hour (including testing time). Now we can build those changes in with full confidence in the uplift value.”

Fearlessness as a value

At Optimizely, one of our core values is fearlessness—to take on big things and to not fear failure. It’s really inspiring to see that our customers also hold this as a core value and are pushing the boundaries with their experiments every day.

To foster the fearless spirit, we’ve launched a specific space within Optiverse to discuss “Experimental Marketing.” Within this forum, we encourage experimenters to share and discuss the most bold, imaginative and experimental endeavors across the marketing universe. I encourage you to take a look and share your own fearless endeavors!