Optimizely Blog

Grow your Optimization and A/B testing skills

X

Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!

Get It Now




facebook-like-button-redesignOne of the most iconic and frequently clicked buttons on the internet is getting a redesign. The Facebook like button that 1.6 billion users click more than 6 billion times per day is officially being replaced with six buttons to better capture the range of emotions we exhibit on Facebook — like, love, wow, haha, angry, sad. If you’re thinking, OK…so what? think again, because this is huge. The way Facebook designed Reactions — their name for this new capability — demonstrates the magnitude of this change.

Before taking the redesign plunge, Facebook did their homework: they reportedly studied user behavior, did user-testing, ran a test-pilot in multiple countries, and measured every interaction with Reactions. The way Facebook redesigned the like button is a shining example of data-driven decision-making and customer empathy in action. It’s a lesson on why testing is the real secret sauce to making smart, customer-first decisions.

Companies who embrace this approach are going to outperform those who don’t. They will acquire and retain users more efficiently because they can mitigate their risks while still doing fearless things. They will ultimately go farther because they are designing experiences for what their customers need.

this-is-hugeEmpathizing with the customer and making decisions with data instead of just relying on intuition is what is keeping online travel giant, Priceline.com, ahead of competitors in the hyper-crowded world of online travel booking.

It’s what gave validation to a new feature that online retailer Trunk Club rolled out on its mobile app, providing proof that the new feature increased sales by 9%.

It’s what Sony used to increase computer purchases by 20%.

Empathetic, user-centric choices are table stakes for companies who provide great digital experiences. If you are serious about designing delightful customer experiences, it’s expected that you’re testing.

NetflixLogoTake Netflix for example. Over the years, their interface has evolved drastically, and at every step of the way, the risk of making those changes is mitigated with A/B and multivariate tests.

This was the case in 2011 when they rolled out a new design to the Watch Instantly web interface — scrollable rows with thumbnails instead of a sparser page with 5 titles to choose from.

user-centric-decisionsWhile user comments and blogosphere reactions may have been negative, the data proved that the new, denser experience increased engagement metrics.

“What people say and what they do are rarely the same,” Bryan Gumm, Netflix Manager of Experimentation told Optimizely in an interview. “We were looking at the metrics and people were watching more, they liked it better, and they were more engaged in the service.”

Netflix is transparent about its in-house solution to testing and the frequency with which it tests to weed out what doesn’t work and implement design that does. Netflix mixes a deep empathy for their customer with rigorous hypothesis testing, and it’s working; year over year, they are growing revenue 22.8% (source).

Secret Escapes LogoAnother growing company applying empathy and testing to answer crucial questions about their digital properties is Secret Escapes, a flash-sale luxury travel company.

Since launching in 2011, it has reported 260% YoY growth, and currently boasts 10 million+ users between its iOS/Android apps and website. With this kind of momentum, Secret Escapes cannot afford to make changes to any of their digital properties that might throw off their competitive edge. That’s why Secret Escapes runs all app and website changes—no matter how small—as Optimizely experiments before rolling them out live.

“Virtually no change will go live on the website just like that. Everything is tested,” Sebastian Fallert, the company’s UK General Manager, said.

User acquisition and retention is a top priority for Secret Escapes. Before releasing their mobile app, the team had a very important question to answer that applies broadly across industries: should the app require users to sign up to view travel deals? The debate between launching an open app or a closed app could have been catastrophic because the team making the decision had a hunch but no answers.

testing-secret-sauceWisely, the team decided to run an A/B test on the experience and gather data to inform their decision. They quickly learned that — despite their initial qualms — requiring sign up to view travel deals led to positive business outcomes, doubling the signup rate and increasing customer lifetime value.

Without having data to back up decisions, feature rollouts and redesigns hold less validation and the backlash from the vocal minority seems louder.

As Digg.com’s former VP of Product Keval Desai once said, “You can’t [always] prevent the user backlash. But you can know you did the right thing.”

Around 2010 the online media site Digg.com infamously redesigned its website with a “big-bang launch” — no control vs. variation experiment, no small rollout. The redesign met an uproar of negative reactions and user abandonment, but since Digg wasn’t measuring the key metrics, they did not have data to validate that the new experience was actually working.

People are resistant to change, and will speak out when familiar experiences differ from the norm. That’s why it’s vital to have the data to validate your decision — to know for certain that, in the long run, the change will pay off and improve the customer experience.

“I think any good company is trying things, is forcing itself to try things, and you need to be able to put things out there and try and learn,” Chris Cox, the Director of Product at Facebook and brain behind Reactions told Bloomberg Business. “People only get in trouble if they’re not honest about failure.”

failure-chris-cox-quoteAnd without measuring, it becomes much harder to detect and admit failure.

Big or small, making changes to your digital experience is hard. At worst, a change can make users lose loyalty, feel frustrated, or “rage quit,” as Facebook puts it. You cannot afford to not apply empathy, humility, and rigorous testing to each and every decision about the customer experience.

You know change is necessary. Nothing thrives by staying stagnant. Empathizing with the customer and making decisions with data instead of relying on intuition is the not-so secret sauce Facebook and an increasing number of the world’s most innovative companies use to design their best customer experiences.

Opticon Live
comments powered by Disqus