To Make Optimization Stick, Focus on a Quantifiable Goal

As we discussed in the first installment of the Move story, Investing in Optimization – The First Steps Towards Building a Team, a key component of the success of the optimization program was their ability to communicate. For the core team, this enabled them to seamlessly do cross-functional work, remove blockers, and execute their experiments at a rapid pace.

In this post, we’ll talk about why communication was an essential for another part of the company’s testing culture: setting and communicating a primary goal to raise awareness and understanding of optimization internally. (We’ve written a detailed guide about selecting a primary conversion goal called Building your Company’s Data DNA. You can download it here.)

Optimization as Company-Wide KPI: Focusing on Revenue

The team chose to focus their optimization efforts on a revenue goal. This served a dual purpose: First, to help with prioritization: experiment ideas with greater potential revenue impact would be prioritized above ones with less potential to create valuable customer accounts. Second, the goal helped Suma and her team to raise visibility of the optimization and evangelize the program internally.

“If you’re doing optimization in a vacuum, the wins won’t have any meaning to the other teams.” 

At the end of 2013, the revenue goal was set and communicated across the company: 2014 would become a year where A/B testing and optimization contributed to a portion of the company’s revenue. The goal was a stretch, which drove strong focus and prioritization.

Once the team reached their first 2014 benchmark, they trumpeted their success and hosted an internal celebration, inviting members of the Design, Research, Development, and QA teams out for drinks. This positive communication helped to spread awareness and generate excitement for the optimization program internally.

Turning the Tide: A Year-Long Internal Campaign

The communication didn’t stop there; Suma and her team continued to lather, rinse, and repeat their testing and communication process. The tide began to turn as evangelizing testing evolved into helping with inbound requests to run experiments. As 2014 progressed, Suma found that she and her team were fielding more and more requests from other teams to help them set up and run their own experiments. 

According to Suma, communication and celebration were absolutely essential to the success of the optimization program at Move. She sums up their year-long process in the following steps:

The process of optimization is ongoing, but keeping team members at the company in the know is essential to recognizing the power of optimization and continuing to commit resources to the program.

As Move’s optimization program has become more visible and successful, many other teams have started to brainstorm areas where they wanted to test and optimize. Suma hopes that in 2015, the team will successfully scale best practices for ideating, prioritizing, and executing experiments.

“Many teams come to us with ideas that either are something they’re stuck on, or a small change they want to make. We take a step back and look at the big picture, help them to unpack the problem and design a meaningful experiment.”

In the meantime, Suma and her team are now focused on ramping up new channels where they want to be optimizing, including optimizing for mobile visitors.

Stay tuned for the third and final installment of the Move story, where we’ll talk about their hypothesis generation process and showcase their experiments based on analytics and user research data.