Optimizely Blog

Grow your Optimization and A/B testing skills


Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!

Get It Now

Top 5 ISSUESA-B TestingTEAMS SHOULD FIX“Optimization is a journey, not a destination.”

We’ll often point to this statement as a guiding light for our own testing, and as a recommendation to our thousands of customers—testing and optimization are iterative, learning-driven processes that are ongoing.

Experimentation means that nothing about your digital experience should remain fixed for too long. Changes to your website or app are one thing, but what about your process, team, or culture?

Chances are, if you’re an optimizer at heart, you’re experimenting with just about everything.

In our recent benchmark report survey, we asked optimizers at companies of all sizes and industries – what would you change about your optimization efforts, if anything?

changes to optimization

Only 3% of respondents said they wouldn’t change a thing about their optimization efforts! They reported that there are always opportunities to improve, in true optimizer fashion. The changes that they mentioned fell into two main categories:

  • Changes to the people working on optimization
  • Changes to their optimization processes

These are difficult, but approachable challenges that we’ve researched and written articles about before. Here is a roundup of articles and resources to help you:

  • Get more engineering and design resources for your testing program
  • Add new members to your optimization team
  • Establish more consistent processes for optimization
  • Decide on your overarching goal/success metric for optimization
  • Secure buy-in for optimization from your executive team

1. What to Say When: You Need Engineering and Design Help to Run More Complex A/B Tests

You’re not alone: 53% of the optimizers we surveyed for our Benchmark Report plan to increase their budget for optimization in the next year. Hiring additional technical help was high on their wish list when we asked about why they were planning a budget increase.

There’s no question; if you had technical know-how on your side, your optimization program could scale faster, experiment with bigger, more complex tests, or move from testing on the marketing site to testing in your product.

Educate and align your design and technical teams by addressing their concerns and speaking their language:

changes to optimization team structure

  • Show how optimizing through a redesign process can help ensure that a new design is both performing well and visually delightful. We love these redesign examples from Smartwool and Soccerloco.
  • They might not know that A/B testing is for engineers too, according to Kyle Rush, Director of Frontend Engineering and Optimization at Hillary for America. When he led optimization at the Obama 2012 campaign, his team A/B tested a new architecture for processing donations, and discovered a huge bug in the process.

If you’re in need of resources, bringing in an agency partner to help can be a flexible short or long-term solution. Take a look at Optimizely’s directory of approved Solutions Partners for more information.

2. What to Say When: You Want to Hire Dedicated Team Members for Optimization

41% of optimizers would like to add more people to their teams in order to scale their efforts.

Maybe you’re only just getting started with optimization, or you’ve been championing optimization while doing all of the work yourself. You want to scale your program, but you need help. Here are some of the roles like you are looking to add to their teams:

changes to optimization people

If you’re looking to bring on more help, share these resources with your higher-ups:

Already have a job req ready to go? Post open positions to our Job Opportunities board in Optiverse.

3. What to Say When: You Need More Consistent Processes For Optimization

Almost half – 48% – of optimizers wish they had more consistent processes.

Excellent optimization is only possible when you pair excellent team members with a consistent group of processes. Sometimes, getting process right takes time and iteration, and some self-reflection.

When you want to streamline you brainstorming, hypothesis generation, and communication processes, bring these tips to your team:

4. What to Say When: Each Experiment is Measured Against Different Metrics

35% of benchmark survey respondents said they were still in search of an overarching metric for optimization. The key to optimization alignment with the broader business is choosing the right metric to measure. You’ll always be tracking micro-goals, like conversions and engagement, but it’s important to track the downstream effects of your experiments so you can articulate the business impact of your program.

changes to optimization goals

Great optimization programs have nested goals, like this example: your conversion metrics should ladder up to larger company goals.

  • To make optimization stick, focus on a quantifiable goal. Many companies will measure revenue as a straightforward, iron-clad method of measuring and communicating the value of their testing.
  • Finally, for a more in-depth read, learn how to leverage the right metrics to delight your customers, engage your audience, and grow your business with our complete guide: Building your Company’s Data DNA.


5. What to Say When: You Need Executive Support for Testing to Get Started

About one in five optimizers (22%) are looking to establish executive buy-in for optimization. This can be a daunting challenge, but we’re here to help.

Looking to dig further into metrics like these? Join tomorrow’s webinar: Sharing Findings from the Optimization Benchmark.


Optimizely X
comments powered by Disqus