Optimizely Blog

Grow your Optimization and A/B testing skills


Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!

Get It Now

Hi I’m Hudson Arnold, a Strategy Consultant at Optimizely. I want to share some thoughts on how to prioritize test ideas for this week’s One Minute Monday.

My guess is that if you’re watching One Minute Monday you’ve probably spent a ton of time looking around the web for great test ideas — so many ideas, that you don’t even know what to run next. That’s where prioritization comes in. How do we do that? It’s pretty simple. We want to estimate what the ROI of each test will be. So you estimate:

1. how hard it will be to build


2. how much value you think you will get out of that test.

To give this idea a little more clarity, let’s pretend we’re at the buffet line looking at a long table of foods we might want to eat.

breakfast buffet

Can you smell the bacon?

So we’re at the buffet line, we have three options in front of us: oatmeal, waffles, and bacon. To decide what to eat, we’re going to assess just how delicious each of these foods are and how many calories we’ll take in from each one.

prioritizing breakfast

Based on this reasoning… definitely choosing bacon.

OK, now let’s move on to an example with real tests to help prioritization make more sense. Let’s say we’re looking at three tests. You and your team would assign points to each test around different criteria like the three listed below in this example:

Test Estimated Value Estimated Difficulty Value/Effort
Homepage Headline 2 1.5 1.33
Product Page CTA 3 2 1.66
Streamlined Checkout 5 2.5 2

You can see the estimated value for homepage headline is 2 and the estimated value for a streamlined checkout is 5. According to this scale, the checkout test is also the hardest tests to set up, but because it’s so much more valuable, we’re going to choose to invest time to run it first and except that it will take more time to build.

Your criteria might change over time as you get better at assessing value and develop more technical proficiency. (Over time, oatmeal might start making more and more sense).

Use this framework to ensure you’re running the right tests at the right time! That’s the idea of prioritization: establishing how hard a test is to run vs. how much value you’ll get out of it.

Hopefully, all of our tests will be bacon.

More article on prioritizing tests:

A Method for Prioritizing A/B Test Ideas That Won’t Hurt Feelings

What’s Next? Steps For Prioritizing Your Experiment Backlog

More One Minute Mondays:

ehsan-1mm-mug Ryan Krebs One Minute Monday Optimiely Becca one minute monday mobile
Targeting an Optimizely Experiment to Specific URLs Brainstorm Your Next A/B Test  Preventing Buggy iOS Updates
Optimizely X
comments powered by Disqus