Hi I’m Hudson Arnold, a Strategy Consultant at Optimizely. I want to share some thoughts on how to prioritize test ideas for this week’s One Minute Monday.
My guess is that if you’re watching One Minute Monday you’ve probably spent a ton of time looking around the web for great test ideas — so many ideas, that you don’t even know what to run next. That’s where prioritization comes in. How do we do that? It’s pretty simple. We want to estimate what the ROI of each test will be. So you estimate:
1. how hard it will be to build
2. how much value you think you will get out of that test.
To give this idea a little more clarity, let’s pretend we’re at the buffet line looking at a long table of foods we might want to eat.
So we’re at the buffet line, we have three options in front of us: oatmeal, waffles, and bacon. To decide what to eat, we’re going to assess just how delicious each of these foods are and how many calories we’ll take in from each one.
Based on this reasoning… definitely choosing bacon.
OK, now let’s move on to an example with real tests to help prioritization make more sense. Let’s say we’re looking at three tests. You and your team would assign points to each test around different criteria like the three listed below in this example:
|Test||Estimated Value||Estimated Difficulty||Value/Effort|
|Product Page CTA||3||2||1.66|
You can see the estimated value for homepage headline is 2 and the estimated value for a streamlined checkout is 5. According to this scale, the checkout test is also the hardest tests to set up, but because it’s so much more valuable, we’re going to choose to invest time to run it first and except that it will take more time to build.
Your criteria might change over time as you get better at assessing value and develop more technical proficiency. (Over time, oatmeal might start making more and more sense).
Use this framework to ensure you’re running the right tests at the right time! That’s the idea of prioritization: establishing how hard a test is to run vs. how much value you’ll get out of it.
Hopefully, all of our tests will be bacon.
More article on prioritizing tests:
More One Minute Mondays:
|Targeting an Optimizely Experiment to Specific URLs||Brainstorm Your Next A/B Test||Preventing Buggy iOS Updates|