Best Practices and Templates for Demonstrating Experimentation ROI

I’m Becca Bruggman, Optimizely’s Experimentation Program Manager. My job is to make sure we are “drinking our own champagne” and run a best-in-class experimentation program.

This is the final installment of my series designed to help you run a best-in-class experimentation program. In the series we covered everything you need to build and scale your program as well as make it visible all with downloadable templates to simplify all of these tasks for you going forward.

All good things must come to an end! In this final post, we will discuss how to show the overall ROI of your program and its impact as well as to get executive visibility.

A summarized, executive-level digestible recap can be really impactful for your program. Developing and distributing it provides the opportunity to demonstrate the overall ROI of your experimentation program. With this documentation you can show the aggregate value of the program, tying it back to top-line business goals and demonstrating the additional value of the program. In addition to the experiments themselves, you can include things like training, onboarding or ideation sessions which show the broader value of the program to the entire company and, importantly, executives.

If you’ve been leveraging the Experiment Start and Experiment Results emails, which we covered in my previous blog post, creating this executive level recap can be fairly straightforward as you can re-use the information from the individual experiment emails within the recap as well.

I recommend sending these at a consistent cadence. At Optimizely, we do quarterly goal setting of Objectives & Key Results (OKRs), so I send an update of the program against our OKR goals to align with that cadence. As I mentioned in the last post, piggybacking on processes that already exist versus creating new ones is always recommended. I also previously wrote extensively about the value of sharing your program.

Before getting your recap email together, it is helpful to think through the ROI of your program as it aligns to top level metrics for your company. This usually entails assigning a value to specific user actions and then identifying how much experimentation has influenced these behaviors. If you have your goal tree created from a few posts back, you can assign values to those events to help with this ROI calculation. For some companies, it might make sense to have something custom that aligns to current metrics monitoring that are already in place. If you’re getting started or looking for some additional guidance on how to approach ROI for experimentation, Optimizely has a great ROI calculator.

Ok, so you have your goal tree values and ROI calculations all sorted, now let’s bring everything together!

Here is an example of a quarterly recap I sent, which you can find a template for in my Experimentation Program Toolkit.

I include a summary with the total number of experiments we’ve run, any scalable work such as blog posts or trainings that have happened, a plug for experiment review and thank the rest of my team.

Summary of Optimizely’s Experimentation Program in Q4FY19

Below that I include a table that has the top-level information for all experiments launched. Finally, I include all the details from the experiment launch emails below so it’s all in one summarized place.

I know that making a program visible can be intimidating for some people, however, if you don’t make the work visible, it’s hard for anyone to know it’s happening and it will be difficult to ask for support and resources if you do need help. While getting an email back from an executive with questions about the program can be anxiety-inducing, it’s important to remember that you are not the project and the fact that they care is great! It opens up the door to have a conversation about what’s needed to increase the impact of the program even further.

Sometimes, you even get a congratulatory email from your CEO – which can make it all worth it!

You can find all templates noted above [here].

Hope you’ve enjoyed the series! What should I write about next? Tweet me @bexcitement.