Download our FREE ebook of 43 A/B testing case studies from the world's leading companies for test ideas and inspiration. Download Now

Optimizely Blog

Tips & Tricks for Building Your Experimentation Program

X

Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!

Get It Now



In our first glimpse at Professor Stefan Thomke’s new book “Experimentation Works“, we focused on his insights into what makes a good business experiment. We are now going to summarize the next section that caught our eye, wherein Professor Thomke outlines key factors that drive a culture of experimentation. And as an additional bonus we are excited to share that Optimizely will be hosting a virtual meetup with Professor Thomke on April 22nd and you can apply to attend here.

In the first few chapters of “Experimentation Works”, Professor Thomke highlights the quantifiable potential of experimentation. He then proceeds to the logical question: Why aren’t more companies experimenting–and innovating–more frequently?

Two answers spring to mind. One: the move toward increased efficiency has resulted in company cultures that strive for predictability. And two: senior management with strong incentives to focus on the near term and sticking to plans, rather than focusing on long-term gains. Yet the true obstacle to successful innovation, Thomke posits, is the need to build a culture over time that supports large scale experimentation – even when budgets are tight.

Professor Thomke explains how computer simulations, A/B testing tools, and other approaches allow teams to ask an endless stream of “what if” questions. And as they do, running more experiments is no longer a matter of technology, but of developing an organization-wide culture that buys into the whole idea.

Below Professor Thomke identifies and assesses the seven attributes he sees as  absolutely fundamental to building a culture of experimentation.

1. Be open to learning. Surprises should be savored. Not winning is not the same as losing and failures are not the same as mistakes.

2. Stay consistent. Managers need to avoid mixed messages, for example by encouraging teams to experiment while maintaining a reward system that punishes failure.

3. Don’t let pride get in the way of progress. Accept results from experiments even if they go against your interests, beliefs, and norms.

4. Keep your integrity. An experiment’s ethics need to be built into your training, guidelines and discussions.

5. Trust your tools. You need to place your trust in the tools you use, or how will they ever be fully adopted and integrated?

6. Find the right balance. The most successful organizations maintain a healthy balance between failure and success, experimentation and standardization, and short-term and long-term thinking.

7. Embrace a new leadership model. The greater the distance between upper and lower levels of an organization, the less innovative companies generally become.

So that’s it in a nutshell. Needless to say, this eminently readable and highly insightful book offers far more detail, with fascinating real-world examples bringing these attributes to life – most notably from Facebook, 3M and Intuit to name just a few.

The takeaway is simple: building an effective culture of experimentation takes time, but the rewards are worth it. Professor Thomke’s book provides a great navigation tool that will help you to make sure you are going in the right direction.

Experimentation Works by Professor Stefan Thomke is now available.

To learn more about experimentation best practices pick along with the book you can view our exclusive Thomke Talks video series with the professor.

 

 

Optimizely X