Download our FREE ebook of 43 A/B testing case studies from the world's leading companies for test ideas and inspiration. Download Now

Optimizely Blog

Tips & Tricks for Building Your Experimentation Program

X

Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!

Get It Now

product experimentation

Ask an Experimenter: Olivia Simon of Dow Jones

Ask an Experimenter: Olivia Simon of Dow Jones

Olivia Simon will be speaking at Test and Learn: The Product Experimentation Virtual Summit, a free half-day virtual event for product managers, data scientists, engineers, and designers to dive deep into the intersection of insights-driven product development and agile software delivery. Save your seat to listen to Olivia and other product and engineering leaders on how to create […]

Nir Eyal on Habits, Experimentation, and Becoming Indistractable

Nir Eyal on Habits, Experimentation, and Becoming Indistractable

Nir Eyal is the author of Hooked: How to Build Habit-Forming Products, a bestselling guide to using behavioral psychology to inform product design and development, and the forthcoming Indistractable: How to Control Your Attention and Choose Your Life. Nir is the keynote speaker at our upcoming virtual Test & Learn conference, which will be broadcast […]

An Interview with Emily Nakashima, Honeycomb, on Building a Test and Learn Culture within Engineering Organizations

An Interview with Emily Nakashima, Honeycomb, on Building a Test and Learn Culture within Engineering Organizations

Emily will be speaking at Test and Learn: The Product Experimentation Virtual Summit, a free half-day virtual event for product managers, data scientists, engineers, and designers to dive deep into the intersection of insights-driven product development and agile software delivery. Save your seat to listen to Emily and other product and engineering leaders on how […]

Ask an Experimenter: Kevin Chen, Growth PM at Reddit

Welcome to the third installation of Ask an Experimenter, a new series where we interview experts about how they are building a culture of experimentation at their companies. This interview features Kevin Chen, Growth Product Manager at Reddit. Kevin works on the strategy, roadmap, and execution of push notifications and email products for Reddit. Tell us a […]

How to Avoid Building the Wrong Product? Make Your Product Roadmap Your Experimentation Roadmap

We often hear that product development teams don’t have an experiment backlog or that they struggle to come up with experiment ideas. Almost certainly, though, you have a product roadmap. This list of problem-solution pairs includes the features that will certainly help you defeat the competition. Well, maybe. No matter how much data analysis, user […]

Product Experimentation Pitfalls, Post #5: Failing to Scale Across Teams

Product Experimentation Pitfalls, Post #5: Failing to Scale Across Teams

This is the final post in our Product Experimentation Pitfalls blog series, written by Optimizely’s Director of Product Management, Jon Noronha. See here for more information on this 5-part series.    So far in this series, I’ve highlighted several pitfalls on the path to product experimentation. We’ve seen how some common mistakes can slow down […]

Product Experimentation Pitfalls, Post #4: Measuring and Driving Experiment Velocity

Product Experimentation Pitfalls, Post #4: Measuring and Driving Experiment Velocity

This is the fourth post in our Product Experimentation Pitfalls blog series written by Optimizely’s Director of Product Management, Jon Noronha. See here for more information on this 5-part series.  So far in this series, we’ve talked about the hidden challenges that can stop experimentation from getting off the ground, like getting tricked by statistics, or choosing […]

Product Experimentation Pitfalls, Post #3: Getting Tricked by Statistics

Product Experimentation Pitfalls, Post #3: Getting Tricked by Statistics

This is the third post in our Product Experimentation Pitfalls blog series written by Optimizely’s Director of Product Management, Jon Noronha. See here for more information on this 5-part series.  Analyzing an A/B test is deceptively simple. You count up how many conversions happened in each variation, you divide by the number of users exposed […]

Product Experimentation Pitfalls, Post #2: Experimenting Without Enough Traffic

Product Experimentation Pitfalls, Post #2: Experimenting Without Enough Traffic

This is the second post in our Product Experimentation Pitfalls blog series written by Optimizely’s Director of Product Management, Jon Noronha. See here for more information on this 5-part series.  An experiment is like a metal detector. When there’s valuable treasure close to the surface, it’s an incredible tool that can lead you straight to the gold. […]