Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more! Download Now

Optimizely Blog

Tips & Tricks for Building Your Experimentation Program

X

Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!

Get It Now

Developers

Articles by developers for developers about how to best make use of the Optimizely platform.

Product Experimentation Pitfalls, Post #5: Failing to Scale Across Teams

Product Experimentation Pitfalls, Post #5: Failing to Scale Across Teams

This is the final post in our Product Experimentation Pitfalls blog series, written by Optimizely’s Director of Product Management, Jon Noronha. See here for more information on this 5-part series.    So far in this series, I’ve highlighted several pitfalls on the path to product experimentation. We’ve seen how some common mistakes can slow down […]

Product Experimentation Pitfalls, Post #4: Measuring and Driving Experiment Velocity

Product Experimentation Pitfalls, Post #4: Measuring and Driving Experiment Velocity

This is the fourth post in our Product Experimentation Pitfalls blog series written by Optimizely’s Director of Product Management, Jon Noronha. See here for more information on this 5-part series.  So far in this series, we’ve talked about the hidden challenges that can stop experimentation from getting off the ground, like getting tricked by statistics, or choosing […]

Product Experimentation Pitfalls, Post #3: Getting Tricked by Statistics

Product Experimentation Pitfalls, Post #3: Getting Tricked by Statistics

This is the third post in our Product Experimentation Pitfalls blog series written by Optimizely’s Director of Product Management, Jon Noronha. See here for more information on this 5-part series.  Analyzing an A/B test is deceptively simple. You count up how many conversions happened in each variation, you divide by the number of users exposed […]

Product Experimentation Pitfalls, Post #2: Experimenting Without Enough Traffic

Product Experimentation Pitfalls, Post #2: Experimenting Without Enough Traffic

This is the second post in our Product Experimentation Pitfalls blog series written by Optimizely’s Director of Product Management, Jon Noronha. See here for more information on this 5-part series.  An experiment is like a metal detector. When there’s valuable treasure close to the surface, it’s an incredible tool that can lead you straight to the gold. […]

Product Experimentation Pitfalls Blog Series, Post #1: The Perils of Experimenting With the Wrong Metrics

Product Experimentation Pitfalls Blog Series, Post #1: The Perils of Experimenting With the Wrong Metrics

This is the first post in our Product Experimentation Pitfalls blog series written by Optimizely’s Director of Product Management, Jon Noronha. See here for more information on this 5-part series.  Highlights: Measuring experiment results is relatively straightforward when you are driving toward a simple goal such as conversion. When product teams optimize for long-term or […]

How Vivid Seats Ensures Experimentation Success

How Vivid Seats Ensures Experimentation Success

Last week, we had an opportunity to host a webinar featuring Bill Hinderman, the Experimentation Engineering Lead at Vivid Seats. During the hour long webinar, Bill covered everything from his own experience with experimentation to the Optimizely Full Stack implementation at Vivid Seats, and he even dropped in some fun analogies along the way. There […]

New Blog Series: Product Experimentation Pitfalls and How to Avoid Them

New Blog Series: Product Experimentation Pitfalls and How to Avoid Them

We’re excited to announce our very first blog series, written by Jon Noronha, the Director of Product Management at Optimizely. Hot on the heels of the release of Feature Management, we’re launching a 5-part blog series, using stories, research, and data to help you avoid common product development experimentation pitfalls. Whether your team has experiments […]

Experimentation Helps Product Teams Build Innovative Software

Experimentation Helps Product Teams Build Innovative Software

Market competition is intensifying, and customer expectations continue to rise. Staying in business requires addressing them both. Fortunately, by adding experimentation to your development process, and discovering how it complements initiatives like Agile, your team can learn to innovate rapidly and deliver a better customer experience. The Role of Experimentation The days of completing development […]

To Move Faster, Experiment More


If you’ve ever had to forecast a deadline for a software project, you’ll be all too familiar with Hofstadter’s Law: It always takes longer than you expect, even when you take into account Hofstadter’s Law. So it’s no surprise that every product development team wants to be more “agile.” To the naive CTO, “agile” is just […]