Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!
Get It Now
Why Product Experimentation? Before I joined Optimizely, I worked at Microsoft on what was once the world’s leading internet browser, Internet Explorer. For those of us who have forgotten about this great browser, here’s a screenshot: As an engineer, I was tasked to work with a product manager on developing an algorithm for determining the […]
Creating a faster and more engaging site experience often requires tradeoffs between various revenue streams of the business. Can the increase in conversion from more engaged users more than offset any loss in ad revenue? It’s a bold bet that Realtor.com’s analytics and experimentation team set out to answer at the beginning of 2018. A […]
Welcome to the second installation of Ask an Experimenter, a new series where we interview experts about how they are building a culture of experimentation at their companies. Our interview today is with Joy Scharmen, Optimizely’s Director of DevOps. Tell us a little about yourself. I’m Joy, an engineering director here at Optimizely. Our team […]
Last week, we hosted our second annual Developer Night @ Opticon18, sponsored by Heap. Over 170 people showed up to listen to the lightning talks and watch the live demos. From experimenting on Alexa skills to integrating with analytics platforms, and integrating with other developer tools we covered a lot of ground. Read on for […]
At Optimizely, we’re always looking for ways to eat our own dogfood. Once we added feature flags to Optimizely Full Stack, we started adopting flags to remotely configure our own application. In the last few months, we’ve found this has been especially helpful for our end-to-end testing. Being able to toggle features on and off […]
This is the final post in our Product Experimentation Pitfalls blog series, written by Optimizely’s Director of Product Management, Jon Noronha. See here for more information on this 5-part series. So far in this series, I’ve highlighted several pitfalls on the path to product experimentation. We’ve seen how some common mistakes can slow down […]
This is the fourth post in our Product Experimentation Pitfalls blog series written by Optimizely’s Director of Product Management, Jon Noronha. See here for more information on this 5-part series. So far in this series, we’ve talked about the hidden challenges that can stop experimentation from getting off the ground, like getting tricked by statistics, or choosing […]
This is the third post in our Product Experimentation Pitfalls blog series written by Optimizely’s Director of Product Management, Jon Noronha. See here for more information on this 5-part series. Analyzing an A/B test is deceptively simple. You count up how many conversions happened in each variation, you divide by the number of users exposed […]
This is the second post in our Product Experimentation Pitfalls blog series written by Optimizely’s Director of Product Management, Jon Noronha. See here for more information on this 5-part series. An experiment is like a metal detector. When there’s valuable treasure close to the surface, it’s an incredible tool that can lead you straight to the gold. […]
This is the first post in our Product Experimentation Pitfalls blog series written by Optimizely’s Director of Product Management, Jon Noronha. See here for more information on this 5-part series. Highlights: Measuring experiment results is relatively straightforward when you are driving toward a simple goal such as conversion. When product teams optimize for long-term or […]
