Showing posts in “Using Optimizely

Statistics for the Internet Age: The Story Behind Optimizely’s New Stats Engine

Optimizely Stats Engine

Classical statistical techniques, like the t-test, are the bedrock of the optimization industry, helping companies make data-driven decisions. As online experimentation has exploded, it’s now clear that these traditional statistical methods are not the right fit for digital data: Applying classical statistics to A/B testing can lead to error rates that are much higher than most experimenters expect. We’ve concluded that it’s time statistics, not customers, change.

Working with a team of Stanford statisticians, we developed Stats Engine, a new statistical framework for A/B testing. We’re excited to announce that starting January 21st, 2015, it powers results for all Optimizely customers.

This blog post is a long one, because we want to be fully transparent about why we’re making these changes, what the changes actually are, and what this means for A/B testing at large.

Introducing a Faster, More Powerful Optimizely Results Page


Today we’re excited to announce that Optimizely’s new results page is available to all customers.

The results page is Optimizely’s #1 most viewed page, so rebuilding it was no small task. We ran multiple rounds of user research and conducted an extended opt-in beta with more than 6,000 users.

In this post, we walk through the features of the brand new experiment results page.