“Is it done yet?” Getting real about calling a test

Updated May 8, 2014

Optimizely is a product that was created so that anyone—regardless of technical ability—could run A/B tests on their website. We know that getting up and running with a testing tool is only half the battle. In order to truly excel with website optimization, you also need to understand how to interpret the results from your tests. Just like our initial intention of making the creation and execution of tests easy, we want to arm you with an understanding of the math behind your tests, and confidence of how to call a winner (regardless of your comfort level with statistics.)

Part of this process is determining the ideal sample size for your test. If you know how many web visitors need participate in an experiment in order to reach a statistically significant outcome, you can then project how long a test will need to run based on your knowledge of your daily website traffic. We’ve broken it all down for you in this article. Let us know what you think!