As a Web Strategist and A/B testing consultant, I’ve worked with everything from Fortune 100 companies to freshly minted start-ups. Regardless of the length of their customer list or the volume of their website traffic, they all ask me the same question: “How long should I run this A/B test for? How long will it take to get accurate results?” It’s a natural question and one that may be even more fundamental if you’re trying to reach a particular quarterly goal for your testing program. And while I’m not a statistician, I’m not afraid to say that it’s very rare that you’ll be able to conclusively say how long a test will need to run. Continue reading
The goal of any A/B test is to compare two (or more) versions of a web page and see which version performs best. What “performs best” actually means is entirely up to you and depending on the test’s goals, it could mean any number of things. But however you define “performs best,” you’ll measure this by a conversion goal, an aggregate goal, or an what I like to call an ultimate conversion goal.