Optimizely Blog

Grow your Optimization and A/B testing skills

X

Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!

Get It Now

Multivariate Testing

Multivariate testing is a testing method whereby all possible combinations of a change are tested simultaneously. This method of testing can lead to the discovery of a winning variation sooner and uncover interactions between different site changes. The downside of MVT is that it requires much more traffic than simple A/B testing, which may make it impractical for websites that do not receive as much traffic.

How an Online-First Model for Retailers’ New Initiatives Drives Down Costs and Reduces Risk

How an Online-First Model for Retailers’ New Initiatives Drives Down Costs and Reduces Risk

In an increasingly challenging and fast-moving retail environment, organizations are looking for every opportunity to reduce cost and risk while improving the end-to-end customer experience.

Given the disruption created by online pure players, how can traditional retailers maximize the value of the multi-channel model of customer engagement and also gain new levels of agility and innovation, from introducing new products through to marketing?

For Killer Response Rates in Large Campaigns, Try Experimental Design

For Killer Response Rates in Large Campaigns, Try Experimental Design

Your customer base is dynamic and complex. If you’re trying to optimize across your entire population of users, traditional A/B experiments may fall short. In this case, you should consider a technique we call experimental design because it allows for massive increase in the amount of variance in your digital marketing campaigns.

In this post, we’ll discuss how to make large-scale experimental design campaigns actionable.

How to Use Data to Choose Your Next A/B Test

How to Use Data to Choose Your Next A/B Test

Your optimization strategy is an exercise in rigorous prioritization. When your most valuable resource is time, it is essential that your experiments run on high-priority pages and generate a measurable effect. Learn how to estimate test impact, leverage your website analytics, and develop a robust testing strategy with data that will let you you make the most of your limited time.

6 Best Practices from 251,391 Website Experiments (and How to Apply Them)

I’ve had the good fortune to attend several optimization talks & conferences with our CEO, Dan Siroker. One thing I admire about good speakers is their ability to distil complex ideas into simple truths.

Simple is good.

In this forum, we try to keep things short & sweet because we know you are frantically busy. After all, this is a blog—not API documentation.

With that in mind, here are Dan’s 6 “Best Practices” from 251,391 A/B and Multivariate tests

How Obama Raised $60 Million by Running a Simple Experiment

On October 21, 2010 President Obama visited Palo Alto to raise money the good ol’ fashioned way: a $30,400-per-plate fundraising dinner. This tried and true fundraising technique is a great way to raise money if you can get the President of the United States to show up to dinner. But how can you raise money if no one has heard of you and all you have is a website? Back in 2007, when Obama was running for the nomination and trailing by double digits in the polls, that’s pretty much all we had.

As Director of Analytics for the Obama 2008 campaign, my job was to use data to help the campaign make better decisions. We started with just one simple experiment back in December of 2007.