Name: Jason Burt
Role: Web Marketing Analyst
Fun fact about sales tax: Even if I came up with one, the law changes so much that it would probably be different tomorrow.
Jason says, “testing has dramatically changed our company culture as far as decision making goes. The conversation used to be, ‘this is the way the site is going to be.’ Now the conversation goes, ‘here are some ideas to test.'”
Optimizely: What’s your testing process like?
Jason: Since we started testing in December, I feel like our process has been about the same. My background is in physics and economics so I’m really used to the scientific method and working with statistical sample sizes. We try to test everything as much as possible. We’ll run tests on pages that receive the most traffic. If we find any UI improvements, we’ll implement them across all the pages that don’t get as much traffic. Likewise, if we see certain photos or elements that have high conversions on our site, we’ll test them again in an email. Testing has dramatically changed our company culture as far as decision making goes. The conversation used to be, “this is the way the site is going to be.” Now the conversation goes, “here are some ideas to test.”
How often do you run experiments?
Every few weeks we start a new test, we aim for two weeks minimum. We try to test everything and it starts when we come together and have idea for something we want to test. I’m a purist, I only do 50/50, A or B, not multivariate. Testing is a continual process. There are a lot of cool things going on in behavioral economics and UI right now. So it’s a great time to be getting into testing.
What were some of the initial tests you ran on avalara.com?
Our site is orange, we love orange, but at the same time we want to balance it properly with other colors to create contrast to make the user experience better. Working within our color palette, we ran A/B tests with a number of color combinations on call to action buttons. We tested reactions to red, green, orange, and blue. We ended up going with a blue button because it significantly improved conversions by 20%. At year end, just that button being blue is going to probably be a couple thousand more dollars in revenue.
Which tools have been most effective for your A/B testing?
The cross browser feature makes our lives much easier. I can see how our pages load across all different browsers and immediately make changes if there’s something loading incorrectly.
Most companies still browse with IE. One week, something on our site was loading improperly in certain versions of IE and it played into our conversion rates. Though it might appear to be minor, by year end that could be a 2-5% drop in business – that’s very significant. That’s one of the reasons why Optimizely is awesome. We are constantly testing our loads and can quickly see if something is testing correctly; you can do it all in one spot.