Brilliant Testing Ideas in 6 Simple Steps

It always goes exactly like this:

A/B Testing Workflow

1. You have a brilliant idea. (Even your boss loves it.)
2. You set up the test. (In seconds.)

   

3. You run it. (In a day.)
4. You get statistically relevant results. (From your massive volume of site traffic.)
5. You share the results with your team. (And get a raise.)
6. You implement the winner. (And eat some cake.)
7. You win the nobel prize for web genius. (And eat some more cake.)

Not quite, you say? I’ve seen tons of companies have fantastic success with A/B testing but even I’ll admit it doesn’t always go like the process outlined above.

Today, I’d like to share a few strategies to help you with one of the hardest parts about A/B testing: getting that brilliant idea. 

Like so many things, the hardest part is getting started. And like so many things “Step 1” of the A/B testing workflow is actually multiple steps in and of itself. Since I like numbered lists, I’m going to provide you with a realistic, six-step process for getting that brilliant idea.

Step One: Define The Metrics You Want To Impact

It sounds elementary but in the world of website optimization, it’s forgotten all too often. I’ve actually heard multiple companies say, “We don’t know what we want to improve but we definitely want to make the site better.” At this point, I usually ask why the company has a website (after all, they could probably save a lot of money by taking it down). And immediately the client answers their own question.

A lead gen site will say, “92% of our sales leads come through this form and 96% of our revenue comes from sales made through those leads so we absolutely have to improve the form completion rate.” Then I write down, “Improve Form Completion Rate.”

An e-commerce site will say, “We sell products. People have to see those products and have an easy way to purchase them.” I write, “Increase product page views and improve checkout flow conversion rates.”

(Are you starting to see why I have so many web genius Nobels and why I’m addicted to cake?)

Step Two: Know Your Users’ Pain Points

To understand the user, trying being the user. Block out 30 minutes on your calendar after lunch. Go outside get some fresh air. Eat a sandwich (or some cake…). Forget everything you ever knew about your website. Then…use it.

What are the pain points? Try searching for something – can you find it? What’s the point of creating a username and password? Can you checkout as a guest? Are you bored? Are you overwhelmed? Are you going to shop here again? How about that rotating banner on your homepage? Did you feel compelled to click on it? Or did the image disappear before you even had a chance to?

If something is affecting your own ability to “convert” (however defined), it’s definitely affecting your users as well. If you had a hard time walking in your users’ shoes, I suggest looking at the top user path flows in your analytics engine. See what a common visit to your site looks like and see where the biggest areas of relative drop-off occur.

Step Three: Get Talking

Believe it or not , you probably work with some really smart people. (Or some really opinionated people.) Talk to people from all departments (including from outside of the web division) and ask them what they like or don’t like about the site. Explicitly ask them what they would test too. A few (caked-related) ways you could do this include:

• Instant gratification cake: Attract interested colleagues by setting up shop in a conference room with a laptop and a spreadsheet. Offer co-workers who pass by a piece of cake for a few ideas.

• Delayed gratification cake: Set up a simple Google or SurveyMonkey survey asking co-workers to list their top ideas. Tell them that the following Friday everyone who completes it will get an email notifying them where they can attend the in-office cake party.

Not every idea will be a good one or fully thought out but you’ll still get a variety of perspectives and a solid sense of how co-workers view the site.

Oh and did I mention cake?

Step Four: Get Stalking

Check out what other companies in your industry are doing. How do your competitors tackle similar problems? A great way to do a fast comparison is taking screenshots of competitors’ websites and viewing those screenshots side-by-side. Force yourself to jot down four or five bulleted points about how each differs from each other. Even if your competitors’ sites are “worse” than yours, they’re still doing a few things right – otherwise they wouldn’t be in business.

Step Five: Get Thinking

Look at the pages you want to change and ask as many questions (even some “dumb” ones) as you can.

• What makes our site better than the competition? Do we make that clear?

• Is it obvious that’s a link?

• Does that look like an ad?

• What is our business model?

• How do I get back to the product category? Am I already in the product category or is that a sub-category?

• Does that button color stand out?

• Do I understand why I’m being asked to fill out a form? Am I motivated to fill out this form? What is compelling me to fill this form out in the first place?

• Does “Buy Now” work as well as “Complete Purchase”, “Place Your Order” or “Get Your Deal?

• Do I enjoy reading the content on this page?

Some questions will be geared toward usability while others will be geared toward improvement. However you frame it, if you can ask a good question you can probably get a good test idea.

Step Six: Focus On The Losers

It’s time to get analytical. If you’ve done past tests, review those results. Sure you got a winner (hence all that cake you’ve been eating) but what else did you learn? Sometimes, the tests that lose are actually the ones that provide you with the best learnings. Take a close look at those too.

Look for dips or spikes in conversion rate. Did it dip during certain times of the day, week or month? Did that coincide with any internal or external events (promotions, economic, news etc.). Did it spike or dip more for some variations than it did for others? Why? Did the user-base change? Did it go up for some segments but down for others?

Reviewing past results will give you more questions, and more intelligent questions which you may be able to answer with additional testing.

So, Now What?

These six steps will set you up for success at the beginning of the testing cycle. So, you got that goin’ for you, which is nice.