About Ryan Lillis

Ryan Lillis is a Website Optimization and Digital Marketing strategist. Formerly with Adobe/Omniture, Ryan joined Optimizely in August 2012 because he was blown away by Optimizely's testing platform and believes it represents the future of website optimization.

Brilliant Testing Ideas in 6 Simple Steps

The great thing about website optimization is that it’s easy. Insanely easy. It always goes exactly like this:

A/B Testing Workflow

  1. You have a brilliant idea. (Even your boss loves it.)
  2. You set up the test. (In seconds.)

    thumbs-up
  3. You run it. (In a day.)
  4. You get statistically relevant results. (From your massive volume of site traffic.)
  5. You share the results with your team. (And get a raise.)rain
  6. You implement the winner. (And eat some cake.) 
  7. You win the nobel prize for web genius. (And eat some more cake.)dog1

Not quite, you say? I’ve seen tons of companies have fantastic success with A/B testing but even I’ll admit it doesn’t always go like the process outlined above.

Today, I’d like to share a few strategies to help you with one of the hardest parts about A/B testing: getting that brilliant idea.  Continue reading

Posted in AB Testing ideas, Best Practices | 5 Replies

What Candy Land Can Teach You About Optimizing Mobile

Remember the board game Candy Land? Sweet-toothed contestants traveled down a path of colored squares on their quest to reach the Candy Castle. If you were lucky, you landed on the Gumdrop Pass or Rainbow Trail – shortcuts that helped you bypass your opponents to reach the finish that much faster.  Draw the wrong card, and you’d get lost in the Lollipop Woods or stuck in the Molasses Swamp. An unfortunate turn of events that could prevent you from reaching Candy Castle altogether.

candyland2

Now, think of your mobile site like Candy Land. Just like the Candy Castle, users enter your site looking for something specific. To help decrease bounce rates and take users to the pages they’re looking for faster, here are a few A/B tests to get started optimizing your mobile experience.

Continue reading

Avoid Sidewalk Catastrophes: Optimize Your Mobile Website

Trips, falls, and head-on collisions. These days, a quick stroll down the street is an accident waiting to happen. It’s easy to point fingers at uneven pavement, crazed skateboarders, or fanny-packed tourists, but the truth of the matter is clear: smartphones are the culprit. Young, old, commuter, tourist, sometimes even bikers (yikes!) – everyone is face down, glued to their mobile device. While the act of walking-and-browsing can be aggravating, it speaks to a very important truth: mobile web is pervasive. Improving your mobile website will not only help prevent sidewalk catastrophes, it will also help create an optimized experience that generates more clickthroughs, revenue, and conversions.

Here’s where mobile testing comes in.

Mobile versions of target.com, reddit.com and taskrabbit.com

Mobile versions of target.com, reddit.com and taskrabbit.com.


Continue reading

“Is it done yet?” Getting real about calling a test

As a Web Strategist and A/B testing consultant, I’ve worked with everything from Fortune 100 companies to freshly minted start-ups. Regardless of the length of their customer list or the volume of their website traffic, they all ask me the same question: “How long should I run this A/B test for? How long will it take to get accurate results?” It’s a natural question and one that may be even more fundamental if you’re trying to reach a particular quarterly goal for your testing program.  And while I’m not a statistician, I’m not afraid to say that it’s very rare that you’ll be able to conclusively say how long a test will need to run. Continue reading

What exactly is a ‘conversion’? It’s up to you!

The goal of any A/B test is to compare two (or more) versions of a web page and see which version performs best. What “performs best” actually means is entirely up to you and depending on the test’s goals, it could mean any number of things. But however you define “performs best,” you’ll measure this by a conversion goal, an aggregate goal, or an what I like to call an ultimate conversion goal.

Continue reading