Why We Switched to Optimizely

The biggest difference between conversion optimization and all other sub-sectors of online marketing is the technology required to do it well. Website optimization requires qualitative analysis, via tools such as SurveyMonkey and Qualaroo, as well as quantitative analysis via tools like Google Analytics, Clicktale, and Mouseflow. Most important of all is A/B testing software. The better your testing software, the better your testing will go.

At Landers Optimized, we’ve tried just about every testing software out there. Google Website Optimizer (GWO), Visual Website Optimizer (VWO), Google Content Experiments, Google Optimize, Test&Target…

You name it. We’ve probably tried it.

After testing the gamut of split testing tools, we now use Optimizely 100% of the time because it’s hands down the most advanced platform out there. Here are the top 5 reasons we made the switch (and you should too).

1. Regular Expressions in URL Targeting

We tend to run many multi-page tests (especially on e-commerce sites), and this was a very frustrating thing to accomplish back when we were using Visual Website Optimizer (VWO). The following example illustrates why.

Let’s assume you wanted to set up a test that ran on two separate pages: website.com/page1 and website.com/page2.

With a tool like VWO, you can’t run this test. VWO gives you a wildcard which would theoretically work by putting website.com/page* into the URL pattern, but would also make the test run on any page that matched that pattern, such as website.com/page3.

By allowing you to use real regular expressions, Optimizely makes this a very easy test to setup.

There are countless other cases where using regular expressions is simply mandatory, such as:

• Targeting URLS with optional characters: For example, https://website.com BUT the “s” in “https” is optional.

• Targeting a range of digits: In this case, website.com/page[2-5] would match website.com/page2, website.com/page3, all the way through website.com/page5, but NOT website.com/page6.

• Making query parameters optional: For example, website.com/page\/?(\?.*)? would match website.com/page, website.com/page/, AND website.com/page?some_param=value. Although Optimizely’s simple matching does this by default, this is necessary if you want to use regular expressions somewhere else in the URL as well.

2. GET Parameters for QA and Testing Purposes

An extremely useful feature Optimizely provides is the ability to force yourself into a certain variation, force tracking or a debug log, or disable Optimizely by adding a simple GET parameter. For now, I’ll simply illustrate how we take advantage of them.

The debug is extremely helpful in making sure your testing goals work properly. Simply append ‘optimizely_log=true’ to any URL and you will see a complete log of exactly what Optimizely is doing, how you are being bucketed, and much more. Never again will you launch a test and lose out on precious data due to a flaw in the goal setup.

Just as useful as logging is the ability to force yourself into a variation. A few months back, my team was running two tests on one site – one that tested a new navigation across an entire site and another that tested a free shipping icon in the site’s header. We ran into an obvious obstacle. Since the navigation and the the shipping icon were both in the header, the two tests conflicted with each other.

With Optimizely, we solved this problem easily. We wrote a conditional in our jQuery code to check which variations were displayed, and adjusted the size of the shipping icon based on which version of the navigation was shown. We visited the following URLs in our browsers to write and test our code:

• Navigation A + Shipping Icon: store.com/page?optimizely_x[Test1ID]=0&optimizely_x[Test2ID]=1

• Navigation B + Shipping Icon: store.com/page?optimizely_x[Test1ID]=1&optimizely_x[Test2ID]=1

See how this is useful? Because Optimizely allows you to actually append more than one parameter for forcing variations, you can make sure that whenever you have multiple tests running on a page, the experiments don’t conflict with each other. No other software allows this.

3. The Optimizely Data Object

For the previous experiment example to work, each variation needed to adjust based on which variation of the other experiment was showing. How was this possible? Enter: The Optimizely Data Object.

The Data Object provides lots of useful information; we use the variation map feature most. With the variation map, your code can be aware of not only what other experiments are running, but also what variations of that experiment are being shown. Without this, the previous test example would not have been possible.

4. Event-Based Activation

This is by far my favorite feature of Optimizely. It allows you to ignore URL-based targeting altogether and activate experiments manually via a Javascript event.

Why is this so useful?

Let me explain.

A large amount of today’s websites are built on CMS platforms such as WordPress. If you’re familiar with the template structure of WordPress at all, then you probably know that the different layouts (category pages, homepages, single posts, etc.) are just PHP files in a template folder. Now, assume that you wanted to easily run an experiment across all of your product pages, and your template on your server is at /path/to/theme/product.php.

Moral of the story: If you’ve got a specific template file that applies to all the pages you want to test, this is (BY FAR) the easiest to target your test.  This is a feature I’ve yet to see on any other website optimization platform.

5. Under the WYSIWYG Editor’s Hood

Of course, for many experiments, you will only be testing a single page and won’t need any fancy targeting strategies. In these situations, I turn to  Optimizely’s WYSIWYG editor – it’s the easiest choice for building an A/B test. Change your headline size, edit your CTA text, move your navigation menu across the page – the editor will instantly generate the corresponding Javascript code to make your changes a reality. A big advantage of Optimizely’s WYSIWYG editor is that every time you make a change, the corresponding code is displayed (and made editable) in the ‘Edit Code’ section. You can always modify the code to fix any issues you may encounter along the way. This is not possible with VWO’s editor.

And what about those times you want to change something you did say, 10 changes ago, but not everything you did since? With Optimizely, you can identify which line of code made that change because when you hover your mouse over a line of code spit out by the editor, it literally draws a box around the element it’s affecting. How useful!

These are just some of the reasons we are now 100% Optimizely.