8 Things You Didn’t Know You Could Do With Optimizely

Optimizely gives us a strategic advantage when building and validating solutions for our clients. To date, we’ve used the platform to run hundreds and hundreds of tests. During this time, we’ve discovered that we can dramatically scale our strategic advantage by becoming more nuanced in how we craft and deploy Optimizely experiments.

Specifically, my Rocket Code colleagues and I have discovered eight core insights and workarounds that empower us to deliver some truly awesome e-commerce firepower. These insights and workarounds have mostly arisen from engineering difficulties we’ve run into while running a variety of experiments. Because Optimizely is so feature rich, some of these tips may not be obvious, which is why I’m eager to share them with you.

Collectively, these tips will help your Optimizely experiments run faster with fewer errors and more visibility into how users are interacting with the test variants.

First up, mastering the art of timing…

1. Fire Optimizely When You Need/Want To

Optimizely uses an intelligent algorithm to know when to kick off your experiment code every time the page loads. Sometimes, however, this algorithm can cause problems if you have a specific order of operations that needs to execute on the page before your experiment starts.

In an ideal world, you could just tell Optimizely when to start the experiment code. Luckily, we live in that ideal world today. Here’s how:

Optimizely gives you the ability to force it to run JavaScript before the rest of the page and experiment runs. Such flexibility grants you a great deal of control to set up any variables, functions, and/or JavaScript modules before the rest of your experiment and page functions fire.

For example, if you want to test a particular feature on a page that requires a tweak to a global variable or plugin configuration like changing a variable flag for a feature or adding a modal configuration, using this feature allows you to reliably know that you will not run into a race condition where your Optimizely code is firing before any essential setup has happened on the page.

Here’s the code snippet you need to make this work:

/* _optimizely_evaluate=force */

var moduleConfig = moduleConfig || {};

moduleConfig.showAltText = true;

moduleConfig.altText = “Show me in a modal headline”;

window.moduleConfig = moduleConfig;

/* _optimizely_evaluate=safe */

What’s going on here? Well, the first line, `/* _optimizely_evaluate=force */`, tells Optimizely to read, then run, this block of code first. After that, the last line, `/* _optimizely_evaluate=safe */`, lets Optimizely know that this block of code is safe to run “as is” before any other scripts you have in your variation code or elsewhere on the page

As a bonus, this method works with JavaScript written in your Experiment JavaScript and Variation JavaScript (a.k.a. Javascript from the “< edit code >” in the editor) . The thing to remember, though, is that the Experiment JavaScript will always run on every variant and will run before any Variation JavaScript that you may force; keep this in mind so that you are not accidentally setting or overriding variables in the wrong place!

2. Implement Experiment “Inception”

One of the most interesting ways to start an experiment with Optimizely is via another experiment. That may seem counterintuitive but it is possible and can come in handy in several circumstances.

Not having direct access to a website’s codebase is one such circumstance. In this scenario, you may need to constrain an experiment to a subset of conditions such as precise click events but cannot manipulate what those events trigger. Triggering an experiment immediately after a user updates the quantity of an item in their cart is a great use case for this capability if you can’t rely on adding the trigger to the original JavaScript event.

Thankfully, you can gain indirect access to those events by using Optimizely’s Manual Activation Mode (MAM).

MAM enables you to “bootstrap” your primary experiment by passing the activation command to the Optimizely experiment you need through another experiment that is set to 100 percent of the website’s traffic.

Here’s an example code snippet to pull off this experiment inception trick:

// This snippet goes into the Experiment JavaScript or Variation JavaScript for the experiment that is that is set to 100% traffic and doing the “boostrapping”.

$(‘a.my-button-trigger’).on(‘click’, function(){

// Make sure any other JavaScript you have is scoped correctly to avoid any errors

window.optimizely = window.optimizely || [];

window.optimizely.push(["activate", YOUR_EXPERIMENT_ID]);

// Now the main experiment will start!

}

If you do in fact have access to your site’s codebase, the two lines that begin with `window.optimizely` can be put in the `<head>` of your site or from within any scripts that you may wish to fire your experiments manually.

Additionally, if you’re using the Optimizely Enterprise Plan, then you have access to the Conditional Activation Mode (CAM). CAM allows you to execute the type of inception I described above using MAM within your experiment rather than relying on any triggers from another experiment or script on the website.

3. Test Entire Page Redesigns

Testing small UI/UX treatments and interactions to drive big lifts is a staple optimization strategy. However, sometimes going small isn’t good enough to achieve truly massive economic goals. In those moments, don’t be afraid to go bigger with your experiments.

At Rocket Code, we have built and deployed wholly new product detail pages and other key page types via Optimizely experiments. Such tests enable you to observe and evaluate larger consumer behavior patterns and commerce workflows

Some server-side tools exist already that can enable you to run such large scale tests. That said, these tools are typically difficult to configure, can cause problems at scale, and can risk site-wide uptime if the tool goes down.

Thankfully, Optimizely provides a tool that can achieve the same outcome without the massive amounts of overhead and uptime risk. Optimizely’s tool provides such ease and stability in large part because it doesn’t need to load both page variant’s content for each user.

Using what is called a Redirect Test, you can set up how to match and redirect users to URLs that will show your users a different page design. There are three things to look out for when doing a redirect test:

1. Like most Optimizely tests, you need to have your Optimizely snippet on all pages involved, not just the original page.
2. If you are using anything beyond exact URL matching, you need to make sure the new page does not fall within that match as it would cause an infinite loop that redirects users to the same page over and over again. Luckily, Optimizely is smart enough to catch and prevent a possible infinite redirect loop, but it could still cause the user to not be directed to your intended page.[i][j][k]
3. You still need to have some mechanism on your website to differentiate the pages you are testing. Depending on how your website is setup, it can be something as simple as keeping the same URL but appending a query parameter such as `?layout=2` that your server or client-side code can interpret to show a different page template or layout.

Even with these small hurdles, setting up a redirect test through Optimizely can be radically faster, easier, and lighter on page load than going another route.

4. Integrate With Heatmaps

Heatmaps are a powerful, often underutilized tool that can sit between quantitative and qualitative testing. Leveraging heatmaps with your tests can visualize users’ preferred browsing and shopping behavior.

For instance, by integrating a heatmap into a new test that involves enhanced product photography on, say, a collection page, you can see just how many clicks those new photography assets are getting versus their controls. Further, you can more clearly evaluate the relationship between clicks on descriptive product text, like product names, and their photographic counterparts.

Additionally, heatmaps can show where click activity is happening within your commerce flow where the UI/UX may not be optimized for it. For instance, if some elements within your side cart are getting a lot of clicks but they aren’t clickable, then you have clear evidence that exploring some usability optimization in your cart might be needed and would drive more order completions from the cart

Optimizely makes heatmap integration a cinch by already integrating with existing heatmap platforms such as ClickTale. Don’t use such a heatmap platform and instead use a custom analytics database? No worries. With Optimizely, you can still achieve such a heatmap integration easily via the Optimizely JavaScript API and tying what variant a user is in with a particular set of analytics or heatmaps. No matter which technology you integrate with or how, heatmaps can add another great layer of data for deeper user analytics and consumer behavior research.

5. Vanquish Page Flicker

Optimizely does its part to be as fast as possible on your page. That said, it’s up to you to keep your experiments as lean as possible to avoid the dreaded page “flash” of old/incorrect content (often referred to as FOOC)

Eliminating that flicker can be accomplished a few ways. For now, I’m going to concentrate on the quickest and easiest options available to you for a speed boost.

First, ensure that old, completed, and abandoned experiments are properly archived. Note that experiments that are paused or set to draft will still load along with your active experiments in your Optimizely snippet by default, which means that the JavaScript and CSS for each variation will be loaded (but not run) as well. Leaving bloat around can pick away at the loading times of the experiments that you do care about, so lean down as much as possible.

If you are a prolific tester, weeding out your inventory of legacy experiments can yield dramatic speed gains for your website. Hundreds of kilobytes of deadweight can be shed rather quickly, especially if you have done extensive tests that involved a multitude of changes within a variation. Along these lines, you can even exclude all draft and paused experiments if you do not need to preview them.

Second, clean up any unused or duplicate code within your variations. Doing so requires that you access the code editor for each variation and checking your Experiment JavaScript and CSS options. Since the editor does everything in order, if you forgot to hit undo a few times, you may have some extraneous changes that will run and invisibly slow things down.

Speed gains aside, I’d be remiss if I didn’t say that auditing the integrity of the changes within your experiments is just good business practice. It keeps your experiments in line with each other as well as any standards you have set up. The speed gains you’ll enjoy are the delightful side effect of well groomed code.

Here’s a code snippet that might seem familiar if you’ve forgotten to use undo before:

// Uh-oh, looks like I forgot to hit “Undo”

$(".search--trigger_open").css({"font-weight":400, "text-align":"justify"});

$(".search--trigger_open").css({"font-weight":400, "text-align":"center"});

$(".search--trigger_open").css({"text-align":"left"});

$(".search--trigger_open").css({"font-weight":"bold", "text-align":"justify"});

// … After some clean up …

// Much better

$(".search--trigger_open").css({"font-weight":"bold", "text-align":"justify"});

Third, add as much of the test’s functionality directly into your website’s CSS and JavaScript files. Why? Because doing so allows your Optimizely snippet to do the lighter lifting job of adding classes or triggering events. The benefits of this are several fold:

• You get the ability to easily reuse any code from your existing website and easily tweak it at any time
• You get the added performance bonus of having the CSS and JavaScript load on the page, which more closely simulates the normal user experience
• You reduce the overhead waiting for you when it comes time to deploy the winning variant natively to the website

6. Use the API to Start, Pause, and Schedule Experiments

Optimizely’s API allows you to do much of what you can from their admin but from the comfort of your own system. So, if you have your own custom admin dashboard for your website, you can integrate much of Optimizely directly into it so you have more control over your experiments.

Specific controls include starting, pausing, and archiving experiments. To execute these commands, send a PUT request using the experiment ID and a status of `{“status”: “Running”}`, `{“status”: “Paused”}`, or `{“status”: “Archived”}`, respectively. You can manipulate your experiment schedules from the API too. Doing so gives you the same level of autonomy to integrate Optimizely into your own systems.

Enabling these controls directly within your own admin dashboard is fantastic for stakeholders who may need to have such access but are not familiar with the Optimizely dashboard. Similarly, granting access to stakeholders in this manner is a way to control who has full access to the entire Optimizely account.

Beyond your custom admin dashboard, you can also integrate the Optimizely experiment start, stop, archive, and scheduling controls into your development build process. Putting this integration in place can easily streamline your launch processes as well as add in an additional QA layer—namely, the experiment cannot launch until the correct build is complete and deployed successfully. Of course, such an implementation would differ based on what version control system and continuous integration scheme you use. Regardless, the benefit is universal and worth considering.

7. Use the API to Show Experiment Stats Within Your Dashboards

You can leverage the Optimizely API to display experiment data and results directly within the dashboard. This integration gives you and other stakeholders using the dashboard a more focused look at your experiments. It also improves the clarity and ease of seeing how your experiments are affecting various metrics across your website without having to manually compare them from different sources.

Bottom-line: Having a richer dashboard of metrics will empower you to make better business decisions faster. And isn’t that why we’re testing in the first place?

8. Change the Editor Preview Page

Confession: I routinely forget to change the editor preview page on my experiments, which drives me a bit batty because it’s such a no brainer. Forgetting to make this change spawns needles of frustration thinking I needed to access the editor’s Interactive Mode to get to the page I needed.

For example, the first time worked on an experiment that contained a page hidden from the site’s navigation, I continuously got frustrated trying to navigate to the page within the Optimizely editor because I couldn’t just click on a link to the page. This is a common occurrence when working on redirect tests as the other page is typically not accessible to users outside of the test.

At the End of the Day…

Optimizely is an immensely powerful platform with more options to tweak and tools to use than you can absorb in a short time. That can be overwhelming. Don’t let it be. Take your time, learn by doing, and stay hungry to maximize your strategic advantage by mastering the capabilities of this platform.

Happy testing, and happy optimizing.