Scale your Experimentation Program with Optimizely’s REST API

This visual interface is a great way to get started with experimentation, but what many users don’t realize is that it’s just scratching the surface of what they can do.

With some development work, you can go much deeper. For every feature in our user interface, we expose a corresponding developer API. These APIs allow for much finer-grained control over your experiments, and they make it easy to plug Optimizely into a broader workflow or technology stack. In this series, we’ll walk through how some of the most sophisticated experimentation teams are using our APIs to take their programs to the next level.

Choosing the Right Tool for the Job

The first thing to know about Optimizely’s APIs is that there are several, each designed to enable different use cases. To highlight a few:

• The REST API lets you create, manage, and monitor your experiments programmatically. It’s the best tool for integrating experimentation into an existing workflow, like your CMS, or building tools to oversee your testing program
• The Event API lets you track offline conversions – for example, a real-world purchase or call to phone support. It also allows tracking events in a website or app where no Optimizely snippet or SDK is implemented. Another use case is tracking events from third party sources like measuring advertising revenue on a media site.
• For client-side testing, the Javascript API allows for customizing how Optimizely runs on your page. You can use it to set up complex event tracking, integrate with tools like tag managers, or carefully control the timing of code execution.
• For server-side or mobile testing, each Full Stack SDK offers APIs for triggering events, activating experiments, and controlling where and how the SDK runs.

In the upcoming weeks, we’ll do a deep dive into each of these APIs. Today, we’ll start by going into more depth on the REST API.

REST API

The REST API allows you to run experiments without ever logging into the Optimizely UI. More broadly, it exposes all the resources in an Optimizely project through a RESTful interface, mirroring the UI as closely as possible but allowing for deeper integrations and scalable automation.

The most common use case for the REST API is integrating with an existing workflow. For example: let’s say you’re a newspaper that wants to A/B test all the headlines on your site, which has been shown to increase click-through rates by over 20%. One way to do this would be to log into Optimizely and create a new experiment for every article you publish. But that can get tedious. Each editor has to step out of their existing workflow, log into a separate tool, get an account, learn how to use the tool, start the experiment, and then remember to check the results later and move forward with the winning variation.

To shortcut this process, media sites like the New York Times are integrating A/B testing directly into their content management systems. Editors write one main headline, and then a few alternative variations right in the same spot. Then, the REST API automatically starts a new experiment with the different headlines. You can see this in action through our WordPress plugin and explore all the PHP source code, or see a simple Python example.

This isn’t just for media sites. We’ve also seen retailers integrate with e-commerce platforms to test out different product descriptions and images, travel sites integrate different offers, etc. This kind of integration is powerful any time you have an existing workflow that could benefit from built in experimentation, like writing Extensions in your local environment and syncing with Optimizely via a command line tool. Some teams have used integrations to scale their programs to run over 1000 experiments per month.

Another way to use the API is building custom dashboards. You can build your own listing of all the experiments you’re running and tie them to other data that matters to your program, integrating with tools like JIRA or Asana. You can also use our results API to track uplift programmatically. Companies like Atlassian have built internal tooling, including graphs, that allow them to more granularly track lift between product lines.

We’re excited to see what you build with the REST API. Stay tuned for upcoming blog posts on how to get the most out of the Event API, JavaScript API, and Full Stack SDKs.

Today, we’re pleased to share a pair of new updates to the Optimizely JavaScript snippet which are launching in general availability: a more performant event logging endpoint (/events), and new JavaScript methods for event timing.

/events Logging Endpoint: Better performance through batching

Our customers care a lot about the performance impact of including our snippet in their web frontend. We’re constantly improving our snippet to make it faster, and we’re happy to announce the /events endpoint: a new logging endpoint for the Optimizely X Web snippet.

Today, our snippet generates one network request per event to be tracked, and there are a lot of events being tracked on any page including:

• Decision events (one per activated experiment)
• Pageview events (one per Page activated)
• Conversions events (clicks, custom events, etc.)

We know that each network request has an impact on page performance, especially during the critical first seconds of page load. This is especially important on mobile devices where each request consumes finite data and battery resources.

With the release of the Event API, we introduced the capability to submit batches of events in a single request. The /events logging endpoint for the JavaScript snippet is an implementation of the Event API.

A few things happen when a snippet is configured to use the /events endpoint:

• As events are triggered, they are appended to a localStorage queue
• Events are pulled off the queue, batched together and sent once per second for the first ten seconds after snippet activation
• Any events that are triggered after this point will be sent immediately, without queueing or batching
• Note: using the /events logging end point may result in a slight decrease in the numbers of unique visitors and conversions displayed on the Optimizely results page as some visitors who load a page and then bounce within the first second will not be tracked

These optimizations may sound simple, but they have the potential to deliver pretty astonishing performance improvements. One of our early adopters reported that:

“[The new snippet] went live this morning. It reduced page load by a full second. That made quite an impact! Keep these changes and improvements coming!”

holdEvents and sendEvents: Granular control over event logging

In addition to improved performance, we’ve also added the ability to control the timing of when the snippet starts to generate requests.

This is implemented through a pair of new JavaScript methods: holdEvents and sendEvents. Here’s how they work:

1. Call holdEvents before the snippet initializes to prevent any events from being logged
2. Call sendEvents later on to start dispatching events

So why would a developer want to control the timing of Optimizely event logging? Two common use cases are:

1. You want to further optimize page load performance by delaying event logging more than one second
2. You want to align the timing of Optimizely’s tracking with third-party analytics

This second use case is really important, and something we think about a lot at Optimizely. We know that our customers expect the numbers on Optimizely’s results page to line up with their analytics solution, which is often the “source of truth” for the metrics they care about most. When we diagnose analytics discrepancy issues, we often find that the root cause is a mismatch in timing of event logging between Optimizely and third-party analytics, where Optimizely begins logging events earlier during page load. By calling holdEvents before the Optimizely snippet begins logging events and then calling sendEvents part of a callback method provided by third party analytics, it’s possible to significantly reduce this timing gap.

Originally, without this update, Optimizely events could be sent before and after 3rd party events. This made it harder to align Optimizely with 3rd party analytics tools as shown in the image below.

With batching in the new snippet, fewer network requests are generated, and request timing is delayed to be closer to that of third-party analytics..

Finally, using the holdEvents and sendEvents methods in conjunction with batching can align the timing of events to happen closer to the timing of third party analytics tools to reduce data discrepancies.

One of our customers implemented these APIs in an effort to more closely align their Optimizely results with those of their analytics platform. Prior to implementation, Optimizely was showing ~12% more unique visitors. Today, Optimizely’s count of unique visitors is within 3% of the analytics platform, which our customer considers within the acceptable margin of error.

One thing to keep in mind is that implementing these features will have an impact on experiment results. Because these features prevent Optimizely from logging events immediately, there will be some number of visitors who “bounce” from your website before any events are logged. This could lead to a decrease in the number of unique visitors and conversions displayed on the results page. To mitigate this factor, snippets using the /events endpoint use a localStorage queue for unsent events, enabling Optimizely to log events generated when a visitor quickly navigates to another page where the snippet is implemented. So the only time the snippet would fail to track events entirely is if the visitor triggered events and then exited your site before those events could be logged.

Getting access

Ok, so if you’re as excited about these new features as we are you’ll probably want to get started using them right away. These features are now generally available and fully supported, but you need to opt-in to get access. Here’s how it works:

1. Submit a support ticket requesting access (optimizely.com/support).
2. Our support team will enable access to the new features in your account.
3. The next time any snippet associated with your account is resaved, it will begin using the new logging endpoint.
4. You can then start calling holdEvents and sendEvents. Note that any calls to these APIs prior to enabling the new logging endpoint will have no effect.

We’ll be rolling these new features out to all snippets by default in January of 2018, but if you’d like to get your hands on them now go ahead and opt-in!