Download our FREE ebook of 43 A/B testing case studies from the world's leading companies for test ideas and inspiration. Download Now

Optimizely Blog

Tips & Tricks for Building Your Experimentation Program

X

Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!

Get It Now



Determining the best customer experience can be challenging. Teams have often relied on surveys, focus groups, and analytics to determine what customers want and then build based on that understanding. Machine learning can be used to help deliver more relevant content but tuning those models can be time-consuming and require a lot of manual analysis to determine the best experience for customers. We’re excited to announce an integration with Amazon Personalize to combine AWS’s powerful machine learning technology with Optimizely’s experimentation platform to help teams test, iterate, and roll out machine learning models to drive customer engagement and foster innovation.

Amazon Personalize makes delivering machine-learned powered experiences easy

Amazon Personalize provides access to the same personalization models Amazon has developed over the last 20 years to power their own business. Optimizely for Amazon Personalize enables software teams to A/B test and iterate on different variations of Amazon Personalize models using Optimizely’s progressive delivery and experimentation platform. Once a winning model has been determined, users can roll out that model using Optimizely’s feature flags without a code deployment. With real-time results and statistical confidence, users can continually monitor and optimize their digital experiences. 

Optimizely + Amazon Personalize

How the integration between Amazon Personalize and Optimizely works

Companies use Amazon Personalize to train different machine learning models. Amazon Personalize provides customized personalized algorithms like user recommendations, personalized rankings, and related Items. Before today’s integration, teams questioned: which algorithm will have the biggest positive impact on our business? And how can we know for sure that product recommendations are improving our business at all?

Improved customer engagement with measurable impact

Using Optimizely for Amazon Personalize, teams can easily segment and test model variations with a percentage of their customer base by setting up a feature flag in Optimizely that controls which machine-learning powered experience their customers are shown. Teams receive automated experiment results and statistical reporting to then identify the best-performing models with confidence. With the ability to test and iterate on multiple versions of machine learning models, teams can improve user experiences more quickly. In addition, real-time results with statistical significance on how each experience is performing further reduces guesswork and quantifies the impact on application and business metrics.

Experiment Results

Example: Experiment results showing winning Amazon Personalize recommendations model

Safer machine learning releases 

To further mitigate risk, the winning model and experiences can be rolled out to a percentage of customers rather than everyone to limit a blast radius should there be an issue and can be rolled back without a code deploy. Individual model parameters can be remotely updated and validated using Optimizely’s dashboard. For example, Optimizely offers flexible audience targeting so Personalize model variations can be rolled out only to specific customers, geographies, or other key segments.

Rolling out Personalize Model

Example: Rolling out an Amazon Personalize model to just frequent customers

Demo Video: How Optimizely + Amazon Personalize drives better e-commerce experiences

An online retailer could use the Optimizely for Amazon Personalize integration to run experiments to see which machine learning model drives the highest purchase rate. Watch the video below to learn how teams can use this integration to drive revenue.

How to get started

To get started with this integration go to optimizely.com/aws and check out its GitHub repo.

Optimizely X