This is part two of a series on Stats Accelerator. In the first part, we explained the when, why, and how of Stats Accelerator. In today’s installment we will discuss a major roadblock to successful productionization of bandits in the A/B testing context and how we eventually overcame it. This is a high-level overview. For […]
At Opticon last month, we launched a rebrand of Optimizely. As product designers, we knew we’d have to move quickly to roll out the new brand in our product to coincide with the big Opticon launch. But redesign projects are often complex, fraught with challenges, and take resources away from higher priorities such as designing […]
In this post by customer Kyle Rush, VP of Engineering at Casper and former Deputy CTO on the Hillary For America campaign, we can see firsthand how his team dramatically improved the site performance of casper.com. Using a new recommendation from Optimizely that encourages more flexible implementation options for customers through Optimizely Web, Rush and […]
You’ve probably heard that performance matters on the web. But in recent years, performance has taken center stage. It’s gone from being a priority for technical teams to becoming a top priority for business and product leaders as well. Why? Performance can impact key business metrics like page or cart abandonment, conversion rates, and ultimately […]
This content originally appeared on Medium. What is Stats Accelerator and why use it? When a customer sets up an A/B/n test using Optimizely, they choose what percentage of visitors (or samples) will be exposed to each of the test’s variations. This is the test’s traffic allocation. The traffic allocation stays the same, unless a customer manually changes […]
Why Product Experimentation? Before I joined Optimizely, I worked at Microsoft on what was once the world’s leading internet browser, Internet Explorer. For those of us who have forgotten about this great browser, here’s a screenshot: As an engineer, I was tasked to work with a product manager on developing an algorithm for determining the […]
If there was a glass ceiling for experimentation, it would be scalability and measurement. One of the key challenges for experimenters lies in developing a strategy that will enable them to scale their optimization program and create the right metrics to effectively measure its success. The most innovative of organizations have multiple teams testing, […]
Creating a faster and more engaging site experience often requires tradeoffs between various revenue streams of the business. Can the increase in conversion from more engaged users more than offset any loss in ad revenue? It’s a bold bet that Realtor.com’s analytics and experimentation team set out to answer at the beginning of 2018. A […]
Welcome to the third installation of Ask an Experimenter, a new series where we interview experts about how they are building a culture of experimentation at their companies. This interview features Kevin Chen, Growth Product Manager at Reddit. Kevin works on the strategy, roadmap, and execution of push notifications and email products for Reddit. Tell us a […]
We often hear that product development teams don’t have an experiment backlog or that they struggle to come up with experiment ideas. Almost certainly, though, you have a product roadmap. This list of problem-solution pairs includes the features that will certainly help you defeat the competition. Well, maybe. No matter how much data analysis, user […]
