Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more! Download Now

Optimizely Blog

Tips & Tricks for Building Your Experimentation Program

X

Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!

Get It Now

Optimization Culture

A successful optimization is much more than the software. Building a culture that embraces data-driven decision making and making hypotheses is vital for optimization to thrive.

Investing in Optimization – The First Steps Towards Building a Team

Investing in Optimization – The First Steps Towards Building a Team

Suma Warrier is an optimization veteran. Over the past 10 years, her digital marketing career has focused on building and scaling optimization programs at the likes of Symantec, Sephora, and most recently Move, Inc. (operator of Realtor.com), a News Corp company.

Her latest challenge? Building an optimization program from the ground up, and enabling her organization to unlock the power of rapid testing while working towards an aggressive revenue goal. In this series of three posts, we’ll cover how she approached team structure and process, how the team set goals, and examples of experiments her team has run.

In 2014, Suma and her team ramped up ideation, prioritization, execution and analysis of Realtor.com’s tests. In this post, we’ll cover how the team is structured, the importance of their cross-functional stakeholders, and the agile processes they adopted to ensure a smoothly running optimization program.

What’s Next? Steps For Prioritizing Your Experiment Backlog

What’s Next? Steps For Prioritizing Your Experiment Backlog

Prioritization is a critical skill to master when building out a testing program. It’s about making smart choices and applying discipline to the decision-making process.

In my experience helping companies build their test programs from scratch, as well as optimizing more mature programs, I’ve seen the benefits of adopting a rigorous prioritization scheme time and time again. Based on experience working with Optimizely customers, here are three crucial key steps to follow when bringing prioritization into your optimization strategy.

How to Write a Solid A/B Test Hypothesis

How to Write a Solid A/B Test Hypothesis

Imagine you set out on a road trip. You packed the car, made a playlist, and set out to drive 600 miles—but you don’t actually know where you’re headed. When you arrive at a destination, and it’s not at all what you imagined it would be.

Running an experiment without a hypothesis is like starting a road trip just for the sake of driving, without thinking about where you’re headed and why. You’ll inevitably end up somewhere, but there’s a chance you might not have gained anything from the experience.

In this post, we’ll show you how to craft great hypotheses, how they fit into your experiment planning, and what differentiates a strong hypothesis from a weak one.

How to Prioritize Your Test Ideas and Other Critical Questions

How to Prioritize Your Test Ideas and Other Critical Questions

When I’m not running experiments on Optimizely’s conversion funnels, I love to interact with the optimization community. GrowthHackers has one of the best communities out there and last week I hosted an Ask Me Anything (AMA). The questions were very high quality and covered topics like running multiple tests at the same time, how to overcome technical hurdles, how multi-armed bandits can be helpful, what to do with inconclusive tests, and more.

If this piques your interest, have a read through the questions and, of course, continue to ask me anything.

Optimizing Content: How Kevy Writes More Without Writing Worse

Optimizing Content: How Kevy Writes More Without Writing Worse

Brooke Beach has a challenge common amongst many: producing a lot of content with limited resources without sacrificing quality.

Sound familiar?

Her marketing team has come up with a system that combines data from website analytics, marketing automation, and live chat to help create the right content for the right audiences. Intrigued as to how live chat contributes to this optimization equation, I talked to Brooke about how they go about it, and the impact it’s had on the business.

Building an MVPP: Minimum Viable Product We’re Proud Of

Building an MVPP: Minimum Viable Product We’re Proud Of

On November 18th, 2014, we publicly released Optimizely’s iOS editor. Before we launched, there was one problem the whole team rallied behind to fix: we weren’t proud of the product. To fix this issue, we went beyond a Minimum Viable Product (MVP) to an MVPP—the Minimum Viable Product we’re Proud of.

This is the story of how we pulled this off, what we learned along the way, and product development tips to help you ship great products, from the perspective of someone who just did it.

What Does it Mean to Be a Growth Marketer?

What Does it Mean to Be a Growth Marketer?

From conversations with our customers, we’ve learned that some folks who do A/B testing also have day-to-day responsibilities ranging from SEO to front end development to content marketing. But, it’s apparent that the folks who do conversion optimization have one thing in common: an obsession with growth. Their work directly contributes to company growth and impacts wins and fails. In essence, everyone who practices A/B testing and optimization is a Growth Marketer.

We’re curious about what it means to be a Growth Marketer… So we’re asking you.

5 Traits of Best-in-Class Optimization Teams

5 Traits of Best-in-Class Optimization Teams

What are your best customers doing?

That is the #1 question I hear from customers on a day-to-day basis. How do others companies do optimization and testing? It’s a great question.

Based on thousands of interactions with Optimizely customers and four years of enterprise enablement, I can confidently point to five traits that all best-in-class optimization teams possess…

For People Who Need Help Memorizing Their Coworkers’ Names

For People Who Need Help Memorizing Their Coworkers’ Names

Employees at extremely fast growing companies have an exciting opportunity to meet many smart, interesting new people. This also presents a challenge: memorizing their names. It might seem impossible after you reach a certain size and have multiple offices around the world. Maybe knowing everyone is unrealistic, but at least we can strive to match a name with a face.

During our last hack sprint, I thought, wouldn’t it be nice to have a tool that could help us recognize (and maybe even get to know) each other?

Why Prioritizing Social Responsibility Will Strengthen Your Startup

Why Prioritizing Social Responsibility Will Strengthen Your Startup

At first glance, “doing good” might seem impossible for today’s tech startup. When you’re staying up into the wee hours of the morning to ship features and investors are your main source of revenue, thinking about your social impact strategy might seem impractical.

Thankfully, we’re seeing more and more companies ditch the t-shirt clad garbage picking in favor of experimental, integrated approaches to social impact. Here are 4 ways your startup can prioritize social responsibility.