Download our FREE ebook of 43 A/B testing case studies from the world's leading companies for test ideas and inspiration. Download Now

Optimizely Blog

Tips & Tricks for Building Your Experimentation Program

X

Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!

Get It Now




Widerfunnel, one of the leading experimentation consultancies, has just published their new 2019 research report on the experimentation industry. You can download your copy here.

Becoming an insight-driven business has become a competitive necessity for companies across all industries. The most innovative, fast-growth companies are rapidly developing best-practices for their experimentation programs to help fuel their decision-making process.

Business leaders need to understand the characteristics of the best programs to compete today.

A new research report shows the industry’s progress

In this new report published September 2019, “Harnessing the experimentation mindset to drive growth”, we’ve  gathered new research data from decision-makers in Digital, Marketing, and Product from leading brands across multiple industries including Microsoft, Target, Allstate, Domino’s, BMW, and more, to uncover the characteristics of a healthy culture of experimentation.

The study reveals four components of thriving experimentation programs, which are:

  1. Overall organizational agility
  2. Accountability via strategic metrics
  3. Executive participation in experimentation
  4. Customer obsession throughout the organization

Below are a few specific insights you’ll find in the report, or you can download the full report now.

Centralized or decentralized, agility is what matters most

Widerfunnel’s research in 2018 found that more mature experimentation organizations structure their experimentation programs in a hybrid model, sharing characteristics of both a centralized and decentralized program.Centralized Decentralized Hybrid Stack

Last year, research showed that this hybrid model enables higher experiment velocity. 

widerfunnel experiment velocity

This year, we found a similar insight. As experimentation programs begin, they tend to start without much organization, or as a centralized team of experts. As they mature and the organization demands more experiments for decision-making, the decision to centralize or de-centralize becomes increasingly important. Fast-growth organizations are also more likely to adopt a hybrid model, retaining characteristics of centralized and decentralized structures. (There’s more on the topic of experiment velocity in the recent blog post, Going from 10 to 100 experiments per year.)

The speed of decision-making and acting has shown up in the research as a primary driver of success this year. 

Graph Widerfunnel

It’s especially difficult for large organizations to maintain decision-making agility. We found that, while 90% of companies that reported strong year-over-year revenue growth made quick decisions, larger companies were less likely to.

dot graph

Experimentation program accountability is most effective when executives have been involved in creating the metrics

Experimentation, like any other core strategy, must be held accountable to deliver business impact. Organizations that outline clear goals for experimentation and assign clear metrics and KPIs to experimentation efforts are more successful in the long-term.

Leadership involvement in setting these metrics is particularly important.

Among respondents in companies showing the strongest growth, 40% designed an Overall Evaluation Criteria (OEC) with the executive team, compared to just 17% of those in companies showing no growth.

no growth percentages

Executive-level support of experimentation is a game-changer.

This year’s research shows a correlation between how an organization’s executive team views the experimentation program and reported year-over-year revenue growth.

A full 80 percent of companies with the strongest YoY revenue growth responded that their executives were fully bought-in or at least see value in experimentation.

Fast-growth companies prioritize a culture of knowledge-sharing more than other companies.

Sixty-eight percent of respondents in fast-growth companies reported a significant knowledge-sharing culture, where employees are encouraged to learn and share knowledge throughout the organization; compared to only 40% of respondents whose companies did not grow last year.

This is just a sneak peek at the insights in the full report. Get all the research findings, charts, and expert insights in the full 38-page report

About Widerfunnel

Widerfunnel is a leading optimization and experimentation company. We partner with change-makers in growing organizations to launch, accelerate, and scale our experimentation programs—leveraging industry-leading frameworks and methodologies. Our clients, such as HP, GM, Intercom, Sport Chek, and The Motley Fool, see relentless growth and gain insights that empower them to make confident, evidence-based business decisions. You should subscribe to our blog.

 

Optimizely DXE Report