Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more! Download Now

Optimizely Blog

Tips & Tricks for Building Your Experimentation Program

X

Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!

Get It Now




Around 2007, Matt Wishnow had a revelation about decision-making: most decisions businesses make—whether it’s a marketing plan, a design brief, or a feature on a product roadmap—are “really a bunch of hypotheses masquerading as priorities.”

This epiphany drove him to start a digital agency that helps some of the largest retail brands in the world make smart decisions.  Through many experiences at large corporations, he realized that most businesses didn’t really employ data or testing to drive their decisions. Any testing they did was unrelated to big decisions around product design and development.

“It was almost just like something that somebody had to check off on a checklist, all the analysis was postmortem; everything was still driven by gut,” Matt told me.

In 2012, Matt paired up with former co-worker Ryan Garner and founded a digital optimization agency. “I just got really excited about filling what I thought was a massive hole in the market—not the software, because the software was becoming more usable—but just really the people and process side of it,” Matt said.

GrowingMatt and Ryan called the company Clearhead, inspired by the phrase they would use in their previous roles at Warner Music Group when balancing many differing opinions. Today, some of Clearhead’s clients include Adidas, Fresh Direct, Blu Dot, Sephora, Keurig and Patagonia.

I spoke with Matt and Brian Cahak, Clearhead’s VP of Business Development and Marketing, about how they approach client engagements, how they’ve seen the optimization landscape change over the years, and advice for agencies starting to offer optimization.

What do you think that the market for A/B testing consulting services will look like over the next five years?

Matt Wishnow is the co-founder and CEO of Clearhead.

Matt Wishnow is the co-founder and CEO of Clearhead.

Matt Wishnow: I think increasingly companies will realize that testing and targeting are both more foundational and supportable than ever before and less of a “nice to have.” I think a lot of companies still think, “Oh, build feature and then test it,” or, “Test a few things but not everything.” The product roadmap is still this gorilla in the room that has some bare relations to A/B testing or hypothesis validation but not necessarily one-to-one core relation.

Once businesses start saying nothing gets designed or built unless the hypothesis has been validated, then I think, what Clearhead does as a business is unleashed to maximum value. Because then the budgets and resources that people are currently applying to design and build, are really rationalized against testing and validation.

I don’t have a crystal ball but in five years it seems reasonable that most data-driven businesses would operate that way. And if that is true, then what we are doing at Clearhead, in my estimation, is the future model for what a digital agency is.

Are there any surprises in the impact that you’ve seen optimization have on your clients’ business beyond increasing conversion rates or driving more revenue?

Wishnow: At Clearhead, we don’t call ourselves a conversion rate optimization agency. I think that’s a term that tends to come from the legacy of landing page optimization. I always remind our clients that if you’re looking to improve conversion rate, you can put everything on sale tomorrow and you will improve conversion rate.We’re in the business of finding the most efficient path to profitability through design and build using data.

For us it’s always about business profitability. Some of the surprising benefits come up when a client is certain that a new feature they’re contemplating is going to move the needle. They’ll say, “As a courtesy, we’ll do some quick testing to validate it.” And then they find that what they thought, was absolutely not true. Now there might be good reasons for the feature and hypotheses worth testing, but the big surprising benefit is not that we improved conversion rate, but we saved the client a lot of money and opportunity cost of building something that wasn’t going to move the needle. That’s one really big one.

“Testing is absolutely a gateway drug to a data-driven business in general because you get the data so quickly and it’s such high confidence data when done properly.”

-Matt Wishnow

Once our clients, both the stakeholders and executives feel what it’s like to be data-driven in their approach, they can’t get enough of it. What we’re absolutely seeing is their priorities and intention shifting from more traditional product road mapping approaches to a much more iterative approach.

How do you sell an optimization engagement or project? Who is the typical buyer?

Brian Cahak

Brian Cahak is the VP of Business Development and Marketing at Clearhead.

Brian Cahak: We typically sell to the head of digital, chief digital officer, or the vice president of e-commerce, marketing or product. They tend to be the buyers because they have strategic goals that are much higher level than just running some tests. While it doesn’t take much to simply run tests, it takes a great deal of care and effort to commit to continuous hypothesis development, testing, and improvement.

Initially, we typically sell a proof of concept. We want to give our clients enough of a taste of what we do so that they can see value and then have a real vision for how we could work together long-term to drive value. So rather than saying, “Hey, you’ve got to swallow the whole elephant right now,” I say, “Well, nibble on the ear a little bit, and maybe over a six month project we work together on a number of tests. Whatever makes sense so that you can see full cycles and trust us, trust [the optimization platform], and trust that we can deliver results with all of that.” Then they’ll re-up for typically a much longer retainer. So that’s what we start with, the POC.

How does Clearhead organize the team to service each client?

Cahak: Our account team tends to be four people: a program manager, who is the day to day quarterback who works directly with the client. Some agencies might call this person a client or account manager. Then there are three people who actually do the testing and analysis: a front-end developer, a designer, and an analytics person who can extract the data and do all the segmentation to tell a story with it.

Do you think that there’s a certain number of characteristics that qualify a client to be ready for testing and optimization?

Wishnow: I think, first and foremost, there has to be an acknowledgement that testing is not a tactic separate from product design and development. That comes from a very senior leadership position. In companies where testing is fragmented, where it’s run by marketing exclusively, or UX exclusively, there are challenges. But businesses where all of those roll up into one executive and that executive, or group of executives, acknowledges that testing can’t be a side project, that it has to be fundamental to product design development, I think those businesses are most likely to succeed.

What advice would you give a digital agency that doesn’t offer conversion optimization today?

Cahak: I think the easy way to learn is by partnering with an optimization specialist. Instead of spending a lot of time and potentially fumbling it, you can see how a handful of the engagements work, and see if this is something you want to get into. Talk together on an opportunity. Watch and draft off of of someone who has done this before.

What about advice for an optimization focused agency trying to grow their business?

Cahak: I think the relational credibility is a big one.You’ve just got to know the space, know the decision makers. I don’t mean know them personally, I mean know how they think. How do they spend on this? How do they think about this space?

Then you really have to show proof points. At some point, relationships can only get you so far. Now you’ve got to actually go out and show that, “I did A, B, and C for these three clients. And because of that, you should hire me too.” That’s the next level. Get those proof points as quickly as possible. Because I might trust you, you might sound like you’re smart, but you are way more credible if you can show me case studies.

This interview has been condensed and edited.

Optimizely X
comments powered by Disqus