How Discovery Increased Ad Viewability & CTR Through A/B Testing

Discovery Communications is a global mass media and entertainment company with an astonishing portfolio of television networks, including the Discovery Channel, Animal Planet, TLC, the Oprah Winfrey Network, and Science Channel. With more than 2.9 billion global subscribers, Discovery Communications has the potential to reach 2 in 5 people on the planet solely with its television networks.

But for Discovery, reach is just the start of a long relationship with its fans. The Digital Media product team has been leveraging A/B testing to increase audience engagement with its content and maximize revenue from online display ads.

Engaging with Super Fans

Within that huge pool of 2.9 billion subscribers lives a group of people that Discovery cares deeply about, the highly engaged and loyal customers, aka the “super fans.”

“Many people have a favorite TV show, they may really love that show, but do they ever visit the website? The folks that visit a TV show website want to dig even deeper and get exclusive content or catch up on something they missed,” says Jeffrey Douglas, Director of Product at Discovery Digital Media. “That’s the audience we cater to; engaged fans who want more than what’s on their TV screens.”

Loyal customers, VIPs, super fans; regardless of what any company calls them, it’s imperative for Discovery to provide this customer segment with as much of their desired content as possible.

Discovery’s video hub is an engagement center for super fans, and an opportunity to serve ads to monetize their content.

One of the ways Discovery Communications does this is by optimizing their website to make it as easy as possible for super fans to engage with content. Testing is a key part of the product team’s engagement strategy.

The Ken Burns Effect

By experimenting to improve their site visitor’s experience and ensure that they discover and watch the content they’re looking for, Douglas has found ways to have a meaningful impact simply by A/B testing the few elements of the site.

“When it comes to optimizing, we don’t have a lot to work with. We have the images and the headline, which we will optimize on an individual basis,” Douglas said. “As a product person, I control the rest of the elements on the screen, the interactions, the design and the experience.”

He started by A/B testing something simple: the video preview image. Despite being beautiful and compelling, some of the images on Discovery Communication’s video players were not enticing enough viewers to click. Douglas decided to run an experiment he dubbed “the Ken Burns test”.

Ken Burns is documentary filmmaker who often focuses on subjects that pre-date film. He utilizes a motif in which the camera pans across still images, creating a sense of motion and bringing them to life.

“I don’t purposefully give all my tests creative names, but it makes the process a little more fun.”

“I don’t purposefully give all my tests creative names, but it makes the process a little more fun,” says Douglas. He asked, “What would happen if we did the ‘Ken Burns effect’ when a user on a desktop hovers over this image?“

Douglas and his team didn’t know if this subtle change would set the world on fire or not. They thought users would click more. It was a guess.

Douglas says they ran the experiment on 20,000 visitors and it took him and a developer 15 minutes to set up. The Ken Burns Effect resulted in a 6% increase in click-throughs on the images.

“We’ll take these small incremental wins and continue to build on them,” says Douglas. 

Improved Ad Viewability 

Of course, once Discovery Communications has enticed a visitor to watch a video, it’s important to understand the impact of their test results on another key metric: ad viewability.

The Benefits of a Testing Culture

Even though he isn’t a designer or developer, Douglas and the Discovery product team can test, iterate, and make changes to the website that have meaningful impact on Discovery Communication’s business.

He says Discovery’s goal is to run dozens of tests a month, build a backlog of experiments, and prioritize the ones with the highest potential to impact fan engagement.

“Ultimately, we want to be in a place where the we’re making sure the developers are able to focus on work that has the highest potential for impact for our company.”

“Our product team can fail and make mistakes that are inexpensive versus very expensive mistakes that end up using lots of resources,” explains Douglas.“Everything from QA to deploying code and so on. We can prevent developers from making changes that won’t ultimately matter… that’s really the kind of culture that we’re building here.”

Next on the roadmap, the Discovery product team is focused on page load performance and how it impacts engagement, especially on mobile devices. Their core hypothesis, not unlike other experiments, seems obvious. If pages load faster, especially on mobile, users will engage more with their content.

Read the full Discovery and Optimizely case study.