Ask an Experimenter: the Fox Networks Group Edition

In addition to leading a well attended and informed conversation at Opticon19 about In-App Testing, Alex agreed to have a chat about his path to experimentation and his career.

How did you get involved with experimentation?

Alex Alwan: I got involved gradually. I was working at Airbrake as a Product Manager and my job was to ship more features and release code. In that role, I was looking at engagement and retention as well as new customer conversion rate. 

Airbrake is an error monitoring tool. At the time it had three paid plans in addition to an enterprise plan. While looking at the quality of leads and the conversion rates, I set up Segment and Kissmetrics. Once these tools were integrated with the A/B testing tool, I was able to produce experiments reports instantly.

I was able to see the immediate impact when we shipped a new feature or UX improvement. That was an important moment for the entire team as we realized the value of A/B testing. 

It took some time to spread the culture of experimentation but what helped me convince the team was the fact that our experiments had a direct impact on the company’s revenue. We ran multiple experiments after that and launched many features as experiments which resulted in doubling the revenue within a year only by improving the purchase conversion funnel and product features.

That seems like a successful start? Where did you go next? 

I joined Spark Networks where I worked on JDate and Christian Mingle online dating apps. We ran multiple experiments to test different user experiences related to timing and frequency of paywall where we ask users to upgrade from free to paid plan. My responsibility as a product manager is to put myself in the customer’s shoes and ask; what do I need and what do I want? 

You are now at Fox Networks Group. How is your experimentation team structured?

I work with product and engineering teams at Fox that build web, mobile, OTT apps for some of the world’s leading and most valued brands, including FOX News, FOX Sports, Fox Network and FOX Television Stations. At Fox, we are keen on having experimentation as part of the culture. Experimentation bridges between the various teams, and the optimization team is the center of excellence that empowers, enables and educates all teams to run experiments to achieve their OKRs. 

We have great support from the product leadership. Without that support scaling optimization programs would be difficult. Our VP of Product Amit Dudakia encourages Product Managers to adopt A/B testing approaches and use it with every major new feature and design. Fox Nation is Fox’s major subscription-based product launched early this year, and we have been working heavily on optimizing its subscription conversion as well as engagement rates. 

Is there a ‘Test & Learn’ mentality at Fox? 

Absolutely. We understand that even a failed experiment is a win because of the learnings associated to it that help avoid the risk of rolling out blindly. With every experiment we learn something and in many cases, we iterate and keep improving.

You mentioned you have a new role. Congratulations, could you share with our audience what that is? 

Sure thing! Recently I started a new role that allows me to help the product team identify areas of improvement and new product ideas. The role is Sr. Product Manager, Product Strategy. I work closely with product leadership to drive initiatives from the company’s OKRs and track the impact of initiatives in general. 

While my focus in the last year and a half was on how to build AB tests, my focus now is on what to build and why to build it. 

This role requires working with a team of data analysts to create reports and dashboards that enable all teams to gear their efforts towards achieving our common goals.