Four takeaways from our virtual product experimentation summit: Test and Learn

Last week we held our first-ever virtual product experimentation summit: Test & Learn.

Around 1,500 professionals spanning product, design, analytics, marketing, engineering, and growth roles joined us live for the half-day master class in experimentation. There was incredible engagement across all of the sessions, and below, you’ll find some valuable insights from the event.

You can sign-up here to watch all of the sessions from Test & Learn on-demand, and crack the code on how industry leaders bring habit-forming products to life.

4 Key Takeaways from the Event:

#1: Products that become (and remain) part of our daily rituals are founded on data, experimentation, great design, and iteration

In his Keynote session, “Hooked: How to Build Habit Forming Products“, Nir Eyal talked about identifying customer needs through human psychology. Products that serve those needs (and motivate users to keep coming back) are not simply the outcome of terrific ideas. Products that hook users are the result of a commitment to innovation embodied by the most successful companies in the world. Applying data-driven and experimentation techniques are the keys to making a product incredibly rewarding, habitual, and easy to use.

Learn even more from Nir about his take on experimentation and becoming ‘indistractable’ in this interview he did with us before the big event.

#2: Behind every ingenious product is an insightful, guiding metric

Indeed’s Ketan Gangatirkar discussed how he takes a scientific approach to metrics. His model focuses on using product data to increase Lifetime Value, which he asserted is the best guiding metric.

Meanwhile, Alicia Shiu from Amplitude shared how to identify a north star metric. She explained how to define the north star metric based on which of the 3 “product games” (attention, transaction, or productivity) is in question. Her hypothesis-driven approach to optimize said metric yields greater customer value, strategic alignment, and revenue.

In the session with Matt Gomes from OppLoans, he shared how to put a data-driven strategy into action by running light-weight, iterative tests to move the needle faster.

#3: Leading product development teams sustain and accelerate innovation by rooting experimentation in all practices

Feature flagging and testing are becoming the gold standards when it comes to building and maintaining highly resilient applications.

In Emily Nakashima’s session from Honeycomb.io, she talked about empowering software teams to take ownership, ship higher-quality code, and improve faster than ever. You can also check out the interview with Emily to learn more about building a Test and Learn culture within engineering organizations.

Booking.com, a longtime industry leader in experimentation, approaches every challenge and idea as an opportunity to test. In Lukas Vermeer’s session, he reviewed how adopting a culture of experimentation enables data-driven decision-making and faster, safer feature releases. He explains how practices like asynchronous releasing and guarding against regressions are achieved by incorporating feature flagging into the development process.

He also highlighted how to use experimentation to identify problems as well as to test for the best solution.

#4: Results-driven prioritization is key to choosing which user behaviors to drive and optimize throughout the customer lifecycle

From her experience at The Wall St Journal, Olivia Simon described how she uses data and experimentation to create user habits that drive value. One of her tests increased mobile app downloads by a factor of more than 70x!

Read more about how Olivia uses a Test and Learn approach to build better products.

In her session, Dropbox’s Angel Steger shared how the role of design in the experimentation process increased subscription upgrades by more than 3x.

She described her method as starting with a chainsaw to experiment with big changes; before moving to a chisel to test and polish.

Last but not least, Lauren Schuman at MailChimp told the story of how she used Optimizely to overhaul a subscription funnel that translated into millions of dollars in revenue. She shared how scaling experimentation across the entire organization was essential to running experiments that improve the customer experience at every touchpoint.

Sign-up for free to catch up on all of the sessions from our Test & Learn Summit.