This article is part of our Mobile Teardown Tuesday blog series. Each Tuesday, we choose a mobile app and suggest a few experiments the team might consider running to optimize the user experience. Learn more about mobile a/b testing from Optimizely.

Teardown Tuesday: OpenTable

OpenTable was recently acquired by Priceline in a deal that valued the online reservation system at $2.6 billion.

OpenTable recently launched the ability to pay your bill with the same app you use to make your reservation. My friends have affectionately dubbed this “Uber for dinner” for how it mirrors the payment convenience of a ride with Uber. This new “Pay without the wait” feature is presently being rolled out in San Francisco only, with some new in-app banners to spread the good word to hungry Bay Area residents.

Testing through a new feature release

Rolling out a new service, feature, or redesign is a rare opportunity to build your understanding of how your users think about your product/service. When Uber launched in San Francisco in 2010, they positioned the service as “Your Own Private Driver.” If they had messaged the service as “Black Cars On-Demand”, they might not have seen the same success and adoption. The latter is nearly meaningless to your average San Franciscan, but the former  sounds badass and status-conferring.

As any Product Marketing Manager will tell you, it’s incredibly difficult to understand pre-launch how the market will think about your service/product. But this creates a catch-22 : if your messaging is off, your marketing will fall on deaf ears. Testing your messaging at launch is a great way to find the line of positioning that resonates with the audience.

In this type of testing, OpenTable might consider a number of different goals or success metrics. First, a tap on the banner itself is a good micro-conversion goal to indicate how effective the messaging is. Downstream, additional goals might be the number of confirmed reservations, or the number of successful in-restaurant transactions.

Let’s take a look at how OpenTable can experiment with a similar strategy  for their “Pay with OpenTable” feature launch.

Variation Ideas: New feature messaging

Here’s the current view for someone opening the OpenTable app from cold start (regardless of login state):The prominent “Pay without the wait” banner at the top of the view conveys one of the key value propositions of this new offering. I applaud the use of a value statement here, as these tend to perform better than descriptions of what the service is/does. That said, there is definite room for improvement. OpenTable might test a new tagline that focuses on features, rather than benefits:

Although I don’t expect this to outperform the original because of this feature-focus, we shouldn’t assume! What is true for the average visitor or user is not necessarily true for your visitors or users!

OpenTable might also explore a different value proposition for the new feature. I can’t say I’m frequently frustrated by having to wait to pay my bill at a restaurant and perhaps I’m not the only one! Here’s another hypothesis worth testing: focusing on convenience versus long wait times will engage more users to try the feature.

We should also consider the possibility that the original value proposition of ease and speed is a fantastic one. In this case, we should try conveying the same benefit in a different way. Perhaps “Pay without the wait” doesn’t resonate, but “Pay with a tap” does; here’s what that might look like:

In any test I run, I enjoy having a somewhat ridiculous comedy option. You’d be surprised how often these win! Here’s something fun that may also suit as a catchy tag line:

You can always leverage A/B testing and other techniques to dial-in your messaging, but this is not done in a vacuum. Your users have a history with you and your service/product that cannot be separated from the experiment itself. A new launch is as close to a tabula rasa as you’re going to get; users know next to nothing about what you’re selling/offering, and marketing “noise” is minimal.

A new launch is that single incredible opportunity to learn about your users, how they think about your offering, and what benefits they value most. Don’t waste it.