Experiments to Drive More App Store Rankings & Reviews

It’s a special edition of Teardown Tuesday, it’s Teardown Taco Tuesday. Being the taco-fiend I am, I had to download the brand new, highly anticipated, Taco Bell app to check it out. Between the bright images and ease of use, I was very impressed (and more hungry). My experience ordering a Burrito Supreme was delightful for the most part. Mid-order, a text message interrupted me (rude) and when I foregrounded the app to continue ordering, it asked me, “Wanna Rate the App?”

My immediate reaction: Pump the brakes Taco Bell! I haven’t even gotten to the Burrito section yet! I tapped “Remind me later” and moved on.

This experience inspired a question: when could Taco Bell ask me for a review to increase my chances of actually doing it? Ratings — the quality and number — matter for an app’s success, so prompting users for ratings at the right time is crucial.

Ordering experience on Taco Bell’s iOS app.

Why are ratings important? Ratings are one of the most visible components of an app’s listing and do influence a person to tap through or download. They matter for optimizing your app store rankings. Especially the number of ratings. Recently, the number of ratings has become more influential in the app ranking algorithm. Analysis from MobileDevHQ shows that “on average, the apps in the top 5 of any Top Chart have more than twice the number of ratings as apps in ranked between 6 and 10.”

So what does this mean for product managers? It means you want as many users as possible to rate your app (and of course give good ratings). At time of writing, the Taco Bell app is doing well with 1,576 App Store ratings. Whether the app comes from a huge brand like Taco Bell, or a brand new developer, the question to explore in this post is: how can apps determine the optimal way to generate lots of ratings to get noticed?

Experimenting with ratings is one of them. There are three elements to consider when experimenting with ratings:

1. When you ask
2. How you ask
3. Who you ask

1. When you ask

There are moments in your app that illicit a sense of accomplishment from people. Maybe it’s checking out or finishing an article. The time you ask for a rating should map to points when users feel satisfied or accomplished. How you prompt a user to give you a rating if you’re a transactional app like e-commerce or food delivery, should probably look different than if you’re a news or media app. A couple test ideas:

Ask for a rating after someone completes the first transaction/goal successfully.

For Taco Bell, the moment after someone orders might be a great time to ask “How would you rate your ordering experience?” The day after someone orders could work well too. (Though the stereotypical late-night taco eater may have a hazy-at-best memory of their seamless ordering experience the night before.)

Ask for a rating after a certain number of sessions or amount of time.

How much interaction does it take for a user to be engaged enough to want to leave your app a review? Should it be based on number of sessions, or the amount of time that they’ve spent in your app? These are great metrics to base tests on, and more importantly it challenges the assumptions that your team may have made when they first built your app. Another great idea is to combine the two (time and sessions) to create a custom threshold. One mobile team I spoke with only prompts users for ratings that have had the app installed for 50+ days AND completed 30 sessions.

Taco Bell app rating prompt.

2. How you ask

Try asking people to rate a specific part of the app.

Imagine you get to the front of the Taco Bell line and the cashier taking your order asks you, ‘How would you like your burrito?’ They’ve given you no direction, no place to start in describing how you’d like your burrito. Asking a user to simply ‘Rate My App’ does the same thing. Try asking people to rate a specific experience of the app. Giving people a certain prompt or some guidance might lead to not just an increase in reviews, but also an increase in the quality of reviews.

The brand and tone of voice for the app also plays a part in how you ask. Try wording your message differently–for example, buttoned up vs. playful, serious vs. humorous–to solicit more positive responses.

3. Who you ask

Try requesting ratings from certain user segments.

This extends beyond the experiment of only prompting users who are engaged and likely to give you a good review. Just as your user base probably varies widely, your messaging should vary based on who you’re prompting — you want your message to resonate with each user. By using a mobile a/b testing platform on your app, you can deliver different messaging to different users to give them a more personalized ratings prompt. In Taco Bell’s case, maybe delivering a different prompt to burrito-orderers versus taco-orderers might lead to a higher completion rate.

¿Que quieres?

App store ratings are something that all mobile teams care about — from seasoned developers to companies submitting for their first time. Having good ratings makes it more likely that your app will get noticed and downloaded more.

What makes you likely to rate an app? Which apps prompt for ratings particularly well?

Learn more about how Optimizely works on iOS apps.