Implementing low-latency and dynamic feature flags

When the logic of your codebase depends on the return value of a feature flag, say a function isFeatureEnabled, you’ll want to make sure that isFeatureEnabled returns its decision as fast as possible. 

In the worst case, if you rely on an external database to store whether the feature is enabled, you risk increasing the latency of your application by requiring a roundtrip external network request across the internet, even when the new feature is not enabled. 

In the best-case performance, the isFeatureEnabled function is hardcoded to true or false either as a variable or environment variable, but then you lose the ability to dynamically change the value of the feature flag without code deploys or reconfiguring your application. 

In my experience as an engineer and an engineering manager, I’ve seen multiple methods for achieving the balance between fast and dynamic feature flags to suit the needs and capabilities of different applications. Below, I’ll walk through an example architecture from my free e-book Ship Confidently with Progressive Delivery and Experimentation that strikes this balance well and is suitable on many platforms.

An architecture that balances fast and dynamic feature flags will:

• Fetch the latest feature flag configuration when the application starts up
• Cache feature flag configuration in-memory and on-disk so that decisions can be made with low latency and subsequent application starts are fast
• Listen for updates to the feature flag configuration so that updates are pushed to the application in as real time as possible
• Poll for updates to the feature flag configuration at regular intervals, so if a push fails, the application is still guaranteed to have the latest feature configuration within some well-defined interval

Fetching and caching the configuration at startup ensures you can evaluate an API like isFeatureEnabled without having to make any slow blocking network requests for the state of a feature flag while a user is using your application. Storing the configuration on-disk makes sure subsequent application starts don’t have to wait for the initial fetch to complete.

Listening and polling for updates ensures that your feature flags stay dynamic even if your application does not restart often.

The below diagram shows how this architecture plays out in a timeline between an admin panel controlling the state of your feature flags and the client apps or application servers that rely on that feature flag information. Read the diagram as a timeline from top to bottom:

Example Feature Flag Service Architecture

As an example, a mobile application may initially try to (1) load feature flag configuration if it is already cached on the phone. Regardless of whether a feature flag configuration was there, the phone will (2) fetch the latest feature flag configuration as the app is started, then (3) cache this latest feature flag configuration in-memory on the phone. The mobile app can then (4) poll on regular, 10-minute intervals to ensure that the feature flags are still up to date as well as (5) listen to more real-time push notifications of any changes in feature flags. Anytime the phone receives updates of feature flag changes, it can again (3) cache those changes in-memory and on-disk for subsequent (1) startups to be faster.

With this setup, the performance of application startup is minimized, feature flag decisions are made quickly based on in-memory data and feature flags still stay up-to-date with any changes made to their configuration either because of a real-time push or regular poll.

For an application server example, each step is the same except instead of using push notifications, you would use (5) webhooks or server-side events to listen for any updates in real-time.

For a frontend application example, each step is the same except instead of using push notifications or webhooks, you might use (5) web sockets or server-sent events to listen for any updates in real-time.

Developers might find this architecture appealing because it’s relatively simple to set up, results in fast and dynamic feature flags, and is easy to tweak to fit a large variety of applications. If you don’t want to spend the time building this type of infrastructure, though, definitely checkout Optimizely’s free feature flag offering, which packages a similar architecture as a free service!

Let me know what you think!

How do you set up feature flag infrastructure to balance fast and dynamic decisions? Message me in our Slack community or find me on Twitter at @asametrical.

This is part of a series of best practices to help your company successfully implement progressive delivery and experimentation to ship faster with confidence.

If you like this content, check out my free e-book: Ship Confidently with Progressive Delivery and Experimentation which offers more best practices from just getting started to scaling these technologies organization-wide.

And if you are looking for a platform to get started, check out Optimizely’s free offering. Thanks!