How's about Git?

Today we’re excited to announce that we’re exploring a new type of backend for Flipt!

The team has been discussing and exploring ways Flipt could be better positioned to support GitOps-style workflows. For those unfamiliar with GitOps practices, we recommend the WeaveWorks Guide To GitOps.

Our new filesystem backends proposal adds two ways to configure Flipt to serve flag state content encoded as yaml stored in files, local and git.

We’re keen to understand if this idea resonates with you and your use case, this will help us shape the implementation, which is currently underway.

Problem

We feel that many feature flags offerings (including Flipt today) work at odds with GitOps practices. GitOps prescribes that you store all your configuration in one or more Git repositories. This gives your entire system the benefits of version control. Feature flag configuration is just like any other configuration in that it adjusts the behaviour of your applications at runtime. However, we find that it is rarely stored and tracked within Git itself.

If you’re using GitOps practices for your wider configuration needs, but your feature flag solution is completely isolated, then you’re subverting the benefits and traceability of Git. When the alerts start firing you'll have to correlate your configuration changes in Git with the auditing features of your flagging solution.

Alternatively, some existing feature flag solutions support GitOps by bolting a declarative syntax onto their existing imperative API. For example, some offerings provide a Terraform provider which is an entirely legitimate way to create a GitOps-style contribution experience. In fact, we’re still considering having one for Flipt and our existing relational backends. However, this approach does have drawbacks, primarily, there are now two sources of truth.

Reconciling from your Git repository into your online feature flag service also has to be performed. This process is often performed during some kind of continuous delivery step (e.g., via a GitHub action), which likely pushes changes into your feature flag system. You may have to manually intervene when this process fails for intermittent reasons. Your environment must also have credentials and access to your application repository and your feature flag solution. Ideally, your feature flag solution provides tooling to ease the extra work required to set up this CI/CD process and make it resilient.

In this scenario, given you’re using a self-hosted solution (such as Flipt), you now have the following set of applications to run and configurations to manage:

• The feature flag service itself (e.g. the flipt binary)
• Backing database for storing flag state (e.g. SQLite, PostgreSQL etc)
• Flag state configuration (e.g. in Terraform HCL or via some other process such as flipt import)
• CI processes for reconciliation (plus monitoring)
• For solutions such as Terraform, storage of reconciliation state (tfstate files)

This is all achievable, but care must be taken when building the reconciliation process to ensure a predictable end state.

Ultimately, feature flag offerings have to build a host of features you’re already getting from your SCM (e.g. GitHub or Gitlab). Version control, peer review, access control and backup to name a few.

How We Intend to Serve GitOps

We intend to have Flipt pull feature flag state directly from Git at runtime.

We want Flipt to be able to drop its dependency on a database for flag state and go directly to the source. This removes the need for a reconciliation process altogether. By adopting a pull model, we embrace eventual consistency and allow Flipt to recover automatically during transitive errors without intervention.

You'll simply be able to configure Flipt with access to your repository and with some additional configuration, you will be able to control how and where to locate your flag state files. Your repository could be used solely for Flipt, it could be your existing configuration repository, or it could be stored within a monorepo.

Flipt’s evaluation and read APIs won't change, meaning your existing interactions with the Flipt SDKs can remain the same. The existing flipt export command produces configuration files valid for committing straight to your repositories for ease of transition.

Flipt will be configured to serve a particular branch for a target repository. This branch doesn’t always have to be main, meaning you can deploy Flipt in multiple ways to support validating flag state changes in arbitrary Git branches or tags.

You'll also be able to run Flipt over a local directory. In this configuration, Flipt will simply serve your flag configuration without the need to go through Git. Updates to the local directory will update Flipt automatically. There is no need to access an external running instance of Flipt to replicate or alter your production feature flag state in your local environment. Simply check out your repository and run flipt to begin experimenting.

Flipt in CI

Alongside building a new core backend for Flipt, we’re also developing a suite of tools for validating your use of Flipt during your existing continuous integration workflows.

Before Flipt can serve your configuration files, it is important to validate they are fit for consumption. Failure to do so could lead to runtime errors or unexpected behaviour. flipt validate is a new subcommand currently being added to Flipt. This will be useful for local validation or running directly in a CI pipeline.

We’re also developing a GitHub action which simplifies running flipt validate which will make it trivial to add alongside your existing GitHub workflows.

Just the Beginning

We’re just getting started turning our proof-of-concept into something ready for production Flipt usage. While we have a good feeling that the experience will resonate with users, there are still a number of interesting technical challenges we expect to have along the way. This is why we wanted to get this plan out early in order to hear your feedback.

Help come shape the future of this solution on GitHub, or in our Discord.