Experimentation
Experimentation is in beta on Enterprise plans. Get in touch to join.
Flagsmith Experimentation lets you run A/B tests end to end on the platform: serve variations of a feature with a multivariate flag, collect events from your application into a managed data warehouse, and read the results with a built-in Bayesian statistics engine.
Experiments are scoped to an environment.
Set up and run an experiment
- Connect a warehouse: enable the managed Flagsmith warehouse for your environment. Done once, in a few clicks.
- Create metrics: define the outcomes you measure, computed from the events your application sends.
- Create an experiment: pick a multivariate flag, decide how much traffic enters the test and how it is split, and choose a primary metric.
- Run the experiment: instrument your application to record exposures and conversion events with the Flagsmith SDKs.
- Analyse the results: read lift, credible intervals and win probability on the experiment's results page, then roll out the winner.
Prefer to see it end to end? Follow the PayPal button example.
Unfamiliar with a term? See the Statistics Glossary.
Event collection is currently available in the JavaScript and Python SDKs, with more to follow. Check Run an Experiment for instrumentation details.
A/B testing without the Experimentation feature
You can also run A/B tests with multivariate flags and your own analytics platform. See A/B Testing with Integrations. The self-managed approach remains fully supported and is not going away.