Product analytics
Product analytics platform for funnels, retention and cohorts, with built-in experimentation and session replay.
- Build-Measure-Learn
- Cohort Analysis
- Funnel Metrics
- A/B Testing
- Pricing
- Free plan + paid
- Deployment
- Cloud
A/B testing tools: how to choose A/B testing softwareLet customer behavior settle the argument.
A/B testing tools, also called A/B testing software or experimentation platforms, run controlled experiments on websites, apps and product features. They decide which users see which version, record what each group does and report whether the difference is likely to be real.
Showing 4 of 4 tools
1 2 GrowthBook
Open sourceOpen-source A/B testing & flags
Open-source, warehouse-native experimentation and feature-flag platform with Bayesian and frequentist stats engines.
- A/B Testing
- Feature Flags
- Hypothesis testing
- Build-Measure-Learn
- Pricing
- Free plan + paid
- Deployment
- Cloud, Self-hosted
3 Enterprise experimentation
Enterprise digital experience platform offering web and feature experimentation alongside CMS and personalization.
- A/B Testing
- Multivariate Testing
- Feature Flags
- Split Testing
- Pricing
- Price on request
- Deployment
- Cloud
4 PostHog
Open sourceOpen-source product analytics suite
Open-source all-in-one product analytics with feature flags, A/B tests, session replay and surveys for fast iteration.
- Build-Measure-Learn
- A/B Testing
- Feature Flags
- Cohort Analysis
- Actionable Metrics
- Pricing
- Free plan + paid
- Deployment
- Cloud, Self-hosted
The basics
What is a/b testing tools?
A/B testing tools split users between two or more versions of a page or feature and compare how each group behaves. Many also manage feature flags and product analytics.
In lean startup terms they turn vanity metrics into actionable ones. A controlled experiment shows whether a change caused a result. A shift in the numbers alone does not show that.
The book Trustworthy Online Controlled Experiments, by Kohavi, Tang and Xu, is a common reference for running tests well.
Paper, spreadsheet or software?
Low-traffic products often cannot reach a clear result in a reasonable time. Interviews, usability tests or before-and-after comparisons may teach more.
Testing tools earn their place once you have steady traffic on the steps you want to improve. Plan the sample size before the test starts, and keep the test running for its planned length.
The routine it supports
- 1Write a hypothesis
- 2Set the sample size
- 3Release the variant behind a flag
- 4Run for the planned length
- 5Read the cohort result
- 6Ship, drop or retest
How the listed tools are priced
- Free plan + paid: 3
- Price on request: 1
How A/B testing software works
The software randomly assigns each user to a control or a variant and keeps them in that group for the length of the test. It records a chosen metric for each group, such as sign-ups or purchases, and compares them with a statistical method.
Tools differ mainly in that last step. Some use frequentist statistics with a fixed sample size, some use Bayesian methods, and some use sequential testing that allows checking results early. Ask which method a tool uses and how it guards against stopping a test too soon. The glossary entry on A/B testing gives the short definition.
Client-side vs server-side A/B testing
Client-side tools change the page in the user's browser, often through a visual editor. They suit marketing pages and copy tests, and they need little developer time, but the change can cause a brief flicker and is limited to what the browser can alter.
Server-side tools decide the variant in your own code, usually through feature flags. They suit pricing, algorithms, back-end changes and mobile apps, and they need developers to set up. Many product teams use server-side testing and keep client-side tools for marketing sites.
Free and open source A/B testing tools
Free A/B testing tools come in two forms: free plans of commercial tools, usually limited by traffic, tests or features, and open source platforms you host and run yourself. Open source options often read results from your own data warehouse, which keeps data in your hands.
Before choosing, check the cost of the data you will send, the effort to host and update the tool, and whether its statistics are documented. For a product with little traffic, customer discovery or a simple MVP may teach more than any test.
Buying guide
How to choose A/B Testing Tools
Who it's for: Product, growth and engineering teams that want decisions based on measured behavior rather than opinion.
What to look for
Trustworthy statistics
Clear handling of sample size, significance and peeking, so results can be trusted.
Feature flags
Release to a small group first, then roll out or roll back safely.
Actionable metrics
Cohorts, funnels and retention instead of totals that only go up.
Privacy and data control
Clear data ownership, consent handling and regional hosting if you need it.
Common mistakes
- Stopping tests as soon as the result looks good.
- Testing small cosmetic changes while big assumptions go untested.
- Tracking everything and deciding nothing.
Before you choose
Questions to ask vendors
- 1Which statistical approach do you use, and how do you handle early stopping?
- 2Can feature flags and experiments be managed in one place?
- 3Where is our data stored, and can we export it?
- 4How does pricing scale with traffic or events?
FAQ
A/B Testing Tools: common questions
What are actionable metrics?
Metrics that tie a specific change to a specific result, such as conversion for a cohort, as opposed to vanity metrics like total sign-ups.
How long should an A/B test run?
Until it reaches the sample size you planned before starting, and usually for at least one full business cycle such as a week.
Are feature flags only for engineers?
Engineers set them up, but product teams use them for gradual rollouts, beta programs and experiments.
How much traffic do I need for an A/B test?
It depends on your baseline conversion rate and the smallest effect you want to detect. Use a sample size calculator before the test, not after.
What is the difference between A/B and multivariate testing?
An A/B test compares whole versions. A multivariate test changes several elements at once to see which combination works, which needs much more traffic.
What is statistical significance in A/B testing?
It means the difference between versions would be unlikely if the change had no real effect. It does not tell you the size of the effect or whether it matters to the business.
What is an A/A test?
A test that shows the same version to both groups. It checks that the tool splits traffic and measures results correctly, because no real difference should appear.
What is the difference between A/B testing and split URL testing?
In an A/B test the variants usually share one page address. In a split URL test each variant lives at its own address, which suits full page redesigns.
Sources
Written by the lean-stack editors following our methodology. Updated October 3, 2026.
For vendors
Built a lean tool?
Submit it for review. Every tool is checked against our listing criteria and researched from public sources. Approved tools join their category in the next published batch.