Only 12% of experiments produce a statistically significant improvement on the primary metric, according to Optimizely's analysis of 173,000 experiments across more than 1,200 companies. That figure is the one to start from, because every other A/B testing statistic depends on how "win" is defined, and most of the ones in circulation do not say.
Below are the numbers from the platforms that hold the data, with the definition next to each, and the gaps where the vendors disagree.
How often tests win depends on what counts as winning
| Source | Dataset | Definition | Win rate |
|---|---|---|---|
| Optimizely | 173,000 experiments, 1,200+ companies, 2018 to 2026 | Primary metric improved at statistical significance | 12% |
| Optimizely | Same programme | Any conclusive result, positive or negative, in a healthy programme | 35% to 40% |
| Optimizely, as reported | Client averages | Any win across all experiments; revenue-tied experiments | About 20%; about 10% |
| VWO, as reported | 193,000 experiments, 38,000 websites, 17 industries | CTA copy tests and click-through tests that won | About 10% |
| VWO, as reported | Same | Variation beat control, no significance test | About 14% |
| Microsoft, Kohavi | Bing and other heavily optimised products | Significant positive result | 10% to 20%; about 33% for teams new to testing |
| Convert | 2025 experiments on its platform | Reached 95% confidence in either direction | 70% |
Sources: Optimizely, VWO benchmark page and secondary reports, Convert original data, Microsoft experimentation platform publications
Read together, the figures say the same thing. Somewhere between one in ten and one in five tests improves the number it was built to improve, teams new to testing win more often because the easy problems are still unsolved, and the majority of tests end without a clear answer. The 70% figure from Convert is not a contradiction: reaching confidence includes confident losses.
Lifts are smaller than the case studies suggest
Convert's distribution of lifts across completed experiments is the clearest published picture of what a win looks like.
| Reported lift | Share of completed tests |
|---|---|
| Under 10% | 39.8% |
| 10% to 19% | 20.4% |
| 20% to 29% | 9.7% |
| 30% to 49% | 14.3% |
| 50% to 99% | 8.0% |
| 100% or more | 7.8% |
Source: Convert original data, 2025 experiments
Sixty percent of completed tests moved the metric by less than 20%, and 84% by less than 50%. The 7.8% claiming a doubling or better are the ones Twyman's Law applies to: a result that looks too good is usually a tracking error, a sample-ratio mismatch or a test called too early. Optimizely's revenue figure fits the same picture: an average 0.4% incremental lift in digital revenue per revenue-focused experiment, which compounds across a programme and disappoints as a single headline.
Confidence, traffic and the tests that cannot conclude
| Measure | Share |
|---|---|
| Reached 99% or higher confidence | 48.9% |
| Reached 95% to 98% | 21.0% |
| Reached 90% to 94% | 12.1% |
| Reached 80% to 89% | 8.7% |
| Below 80% | 9.3% |
| Experiments with fewer than 1,000 visitors | About 1 in 10 |
| Experiments with 10,000 to 50,000 visitors, the most common band | 37% |
| Experiments reaching 100,000 or more visitors | About 9% |
Source: Convert original data
About one test in ten runs on fewer than 1,000 visitors, which for most conversion rates cannot reach a reliable result at all. That is the B2B problem in one row: a site with 5,000 monthly visitors and a 2% conversion rate produces 100 conversions a month, and a test needs hundreds per variation to detect a 20% change. The sections on low-traffic testing elsewhere on this site exist because of this row.
What gets tested
Convert's 2025 data shows how programmes spend their tests: 67.6% are standard A/B tests, 16.9% split-URL tests, 4.6% personalisation experiments, 4.5% A/A tests, under 1% multivariate and under 3% multi-armed bandit. VWO's benchmark reports CTA copy as the single most tested element, around 3,500 times across its set with about a 10% win rate, and click-through rate as the most tested metric at around 8,000 tests.
Optimizely's velocity figures show the spread between programmes: the median company runs 34 experiments a year, the top 10% run about 200, and the top 3% run more than 500. Its analysis also finds that experiments with four variations deliver 3.5 times the expected impact of a typical two-variant test, which argues for fewer, bigger tests on sites with limited traffic.
How few sites test at all
BuiltWith's technology data, as cited by Convert, puts about 2.2 million websites on an A/B testing or experimentation platform, against 1.1 to 1.2 billion active sites, or roughly 0.2% of the web. Adoption rises steeply with size.
| Site rank | Share using testing tools |
|---|---|
| Top 10,000 | 32% |
| Top 100,000 | 20.95% |
| Top 1 million | About 11.5% |
| All active sites | About 0.2% |
Source: BuiltWith, as cited by Convert, 2025
For a B2B marketing team this is the competitive context: most competitors are not testing, and the ones that are tend to be the largest.
Programmes have matured, and still skip the basics
Speero's 2025 Experimentation Maturity Programme Report, cited by Convert, tracks the same questions since 2021.
| Measure | 2021 | 2025 |
|---|---|---|
| Companies at strategic or transformative maturity | 35% | 54% |
| Progressive middle | 38% | 33% |
| Beginners | 9% | 2% |
| Beginner or aspiring, combined | Over 25% | 13% |
Source: Speero 2025, as cited by Convert
The maturity numbers rise while the process numbers lag. In the same report, 52% of companies have no specific process for QA-ing an experiment, 58% have no clear prioritisation framework, 10% have no dedicated experimentation staff and 25% have exactly one person, and only 26% strongly agree they have an accountable senior sponsor. Half have a centralised knowledge base. A programme that calls itself strategic and does not QA its tests is producing some of the 7.8% of results that look too good.
Convert's 2025 survey of 237 experimentation agencies adds the supply side: over half have one to five staff, 76% say AI is a key part of their workflow, and 65% report clients bringing experimentation in-house, rising to 87% among large agencies.
What this means for a B2B testing budget
Plan for one win in eight to ten tests, each moving its metric by 10% to 20%, and for most tests to end inconclusive. That arithmetic favours fewer, larger tests on the pages with the most traffic, four-variation designs where traffic allows, and a written QA step before any result is believed. It also means the first dozen tests on a site that has never been tested will win more often than the next hundred, and a programme should expect its win rate to fall as it improves. Our CRO and testing service is built around that curve; the landing page benchmarks on this site give the starting numbers to test against.
