On a page with 10,000 visitors a month and a 2% form conversion rate, a four-week A/B test can only detect a lift of about 45% or more. Convert's 2025 platform data shows that 84% of completed tests move their metric by less than 50%. Put those two numbers side by side and the problem with hiring an A/B testing agency for a typical B2B site is clear: most of the changes the agency would test are too small for your traffic to see.
This guide is for B2B marketing leads deciding whether to hire an A/B testing company, and what to ask for if the answer is "not yet". The pricing and entry requirements come from agency pages we opened on 26 October 2026; the detection limits are our own calculation, with the raw file in our research folder.
What an A/B testing agency actually sells
An A/B testing agency, often called a CRO or experimentation agency, sells a programme: research to find what is wrong, hypotheses, design and build of variations, QA, the test itself, and analysis. The test is the visible part. On a site with plenty of traffic it is also the part that turns opinions into evidence.
Convert's 2025 agency report found 237 experimentation agencies worldwide. Over half have one to five people, nearly a third run ten or fewer experiments a month across all clients, and about 60% manage two or fewer tests per person per month. That matters when you read a proposal: a small agency promising four tests a month on your site is promising a large share of its capacity, and the tests will only be as good as the research behind them.
The same report says 65% of agencies see clients bringing experimentation in-house, rising to 87% among large agencies. Many agencies are moving towards programme design and training rather than running tests for you. For a low-traffic B2B site, that shift happens to fit: the useful work is often research and decision-making, not test volume.
Why low traffic decides what can be tested
A test needs enough conversions in each version to separate a real change from noise. The smaller the real change, the more conversions you need. We calculated the smallest relative lift a two-version test can reliably detect, at 95% confidence and 80% power, for different traffic levels and two kinds of metric: a 2% form conversion and a 10% click-through to the next step.
| Monthly page visitors | 2% form rate, 4 weeks | 2% form rate, 8 weeks | 10% click rate, 4 weeks | 10% click rate, 8 weeks |
|---|---|---|---|---|
| 2,000 | 113% | 75% | 43% | 29% |
| 5,000 | 66% | 45% | 26% | 18% |
| 10,000 | 45% | 31% | 18% | 13% |
| 25,000 | 28% | 19% | 11% | 8% |
| 50,000 | 19% | 13% | 8% | 6% |
| 100,000 | 13% | 9% | 6% | 4% |
Source: Octo calculation, standard two-proportion sample size formula, October 2026; raw file research/OB-0109-minimum-detectable-lift-2026-10-26.csv
Now compare the table with what tests actually produce. On Convert's 2025 data, 39.8% of completed tests lifted their metric by under 10%, 60% by under 20% and 84% by under 50%. Convert also notes that the 7.8% showing a lift of 100% or more are usually errors, an effect known as Twyman's Law. A B2B page with 5,000 visitors a month and a 2% demo request rate can only detect a lift of 66% in four weeks. That is a band where few real effects live, and where false ones collect.
Other published figures say the same. Convert's own worked example needs 174,369 visitors per variation to detect a 5% relative lift on a 3.5% baseline. Kameleoon, in a 2015 article, put a 5% gain on a 2% conversion rate at almost 310,000 visits per variation. Convert's platform data shows the most common test size is 10,000 to 50,000 visitors, with about one test in ten running on fewer than 1,000.
Our earlier post, A/B Testing Tools for Low-Traffic B2B Sites, covers the statistics engines and why a Bayesian dashboard does not shrink the sample you need. This guide is about the buying decision.
What can be tested on a low-traffic B2B site
Low traffic does not rule out testing. It rules out small changes measured on rare conversions. Three kinds of test still work.
Big swings on the busiest page. A new hero message, a different offer (a demo against a self-serve trial, say) or a much shorter form can move a metric by 30% or more. Those are the changes worth testing when traffic is thin. Button colour and headline tweaks are not.
Earlier metrics with higher rates. A 10% click to the pricing or demo page reaches a detectable result far sooner than a 2% form submit, as the table shows. The trade-off is that a click is not a lead. Use it to choose between messages, then watch the downstream rate over the following months.
Paid traffic you control. If you run paid search or LinkedIn ads, you can buy the sample. Sending a fixed budget to two landing pages for four weeks is a test you pay for in media rather than waiting for in organic traffic.
What does not work: testing several elements at once on a page with a few thousand visitors, testing pages deep in the site, and checking results daily and stopping when one version pulls ahead. Evan Miller showed in 2010 that stopping at the first significant reading, with up to 150 checks, produces a false winner 26.1% of the time when the two versions are identical.
What A/B testing agencies charge and who they take on
Few agencies publish prices. The ones that do, plus the commonly quoted tiers, give a usable range.
| Agency or source | What is published | Entry signal | Type of source |
|---|---|---|---|
| Conversion Fanatics | Starter $4,995 a month, Standard $6,500, Pro $9,750, Fanatic $13,650 (monthly billing) | Starter covers 2 to 10 daily conversions and one test a month; testing tool not included | Agency pricing page |
| Speero | Budget options on its enquiry form start at $5,000+, then $10,000+ and $20,000+ a month | Names companies with $50 million+ in annual revenue as its ideal client | Agency homepage |
| Speero, as reported | Programmes from $10,000 a month on a 12-month term | At least 100,000 unique monthly visitors | Passionates, a competing agency, December 2025 |
| Invesp tiers, as reported | Top tier from $10,000, typically about $16,000, up to $30,000+ a month; mid tier $6,000 to $8,000; newer agencies $2,000 to $5,000 | Seven years or more of CRO experience for the top tier | Convert, updated April 2026 |
| Convert summary | Audits $2,500 to $4,500 one-off | None stated | Convert, updated April 2026 |
Sources: Conversion Fanatics and Speero pages read 2026-10-26; Passionates 2025; Convert 2026
The Conversion Fanatics table is the most useful because it ties price to traffic. Its Starter plan is for sites with 2 to 10 conversions a day, which is 60 to 300 a month, and includes one test a month. On a B2B site with 60 demo requests a month, one test a month at about $5,000 is roughly $60,000 a year for twelve tests. At a 12% win rate, Optimizely's figure, that buys one or two significant wins a year, and the table above suggests most tests at that volume will not reach a conclusion at all.
The 100,000 visitor minimum attributed to Speero comes second-hand, from a competing agency's blog, and Speero's own page states a revenue threshold instead. We could not find the visitor figure on a Speero page, so treat it as indicative.
What to buy instead when traffic is below the line
If the table says your traffic cannot detect the changes you would make, the budget is better spent on methods that do not need traffic. Each of these answers a narrower question than an A/B test, and each is cheaper.
| Method | What it tells you | Traffic needed | Published cost signal |
|---|---|---|---|
| Moderated usability tests, five people | Where buyers get stuck or misread the page | None | Nielsen Norman Group: five users find about 85% of problems |
| Message testing with a B2B panel | Which headline and value proposition your buyers understand and prefer | None | Wynter Pro $20,000 a year for 20,000 credits; pay-as-you-go about 1.5 times more per test |
| Session recordings and heatmaps | Where visitors scroll, click and abandon | Any | Microsoft Clarity: free, no traffic limits, no sampling |
| Sales call and win-loss review | The objections the page never answers | None | Internal time only |
| Paid-traffic split | Which landing page converts paid visitors | Bought | Media spend |
Sources: Nielsen Norman Group 2000; Wynter and Microsoft pages read 2026-10-26
Nielsen Norman Group's argument for small usability studies is that the budget is better spent on three rounds of five users than one round of fifteen, fixing problems between rounds. The second round finds most of the remaining 15% of the original problems. For a B2B site with a long sales cycle, that loop of watch, fix and watch again moves faster than any A/B test could.
Message testing tools such as Wynter recruit verified B2B panels and return results in 12 to 48 hours, according to Wynter's pricing page. They test comprehension and preference, not behaviour, so they tell you which message buyers say is clearer, not which one converts. That is still the right first question on a site that cannot run a conclusive test.
How to brief an A/B testing agency so the money is not wasted
If your traffic is above the line, or you plan to buy traffic, an agency can earn its fee. The brief should make the arithmetic visible from the start.
Give them the numbers. Monthly visitors to each candidate page, the conversion rate of each metric, and monthly qualified leads. An agency that does not ask for these before proposing a test count is selling volume, not evidence.
Ask for a detection table. The smallest lift each test can detect in four and eight weeks, per page. If the agency's first tests sit below that line, ask why.
Ask how they define a win. Optimizely's 12% is for statistically significant improvement on the primary metric. A proposal quoting a 30% or 40% win rate is usually counting something else.
Ask for research before tests. On a low-traffic site, the first month should produce interviews, usability findings and a ranked list of problems. The tests come after.
Ask who owns the testing tool. Conversion Fanatics, for one, excludes the platform subscription from its plans. Clarify whose account it is and who keeps the data when the contract ends.
If we were scoping this for a B2B client below the line, we would start with research and a redesigned hero and form on the busiest page, run usability rounds before and after, and only propose A/B tests once paid traffic or growth made the table work. That is how we would approach it; we quote per project.
What this means for hiring an A/B testing company
Work out the smallest lift your traffic can detect before you speak to any A/B testing agency. If it is under about 20% on the metric you care about, a testing programme can pay off, and the published prices give you a range to negotiate against. If it is above 30%, a testing retainer will mostly buy inconclusive results; spend the first quarter on usability rounds, message testing and fixing what they find, then revisit testing when traffic grows. Our CRO and website service page explains how we scope that kind of work.
