Zway.ai
Book a demo

Free tool

How much traffic the test actually needs

This calculates how many visitors each variant needs to detect a given relative lift at ninety five percent confidence and eighty percent power, then converts that into weeks at your traffic. For most early stage sites the honest answer is that the test would take longer than the company has.

Last reviewed 27 August 2026

%
%

A 20 percent relative lift on a 3 percent rate means moving to 3.6 percent.

%

Result

Visitors needed per variant

13,896

To detect 3.00% moving to 3.60%

Total visitors needed

27,792

2 variants including the control

Time to complete

6.9 weeks

At 4,000 visitors a week entering the test

Bigger changes are cheaper to prove

Bigger changes are cheaper to prove
Lift to detectTarget ratePer variantTime
10% relative lift3.30%53,14826.6 weeks
20% relative lift3.60%13,8966.9 weeks
30% relative lift3.90%6,4453.2 weeks
50% relative lift4.50%2,5121.3 weeks

At 4,000 visitors a week this finishes in 6.9 weeks, which is short enough that the site is unlikely to change underneath it. Decide now that you will not look at the result until it completes.

How this works

What the numbers mean.

  • 01The sample size uses the standard two proportion formula: the squared sum of the two z values, multiplied by the combined variance of both rates, divided by the squared absolute difference between them.
  • 02The z values are fixed at 1.96 for ninety five percent confidence and 0.84 for eighty percent power. Power is the chance of detecting a real effect of the stated size, and eighty percent means you would miss it one time in five.
  • 03Time is total sample divided by the traffic actually entering the test, so the variant count and the traffic share both extend it. Adding a third variant is a fifty percent increase in how long you wait.

Assumptions and limits

  • This is a fixed horizon calculation. It assumes you set the sample size first and read the result once, which is the only way the confidence level means what it says.
  • It does not apply a correction for multiple variants. Comparing several variants against one control raises the false positive rate, so a stricter threshold is needed than the one priced in here.
  • Small differences are enormously expensive to prove. Halving the lift you want to detect roughly quadruples the traffic you need, which is the arithmetic behind most abandoned testing programmes.

Questions about this tool

Why does a smaller lift need so much more traffic?
Because the required sample scales with the inverse square of the difference. Going from detecting a twenty percent lift to a ten percent lift is not twice the traffic, it is roughly four times. This is the single most useful thing to know before planning a testing programme, because it turns most small-site test ideas into non-starters immediately.
What is power, and why 80 percent?
Power is the probability that the test detects a real effect of the size you specified. Eighty percent is the common convention, and it means one real winner in five goes undetected. Raising power to ninety percent is defensible and costs roughly a third more traffic.
My site is too small to test. What should I do instead?
Test further up the funnel where volumes are larger, such as click rate rather than purchase rate, and accept that it is a proxy. Otherwise make changes large enough that a test is not needed to see the difference, and rely on qualitative evidence: session recordings, support tickets, and talking to people who did not convert.
Can I stop early if the result looks clear?
Not with this method. A fixed horizon test assumes one look at one predetermined sample size, and stopping early because the numbers look good is exactly how false positives get shipped. If you need to monitor as you go, use a sequential or Bayesian approach designed for it.

Related

This tool is free and there is nothing to sign up for. If you would rather have the work done than calculate it, that is what Zway does.

All toolsBook a demo