Here is the number nobody selling you a testing tool leads with: to detect a 20 percent lift on a landing page that converts at 3 percent, you need about 14,000 visitors per variant, roughly 28,000 visitors total, at the standard 80 percent power and 95 percent confidence. If your page gets 5,000 visitors a month, that single test takes over five months. If it gets 1,000 a month, it takes more than two years. For most small sites, the honest answer to "how do I A/B test with low traffic" is: you mostly don't, and pretending otherwise produces random results with a scientific costume on.
That is not a reason to give up on conversion work. It is a reason to change methods. Big swings instead of button-color tweaks, because large effects need far smaller samples. Before-and-after comparisons with guardrails. Watching five real people use your page, which finds problems no test ever will. Cutting form fields and fixing speed, where the evidence is already strong enough that you do not need your own experiment.
This guide does the sample-size math in public so you can run it on your own numbers, then walks through each alternative in order of effort. It also covers the practical stuff that changed recently: what actually replaced Google Optimize for free testing after it shut down in 2023, and why INP replacing FID matters for your page speed work. Written for a founder with one landing page, real traffic numbers, and no data scientist down the hall.
How much traffic do you actually need for an A/B test?
An A/B test needs enough visitors that the difference you observe is unlikely to be luck. Two settings control this. Significance (alpha, usually 0.05) is how often you will "detect" a difference that is not real. Power (usually 0.8) is how often you will catch a real difference of the size you care about. Plug those into the standard two-proportion formula, or into Evan Miller's sample size calculator, the tool most practitioners use.
For a page converting at 3 percent, per variant you need roughly:
- To detect a 10 percent relative lift (3 percent to 3.3 percent): about 53,000 visitors per variant, over 106,000 total.
- To detect a 20 percent lift (3 percent to 3.6 percent): about 14,000 per variant, 28,000 total.
- To detect a 50 percent lift (3 percent to 4.5 percent): about 2,500 per variant, 5,000 total.
Notice the shape: halving the effect size roughly quadruples the traffic you need. This is why "we tested a new headline and got a 4 percent lift" from a site with 2,000 monthly visitors is noise, not knowledge. And no, checking the dashboard daily and stopping when it says significant does not rescue you. Peeking at results and stopping early inflates false positives badly; decide your sample size up front and let the test finish.
A workable rule: if the calculator says the test takes more than six to eight weeks at your traffic, do not run it. Tests that run for months get polluted by seasonality, campaigns, and your own impatience.
Do this now: open the calculator, enter your real baseline conversion rate and a 20 percent relative effect, and write down the visitor count. That number decides everything below.
What should you do instead when traffic is thin?
Five moves, in the order I would make them.
Test radical redesigns, not tweaks. The math above is your friend here: a change big enough to plausibly produce a 50 percent lift needs only around 2,500 visitors per variant at a 3 percent baseline. New offer, restructured page, different headline promise, different social proof. Button colors and comma placement produce effects too small for you to ever measure, so stop testing things you cannot detect.
Run pre-post comparisons with guardrails. Ship the new page to everyone, then compare two to four weeks after against the same window before. This is weaker evidence than a randomized test, so add guardrails: same traffic sources in both windows, no overlapping promotions or price changes, no seasonal spikes, and demand a difference big enough to notice without a calculator. Treat the result as evidence, not proof, and be willing to roll back.
Watch five real people use the page. Recruit five people who resemble your customers, give them a task ("find out what this costs and sign up"), and watch in silence. You will find broken assumptions in the first two sessions. This is the highest information-per-visitor method that exists, and it needs zero traffic.
Fix obvious friction without testing anything. Broken mobile layout, a form that rejects valid emails, a price hidden four clicks deep, a headline that does not say what the product is. You do not need an experiment to justify fixing a leak.
Pool everything into one primary metric. With thin traffic you cannot afford to split attention across signups, clicks, scroll depth and email captures. Pick the one money event, judge every change against it, and log every change with its date so your pre-post comparisons stay honest.
Do this now: install Microsoft Clarity, which is free with no traffic limits, and watch ten session recordings of your landing page tonight. Write down every hesitation, rage click and dead end you see.