How long should you run an A/B test?

A methodology guide for CRO teams. Published 7 July 2026.

The honest answer is not a fixed number of days. Duration falls out of two things: the sample size your test needs, and how fast your traffic delivers it. Everything else, the seasonality, the cycle floor, the discipline not to stop early, is about making sure those two numbers give you a result you can trust.

Duration is sample size divided by traffic

Start from the sample size. The number of visitors you need per variant comes from four inputs: your baseline conversion rate, the minimum detectable effect you care about, your significance level (conventionally 95%), and your power (conventionally 80%). Our MDE guide covers how to choose those, and the A/B test calculator does the arithmetic.

Once you have the required sample per variant, duration is simple division. Take the total sample size across both arms and divide by the number of eligible visitors you send into the test each day. If you need 79,000 visitors in total and 4,000 qualifying visitors enter the test daily, that is about 20 days of raw collection. That figure is the floor, not the finish line, because a few adjustments still apply.

Run in whole weeks

Round that raw number up to a whole number of weeks. The reason is seasonality inside the week. Your Tuesday visitors and your Saturday visitors are often different people with different intent: weekday buyers researching for work, weekend browsers shopping at leisure, payday spikes, quiet Sunday mornings. If you stop mid-week, one variant may have soaked up an extra busy Saturday or missed a slow Monday, and that imbalance shows up as a difference that has nothing to do with your change.

Running in whole weeks guarantees every day of the cycle appears the same number of times in both variants. Twenty raw days becomes three full weeks. It costs you a couple of extra days and removes a whole class of avoidable error.

A practical floor of one to two business cycles

Even when your traffic is enormous and the maths says you could hit the sample size in two days, hold a floor of at least one full business cycle, and preferably two. For most sites a cycle is a week, so the floor is one to two weeks minimum regardless of how fast the numbers arrive.

The floor exists because a test that finishes too quickly only ever saw one narrow slice of your audience and one mood of the market. Two cycles also lets you confirm the effect holds up in the second week rather than resting on a single unusual one. If your buying cycle is longer, a considered B2B purchase over several weeks for example, stretch the floor to match it.

Watch for novelty and primacy effects early

The first days of a test can mislead in a specific way. Returning visitors notice a change simply because it is new. Some react to the novelty and engage more than they will once it becomes ordinary, which flatters the variant. Others are thrown by the unfamiliar layout, a primacy effect that penalises the variant until they adjust. Both fade as the audience gets used to the change.

Early readings are contaminated by these effects, which is another reason a two-day test is untrustworthy and a full run is not. Give the numbers time to settle into steady-state behaviour before you read anything into them.

Do not stop early, even when it looks significant

This is the discipline that ties the rest together. It is tempting to watch a running test and end it the moment it turns significant. Do not. With a fixed-horizon test, stopping at the first significant reading inflates your false-positive rate well above the nominal 5%, because every time you look you give random noise another chance to cross the line. A green result on day four of a planned three-week run is far more likely to be luck than a real win. Our guide on the peeking problem works through exactly why, and why the effects you ship on the back of early stops tend to evaporate.

The fix is to let the required sample size and the whole-week schedule set the end date, then read the result once you get there. The plan ends the test, not a gut feeling that today's number looks good.

The short version

Compute the sample size from your baseline, MDE, 95% significance, and 80% power. Divide by daily traffic to get the raw days, round up to whole weeks, and never go below a floor of one to two full business cycles. Ignore the noisy first days where novelty and primacy distort behaviour, and hold your nerve when an early reading looks significant. ABTestly supports this by showing the sample size you need and the uncertainty measure for the method you chose, and, on the fixed-horizon frequentist method, by reading Still collecting until the test has the evidence to say more. How we count and compute is public in the results methodology docs, and the SRM docs explain the chi-square check that catches a broken traffic split.

FAQ

How do I calculate how long to run an A/B test?

Work out the sample size you need per variant from your baseline conversion rate, your minimum detectable effect, 95% significance, and 80% power. Then divide the total sample size by your daily traffic into the test. That gives the raw number of days, which you round up to whole weeks.

Why should I run an A/B test in whole weeks?

Behaviour varies across the week. Weekday and weekend visitors convert differently, and payday and browsing patterns shift too. Running in whole weeks means every day of the cycle is represented equally in both variants, so a stray busy Saturday does not skew the result.

Can I stop an A/B test early if it already looks significant?

No. Stopping the moment a fixed-horizon test crosses significance inflates your false-positive rate well above 5%, because each look is another chance to cross the line by noise. Let the required sample size and your whole-week schedule set the end date instead.