About A/B Test Sample Size Calculator
This A/B test sample size calculator gives the number of people needed in each group to detect a chosen change in a conversion rate, with a chosen power and significance level. It is free, needs no sign-up and runs in your browser, so nothing you enter is uploaded.
It uses the standard normal-approximation formula for comparing two proportions, the same one as R’s power.prop.test. You choose the baseline rate, the smallest change that would matter (as a relative lift or in percentage points), the power and the significance level, and whether the test is one-sided or two-sided.
The page opens on a 10% baseline and a 20% relative lift, which is a move to 12%. The estimate assumes one comparison and a fixed sample size. Looking at results every day and stopping when the p-value dips below 0.05 raises the false-positive rate, so fix the sample size first and wait.
How to use A/B Test Sample Size Calculator
- Choose what to calculate: survey size, margin of error, a mean, or an A/B test.
- For a survey, set the confidence level, the margin of error in percentage points and the expected proportion. Use 50% if you are unsure.
- If your population is small, type its size to apply the finite population correction.
- For an A/B test, enter the baseline rate, the change to detect, the power and the significance level.
- Read the result, then open the working to copy the formula into your methods section.
Features
- Survey sample size for a proportion at 90%, 95% or 99% confidence.
- Margin of error for a given sample size, with finite population correction.
- Sample size to estimate a mean from the standard deviation and the margin of error.
- A/B test sample size per group from baseline rate, minimum detectable change, power and significance.
- Finite population correction when the population is known and small.
- A bell curve of the confidence level and step-by-step KaTeX working.
Sample size per group, two proportions
With p̄ the average of the two rates and q = 1 − p:
n = ( z_α √(2 p̄ q̄) + z_β √(p1 q1 + p2 q2) )² / (p1 − p2)²Worked examples
50% to 75%, power 90%, α = 0.05 two-sided gives 77 per group (76.71 unrounded)
R documentation: power.prop.test(p1 = .50, p2 = .75, power = .90) gives n = 76.7.
10% to 12%, power 80%, α = 0.05 two-sided gives 3841 per group
10% to 12%, power 80%, α = 0.05 one-sided gives 3026 per group
Privacy: how it works
- All formulas run in your own browser tab. Nothing you type is sent to Ulkaa or to any other server.
- There is no account and nothing is stored once you close the tab.
Tips
- If you do not know the expected proportion, use 50%. It gives the largest sample and so a safe margin.
- Smaller margins are expensive: moving from ±5 to ±2.5 points roughly quadruples the sample.
- For an A/B test, decide the smallest change that would matter to the business. Detecting tiny lifts takes very large samples.
- A population much larger than the sample barely changes the answer. The correction matters when the sample is more than about 5% of the population.
Frequently asked questions
Is it really free?
Yes. There is no price, no trial and no credit card. Ulkaa Tool does the work with your own device, so there is no server bill to pass on to you.
Do I need an account?
No. There is no sign-up and no login. Open the page and use it. An Ulkaa account is not needed for anything on this tool.
Does it run in my browser?
Yes. Your numbers are processed by code running in your browser tab, on your own device.
Is my data uploaded?
No. Your numbers are never sent to Ulkaa or to anyone else. You can check this yourself by watching the Network tab in your browser developer tools while you use the tool.
What is a minimum detectable effect?
The smallest change in the rate that you would want the test to be able to detect. A smaller effect needs a much larger sample, roughly four times as many people for half the effect.
Should I use a one-sided test?
Only if a change in the other direction would lead to the same decision as no change. A two-sided test is the safe default, and it needs more people.
More Sample Size Calculator pages
- Sample Size CalculatorFind how many responses a survey needs, the margin of error of a sample, or how many people an A/B test needs.
- Survey Sample Size CalculatorSet a confidence level, a margin of error and your population size to find how many responses to collect.
- Margin of Error CalculatorType the sample size and get the margin of error for a survey result.
- Sample Size for a Mean CalculatorEnter the standard deviation and the margin of error you can accept to find the number of observations.