maratoolHow to check A/B test statistical significance
Your variant converted better — but is it real or random noise? Enter the numbers and get a p-value in seconds.
You ran an A/B test. The variant beat the control. Before you ship it to 100% of traffic, you need to know whether the difference is statistically significant or just luck. The A/B Test Significance Calculator runs a two-proportion z-test on your data and tells you.
How it works
- Enter control (A) data — total visitors and how many converted.
- Enter variant (B) data — same fields for the test group.
- Pick a confidence level — 95% is standard. The tool shows conversion rates, relative lift, p-value, and whether the result is significant.
What the numbers mean
- Conversion rate — conversions ÷ visitors for each group.
- Lift — how much better (or worse) the variant is relative to control. +20% lift means the variant converts 20% more.
- p-value — the probability you'd see this difference (or larger) if there were actually no real effect. Lower is better.
- Significant? — at 95% confidence, p < 0.05 means yes. The variant likely has a real effect.
Common mistakes
Stopping early. If you peek at results every hour and stop when p < 0.05, your false-positive rate is much higher than 5%. Decide your sample size upfront or use sequential testing methods.
Too little traffic. A 0.5% absolute lift on 200 visitors per group will almost never reach significance. You need enough data for the signal to outweigh noise.
Track where traffic came from
Pair your tests with UTM-tagged URLs from the UTM Builder so analytics attributes conversions to the right campaign.