kitrate

Home✦Tools✦A/B Test Significance Calculator

A/B test significance calculator.

Drop in the visitors and conversions for each variant and get the conversion rates, relative lift and the statistical confidence that your winner is real — not just noise.

Your test data

Variant A — control

Variant B — challenger

Sample-size planner

Before you run a test, find out how much traffic you need to reliably catch a given lift at 95% confidence.

—

Result

—

Statistical confidence

Enter your numbers

—

Variant A rate

—

Variant B rate

—

Relative lift (B vs A)

—

P-value

Enter visitors and conversions to see the result.

Significance tells you how likely this difference is real rather than random chance — 95%+ is the usual bar to call a winner.

How to read this

Three numbers that decide your test

Conversion rate
conversions ÷ visitors for each variant. This is the raw performance before any statistics.
Relative lift
how much better (or worse) B does versus A in percentage terms. A "+20% lift" means B converts 20% more often than A, not 20 points more.
Confidence
from a two-proportion z-test. At 95% or higher the result is statistically significant; 90–95% is trending — keep running; below 90% means you don't have enough evidence yet.
Don't stop early.
Calling a test the moment it crosses 95% inflates false positives. Decide a sample size up front and let it finish.

Get the full picture

Want this test read by a CRO team — and the next one designed?

Drop your email and we'll send your significance results plus a short, specific note on what we'd test next to compound the win. No deck, no pitch — just the next move.

Or book a free expert review →

Calculator questions

Asked, answered.

What does statistical significance mean in an A/B test?+

Statistical significance is how confident you can be that the difference between your two variants is real and not just random chance. This calculator reports it as a confidence percentage from a two-proportion z-test. The industry convention is 95% — at or above that, the result is treated as significant and the winning variant is unlikely to be a fluke.

How many visitors do I need before I can trust the result?+

There's no fixed number — it depends on your baseline conversion rate and the size of the lift you're trying to detect. Smaller lifts need far more traffic. As a rule of thumb, keep the test running until confidence reaches 95% and each variant has at least a few hundred conversions, and never stop the moment it crosses the line, since early peeks inflate false positives.

Why is my huge lift still not statistically significant?+

A big relative lift on a small sample is fragile — a handful of extra conversions can swing it. Significance weighs the size of the difference against how much data backs it up, so a 40% lift on 50 visitors per variant can easily land below 95% confidence. Keep the test running to gather more data before you call it.