kitrate

Home✦Tools✦A/B Test Significance Calculator

A/B test significance calculator.

Drop in the visitors and conversions for each variant. You get the conversion rates, relative lift, and the statistical confidence that your winner is real and not noise.

Your test data

Variant A (control)

Variant B (challenger)

Sample-size planner

Before you run a test, find out how much traffic you need to reliably catch a given lift at 95% confidence.

-

Result

-

Statistical confidence

Enter your numbers

-

Variant A rate

-

Variant B rate

-

Relative lift (B vs A)

-

P-value

Enter visitors and conversions to see the result.

Significance tells you how likely this difference is real rather than random chance. 95%+ is the usual bar to call a winner.

How to read this

Three numbers that decide your test

Conversion rate
conversions ÷ visitors for each variant. This is the raw performance before any statistics.
Relative lift
how much better (or worse) B does versus A in percentage terms. A "+20% lift" means B converts 20% more often than A, not 20 points more.
Confidence
from a two-proportion z-test. At 95% or higher the result is statistically significant. 90-95% is trending, so keep running. Below 90% means you don't have enough evidence yet.
Don't stop early.
Calling a test the moment it crosses 95% inflates false positives. Decide a sample size up front and let it finish.

Get the full picture

Want a CRO team to read this test and design the next one?

Drop your email and we'll send your significance results plus a short, specific note on what we'd test next to compound the win. No deck, no pitch, just the next move.

Or book a free expert review →

Calculator questions

Asked, answered.

What does statistical significance mean in an A/B test?+

Statistical significance is how confident you can be that the difference between your two variants is real and not just random chance. This calculator reports it as a confidence percentage from a two-proportion z-test. The industry convention is 95%. At or above that, the result is treated as significant and the winning variant is unlikely to be a fluke.

How many visitors do I need before I can trust the result?+

There's no fixed number. It depends on your baseline conversion rate and the size of the lift you're trying to detect. Smaller lifts need far more traffic. As a rule of thumb, keep the test running until confidence reaches 95% and each variant has at least a few hundred conversions, and never stop the moment it crosses the line, since early peeks inflate false positives.

Why is my huge lift still not statistically significant?+

A big relative lift on a small sample is fragile. A handful of extra conversions can swing it. Significance weighs the size of the difference against how much data backs it up, so a 40% lift on 50 visitors per variant can easily land below 95% confidence. Keep the test running to gather more data before you call it.