---
title: "A/B Test Significance Calculator: Free Tool | AI Digital"
description: "Free A/B test significance calculator: enter visitors and conversions to see conversion rates, uplift, confidence and the sample size you need."
---

[Skip to content](https://aidigital.org/ab-test-calculator#main-content)

[![](https://aidigital.org/hs-fs/hubfs/site/brand/ai-digital-winner.png?width=1068&height=400&name=ai-digital-winner.png) AI Digital](https://aidigital.org/?hsLang=en)

[About](https://aidigital.org/about?hsLang=en) [Services](https://aidigital.org/services?hsLang=en) [Pricing](https://aidigital.org/pricing?hsLang=en) [Testimonials](https://aidigital.org/testimonials?hsLang=en) [Contact](https://aidigital.org/contact?hsLang=en) [Book a consultation](https://meetings.hubspot.com/dalibor-selucky-tc5?uuid=8928dc70-03c9-49ba-a97f-ce1f3af89474)

Free tool

# Did your A/B test *really win?*

Check whether the difference between two versions of a page, email or form is real or just chance, or plan how many visitors your next test needs. Nothing you enter leaves your browser.

On this page

A B

Calculator

## Check a result *or plan a test.*

Check a result

Plan a test

AVersion A (original)

Visitors Conversions 

BVersion B (variation)

Visitors Conversions

Confidence level 90% 95% (recommended) 99%

The numbers shown are an example. Replace them with your own; results update as you type.

 Copy result

Two-sided z-test for two proportions. Nothing you enter leaves your browser.

1Your page today

Conversion rate (%) Visitors per day 

2What you want to detect

Smallest uplift (%) Versions, incl. original 2 (A and B) 3 (A, B and C) 4 (A, B, C and D) 5 (A, B, C, D and E) 

3How sure you want to be

Confidence level 90% 95% 99% Power 80% 90%

The uplift is relative: 10% on a 3% conversion rate means 3.3%. Count visitors to the tested page only, across all versions. 95% confidence and 80% power are the usual standard.

 Copy plan

Two-sided test for two proportions, equal split between versions. Nothing you enter leaves your browser.

Reading the result

## Every result has *a next step.*

Don't expect every test to win. At Microsoft, about a third of experiments improve the result, a third make no difference and a third make it worse. At Google and Bing, only about 10% to 20% produce a positive result.

How experiments at Microsoft turn out

1 in 3 win

- Improved the result**1/3**
- No difference**1/3**
- Made it worse**1/3**

| Improved the result | 1/3 |
| --- | --- |
| No difference | 1/3 |
| Made it worse | 1/3 |

Even at Microsoft, only about one test in three wins.

Harvard Business Review, 2017

1. B wins
   
   ### Make B the new default.
   
   Write down what changed and why you think it worked, so the next test builds on it.
2. A wins
   
   ### Keep the original.
   
   The test still taught you something: note why the variation may have lost, and test a different idea.
3. Not significant yet
   
   ### Check the visitor count.
   
   If you haven't reached the planned number of visitors, keep the test running. If you have, the change made less difference than you hoped. Treat the versions as equal and test a bolder idea.
4. Uneven split
   
   ### Fix the setup first.
   
   When one version gets clearly more visitors than planned, the result can't be trusted, whatever it says. Check redirects, bot filtering and tracking, then restart the test.

Before you test

## Habits that make *results trustworthy.*

- Change one thing at a time, so you know what made the difference.
- Decide how many visitors you need before the test starts.
- Run whole weeks, so weekday and weekend visitors are both included.
- Split traffic randomly and keep the split the same for the whole test.
- Don't stop the moment a winner appears. Wait for the planned sample.
- Check the winner by device and traffic source before rolling it out.

Planning

## How many visitors *do you need?*

Two numbers decide it: your current conversion rate, and the smallest lift that would make you switch versions. The smaller the lift you want to detect, the more visitors you need. Halving it takes roughly four times the traffic. Divide the total by your daily visitors, then round up to whole weeks.

Low traffic? Test bigger changes, such as a new offer or a shorter form, rather than small tweaks to button color or wording. Big changes show up with fewer visitors.

Visitors needed per version at a 3% conversion rate

- 40% lift **3,350**
- 20% lift **13,051**
- 10% lift **51,487**
- 5% lift **204,494**

| 40% lift | 3,350 |
| --- | --- |
| 20% lift | 13,051 |
| 10% lift | 51,487 |
| 5% lift | 204,494 |

Halve the lift you want to detect and you need about four times the visitors.

Calculated with this calculator (95% confidence, 80% power)

[Plan your test](https://aidigital.org/ab-test-calculator#plan)

Method

## How the *calculator works.*

1. 01
   
   ### Conversion
   
   The action the page is meant to produce: a form submission, a booked call, a purchase or a click on the main button. Count the same action for both versions, over the same dates.
2. 02
   
   ### p-value and confidence
   
   The p-value is how often a gap this large would appear by chance if both versions performed the same. At 95% confidence, a result is significant when the p-value is below 0.05. "Confidence" in the result is 1 minus the p-value. Use 99% when a wrong call would be costly.
3. 03
   
   ### Likely range
   
   The range under the uplift is a confidence interval: where B's true uplift over A probably lies. A range that crosses zero means the two versions may still perform the same.
4. 04
   
   ### Chance B is better
   
   A Bayesian reading of the same numbers: the probability that B's true conversion rate is higher than A's, starting from no prior assumption. It is easy to read, but it doesn't replace the planned visitor count.
5. 05
   
   ### Power
   
   How likely a test of this size is to detect a real difference as big as the one measured. Plan for 80% or more. Below that, a result that isn't significant says little: the test may simply be too small.
6. 06
   
   ### Visitor split check
   
   With a planned 50/50 split, both versions should get about the same number of visitors. A chi-square test flags the split when a gap that big would happen by chance less than once in 100 tests (p \< 0.01). That usually points to a setup fault, and the result shouldn't be trusted until it's fixed.
7. 07
   
   ### Planning a test
   
   Visitors per version come from the standard two-sided sample size formula for two proportions, at the confidence and power you choose. With more than one variation, the confidence is split between the comparisons (Bonferroni correction), so each extra version needs more visitors.

**12%** more revenue from one Bing test of how ad headlines were displayed

Need help testing?

## We plan, build and read *your page tests.*

Small changes can pay off: that one test was worth more than $100 million a year in the US alone. We run A/B tests with you, from landing pages and ad campaigns to HubSpot forms and calls to action. Where the platform allows it, we use adaptive testing: more traffic goes to the better version while the test is still running.

[Book a consultation](https://meetings.hubspot.com/dalibor-selucky-tc5?uuid=8928dc70-03c9-49ba-a97f-ce1f3af89474)

*·*[Landing pages and websites](https://aidigital.org/services/web-development?hsLang=en)*·*[A/B tests in paid campaigns](https://aidigital.org/services/paid-campaigns?hsLang=en)*·*[HubSpot forms and calls to action](https://aidigital.org/services/hubspot?hsLang=en)

[![](https://aidigital.org/hs-fs/hubfs/site/brand/ai-digital-winner.png?width=1068&height=400&name=ai-digital-winner.png) AI Digital](https://aidigital.org/?hsLang=en)

AI Digital is a HubSpot, SEO, web and AI automation agency. We connect your CRM, website and marketing into one system, for startups and global brands in 16 countries.

## Services

- [Strategy & Consulting](https://aidigital.org/services/strategy?hsLang=en)
- [Inbound Marketing](https://aidigital.org/services/inbound-marketing?hsLang=en)
- [SEO & Content](https://aidigital.org/services/seo-content?hsLang=en)
- [PPC](https://aidigital.org/services/paid-campaigns?hsLang=en)
- [Social Media](https://aidigital.org/services/social-media?hsLang=en)
- [HubSpot CRM](https://aidigital.org/services/hubspot?hsLang=en)
- [AI & Automation](https://aidigital.org/services/ai-automation?hsLang=en)
- [Website Design](https://aidigital.org/services/website-design?hsLang=en)
- [Website Development](https://aidigital.org/services/web-development?hsLang=en)
- [Custom Apps](https://aidigital.org/services/custom-applications?hsLang=en)

## Company

- [About](https://aidigital.org/about?hsLang=en)
- [Pricing](https://aidigital.org/pricing?hsLang=en)
- [Industries](https://aidigital.org/industries?hsLang=en)
- [Testimonials](https://aidigital.org/testimonials?hsLang=en)
- [Contact](https://aidigital.org/contact?hsLang=en)

## Resources

- [Assessments](https://aidigital.org/assessments?hsLang=en)
- [HubSpot Audit](https://aidigital.org/hubspot-audit?hsLang=en)
- [HubSpot Onboarding](https://aidigital.org/hubspot-onboarding?hsLang=en)
- [Design Gallery](https://aidigital.org/services/website-design-gallery?hsLang=en)
- [A/B Test Calculator](https://aidigital.org/ab-test-calculator?hsLang=en)
- [Knowledge Hub](https://aidigital.org/blog?hsLang=en)

© 2026 AI Digital, brand by Tranquila Casa, VAT CZ04976207. All rights reserved.

[Legal notice](https://aidigital.org/legal#notice) [Privacy policy](https://aidigital.org/legal#privacy) [Cookie policy](https://aidigital.org/legal#cookies)