A/B test results summary
Showing results per visitor.
If the same visitor has passed through an experiment multiple times,
only one trial is counted and a maximum of one goal conversion.
Show unfiltered results.
Showing all results.
If the same visitor has passed through an experiment multiple times,
all trials and conversions are shown.
Show per visitor results.
Experiment: $exp.name
| Variant |
Number of tests |
Goal: $goal |
| conversions |
rate |
z-score |
confidence |
probability of beating the control |
|
$variant
control
|
${vdata['trials']} |
no data yet |
${goaldata['conversions']} |
${'{0:.1%}'.format(goaldata['rate'])} |
${'{0:.3g}'.format(goaldata['z'])} |
${'{0:.1%}'.format(goaldata['confidence'])} |
${'{0:.1%}'.format(goaldata['p_beats_control'])} |
Insufficient data
For the $goal goal, the $variant variant resulted in
${'{0:.2%}'.format(difference - 1)} more conversions than the $control variant
${'{0:.2%}'.format(difference - 1)} fewer conversions than the $control variant
-
You have now collected enough data to be 99% certain that this conclusion is valid.
-
Recommendation:
adopt $variant variant.
adopt $control variant.
-
You have collected enough data to be 95% sure that this difference is not simply due to chance factors.
-
Recommendation: let the test run for longer to be completely certain.
-
You don't have enough data to be sure yet that this difference is due to the variant you are testing or purely chance factors.
-
Recommendation: let the test run to collect more data.