Configure an A/B test

Setting up A/B testing requires two steps:

• Configuring targeting and policy overrides for an application
• Configuring a dashboard to view A/B test results

This article explains how to configure targeting and policy overrides. See View A/B test results to learn how to configure dashboards to view the results.

Targeting

Targeting refers to the practice of selecting specific segments of users or audiences that will participate in an A/B test experiment. By targeting specific segments, businesses can obtain more relevant and actionable insights, ensure the test results are statistically significant, and make informed decisions on how to optimize their product or service for different user groups.

Baselines and variants

Baselines and variants are used to group users who went through the normal set of policies vs those who did not. Strivacity’s dashboard allows for filtering by baselines and variants to facilitate comparisons of the two groups to determine how the A/B test experiment performed.

1. Baseline: Represents customers who went through the standard set of policies
2. Variants: Represents customers and the policies who experienced the modified policy set

The goal of A/B testing is to compare the performance of these variants to determine which one produces better results based on predefined metrics such as registration failure rates and abandonments.

Setting up a test

To set up an A/B test, first select the application you want to perform testing on in the Admin Console.

1. Once you've selected an Application in the Admin Console, select the A/B testing tab.
2. Create a new variant by clicking +Create variant.
3. Specify a Variant name. This will be the name used to identify variants in the dashboard. Add an optional Description to explain the purpose of the test, viewable only by brand admins.
4. Define your targeting based on Native claims and Context variables.
5. Define which Policy variants will be applied to the targeted customers.
   1. You can change (or in the case of Identity Verification, remove) any policy that is applied to the current application.
6. Define which Consent variants will be applied to the targeted customers.
   1. You can add a new consent, change how the consent is collected (via Consent type), or remove an existing consent.
7. Define which Lifecycle event hook variants will be applied to the targeted customers.
   1. You can add a new Lifecycle event hook, change an existing hook, or remove a hook
8. Decide whether you want to enable or disable the experiment before saving.
9. Click Save.

🚧

Like all policies in Strivacity, A/B tests take effect as soon as you click "Save". Be sure the variant is disabled before clicking save, unless you are ready to launch the test.