Audience Splits are part of the Advanced Customer Studio, available as an add-on to Business Tier plans.
Say you're a marketer and have just defined a new audience of users. You might have questions like, "Which channel works best here?" or "What copy or offer should I use?"
Hightouch Splits empowers you to use A/B and multivariate testing to answer these questions. With Splits, you can create randomized split groups or splits of users within your audience. You can specify:
- The number of splits you want to use
- The percentage of the total audience membership that each split should have; for example, split A gets 40% and split B gets 60%
- The destination to send each split to; for example, split A goes to Facebook, Split B goes to Google, and split C is a holdout that doesn't get sent to any destination
Using Splits, you can run experiments such as:
- Holdout Tests: Send one split to a marketing destination (email, Google, Facebook, etc.) and have the other split act as a control and receive no messaging. These tests use the holdout group feature and can prove if there was an actual lift from marketing efforts.
- Channel Tests: Send splits to different channels to determine the most effective one.
- Offer/Creative Tests: Deploy different messaging on each split to see what resonates the best.
To configure splits, click into an existing audience on the Audiences overview page, and select the Splits tab. From there, you can toggle Enable splits for the selected audience.
Once you've enabled splits for an audience, you need to configure the following:
- Group Column Name: This is the name of a column that Hightouch automatically creates to denote which split group a row is part of.
- Split Group Names and Percentages: This is a list of the split group names you want to assign to your audience and the percentage of members to assign to each group.
- Syncs : This is the destination you want to send each split to. To enable holdout groups, select the Sync Logs Only destination. For more information, see the holdout groups section.
- Stratification Variables (optional): This advanced beta feature lets you select columns to use to evenly distribute your splits. For example, you may want an even distribution of “Age” across your split groups. For more information, see the stratified sampling section.
Enable holdout groups
If you want one of your splits to act as a control, you can send it to the Sync Logs Only destination.
This destination doesn't actually send data anywhere but uses the same sync logic as other destinations. That means data synced to this destination appear in your audit and warehouse logs. From there, you can run analytics to determine how this split compares to other splits that were sent to a marketing destination.
If you want access to the Warehouse Sync Log Destination, don't hesitate to .
Stratified sampling lets you define subgroups based on shared characteristics such as demographic traits. Hightouch uses these characteristics to first divide the audience into subgroups or strata before assigning split groups.
Be aware of the following caveats when using stratified sampling:
- Degrading split percentage accuracy: The more stratification variables you define, the less accurate the split percentages may become. This happens because it becomes difficult to represent the appropriate percentage of users in each split group as stratification becomes more granular.
- Inconsistent assignment: Not all users are guaranteed assignment to the same split group per sync. This removes the ability to perform incremental syncs and forces users to a perform a full resync every time they sync their audience data.
How does audience size affect split percentage accuracy?
A general rule of thumb is the larger the audience size, the more accurate the split percentages will be. This is because there are more people to achieve the desired split percentage for each group.
Other factors that can affect the accuracy of your split percentages include the number of split groups you define and their associated percentages. Generally, the lower the number of groups, the more accurate split percentages can be. The overall audience size and percentages you select for split groups also affect achieved percentage accuracy. For example, it's easier to achieve a perfect 20/40/40 split for an audience whose overall size is a multiple of 20.
How are split groups assigned?
Under the hood, Hightouch uses a hashing algorithm that uses the parent model's primary key to consistently assign split groups.