Group assignment

Overview

This guide outlines best practices for assigning experiment groups for AI Decisioning (AID).

Hightouch includes built-in functionality to manage group assignments within AI Decisioning. In most cases, this is the recommended approach. However, customer-assigned groups are also supported when required.

Depending on your setup, group assignment may be:

• Fully managed by Hightouch
• Fully customer-managed
• A hybrid of both

This guide explains each approach and when it is most appropriate.

Why experiment group assignment is required

AI Decisioning’s measurement methodology is designed to help marketers understand whether personalized messaging drives incremental conversions beyond what would happen otherwise.

Properly managed treatment and holdout group assignment is critical to valid experiment reporting. It allows AID to isolate the true impact of personalization delivered by its models.

Throughout this document, group assignment refers to assigning users to:

• Treatment
• Holdout
• Business-as-usual (BAU) / customer-managed groups

Setup options

Choose the setup path that best fits your implementation:

• Hightouch-managed group assignment
• Customer-managed group assignment

Hightouch-managed group assignment

In most cases, Hightouch recommends using native AI Decisioning functionality to assign experiment groups. This is configured directly in the Hightouch UI when you set up an agent →.

When users qualify for an agent’s audience, Hightouch assigns each user to a group by:

• Computing a deterministic, uniform hash of a stable user identifier plus a static experiment key (salt)
• Taking the result modulo 100
• Mapping contiguous bucket ranges to configured percentages
  (for example, 0–9 = 10% holdout, 10–29 = 20% BAU, 30–99 = 70% treatment)

Although group assignment is evaluated daily when the AID job runs, deterministic hashing ensures users retain the same group once assigned.

This approach is randomized and independent of demographic or attribute bias. Assignments depend only on (user_id, salt) and do not rebalance over time. If percentage allocations change later, existing assignments remain stable and only marginal bucket ranges expand or contract.

Advantages

• Guaranteed group stability using proven randomization logic
• Simplified configuration for most implementations

Considerations

• Group assignments are evaluated daily when the AID job runs. If many new users enter the audience each day, users may briefly be eligible for BAU messages before suppression occurs.
• Group assignments must be synced to downstream tools. Suppression must be configured to ensure holdout and AID treatment users do not receive BAU messaging.

Hightouch-managed setup steps

Prerequisites

• Initial schema configuration
  Learn more about defining your schema →
  • Parent model with a unique primary key
  • Related models configured
• Audience defined in Hightouch
  Learn more about building audiences →
• Destinations configured for syncing group assignments to your ESP
  See setup instructions for Braze →, Iterable →, or SFMC →

1. Set the holdout percentage

When you create a new agent, you set a holdout percentage. The standard holdout is 10%.

For live agents, you can view or modify the holdout percentage in the agent’s Configuration tab.

If you chose to manage group assignment yourself in the data warehouse or via another tool, the source field can be referenced here instead of a raw percentage assignment.

2. Define the BAU / customer-managed percentage

During onboarding, you will work with your Hightouch account team to determine what percentage of treatment is allocated to AI Decisioning vs your BAU campaigns.

Your account team configures the BAU percentage in the agent.

3. Generate experiment groups and sync downstream

Prior to launch, your Hightouch account team will:

1. Generate initial group assignments
2. Join assignments back to your schema
3. Sync groups to your ESP as Cohorts, Segments, or Lists (terminology varies)

4. Set up BAU suppression

To ensure a valid experiment, holdout and treatment users must be suppressed from BAU campaigns.

For example:

• Sync holdout and treatment users to a Braze cohort
• Use that cohort as suppression for all BAU campaigns

Where group assignments are used

Group assignments are stored in your data warehouse and are:

• Used by the Hightouch UI to populate reporting
• Available as list memberships or user attributes in your ESP

Performance overview reporting supports breakdowns for Treatment, BAU, and Holdout.

Experiment groups table in the warehouse

This table tracks which experiment bucket each user belongs to over time for a given agent (e.g., AID treatment vs BAU vs holdout).

Example table name
hightouch_planner.de_user_experiment_groups_<agent_id>

Key columns

This table is typically used to:

• Build BAU and holdout audiences
• Slice reporting by experiment group membership over time

The group assignment table is pulled into the Schema:

Customer-managed group assignment

The following setup applies only if you manage group assignment outside of Hightouch.

Customer-managed assignment via SQL

You create a column in your parent model using SQL to denote split group assignments.

Advantages:

• Enables you to use your existing experiment assignment infrastructure
• Allows for more frequent experiment group updates if your users enter your parent model at high frequency and BAU suppression groups need be updated more frequently than AID run frequency

Considerations:

• Requires careful SQL logic to ensure proper randomization and group stability
• Must preserve assignment stability when users re-enter audiences

Other tool-managed assignment

If you have a dedicated testing platform or leverage split capabilities within another tool, we can use the split assignments as long as they are populated into the data-warehouse. Timing considerations must be taken into account here to ensure users are not eligible for an audience without having a split assignment.

Customer-managed setup steps

Prerequisites

Before configuring group assignment, ensure the following are in place:

• Initial schema configuration
  See Define data schema →
• A parent model with a unique primary key
• Related models configured
• An audience defined in Hightouch
  See Audiences →
• Destinations configured for syncing group assignments to your ESP
  See destination setup docs →
• Confirmation of your split group assignment, typically one of the following:
  • 2-way split: 90% Hightouch AID treatment, 10% holdout
  • 3-way split: 45% Hightouch AID treatment, 45% BAU, 10% holdout

1. Create a related model for group assignment

Create a dedicated related model in your schema that deterministically assigns each user to a group.

The following example shows a Snowflake SQL query that can be used directly in a related model to assign a 45 / 45 / 10 split.

-- Snowflake example
WITH bucketed_users AS (
    SELECT
        user_id,
        -- Modulo 100 converts the hash into a uniform integer bucket from 0–99
        -- Rotate the salt '1' to redistribute users in future experiments
        ABS(HASH(CONCAT(user_id, '1'))) % 100 AS bucket
    FROM parent_model_table
)

SELECT
    user_id,
    CASE
        WHEN bucket < 45 THEN 'treatment'
        WHEN bucket < 90 THEN 'bau'      -- business as usual
        ELSE 'holdout'
    END AS aid_agent1_assignment
FROM bucketed_users;

In your related model setup:

• Use user_id as the join key
• Set the relationship to 1:1
• Merge fields from the related model onto the parent model

This makes the group assignment available as a user-level attribute throughout Hightouch.

Example related model configuration

2. Create a Hightouch sync for the BAU audience

Create an audience that includes all users where: aid_agent1_assignment = bau (using the field name defined in the group assignment SQL.)

Set up a recurring sync to your ESP to populate an audience that will be used as a filter for BAU campaigns.

• Choose a sync frequency that fits your send schedule (for example, hourly)
• Ensure the audience stays up to date as users enter and exit eligibility

Where are group assignments used in AID?

Your Hightouch account team will add the group assignment variant into your agent configuration.

Group assignments are used in the Hightouch UI to automatically populate reporting.

Performance overview reporting supports breakdowns for:

• Treatment
• BAU
• Holdout

Frequently asked questions

Will Hightouch reporting match my other attribution?

While raw event counts will match when using the same source data, Hightouch reporting is incrementality-based and will not always align with attribution-based reporting in your ESP or other tools.

When you look at Hightouch incrementality reporting we are not claiming full responsibility for every conversion. We are simply reporting the conversion rate for the group of users who received any treatment vs the conversion for users who did not. The result is a clear overview of whether the marketing efforts are effective and how much lift they drive.

What do Treatment and Holdout users actually receive?

• Treatment
  AI-optimized messages (content and timing) from the agent.
• Holdout
  No campaign messaging for that use case, but users are still tracked to observe organic behavior.
• BAU / customer-managed
  Users in this group continue to receive your existing marketing messaging.

How does Hightouch assign users to Treatment, Holdout, and BAU?

When a user qualifies for an audience, Hightouch:

1. Computes a deterministic hash of the user’s stable ID and an experiment key
2. Applies modulo 100 to create a bucket from 0–99
3. Maps bucket ranges to groups (for example, 0–9 = holdout, 10–29 = BAU, 30–99 = treatment)

This produces random, unbiased, and stable group assignments that never change for a given agent.

What happens if we change percentages later?

Changing split assignments only affects new users entering the audiences. Existing users remain in their original group. There is no re-randomization of users once a group is assigned.

What if users fall out of the audience and later re-enter?

Group assignment is deterministic. Users re-enter the same group they were originally assigned to.

What happens if I want to put all users in my treatment group?

Once you have established a high-performing agent, you can merge BAU users into the Hightouch treatment audience.

Contact your Hightouch account team to request this change. We do not recommend merging the holdout group and suggest maintaining at least a small holdout for ongoing measurement.