# Hightouch Documentation — Full Content
> This file contains the full text content of all public Hightouch documentation pages. It is generated automatically on every push to master.
---
## Daily usage (Marketers & business teams)
**URL:** https://hightouch.com/docs/getting-started/business
**Description:** Learn where marketers work in Hightouch and how to build audiences, run campaigns, and measure results.
**Section:** Getting started
| | |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers |
| **Prerequisites** |
Access to the correct Hightouch workspace or [Space](/workspace-management/spaces)
At least one [schema](/customer-studio/schema) configured by your data team
At least one [destination](/destinations/overview) available for activation
A role with permission to create audiences or launch campaigns
|
_Hightouch lets marketers work directly with governed warehouse data. Your platform admin and data team set up access, schema, and guardrails first. After that, you can build audiences, run campaigns, and measure results within those boundaries._
Platform admins and data teams set up access, schema, and governance before
marketers start working. This guide focuses on what marketers do after that
setup is complete. See [Workspace setup](/getting-started/it) and [Technical
setup](/getting-started/data) for setup details.
---
## Before you start
If something looks missing after you log in, it may be expected governance rather than a bug.
- **Can't see certain audiences or syncs?** Your platform admin may use [Spaces](/workspace-management/spaces) to limit which resources are visible to each team, organized by region, brand, or business unit.
- **Missing customer attributes or events?** Ask your data team. They define which data appears in the audience builder through the [schema](/customer-studio/schema).
- **Can't access a destination?** Platform admins control which marketing tools are connected and may use [destination rules](/customer-studio/destination-rules) to restrict which records can be sent to specific channels, or [approval flows](/workspace-management/approval-flows) to require review before changes go live.
---
## What you can do yourself vs. what requires setup
| You can usually do this yourself | Your platform admin sets this up | Your data team sets this up |
| ------------------------------------------------ | ----------------------------------------------------------------- | ------------------------------------------ |
| Build and save audiences | Create workspaces and manage access (roles, SSO, groups) | Connect sources and configure sync engines |
| Preview and validate audiences | Set governance guardrails (subsets, destination rules, approvals) | Define schema, models, and traits |
| Reuse audience or sync templates | Create templates for audiences and syncs | Expose new fields when you need them |
| Launch syncs or journeys within your permissions | Connect destinations (either team) | Connect destinations (either team) |
| Review sync results and campaign performance | | |
---
## Choose your starting point
What you do first depends on your goal.
| If you want to... | Start here |
| ----------------------------------------------------- | -------------------------------------------------------- |
| Build an audience and send it to a marketing tool | [Create audiences](/customer-studio/usage) |
| Run a multi-step campaign with branching logic | [Journeys](/customer-studio/journeys) |
| Measure campaign lift or compare audience performance | [Intelligence overview](/campaign-intelligence/overview) |
| Let AI choose the best channel, message, and timing | [AI Decisioning overview](/ai-decisioning/overview) |
**Lifecycle Studio** and **Ad Studio** are two additional product areas for
marketers. Documentation for both is coming soon.
---
## Core workflow
Most marketer workflows in Hightouch follow the same pattern: explore data, build an audience, activate it, and measure results.
### 1. Explore the data available to you
Start by understanding what you're building with. Your data team defines a [schema](/customer-studio/schema) that determines which customer attributes, related models, and events appear in the audience builder.
- The **parent model** is your main dataset — typically a `users` or `customers` table.
- **Related models** add supporting context like accounts, subscriptions, or product catalogs.
- **Events** are timestamped actions like purchases, logins, or cancellations.
- [**Traits**](/customer-studio/traits) are computed values (aggregations, most-recent values, or custom formulas) defined by your data team. You use them as filters when building audiences.
Your platform admin may have configured [subsets](/customer-studio/subsets) that limit which records you can query, for example by region or brand. If you're only seeing a portion of the data, you're likely seeing the data relevant to your team.
### 2. Build an audience
Use [Customer Studio](/customer-studio/usage) to create audiences from the data available in your schema.
1. Choose a parent model.
2. Add filters using customer attributes, related data, or events.
3. Preview the audience to verify the results look right.
4. Save the audience.
If your team uses [audience templates](/customer-studio/templates), start
there. Templates reduce setup time and help enforce consistent campaign logic
across teams.
### 3. Activate your audience
Once you've built an audience, decide how to get it into your downstream tools.
- **Sync it directly** to a destination your data team has already connected (ad platforms, ESPs, CRMs) using [audience syncs](/customer-studio/syncs).
- **Orchestrate a multi-step flow** with [Journeys](/customer-studio/journeys), which support branching, delays, and conditional logic.
- **Optimize delivery with AI** using [AI Decisioning](/ai-decisioning/overview), which selects the best channel, message, and timing for each customer.
### 4. Launch with guardrails
Depending on how your workspace is configured, some controls may apply automatically when you activate an audience.
- [**Approval flows**](/workspace-management/approval-flows) may require a teammate or admin to review changes before they go live.
- [**Destination rules**](/customer-studio/destination-rules) may automatically exclude records that don't meet channel or consent requirements.
- [**Subsets**](/customer-studio/subsets) may limit which records you can include in an audience based on region, brand, or compliance boundaries.
These guardrails are configured by your platform admin or data team. They're designed to let you move quickly without bypassing data controls.
### 5. Monitor results and iterate
After launch, check how your audiences and campaigns are performing.
- Review **sync status** from the sync detail page to see whether records were delivered, rejected, or errored.
- Use [**Intelligence**](/campaign-intelligence/overview) to measure campaign lift, compare audience segments, and track performance over time.
- Run [**Experiments**](/customer-studio/experiments) to test audience variants and measure which performs better.
- Revisit your audience logic based on performance data, approval feedback, or error patterns.
---
## Related products
These products work alongside Customer Studio and may already be set up in your workspace. Ask your admin if you're unsure what's available.
- [**Identity Resolution**](https://hightouch.com/platform/identity-resolution): Unified customer profiles across devices and channels.
- [**Events**](/events/overview): Campaign triggers based on customer actions, either as scheduled batches or in real time.
- [**Match Booster**](/match-booster/overview): Improved ad platform match rates through identifier enrichment.
- [**Real-Time Personalization**](/real-time/overview): In-session experiences that respond to what a customer is doing right now.
For an overview of how the marketing workflow fits together, see [Hightouch for marketers](/getting-started/cdp).
---
## Hightouch for marketers
**URL:** https://hightouch.com/docs/getting-started/cdp
**Description:** How marketers use Hightouch to build audiences, explore data with AI agents, orchestrate campaigns, and measure results — all from warehouse data, without writing SQL.
**Section:** Getting started
## Overview
This page walks through how marketers work in Hightouch — from building your first audience to measuring campaign results. Each section covers one step in the workflow. You can adopt them independently, but they work best together.
Everything here runs on warehouse data that your data team has already prepared. For the data infrastructure underneath, see [How data activation works](/getting-started/concepts).
[**Explore the interactive architecture diagram**](https://architecture.hightouch.com/?from=docs&product=overview&utm_source=hightouch-docs&utm_medium=referral&utm_campaign=architecture-explorer&utm_content=overview)
See how data flows and where each product fits in your stack.
---
## Build audiences
[Customer Studio](/customer-studio/overview) is where you define who to target. The audience builder lets you combine traits, events, and relationships in a visual interface — no SQL required.
You can define conditions like "purchased in the last 30 days" or "viewed a product but didn’t check out," preview audience size and membership before syncing, and create [calculated traits](/customer-studio/traits) like lifetime value or days since last purchase. Audiences stay current as your data changes and sync automatically to your connected destinations.
**Example**: Target users who viewed a product but didn’t complete checkout.
One audience can power multiple syncs to different tools — the same "high-value customers" segment can feed a Braze campaign, a Meta lookalike audience, and a Salesforce list.
[**Customer Studio overview**](/customer-studio/overview)
---
## Explore data with agents
Agents let you ask questions about your customer data and marketing performance in natural language. Instead of writing SQL or filing a ticket with your data team, you describe what you want to know and the agent returns answers grounded in your warehouse data.
You can explore customer segments and trends conversationally, analyze campaign performance across channels, and save prompts as recurring reports that run on a schedule. Agents draw on the same warehouse data that powers your audiences, so you can build an audience visually and then use an agent to analyze its performance or compare it against other segments.
---
## Orchestrate journeys
[Journeys](/customer-studio/journeys) automate multi-step campaigns across channels. You define the entry criteria, add steps like emails, SMS, or webhook calls, set time delays and branching conditions, and Hightouch handles the execution.
**Example**: A customer abandons checkout → wait 2 hours → send a reminder email → if no purchase after 24 hours → send an SMS with a discount code.
Journeys support holdout groups for measurement, re-entry rules, and branching based on customer behavior or attributes.
---
## Optimize with AI Decisioning
[AI Decisioning](/ai-decisioning/overview) uses reinforcement learning to recommend the best message, channel, and timing for each customer. Rather than manually creating rules for every scenario, you define the goal and AI Decisioning learns what works over time.
It optimizes across message content, send channel, and timing simultaneously. You can inspect individual decisions to understand why a customer received a specific treatment and measure incremental lift against holdout groups.
AI Decisioning works alongside journeys and audiences — it doesn’t replace your campaign logic, it makes each touchpoint more effective.
---
## Draft ad creatives
[Ad Studio](https://hightouch.com/platform/ad-studio) helps you draft and refine ad creatives for paid channels. You provide brand assets and guidelines, and the AI generates variations that follow them. You review, edit, and export the final creatives to your ad platforms.
---
## Measure results
[Intelligence](/campaign-intelligence/overview) provides measurement tools so you can understand what’s working and what isn’t.
**Splits** divide audiences into test and control groups to measure the incremental impact of a campaign. **Holdouts** withhold a portion of an audience to establish a baseline. **Dashboards** track campaign metrics over time and let you drill into performance by audience, treatment, or trait.
Results write back to your warehouse, so your data team can include them in broader reporting and analysis.
[**Intelligence overview**](/campaign-intelligence/overview)
---
## How it connects
The marketing workflow in Hightouch follows a natural sequence, but each tool works independently.
1. Your data team connects a source and sets up the [schema](/customer-studio/schema)
2. You build [audiences](/customer-studio/usage) in Customer Studio
3. Agents help you explore data and find opportunities
4. [Journeys](/customer-studio/journeys) automate multi-step campaigns
5. [AI Decisioning](/ai-decisioning/overview) optimizes message, channel, and timing per customer
6. [Intelligence](/campaign-intelligence/overview) measures what worked
---
## Next steps
**Marketers**: Get started with the [Daily usage guide](/getting-started/business).
- Build your first [audience](/customer-studio/usage)
- Set up a [journey](/customer-studio/journeys)
- Explore [AI Decisioning](/ai-decisioning/overview)
- Learn [how data activation works](/getting-started/concepts) — the technical foundations underneath everything in Hightouch
---
## Data activation concepts
**URL:** https://hightouch.com/docs/getting-started/concepts
**Description:** Learn the foundational concepts in Hightouch—sources, models, schema, syncs, destinations, and change data capture—that power Reverse ETL and the Composable CDP.
**Section:** Getting started
## Overview
Data activation is the process of moving trusted warehouse data into the tools where teams take action — ad platforms, CRMs, marketing automation, support desks, and more. Hightouch calls this process **Reverse ETL**.
This page covers the building blocks that make Reverse ETL work: **sources**, **models**, **schema**, **syncs**, **destinations**, and **change data capture**. It is written for data teams and engineers who set up these components, but it also helps marketers understand the concepts they encounter in Customer Studio.
[**Explore the interactive architecture diagram**](https://architecture.hightouch.com/?from=docs&product=reverse-etl&utm_source=hightouch-docs&utm_medium=referral&utm_campaign=architecture-explorer&utm_content=reverse-etl)
See how data flows and where Reverse ETL fits in your stack.
---
## Sources
A **source** is any system where your data resides. Common examples include Snowflake, BigQuery, Databricks, and PostgreSQL. Sources can also be files, APIs, or BI tools. In practice, this is where customer and business data lives — purchase history, campaign activity, product usage, and so on.
See [Sources overview](/sources/overview) for setup instructions and a full list of supported connectors.
---
## Models
A [**model**](/models/creating-models) defines what data to query from a source. Models can represent users, events, products, or any other entity. Data teams typically create models by writing SQL in the editor, selecting from tables or views, or re-using dbt models and BI queries from Looker or Sigma.
Every model requires a **unique primary key** to identify each row and track changes between syncs. In practice, a model is a reusable dataset — “all active customers,” “abandoned carts,” or “email subscribers” — that can power multiple syncs and audiences.
See [Models overview](/models/creating-models) for details on each modeling method.
---
## Schema
A **schema** is a one-time configuration that connects your models to [Customer Studio](/customer-studio/overview). Data teams define a parent model (typically users), then add related models, events, and relationships. Once the schema is configured, marketers can build [audiences](/customer-studio/usage) and [journeys](/customer-studio/journeys) in a no-code UI. Audiences cannot be created until this step is complete.
In Customer Studio, the schema determines which fields, events, and relationships appear in the audience builder — it is how marketers access model data without writing SQL.
See [Customer Studio schema setup](/customer-studio/schema) for configuration steps.
---
## Syncs
A [**sync**](/syncs/overview) defines how data appears in the destination and when. Syncs specify:
- **Type**: object, event, audience, etc.
- **Mode**: insert, update, upsert, or archive
- **Mapping**: how source columns map to destination fields
- **Schedule**: interval, cron, or triggered by tools like dbt Cloud or Airflow
Multiple syncs can be created from the same model to support different use cases. In practice, a sync is how a dataset shows up in tools like Braze, Salesforce, or Meta Ads so teams can act on it.
See [Syncs overview](/syncs/overview) for configuration details and scheduling options.
---
## Destinations
A **destination** is any system where data is consumed — CRMs, ad platforms, support tools, analytics, or custom APIs. These are the tools where teams build campaigns, measure results, and engage with customers.
See [Destinations overview](/destinations/overview) for a full list of supported destinations.
---
## Change data capture (CDC)
Hightouch optimizes syncs with **change data capture (CDC)**. Instead of sending the full query result on every run, CDC compares the current results to the previous run and only sends new, changed, and removed rows. This keeps downstream tools current without redundant updates.
By default, CDC runs on Hightouch-managed infrastructure. For higher performance at scale, you can enable the [Lightning sync engine](/syncs/lightning-sync-engine) to compute CDC directly in the warehouse.
Hightouch recommends enabling the [Lightning sync engine](/syncs/lightning-sync-engine) during source setup. Beyond performance, it’s required for features like [Journeys](/customer-studio/journeys), [Sampling](/customer-studio/sampling), [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Audience Snapshots](/customer-studio/audience-snapshots), and [Identity Resolution](https://hightouch.com/platform/identity-resolution). See the [full list of requirements](/syncs/lightning-sync-engine#when-the-lightning-sync-engine-is-required).
See [Change data capture overview](/syncs/cdc) for details on how diffing works and how to choose a sync engine.
---
## How it fits together
1. Connect a **source** (e.g., Snowflake)
2. Define **models** that query the data you need (e.g., “active customers in the last 30 days”)
3. Configure the **schema** so marketers can access model data in Customer Studio
4. Set up **destinations** and **syncs** to deliver data to downstream tools
From there, marketers build [audiences](/customer-studio/usage), orchestrate [journeys](/customer-studio/journeys), run [experiments](/customer-studio/experiments), and measure results — all from warehouse data, without writing SQL.
These primitives are the foundation for Hightouch's marketing tools. For a walkthrough of the marketing workflow — audiences, agents, journeys, AI Decisioning, and measurement — see [Hightouch for marketers](/getting-started/cdp).
---
## Next steps
**Data teams**: Get started with the [Quick start guide](/getting-started/data).
- Set up your first [source](/sources/overview)
- Create a [model](/models/creating-models)
- Configure a [sync](/syncs/overview)
- Explore [destinations](/destinations/overview)
- See the marketing workflow in [Hightouch for marketers](/getting-started/cdp)
---
## Technical setup (Data teams & engineers)
**URL:** https://hightouch.com/docs/getting-started/data
**Description:** Connect your data warehouse, configure sync infrastructure, and prepare reusable datasets in Hightouch.
**Section:** Getting started
| | |
| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Data teams |
| **Prerequisites** |
Workspace created by your platform admin
Warehouse credentials (Snowflake, Databricks, BigQuery, or Redshift)
A role with permission to create sources, models, and syncs
|
_Connect your data warehouse, configure sync infrastructure, and prepare datasets so marketers can build and activate audiences directly in Hightouch._
New to Hightouch's data model? The [**Data activation
concepts**](/getting-started/concepts) page explains sources, models, syncs,
destinations, and Change Data Capture (CDC). This article focuses on the
hands-on setup work for data teams.
---
## Setup steps
### 1. Connect your data
Connect your warehouse as the **source of truth** for all activation.
1. Add your primary data source (e.g., Snowflake, Databricks, BigQuery, Redshift) from **Integrations**, then **Sources**.
2. During source setup, select the [**Lightning sync engine**](/syncs/lightning-sync-engine) **(recommended)**. Lightning computes [change data capture](/syncs/cdc) directly in your warehouse for faster, more reliable syncs. It is required for [Journeys](/customer-studio/journeys), [Sampling](/customer-studio/sampling), [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Identity Resolution](https://hightouch.com/platform/identity-resolution), and other advanced features.
3. **(Optional)** Connect additional data sources such as databases, APIs, or SaaS tools if they’re part of your activation strategy.
[**Sources overview**](/sources/overview)
_Hightouch syncs modeled data from your warehouse to destinations through sources, models, and syncs._
---
### 2. Configure sync infrastructure
Set up the remaining systems that power reliable syncs between your warehouse and marketing tools.
1. [**Warehouse Sync Logs**](/syncs/warehouse-sync-logs)
Write sync metadata (row counts, errors, and rejected records) back to your warehouse for monitoring and visibility.
2. [**Destinations**](/destinations/overview)
Connect downstream tools (e.g., Braze, Salesforce, ad platforms) where marketers will activate audiences and campaigns. Either data teams or platform admins can own this step.
---
### 3. Define your schema
Define the data structure that marketers will use in Customer Studio to build audiences and campaigns. This is typically a **one-time setup owned by data teams**, though you may revisit it as new data becomes available.
- **Parent model:** The main dataset (for example, your `users` table)
- **Related models:** Supporting entities such as accounts, subscriptions, or products
- **Events:** Timestamped actions (purchases, logins, cancellations)
- **Relationships:** Connections between models via foreign keys
- **[Traits](/customer-studio/traits):** Computed values like aggregations, most-recent values, or custom formulas that marketers can use as filters. If a marketer can't find the right attribute, it usually means a trait or schema change is needed here.
- **[General models](/customer-studio/general-models) (optional):** Warehouse tables available for [agent](/agents/overview) queries but not used in audiences or syncs. Add these when agents need access to data outside your Customer Studio schema, such as sales figures or product catalogs.
[**Define data schema**](/customer-studio/schema)
_Data teams define parent, related, and event models, including traits. Marketers use these attributes in Customer Studio to build audiences._
---
### 4. Security and change management
Security and workspace-level governance are usually managed by platform admins.
As a data team, coordinate with IT to ensure:
- **Environments** are available for staging models and syncs
- **Audit logs** and **approval flows** are enabled per compliance policies
[**Workspace setup (Platform admins)**](/getting-started/it)
---
### 5. Set up additional products
If your team uses other Hightouch products, each has its own setup guide.
| Product | Setup docs |
| ------------------------- | ---------------------------------------------------------------------------------- |
| Identity Resolution | [Identity Resolution overview](https://hightouch.com/platform/identity-resolution) |
| Events | [Events overview](/events/overview) |
| Match Booster | [Match Booster overview](/match-booster/overview) |
| Real-Time Personalization | [Set up realtime audiences](/real-time/setup) |
For an overview of how the marketing workflow fits together, see [Hightouch for marketers](/getting-started/cdp).
---
## Readiness checklist
Before marketers can build audiences and activate data, confirm the following is in place.
- [ ] Primary data source connected with the [Lightning sync engine](/syncs/lightning-sync-engine) (recommended)
- [ ] At least one [destination connected](/destinations/overview) (either data team or platform admin)
- [ ] [Warehouse Sync Logs](/syncs/warehouse-sync-logs) enabled for monitoring
- [ ] [Schema defined](/customer-studio/schema) with parent model, related models, events, and relationships
- [ ] [Traits](/customer-studio/traits) created for any computed values marketers will need as filters
- [ ] Marketers can see the right data in the audience builder (check with your platform admin if [Spaces](/workspace-management/spaces) or [subsets](/customer-studio/subsets) are in use)
---
## Workspace setup (IT & platform admins)
**URL:** https://hightouch.com/docs/getting-started/it
**Description:** Set up secure access, governance, and user roles in your Hightouch workspace so data and marketing teams can activate safely.
**Section:** Getting started
| | |
| ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Platform admins |
| **Prerequisites** |
Admin or organization-owner access to your Hightouch account
Your organization's SSO provider details (if using SSO)
|
_Configure your Hightouch workspace to ensure secure access, clear permissions, and consistent governance across teams._
---
## Setup checklist
### 1. Learn the building blocks
Before configuring anything, familiarize yourself with how Hightouch is organized: organizations, workspaces, user groups, roles, environments, and Spaces.
[**Core concepts**](/workspace-management/overview) — organizations, workspaces, user groups, roles, environments, and Spaces
Most teams use a single production workspace with [Spaces](/workspace-management/spaces), permissions, and [subsets](/customer-studio/subsets) to separate teams. Use additional workspaces when you need strict data isolation or separate dev and prod environments. See [Workspaces](/workspace-management/workspaces) for guidance on when to add a workspace.
---
### 2. Create your workspace
Set up your organization and first workspace. Choose a region, pick a workspace name and slug, and walk through the onboarding checklist.
[**Workspaces**](/workspace-management/workspaces) — create, configure, switch, and delete workspaces
---
### 3. Set up access control
Configure how your team authenticates and what they can access. Set up SSO and SCIM to automate provisioning, map identity provider groups to Hightouch user groups, and assign roles.
[**SSO setup**](/workspace-management/sso) — connect your identity provider for centralized authentication
[**Manage users and groups**](/workspace-management/user-management) — invite users, assign groups, manage membership
[**Roles**](/workspace-management/roles) — pre-built and custom permission sets
[**Transfer org admin access**](/workspace-management/user-management#transferring-organization-admin-access) — hand off admin privileges when team members change
_User groups manage access across workspaces, with roles assigned at the group level._
Start with pre-built roles during onboarding. Move to custom roles once your
org structure or compliance needs require it.
---
### 4. Plan your governance
Decide how teams should collaborate within your workspace: who can publish changes, what data they can reach, and what requires approval. Hightouch offers several controls that layer on top of each other.
[**Governance best practices**](/workspace-management/governance) — when to use each control and how they layer together
[**Environments**](/workspace-management/environments) — dev/staging/prod promotion within a workspace
[**Approval flows**](/workspace-management/approval-flows) — require review before changes go live
[**Spaces**](/workspace-management/spaces) — scope resource visibility across teams
[**Subsets**](/customer-studio/subsets) — limit which records marketers can query by region, brand, or compliance boundary
[**Destination rules**](/customer-studio/destination-rules) — control which records can be sent to specific destinations
[**Templates**](/customer-studio/templates) — standardize audience and sync setup so marketers start from approved patterns
[**Audit logs**](/workspace-management/audit-logs) — track all in-app user actions for 90 days
---
### 5. Secure your infrastructure
Configure storage, networking, and monitoring to meet your security and compliance requirements.
[**Security overview**](/security/overview) — encryption, compliance, and hosting details
[**Storage**](/security/storage) — configure self-hosted storage for data-at-rest
[**Networking**](/security/networking) — tunnels, PrivateLink, IP allowlisting
[**Alerting**](/syncs/alerting) — sync failure notifications via Slack, email, PagerDuty, or Datadog
[**Sync logs**](/syncs/warehouse-sync-logs) — write sync history to your warehouse
---
### 6. Organize resources (optional)
As your workspace grows, use folders, labels, and filters to keep things navigable.
[**Folders, filters, and labels**](/workspace-management/folders-filters)
---
## Readiness checklist
Before marketers can start building audiences, the workspace needs to be fully configured. Use this checklist to verify that all setup is complete across teams.
**Platform admin responsibilities:**
- [ ] Workspace created and region selected
- [ ] SSO and SCIM configured (if applicable)
- [ ] User groups created and roles assigned
- [ ] Governance configured: environments, approval flows, Spaces, subsets, destination rules, and templates
- [ ] Storage, networking, and alerting configured per security requirements
- [ ] Marketers have the right roles and [Space](/workspace-management/spaces) access to see the data they need
**Confirm with your data team or whoever owns the technical setup:**
- [ ] At least one [source connected](/sources/overview) with the [Lightning sync engine](/syncs/lightning-sync-engine) (recommended)
- [ ] At least one [destination connected](/destinations/overview)
- [ ] [Schema defined](/customer-studio/schema) with parent model, related models, events, and traits
In smaller teams, one person may own both platform admin and data team responsibilities. The checklist above is organized by responsibility, not job title.
## Next steps
- **Data teams:** [Technical setup](/getting-started/data) — connect sources, configure sync infrastructure, and define schema.
- **Marketers:** [Daily usage](/getting-started/business) — build audiences, run campaigns, and measure results.
---
## Welcome
**URL:** https://hightouch.com/docs/getting-started/welcome
**Description:** Learn what Hightouch does, choose the right starting point, and find setup guidance for your team.
**Section:** Getting started
Use the new [changelog](/changelog) to track recent product updates.See something wrong or have a request? Use the **Send feedback** button in the right sidebar on any page.Not sure which product fits your use case? Try the [product finder](#product-finder) below.
## What is Hightouch?
Hightouch is an Agentic Composable CDP for marketing and personalization.
It runs on top of your data warehouse, so teams can use customer data without copying it into a separate system. Data teams define and govern datasets at the source. Marketers use those datasets to build audiences, run campaigns, personalize experiences, and measure results.
Agents are built into Hightouch to help marketers explore data, build audiences, generate creative, and analyze campaign performance. [AI Decisioning](/ai-decisioning/overview) uses reinforcement learning to recommend the best message, channel, and timing for each customer. Marketers review, refine, and approve AI outputs before taking action.
---
## Find the right starting point
Start with what you want to do. Each path points you to the product area and setup guide that best matches your goal.
Create audiences from warehouse data, build automated journeys, and use the built-in agent to explore segments and analyze campaign performance in natural language.
Sync modeled data to sales, support, analytics, and internal tools on a schedule or when data changes.
AI Decisioning uses reinforcement learning to recommend the best message, channel, and timing for each customer, then measures what worked.
Tailor website and app experiences based on what a customer is doing right now.
Resolve identities across devices and channels so teams can build on a more complete customer record.
Set up permissions, approvals, and ownership boundaries so each team has the right level of access.
Help ad platforms recognize more of your customers by adding additional identifiers to synced audiences and conversions.
---
Not sure where to start? Read [How data activation works](/getting-started/concepts) to learn the technical foundations, or see [How marketers use Hightouch](/getting-started/cdp) for the marketing workflow from audiences to measurement.
---
## Explore products
Hightouch products fall into two groups. You can start with one and expand over time — they all work from the same warehouse data.
### Marketing
Build audiences, create campaigns, and measure results — with AI agents that help at every step.
- **[Customer Studio](/customer-studio/overview)** — Build audiences, orchestrate journeys, and use the built-in agent to explore data in natural language
- **[Agents](https://hightouch.com/platform/agentic-marketing-platform)** — Ask questions, analyze campaigns, and draft reports grounded in your warehouse data
- **[Context Hub](https://hightouch.com/platform/agentic-marketing-platform)** — Configure brand context, channel connections, and review settings shared across Agents, Ad Studio, and Lifecycle Studio
- **[Ad Studio](https://hightouch.com/platform/ad-studio)** — Draft and refine ad creatives, then export to ad platforms
- **[AI Decisioning](/ai-decisioning/overview)** — Recommend the best message, channel, and timing for each customer using reinforcement learning
- **[Intelligence](/campaign-intelligence/overview)** — Run experiments, compare results, and track campaign metrics
### Data platform
Infrastructure for collecting data, modeling it, delivering it to downstream tools, and powering real-time experiences.
- **[Models](/models/creating-models)** — Define reusable datasets with SQL, dbt, or a visual builder
- **[Syncs](/syncs/overview)** — Deliver data to destinations on a schedule or when data changes
- **[Hightouch Events](/events/overview)** — Collect behavioral data from apps and websites
- **[Real-Time Personalization](/real-time/overview)** — Personalize onsite and in-app experiences
- **[Identity Resolution](https://hightouch.com/platform/identity-resolution)** — Unify customer profiles across devices and channels
- **[Match Booster](/match-booster/overview)** — Increase ad match rates with additional identifiers
---
## Get started by team
Hightouch is a cross-functional platform. Different teams own different parts of setup and daily usage.
| Role | What you'll set up | Start here |
| --- | --- | --- |
| Platform admins | Configure SSO, roles, approval flows, and workspace settings | [Workspace management](/getting-started/it) |
| Data teams | Connect data sources, define models, manage syncs, and set up schema | [Data modeling and access](/getting-started/data) |
| Marketers | Build audiences, launch campaigns, manage journeys, and measure experiments | [Daily usage](/getting-started/business) |
These paths are designed to be followed in order: platform admins set up the workspace, data teams connect and model data, then marketers start building. Each guide tells you what other teams need to have completed before you begin.
---
## Understand how Hightouch works
Before diving into setup, these pages help you build a mental model of the platform:
- [How data activation works](/getting-started/concepts) — Sources, models, syncs, destinations, and change data capture: the building blocks underneath everything in Hightouch.
- [How marketers use Hightouch](/getting-started/cdp) — Audiences, agents, journeys, AI Decisioning, and measurement: the marketing workflow from data to results.
- [Governance best practices](/workspace-management/governance) — How to structure roles, approval flows, environments, and audit logs so the right people can move fast without creating risk.
---
## Models overview
**URL:** https://hightouch.com/docs/models/creating-models
**Description:** Create Hightouch models using the SQL editor or the table selector which queries existing tables and views within your source.
**Section:** Models
To learn about defining data models for Audiences, refer to the [Customer Studio documentation](/customer-studio/schema).
Hightouch models define and organize the data you want to query from a [source](/getting-started/concepts).
To create a new model, navigate to the [**Models** page](https://app.hightouch.com/models?from=docs) and click **Add model**. Next, select one of you the sources you've setup. Then choose a modeling method:
- SQL editor
- Table selector
- dbt model selector
- Looker
- Sigma
Supported modeling methods depend on the source. Refer to source documentation to learn about supported modeling methods.
Refer to the [SQL editor](/models/sql-editor) and [table selector](/models/table-selector) docs for details on how to use those modeling methods.
Refer to the Extensions documentation to learn more about how to connect Hightouch to these platforms and build Hightouch models using existing [dbt models](/extensions/dbt-models), [Looker Looks](/extensions/looker-models), or [Sigma workbooks](/extensions/sigma).
## Unique primary key requirement
Regardless of which method you use to build your models, you must always designate a column as the model's primary key, which is a non-repeating, non-null value that uniquely identifies each row. For example, if you have a table containing customer profiles, you might have a column named `CustomerID` that distinguishes each customer from all the others. (While several customers can share the same first name, location, or even phone number, they cannot share the same `CustomerID`.)
Select your primary key column when prompted during model setup.
If your dataset doesn't include any truly unique columns, you can use the [SQL editor](/models/sql-editor) to either [filter out](/models/sql-editor#filter-out-duplicates) duplicate rows or create a [composite column](/models/sql-editor#composite-primary-keys) to use for your primary key. If you're not using the SQL modeling method or if SQL isn't supported by your source, you need to make changes to your data upstream to ensure it includes a unique column.
Rows with `null` primary keys will be ignored by the sync.
Hightouch's sync engine relies on unique primary keys to detect when rows are added, changed, or removed in a data model. See the [change data capture](/syncs/cdc) docs to learn how Hightouch accomplishes this.
As a safeguard to prevent duplicate records from reaching to your destination, **Hightouch will automatically filter out rows with non-unique primary keys**. In the sync logs, they will be marked as “Rejected”, just like any other row-level error. It is safer to reject these rows than to risk syncing wrong or duplicate data.
Make sure to read through the [primary key updates](#updating-a-primary-key) section before making any changes to your primary key.
### Updating a primary key
Hightouch relies on unique primary keys to detect when rows are added, changed, or removed. Therefore, altering the primary key requires Hightouch to reset its [change data capture (CDC)](/syncs/cdc) for syncs that depend on your model.
When changing a primary key, you will be asked whether or not a backfill should be performed. Two options are available:
**Reset CDC without backfill**: By electing to reset CDC without a backfill, you are instructing Hightouch to ignore the current state of your model and to track only rows that are added, changed, or removed in the future. The next sync run will be used as an opportunity to capture a snapshot of your model, but no data will be sent to the destination. Thereafter, future sync runs will track new changes only. (This option is recommended for insert-only use cases like conversion events, operational alerts, and other situations where a backfill would create undesired duplicate records. Beware that skipping the backfill may cause your destination to drift out of sync with your data model.)
**Reset CDC with backfill (i.e., [full resync](/syncs/overview#resync-full-query))**: By electing to reset CDC with a backfill, you are instructing Hightouch to perform a full resync of your entire data model. This means that Hightouch will not perform any diffing and will instead process every row as if it's the first time the sync has ever run. All rows will be sent to the destination again. (This option may be preferred for syncs to CRMs and other services that are configured to update existing records instead of creating new ones. Do not use this option if a backfill will create duplicates or trigger downstream actions.)
By default, proceeding with your primary key change will immediately trigger a run for every sync that depends on your model. It is necessary to reset CDC as quickly as possible after the primary key is updated. However, you can opt out of this behavior, in which case the CDC reset will be deferred until the next sync run.
You don't need to manually trigger a full resync if you change the primary key column's [data type](/models/data-types-casting#view-model-data-types). If you change the primary key's data type in the [model configuration](/models/data-types-casting#casting-from-your-model-configuration), your sync will process normally. If you make this change in your source or in the [SQL editor](/models/sql-editor#unique-primary-key-requirement), the entire model query result set is automatically resynced as if you triggered a full resync. As outlined in the [full resync prerequisites](https://hightouch.com/docs/syncs/overview#full-resync-prerequisites) section, this can create duplicates in your destination data. You may wish to reset change data capture without a backfill instead.
## Row and column ordering
Hightouch’s sync engine is optimized for performance and scale. It divides data into batches and processes them in parallel so large syncs complete quickly. This distributed design means rows and columns may be sent to destinations in a different order than they appear in your model.
This is normal behavior and does not affect which records are synced or the values written. If you notice that files or datasets appear in a different order between runs, that variation is expected. Any apparent ordering you observe is incidental and not guaranteed to remain consistent over time.
This applies to all destination types, including spreadsheets and file uploads. Even if you use a custom SQL model with an `ORDER BY` clause, that ordering applies only when previewing the model. The order in which rows and columns are processed during syncs can still vary.
## Data types and casting
When you define a model, Hightouch doesn't change the [data types](/models/data-types-casting#data-types) found in your [source](/getting-started/concepts#sources) unless otherwise specified.
You can view a column's type in a model's **Columns** tab.
To safeguard your syncs from failing, the data types your model returns must align with your [destination's](/getting-started/concepts#destinations) data type expectations.
If your source data types don't match your destination's expectations, you can use [data casting](/models/data-types-casting#casting) while setting up your models.
Hightouch intentionally stringifies your chosen [**primary key**](/models/creating-models#unique-primary-key-requirement) column for enhanced performance during [change data capture](/getting-started/concepts#change-data-capture-cdc).
If you need to sync the primary key column as a non-string value, use [SQL aliasing](/models/data-types-casting#sql-aliasing) in your model to create a new column specifically for syncing.
## Preview model results
When first defining a model, it's highly recommened to **Preview results** before you continue with setup. Previewing lets you validate the data your model returns before syncing it to downstream destinations.
Previewing a model's query results can also be helpful when troubleshooting issues. To preview an existing model's results, open the model's overview page and click **Edit**.
You can then select to **Preview results**.
If you want to display all model rows, click the gear symbol, then toggle off **Limit preview to 100 rows**.
You can also turn off **Show row count** to improve loading performance for larger data models.
## Column descriptions
Hightouch automatically pulls in column descriptions from supported sources. Column descriptions are supported for
Snowflake, BigQuery , Databricks, and other SQL-based sources that provide the standard `INFORMATION_SCHEMA.COLUMNS`
table, such as PostgreSQL, MySQL, MSSQL, and their cloud-managed variants. You can add descriptions to your models'
columns so that team members (and your future self) can better understand the data. Go to a model's **Columns** tab and
click the a column's **description** to edit it. If you edit a description in Hightouch, your version displays instead
of the comment on the source column.
If you're using the [dbt extension](/extensions/dbt-models) to build models, Hightouch automatically [pulls in columns descriptions](/extensions/dbt-models#model-column-descriptions) from dbt.
### Updating column descriptions from the source
To update column descriptions from your source, you can navigate to any model from that source -->
click the `Refresh columns available in source` button.
Alternatively, you can trigger this refresh by navigating to the table selector and refreshing the table list.
The table selector is available in two places:
1. Activation -> Models -> any model within the source
2. Customer Studio -> Schema -> select a model -> Query -> edit.
Once in the table selector, click the refresh icon to refresh the tables available from that source, this also updates the column descriptions.
## Clone models
After creating a model, you can clone it by opening the horizontal three-dot menu on its overview page and clicking **Clone**.
This opens a new page where you can:
- edit and preview the cloned model's query
- select which syncs to clone, if the model you're cloning has syncs associated with it
- give it a custom name and description
- confirm which model column to use as the [primary key](/models/creating-models#unique-primary-key-requirement)
Once you finish creating the cloned model, you can further edit it and configure additional syncs on it, like any other model.
---
## Data types and casting
**URL:** https://hightouch.com/docs/models/data-types-casting
**Description:** Third-party APIs require the data you send to be of a specific data type. Unless otherwise specified, Hightouch takes the data from your source and leaves the data type unchanged to send to your destination. You can choose to cast data into different types using SQL.
**Section:** Models
## Overview
When syncing data to a destination, their underlying APIs require the data you send to be correctly typed. For example, a `Name` field may require a `string` data type, while a `timestamp` field may require a particular `datetime` format. If you try to sync an incorrect type, for example, a `number` type for an identifier, when a `string` type is expected, APIs usually reject the request, and the sync fails.
When you define a [model](/getting-started/concepts#models), Hightouch doesn't change the data types found in your [source](/getting-started/concepts#sources) unless otherwise specified. To safeguard your syncs from failing, the data types your model returns must align with your destination's data type expectations. If your source data types don't match your destination's expectations, you can use [casting](#casting) while setting up your models.
## Data types
Hightouch recognizes the following data types for [model columns](#view-model-data-types):
- Boolean
- Number
- String
- Timestamp
- Date
- Object / Array
And the following for [destination fields](#view-destination-data-types):
- Boolean
- Number
- String
- Timestamp
- Date/time
- Associated field (linked to another table)
- Null value
- Unknown type
The Hightouch UI uses these icons to distinguish various data types for destination fields:
### View model data types
If you don't know the data types of the columns returned from your model, you can view them from the **Columns** tab on the model's overview page:
You can also [change the data types](#casting-from-your-model-configuration) returned from a query here.
Make sure to read through the [primary key
updates](/models/creating-models#updating-a-primary-key) section before making
any changes to your primary key.
Hightouch intentionally stringifies your chosen [**primary
key**](/models/creating-models#unique-primary-key-requirement) column for
enhanced performance during [change data
capture](/getting-started/concepts#change-data-capture-cdc). If you need to sync
the primary key column as a non-string value, use [SQL
aliasing](#sql-alaising) in your model to create a new column specifically for
syncing.
### View destination data types
When creating a sync configuration, you can view the destination fields' expected data types by looking at their icons when setting up [field mappings](/syncs/mapping-data):
## Casting
For your data to reach its destination without error, its data type must match the data type required by the destination. You can use **[type casting](https://en.wikipedia.org/wiki/Type_conversion)** to change column data types. You have these options:
- Click and point casting from a model's **Columns** tab: suitable for all [modeling methods](/models/creating-models)
- Casting via SQL in your model definition: suitable if you've built your model using the [SQL editor](/models/sql-editor) and prefer to encode casting in your SQL query
- SQL aliasing: recommended if your [primary key](/models/creating-models#unique-primary-key-requirement) data type is incompatible with the expected destination data type
### Casting from your model configuration
1. From the [**Models** overview page](https://app.hightouch.com/models?from=docs), select the model whose columns you want to cast.
2. Select the **Columns** tab.
3. Use the picklist to the right of each column to choose the data type you want for that column.
### Casting in SQL
You can cast values from one data type to another when writing the SQL query that defines a model.
For example, this query casts a column called `total_storage_used_mb` as an integer:
```sql
SELECT *,
CAST (total_storage_used_mb AS int)
FROM public.organizations;
```
For more on SQL casting, see [this guide on SQL type casting](https://learn.microsoft.com/en-us/sql/t-sql/functions/cast-and-convert-transact-sql?view=sql-server-ver16/).
### SQL aliasing
If your model's primary key isn't a string, Hightouch stringifies it for enhanced record-matching performance. If stringification makes your primary key incompatible with the data type your destination is expecting, you can use **SQL aliasing** to create an additional field for mapping.
For example, if you use a field called `id` as your primary key, Hightouch stringifies its value to perform the matching and [diffing operations](/getting-started/concepts#change-data-capture-cdc).
Many destinations expect a field called `id` to be a string. Some destinations, however, expect `id` to be an integer. This mismatch causes a data type conflict if you need to map your primary key in addition to using it for matching.
Using SQL aliasing and casting can solve this issue. This example query selects the `customer_id` column twice. The second time, it casts the column as an `int`, and aliases it as a new `id_for_mapping` column.
```sql
SELECT customer_id, CAST(customer_id AS int) AS id_for_mapping
FROM customers;
```
The query results will now include a string `customer_id` column and an integer `id_for_mapping` column. You can then use `customer_id` as the primary key in your model setup and `id_for_mapping` for field mapping in a sync configuration.
---
## Match Booster on models
**URL:** https://hightouch.com/docs/models/match-boosting
**Description:** Configure Match Booster on a model to enrich your data with additional identifiers.
**Section:** Models
This page has moved. See [Configure Match Booster on a model](/match-booster/configure-model) for the full setup guide.
---
## Resolve SQL compilation errors
**URL:** https://hightouch.com/docs/models/sql-compilation-errors
**Description:** Troubleshoot SQL compilation errors that appear in model previews, Customer Studio audiences and journeys, or sync runs.
**Section:** Models
SQL compilation errors in Hightouch usually come from your warehouse when Hightouch runs a query to preview a model, compute an audience, evaluate journey logic, or execute a sync.
In most cases, Hightouch is surfacing the warehouse's error message directly.
## Where these errors appear
You may see SQL compilation errors in:
- Model preview and model edits
- Customer Studio audience preview or count calculations
- Customer Studio journeys that evaluate audience/event conditions
- Sync run failures
## Quick checklist
When you hit a SQL compilation error:
1. Copy the full error message and note object names, column names, and line/position details.
2. Run the same SQL in your warehouse using the same role/context as Hightouch.
3. Classify the error pattern below.
4. Fix the issue at the right layer (warehouse object/permissions vs model/filter/mapping config).
5. Re-run model preview, then re-run the audience/journey/sync flow.
## Common error patterns
### Invalid identifier (including `_ht_to_filter`)
Example patterns:
- `SQL compilation error: invalid identifier ...`
- `invalid identifier '"_ht_to_filter"."COLUMN_NAME"'`
What it means:
- A referenced column is missing from the query result Hightouch is filtering or mapping.
Common causes:
- Column renamed or removed in model SQL.
- Audience filter, trait, journey condition, or sync mapping still references old column.
How to fix:
1. Confirm the column still exists in model output.
2. Update filters/conditions/mappings that reference the old name.
3. Preview and re-save the model if metadata is stale.
### Unresolved column with suggestions
Example pattern:
- `[UNRESOLVED_COLUMN.WITH_SUGGESTION] ... cannot be resolved. Did you mean ...`
What it means:
- A referenced column name does not exist in the active query output.
Common causes:
- Typo in a field name.
- Renamed/removed model column (for example after changing from `SELECT *` to an explicit column list).
- Stale audience, trait, journey filter, or suggestion-job configuration.
How to fix:
1. Verify the column exists in the parent model/event model output.
2. Preview and re-save the model to refresh available columns.
3. Update filter logic to use valid column names.
### Ambiguous column name
Example pattern:
- `SQL compilation error: ambiguous column name 'COLUMN_NAME'`
What it means:
- The final query references a column name that exists in multiple joined sources, and the warehouse cannot infer which one you intended.
Common causes:
- Multiple tables expose the same column name and the model output does not disambiguate with aliases.
- Downstream filters/mappings reference a shared name after query wrapping.
How to fix:
1. Alias ambiguous columns in model SQL (for example, `a.faq_url AS faq_url_from_a`, `b.faq_url AS faq_url_from_b`).
2. Update filters, traits, journeys, and sync mappings to reference the intended alias.
3. Re-run preview and dependent workflows.
### Object does not exist or not authorized
Example pattern:
- `Object 'DB.SCHEMA.OBJECT' does not exist or not authorized`
What it means:
- Warehouse object is missing, moved, renamed, or inaccessible to the configured role.
How to fix:
1. Verify object path (database/schema/object) is correct.
2. Confirm the Hightouch role has required `USAGE`/`SELECT` privileges.
3. Update model SQL or restore the object.
### View definition column mismatch
Example pattern:
- `View definition ... declared N column(s), but view query produces M column(s)`
What it means:
- The warehouse view definition is out of sync with the underlying query.
How to fix:
1. Recreate or repair the view in the warehouse.
2. Prefer explicit column lists over `SELECT *` in long-lived views.
3. Re-test the view in warehouse SQL before re-running in Hightouch.
### Invalid data type for predicate / row comparison
Example pattern:
- `Invalid data type ... for predicate [ROW(...)]`
What it means:
- Filter predicates compare incompatible data types.
How to fix:
1. Inspect predicate/filter logic for mixed types.
2. Align types with explicit `CAST(...)` where needed.
3. Re-test in warehouse, then in Hightouch.
### Works in warehouse UI but fails in Hightouch
What to check:
- Role differences between your user and Hightouch connection.
- Different default database/schema/warehouse context.
- Model points to a different object/version than expected.
- SQL succeeds in UI context but fails under Hightouch role with a standard pattern above.
How to fix:
1. Re-run under the same role/context as Hightouch.
2. Use fully qualified names (`DATABASE.SCHEMA.OBJECT`) in SQL.
3. Confirm Hightouch model references the intended object.
## When to contact support
Contact support with:
- Link to model/audience/journey/sync run
- Full SQL compilation error text
- Whether the error reproduces in warehouse SQL with the same role/context
---
## Create models with the SQL editor
**URL:** https://hightouch.com/docs/models/sql-editor
**Description:** Create Hightouch models using the SQL editor.
**Section:** Models
With the SQL editor modeling method, you can write a SQL query to define your model. The SQL editor can execute whatever SQL is native to your warehouse or database.
If you encounter warehouse query failures while previewing or running downstream syncs, see [Resolve SQL compilation errors](/models/sql-compilation-errors).
Your query depends on the data in your model. For example, if you are using the [Sample B2B SaaS source](/sources/sample-data), you can use the following query to select the entire `public.users` table:
```sql
SELECT * FROM public.users
```
Before you can continue, you must click **Preview** to see a preview of the data. By default, Hightouch limits the preview to the first 100 rows. Once you've confirmed that the preview contains the data you expect, click **Continue**.
You can't save a model if your query doesn't return any results. If you need to save a model with such a query because you expect there to be results in the future, see the [save a model without results section](#save-a-model-without-results).
The last step for model setup is to enter a descriptive **Name** and select the column to use as a [**Primary key**](/models/creating-models#unique-primary-key-requirement). You can optionally select a [folder](/workspace-management/folders-filters#folders) to move the model to.
Click **Finish** to complete your model setup.
## Unique primary key requirement
All models require a [unique **primary key**](/models/creating-models#unique-primary-key-requirement).
If your dataset doesn't inherently include any truly unique columns, you can use SQL to either:
- [filter out](#filter-out-duplicates) duplicate rows, if you're not concerned about losing data from one or more of the duplicated rows
- create a [composite column](#composite-primary-keys) to use for your primary key
As explained in the [primary key updates](/models/creating-models#updating-a-primary-key) section, if you alter a model's primary key by selecting a different column, you will be prompted to perform a full sync or reset the change data capture for all syncs that depend on that model. Be careful when modifying primary keys. If you keep the same column name but alter the way its values are calculated, some records may be added or deleted in your destination, depending on how your sync is configured.
Hightouch intentionally stringifies your chosen [**primary key**](/models/creating-models#unique-primary-key-requirement) column for enhanced performance during [change data capture](/getting-started/concepts#change-data-capture-cdc).
If you need to sync the primary key column as a non-string value, use [SQL aliasing](/models/data-types-casting#sql-aliasing) in your model to create a new column specifically for syncing.
### Filter out duplicates
When you write a `SELECT` statement to define your model, you can use a `GROUP BY` clause to retrieve only the unique rows based on the column you want to use for your primary key:
```SQL
SELECT *
FROM users
GROUP BY primary_key_column
```
This query returns only one row for each unique `primary_key_column` value, while ignoring the duplicates.
This query returns one row for each `primary_key_column` value, so if there are multiple rows with the same `primary_key_column` value, only one of them will be returned.
If you want your model's query results to return every row in your dataset, it's better to create a [composite primary key](#composite-primary-keys).
### Composite primary keys
If your dataset doesn't inherently include a truly unique primary key, you can create one. For example, in a CRM, you may have users that are identified by email, but belong to multiple organizations:
{/* */}
| org_id | email | created_at |
|--------|-------------------|------------|
| 3 | alice@example.com | 2023-01-18 |
| 1 | bob@example.com | 2022-05-06 |
| 3 | bob@example.com | 2022-12-13 |
| 2 | carol@example.com | 2023-03-29 |
The user `bob@example.com` belongs to both `org_id:1` and `org_id:3`. Their email address is unique within each organization but non-unique across organizations. Therefore, if you were to select `email` as the primary key, downstream destinations receiving the data would likely reject these rows.
To ensure a sync with this type of data includes all records, you can use a **hash function** for the model's primary key. The hash should combine enough columns in the data to create a unique value, for example, `org_id` and `email`.
```SQL
SELECT org_id,
email,
created_at,
HASH(CONCAT(org_id, '-', email)) AS composite_key
FROM users
```
This query selects the `org_id`, `email`, and `created_at` columns, and creates a `composite_key` column.
The composite_key column contains a hash of the concatenation of `org_id` and `email`, creating a unique value for each row.
| org_id | email |created_at |composite_key|
|--------|-------------------|------------|-------------|
| 3 | alice@example.com | 2023-01-18 | 123456780 |
| 1 | bob@example.com | 2022-05-06 | 987654321 |
| 3 | bob@example.com | 2022-12-13 | 709230458 |
| 2 | carol@example.com | 2023-03-29 | 382910382 |
{/* */}
You would then select the `composite_key` column as the primary key for the model.
Because the composite keys for the `bob@example.com` rows are unique, Hightouch would sync both rows to the destination.
### Save a model without results
You can't save a model if your query doesn't return any results.
If you need to save a model with such a query because you expect there to be results in the future, add the following SQL to the end of your query:
```sql
UNION ALL
select 'ignore', 'ignore', 'ignore'
```
This SQL adds a row of 'ignore' values that don't match against anything in your destination but are always present so you can save the query.
If your column datatype isn't a `string`, you can replace 'ignore' with a `null` value.
The number of 'ignore' or `null` values needs to be the same as the number of columns in your model.
### Exclude rows with null values
Some [destinations](/getting-started/concepts#destinations) reject requests when required fields contain `null` values.
You can prevent syncs from sending `null` values using the [**Don't sync null values** option](/syncs/mapping-data#dont-sync-null-values) in the [advanced mapper](/syncs/mapping-data#advanced-mapper).
If you want to exclude rows with `null` values at a model level, you can include the following SQL in your model defintiion:
```sql
SELECT *
FROM your_table
WHERE column1 IS NOT NULL AND column2 IS NOT NULL AND column3 IS NOT NULL;
```
Be sure to change `column1`, `column2`, etc. to your column names and to include any columns where you believe there could be `null` values.
---
## Create models with the table selector
**URL:** https://hightouch.com/docs/models/table-selector
**Description:** Create Hightouch models using the table selector which queries existing tables and views within your source.
**Section:** Models
With the table selector modeling method, you can build Hightouch models using existing tables in your source without writing any SQL.
The first time you use the table selector you may not see your tables populated. Click the **Refresh** button to the left of the search bar and your tables should appear.
Start by selecting the table or view that you want to use as your model.
Before you can continue, you must click **Preview** to see a preview of the data. By default, Hightouch limits the preview to the first 100 rows. Once you've confirmed that the preview contains the data you expect, click **Continue**.
The last step for model setup is to enter a descriptive **Name** and select the column to use as a [**Primary key**](/models/creating-models#unique-primary-key-requirement). You can optionally select a [folder](/workspace-management/folders-filters#folders) to move the model to.
Click **Finish** to complete your model setup.
---
## Athena
**URL:** https://hightouch.com/docs/sources/amazon-athena
**Description:** Amazon Athena is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. Athena is serverless, so there is no infrastructure to manage, and you pay only for the queries that you run.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in Amazon Athena and push it to downstream destinations. Connecting Hightouch to Amazon Athena requires some setup in both platforms. In Amazon, you need to make sure the AWS credentials you will use to connect Hightouch has the correct permissions.
## Required permissions
The AWS user that you use to connect to Athena must have the following permissions:
Read and list permissions for Athena resources and execute permissions for Athena queries:
- `ListWorkGroups`
- `ListDataCatalogs`
- `ListDatabases`
- `ListTableMetadata`
- `StartQueryExecution`
- `GetQueryExecution`
- `GetQueryResults`
- `GetQueryResultsStream`
Read and write permissions for the output location in S3 to both write the results to S3, read them back, and perform cleanup:
- `PutObject`
- `ListBucket`
- `ListBucketMultipartUploads`
- `ListMultipartUploadParts`
- `GetBucketLocation`
- `GetObject`
- `AbortMultipartUpload`
- `DeleteObject`
See the AWS managed policy [`AmazonAthenaFullAccess`](https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AmazonAthenaFullAccess.html)
for reference.
If you are using Lake Formation for [managing permissions](https://docs.aws.amazon.com/lake-formation/latest/dg/managing-permissions.html) you need to [grant database permissions](https://docs.aws.amazon.com/lake-formation/latest/dg/granting-database-permissions.html) to the AWS user that you use to connect to Athena, otherwise no databases load when configuring the source.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Amazon Athena** and follow the steps below.
### Configure your credentials
You can either select AWS credentials you've previously configured in Hightouch or choose to add **New credentials** directly from this page.
The credentials must be for a user who has permission to access Athena and the S3 output location. Refer to the [required permissions](#required-permissions) section for permission details and to the [AWS credential configuration](/security/aws) docs for credential setup information.
### Configure your Amazon Athena source
Enter the following fields into Hightouch:
- Athena **workgroup** you previously set up in AWS.
- (Optional) **Query output location**—you only need to enter this if your workgroup doesn't have an output location configured. This is an S3 path.
- **Data catalog** name
- **Database** name
- (Advanced) **VPC connection endpoint**-if you wish to connect to Athena through a private VPC endpoint, please contact Hightouch and we can provide the endpoint necessary for this step.
Workgroup and data catalog options populate depending on your AWS credentials. If you don't see the expected values, confirm your AWS credentials and click **Refresh**. Your database options populate depending on your selected data catalog.
### Choose your sync engine
#### Basic versus Lightning engine comparison
#### Lightning engine setup
Hightouch uses two schemas (`hightouch_planner` and `hightouch_audit`) for storing logs of previously synced data. Hightouch must be able to read and write to these schemas, but the specific schema names might vary.
```sql
-- Required for the Lightning engine
CREATE SCHEMA IF NOT EXISTS .hightouch_planner;
-- For sync logs, optional
CREATE SCHEMA IF NOT EXISTS .hightouch_audit;
```
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Athena.
The Athena source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Looker Looks](/extensions/looker-models)
## Data types
Hightouch parses most Athena data types into JavaScript types before sending them to your destination. Hightouch leaves the following Athena data types as strings:
- `BIGINT`
- `ARRAY`
- `MAP`
Read more about Athena data types in the [Athena documentation](https://docs.aws.amazon.com/athena/latest/ug/querying-iceberg-supported-data-types.html).
## Tips and troubleshooting
### Performance considerations
The following Service Quota changes to Amazon Athena are recommended in order to support fast and reliable syncs:
- Active DML queries: 400
- To support concurrent `CREATE TABLE AS` queries which are [DML statements](https://docs.aws.amazon.com/athena/latest/ug/create-table-as.html).
---
## Redshift
**URL:** https://hightouch.com/docs/sources/amazon-redshift
**Description:** Amazon Redshift is a fully managed, petabyte-scale data warehouse service in the cloud. It is built on Amazon Web Services
**Section:** Sources
## Overview
Hightouch lets you pull data stored in your Amazon Redshift data warehouse and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to Redshift for information like your host, port, database name, and credentials.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Amazon Redshift** and follow the steps below.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your Redshift cluster. The hostname can be found by visiting the [Redshift web console](https://console.aws.amazon.com/), navigating to the **Clusters** panel, and clicking your cluster. Copy the **Endpoint string**, excluding the port and database name.
- **Port**: The port number of your Redshift cluster. The default is 5439, but yours may be different. To confirm, visit the [Redshift web console](https://console.aws.amazon.com/), navigate to the **Clusters** panel, and click your cluster. The port number is shown in the **Properties** tab.
- **Database**: The name of the database in your Redshift cluster. Most clusters have only one database. Visit the [Redshift web console](https://console.aws.amazon.com/), navigate to the **Clusters** panel, and click your cluster. The database name is shown in the **Properties** tab.
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
To set up the Lightning engine, you need to grant Hightouch write access to Redshift. You can do so by running the following SQL snippet.
```sql
CREATE USER hightouch_user WITH PASSWORD '********';
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT CREATE, USAGE ON SCHEMA hightouch_audit TO hightouch_user;
GRANT CREATE, USAGE ON SCHEMA hightouch_planner TO hightouch_user;
```
The snippet creates a dedicated Amazon Redshift user for Hightouch. It also provisions two schemas, `hightouch_planner` and `hightouch_audit`, for storing logs of previously synced data.
### Provide credentials
Enter the following fields into Hightouch:
- **User**: This can be your personal Redshift login or a dedicated user for Hightouch. At minimum, this user must have read access to the data you wish to sync. If using the Lightning sync engine, you must also grant this user additional permissions as described above.
- (Optional) **Password**: The password for the user specified above.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Amazon Redshift.
The Amazon Redshift source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Looker Looks](/extensions/looker-models)
- leveraging existing [Sigma workbooks](/extensions/sigma)
## Workload management
Amazon Redshift offers powerful and flexible settings for [workload management (WLM)](https://docs.aws.amazon.com/redshift/latest/dg/cm-c-implementing-workload-management.html).
Hightouch cannot automatically manage your WLM rules, but does enable the use of WLM to manage resource use by Hightouch by
[setting query groups](https://docs.aws.amazon.com/redshift/latest/dg/cm-c-executing-queries.html#cm-c-executing-queries-assigning-a-query-to-a-query-group).
Hightouch will automatically set the query group to either
- `ht_interactive` for interactive queries which need low latency, such as running [model previews](/models/creating-models#preview-model-results) or schema discovery
- `ht_batch` for batch operations that run in the background, such as [rETL syncs](/syncs/overview)
If no WLM configuration references these query groups, all Hightouch queries will run using Redshift’s default queue behaviour.
## Tips and troubleshooting
### Spectrum nested query error
You may receive a `Spectrum nested query error` if you are using [Amazon Redshift Spectrum](https://docs.aws.amazon.com/redshift/latest/dg/c-getting-started-using-spectrum.html) as your data source.
Hightouch uses a wrapper query around your model query for obtaining the `COUNT` of query results. Spectrum has [limitations around nested data](https://docs.aws.amazon.com/redshift/latest/dg/nested-data-restrictions.html) that this can causes issues with.
To resolve the error, you can try the following:
- Create a [materialized view](https://docs.aws.amazon.com/redshift/latest/dg/materialized-view-overview.html) of the results set you want Hightouch to query.
- Add a commented out `-- ORDER BY` statement to your model definition. Hightouch's Redshift source disables the row counter wrapper query if there's an `ORDER BY` in the query. However, Redshift Spectrum doesn't allow `ORDER BY` statements, hence the need to comment it out. Even commented out, Hightouch disables the wrapper query.
### Error: could not identify an ordering operator for type "unknown"
You may receive a this error if you are using many UNION set operators within your SQL query.
To resolve the error, you can try replacing UNION with UNION ALL.
To learn more about UNION and UNION ALL, check out the article by [The Data School](https://dataschool.com/learn-sql/what-is-the-difference-between-union-and-union-all/#:~:text=The%20main%20differenc[…]ds%2C%20including%20duplicates).
### Error: permission denied for relation 'objectname'
You may receive this error when trying to access newly created objects in the schema and you lack permissions to do so. This error happens when access is granted for only the objects present in a schema when the access was first granted. By default, access isn't automatically granted for objects that are created under the current schema.
To learn about how to grant permissions you can check the AWS documentation [here](https://repost.aws/knowledge-center/redshift-grant-permissions-new-objects).
---
## Azure Blob Storage
**URL:** https://hightouch.com/docs/sources/azure-blob
**Description:** Azure Blob Storage helps you create data lakes for your analytics needs, and provides storage to build powerful cloud-native and mobile apps. Optimize costs with tiered storage for your long-term data, and flexibly scale up for high-performance computing and machine learning workloads.
**Section:** Sources
## Overview
Hightouch lets you pull data from CSV and JSON files stored in Azure Blob Storage and push them to downstream destinations. To get started, you need an Azure Blob Storage account and container.
## Azure credential setup
See the guide for [configuring Azure credentials](/security/azure) to learn how you can set up a service account and use its credentials. The IAM user whose credentials you use must have programmatic access enabled and permission to read from the Blob Storage path you want to use.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Azure Blob Storage** and follow the steps below.
### Configure your service account
Select the credentials you previously created or click **Create new** to set them up now.
### Configure your source
Enter your Blob Storage **Account name** and **Container name**.
## Test your connection
When setting up Azure Blob Storage as a source for the first time, Hightouch validates your Azure credentials and access to your Blob Storage. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from your Blob Storage. The file must have either a `.csv` or `.json` extension.
### CSV requirements
CSV files must meet the following requirements:
- They must have a header row. The values in the header row are automatically available as column names when you set up a sync.
- They must use comma-separated values; tabs and other delimiters aren't supported.
- They must use double quotes (`"`) for quoted values.
### JSON requirements
JSON files must meet the following requirements:
- The input file must contain an array at the top level.
- Each element in the input file must be an object with the same keys.
The keys of the array elements are automatically available as column names when you set up a sync.
### Model setup
1. In Hightouch, go to the [**Models** overview page](https://app.hightouch.com/models?from=docs).
1. Click **Add model.**
1. Select the Azure Blob Storage source you previously created.
1. Enter the **relative path** to the CSV or JSON file that you want to sync data from.
1. Preview your model's query results.
1. Click **Continue**.
1. Name your model and select its [**primary key**](/models/creating-models#unique-primary-key-requirement). Hightouch uses the primary key to determine which rows have been added, changed, or removed since the last sync.
## Tips and troubleshooting
---
## ClickHouse
**URL:** https://hightouch.com/docs/sources/clickhouse
**Description:**
**Section:** Sources
Hightouch only supports ClickHouse v20 and newer.
## Overview
Hightouch lets you pull data stored in [ClickHouse](https://clickhouse.com/) and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to ClickHouse for information like your host, port, database name, and credentials.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **ClickHouse** and follow the steps below.
### Choose authentication method
You can select to use either a **password** or **connection string**. A **connection string** is useful if you need to pass additional configuration options. Hightouch uses the [official ClickHouse JDBC driver](https://github.com/ClickHouse/clickhouse-java/tree/main/clickhouse-jdbc).
If you select **connection string**, enter a JDBC connection string and skip to the [connection test section](#test-your-connection) of this page.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your ClickHouse server.
- **Port**: The port number of your ClickHouse server's HTTP API. The default is 8123, but yours may be different.
- **Database**: The name of the database to use when Hightouch executes queries in ClickHouse.
- **Username**: This can be your personal ClickHouse login or a dedicated user for Hightouch.
- (Optional) **Password**: The password for the user specified above.
## Test your connection
When setting up ClickHouse as a source for the first time, Hightouch checks that you have the correct permissions. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from ClickHouse.
The ClickHouse source support writing queries in [the SQL editor](/models/sql-editor) as its modeling method.
## Tips and troubleshooting
### Table selector fails to populate
{/* Unfortunately, this is entirely opaque to the user at the moment; there's no error information in the UI. */}
If the [table selector](/models/table-selector) fails to populate, it may be that the Clickhouse user
configured for the connection lacks necessary privileges. In order for Hightouch to automatically inspect the
schema and populate the table selector, the user configured for the connection must have `SELECT` permissions
on `INFORMATION_SCHEMA.COLUMNS`.
---
## Databricks
**URL:** https://hightouch.com/docs/sources/databricks
**Description:** Databricks is a data science and analytics platform built on top of Apache Spark. Databricks implement the Data Lakehouse concept in a single unified, cloud based platform.
**Section:** Sources
## Overview
Databricks is a popular source that supports our Reverse ETL, Composable CDP, and AI Decisioning features. This article discusses the steps for connecting to Databricks.
You can connect Databricks to Hightouch using
[Databricks Partner Connect](https://hightouch.com/blog/hightouch-databricks-partner-connect)
to bypass the setup steps outlined below. You can learn more about this in
[Databricks' documentation](https://docs.databricks.com/en/partner-connect/index.html).
## Databricks credential setup
Hightouch allows you to authenticate with Databricks using one of two methods:
Either [OAuth](https://docs.databricks.com/en/dev-tools/auth/oauth-u2m.html) or
a [Personal Access Token (PAT)](https://docs.databricks.com/en/dev-tools/auth/pat.html).
We recommend using OAuth for configuring new connections.
### Configuring a service principal with OAuth
1. Create a [service principal](https://docs.databricks.com/en/admin/users-groups/service-principals.html).
1. In your Databricks workspace, go to the user dropdown at the top right and select **Settings**.
1. In the **Identity and access** page, find **Service Principals** and click **Manage**.
1. Choose **Add new**, give your service principal a name, and click **Add**.
1. Take note of the Service Principal name as well as the **Application id** listed on the Service Principal
Configurations page. This is a UUID (for example, `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`). You may
need this later to complete setup.
1. Create an OAuth secret pair.
1. On the page for the Service Principal created above, to go the Secrets tab.
2. Click **Generate secret**. A dialog will pop up with a **Secret** and a **Client ID**.
Take careful note of these values as it is only visible now; you can't look it up later.
1. Grant the necessary catalog permissions to your new Service Principal. The above steps are sufficient
to connect and authenticate, but the Service Principal may not be _authorized_ to execute
queries against your Databricks catalog. Depending on your Databricks workspace configuration,
it may already be configured with the necessary permissions; but if the connection test fails,
you may need to follow these steps.
1. In the Databricks sidebar on the left, click **Catalog** and locate your schema in the tree view.
2. Select the schema and open the **Permissions** page. Click the **Grant** button.
3. In the **Principals** input box, type the **name** of your new Service Principal.
* If you are using the Lightning sync engine (recommended; see below),
select the **Data Editor** preset or manually grant the following permissions:
* `USE SCHEMA`
* `EXECUTE`
* `READ VOLUME`
* `SELECT`
* `MODIFY`
* `CREATE TABLE`
* Make sure to enable the checkbox at the bottom to "Also grant `USE CATALOG`"
* If you are using the Basic sync engine,
select the **Data Reader** preset or manually grant the following permissions:
* `USE SCHEMA`
* `EXECUTE`
* `READ VOLUME`
* `SELECT`
* Make sure to enable the checkbox at the bottom to "Also grant `USE CATALOG`"
1. You may also need to configure _warehouse_ permissions. As with the previous step
concerning _catalog_ permissions, your Databricks workspace configuration may or may not
automatically grant the Service Principal permission to use the compute resources.
You can read more about it in the [Databricks docs](https://docs.databricks.com/en/compute/clusters-manage.html#compute-permissions).
1. In the Databricks sidebar, click **Compute** and find your data warehouse. It may
be located on the **SQL warehouses** page.
1. On the far right column of the row corresponding to the desired warehouse, click
the icon for the kebab menu **⋮**, then **Permissions**.
If you do not see the **Permissions** option, you may lack the privileges
necessary to configure permissions—even if you are a Databricks workspace
admin. If this happens, please contact your Databricks account admin.
1. Enter the name of your Service Principal and ensure that the setting in the
dropdown is set to **Can use**. Click **Add**.
### Alternative: Configuring a Personal Access Token (PAT)
1. In your Databricks Account console, go the [**Workspaces** page](https://accounts.cloud.databricks.com/workspaces).
Select the relevant workspace and then click **Open workspace**.
2. In your workspace, go the **Compute** page and click the relevant cluster. This brings you to its **Configuration** tab.
3. At the bottom of the page, expand the **Advanced Options** toggle and open the **JDBC/ODBC** tab.
4. This tab displays your cluster's **Server Hostname**, **Port**, and **HTTP Path**, which you need to connect to Hightouch.
Keep the tab open, or save these to a secure location.
5. Create a **Personal access token** by following [the Databricks documentation](https://docs.databricks.com/dev-tools/auth.html#pat).
Once you've saved these Databricks credentials, you're ready to set up the connection in Hightouch.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button.
Select **Databricks** and follow the steps below.
### Configure your source
Enter the following required fields into Hightouch:
- **Server hostname**: To find your server hostname, visit the Databricks web console and locate your cluster.
Then, click to reveal Advanced options and navigate to the JDBC/ODBC tab.
- **Port**: The default port number is 443, but yours may be different.
To find your port, visit the Databricks web console and locate your cluster.
Then, click to reveal Advanced options and navigate to the JDBC/ODBC tab.
- **HTTP path**: To find your HTTP path, visit the Databricks web console and locate your cluster.
Then, click to reveal Advanced options and navigate to the JDBC/ODBC tab.
You can choose to route all queries to a single HTTP path, or configure separate HTTP paths optimized for different workloads (recommended):
- **Route all queries to a single HTTP path**: Use this option if you want to use a single HTTP path for all Hightouch operations.
- **Route queries to different HTTP paths (split by workload)** (Recommended): This option allows you to optimize performance by routing different types of queries to different HTTP paths:
- **Batch HTTP path**: For background jobs like sync runs, schema sampling, audience snapshots, and other workloads that are not latency-sensitive.
- **Interactive HTTP path**: For latency-sensitive queries (e.g., audience previews) and heavier workloads that need more compute (e.g., identity resolution).
- (Optional) **Catalog**: You can optionally include a Databricks catalog.
Catalogs are the first level of [Unity Catalog](https://docs.databricks.com/aws/en/data-governance/unity-catalog/)'s three-level namespace.
If Lightning engine is enabled, the 'hightouch_audit' and 'hightouch_planner' [schemas](https://hightouch.com/docs/syncs/lightning-sync-engine#warehouse-schemas)
need to be created in the catalog specified here. If this field is left empty,
these schemas should be created in the [default catalog](https://docs.databricks.com/aws/en/catalogs/default).
Note that data in other catalogs can still be accessed for model creation by specifying
the full name (catalog.schema.table) in the Hightouch SQL interface.
- **Schema**: The initial schema to use for the connection.
- **SQL dialect**: By default, Hightouch assumes that your queries use the Databricks SQL dialect.
You may wish to override this behavior if your queries use legacy ANSI SQL-92 syntax.
Some features are not available when legacy syntax is used.
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse,
which makes its setup more involved than the Basic sync engine.
However, it is more performant and reliable than the Basic engine.
This makes it the ideal choice to guarantee faster syncs, especially with large data models.
It also supports more features, such as
[Warehouse Sync Logs](/syncs/warehouse-sync-logs),
[Match Booster](/match-booster/overview),
and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
To set up the Lightning engine, ensure that your Databricks user or service principal has the appropriate permissions.
You can do so by running the following SQL snippet.
Before running the snippet, make sure to include your own username, which can be found in the top right corner
of the Databricks web console. Alternatively, you can create an API-only service principal by following
[these instructions](https://docs.databricks.com/administration-guide/users-groups/service-principals.html).
{/* Conflicting info: Used to have */}
{/* GRANT OWNERSHIP ON SCHEMA hightouch_audit TO ROLE identifier($ht_default_role); */}
{/* We don't really know which is correct; the below mirrors in-app instructions. */}
```sql
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT ALL PRIVILEGES ON SCHEMA hightouch_audit TO ;
GRANT ALL PRIVILEGES ON SCHEMA hightouch_planner TO ;
```
{/* We've had users confused about permissions around "temporary tables", which they seem */}
{/* to have interpreted as some kind of `CREATE TEMPORARY TABLE` or equivalent. */}
The `hightouch_audit` and `hightouch_planner` schemas exist to keep Hightouch's temporary tables, logs, and other
artifacts separate from your source data. This simplifies governance because you can restrict write access to only
these Hightouch-managed schemas. We recommend using `GRANT ALL PRIVILEGES` for simplicity, but you can also grant the
[required privileges](#configuring-a-service-principal-with-oauth) separately to achieve the same effect.
(“Temporary tables” here refers to tables that Hightouch will automatically clean up once they are no longer
needed; it doesn't necessarily refer to special table types in the warehouse.)
If you are using a Service Principal to connect, you may need to specify the user
above by UUID rather than name. For example, if you have created a Service Principal
named `test_principal` with an **Application Id** of `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`,
you may need to run
```sql
GRANT ALL PRIVILEGES ON SCHEMA hightouch_audit TO `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`;
GRANT ALL PRIVILEGES ON SCHEMA hightouch_planner TO `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`;
```
### Provide credentials
Enter the **Access token** you generated in the [Databricks credential setup](#databricks-credential-setup).
### Unity Catalog, external locations, and managed storage
The Lightning sync engine persists change data capture state in managed tables under `hightouch_planner` and `hightouch_audit`. When Unity Catalog manages these tables, Hightouch can apply extra optimizations during the CDC phase that improve throughput and stability on large models.
Hightouch only supports S3 for self-hosted managed storage today. If your workspace uses a cloud provider other than AWS, contact Support before proceeding.
#### Prerequisites
- You plan to use Lightning with Databricks Unity Catalog.
- You have the [service principal](#configuring-a-service-principal-with-oauth) Hightouch uses to connect, including its Application id for `GRANT` statements.
- You have a metastore or account administrator (or delegate) who can create Unity Catalog objects, specifically:
- To create the [storage credential](https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/manage-storage-credentials): `CREATE STORAGE CREDENTIAL` on the metastore.
- To create the external location: `CREATE EXTERNAL LOCATION` on both the metastore and the storage credential that backs it.
- To create the Lightning schemas: `CREATE MANAGED STORAGE` on the external location plus `USE CATALOG` and `CREATE SCHEMA` on the catalog ([Create schemas](https://docs.databricks.com/aws/en/schemas/create-schema)).
- To run the `GRANT` statements in step 3: `MANAGE` (or ownership) on the external location and storage credential.
- Someone who can attach IAM roles to your bucket path is usually involved ([S3 + Unity Catalog](https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/s3/s3-external-location-cfn)).
#### What to do
1. Pick an S3 path for Hightouch-managed data — typically the same bucket as [self-hosted workspace storage](/security/storage#amazon-s3) (for example `s3://your-company-hightouch/hightouch-databricks/`). Every `MANAGED LOCATION` you set later must sit inside the external location URL you register.
2. Create a storage credential and an external location in Databricks. Follow [CloudFormation quickstart for S3](https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/s3/s3-external-location-cfn), [Create a storage credential](https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/manage-storage-credentials), and [Create an external location](https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/manage-external-locations#create-an-external-location-using-catalog-explorer). Record the external location name and storage credential name.
3. Grant file access to the Hightouch service principal. Replace the names and the Application id (in backticks) below.
```sql
GRANT BROWSE, CREATE EXTERNAL TABLE, READ FILES, WRITE FILES
ON EXTERNAL LOCATION your_hightouch_external_location
TO `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`;
GRANT CREATE EXTERNAL TABLE, READ FILES, WRITE FILES
ON STORAGE CREDENTIAL your_hightouch_storage_credential
TO `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`;
```
Always grant `READ FILES` with `WRITE FILES` — `WRITE FILES` alone returns `PERMISSION_DENIED` ([privilege reference](https://docs.databricks.com/aws/en/data-governance/unity-catalog/manage-privileges/privileges#write-files)).
4. Create the Lightning schemas inside the catalog you configured in the Hightouch source (`my_catalog` below).
If your catalog already has a `MANAGED LOCATION`, omit the `MANAGED LOCATION` clause below — schemas inherit catalog storage automatically ([managed storage hierarchy](https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/managed-storage)).
```sql
CREATE SCHEMA IF NOT EXISTS my_catalog.hightouch_audit
MANAGED LOCATION 's3://your-company-hightouch/hightouch-databricks/audit';
CREATE SCHEMA IF NOT EXISTS my_catalog.hightouch_planner
MANAGED LOCATION 's3://your-company-hightouch/hightouch-databricks/planner';
```
5. Grant catalog and schema privileges to the Hightouch service principal. `USE CATALOG` is required in addition to schema grants; if you didn't already grant it via the [credential UI](#configuring-a-service-principal-with-oauth), include the first statement below.
```sql
GRANT USE CATALOG ON CATALOG my_catalog TO `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`;
GRANT ALL PRIVILEGES ON SCHEMA my_catalog.hightouch_audit TO `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`;
GRANT ALL PRIVILEGES ON SCHEMA my_catalog.hightouch_planner TO `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`;
```
6. Ensure the service principal has `Can use` on the SQL warehouse Hightouch uses ([warehouse permissions](#configuring-a-service-principal-with-oauth)).
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up
[models](/getting-started/concepts#models) to define which data you want to pull from Databricks.
The Databricks source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Sigma workbooks](/extensions/sigma)
The [SQL editor](/models/sql-editor) allows you to query data from all databases that your Databricks credentials
[have access to](#databricks-credential-setup), unless you [specify a catalog](#configure-your-source).
The [table selector](/models/table-selector) only supports querying from the schema specified in
the [source configuration](#configure-your-source).
## Tips and troubleshooting
### Unity Catalog support
Hightouch integrates with Databricks' [Unity Catalog](https://docs.databricks.com/en/data-governance/unity-catalog/index.html) feature.
During [initial configuration](#configure-your-source), include a **catalog** when you use Unity Catalog.
### Subquery error
Some Databricks [models](/getting-started/concepts#models) return the following error:
{/* */}
`java.lang.IllegalArgumentException: requirement failed: Subquery subquery#485, [id=#937] has not finished {...}.`
{/* */}
This error can occur when the model query contains a subquery, especially when the subquery has an `ORDER BY` clause.
For example:
```sql
SELECT user_query.*
FROM (
SELECT * FROM default.subscriptions_table
ORDER BY last_name
) user_query.*
```
The best way to resolve the error is to rewrite your model query to remove subqueries.
[Common table expressions (CTE)](https://docs.databricks.com/sql/language-manual/sql-ref-syntax-qry-select-cte.html) are supported.
If you require assistance, don't hesitate to .
### Error connecting to the database. Unauthorized/Forbidden: 403
This error can occur when [testing the source connection](#test-your-connection)
or when running a sync that uses a Databricks model.
It is typically caused by an expired Databricks access token.
To solve this, [generate a new token](https://docs.databricks.com/dev-tools/auth.html#pat)
and insert it in the source configuration.
---
## Microsoft Fabric
**URL:** https://hightouch.com/docs/sources/fabric
**Description:** Microsoft Fabric is a comprehensive analytics and data integration solution that includes cloud-based data warehousing as part of its extensive suite of capabilities.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in Microsoft Fabric and push it to downstream destinations. Connecting Hightouch to Microsoft Fabric requires some setup in both platforms. In Microsoft Fabric, you need to create a service principal and grant it access to your Fabric resource.
## Service principal creation
First, we need to create a service principal. These are the credentials Hightouch will use to access your Microsoft Fabric resources.
1. Sign in to the [Microsoft Entra admin center](https://entra.microsoft.com/).
2. On the sidebar, select **Identity** > **Applications** > **App registrations**.
3. Select **New registration**.
4. Pick a name for your application and select **Accounts in this organizational directory only** as the supported account type.
5. Click **Register**.
6. Note your newly created **Application (client) ID** and **Directory (tenant) ID**.
7. Click **Add a certificate or secret**.
From here, you can opt to use either a **Client secret** or a **Certificate**.
### Client secret
1. Select the **Client secrets** tab.
2. Click **New client secret**.
3. Name your client secret and select its expiration date. To maintain access to Fabric, make sure to refresh your client secret prior to its expiration.
4. Note your client secret **Value** now as it cannot be retrieved later. It will be used to connect Hightouch to Microsoft Fabric.
### Certificate
1. Select the **Certificates** tab.
2. Click **Upload certificate** and upload a valid signed certificate.
3. Note your certificate's **Thumbprint**.
For more detailed instructions on service principal creation, you can check out [Microsoft's documentation](https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal).
## Fabric configuration
By default, Microsoft Fabric may not be accessible via service principal. To connect, you'll need to take the following steps.
1. Log in to [Microsoft Fabric](https://app.fabric.microsoft.com/).
2. Click **Synapse Data Engineering**.
3. In the top right of the window, select **Settings** > **Governance and insights** > **Admin portal**.
4. On the sidebar, select **Tenant settings**.
5. Scroll down to **Developer settings** and select **Service principals can use Fabric APIs**. Make sure the toggle is set to **Enabled**.
6. Next go to **OneLake settings** and select **Users can access data stored in OneLake with apps external to Fabric**. This toggle should be **Enabled** as well.
## Attaching your service principal to your Fabric warehouse
1. Log in to [Microsoft Fabric](https://app.fabric.microsoft.com/).
2. Select the workspace that contains your warehouse.
3. Hover over your warehouse and select the three dots menu.
4. Choose **Manage permissions** from the menu.
5. On the next page, select **Add user**.
6. Type in the name of your service principal and check the box for **Read all data using SQL (ReadData)**.
7. Select **Grant**.
## Connection configuration
In Hightouch, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Microsoft Fabric** and follow the steps below.
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your Microsoft Fabric instance. If using the hostname, make sure to only insert the hostname and not the full URL (remove the leading `http://` and the final `/`).
- **Port**: The port number of your Microsoft Fabric instance. The default is 1433, but yours may be different.
- **Database**: The name of the database in Microsoft Fabric.
Optionally you can enter a **Request timeout duration** and whether to **Trust server certification**.
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
To set up the Lightning engine, run the following commands. Make sure to replace **your_service_principal** with the name of the service principal created earlier.
```sql
IF NOT EXISTS (SELECT 1 FROM sys.schemas WHERE name = 'hightouch_audit')
BEGIN
EXEC('CREATE SCHEMA [hightouch_audit]');
END;
IF NOT EXISTS (SELECT 1 FROM sys.schemas WHERE name = 'hightouch_planner')
BEGIN
EXEC('CREATE SCHEMA [hightouch_planner]');
END;
GRANT CONTROL ON SCHEMA::hightouch_audit TO [your_service_principal];
GRANT CONTROL ON SCHEMA::hightouch_planner TO [your_service_principal];
```
### Provide credentials
Then you'll enter the credentials from the previous service principal creation step
- **Tenant ID** (or Directory ID)
- **Client ID** (or Application ID)
If you chose to create a **Client Secret** for your service principal, you'll enter it here.
If you instead chose to create a **Certificate**, enter in your certificate's **Thumbprint** and the associated **private key**. Make sure that the private key is in PEM format and includes the header and footer.
### Test your connection
When setting up Microsoft Fabric as a source for the first time, Hightouch validates your credentials and access to your database. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, you've completed setting up Microsoft Fabric as a source. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Microsoft Fabric.
The Microsoft Fabric source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
## Tips and troubleshooting
Most issues with the Microsoft Fabric source can be resolved by verifying that your Microsoft Fabric configuration is correct. It must allow external connections via service principal.
---
## Google Cloud Storage
**URL:** https://hightouch.com/docs/sources/gcs
**Description:** Cloud Storage is a managed service for storing unstructured data. Store any amount of data and retrieve it as often as you like.
**Section:** Sources
## Overview
Hightouch lets you pull data from CSV and JSON files stored in Google Cloud Storage (GCS) and push them to downstream destinations. To get started, you need a GCS bucket and Google Cloud credentials.
## Cloud credential setup
See the guide for [configuring GCP credentials](/security/gcp) to learn how you can set up a service account and use its credentials. The IAM user whose credentials you use must have programmatic access enabled and permission to read from the Cloud Storage path you want to use.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select Google Cloud Storage and follow the steps below.
### Configure your service account
Select the credentials you previously created or click **Create new** to set them up now.
### Configure your source
Enter the **Project ID** or your GCS project and the **Bucket name**. The bucket name should be just the name of the bucket, not a URL.
## Test your connection
When setting up GCS as a source for the first time, Hightouch validates your GCP credentials and access to your GCS bucket. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from your GCS bucket. The file in your GCS bucket must have either a `.csv` or `.json` extension.
### CSV requirements
CSV files must meet the following requirements:
- They must have a `.csv` extension.
- They must have a header row. The values in the header row are automatically available as column names when you set up a sync.
- They must use comma-separated values; tabs and other delimiters aren't supported.
- They must use double quotes (`"`) for quoted values.
### JSON requirements
JSON files must meet the following requirements:
- They must have a `.json` extension.
- The input file must contain an array at the top level.
- Each element in the input file must be an object with the same keys.
The keys of the array elements are automatically available as column names when you set up a sync.
### Model setup
1. In Hightouch, go to the [**Models** overview page](https://app.hightouch.com/models?from=docs).
1. Click **Add model.**
1. Select the GCS source you previously created.
1. Enter the **relative path** to the CSV or JSON file that you want to sync data from, like `path/file.csv`. The path shouldn't contain the bucket name.
1. Preview your model's query results.
1. Click **Continue**.
1. Name your model and select its [**primary key**](/models/creating-models#unique-primary-key-requirement). Hightouch uses the primary key to determine which rows have been added, changed, or removed since the last sync.
## Tips and troubleshooting
---
## BigQuery
**URL:** https://hightouch.com/docs/sources/google-bigquery
**Description:**
**Section:** Sources
## Overview
Hightouch lets you pull data stored in your Google BigQuery warehouse and push it to downstream destinations.
Connecting Hightouch to BigQuery requires some setup in both platforms. It's recommended to set up a service account with the correct permissions in BigQuery before configuring the connection in Hightouch.
## BigQuery credential setup
Setup in BigQuery has three main steps:
1. Enable BigQuery for your Google Cloud project
2. Create a service account
3. Grant the Hightouch service account access to your project
### Create a project and enable BigQuery API
1. Login to the [Google Developers Console](https://console.developers.google.com/).
2. Configure the [Cloud Platform](https://console.cloud.google.com/):
- If you don't have a project already, [create one](https://support.google.com/cloud/answer/6251787?hl=en&ref_topic=6158848).
- Once you have a project, [enable the BigQuery API](https://console.cloud.google.com/flows/enableapi?apiid=bigquery&_ga=2.71379221.724057513.1673650275-1611021579.1664923822&_gac=1.213641504.1673650813.EAIaIQobChMIt9GagtPF_AIVkgB9Ch331QRREAAYASAAEgJfrfD_BwE) for it.
3. Copy your **Project ID** for later use.
4. Find the **location** of your BigQuery dataset or sets. You can find this by querying the `INFORMATION_SCHEMA.SCHEMATA` view or by visiting the [Google Cloud web console](https://console.cloud.google.com/) and clicking on a BigQuery dataset in the Explorer panel. You need both **Project ID** and **Data location** when connecting Hightouch to BigQuery.
Make sure [billing is enabled](https://support.google.com/cloud/answer/6293499#enable-billing) on your project, otherwise Hightouch can't write into the cluster.
### Create a service account
To create a service account, follow the setup instructions in our [Google Cloud Provider (GCP) documentation](/security/gcp).
### Grant access
By default, your GCP service account doesn't have [permission to read data](https://cloud.google.com/bigquery/docs/access-control#permissions-predefined-roles) from BigQuery.
You can set up your service account to have [full access](#grant-full-access) to your project using a [predefined role](https://cloud.google.com/iam/docs/understanding-roles#predefined_roles).
Otherwise, you can create a [custom role](https://cloud.google.com/iam/docs/roles-overview#custom) and provide [limited access](#grant-limited-access) according to a user-specified list of permissions.
#### Grant full access
You can grant full access by assigning the `bigquery.user` and `bigquery.dataViewer` roles to your service account.
You can do this in the [Google Cloud web console](https://cloud.google.com/iam/docs/granting-changing-revoking-access#grant-single-role) or by running these snippets in the [Cloud Shell](https://cloud.google.com/shell/docs/run-gcloud-commands).
Grant permission to **read metadata and list tables**:
```bash
gcloud projects add-iam-policy-binding \
--member serviceAccount: \
--role roles/bigquery.user
```
Grant permission to **read data from tables and views**:
```bash
gcloud projects add-iam-policy-binding \
--member serviceAccount: \
--role roles/bigquery.dataViewer
```
#### Grant limited access
If you don't want to grant full access to your BigQuery service account, you can opt to grant limited access instead.
You can do this by assigning the `bigquery.dataViewer` role only to the specific datasets, tables, or views you want to use in Hightouch.
Since you are assigning the `bigquery.dataViewer` role only to specific resources, you need to assign the `bigquery.user` role **and** grant the `bigquery.tables.get` permission at the project level.
For this, you can create a [custom role](https://cloud.google.com/iam/docs/creating-custom-roles#creating) in the Google Cloud web console based on an [existing predefined role](https://cloud.google.com/bigquery/docs/access-control#bigquery.user) (`bigquery.user`), which you can name `custom.bigquery.user`.
When setting up the custom role, click **Add permissions** to add the `bigquery.tables.get` permission to this custom role.
Then assign this role to your service account at the project level.
You can do this in the [Google Cloud web console](https://cloud.google.com/iam/docs/granting-changing-revoking-access#grant-single-role) or by running this snippet in the [Cloud Shell](https://cloud.google.com/shell/docs/run-gcloud-commands):
```bash
gcloud projects add-iam-policy-binding \
--member serviceAccount: \
--role roles/custom.bigquery.user
```
If this `custom.bigquery.user` role still isn't limited enough, you can try assigning the [`bigquery.jobUser`](https://cloud.google.com/bigquery/docs/access-control#bigquery.jobUser) role **and** granting the `bigquery.dataset.get`, `bigquery.tables.get`, and `bigquery.tables.list` permissions at the project level.
Hightouch needs to be able to list the schemas and tables in your BigQuery project when [creating a model](/models/creating-models).
Therefore, the [source connection test](#test-your-connection) fails if you don't grant the `bigquery.tables.get` permission at the project level.
You can then decide which datasets, tables, or views your GCP service account has access to by [granting access to a resource](https://cloud.google.com/bigquery/docs/control-access-to-resources-iam#grant_access_to_a_resource) in the Google Cloud web console.
For every resource you would like to use in Hightouch, select your BigQuery service account as the **Principal** and the `bigquery.dataViewer` role as the **Role**.
Hightouch lists all tables in your BigQuery project when creating a model using the [table selector](/models/table-selector).
However, Hightouch can only query data from tables assigned the `bigquery.dataViewer` role.
Tables that weren't assigned this role return an error if you attempt to query them.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **BigQuery** and follow the steps below.
### Configure your service account
Select the **GCP credentials** you [previously created](#create-a-service-account) or click **Create new**. To learn more about these credentials, see the [Google Cloud Provider (GCP) documentation](/security/gcp).
### Configure your source
Enter the **Project ID** for the project you [enabled the BigQuery API](#create-a-project-and-enable-bigquery-api) for and the **Dataset location**.
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
When using the [Lightning sync engine](/syncs/lightning-sync-engine) with BigQuery, Hightouch recommends configuring [self-hosted workspace storage](/security/storage#google-cloud-storage) (Business tier) in Google Cloud Storage. Keeping your workspace data in GCS aligns with BigQuery's native export paths and region constraints and helps maximize sync performance, especially for large models.
#### Lightning engine permissions
To enable the Lightning engine, you need to provide your service account additional permissions to create schemas and read/write data.
Run the following snippet to provision the `hightouch_planner` and `hightouch_audit` schemas, which are used for storing logs of previously synced data.
```sql
CREATE SCHEMA IF NOT EXISTS `hightouch_audit`;
CREATE SCHEMA IF NOT EXISTS `hightouch_planner`;
GRANT `roles/bigquery.dataViewer`, `roles/bigquery.dataEditor` ON SCHEMA `hightouch_planner` TO "serviceAccount:";
GRANT `roles/bigquery.dataViewer`, `roles/bigquery.dataEditor` ON SCHEMA `hightouch_audit` TO "serviceAccount:";
```
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from BigQuery.
The BigQuery source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Looker Looks](/extensions/looker-models)
- leveraging existing [Sigma workbooks](/extensions/sigma)
The [SQL editor](/models/sql-editor) allows you to query data from all projects that your GCP service account [has access to](#create-a-service-account).
The [table selector](/models/table-selector) only supports querying from the project specified in the [source configuration](#configure-your-source).
When syncing large amounts of data, it can take a long time for your model to reflect changes made in BigQuery. To speed up BigQuery model updates in the Hightouch UI, you can preview a model and save it. New or updated columns should then be reflected in the Hightouch UI.
## Tips and troubleshooting
### Hashed values are displayed as objects of bytes
When [previewing your model](/models/creating-models#preview-model-results) or when syncing your data to the destination, you might see an object of bytes, such as:
```bash
{"0":123,"1":456,"2":789,"3":012,...}
```
This happens if you use certain [hashing algorithms](https://cloud.google.com/bigquery/docs/reference/standard-sql/hash_functions) in BigQuery to hash your data, such as [SHA256](https://cloud.google.com/bigquery/docs/reference/standard-sql/hash_functions#sha256). To resolve this, open the [SQL editor](/models/sql-editor) and use [TO_HEX](https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#to_hex) or [TO_BASE64](https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#to_base64) to query this model column:
```bash
SELECT TO_HEX(model_column_name) as string_model_column_name, * FROM dataset.table
```
### No matching signature for operator \{...\} for argument types: \{...\}, \{...\}.
An example of the error message is `No matching signature for operator != for argument types: DATE, TIMESTAMP.`
This error means that your [model definition](/getting-started/concepts#models) is using the named operator, for example, `!=` to compare two incompatible types, for example, a `date` to a `timestamp`.
To resolve the issue, ensure your model columns are properly [typed](/models/data-types-casting#casting-from-your-model-configuration) or revise your model query.
### Unable to process number due to [big.js]
An example of the error message is `Unable to process number due to [big.js] Imprecise conversion: [big.js] Imprecise conversion`
This error occurs when one or more values have a very high precision, for example `123.12345678910`. Hightouch attempts to convert the value to a `number` (based on the data type selection of your model). An error occurs if the number can't be precisely represented by a JavaScript number.
To resolve the issue, ensure the values in your model columns have a lower precision. Additionally, if your value doesn't need to be a `number` datatype, casting the value to a `string` datatype can also resolve this issue.
### Dataset was not found in location
You may encounter this error if you customized the Lightning engine cache location in the source configuration. When using a customized location, you must use fully qualified table names (`project_id.dataset.table`) in model queries. This requires using the [SQL editor](/models/sql-editor) to create your model, because all other modeling methods ([table selector](/models/table-selector), dbt, Looker, Sigma) aren't compatible with this setup.
Otherwise, this error can happen if your dataset is located in a region other than `US` (such as `EU`). By default, BigQuery creates new schemas in the US region, so running the [Lightning engine permissions](#lightning-engine-permissions) SQL snippet (shown in the source setup UI) creates them in that region. If your dataset is not in the US region (for example, it might be in `EU`), this causes a region incompatibility between the main dataset and those two schemas, which results in that error message. To fix this, you need to manually check the [schema details](https://cloud.google.com/bigquery/docs/managing-table-schemas) in BigQuery and update their region/location value to be the same as the main dataset.
---
## BigQuery Iceberg
**URL:** https://hightouch.com/docs/sources/google-bigquery-iceberg
**Description:** BigQuery is a fully-managed enterprise data warehouse that helps you manage and analyze your data with built-in features like machine learning, geospatial analysis, and business intelligence.
**Section:** Sources
## Overview
This source is currently in preview. Please for more details.
Hightouch lets you sync data from BigLake tables using the Apache Iceberg REST catalog to downstream destinations.
BigLake Iceberg tables use the [Apache Iceberg](https://iceberg.apache.org/) open table format, with data stored as Parquet files in Google Cloud Storage (GCS). Table metadata is managed by the [BigLake Metastore](https://cloud.google.com/biglake/docs/overview) and accessible via the **Iceberg REST Catalog**, enabling interoperability between BigQuery, Apache Spark, and other Iceberg-compatible engines.
Hightouch discovers your tables and schemas via the BigLake Iceberg REST Catalog API, and uses BigQuery as the query engine to read your data.
If you already have a standard BigQuery source in Hightouch, you still need to create a separate BigQuery Iceberg source. The two source types use different table discovery mechanisms and cannot be combined.
## GCP prerequisites
Before configuring the source in Hightouch, you need to set up several GCP resources. Each step below shows both **Google Cloud Console** and **gcloud CLI** methods—you only need to follow one.
### Step 1: Enable required APIs
BigLake Iceberg requires the following APIs on your GCP project:
- **BigQuery API** (`bigquery.googleapis.com`)
- **BigLake API** (`biglake.googleapis.com`)
#### Console
1. Go to the [APIs & Services dashboard](https://console.cloud.google.com/apis/dashboard).
2. Click **Enable APIs and Services**.
3. Search for **BigQuery API** and click **Enable** (if not already enabled).
4. Repeat for **BigLake API**.
#### gcloud CLI
```bash
gcloud services enable bigquery.googleapis.com \
--project=YOUR_PROJECT_ID
gcloud services enable biglake.googleapis.com \
--project=YOUR_PROJECT_ID
```
### Step 2: Create a Cloud Storage bucket
Iceberg table data (Parquet files and metadata) is stored in a GCS bucket. If your Iceberg tables already exist in a bucket, you can use that bucket. Otherwise, create a new one.
The bucket must **not** use any of the following features, which are incompatible with BigLake Iceberg:
- Hierarchical namespaces
- Object versioning
- Object lock or bucket lock
- Customer-supplied encryption keys (CSEK)
#### Console
1. Go to [Cloud Storage > Buckets](https://console.cloud.google.com/storage/browser).
2. Click **Create**.
3. Enter a globally unique bucket name (e.g., `your-company-iceberg-data`).
4. Choose a **Location type** that matches where your BigQuery jobs run (e.g., `us-central1` or `US` multi-region).
5. For **Default storage class**, select **Standard**.
6. For **Access control**, select **Uniform**.
7. Leave all other settings at their defaults—do **not** enable versioning, retention policies, or object lock.
8. Click **Create**.
#### gcloud CLI
```bash
gcloud storage buckets create gs://YOUR_BUCKET_NAME \
--project=YOUR_PROJECT_ID \
--location=YOUR_REGION \
--uniform-bucket-level-access
```
### Step 3: Create a service account for Hightouch
Hightouch needs a GCP service account with permissions to query BigQuery, read from the Iceberg REST catalog, and access GCS. If you already have a service account from a standard BigQuery source, you can reuse it—just add the additional roles below.
| Role | Purpose |
|---|---|
| **BigQuery User** (`roles/bigquery.user`) | Run queries |
| **BigQuery Data Viewer** (`roles/bigquery.dataViewer`) | Read table data via BigQuery |
| **BigLake Viewer** (`roles/biglake.viewer`) | Read table metadata from the Iceberg REST catalog |
| **Storage Object Viewer** (`roles/storage.objectViewer`) | Read Parquet data files from GCS |
| **Service Usage Consumer** (`roles/serviceusage.serviceUsageConsumer`) | Required for REST catalog API billing attribution |
#### Console
1. Go to [IAM & Admin > Service Accounts](https://console.cloud.google.com/iam-admin/serviceaccounts).
2. Click **Create Service Account**.
3. Enter a name (e.g., `hightouch-bigquery-iceberg`) and click **Create and Continue**.
4. Add the roles listed above and click **Continue**.
5. Click **Done**.
6. Click on the newly created service account, go to the **Keys** tab, and click **Add Key > Create new key > JSON**. Download and save the key file.
#### gcloud CLI
```bash
SA_EMAIL="hightouch-bigquery-iceberg@YOUR_PROJECT_ID.iam.gserviceaccount.com"
gcloud iam service-accounts create hightouch-bigquery-iceberg \
--display-name="Hightouch BigQuery Iceberg" \
--project=YOUR_PROJECT_ID
for role in roles/bigquery.user roles/bigquery.dataViewer roles/biglake.viewer roles/serviceusage.serviceUsageConsumer; do
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
--member="serviceAccount:$SA_EMAIL" \
--role="$role"
done
gcloud storage buckets add-iam-policy-binding gs://YOUR_BUCKET_NAME \
--member="serviceAccount:$SA_EMAIL" \
--role=roles/storage.objectViewer
gcloud iam service-accounts keys create hightouch-key.json \
--iam-account="$SA_EMAIL"
```
### Step 4: Verify your setup
Verify that your bucket is accessible as an Iceberg REST catalog. The BigLake Metastore automatically maps GCS buckets to Iceberg catalogs—no explicit catalog creation is needed.
You can verify by querying the REST catalog config endpoint (requires `gcloud` authentication):
```bash
curl -s -H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "x-goog-user-project: YOUR_PROJECT_ID" \
"https://biglake.googleapis.com/iceberg/v1/restcatalog/v1/config?warehouse=gs://YOUR_BUCKET_NAME"
```
A successful response includes a `prefix` field and a list of supported endpoints:
```json
{
"overrides": {
"prefix": "projects/YOUR_PROJECT_NUMBER/catalogs/YOUR_BUCKET_NAME"
},
"endpoints": ["GET /v1/{prefix}/namespaces", ...]
}
```
If you already have Iceberg tables in your bucket (created by Spark, Flink, or another engine), they will be automatically discoverable through the REST catalog. No migration is needed.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **BigQuery Iceberg** and follow the steps below.
### Configure your service account
Select the **GCP credentials** you [previously created](#step-3-create-a-service-account-for-hightouch) or click **Create new**. To learn more about these credentials, see the [Google Cloud Provider (GCP) documentation](/security/gcp).
### Configure your source
Enter the following required fields into Hightouch:
- **Project ID**: Your GCP project ID.
- **Dataset location**: The geographic location of your data (e.g., `us-central1` or `US`).
- **Iceberg catalog warehouse URI**: The GCS bucket URI that backs your Iceberg REST catalog (e.g., `gs://your-company-iceberg-data`). Hightouch uses this to discover namespaces, tables, and schemas.
### Choose your sync engine
BigQuery Iceberg currently supports the **Basic sync engine** only. The Lightning sync engine will be supported in a future release.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from your Iceberg tables.
The BigQuery Iceberg source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
The [table selector](/models/table-selector) browses tables discovered from your Iceberg REST catalog. The [SQL editor](/models/sql-editor) allows you to write arbitrary BigQuery SQL, including queries that join Iceberg tables with standard BigQuery tables.
## Known limitations
- **Lightning sync engine**: Not yet supported. BigQuery DML (INSERT, UPDATE, DELETE) is not available on REST-catalog-managed Iceberg tables. This is a Google Cloud limitation expected to be resolved in a future BigQuery release.
- **Write operations**: Hightouch reads from Iceberg tables but does not write to them. Data must be loaded into Iceberg tables using an Iceberg-compatible writer (Spark, Flink, PyIceberg, etc.).
## Tips and troubleshooting
### "Not found" errors when querying tables
BigQuery queries against REST catalog tables use four-part naming: `project.catalog.namespace.table`. If you see "Dataset not found" errors, verify that:
- Your GCS bucket name is correct in the **Iceberg catalog warehouse URI** field
- The Hightouch service account has the `biglake.viewer` and `serviceusage.serviceUsageConsumer` roles
- The BigLake API is enabled on your project
### Tables not appearing in the table selector
If your Iceberg tables don't appear in the table selector:
- Verify the tables exist in the REST catalog by querying the namespaces endpoint (see [Step 4](#step-4-verify-your-setup))
- Ensure the tables are in a namespace other than `hightouch_planner` or `hightouch_audit` (these are filtered from discovery)
- Try refreshing the schema in the Hightouch UI
---
## Google Sheets
**URL:** https://hightouch.com/docs/sources/google-sheets
**Description:** Google Sheets is a web-based spreadsheet offered by Google as part of their Drive office services. It makes it easy to create, edit and collaborate with others from any device.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in Google Sheets and push it to downstream destinations. Before connecting your Google Sheets, it's best to make sure they meet formatting requirements.
## Google Sheets formatting requirements
- Your Google Sheet must be a Sheets file, not an `.XLS` or `.XLSX` file. If this is the case, you need to convert it to a Sheets file. To do so, go to [**File > Save as Google Sheets**](https://support.google.com/docs/answer/9331167?hl=en#2.5).
- Your Google Sheet must have column names in the first row of your sheet. In the following image, you can see that the column names are `id`, `created_at`, `name`, `website`, and `description`.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Google Sheets** and follow the steps below.
### Authorize connection
1. Click **Log in to Google Sheets** and select the desired Google Account.
2. After reading what authorization you're giving to Hightouch, select **Allow**.
3. Select the **Spreadsheet** containg the data you wish to sync from the dropdown.
4. Click **Continue**.
Remember that [`.XLS` or `.XLSX` file types aren't supported](#google-sheets-formatting-requirements) and won't appear in the dropdown.
## Test your connection
When setting up Google Sheets as a source for the first time, Hightouch checks that you have the correct permissions. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Google Sheets.
### Model setup
1. In Hightouch, go to the [**Models** overview page](https://app.hightouch.com/models?from=docs).
1. Click **Add model.**
1. Select the Google Sheets source you previously created.
1. Select the **Sheet name** of the data you want to sync. Hightouch treats each sheet as a separate table. If you want to sync data from multiple sheets, you need to set up a separate model for each one.
1. Preview your model's query results.
1. Click **Continue**.
1. Name your model and select its [**primary key**](/models/creating-models#unique-primary-key-requirement). Hightouch uses the primary key to determine which rows have been added, changed, or removed since the last sync.
## Tips and troubleshooting
### Internal error encountered
This error may occur when the spreadsheet you're syncing to contains sheets with charts or pivot tables.
Try moving the data you would like to update with Hightouch to a separate spreadsheet.
Then reference this data's range using a formula like [`IMPORTRANGE`](https://support.google.com/docs/answer/3093340?hl=en) in the spreadsheet with your charts and analysis.
This way, the sheet that Hightouch updates won't have the timeout issue, and you can still see the data on the sheet that contains charts or pivot tables.
### Unable to parse range
This error indicates that the spreadsheet (in the source configuration) or the sheet (in the model configuration) have either been renamed or deleted. You can fix this by ensuring that both the source and model are querying data from an existing spreadsheet and sheet.
---
## Greenplum Database
**URL:** https://hightouch.com/docs/sources/greenplum
**Description:** Greenplum reduces data silos by providing you with a single, scale-out environment for converging analytic and operational workloads, like streaming ingestion.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in your Greenplum database and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to your Greenplum instance for information like your host, port, database name, and credentials.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Greenplum Database** and follow the steps below.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your Greenplum server.
- **Port**: The port number of your Greenplum server. The default port number is 5432, but yours may be different.
- **Database**: This specifies the database to use when Hightouch executes queries in Greenplum.
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
To set up the Lightning engine, you need to grant Hightouch write access to Greenplum. You can do so by running the following SQL snippet.
```sql
CREATE USER hightouch_user WITH PASSWORD '********';
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT CREATE, USAGE ON SCHEMA hightouch_audit TO hightouch_user;
GRANT CREATE, USAGE ON SCHEMA hightouch_planner TO hightouch_user;
```
The snippet creates a dedicated Greenplum user for Hightouch. It also provisions two schemas (`hightouch_planner` and `hightouch_audit`) for storing logs of previously synced data.
### Provide credentials
Enter the following fields into Hightouch:
- **User**: This can be your personal Greenplum login or a dedicated user for Hightouch. At minimum, this user must have read access to the data you wish to sync. If using the Lightning sync engine, you must also grant this user additional permissions as described above.
- **Password**: The password for the user specified above.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Greenplum.
The Greenplum source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Looker Looks](/extensions/looker-models)
- leveraging existing [Sigma workbooks](/extensions/sigma)
## Tips and troubleshooting
---
## Lakebase
**URL:** https://hightouch.com/docs/sources/lakebase
**Description:** Lakebase is a serverless Postgres-compatible database in the Databricks platform.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in your Lakebase data warehouse and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to your Lakebase instance for information like your host, port, database name, and credentials.
## Connection configuration
Hightouch officially supports Lakebase versions 1.0 and above. While some features may still work with earlier versions, compatibility isn't guaranteed, and future updates might introduce changes affecting older versions.
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Lakebase** and follow the steps below.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your Lakebase server.
- **Port**: The port number of your Lakebase server. The default port number is 5432, but yours may be different.
- **Database**: This specifies the database to use when Hightouch executes queries in Lakebase.
For authentication setup instructions, refer to the [Databricks Lakebase authentication documentation](https://docs.databricks.com/aws/en/oltp/oauth#authenticate-with-postgres-roles-and-passwords).
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
To set up the Lightning engine, you need to grant Hightouch write access to Lakebase. You can do so by running the following SQL snippet.
```sql
CREATE USER hightouch_user WITH PASSWORD '********';
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT CREATE, USAGE ON SCHEMA hightouch_audit TO hightouch_user;
GRANT CREATE, USAGE ON SCHEMA hightouch_planner TO hightouch_user;
```
The snippet creates a dedicated Lakebase user for Hightouch. It also provisions two schemas (`hightouch_planner` and `hightouch_audit`) for storing logs of previously synced data.
### Provide credentials
Enter the following fields into Hightouch:
- **User**: This can be your personal Lakebase login or a dedicated user for Hightouch. At minimum, this user must have read access to the data you wish to sync. If using the Lightning sync engine, you must also grant this user additional permissions as described above.
- **Password**: The password for the user specified above.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Lakebase.
The Lakebase source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Looker Looks](/extensions/looker-models)
- leveraging existing [Sigma workbooks](/extensions/sigma)
## Tips and troubleshooting
---
## Materialize
**URL:** https://hightouch.com/docs/sources/materialize
**Description:** Materialize is a streaming database powered by Timely and Differential Dataflow, purpose-built for low-latency applications. It lets you ask complex questions about your data using SQL, and incrementally maintains the results of these SQL queries as the underlying data changes.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in your Materialize database and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to your Materialize instance for information like your host, port, database name, cluster, and credentials.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Materialize** and follow the steps below.
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your Materialize server.
- **Port**: The port number of your Materialize server. The default port number is 6875, but yours may be different.
- **Database**: This specifies the database to use when Hightouch executes queries in Materialize.
### Provide credentials
Enter the following fields into Hightouch:
- **User**: This can be your personal Materialize login or a dedicated user for Hightouch. At minimum, this user must have read access to the data you wish to sync.
- **Password**: The password for the user specified above.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Materialize.
The Materialize source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Looker Looks](/extensions/looker-models)
- leveraging existing [Sigma workbooks](/extensions/sigma)
## Tips and troubleshooting
### Canceling statement due to statement timeout
This error occurs when the required execution time of your Materialize query exceeds the timeout limit for the database.
To address the error, increase the timeout by executing the following query:
```json
set statement_timeout = '300 s'; -- 300 seconds, 5 minutes
```
Be sure to adjust the time to be as long as it takes for your query to execute.
---
## Microsoft Excel
**URL:** https://hightouch.com/docs/sources/microsoft-excel
**Description:** Microsoft Excel is the industry leading spreadsheet software program, a powerful data visualization and analysis tool.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in Microsoft Excel and push it to downstream destinations. Before connecting your Microsoft Excel source, it's best to make sure they meet the formatting requirements.
## Microsoft Excel formatting requirements
- Your Microsoft Excel workbook must be a `.xlsx` file. Other extension workbooks, for example, `.xls` or `.csv` aren't supported. [Convert your file](https://answers.microsoft.com/en-us/msoffice/forum/all/converting-xls-to-xlsx/ba8d05f4-d38a-462a-a062-d6083db4b479) if necessary.
- Your Excel worksheet must have column names in the first row. In the following image, you can see that the column names are `Country`, `Product`, `Discount Band`, `Units Sold`, and ` Manufacturing Price` etc.
- To ensure accurate processing, all valid data must be present in the top-left portion of the sheet. If a column name is left empty, the preceding column will be treated as the last column in the file. Similarly, if a row is left empty, the preceding row will be considered the last row in the file, and any subsequent rows will not be processed.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Microsoft Excel** and follow the steps below.
### Authorize connection
1. Click **Log in to Microsoft Excel** and select the desired Microsoft Account.
2. After reading what authorization you're giving to Hightouch, click **Accept**.
3. Select the **Drive** containg the data you wish to sync from the dropdown.
4. Click **Continue**.
## Test your connection
When setting up Microsoft Excel as a source for the first time, Hightouch checks that you have the correct permissions. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Microsoft Excel.
### Model setup
1. In Hightouch, go to the [**Models** overview page](https://app.hightouch.com/models?from=docs).
1. Click **Add model.**
1. Select the Microsoft Excel source you previously created.
1. Select the **Workbook** which has the data you want to sync.
1. Next, select a **Sheet** to read from. Hightouch treats each sheet as a separate table. If you want to sync data from multiple sheets, you need to set up a separate model for each one.
1. Preview your model's query results.
1. Click **Continue**.
1. Name your model and select its [**primary key**](/getting-started/concepts#primary-key-requirement). Hightouch uses the primary key to determine which rows have been added, changed, or removed since the last sync.
## Tips and troubleshooting
### Need admin approval
If you see a **Need admin approval** page appear when [accessing Microsoft's OAuth login flow](#connect-to-microsoft-onedrive), your Azure Active Directory user account might not have the necessary permissions to connect applications to your organization.
Ask one of your Azure organization's admins to edit your user account permissions.
You can read more about [Azure user permissions](https://learn.microsoft.com/en-us/azure/active-directory/develop/active-directory-how-applications-are-added#who-has-permission-to-add-applications-to-my-azure-ad-instance) and [configuring user settings](https://learn.microsoft.com/en-us/azure/active-directory/manage-apps/configure-user-consent?pivots=portal#configure-user-consent-settings) in Azure's documentation.
### Files not appearing in model dropdown
Only `.xlsx` files are [supported](#microsoft-excel-formatting-requirements).
Other files won't appear in the dropdown.
[Convert your file type to `.xlsx`](https://answers.microsoft.com/en-us/msoffice/forum/all/converting-xls-to-xlsx/ba8d05f4-d38a-462a-a062-d6083db4b479) to make it available to Hightouch.
---
## MySQL
**URL:** https://hightouch.com/docs/sources/mysql
**Description:** MySQL is an open source relational database management system (RDBMS) with a client-server model.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in MySQL and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to MySQL for information like your hostname, port, and credentials.
Hightouch unofficially supports backwards compatible MySQL forks such as [TiDB by Pingcap](https://en.pingcap.com/).
### MariaDB support
Hightouch supports using MariaDB as a source via the MySQL source. Follow the instructions below, substituting MySQL information and credentials with your MariaDB ones.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **MySQL** and follow the steps below.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your MySQL.
- **Port**: The port number of your MySQL. The default is 3306, but yours may be different.
- **Database**: The name of the database in your MySQL.
### Provide your credentials
Enter the following fields into Hightouch:
- **User**: This can be your personal MySQL login or a dedicated user for Hightouch.
- (Optional) **Password**: The password for the user specified above.
## Test your connection
When setting up MySQL as a source for the first time, Hightouch validates your credentials and access to your database. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, you've completed setting up MySQL as a source. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from MySQL.
The MySQL source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
## Tips and troubleshooting
### Server does not support secure connection
The [connection test](#test-your-connection) returns this error message if you don't enable SSL connections in your MySQL server configuration.
To solve this, make sure to configure MySQL to [use encrypted connections](https://dev.mysql.com/doc/refman/8.0/en/using-encrypted-connections.html).
---
## Oracle DB
**URL:** https://hightouch.com/docs/sources/oracle
**Description:** Power internal tools, in-app experiences, and more
**Section:** Sources
## Overview
Hightouch lets you pull data stored in your Oracle database and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to your Oracle instance for information like your host, port, database name, and credentials.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Oracle DB** and follow the steps below.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your Oracle server.
- **Port**: The port number of your Oracle server. The default port number is 1521, but yours may be different.
- **Database**: This specifies the database to use when Hightouch executes queries in Oracle DB. This is different than the host, but your host address might contain your database name.
- **Username**: The user that has access through Hightouch to the database and tables you want to pull from. It's best to [create a new user](https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/CREATE-USER.html) specifically for Hightouch access. _Do not use the root user._
- **Password**: The password for the specified user.
### Choose your sync engine
For optimal performance, Hightouch [tracks incremental changes](/getting-started/concepts#change-data-capture-cdc)
in your data model—such as added, changed, or removed rows—and only syncs those records.
You can choose between two different sync engines for this work.
The **Basic engine** requires read-only access to Oracle. Hightouch executes a query in your database, reads all query results, and then determines incremental changes using Hightouch's infrastructure. This engine is easier to set up since it requires read—not write—access to Oracle.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Oracle.
The Oracle source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
## Tips and troubleshooting
### Compatible versions
The supported Oracle DB version is `19c`. Other versions aren't officially supported but may work regardless.
### Unsupported data types
Hightouch does not support `INTERVAL` types for Oracle.
---
## Sources overview
**URL:** https://hightouch.com/docs/sources/overview
**Description:** Learn how to connect data sources to Hightouch so you can define models and activate data through syncs.
**Section:** Sources
## Overview
A **source** is any system where your business data lives.
Common examples include cloud data warehouses such as **Snowflake**, **BigQuery**, or **Databricks**. Sources can also be transactional databases, flat files (CSV, SFTP), or BI tools like Looker and Sigma.
Sources are the starting point in Hightouch’s [**Reverse ETL**](/getting-started/concepts) pipeline. Once connected, you can query data from them to build [**models**](/models/creating-models), which are then delivered to **destinations** through [**syncs**](/syncs/overview).
**For marketers**: a source is simply where customer and business data lives,
such as purchase history, website events, or CRM records.
---
## Add a source
To create a new source:
1. Go to **Integrations**, then **[Sources](https://app.hightouch.com/sources?from=docs)**.
2. Click **Add source**.
You’ll be prompted to choose from supported source types and provide the necessary connection details, such as:
- **Connection method** (e.g. username/password, OAuth, API key)
- **Host and port** (for databases)
- **Authentication details**
- **Permissions** required by Hightouch to query data
During source setup, Hightouch prompts you to choose a sync engine. The
[Lightning sync engine](/syncs/lightning-sync-engine) is recommended for all
supported sources. It computes change data capture directly in your warehouse
for faster syncs and is required for features like
[Journeys](/customer-studio/journeys),
[Sampling](/customer-studio/sampling), and [Warehouse Sync
Logs](/syncs/warehouse-sync-logs).
The exact configuration steps depend on the source type. See catalog and refer
to **integration-specific documentation** for instructions.
---
## Security and connectivity
Hightouch connects to sources securely:
- All data in transit is encrypted with TLS.
- For private networks or VPCs, you can connect via [**SSH tunnel**](/security/ssh-tunneling), [**AWS PrivateLink**](/security/aws/privatelink), [**Azure Private Link**](/security/azure/private-link), or [**Google Cloud Private Service Connect**](/security/gcp/private-service-connect).
- By default, no customer data is copied into Hightouch storage—queries are run directly against your source.
See [Security and compliance overview](/security/overview) for details.
---
## Permissions
Access to individual sources is controlled through custom roles. Admins can configure source-specific grants in the **Source** tab of the custom role builder, including per-source read, write, and manage permissions. See [Roles](/workspace-management/roles#source) for details.
---
## How sources fit into data activation
- **Data teams**: configure sources and expose trusted tables or views.
- **Marketers**: work with models and schema that are built on top of these sources, without needing to understand the connection details.
**Example:**
- Source: a Snowflake warehouse containing ecommerce data
- Model: SQL query for “users with >3 purchases in last 90 days”
- Sync: send this model to Meta Ads for retargeting
---
## Next steps
- [Add your first source](https://app.hightouch.com/sources?from=docs)
- Learn how to define [Models](/models/creating-models) on top of sources
- Configure [Syncs](/syncs/overview) to deliver data into destinations
---
## PlanetScale
**URL:** https://hightouch.com/docs/sources/planetscale
**Description:** PlanetScale is a MySQL-compatible serverless database that brings you scale, performance, and reliability — without sacrificing developer experience.
With PlanetScale, you get the power of horizontal sharding, non-blocking schema changes, and many more powerful database features without the pain of implementing them.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in PlanetScale and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to PlanetScale for information like your hostname, database, and credentials.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **PlanetScale** and follow the steps below.
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your PlanetScale.
- **Port**: The port number of your PlanetScale. The default is 3306, but yours may be different.
- **Database**: The name of the database in your PlanetScale.
### Provide your credentials
Enter the following fields into Hightouch:
- **User**: This can be your personal PlanetScale login or a dedicated user for Hightouch.
- (Optional) **Password**: The password for the user specified above.
## Test your connection
When setting up PlanetScale as a source for the first time, Hightouch validates your credentials and access to your database. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, you've completed setting up PlanetScale as a source. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from PlanetScale.
The PlanetScale source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
## Tips and troubleshooting
---
## PostgreSQL
**URL:** https://hightouch.com/docs/sources/postgresql
**Description:** PostgreSQL, also known as Postgres, is a free and open-source relational database management system emphasizing extensibility and SQL compliance.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in your PostgreSQL database and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to your PostgreSQL instance for information like your host, port, database name, and credentials.
## Connection configuration
Hightouch officially supports PostgreSQL versions 15 and above. While some features may still work with earlier versions, compatibility isn’t guaranteed, and future updates might introduce changes affecting older versions.
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **PostgreSQL** and follow the steps below.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your PostgreSQL server.
- **Port**: The port number of your PostgreSQL server. The default port number is 5432, but yours may be different.
- **Database**: This specifies the database to use when Hightouch executes queries in PostgreSQL.
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
To set up the Lightning engine, you need to grant Hightouch write access to PostgreSQL. You can do so by running the following SQL snippet.
```sql
CREATE USER hightouch_user WITH PASSWORD '********';
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT CREATE, USAGE ON SCHEMA hightouch_audit TO hightouch_user;
GRANT CREATE, USAGE ON SCHEMA hightouch_planner TO hightouch_user;
```
The snippet creates a dedicated PostgreSQL user for Hightouch. It also provisions two schemas (`hightouch_planner` and `hightouch_audit`) for storing logs of previously synced data.
### Provide credentials
Enter the following fields into Hightouch:
- **User**: This can be your personal PostgreSQL login or a dedicated user for Hightouch. At minimum, this user must have read access to the data you wish to sync. If using the Lightning sync engine, you must also grant this user additional permissions as described above.
- **Password**: The password for the user specified above.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from PostgreSQL.
The PostgreSQL source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Looker Looks](/extensions/looker-models)
- leveraging existing [Sigma workbooks](/extensions/sigma)
## Tips and troubleshooting
### Canceling statement due to statement timeout
This error occurs when the required execution time of your PostgreSQL query exceeds the timeout limit for the database.
To address the error, increase the timeout by executing the following query:
```json
set statement_timeout = '300 s'; -- 300 seconds, 5 minutes
```
Be sure to adjust the time to be as long as it takes for your query to execute.
### Terminating connection due to conflict with recovery
This error can happen if the `max_standby_archive_delay` and `max_standby_streaming_delay` values in your PostgreSQL configuration are too low. Try increasing them as suggested in the [AWS Knowledge Center](https://repost.aws/knowledge-center/rds-postgresql-error-conflict-recovery).
### Prepared statement did not exist
This error occurs when your PostgreSQL instance is accessed through a connection pooler running in transaction pooling mode.
To resolve this, adjust your source configuration to use a direct connection string instead of the pooled URL.
---
## S3
**URL:** https://hightouch.com/docs/sources/s3
**Description:** Securely sync CSV or JSON data from any S3 bucket
**Section:** Sources
Need to join, transform, or filter your S3 data? Consider using the [AWS
Athena](/sources/amazon-athena) source to directly query your S3 data using
SQL.
## Overview
Hightouch lets you pull data from CSV and JSON files stored in an Amazon S3 bucket and push them to downstream destinations. To get started, you need an S3 bucket and AWS credentials.
## AWS credential setup
See the guide for [configuring AWS credentials](/security/aws) to learn how you can set up an IAM principal and use its credentials. The IAM principal whose credentials you use must have programmatic access enabled and permission to read from the S3 path you want to use.
Hightouch needs the following IAM actions to retrieve items from your bucket:
| Action | Details |
| :----------------------- | :---------------------------------------------------------------------------------------------- |
| `s3:GetObject` | Grants permission to retrieve objects from Amazon S3 |
| `s3:ListBucket` | Grants permission to list some or all the objects in an Amazon S3 bucket (up to 1000) |
| `sts:GetFederationToken` | (Optional) Grants permission to generate short-lived AWS credentials for faster warehouse reads |
If you use an **access key** credential in Hightouch, we recommend adding
`sts:GetFederationToken`. This lets Hightouch generate short-lived credentials
scoped to the S3 prefix being written, which enables syncs to export data from
your warehouse to S3 (faster than reading rows via streaming). If you don't
grant this permission, Hightouch can still work, but syncs will fall back to
the slower streaming method instead. If you use a **cross-account role**,
`sts:GetFederationToken` isn't required.
You can use the following JSON sample to create your IAM policy:
```yaml
{
"Version": "2012-10-17",
"Statement":
[
{
"Effect": "Allow",
"Action": ["s3:GetObject", "s3:ListBucket"],
"Resource":
["arn:aws:s3:::${bucketName}/*", "arn:aws:s3:::${bucketName}"],
},
],
}
```
If you want to add the optional permission, include the following statement:
```json
{
"Sid": "AllowFederationTokenForS3",
"Effect": "Allow",
"Action": ["sts:GetFederationToken"],
"Resource": "*"
}
```
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Amazon S3** and follow the steps below.
### Configure your credentials
Select the credentials you previously created or click **Create new** to set them up now.
### Configure your source
Enter your S3 bucket's **Region** and the **Bucket name**. The bucket name should be just the name of the bucket, not a URL.
## Test your connection
When setting up Amazon S3 as a source for the first time, Hightouch validates your AWS credentials and access to your S3 bucket. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from your S3 bucket. The file must have either a `.csv` or `.json` extension.
### CSV requirements
CSV files must meet the following requirements:
- They must have a header row. The values in the header row are automatically available as column names when you set up a sync.
- They must use comma-separated values; tabs and other delimiters aren't supported.
- They must use double quotes (`"`) for quoted values.
### JSON requirements
JSON files must meet the following requirements:
- The input file must contain an array at the top level.
- Each element in the input file must be an object with the same keys.
The keys of the array elements are automatically available as column names when you set up a sync.
### Model setup
Hightouch supports syncing data from a specific file in your S3 bucket as well
as from the last modified file that matches a given prefix.
1. In Hightouch, go to the [**Models** overview page](https://app.hightouch.com/models?from=docs).
1. Click **Add model.**
1. Select the Amazon S3 source you previously created.
1. Enter the **relative path** to the CSV or JSON file that you want to sync
data from, like `path/prefix/file.csv`. The path shouldn't contain the bucket
name.
1. To sync the last modified file with a given prefix, check the **Use last
modified** box and enter a prefix like `path/prefix` in the path field. See
Amazon's [S3 docs on
prefixes](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-prefixes.html)
for examples of how prefixes match objects. To sync the last modified file in
the entire bucket, check the **Use last modified** box and leave the path field
blank.
1. Preview your model's query results.
1. Click **Continue**.
1. Name your model and select its [**primary key**](/models/creating-models#unique-primary-key-requirement). Hightouch uses the primary key to determine which rows have been added, changed, or removed since the last sync.
## Tips and troubleshooting
---
## Sample datasets
**URL:** https://hightouch.com/docs/sources/sample-data
**Description:** Explore Hightouch using the B2B SaaS and B2C eCommerce sample data sources—no warehouse setup required.
**Section:** Sources
## Overview
Hightouch provides two built-in **Sample datasets** that let you explore the product without connecting your own data warehouse. They're hosted on a read-only database that Hightouch manages for you, so there are no credentials, networking, or permissions to configure.
Sample datasets are ideal for:
- Running your first sync end-to-end before your production source is ready.
- Evaluating Hightouch during a trial or POC.
- Learning how [models](/getting-started/concepts#models), [syncs](/getting-started/concepts#syncs), and [Customer Studio](/customer-studio/overview) work against realistic data.
- Testing a new [destination](/destinations/overview) integration without touching production data.
Sample datasets are real, customer-facing sources—not internal-only demo tooling. They appear in every workspace at the end of the source list, and usage of sample sources is excluded from your workspace's billing and analytics metrics.
Two sample sources are available:
- **B2B SaaS**: models a typical SaaS business with users and organizations, useful for testing CRM, marketing, and product analytics destinations.
- **B2C eCommerce**: contains customer profiles and cart events data for a B2C eCommerce company that retails shoe brands, useful for testing ad platforms, email, loyalty, and retention destinations.
## Add a sample source
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button.
Scroll to the bottom of the source catalog. The **Sample datasets** section contains both **B2B SaaS** and **B2C eCommerce**. Select the one you want to use.
Because the underlying database is managed by Hightouch, there are no credentials to enter and no connection test to run. Hightouch skips directly to the **Finalize source** step.
Give the source a name, review the default permissions, and click **Finish**. The source is ready to use immediately.
## Create a model
Models define what data to pull from your source. You can build [models](/models/creating-models) on top of a sample source using the [SQL editor](/models/sql-editor) or the [visual table selector](/models/table-selector). [dbt](/extensions/dbt-models), [Looker](/extensions/looker-models), and [Sigma](/extensions/sigma) modeling methods aren't supported on sample sources.
### Add a model
Go to the [**Models** overview page](https://app.hightouch.com/models?from=docs) and click **Add model**.
### Select the sample source
On the **Select source** step, choose your **B2B SaaS** or **B2C eCommerce** sample source. Sample sources are clearly labeled with a **Sample** badge.
### Choose a modeling method
On the **Define model** step, choose either the **SQL query** editor or the **Table selector**.
### Use the pre-populated sample queries
When you pick a sample source in the SQL editor, Hightouch shows a **Sample queries** panel with ready-to-run examples tailored to the dataset. Click a query to populate the editor, then click **Preview results** to run it.
The included sample queries are a good starting point for common use cases—product-led upsell, abandoned-cart retargeting, seasonal campaigns, VIP audiences—and can be edited freely.
For a full reference on writing model queries, see [Create models with the SQL editor](/models/sql-editor) or [Create models with the table selector](/models/table-selector).
### Finalize the model
Once you're happy with the preview, click **Continue** to move to the **Finalize model** step. Give the model a name, select a **Primary key**, and click **Finish**.
Your new model appears on the [**Models** overview page](https://app.hightouch.com/models?from=docs) and is ready to use in syncs.
## Send data with a sync
With a model in place, you can create a [sync](/syncs/overview) that delivers sample data to any destination in your workspace.
1. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click **Add sync**.
2. On the **Select model** step, pick the model you just built on top of the sample source.
3. Pick a destination. If you don't have one connected yet, [Google Sheets](/destinations/google-sheets) is the fastest way to see results—it's a [lite destination](/pricing/ss-pricing#lite-destinations) and easy to set up.
4. Map fields from your sample model to the destination's schema.
5. Set a schedule and click **Finish**.
For a full walkthrough, see [Create your first sync](/syncs/create-your-first-sync).
## Use in Customer Studio
Sample sources also work with [Customer Studio](/customer-studio/overview), so you can try audiences, journeys, and other activation features without real warehouse data.
### Create a parent model
Go to **Customer Studio → Schema** and switch the source selector at the top of the page to your sample source. If no parent models exist yet, click **Create parent model**.
On the **Select table** step of the **New parent model** flow, the **Sample queries** panel is pre-populated with starter queries. Pick one to auto-fill the SQL editor, then click **Preview results** to check your data before continuing.
On the **Configuration** step, give the parent model a name, pick a **Primary key**, and optionally select primary and secondary labels used when previewing audiences. Click **Create parent model** to finish.
Your new parent model appears in the **Schema** tab with all its columns visible and ready for audience building.
### Build audiences, journeys, and more
Once a parent model exists, you can use it just like any production source to:
- Build [audiences](/customer-studio/audience-overviews)—for example, VIP customers with items still in their cart, or Self-Serve organizations approaching their storage limit.
- Add [related models](/customer-studio/schema#add-related-models) (like purchases, subscriptions, or events) to enrich filtering.
- Create [traits](/customer-studio/traits) for reusable calculated fields such as lifetime value or last-seen timestamp.
- Orchestrate lifecycle [journeys](/customer-studio/journeys) across multiple destinations.
- Sync audiences to any connected destination with [Customer Studio syncs](/customer-studio/syncs).
See the [Customer Studio quick start](/customer-studio/quick-start) for an end-to-end walkthrough.
## Sample dataset contents
Each sample database is organized under the `public` schema. Use the **table selector** when creating a model to see the full list of tables and columns—the schemas may expand over time as Hightouch adds more examples.
### B2B SaaS
Models a SaaS business with records for users and their organizations. The dataset includes tables such as `public.users` and `public.organizations`, with attributes like `type`, `segment`, `total_storage_used_mb`, `total_users_invited`, and `documents_uploaded_l30d`. This makes it a good fit for testing:
- product-led growth workflows (for example, surfacing upsell opportunities to Self-Serve accounts nearing a storage limit)
- lifecycle marketing to admins and inactive users
- CRM and marketing-automation syncs to destinations like Salesforce, HubSpot, Marketo, Braze, or Segment
The **Sample queries** panel includes starter queries for *Product-led sales upsell opportunities*, *Lifecycle marketing to inactive admins*, and *New Business Tier product introduction*. A typical starter query—surfacing Self-Serve organizations using more than 50 GB of storage—looks like:
```sql
SELECT *
FROM public.organizations
WHERE type = 'Self-Serve'
AND CAST(total_storage_used_mb AS INT) > 50000
```
### B2C eCommerce
Models a direct-to-consumer retailer selling shoe brands. The dataset includes a `public.customers` table with attributes such as `customer_id`, `first_name`, `last_name`, `email`, `city`, `ltv`, `brand_affinity`, `VIP` tier, and `items_in_cart`, alongside cart-event data. This makes it a good fit for testing:
- audience-building workflows (for example, VIP customers or high-intent shoppers)
- abandoned-cart retargeting to ad platforms like Meta, Google Ads, or TikTok
- email and SMS campaigns through tools like Iterable, Klaviyo, or Attentive
- loyalty and retention use cases
The **Sample queries** panel includes starter queries for *Abandoned cart retargeting ads*, *Seasonal Rebook campaign - Nano X2*, and *Annual VIP celebration event in Florida*. A typical starter query—pulling non-VIP customers with items still in their cart—looks like:
```sql
SELECT *
FROM public.customers
WHERE VIP = 'Normal'
AND CAST(items_in_cart AS int) > 0
```
## Limitations
Sample sources are intentionally simple. A few things to know:
- **Read-only.** You can't write to a sample database, create tables, or modify its schema.
- **Shared across workspaces.** The underlying database is the same for every Hightouch customer. Don't store anything you expect to be private in a sample source—you shouldn't be able to, but it's worth stating plainly.
- **Lightning sync engine isn't supported.** Sample sources always use the Basic sync engine (Hightouch's `LocalDiffPlanner`). Lightning engine features such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution) therefore aren't available on syncs powered by a sample source.
- **Not for production.** Sample datasets are meant for exploration and demos only. When you're ready to run real workloads, [connect your own source](/sources/overview).
If a sync against a sample source reports a `permission denied` error, that usually means the sync was configured to use the Lightning engine. Switch the sync to the Basic engine (or recreate the sync) and the error will clear.
## Next steps
With your sample source connected, you can:
- Follow the [Create your first sync](/syncs/create-your-first-sync) tutorial end-to-end.
- Pick a destination from the [destination catalog](/destinations/overview) and map sample data to it.
- Explore [Customer Studio](/customer-studio/quick-start) to build audiences and journeys on top of sample data.
- When you're ready to move to production data, review the [Sources overview](/sources/overview) and add your own warehouse.
---
## SFTP
**URL:** https://hightouch.com/docs/sources/sftp
**Description:**
**Section:** Sources
## Overview
Hightouch supports syncing data from CSV files stored on a remote server via SFTP. To connect to Hightouch you need your hostname, port, and credentials for an SFTP user. This user needs to have permission to read the file you want to use as a data source.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **SFTP** and follow the steps below.
### Choose authentication method
You can select to use either a **password** or SSH **private key**. If using a **private key**, you can optionally include a **passphrase**.
### Configure your source
- **Host**: The hostname or IP address of your SFTP server.
- **Port**: The port number of your SFTP server. The default port number is 22, but yours may be different.
- **Username**: This can be your personal SFTP login or a dedicated user for Hightouch.
- Either your **password** or **private key**: Your private key must be in OpenSSH format and should include header and footer. For example:
```bash
-----BEGIN OPENSSH PRIVATE KEY-----
-----END OPENSSH PRIVATE KEY-----
```
- (Optional) **Private key passphrase**: If using an encrypted private key, you can specify a passphrase to decrypt it.
FTP configurations are not supported at this time.
## Test your connection
When setting up SFTP as a source for the first time, Hightouch validates your credentials. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull via SFTP. The file must have a `.csv` extension.
### CSV requirements
CSV files must meet the following requirements:
- They must have a header row. The values in the header row are automatically available as column names when you set up a sync.
- They must use comma-separated values; tabs and other delimiters aren't supported.
- They must use double quotes (`"`) for quoted values.
### Model setup
1. In Hightouch, go to the [**Models** overview page](https://app.hightouch.com/models?from=docs).
1. Click **Add model.**
1. Select the SFTP source you previously created.
1. Enter the **relative path** to the CSV file you want to sync data from.
1. Preview your model's query results.
1. Click **Continue**.
1. Name your model and select its [**primary key**](/models/creating-models#unique-primary-key-requirement). Hightouch uses the primary key to determine which rows have been added, changed, or removed since the last sync.
## Tips and troubleshooting
### Unsupported Unicode escape sequence
This error may occur when previewing a model from an SFTP source if the file is not encoded in UTF-8. Re-export or convert the file to UTF-8 and retry the preview.
---
## SingleStore
**URL:** https://hightouch.com/docs/sources/single-store
**Description:** SingleStore is a modern relational database for cloud and on-premises delivering immediate insights for modern applications and analytical systems.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in SingleStore and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to SingleStore for information like your hostname, database, and credentials.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **SingleStore** and follow the steps below.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your SingleStore.
- **Port**: The port number of your SingleStore. The default is 3306, but yours may be different.
- **Database**: The name of the database in your SingleStore.
- **Verify CA Certificate**: Enable CA Certificate verification for a more secure connection.
When you enable **Verify CA Certificate**, Hightouch will require you to provide the CA certificate from SingleStore.
Get the certificate file from SingleStore by going to **Connect Directly** option of the workspace your database is in and downloading the **TLS/SSL Certificate**
You can then upload the certificate in Hightouch in the **CA Certificate** section.
### Provide your credentials
Enter the following fields into Hightouch:
- **User**: A dedicated user for Hightouch.
- (Optional) **Password**: The password for the user specified above.
## Test your connection
When setting up SingleStore as a source for the first time, Hightouch validates your credentials and access to your database. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, you've completed setting up SingleStore as a source. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from SingleStore.
The SingleStore source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
## Tips and troubleshooting
---
## Snowflake
**URL:** https://hightouch.com/docs/sources/snowflake
**Description:** Hundreds of industry leaders use Hightouch to turn Snowflake into a marketing, sales, success and operational engine
**Section:** Sources
## Overview
Hightouch lets you pull data stored in your Snowflake warehouse and push it to downstream destinations.
You can connect Snowflake to Hightouch using [Snowflake Partner Connect](https://hightouch.com/blog/hightouch-snowflake-partner-connect) to bypass the setup steps outlined below. You can learn more about this in [Snowflake's documentation](https://docs.snowflake.com/en/user-guide/ecosystem-partner-connect).
Connecting Hightouch to Snowflake requires some setup in both platforms. It's recommended to set up a service account with the correct permissions in Snowflake before configuring the connection in Hightouch.
---
## Snowflake credential setup
To allow Hightouch access to Snowflake, it's best to create a user specifically provisioned with access to the required tables and schemas. You can also use a personal Snowflake login for your credentials, as long as it has the correct permissions.
You can use the following SQL template to create a service account with the necessary roles and permissions with some considerations:
- This snippet provides an example of creating a service account; you may need to alter it depending on your Snowflake implementation details.
- RSA key pair authentication setup is detailed in the [Provide credentials](#provide-credentials) section.
- If you want Hightouch to access multiple databases, run the snippet multiple times, changing `ht_database` each time.
- If you want Hightouch to have read-only access, omit the last line of the query.
- The snippet includes lines for creating a new warehouse and database for Hightouch to use; if you already have databases and warehouses you intend to use, omit these lines.
```sql
-- Edit the following variables
set ht_username='HIGHTOUCH_USER';
set ht_default_warehouse='';
set ht_database='';
set ht_default_namespace='';
set ht_default_role='HIGHTOUCH_ROLE';
set ht_comment='Used for Hightouch integrations';
-- Set role for grants
USE ROLE ACCOUNTADMIN;
-- Create a role for Hightouch
CREATE ROLE IF NOT EXISTS identifier($ht_default_role)
COMMENT = $ht_comment;
-- Only if you want to create a new warehouse for Hightouch to use
CREATE WAREHOUSE IF NOT EXISTS identifier($ht_default_warehouse);
-- Create Hightouch's user
CREATE USER IF NOT EXISTS identifier($ht_username)
TYPE=service
rsa_public_key=''
DEFAULT_WAREHOUSE=$ht_default_warehouse
DEFAULT_NAMESPACE=$ht_default_namespace
DEFAULT_ROLE=$ht_default_role
COMMENT=$ht_comment;
-- Grant permissions to the role
GRANT ROLE identifier($ht_default_role) TO ROLE SYSADMIN;
GRANT USAGE ON WAREHOUSE identifier($ht_default_warehouse) TO ROLE identifier($ht_default_role);
GRANT ROLE identifier($ht_default_role) TO USER identifier($ht_username);
-- Only if you want to create a new database for Hightouch to use
CREATE DATABASE IF NOT EXISTS identifier($ht_database);
GRANT USAGE ON DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT USAGE ON ALL SCHEMAS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON ALL TABLES IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON FUTURE TABLES IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON ALL VIEWS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON FUTURE VIEWS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
-- Grant write access for the Lightning Sync Engine (faster syncs) & Sync Logs - Recommended
USE identifier($ht_database);
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT OWNERSHIP ON SCHEMA hightouch_audit TO ROLE identifier($ht_default_role);
GRANT OWNERSHIP ON SCHEMA hightouch_planner TO ROLE identifier($ht_default_role);
```
If you opt to use [the Lightning engine](#choose-your-sync-engine), you must grant [additional permissions](#lightning-engine-setup).
Once you've created a Snowflake service account, you're ready to set up the connection in Hightouch.
---
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Snowflake** and follow the steps below.
### Configure your source
Enter the following required fields into Hightouch:
- **Account identifier**: You can find this identifier at the beginning of your Snowflake URL, for example, (`https://ACCOUNT_IDENTIFIER.snowflakecomputing.com`). Only enter the `ACCOUNT_IDENTIFIER` and not the full URL.
- **Warehouse**: This specifies the warehouse to use when Hightouch executes queries in Snowflake.
You can choose to route all queries to a single warehouse, or configure separate warehouses optimized for different workloads (recommended):
- **Route all queries to a single warehouse**: Use this option if you want to use a single warehouse for all Hightouch operations.
- **Route queries to different warehouses (split by workload)** (Recommended): This option allows you to optimize performance and cost by routing different types of queries to different warehouses:
- **Batch warehouse**: For background jobs like sync runs, schema sampling, audience snapshots, and other workloads that are not latency-sensitive. This warehouse can be optimized for cost efficiency.
- **Interactive warehouse**: For latency-sensitive queries (e.g., audience previews) and heavier workloads that need more compute (e.g., identity resolution). This warehouse should be sized up for best experience.
- **Database**: This specifies the database to use when Hightouch executes queries in Snowflake.
**Account identifier** format may differ based on Snowflake account age.
For example, older Snowflake accounts often have identifiers that look like `ACCOUNT_LOCATOR.CLOUD_REGION_ID.CLOUD`, whereas newer Snowflake accounts have identifiers that look like `ORGNAME-ACCOUNT_NAME`.
For more details, visit [Snowflake's account identifier docs](https://docs.snowflake.com/en/user-guide/admin-account-identifier.html).
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
To set up the Lightning engine, ensure that the service account you previously created has the appropriate permissions.
Specifically, append these commands to the [service account creation](#snowflake-credential-setup):
```sql
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT OWNERSHIP ON SCHEMA hightouch_audit TO ROLE identifier($ht_default_role);
GRANT OWNERSHIP ON SCHEMA hightouch_planner TO ROLE identifier($ht_default_role);
```
### Provide credentials
Enter the following fields into Hightouch:
- **Username**: This can be your personal Snowflake login or a dedicated user for Hightouch. When using the Basic engine, this user must have read access to the data you wish to sync. If using the Lightning engine, you must also grant the permissions described in the preceding snippet. **Note**: Use the "Login name", which may [differ from the Snowflake username](https://docs.snowflake.com/en/user-guide/admin-user-management#using-classic-console).
- **Role** (optional) - Use this field to specify the role Hightouch should use when executing queries in Snowflake. If left blank, Hightouch uses the user's default role.
You have two options for finalizing your credentials:
- RSA key pair (recommended)
- Password
Snowflake is [deprecating password
authentication](https://www.snowflake.com/en/blog/blocking-single-factor-password-authentification/) in favor of RSA key pair authentication or multi-factor authentication. Hightouch will continue to support password authentication for existing customers until they have migrated.
#### RSA authentication
Generate a [private key](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#step-1-generate-the-private-key) and [public key](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#step-2-generate-a-public-key) by running the following commands in your terminal:
```bash
$ openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8
$ openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub
```
**Save** the password you used for the key, you will need to enter it when creating the Snowflake source in Hightouch.
Then in Snowflake, execute an `ALTER USER` command to assign the public key to your Hightouch Snowflake user:
```bash
alter user ht_user set rsa_public_key='MIIBIjANBgkqh...';
```
In the `Authentication Method` of the Snowflake Source settings, drag and drop or upload the private key you just generated into the **Private key file** field. Enter the password for your private key in the box below here, unless you used an unencrypted key.
---
## Test your connection
---
## Manage warehouse compute
Hightouch runs queries in Snowflake to power audiences, syncs, and other workflows. Because Snowflake bills primarily on compute, your warehouse setup, schema design, and job frequency can affect both cost and performance.
This section outlines recommended patterns to help you:
- Keep Hightouch’s Snowflake usage predictable and easy to review
- Scope Hightouch’s access to only the data it needs
- Avoid unnecessary compute for common activation workflows
For a warehouse-agnostic overview, see [**Manage warehouse compute**](/sources/warehouse-compute).
---
### Recommended Snowflake setup
When you connect Snowflake to Hightouch, we recommend that you:
- **Use a dedicated user, role, and warehouse**
So compute usage and data access are easy to track and control.
- **Set practical warehouse defaults**
Start with a modest size, shorten idle time, and add reasonable query time limits.
- **Separate interactive and scheduled work when needed**
Run UI-driven queries and background jobs on different warehouses if you need stronger performance isolation.
- **Design models with query cost in mind**
Keep parent models focused, handle large event tables carefully, and combine tables only when it clearly helps.
- **Review usage regularly**
Use Snowflake’s cost dashboards, resource monitors, and query history to confirm performance and spend.
---
### Use a dedicated user, role, and warehouse
Snowflake organizes access and compute around three objects:
- A **user** (who runs queries)
- A **role** (what they can access)
- A **virtual warehouse** (where queries run)
Creating all three specifically for Hightouch makes both compute and data access easier to manage.
---
#### Scope access with a dedicated role
The role controls which objects Hightouch can read and write.
- Grant read access only to the databases and schemas that back your Hightouch models.
- Grant write access only when required (for example, for Warehouse Sync Logs or Hightouch-managed audit schemas).
- Avoid broad account-level privileges.
If you need additional governance, Snowflake supports:
- Masking policies on sensitive columns
- Row- or column-level security
- Views that expose filtered or redacted data
These controls ensure Hightouch can only access the data you intentionally expose.
---
#### Separate Hightouch from human users
Create a dedicated Snowflake user for Hightouch:
- Set its default role to the Hightouch role.
- Set its default warehouse to the Hightouch warehouse.
- Use strong authentication (for example, RSA key pair authentication).
With a dedicated user and role, you can filter query history to see exactly what Hightouch is running.
---
#### Isolate compute with a dedicated warehouse
The warehouse is the compute engine Snowflake bills.
- Create a virtual warehouse dedicated to Hightouch (for example, `HIGHTOUCH_WH`).
- Attach it to the Snowflake source configuration in Hightouch.
A dedicated warehouse makes it easier to:
- Attribute Snowflake credits to Hightouch
- Adjust warehouse settings without impacting other workloads
---
### Set practical warehouse defaults
Once Hightouch has its own warehouse, a few settings determine how it behaves.
#### Choose a starting size and adjust as needed
Start with a modest size rather than the largest option available. Many teams begin with **Medium** and adjust based on real usage.
Watch for:
- How long syncs and other Hightouch jobs take
- Whether Customer Studio previews remain responsive
- Whether Snowflake reports consistent queuing
If performance is acceptable and the warehouse is mostly idle, try a smaller size.
If you see frequent queuing or slow UI performance, increase the size or simplify models before scaling up.
#### Reduce idle time with auto-suspend
Auto-suspend controls how long the warehouse stays running when idle.
Snowflake’s default (often 10 minutes) is higher than many teams need. Reducing auto-suspend to around **1 minute**, while keeping auto-resume enabled, is common.
This helps the warehouse shut down soon after Hightouch stops issuing queries while still starting automatically when new work begins.
#### Add query time limits as a safeguard
Statement timeouts limit how long a query can run and help prevent unexpectedly long-running workloads.
The default timeout (2 days) is typically higher than necessary. Set a maximum duration that:
- Covers your typical sync runtimes
- Prevents a query from running indefinitely
If queries hit time limits:
- Review Snowflake query history
- Identify the models or filters causing long runtimes
- Simplify the query before increasing the timeout
#### Use multi-cluster only when concurrency is the issue (optional)
**Concurrency** (sometimes called parallelization) means how many queries run at the same time.
Multi-cluster warehouses can add extra clusters during busy periods to handle higher concurrency.
This can help when:
- Many Hightouch jobs run at once
- You see consistent queuing during peak times
- Usage spikes unpredictably
If a single warehouse handles your workload without queuing, multi-cluster may not be necessary.
If you choose to go with a larger warehouse (M/L), you can ignore this suggestion.
---
### Separate interactive and scheduled work
Hightouch issues both interactive and batch queries:
- **Interactive:** audience previews, traits exploration, dashboards, and other UI actions
- **Batch:** sync runs, Identity Resolution jobs, heavy traits, sampling refreshes, and snapshots
If needed, configure Hightouch to run these on different Snowflake warehouses.
#### Common pattern: two warehouses
Many teams use:
- An **interactive warehouse** for fast UI queries
- A **batch warehouse** for scheduled and background jobs
To set this up:
1. Create two warehouses (for example, `HIGHTOUCH_INTERACTIVE_WH` and `HIGHTOUCH_BATCH_WH`).
2. Configure Hightouch to send interactive work to one and batch work to the other.
3. Set size and auto-suspend settings independently.
This keeps the UI responsive during heavy sync windows and gives you clearer cost control.
---
### Design Customer Studio models with query cost in mind
Warehouse configuration determines how much compute is available. [Schema design](/customer-studio/schema) strongly affects how much work each query must do and which data Hightouch touches.
#### Keep parent and related models simple
For Customer Studio:
- The parent model should represent the main entity (for example, customer or account).
- Include stable identifiers and frequently used segmentation fields.
- Include keys needed to join related models.
Related models can represent 1:many relationships such as orders or sessions.
Where possible, do complex joins and calculations ahead of time (for example, in dbt), and have Hightouch read from tables you’ve already built instead of re-running large joins every time.
#### Limit event tables to what activation requires
Event tables can grow very large. To keep queries manageable:
- Limit time windows to what activation actually needs
- Organize tables based on common filters (often date or tenant)
- Create smaller summary tables for common metrics when helpful
[**Traits**](/customer-studio/traits) can perform aggregations directly, which may reduce the need for additional summary tables.
#### Combine tables only when it clearly helps
As a general guideline:
- Keep data in separate tables by default.
- Combine tables only when it clearly improves common Hightouch queries.
- Avoid flattening very large tables unless it solves a specific performance problem.
For large or time-series tables, consider clustering once you’ve observed real query patterns in Snowflake.
---
### Improve audience building performance
If Customer Studio previews feel slow, you can increase warehouse size—but before doing that, consider enabling [**sampling**](/customer-studio/sampling).
- Sampling runs previews against a subset of rows for faster results.
- You can still run full queries for production syncs or reporting.
Sampling often provides a strong balance between responsiveness and Snowflake cost.
---
### Reduce cost
Overspending typically results from multiple factors rather than a single issue.
#### Reduce unnecessary query volume
Running more queries than needed increases cost by:
- Keeping warehouses active longer
- Increasing concurrency, which may require larger warehouses
If you need frequent updates, consider running those workloads on a right-sized warehouse and separating them from interactive work.
#### Right-size warehouses for your workload
Many teams overspend because their warehouse is larger than necessary.
Aim for balance: enough capacity to meet performance expectations without paying for idle time or excess headroom.
---
### Monitor cost and usage
Use Snowflake’s built-in tools to validate Hightouch usage.
#### Review cost dashboards
- Filter by warehouse name to track credit usage
- Compare Hightouch warehouses with other workloads
#### Use budgets and resource monitors
- Set budgets for Hightouch warehouses
- Configure alerts or automatic suspension for unexpected spikes
#### Inspect query history
- Filter by Hightouch user, role, or warehouse
- Identify expensive or slow queries
- Tie them back to specific models or workflows
Regular monitoring ensures Hightouch stays aligned with your performance, cost, and governance expectations.
---
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Snowflake.
The Snowflake source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Looker Looks](/extensions/looker-models)
- leveraging existing [Sigma workbooks](/extensions/sigma)
The [SQL editor](/models/sql-editor) allows you to query data from all databases that your Snowflake service account [has access to](#snowflake-credential-setup).
The [table selector](/models/table-selector) only supports querying from the database specified in the [source configuration](#configure-your-source).
---
## FAQs
### How can I tell how much compute Hightouch is using?
If you’re using a dedicated warehouse for Hightouch, most high-level cost questions can be answered directly in the Snowflake console.
1. Go to **Admin → Cost Management**.
2. Open the **Consumption** tab.
3. Filter by your Hightouch warehouse name.
This view shows:
- Credit usage over time
- Breakdown by warehouse
- Query and service-level cost trends
You must be an `ACCOUNTADMIN` or have the necessary permissions to access these views.
If Hightouch uses its own warehouse, this report provides a clear view of total Snowflake spend attributable to Hightouch workloads.
---
### How can I identify specific queries driving cost?
For more detailed analysis, Snowflake exposes usage and query telemetry in the shared `SNOWFLAKE` database.
Two views are especially useful:
#### `SNOWFLAKE.ACCOUNT_USAGE.QUERY_ATTRIBUTION_HISTORY`
This view includes the `CREDITS_ATTRIBUTED_COMPUTE` column, which reports how many credits were consumed by each query.
- It accounts for concurrent queries by allocating weighted compute usage.
- It excludes idle time between query completion and warehouse shutdown.
- It allows you to identify the most expensive individual queries.
#### `SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY`
This related view includes:
- Full SQL query text
- Query duration
- Bytes scanned and written
- Rows produced
- Queuing time
- Spill metrics
Together, these views help you determine:
- Which queries consume the most credits
- Whether queuing is contributing to performance issues
- Whether large scans or joins are driving cost
The easiest way to access recent query history for a specific warehouse is via the table function `SNOWFLAKE.INFORMATION_SCHEMA.QUERY_HISTORY_BY_WAREHOUSE`, which allows filtering by `WAREHOUSE_NAME`.
#### Example: Identify expensive Hightouch queries in the last day
The following example joins query attribution and warehouse-scoped history to show the most expensive queries executed on a specific Hightouch warehouse in the last 24 hours:
```sql
SELECT
QAH.*,
QHBW.QUERY_TEXT,
QHBW.QUEUED_PROVISIONING_TIME,
QHBW.ROWS_WRITTEN_TO_RESULT
FROM
SNOWFLAKE.ACCOUNT_USAGE.QUERY_ATTRIBUTION_HISTORY AS QAH
JOIN (
SELECT
*
FROM
TABLE(
SNOWFLAKE.INFORMATION_SCHEMA.QUERY_HISTORY_BY_WAREHOUSE(
WAREHOUSE_NAME => 'HIGHTOUCH_SYNCS',
RESULT_LIMIT => 10000
)
)
) AS QHBW
ON QHBW.QUERY_ID = QAH.QUERY_ID
WHERE
QAH.START_TIME > CURRENT_DATE - INTERVAL '1 DAY'
ORDER BY
QAH.CREDITS_ATTRIBUTED_COMPUTE DESC;
```
You can modify:
- `WAREHOUSE_NAME` to match your Hightouch warehouse
- The time window in the `WHERE` clause
- The `RESULT_LIMIT` parameter
This query helps you:
- Rank queries by compute consumption
- Inspect SQL text to identify inefficient joins or scans
- Confirm that configuration changes (warehouse size, routing, schema adjustments) are having the intended effect
---
## Tips and troubleshooting
### Connection timeout
When initially [testing your connection](#test-your-connection), you may receive a connection timeout error.
Once a connection is established, subsequent API requests should happen more quickly, so it's best to retry the tests if they first fail.
You can do this by clicking **Test again**.
### Network error: Could not reach Snowflake
You may receive this error if the input for [Account identifier](#configure-your-source) is invalid.
Instead of using the complete Snowflake URL, for example, `https://ACCOUNT_IDENTIFIER.snowflakecomputing.com`,
ensure that you're only using the `ACCOUNT_IDENTIFIER` part of the URL, for example, `companyname-xu12345` or `companyname-xu12345.us-east-1`.
### SQL compilation error with quoted identifiers
You may receive a `SQL compilation error: invalid identifier '"column name"'` error when your Snowflake table has column names that contain spaces or special characters. Snowflake requires [double-quoted identifiers](https://docs.snowflake.com/en/sql-reference/identifiers-syntax#double-quoted-identifiers) for these columns, but Hightouch's query engine may not handle them correctly in all contexts.
To work around this, rename the columns in a CTE or view before using them in your Hightouch model:
```sql
SELECT "job number" AS job_number, "first name" AS first_name
FROM my_schema.my_table
```
### JWT token is invalid
You may receive intermittent `JWT token is invalid` errors when using [key pair authentication](https://docs.snowflake.com/en/user-guide/key-pair-auth). This is typically caused by clock skew between your Snowflake account and Hightouch's servers, and is more common after configuring dbt models or other tools that also use key pair auth.
Resetting the key pair in Snowflake and re-configuring the connection in Hightouch may temporarily resolve the issue. If the error recurs, contact support for further troubleshooting.
---
## Snowflake Iceberg
**URL:** https://hightouch.com/docs/sources/snowflake-iceberg
**Description:** Hundreds of industry leaders use Hightouch to turn Snowflake into a marketing, sales, success and operational engine
**Section:** Sources
## Overview
Hightouch lets you pull data stored in Iceberg Tables through your Snowflake warehouse and push it to downstream destinations.
You can connect Snowflake to Hightouch using [Snowflake Partner Connect](https://hightouch.com/blog/hightouch-snowflake-partner-connect) to bypass the setup steps outlined below. You can learn more about this in [Snowflake's documentation](https://docs.snowflake.com/en/user-guide/ecosystem-partner-connect).
Connecting Hightouch to Snowflake requires some setup in both platforms. It's recommended to set up a service account with the correct permissions in Snowflake before configuring the connection in Hightouch.
## Snowflake credential setup
To allow Hightouch access to Snowflake, it's best to create a user specifically provisioned with access to the required tables and schemas. You can also use a personal Snowflake login for your credentials, as long as it has the correct permissions.
You can use the following SQL template to create a service account with the necessary roles and permissions with some considerations:
- This snippet provides an example of creating a service account; you may need to alter it depending on your Snowflake implementation details.
- RSA key pair authentication setup is detailed in the [Provide credentials](#provide-credentials) section.
- If you want Hightouch to access multiple databases, run the snippet multiple times, changing `ht_database` each time.
- If you want Hightouch to have read-only access, omit the last line of the query.
- The snippet includes lines for creating a new warehouse and database for Hightouch to use; if you already have databases and warehouses you intend to use, omit these lines.
- Iceberg tables have limited support for timestamps with timezones. We recommend setting the default timezone of the Hightouch user to `UTC` to maintain consistent behavior.
```sql
-- Edit the following variables
set ht_username='HIGHTOUCH_USER';
set ht_default_warehouse='';
set ht_database='';
set ht_default_namespace='';
set ht_default_external_volume=''
set ht_default_role='HIGHTOUCH_ROLE';
set ht_comment='Used for Hightouch integrations';
-- Set role for grants
USE ROLE ACCOUNTADMIN;
-- Create a role for Hightouch
CREATE ROLE IF NOT EXISTS identifier($ht_default_role)
COMMENT = $ht_comment;
-- Only if you want to create a new warehouse for Hightouch to use
CREATE WAREHOUSE IF NOT EXISTS identifier($ht_default_warehouse);
-- Create Hightouch's user
CREATE USER IF NOT EXISTS identifier($ht_username)
TYPE=service
rsa_public_key=''
DEFAULT_WAREHOUSE=$ht_default_warehouse
DEFAULT_NAMESPACE=$ht_default_namespace
DEFAULT_ROLE=$ht_default_role
DEFAULT_TIMEZONE = 'UTC'
COMMENT=$ht_comment;
-- Grant permissions to the role
GRANT ROLE identifier($ht_default_role) TO ROLE SYSADMIN;
GRANT USAGE ON WAREHOUSE identifier($ht_default_warehouse) TO ROLE identifier($ht_default_role);
GRANT USAGE ON EXTERNAL VOLUME identifier($ht_default_external_volume) TO ROLE identifier($ht_default_role);
GRANT ROLE identifier($ht_default_role) TO USER identifier($ht_username);
-- Only if you want to create a new database for Hightouch to use
CREATE DATABASE IF NOT EXISTS identifier($ht_database);
GRANT USAGE ON DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT USAGE ON ALL SCHEMAS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON ALL TABLES IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON FUTURE TABLES IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON ALL VIEWS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON FUTURE VIEWS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
-- Grant write access for the Lightning Sync Engine (faster syncs) & Sync Logs - Recommended
USE identifier($ht_database);
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT OWNERSHIP ON SCHEMA hightouch_audit TO ROLE identifier($ht_default_role);
GRANT OWNERSHIP ON SCHEMA hightouch_planner TO ROLE identifier($ht_default_role);
```
If you opt to use [the Lightning engine](#choose-your-sync-engine), you must grant [additional permissions](#lightning-engine-setup).
Once you've created a Snowflake service account, you're ready to set up the connection in Hightouch.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Snowflake Iceberg** and follow the steps below.
### Configure your source
Enter the following required fields into Hightouch:
- **Account identifier**: You can find this identifier at the beginning of your Snowflake URL, for example, (`https://ACCOUNT_IDENTIFIER.snowflakecomputing.com`). Only enter the `ACCOUNT_IDENTIFIER` and not the full URL.
- **Warehouse**: This specifies the warehouse to use when Hightouch executes queries in Snowflake.
- **Database**: This specifies the database to use when Hightouch executes queries in Snowflake.
- **External volume**: This specifies the integration point between Snowflake and your cloud provider's storage.
- **Base location**: This specifies the prefix within the External Volume where Iceberg tables will be created.
**Account identifier** format may differ based on Snowflake account age.
For example, older Snowflake accounts often have identifiers that look like `ACCOUNT_LOCATOR.CLOUD_REGION_ID.CLOUD`, whereas newer Snowflake accounts have identifiers that look like `ORGNAME-ACCOUNT_NAME`.
For more details, visit [Snowflake's account identifier docs](https://docs.snowflake.com/en/user-guide/admin-account-identifier.html).
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
To set up the Lightning engine, ensure that the service account you previously created has the appropriate permissions.
Specifically, append these commands to the [service account creation](#snowflake-credential-setup):
```sql
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT OWNERSHIP ON SCHEMA hightouch_audit TO ROLE identifier($ht_default_role);
GRANT OWNERSHIP ON SCHEMA hightouch_planner TO ROLE identifier($ht_default_role);
```
### Provide credentials
Enter the following fields into Hightouch:
- **Username**: This can be your personal Snowflake login or a dedicated user for Hightouch. When using the Basic engine, this user must have read access to the data you wish to sync. If using the Lightning engine, you must also grant the permissions described in the preceding snippet. **Note**: Use the "Login name", which may [differ from the Snowflake username](https://docs.snowflake.com/en/user-guide/admin-user-management#using-classic-console).
- **Role** (optional) - Use this field to specify the role Hightouch should use when executing queries in Snowflake. If left blank, Hightouch uses the user's default role.
You have two options for finalizing your credentials:
- RSA key pair (recommended)
- Password
Snowflake is [deprecating password
authentication](https://www.snowflake.com/en/blog/blocking-single-factor-password-authentification/) in favor of RSA key pair authentication or multi-factor authentication. Hightouch will continue to support password authentication for existing customers until they have migrated.
#### RSA authentication
Generate a [private key](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#step-1-generate-the-private-key) and [public key](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#step-2-generate-a-public-key) by running the following commands in your terminal:
```bash
$ openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8
$ openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub
```
Then in Snowflake, execute an `ALTER USER` command to assign the public key to your Hightouch Snowflake user:
```bash
alter user ht_user set rsa_public_key='MIIBIjANBgkqh...';
```
In the `Authentication Method` of the Snowflake Source settings, drag and drop or upload the private key you just generated into the **Private key file** field.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Snowflake.
The Snowflake Iceberg source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
The [SQL editor](/models/sql-editor) allows you to query data from all databases that your Snowflake service account [has access to](#snowflake-credential-setup).
The [table selector](/models/table-selector) only supports querying from the database specified in the [source configuration](#configure-your-source).
## Tips and troubleshooting
### Connection timeout
When initially [testing your connection](#test-your-connection), you may receive a connection timeout error.
Once a connection is established, subsequent API requests should happen more quickly, so it's best to retry the tests if they first fail.
You can do this by clicking **Test again**.
### Network error: Could not reach Snowflake
You may receive this error if the input for [Account identifier](#configure-your-source) is invalid.
Instead of using the complete Snowflake URL, for example, `https://ACCOUNT_IDENTIFIER.snowflakecomputing.com`,
ensure that you're only using the `ACCOUNT_IDENTIFIER` part of the URL, for example, `companyname-xu12345` or `companyname-xu12345.us-east-1`.
---
## Snowflake RBAC
**URL:** https://hightouch.com/docs/sources/snowflake-rbac
**Description:** Learn how Hightouch honors your Snowflake data governance in in-app previews by mapping user groups to Snowflake roles, so masking policies, row access policies, and role grants apply as they do in Snowflake.
**Section:** Sources
## Overview
Snowflake RBAC lets Hightouch honor your Snowflake data governance when it runs in-app preview queries. When it's enabled, Hightouch maps your user groups to Snowflake roles and runs each user's previews under their resolved role, so Snowflake's masking policies, row access policies, and role grants apply exactly as they do in Snowflake itself.
Without it, every Hightouch warehouse query runs under the single [service account](/sources/snowflake#separate-hightouch-from-human-users) configured on the source. With it, a marketer browsing an audience preview sees the same masked columns and filtered rows that Snowflake would show that marketer directly.
This applies to **in-app previews** only. Syncs, materializations, and other non-preview operations continue to run under the source's service account. See [Scope](#scope) below.
---
## Scope
Snowflake RBAC governs the queries Hightouch runs on your behalf to show you data in the app:
- Model and rETL previews
- Audience previews and the audience member browser
- Trait previews
- Journey-condition previews
- Table browsing and ad-hoc counts
- Raw SQL previews
The following keep running under the service account, by design:
- **Sync runs and materializations.** These are pipeline operations, not per-user views of data.
- **Audience size estimates and live counts.** These are served from a shared cache, so they are not scoped to an individual user's role.
- **Charts.** Charts read pre-computed aggregates produced by syncs running under the service account. To keep governed values out of charts, exclude them from the models that feed your charts.
---
## How it works
- **Per source.** RBAC is configured on each Snowflake source independently.
- **Role mappings.** You map a Hightouch user group to a Snowflake role. Each mapping has a priority.
- **Default role.** The role used for previews when no mapping matches the user. Saving a default role is what enables RBAC on the source.
- **Resolution.** For a given user, Hightouch uses the highest-priority mapping that matches any of the user's groups. If no mapping matches, it uses the default role.
- **Role chip.** Governed previews display a chip showing which role the query ran as and why (for example, *Ran as `MARKETER` · via group mapping*). The chip reflects the role verified at query time.
A workspace with RBAC available but no default role saved behaves exactly as it does today: all queries run under the service account.
---
## Set up Snowflake
Create the roles you want to map and grant them to the Snowflake user your Hightouch source authenticates as. A role must be granted to that user, or Hightouch cannot run queries under it.
```sql
USE ROLE ACCOUNTADMIN;
-- Roles you will map to Hightouch user groups
CREATE ROLE IF NOT EXISTS ht_analyst;
CREATE ROLE IF NOT EXISTS ht_marketer;
-- Each mapped role needs the usage and read it requires
GRANT USAGE ON WAREHOUSE TO ROLE ht_analyst;
GRANT USAGE ON WAREHOUSE TO ROLE ht_marketer;
GRANT USAGE ON DATABASE TO ROLE ht_analyst;
GRANT USAGE ON DATABASE TO ROLE ht_marketer;
GRANT USAGE ON SCHEMA . TO ROLE ht_analyst;
GRANT USAGE ON SCHEMA . TO ROLE ht_marketer;
GRANT SELECT ON ALL TABLES IN SCHEMA . TO ROLE ht_analyst;
GRANT SELECT ON ALL TABLES IN SCHEMA . TO ROLE ht_marketer;
-- The mapped roles must be granted to the source's Snowflake user
GRANT ROLE ht_analyst TO USER ;
GRANT ROLE ht_marketer TO USER ;
```
Define governance against the role names with Snowflake's own [masking policies](https://docs.snowflake.com/en/user-guide/security-column-intro) and [row access policies](https://docs.snowflake.com/en/user-guide/security-row-intro), keyed on `CURRENT_ROLE()`. For example, a masking policy that returns raw `email` only for `HT_ANALYST` and masks it otherwise.
---
## Set up role mappings in Hightouch
1. Open your Snowflake source and go to the **Role Mapping** tab.
2. Click **Refresh roles**. Hightouch discovers the roles currently granted to the source's Snowflake user and records them.
3. Set a **default role**. Saving the default role enables RBAC for this source.
4. Add **mappings** from user groups to roles, and drag to set their **priority**. The highest-priority matching mapping wins.
---
## Keeping roles in sync
- **Refresh is manual.** Hightouch discovers roles when you click **Refresh roles**. There is no background polling.
- **Revoked roles.** If a mapped role is later revoked from the source's Snowflake user, it is marked unavailable at the next refresh and skipped: resolution falls through to the next available mapping in priority order, then to the default role. It never widens to the service account.
- **No usable role.** If none of a user's mapped roles is available and the default role is also unavailable, that user's previews are blocked until a role is restored or re-mapped, rather than falling back to the broader service account.
- **Connection changes.** If the source's connection identity (account, login user, or authentication) changes, all previews on the source are blocked until you refresh roles under the new identity. This prevents serving data under a governance context that no longer matches the connection.
---
## Limitations
- **Snowflake only.** Role-based preview governance is available for Snowflake sources. Other warehouses are not supported.
- **Previews only.** As described in [Scope](#scope), syncs, materializations, audience size, live counts, and charts run under the service account.
- **Group-based mapping.** Mappings are from Hightouch user groups to Snowflake roles. Per-user and identity-provider-driven mapping are not supported.
---
## SQL Server
**URL:** https://hightouch.com/docs/sources/sqlserver
**Description:** SQL Server is a relational database management system, or RDBMS, developed and marketed by Microsoft.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in Microsoft SQL Server and push it to downstream destinations. Connecting Hightouch to Microsoft SQL Server may require some setup in both platforms. If using an Azure provisioned Microsoft SQL Server, you need to configure your firewall settings.
## Azure firewall setup
*This section only applies when connecting to SQL Server instances hosted in Azure.*
By default, Azure protects your SQL Server from any external IP address from connecting to your SQL Server. Follow these steps to add Hightouch's IP addresses to the firewall:
1. Log into your [Azure Console Dashboard](https://portal.azure.com/#home).
2. Click on **SQL databases** under **Services**.
3. Select the SQL database you want to update.
4. Select **Set server firewall** .
5. Under **Public network access**, select **Selected networks**.
6. Scroll to **Firewall rules** and click **Add a firewall rule**.
7. Allowlist the [Hightouch IP addresses](https://hightouch.com/docs/security/networking#ip-addresses) for your region.
8. Click the **Save** button on the bottom left.
## Connection configuration
In Hightouch, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Microsoft SQL Server** and follow the steps below.
### Choose authentication method
When creating a SQL Server source, you can either enter the Host, Port, Database, User, and Password by selecting **Password** or you can provide the full URI **Connection string**.
#### Connection string
Enter your connection string in this format: `Server=server;Database=database;User Id=username;Password=password;Encrypt=boolean`.
Once you've entered your connection string and clicked **Continue**, you can skip to the [Test your Connection](#test-your-connection) section of this doc.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your Microsoft SQL Server. If using the hostname, make sure to only insert the hostname and not the full URL (remove the leading `http://` and the final `/`).
- **Port**: The port number of your Microsoft SQL Server. The default is 1433, but yours may be different.
- **Database**: The name of the database in your Microsoft SQL Server.
- **User**: This can be your personal Microsoft SQL Server login or a dedicated user for Hightouch.
- (Optional) **Password**: The password for the user specified above.
Optionally you can enter a **Request timeout duration** and whether to **Trust server certification**.
## Test your connection
When setting up Microsoft SQL Server as a source for the first time, Hightouch validates your credentials and access to your database. Once the test passes, click **Continue** to finish setup.
If you see the following error message, you need to [update your firewall settings](#azure-firewall-setup) in your Azure Console.
```json
Cannot open server 'hightouch-test' request by login.
Client with IP address '54.196.30.169' is not allowed to access the server.
To enable access, use the Windows Azure Management Portal or run sp_set_firewall_rule
on the master database to create a firewall rule for this IP address or address range.
It may up take up to five minutes for this change to take effect.
```
## Next steps
Once your source configuration has passed the necessary validation, you've completed setting up Microsoft SQL Server as a source. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Microsoft SQL Server.
The Microsoft SQL Server source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
## Tips and troubleshooting
Most issues with the Microsoft SQL Server source can be resolved by verifying the firewall settings according to the setup instructions above.
If you see an error message similar to the following
```json
Login failed for user ''
```
or
```json
login failed for user '\'
```
verify that the username and password work locally. You may also need to adjust your Microsoft SQL Service authentication mode. See [Microsoft's documentation](https://learn.microsoft.com/en-us/sql/relational-databases/errors-events/mssqlserver-18456-database-engine-error?view=sql-server-ver16#login-failed-for-user-username-or-login-failed-for-user-domainusername) for more information about login errors and authentication modes.
---
## Azure Synapse
**URL:** https://hightouch.com/docs/sources/synapse
**Description:** Hightouch can turn Azure Synapse into a marketing, sales, success and operational engine.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in Azure Synapse and push it to downstream destinations. Connecting Hightouch to Azure Synapse requires some setup in both platforms. In Azure Synapse, you need to configure your firewall settings.
## Azure firewall setup
By default, Azure protects your Synapse data from any external IP address from connecting to your Synapse workspace. Follow these steps to add Hightouch's IP addresses to the firewall.
1. Log into your [Azure Console Dashboard](https://portal.azure.com/#home).
2. Click on your Synapse workspace under **Resources**.
3. In the left sidebar, select **Security** > **Networking** .
4. Allowlist the [Hightouch IP addresses](https://hightouch.com/docs/security/networking#ip-addresses) for your region.
5. Click the **Save** button in upper left corner.
## Connection configuration
In Hightouch, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Azure Synapse** and follow the steps below.
### Choose authentication method
When creating an Azure synapse source, you have four options to authenticate:
- **Password**: Use a username and password
- **Connection String**: Provide the full URI Connection String
- **Microsoft Service Principal with Client Secret**: Use a Microsoft Service Principal created using Microsoft Entra ID with a Client Secret
- **Microsoft Service Principal with Certificate**: Use a Microsoft Service Principal created using Microsoft Entra ID with a signed Certificate
### Choose connection type
### Configure your source
If connecting via **Password** or **Microsoft Service Principal**, you must first enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your Azure Synapse.
- **Port**: The port number of your Azure Synapse. The default is 1433, but yours may be different.
- **Database**: The name of the database in your Azure Synapse.
Optionally you can enter a **Request timeout duration** and whether to **Trust server certification**.
### Password
To connect using a username and password, enter the following fields into Hightouch:
- **Username**: This can be your personal Azure Synapse login or a dedicated user for Hightouch.
- **Password**: The password for the user specified above.
### Microsoft Service Principal
To connect using a Microsoft Service Principal, use the following instructions to create a new Service Principal:
1. Sign in to the [Microsoft Entra admin center](https://entra.microsoft.com/).
2. On the sidebar, select **Identity** > **Applications** > **App registrations**.
3. Select **New registration**.
4. Pick a name for your application and select **Accounts in this organizational directory only** as the supported account type.
5. Click **Register**.
6. Note your newly created **Application (client) ID** and **Directory (tenant) ID**.
7. Click **Add a certificate or secret**.
From here, you can opt to use either a **Client secret** or a **Certificate**.
#### Client secret
1. Select the **Client secrets** tab.
2. Click **New client secret**.
3. Name your client secret and select its expiration date. To maintain access to Azure Synapse, make sure to refresh your client secret prior to its expiration.
4. Note your client secret **Value** now as it cannot be retrieved later. It will be used to connect Hightouch to Azure Synapse.
#### Certificate
1. Select the **Certificates** tab.
2. Click **Upload certificate** and upload a valid signed certificate.
3. Note your certificate's **Thumbprint**.
For more detailed instructions on service principal creation, you can check out [Microsoft's documentation](https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal).
#### Synapse configuration
Next, we need to grant the service principal access to the Azure Synapse instance:
1. Navigate to your instance with the [Azure Portal](https://portal.azure.com/#home)
2. On the sidebar, select **Access control (IAM)**
3. Click **Add** > **Add role assignments**
4. Select the role you would like to assign to your service principal (Hightouch will need at least the **Reader** role)
5. On the next screen, select the service principal that was created above and apply the changes.
For more details, please take a look at [Microsoft's documentation](https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-portal).
#### Credentials
Finally, you can enter the following fields into Hightouch that were created during the above instructions:
- **Tenant ID** (or Directory ID)
- **Client ID** (or Application ID)
If you chose to create a **Client Secret** for your service principal, you'll enter it here.
If you instead chose to create a **Certificate**, enter in your certificate's **Thumbprint** and the associated **private key**. Make sure that the private key is in PEM format and includes the header and footer.
### Connection String
If connecting via **Connection String**, simply enter your connection string in this format: `Server=server;Database=database;User Id=username;Password=password;Encrypt=boolean`.
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
Please note that Lightning engine is only supported for Azure Synapse instances that use a [dedicated SQL pool](https://learn.microsoft.com/en-us/azure/synapse-analytics/sql-data-warehouse/sql-data-warehouse-overview-what-is). Serverless Azure Synapse does not have the capabilities to support the Lightning engine.
To set up the Lightning engine, run the following commands. Make sure to replace **your_service_principal_or_user** with the name of the service principal or user used to connect to your Synapse instance.
```sql
IF NOT EXISTS (SELECT 1 FROM sys.schemas WHERE name = 'hightouch_audit')
BEGIN
EXEC('CREATE SCHEMA [hightouch_audit]');
END;
IF NOT EXISTS (SELECT 1 FROM sys.schemas WHERE name = 'hightouch_planner')
BEGIN
EXEC('CREATE SCHEMA [hightouch_planner]');
END;
GRANT CONTROL ON SCHEMA::hightouch_audit TO [your_service_principal_or_user];
GRANT CONTROL ON SCHEMA::hightouch_planner TO [your_service_principal_or_user];
```
## Test your connection
When setting up Azure Synapse as a source for the first time, Hightouch validates your credentials and access to your database. Once the test passes, click **Continue** to finish setup.
If you see the following error message, you need to [update your firewall settings](#azure-firewall-setup) in your Azure Console.
```
Cannot open server 'hightouch-test' request by login. Client with IP address '54.196.30.169' is not allowed to access the server. To enable access, use the Windows Azure Management Portal or run sp_set_firewall_rule on the master database to create a firewall rule for this IP address or address range. It may up take up to five minutes for this change to take effect.
```
## Next steps
Once your source configuration has passed the necessary validation, you've completed setting up Azure Synapse as a source. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Azure Synapse.
The Azure Synapse source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
## Tips and troubleshooting
---
## Teradata Vantage
**URL:** https://hightouch.com/docs/sources/teradata-vantage
**Description:** Teradata Corporation is a leading data and analytics company that specializes in providing enterprise-level data warehousing, data management, and advanced analytics solutions
**Section:** Sources
## Overview
Hightouch lets you pull data from Teradata Vantage and push it to downstream destinations. Hightouch connects to Teradata using [Open Database Connectivity](https://en.wikipedia.org/wiki/Open_Database_Connectivity) (ODBC).
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Teradata Vantage** and follow the steps below.
### Choose connection type
### Configure your source
- **Host**: The hostname or IP address of your Teradata instance.
- **Username**: This can be your personal Teradata Vantage login or a dedicated user for Hightouch.
- **Password**: The password for the specified user.
## Test your connection
When setting up Teradata Vantage as a source for the first time, Hightouch validates your credentials. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models)
to define which data you want to pull via Teradata Vantage.
The Teradata Vantage source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
## Tips and troubleshooting
---
## Trino
**URL:** https://hightouch.com/docs/sources/trino
**Description:** Trino is an open-source distributed SQL query engine designed to query large data sets distributed over one or more heterogeneous data sources.
**Section:** Sources
## Getting started
Hightouch lets you pull data stored in [TrinoDB](https://trino.io/) (previously PrestoSQL) and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to your Trino instance for information like your host, port, database name, and credentials.
Hightouch can also connect to managed Trino services like Starburst.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Trino** and follow the steps below.
### Choose connection type
### Configure your source
Enter the following fields into Hightouch:
- **Host**: The hostname or IP address of your Trino coordinator server.
- **Port**: The port number of your Trino coordinator server.
- (Optional) **Default catalog**: This specifies the catalog that will be used when Hightouch executes queries in Trino.
- **Engine**: Select whether to use the Trino or Presto engine to execute queries.
- **Connector**: Select the connector that will be used to access the data source used by the sync engine. We support the following connectors:
- Hive
- Iceberg
- PostgreSQL
- (Optional) **Disable encryption**: Encryption is enabled by default. Check this option if your Trino instance doesn't support encrypted connections.
### Choose your sync engine
#### Basic versus Lightning engine comparison
#### Lightning engine setup
To set up the Lightning engine, you need to grant Hightouch write access to your Trino connector. The steps to do so depend on the connector. These snippets serve as a reference and should be run on your Trino server.
**Hive & Iceberg**
```sql
CREATE SCHEMA IF NOT EXISTS .hightouch_audit;
CREATE SCHEMA IF NOT EXISTS .hightouch_planner;
```
**PostgreSQL**
```sql
CREATE USER hightouch_user WITH PASSWORD '********';
CREATE SCHEMA IF NOT EXISTS .hightouch_audit;
CREATE SCHEMA IF NOT EXISTS .hightouch_planner;
GRANT CREATE, USAGE ON SCHEMA .hightouch_audit TO hightouch_user;
GRANT CREATE, USAGE ON SCHEMA .hightouch_planner TO hightouch_user;
```
The snippet provisions two schemas (`hightouch_planner` and `hightouch_audit`) for storing logs of previously synced data and, in some cases, creates a dedicated SQL user. Hightouch must be able to read and write to these schemas, but the specific username and schema names might vary.
### Provide credentials
Enter the following fields into Hightouch:
- **User**: This can be your personal Trino login or a dedicated user for Hightouch. At minimum, this user must have read access to the data you wish to sync. If using the Lightning sync engine, you must also grant this user additional permissions as described above.
- **Password**: The password for the user specified above.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Trino.
The Trino source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
## Data types
Hightouch supports all built-in [Trino data types](https://trino.io/docs/current/language/types.html). Custom data types provided by plugins are represented as strings.
## Tips and troubleshooting
---
## Manage warehouse compute
**URL:** https://hightouch.com/docs/sources/warehouse-compute
**Description:** Learn how warehouse compute works with Hightouch and how to optimize cost, performance, and governance through warehouse configuration, schema design, and monitoring strategies.
**Section:** Sources
## Overview
When you connect Hightouch to your warehouse as a source, Hightouch runs directly in that warehouse. When you build audiences, run syncs, calculate traits, or execute journeys in Hightouch, your warehouse runs SQL queries to power those actions.
Because of this, Hightouch contributes to your **warehouse compute** usage.
Many customers find that Hightouch accounts for less than 1% of their annual warehouse compute costs. This guide explains how to plan for, monitor, and optimize that usage intentionally.
You'll learn:
- What typically increases warehouse cost
- How your data structure affects performance
- What you can change in your warehouse
- What you can change in Hightouch
---
## How warehouse billing works
Most warehouses charge separately for:
- **Compute** – the processing power used to run queries
- **Storage** – how much data is stored
- **Platform services** – management and metadata features
Hightouch primarily affects **compute**.
Each time your warehouse evaluates an audience, calculates a trait, runs a sync, or processes a journey step, compute is consumed.
---
## What usually increases warehouse compute
Warehouse cost typically goes up for one of five reasons:
1. Queries run more often
2. Queries do more work
3. More queries run at the same time
4. The warehouse is not the right size for the workload
5. The warehouse stays on longer than necessary
### 1. Queries run more often
More frequent queries increase total compute usage. Common drivers include:
- [Syncs](/syncs/overview) scheduled more frequently
- [Triggered journeys](/customer-studio/journeys) running continuously
- [Audiences](/customer-studio/usage) recalculating often
If a use case doesn’t require real-time updates, consider lowering the schedule. Many background jobs work just as well daily or weekly.
### 2. Queries do more work
Queries that perform large joins, heavy aggregations, or full-table scans consume more compute.
Queries become more expensive when they:
- Join large tables repeatedly
- Scan long time ranges of event data
- Perform heavy calculations each time they run
To reduce compute, move complex joins or calculations earlier in your data pipeline (for example, in dbt) and have Hightouch read from a simpler, materialized table. This avoids repeating the same heavy work every time a sync, audience, or trait runs.
### 3. More queries run at the same time
**Concurrency** (sometimes called parallelization) refers to how many queries run at the same time.
When many syncs, journeys, or recalculations start at once, your warehouse must process them simultaneously.
Higher concurrency can:
- Increase short-term cost spikes
- Slow down interactive work
Adjust concurrency based on what your warehouse can handle and how quickly jobs need to finish. If jobs are not time-sensitive, spreading them out can reduce peak usage.
### 4. The warehouse is not the right size for the workload
Warehouse size (sometimes called a compute engine) determines how much compute is available at once.
- Larger warehouses process more data per unit time but cost more while running.
- Smaller warehouses cost less while running but may introduce wait times during spikes.
Start with a modest warehouse size and scale up only if jobs regularly take too long. Many customers begin with a **Medium** warehouse.
Some warehouses can automatically add capacity during busy periods (for example, multi-cluster or serverless) and scale down when demand drops.
### 5. The warehouse stays on while idle
Warehouses that remain running when no queries are executing still incur compute cost.
Idle behavior depends on:
- **Auto-suspend** (how quickly the warehouse turns off when it’s idle)
- How jobs are **scheduled** throughout the day
- Whether background jobs run **continuously** or in **batches**
Many environments default to ~10 minutes of inactivity. Lowering auto-suspend to around **1 minute**—while keeping **auto-resume enabled**—is often one of the simplest ways to reduce unnecessary compute usage.
---
## How Hightouch uses your warehouse
Understanding how Hightouch workloads behave helps explain why certain cost drivers appear in practice.
Hightouch workloads fall into a few broad categories. Understanding them helps you decide what needs to be fast and what can run in the background.
### Interactive workloads
Interactive workloads are user-driven and show up directly in the platform UI:
- Audience previews in [**Audiences**](/customer-studio/usage)
- Trait previews in [**Traits**](/customer-studio/traits)
- Insights and dashboards in [**Intelligence**](/campaign-intelligence/overview)
- Some AI-assisted workflows
These workloads prioritize quick load times because users expect fast responses.
### Batch workloads
Batch workloads run on a schedule or in the background:
- Sync runs to downstream tools
- Identity Resolution jobs
- Large trait recalculations
- Audience snapshots and backfills
These workloads prioritize processing large volumes of data reliably and can often run on a smaller or lower-priority warehouse.
### Background jobs
Some jobs run periodically but are not time-sensitive:
- Match Booster enrichment refreshes
- Cached column suggestion refreshes
- Non-urgent audience updates
Because these jobs tolerate delay, they are good candidates for:
- Lower-frequency schedules
- Smaller, dedicated warehouses
- Running during off-peak windows
---
## Strategies to reduce warehouse compute
There are three main levers that affect how much compute Hightouch consumes:
1. [**Warehouse configuration**](#1-warehouse-configuration)
2. [**Data modeling and table design**](#2-data-modeling-and-table-design)
3. [**Hightouch configuration**](#3-hightouch-configuration)
Optimizing across all three provides predictable performance and cost control.
---
## 1. Warehouse configuration
### Use a dedicated warehouse and user for Hightouch
Create:
1. A **dedicated warehouse** for Hightouch
- Allows you to clearly see what Hightouch costs and prevents it from competing with other workloads.
2. A **dedicated user** (or service account) and **role** with access only to the data Hightouch needs
- Makes it easier to control access and audit the queries Hightouch runs.
Starting here provides the clearest cost visibility and strongest governance controls.
### Choose the right warehouse size
Start with a modest size (often **Medium**) and scale based on runtime and queueing patterns.
Increase size if jobs regularly exceed acceptable runtimes.
Decrease size if the warehouse is mostly idle.
### Use serverless or multi-cluster options when appropriate
If your warehouse supports dynamic scaling (for example, serverless or multi-cluster configurations), consider enabling it.
These options:
- Add capacity automatically during busy periods
- Scale down when demand drops
- Prevent you from permanently running a larger warehouse
This is especially helpful if you have steady daily usage with occasional spikes.
### Reduce idle time
Lower **auto-suspend** (around **1 minute** is common) and keep **auto-resume** enabled to prevent idle compute waste.
### Control how many queries run at once
Adjust how many queries your warehouse can run at the same time.
- Increasing concurrency allows more work to finish sooner but may increase peak usage.
- Decreasing concurrency smooths load but can extend total runtimes.
Adjust this setting based on whether cost spikes or missed deadlines are the bigger concern.
**For data teams**
Review warehouse query history for:
- Time spent queued before execution
- Peak credit usage during sync windows
- Missed downstream deadlines
Compare before and after making concurrency adjustments.
### Set query time limits
Configure maximum query runtimes to prevent a single query from running indefinitely.
- Set reasonable time limits to guard against runaway compute usage.
- If queries frequently hit the limit, optimize the underlying model before increasing the threshold.
### Use budgets and resource monitors
Most warehouses provide built-in tools to:
- Track credit usage
- Set spending thresholds
- Alert on anomalies
Using these with a dedicated Hightouch warehouse makes unusual usage easier to detect and investigate.
### Route different workloads to different warehouses
If your warehouse supports multiple compute engines, route:
- **Interactive workloads** (e.g., audience previews, dashboards, AI-powered workflows) to a warehouse tuned for low-latency (fast UI queries)
- **Batch workloads** (e.g., syncs, IDR, heavy traits) to a warehouse tuned for throughput (large scheduled jobs)
This keeps the UI responsive during heavy batch windows and separates cost management by workload type.
----
## 2. Data modeling and table design
These strategies reduce how much work your warehouse performs each time Hightouch runs a query.
### Turn complex queries into tables
If a model performs heavy joins or aggregations every time it runs, your warehouse must repeat that work for every sync, audience, or trait.
This can quietly increase compute usage over time.
To reduce repeated work:
- Run those joins and calculations earlier in your data pipeline (for example, in dbt) and have Hightouch read from a simpler, materialized table or view that serves as a presentation layer.
- Simplify the model Hightouch reads from.
If a simple `SELECT * FROM your_model LIMIT 1` query runs slowly, the model is likely doing too much work at read time and should be refactored.
**For data teams**
Check warehouse query history for:
- Repeated large joins across the same tables
- High bytes scanned for simple filters
- Long runtimes even when limiting rows
If the same heavy logic appears across many Hightouch queries, it likely belongs upstream rather than inside the model Hightouch reads.
### Structure your tables intentionally
Hightouch frequently runs queries that filter, join, and aggregate data to compute audiences, traits, and journey steps. Efficient table structure reduces repeated work and lowers compute usage.
A practical default is:
- **Keep core warehouse layers normalized by default** (separate entities into separate tables and follow good modeling practices).
- **Expose a flattened, presentation-layer table or view for Hightouch** when it clearly removes slow, repeated joins for the way you actually query the data (especially for Customer Studio parent models).
- **Avoid denormalizing very large fact tables unnecessarily**, as this can increase how much data the warehouse must read; instead, denormalize surgically where it meaningfully reduces query cost for common patterns.
For time-series and large tables, consider adding **clustering keys** (how your warehouse organizes data to make common filters faster) once you understand real query patterns.
---
## 3. Hightouch configuration
These strategies involve controls inside Hightouch that affect how often and how intensively queries run.
### Simplify the models that power your Customer Studio schema
Customer Studio uses **parent**, **related**, and **event** models to evaluate audiences and traits. Your [schema design](/customer-studio/schema) directly affects how much work your warehouse must perform each time a query runs.
If these models contain complex logic, repeated joins, or large amounts of unnecessary data, every audience preview, sync, and journey step becomes more expensive.
As a general rule, use:
- **Parent models** for users or accounts
- **Related models** for supporting records (like orders or subscriptions)
- **Event models** for behavioral activity
Keeping these entities separate by default makes queries more predictable and reduces repeated work.
**For data teams**
Review models that power Customer Studio and look for:
- Long chains of joins inside a single model
- Heavy aggregations calculated at read time
- Queries that remain slow even when limiting rows
If a model is doing significant transformation work each time it’s queried, consider moving that logic earlier in your data pipeline so Hightouch reads from a simpler structure.
### Limit large event tables to what activation requires
**Event tables** can grow quickly. To keep them efficient:
- Limit time windows to what activation actually requires
- Use pre-aggregated tables for common metrics when helpful
- Organize tables based on the columns most often used in filters (for example, date or tenant)
For large event models, partition event tables by a timestamp column and always include a partition-aligned time filter in audiences and traits to avoid full-table scans.
Traits can perform aggregations directly for lighter or audience-specific metrics. For heavy or widely reused metrics, pre-aggregated tables or materialized views are usually more efficient than re-aggregating on every trait or audience run.
**For data teams**
Review whether:
- Audiences scan long historical windows
- Event volume has increased significantly
- Date filters still result in large scans
If so, narrow time windows or reorganize the table to better support common filters.
### Schedule background jobs at the lowest effective frequency
Many workloads do not require high-frequency updates.
- Reduce sync frequencies for enrichment updates, cached column suggestions (often daily or weekly is sufficient), and non-time-sensitive audiences
- Choose schedules that match when stakeholders actually use the data
See [**Schedule syncs in the UI**](/syncs/schedule-sync-ui) to learn how to set up sync schedules.
### Use sampling and fast queries for exploration
Audience and trait exploration does not always require exact counts.
- **Sampling** maintains a smaller, regularly refreshed copy of your parent model.
- **Fast queries** instruct Hightouch to use that sample for previews instead of scanning the full dataset.
This significantly reduces the amount of data scanned during exploration and provides much faster previews in exchange for approximate (rather than exact) counts.
Fast queries and sampling affect **previews only**. They do not change final audience membership or sync volume.
#### Enable fast queries in audiences
Use fast queries when:
- Brainstorming logic
- Validating filters
- Comparing audience shape
When you need exact numbers (for example, right before go-live, for small or highly targeted audiences, or for compliance and reporting) turn fast queries off to run against the full dataset.
To enable fast queries:
1. Go to **Customer Studio → Audiences**.
2. Open an audience.
3. Enable the **Fast queries** toggle in the audience header, if available.
#### Enable sampling in models
Sampling is especially helpful when:
- Your parent model contains millions or billions of rows
- Audience previews feel slow during filter changes
- Data teams want to reduce compute usage from iterative exploration
See [**Sampling**](/customer-studio/sampling/) for detailed setup instructions and configuration guidance.
### Control how many jobs Hightouch runs at once
Work with your Hightouch team to modify the number of concurrent workers that are available when triggering syncs within your Hightouch workspace.
Hightouch can run many jobs at the same time. This setting controls how many Hightouch syncs and background jobs are initiated in parallel. It is separate from warehouse-level concurrency.
- **Higher concurrency** reduces how long jobs take overall but increases short-term compute usage.
- **Lower concurrency** spreads work out over time and reduces peak load.
In general, keep Hightouch workspace concurrency at or below your warehouse’s effective concurrency limits (and any queue timeouts) so sync queries aren’t killed while waiting for a warehouse slot.
Match this setting to your warehouse limits and job timing requirements.
----
## If warehouse costs increase unexpectedly
Start with these checks:
1. Did sync or journey frequency change?
2. Did event volume grow?
3. Did new audiences or traits launch?
4. Did concurrency settings change?
5. Did auto-suspend settings change?
6. Did a data model change introduce heavier joins or scans?
Most cost increases can be traced to one of these changes.
---
## Alerting
**URL:** https://hightouch.com/docs/syncs/alerting
**Description:** You can set Hightouch alerts to monitor your syncs, including successes and errors, through SMS, Slack, email, and PagerDuty.
**Section:** Syncs
We recently released an upgraded alerting system. If you’d like access to the
new features, please{" "}
## Overview
Hightouch alerting helps you monitor sync health at a glance. Each sync is evaluated against a set of alert triggers that check for issues like failed runs, rejected rows, slow performance, or unexpected data drops.
Rather than showing only the result of the last run, each sync now displays a
**rollup** of its current alerting state. If no triggers are alerting, the
sync shows “No alerts.” If one or more are in a CRITICAL or WARNING state,
you’ll see the number of active alerts.
This gives you a more accurate picture of whether the sync is healthy, degraded, or needs attention—regardless of whether it ran recently.
Each alert is tracked at the **trigger level**. You can fine-tune thresholds, mute specific conditions, or change recipient settings either in bulk (per destination) or individually (per sync).
## Alerts vs. notifications
In Hightouch, an **alert** is a signal that a sync has entered a CRITICAL or WARNING state. A **notification** is what you receive (via email, Slack, SMS, or PagerDuty) when someone is subscribed to alerts at that level.
- All syncs have **fatal error** and **rejected row** triggers enabled by default. These cannot be disabled, but their thresholds can be customized.
- You’ll only receive a **notification** if a recipient is configured to receive alerts at that level.
- Recipients can be set at the **destination level** or overridden per sync.
- Each recipient can be configured to receive only **CRITICAL** alerts, or **CRITICAL + WARNING**.
### When notifications are sent
You receive **one notification per trigger status change**, not per run.
For example:
- A sync triggers a rejected row alert → you receive one WARNING notification
- It keeps rejecting rows in future runs → no new alerts
- It rejects many more rows in a future run, exceeding the CRITICAL threshold → you receive one CRITICAL notification
- It recovers → you receive a recovery alert
- Later, it fatally fails → you receive one CRITICAL notification
This avoids noise while still ensuring you’re informed when something breaks or escalates.
The costs for SMS alerting are covered by Hightouch.
---
## Configuration
Every sync starts with default alerting enabled:
- **Fatal error alerts**: CRITICAL, triggered by syncs that can’t run due to infrastructure or config issues.
- **Rejected row alerts**: WARNING, triggered by row-level sync failures.
These default triggers are always active but can be tuned with custom thresholds.
### How to manage alert configuration
You can manage alerting behavior at two levels:
#### 1. At the destination level (bulk configuration)
1. Navigate to the **Destinations** page.
2. Select a destination and go to the **Alerting** tab.
3. Set default recipients and trigger thresholds for all syncs using this destination.
#### 2. At the individual sync level (override)
1. Navigate to the **Syncs** page and open a sync.
2. Go to the **Alerting** tab.
3. Adjust trigger thresholds and notification recipients for that sync.
Changes here override any destination-level settings for this sync.
When monitors are changed or enabled/disabled, the changes will take effect
the next time the trigger thresholds are evaluated; after the next sync run,
or approximately every five minutes for time-based monitors.
## Trigger types
Each sync supports multiple trigger types that determine when it should be considered alerting. These conditions monitor for issues like failed runs, data quality problems, and unexpected changes in sync behavior.
All syncs have **fatal error** and **rejected row** triggers enabled by
default. These cannot be disabled, but their thresholds can be customized.
Other triggers are optional and configurable at the destination or sync level.
---
### Fatal errors
Detects infrastructure- or configuration-level sync failures that prevent a run from completing.
- **Default behavior**: Alert if one or more sync runs fail to finish
- **Custom options**: Configure a higher failure streak (e.g., alert only after 3 consecutive failures)
- **Evaluation timing**: Evaluated immediately after each sync run ends, regardless of whether the run completed successfully
- **Metric source**: Sync run status (completed, failed)
- **Logic**: Evaluates the number of consecutive failed runs and compares to the configured threshold. A sync run is considered "fatally failed" if it could not complete due to issues like:
- Unreachable source or destination
- Invalid credentials
- Broken SQL or API query logic
- Misconfigured transformation or mapping
---
### Rejected rows
Identifies sync runs that contain rejected records due to schema mismatches, data validation failures, or destination-level constraints.
- **Default behavior**: Alert if at least 1 row is rejected
- **Custom options**: Alert only if rejections exceed a configured absolute count or relative percentage
- **Evaluation timing**: Evaluated immediately after each sync run ends, regardless of whether the run completed successfully
- **Metric source**: Sum of all rejected rows reported by the sync engine across all operation types (i.e., added, changed, or removed)
- **Logic**: Compares rejected row count (or percentage of attempted rows) against user-defined thresholds. You can choose to measure:
- Absolute count of rejected rows OR
- Percentage of **queried** rows that were rejected
---
### Model size
Monitors the number of rows returned by a sync’s source model query, which may indicate issues with upstream filters, joins, or warehouse state.
- **Default behavior**: Alert if the model returns fewer than 1 row at sync runtime
- **Custom options**: Configure a different minimum row count (e.g., alert if fewer than 1000 rows are returned)
- **Evaluation timing**: Evaluated immediately after each sync run ends, regardless of whether the run was successful
- **Metric source**: Raw row count returned by the model at sync run time
- **Logic**: Compares returned row count against the configured lower bound. If the count is below threshold, a warning or critical status is triggered depending on severity configuration.
---
### Sync duration
Tracks sync performance by measuring wall-clock runtime from start to finish.
- **Default behavior**: Alert if runtime exceeds 60 minutes
- **Custom options**: Set a duration threshold (in hours) between 1 and 24
- **Evaluation timing**: Evaluated every 5 minutes; triggers as soon as duration exceeds threshold, even if the run is not complete
- **Metric source**: Calculated as the time from when the sync was enqueued to the time it finished, or to now if the sync is still running
- **Logic**: Evaluates total time spent across all sync phases— queued, querying, syncing, and writing logs— and compares against the configured duration threshold.
---
### Successful operations (Throughput)
Detects stalled or idle syncs that are no longer pushing meaningful data to their destination.
- **Default behavior**: Alert if fewer than 1 successful operation occurred in the last 7 days
- **Custom options**: Customize the operation threshold and time window (e.g., alert if fewer than 50 ops in 24 hours)
- **Evaluation timing**: Evaluated every 5 minutes; triggers as soon as duration exceeds threshold, even if the run is not complete
- **Metric source**: Aggregated sum of successfully added, changed, and removed rows across all sync runs within the time window
- **Logic**: Compares the total number of successful operations in the time window against the configured threshold.
---
## Muting triggers
You can mute individual alert triggers for a sync to temporarily suppress notifications.
- Muting prevents notifications, but the sync's alert status still updates in the UI
- You can mute for a specific duration (e.g., 1 hour, 1 day) or indefinitely
- Muting is helpful during planned maintenance, known issues, or when you need quiet hours
To mute a trigger, click the "Mute..." link at the top of the alert on the sync page, or:
1. Go to the sync’s **Alerting** tab
2. Click the volume button on the trigger
3. Select a mute duration from the options
Muted triggers still appear in rollups and dashboards—but without triggering notifications to your team.
---
## Webhook alert channels
In addition to email, Slack, SMS, and PagerDuty, you can send alert notifications to a **webhook** endpoint. Webhooks deliver a JSON payload to a URL you specify whenever an alert fires or recovers.
### Custom headers
Webhook alert channels support **custom HTTP headers**. Use custom headers to include authentication tokens, routing keys, or other metadata your endpoint requires (for example, `Authorization: Bearer ` or `X-Routing-Key: ops-team`).
Custom headers are sent with every webhook request for that alert channel.
---
## Array expansion
**URL:** https://hightouch.com/docs/syncs/array-expansion
**Description:** Expand array columns into individual sync rows so each array element is synced as a separate record to your destination.
**Section:** Syncs
## Overview
**Array expansion** lets you take a column that contains an array of values and expand each element into its own row during a sync. This is useful when your source data stores multiple identifiers or values in a single array column, but your destination expects one record per value.
For example, if a user has multiple device IDs stored in an array column, array expansion creates a separate sync row for each device ID, allowing you to sync each one individually to your destination.
Array expansion is currently supported with the **lightning sync engine** on
**Postgres**, **Snowflake**, **BigQuery**, and **Databricks** sources.
## How it works
When you map an array-type column as the identifier in your sync configuration, Hightouch automatically enables array expansion. Each element of the array becomes a separate row, and a generated column is used as the identifier value sent to the destination.
### Example
Consider this source table with an array column `device_ids`:
| user_id | name | device_ids |
| ------- | ------- | -------------------------- |
| 1 | Alice | `["ios_a1", "android_b2"]` |
| 2 | Bob | `["ios_c3"]` |
| 3 | Charlie | `[]` |
| 4 | Dana | `["web_d4", "web_d4"]` |
When you map `device_ids` as the identifier for your sync, Hightouch expands each row so that every array element becomes its own record:
| \_ht_device_ids_element | user_id | name |
| ----------------------- | ------- | ----- |
| `ios_a1` | 1 | Alice |
| `android_b2` | 1 | Alice |
| `ios_c3` | 2 | Bob |
| `web_d4` | 4 | Dana |
The generated column `_ht_device_ids_element` contains the individual element value and is used as the identifier sent to the destination.
### Empty arrays
Rows where the array column is empty (`[]`) or `NULL` produce **zero output rows**. In the example above, Charlie's row is omitted entirely because `device_ids` is an empty array.
### Duplicate elements
If an array contains duplicate values (like Dana's two `"web_d4"` entries), the duplicates produce a **primary key collision**. The sync completes with a "Report" status and the duplicate rows are flagged as rejected. Only one row per unique element value is synced.
## Configuration
Array expansion is configured automatically when you map an array-type column as the identifier in the **"How should rows be matched?"** section of your sync configuration.
When an array column is selected:
1. Hightouch sets the `arrayExpansionSourceColumn` in your sync config
2. The identifier mapping is automatically translated to use the generated element column (`_ht_{column}_element`)
3. An info banner confirms that array expansion is active
To disable array expansion, change the identifier mapping to a non-array column.
## Supported column types
Array expansion works with columns of the following types:
- **Native arrays**: Postgres `text[]`, Snowflake `ARRAY`, BigQuery `ARRAY`, Databricks `ARRAY`
- **JSON arrays**: Postgres `jsonb`, Snowflake `VARIANT`, BigQuery `JSON`
- **JSON string arrays**: `VARCHAR` columns containing JSON-encoded arrays like `'["a","b"]'`
Hightouch automatically detects the column type and applies the appropriate expansion strategy.
## Limitations
- Only **one array column** can be expanded per sync
- Array expansion requires the **lightning sync engine** (in-warehouse planner)
- Only supported on **Postgres**, **Snowflake**, **BigQuery**, and **Databricks** sources
- Duplicate values within an array cause row-level rejections
- Array expansion is not available on syncs that use [Match Booster](/match-booster/overview), as Match Booster applies its own identity resolution logic that is incompatible with row expansion
---
## Sync array identifiers to audience destinations
**URL:** https://hightouch.com/docs/syncs/array-identifier-fan-out
**Description:** Automatically expand array-valued identifiers into individual records for audience syncs — maximize match rates across Google Ads, Meta, TikTok, and more.
**Section:** Syncs
## Overview
Audience syncs in Hightouch automatically **fan out** array-valued identifier fields into separate records — one per array element. This allows you to map a column containing multiple identifiers (such as an array of email addresses or mobile advertising IDs) and have each value synced individually to the destination.
This behavior is specific to **audience destination syncs** (for example, Google Ads Customer Match or Meta Custom Audiences). It requires no configuration — it happens automatically at sync time whenever a mapped identifier field contains an array. For enhanced identity resolution and match rates beyond what your source data provides, see [Match Booster](/match-booster/overview).
This is distinct from [array expansion](/syncs/array-expansion), which is a
separate feature that explodes array columns at the model query level for
non-audience sync types. Audience identifier fan-out is automatic and applies
only to identifier fields in audience syncs.
## How it works
If your model contains a row with an array-valued identifier field:
| user_id | mobile_ids |
| ------- | -------------------------------- |
| 123 | `["GAID_1", "GAID_2", "GAID_3"]` |
Hightouch sends **three separate records** to the destination:
| user_id | mobile_ids |
| ------- | ---------- |
| 123 | `GAID_1` |
| 123 | `GAID_2` |
| 123 | `GAID_3` |
Each array element is synced individually so the destination can match on every identifier. Scalar (non-array) values are unaffected and synced as-is.
## Supported destinations
Fan-out works automatically with all audience destinations that support identifier-based matching, including:
- Google Ads (Customer Match)
- Meta (Custom Audiences)
- Snapchat (Customer List)
- TikTok (Audience)
- Google Display & Video 360
- Bing/Microsoft Ads (Customer Match)
- Reddit Ads
- The Trade Desk
- Pinterest Ads
- LinkedIn (Matched Audiences)
- Twitter/X
- Amazon Ads
- Criteo
- Yahoo Ads
- Taboola
This list is not exhaustive — any audience destination in Hightouch that accepts identifier fields supports fan-out.
## Requirements
- The source column must be a **true array type**, not a JSON string. For example, Snowflake `ARRAY`, BigQuery `ARRAY`, or Postgres `text[]` / `jsonb` arrays all work.
- A string like `'["id1","id2"]'` stored as `VARCHAR` will **not** be fanned out. If your data is stored this way, use a SQL transformation (e.g., `PARSE_JSON()` in Snowflake) to convert it to a native array before syncing.
Fan-out increases the number of API requests sent to the destination. If your
arrays are large, this may affect sync duration.
## Linked identifier fields
When multiple identifier fields are mapped (for example, first name and last name), Hightouch preserves the association between values at the same array index. For independent fields like email and mobile ID, all combinations are sent to maximize match rates.
## Related features
- [**Match Booster**](/match-booster/overview) — Enrich your audience data with additional identifiers to improve match rates across audience destinations.
---
## Change data capture (CDC)
**URL:** https://hightouch.com/docs/syncs/cdc
**Description:** Learn how Hightouch uses change data capture (CDC) to detect row-level changes in your models and send only incremental updates to destinations.
**Section:** Syncs
## Overview
Hightouch optimizes syncs by sending only the rows that have changed since the last run.
This process is called [**change data capture (CDC)**](https://en.wikipedia.org/wiki/Change_data_capture), or [**diffing**](https://en.wikipedia.org/wiki/Diff).
Without CDC, every sync would resend the entire dataset, causing unnecessary API calls, slower syncs, and wasted downstream processing.
---
## How CDC works
Whenever a new sync is triggered, Hightouch compares the previous sync run to the current set of query results. To do this, Hightouch keeps a record of the data sent in the last sync. This record is the **diff file**.
CDC only considers mapped model columns when creating and updating the diff
file. The only exception is custom destinations, such as the [HTTP Request](/destinations/http-request) destination, which consider all columns.
Steps in the CDC process:
1. Hightouch queries the source using the defined model.
2. Hightouch compares the current results to the results from the previous sync.
3. Hightouch identifies differences by primary key:
- New rows
- Updated rows
- Removed rows
4. Hightouch syncs only the changes to the destination, including any failed rows from the previous run.
5. A new diff file is created for the next comparison.
---
## Requirements
CDC relies on a **unique primary key** for every row in a model.
If primary keys are missing or duplicated, Hightouch can’t reliably track changes.
See [Primary key requirements →](https://hightouch.com/docs/models/creating-models#unique-primary-key-requirement).
---
## CDC by sync mode
CDC behavior depends on the sync mode:
- **Insert mode**: only syncs rows whose primary key wasn’t present in the previous run.
- **Update / Upsert modes**: syncs rows where values have changed.
- **All and Archive modes**: do not perform CDC; every row is sent every run.
See [Sync types and modes →](/syncs/types-and-modes).
---
## Difference-based CDC
Hightouch uses a [method](https://www.fivetran.com/blog/change-data-capture-what-it-is-and-how-to-use-it) called **difference-based CDC** because it compares the full before/after query results.
This is required when syncing from warehouses, since they cannot produce CDC logs for arbitrary SQL queries or dbt models.
In contrast, OLTP databases (like Postgres or MySQL) log incremental changes natively, which ETL tools often use when sending data into warehouses.
Because Hightouch does the reverse (sending data *from* warehouses), log-based CDC isn’t possible. This is a key distinction between ETL and reverse ETL.
---
## When CDC occurs
CDC happens **after Hightouch receives the query results from the source and before data is written to the destination.**
If you see the sync status **Querying** in the UI, Hightouch is in one of these states:
- Waiting for query results
- Saving the diff file
- Performing CDC computation
---
## Where CDC is computed
By default, CDC is performed on Hightouch-managed infrastructure.
For supported sources, you can enable the [**Lightning Sync Engine**](/syncs/lightning-sync-engine) to compute CDC directly in your warehouse.
- **Basic engine**: CDC runs on Hightouch infra.
- **Lightning engine**: CDC runs in your warehouse, enabling faster syncs at higher volumes.
---
## Where CDC data is stored
Hightouch stores previous query results (diff files) to compute changes.
- By default, diff files are stored in an encrypted Hightouch-managed bucket.
- Some plans allow you to [bring your own bucket →](https://hightouch.com/docs/security/storage#self-hosted-storage) to store diffs in your own infrastructure.
Diff files are retained for 30 days in Hightouch-managed storage.
If a sync doesn’t run for more than 30 days, you’ll need to [reset CDC](/syncs/overview#reset-cdc) or run a [full resync](/syncs/overview#resync-full-query).
---
## Resetting or resyncing CDC
Sometimes you may need to reset how CDC tracks changes. Common cases:
- **Primary key changes**: Hightouch prompts you to reset CDC when you change a model’s primary key.
- **Resync full query**: forces Hightouch to resend all rows, then resume CDC tracking.
- **Reset CDC**: ignores existing state and starts fresh on the next run (no data is sent during the reset run).
See [Resync and reset docs →](/syncs/overview#resync-full-query).
---
## FAQ
**What happens if I change my model configuration?**
- Only mapped columns affect CDC tracking, except for [HTTP Request destinations](/destinations/http-request), which consider all columns.
- If you change a column’s primary key, Hightouch requires a CDC reset.
- For other model changes, see [Model configuration docs](https://hightouch.com/docs/models/creating-models).
As explained in the [primary key
updates](/models/creating-models#updating-a-primary-key) section, if you alter
a model's primary key by selecting a different column, you will be prompted to
reset the change data capture for all syncs that depend on that model. Be
careful when modifying primary keys. If you keep the same column name but
alter the way its values are calculated, some records may be added or deleted
in your destination, depending on how your sync is configured.
Learn more about changes to your model configuration in the [model column
changes](/syncs/mapping-data#model-column-changes) section.
**Does Hightouch keep historical CDC files?**
No. Only the most recent diff file is stored and compared against.
**What happens if my sync fails?**
CDC re-attempts failed rows on the next run.
**What happens if I stop syncing for more than 30 days?**
Diff files in managed storage expire after 30 days. You’ll need to reset CDC or perform a full resync.
**What happens if I change sync mappings?**
Changing mappings may cause a full reprocess. See [Field mapping updates](/syncs/mapping-data#field-mapping-updates).
---
## Related articles
- [Data activation overview →](/getting-started/concepts)
- [Lightning Sync Engine →](/syncs/lightning-sync-engine)
- [Storage →](/security/storage)
---
## Create your first sync
**URL:** https://hightouch.com/docs/syncs/create-your-first-sync
**Description:** Learn the steps to creating a Hightouch sync and follow along to run your first one.
**Section:** Syncs
Running your first sync is satisfying—you can see your source data appear in the tools and apps your organization lives in. To get there, you need to set a few things up first.
It's best to follow this setup sequence when creating your first sync:
1. [**Connect a source**](#connect-a-source)
2. [**Set up a model**](#set-up-a-model)
3. [**Connect a destination**](#connect-a-destination)
4. [**Configure a sync**](#configure-a-sync)
You can learn the specifics of each step by clicking on the section link. Or check out the [core concepts](/getting-started/concepts) to review Hightouch's fundamentals.
## Connect a source
A [source](/getting-started/concepts#sources) is where your organization's business data lives. To add a new source to your Hightouch workspace, follow these steps:
1. Go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button.
2. Select the source you want to add.
If you want to test things out without connecting a production source, you
can use the B2B SaaS or B2C eCommerce [**Sample datasets**](/sources/sample-data).
They're available at the end of the source list and require no credentials
or setup.
3. Enter the relevant credentials. These differ depending on the source. Refer to the relevant source documentation for further details.
4. Click **Continue**. Before continuing, Hightouch validates that you have the necessary access and permissions. If the test fails, you need to confirm and re-enter your credentials.
5. Give your source a **Source name**. This name is how your source appears in the Hightouch workspace. It helps to include details about the sources' environment, data contents, and owners, particularly if you plan on connecting multiple instances of the same source, for example, a development and prod version.
6. Click **Finish**.
Once you've finished, your source appears on the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and you can use it set up models.
## Set up a model
Models define what data to pull from your source. To set up a new model, follow these steps:
1. Go to the [**Models** overview page](https://app.hightouch.com/models?from=docs) and click **Add model**.
2. Select a source you've already connected.
3. Choose a [modeling method](/models/creating-models): SQL editor, table selector, dbt model, or Looker Look selector. If you're using one of the [Sample dataset](/sources/sample-data) sources, the SQL editor comes with pre-populated SQL queries. You can also use the table selector with the Sample datasets.
4. Before continuing, you must **Preview** your model to ensure it's querying the data you're interested in. By default, the preview displays the first 100 records. Once you've validated your data, click **Continue**.
5. Give your model a **Model name**. It's best to include details about the model's contents and business purpose so you can easily identify it. Including more information is beneficial if you plan on using your model for multiple syncs or having many team members in your workspace.
6. Select a **Primary key**. A primary key should be a column with unique identifiers. A column with unique values is essential for Hightouch to [sync data efficiently](/getting-started/concepts#change-data-capture-cdc). Click **Finish**.
Once you've finished, your model appears on the [**Models** overview page](https://app.hightouch.com/models?from=docs) and you can use it to set up syncs.
To set up a sync, you need to connect a destination.
## Connect a destination
[Destinations](/getting-started/concepts#destinations) are the tools and services you want to receive data from your source. You need to connect your destination to Hightouch before you can set up the sync configuration to the destination.
To set up a new destination, follow these steps:
1. Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button.
2. Select the destination you want to add and click **Continue**.
If you want to test things out without connecting a production destination,
you can use the [Google Sheets](/destinations/google-sheets) destination.
Google Sheets is a [lite destination](/pricing/ss-pricing#lite-destinations)
and straightfoward to set up.
3. Authorize connecting to your destination or enter the relevant credentials and inputs. These differ depending on the destination. Refer to the relevant destination documentation for further details.
4. Click **Continue**. Before continuing, Hightouch validates that you have the necessary access and permissions. If the test fails, you need to confirm and re-enter your credentials.
5. Give your destination a **Destination name**. This name is how your destination appears in the Hightouch workspace. It helps to include details about the destination's business purposes and owners, particularly if you plan on connecting multiple instances of the destination, for example, a development and prod version.
6. Click **Finish**.
Once you've finished, your destination appears on the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and you can use it configure syncs.
## Configure a sync
[Syncs](/getting-started/concepts#syncs) declare how your source data should appear in your destination. To configure one, you need to have set up a source, model, and destination first.
1. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button.
2. Select the relevant model connected to the source from which you want to pull data. If you need help determining which model to choose, you can select **View query** to see the model's definition.
3. Select the destination you want to send data to.
4. Configure how you want to sync data to your destination. Configuration inputs differ depending on the destination. Refer to the relevant destination documentation for further details. Click **Continue**.
5. Set a schedule for the sync. You can learn more about scheduling options in the [sync overview docs](/syncs/overview#schedule).
6. Click **Finish**.
Once you've finished configuring a sync, you land on that sync's overview page. This page shows previous sync runs and lets you edit its **Configuration**, [**Schedule**](/syncs/overview#schedule), and set up [**Alerts**](/syncs/alerting). It's also home to the [**live debugger**](https://hightouch.com/docs/syncs/debugger) which helps understand previous runs down to the API request and response level. To learn more about the sync overview page, check out the [sync documentation](/syncs/overview).
To run the sync manually, click **Run**. Check in your destination to see the fruits of your labor.
---
## Data Extraction
**URL:** https://hightouch.com/docs/syncs/data-extraction
**Description:** Write back the data from your ad platforms to your warehouse
**Section:** Syncs
## Overview
Hightouch's data extraction feature allows you to write back the data from your
ad platforms to your data system. With this data, you can conduct dynamic queries
and generate insightful reports for stakeholders.
## Setup
To get started, you will need:
1. A source with [lightning engine](/syncs/lightning-sync-engine) so Hightouch has write-back privileges
2. A destination to any of the [supported](#supported-destinations) ad platforms
If your destination supports this feature, you'll see a `Data Extraction` tab on the destination's page.
In this tab, you can enable this feature and select which data system to write to (one of your configured sources).
Currently, Hightouch only supports the `Audience size report` so by turning this feature on,
Hightouch will write this report back to your data system.
## Audience size report
The audience size report extracts a daily historical log of the estimated size of your audiences in each ad platform, including metadata about the connected Hightouch destination and sync (when applicable).
### How it works
For each enabled destination:
- Hightouch schedules a daily job to fetch and write the data from the ad platform to your data system
- This job pulls _all_ audiences from the active ad account(s) authorized by the user
- Each job is a snapshot of the audiences at that point in time, so no previous data is updated or deleted
- The data is then written back to your data system under the `hightouch_audit` schema.
- This data lives in its own designated table: `{destination-type}_{destination-id}_audiences`
Hightouch also creates a view that combines all the different destination tables into one, allowing you
to query the data holistically.
### Data structure
You can find all of the data from this feature in the `hightouch_audit` schema.
There will be one table per enabled Hightouch destination (`{destination-type}_{destination-id}_audiences`).
Each destination table's schema closely mirrors the ad platform's API's schema. Hightouch tries to
preserve the data in a form as close to the original API response as possible so you can adapt and
transform the data according to your specific needs.
Hightouch combines the table into a view, `external_audiences_metadata_v1`, with a standardized schema of:
- ad_account_id
- audience_id
- audience_name
- audience_size
- ht_fetched_at
- ht_destination
- ht_destination_id
- ht_sync_id (will be `null` if there are no HT syncs for this audience)
To ensure consistency and backwards compatibility, our combined view is versioned.
Any updates or changes to the view schema will result in a new version (e.g., v2, v3, etc.).
Previous versions of the view will remain accessible as long as it existed before a new updated version.
## Supported destinations
- [Amazon Ads DSP](/destinations/amazon-ads)
- [Meta Custom Audiences](/destinations/facebook)
- [Google Ads](/destinations/google)
- [Microsoft Bing Ads Audiences](/destinations/bingads)
- [Snapchat](/destinations/snapchat)
- [TikTok](/destinations/tiktok)
---
## Live debugger
**URL:** https://hightouch.com/docs/syncs/debugger
**Section:** Syncs
## Overview
Hightouch provides a live debugger to provide visibility into each of your sync executions or "runs." The debugger shows all requests and responses Hightouch made to your destination during a sync run. You can access the debugger from the **Runs** tab of any [sync's overview page](/syncs/overview#individual-sync-overview-page).
Hightouch stores row-level sync run data for seven days. If you need older logs, consider using [Warehouse Sync logs](/syncs/warehouse-sync-logs).
### View run details
You can see a table of recent sync runs and their statuses and [other details](/syncs/overview#run-details) from the **Runs** tab of any sync's overview page.
Each row in the table is one sync run.
The table shows:
- the run's [**status**](#sync-run-status): either **Querying** or **Processing** if the run is still processing, **Completed** if the run finished without errors, **Completed with errors** if the run included some rejected rows, **Aborted due to fatal error** if the run did not complete, or **Cancelled by user** if the run was cancelled
- the time the run **started** and its **duration**
- the number of **rows queried**—this is the number of rows in the [model](/getting-started/concepts#models)'s query results
- the number of **operations**, or updates, sent to the destination
- the **results**—the number resulting operations, split into successful and rejected operations
You can click a row to open a run's detail page and access the debugger.
The run detail page shows the number of successful and rejected rows broken down into added, changed, or removed rows.
- **Added** rows are new rows in the model's query results as compared to the previous run.
- **Changed** rows were in the previous run's query results but now have updated values.
- **Removed** rows are those that are no longer in the query results as compared to the previous run.
If the rows appear under the **Successful** tab, it means the run successfully made the changes in your destination.
If the rows appear under the **Rejected** tab, it means that though the model's query results may have changed, the run couldn't propogate the changes to your destination.
The numbers for added, changed, and removed rows reflect your sync configuration.
For example, if you configure a sync using [**insert** mode](/syncs/types-and-modes#sync-modes), Hightouch only shows **added** rows, even if some rows were removed from your model's query results.
If you configure a sync to the [HTTP request destination](/destinations/http-request) and only configure **Rows changed** [request triggers](/destinations/http-request#choose-request-triggers), you'll only see **changed** rows, even if some rows were added to or removed from your model's query results.
#### Sync run status
The preceding example sync run has a **healthy** status, meaning the sync ran as expected without any rejected rows.
The example sync run below has a **Aborted due to fatal error** status, meaning it didn't execute as intended.
This final example run has a **Completed with errors** status, meaning the destination rejected at least one row.
### View error messages
To view an error message for an entire run that failed, click **See error message** from the **Runs** overview tab.
Hightouch displays error messages in the format third-party APIs send them, including standard HTTP status codes and any trace IDs, if the third party provides them.
To see a specific row's error message, click **View error** on the row in the run's detail page table:
Hightouch displays error messages provided by the third-party API related to individual rows.
If the run fails during query execution with warehouse SQL errors, see [Resolve SQL compilation errors](/models/sql-compilation-errors).
### Filter runs
To narrow down the runs to inspect, you can use the **Filter** option to hide runs with either zero operations or zero successful operations.
### Search rows
Use the search input above the run details table to find a row with a primary key that matches the search value. This search only searches through the column you selected as the model's [primary key](/models/creating-models#unique-primary-key-requirement).
### Export rows and row errors
You can also export the data from all rows or just row errors as a CSV or JSON file.
Note: exporting requires workspace admin permissions.
You can then inspect the file to see details of rows the sync run added, changed, or deleted:
Or, download and then inspect just the row errors:
Use your CSV editor's search feature to locate rows or specific error messages more quickly.
## Use the live debugger
To take a closer look at the individual HTTP requests and responses that comprise a sync run, click any row to open the live debugger.
The live debugger shows the request and response JSON payloads of which the row was part and the other rows that were part of the batch.
In this example, the POST request to update this batch failed because of a 400 error. The response displays the third-party error message saying there was an `undefined` object in the request:
Hightouch displays the request's URL for diagnostic purposes. An `undefined` sub-URL means that the row ID number needed for this POST request failed to populate the URL. The next step would be to inspect the query results returned from the model.
### Payload inspection
To expand a payload for closer inspection, click the expand icon in the top right corner of the payload viewer.
Once expanded, you can scroll through the payload or click in the payload body and use Ctrl/Command-F to open a search window. Use the search window to find row IDs, email addresses, or any other payload data you need to inspect or trace.
To copy the payload, click the copy button in the top right corner of the payload viewer; the browser saves the payload to your clipboard.
For large, nested JSON payloads, it's helpful to paste the JSON into a viewer like [CodeBeautify](https://codebeautify.org/jsonviewer) to view the payload's structure and values. Click **Tree viewer** to inspect the JSON as collapsible properties:
## Hightouch debugging tips
### Filter runs
When you have a large number of runs to inspect, use the [run filter](#filter-runs) to narrow down your view to only the runs you want to inspect.
### Run timestamps
Use the run timestamps to isolate the runs you need to inspect based on your observation of when an error occurred. Run timestamps appear in your local timezone under the header **Started**.
### Search the live debugger
Use the live debugger to search payloads for missing rows or to see how Hightouch sent the data to your destination. Expand the payload viewer and use command+F to search for entities you need to inspect.
### Search error messages
Hightouch returns error messages directly from third-party APIs as they occur. Therefore, you will likely find more information and solutions by copying and pasting the error message into your favorite search engine or [Stack Overflow](https://stackoverflow.com/).
Some destinations also have active developer communities where you can find explanations for common error messages:
- [Salesforce Developer Community](https://developer.salesforce.com/forums#!/feedtype=RECENT&criteria=ALLQUESTIONS&)
- [HubSpot Developer Community](https://community.hubspot.com/t5/HubSpot-Developers/ct-p/developers)
- [NetSuite Support Community](https://community.oracle.com/netsuite)
### Understand HTTP status codes
Hightouch displays any HTTP status codes your destination includes in the HTTP response. These universal status codes can help you understand what may have gone wrong with your sync. If you see an HTTP status code in your Hightouch error message, it comes from your destination, not from Hightouch.
HTTP status codes are grouped into five categories:
- **1xx** Informational Response: The destination request was received and understood. Request processing continues.
- **2xx** Success: The request was successfully received, understood, and accepted.
- **3xx** Redirection: The client must take further action to complete the request.
- **4xx** Client Errors: The client caused an error. The request contains the wrong syntax or can't be fulfilled.
- **5xx** Server Errors: The server encountered an error and failed to fulfill the request.
See [Mozilla's documentation](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status) for further information on HTTP status codes.
#### Common status codes in Hightouch
The most common status codes Hightouch syncs receive are:
- 200 success: This is good news and means your destination successfully received and accepted the request from Hightouch.
- 400 status: A 400 error often means there is invalid data. Check that your data conforms to the destination's formatting expectations by referring to the destination's API documentation.
- 401 status: A 401 error often means there is a problem with the credentials you've used to set up your destination. Check that your credentials have the correct permissions to access the destination.
500 series (500, 502, 504, etc.) errors are "server errors," meaning something has gone wrong within the destination you are trying to sync to. Visit the status page of the destination to see if there are any reported outages, for example, [Slack System Status](https://status.slack.com/) or [HubSpot Status](https://status.hubspot.com/).
If a sync fails because of a destination server error, Hightouch [retries](/syncs/retries) the sync.
### Data types and casting
Third-party APIs require the data you send to be of a specific data type. Unless otherwise specified, Hightouch takes the data from your source and leaves the data type unchanged to send to your destination. See the [data types and casting docs](/models/data-types-casting) for tips on resolving data type incompatibilities.
---
## Error codes
**URL:** https://hightouch.com/docs/syncs/error-codes
**Description:** A reference of all Hightouch error codes, what they mean, and how to resolve them.
**Section:** Syncs
When a sync run fails or rejects rows, Hightouch assigns an error code that identifies the cause. This page is a complete reference of all error codes, organized by where they originate.
Some error codes include troubleshooting links directly in the Hightouch UI. If you don't see a link for your error, use this page to find the code and follow the resolution steps.
Don't see your error code? This page covers Hightouch platform error codes only. Hightouch passes error messages from third-party APIs directly, so if you see a vendor error (like a `400` or `409`), check the [destination's docs page](/destinations) under **Tips and troubleshooting** or use the [live debugger](/syncs/debugger) to inspect the full response.
If your error still isn't covered anywhere, use the **Send feedback** button on this page to let us know and we'll add it.
## Sync request errors
These errors occur during sync planning, before Hightouch sends any data to the destination. They typically indicate a problem with your model, primary key, or warehouse configuration.
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `DUPLICATE_COLUMNS_DETECTED` | Your model query returns duplicate column names. | Alias columns in your model SQL so every selected field has a unique name. |
| `NON_UNIQUE_PRIMARY_KEY` | The selected primary key contains duplicate values. | Update your model so each row has a unique identifier. If you're using a SQL model, add a `DISTINCT` clause or `GROUP BY` on the key column. |
| `UNSUPPORTED_PRIMARY_KEY_TYPE` | The primary key's data type isn't supported for this sync path. | Cast the primary key to a supported type (typically `VARCHAR` or `INTEGER`) in your model SQL. |
| `QUERY_COLUMN_MISSING` | A column referenced in your sync configuration no longer exists in the model query results. | Refresh the model schema in Hightouch and update your [field mappings](/syncs/mapping-data) to use valid columns. |
| `WAREHOUSE_TABLE_MISSING` | A warehouse table required for sync planning doesn't exist. | Confirm the table still exists in your warehouse and that the model points to the correct schema and table name. |
| `PREVIOUS_SYNC_RUN_OBJECT_MISSING` | Stored objects from the previous sync run are missing, usually because cached plan data expired. | Retry the sync. If it persists, trigger a [full resync](/syncs/overview#resync-full-query) to rebuild sync state. |
| `REMOVE_PLAN_INCOMPLETE` | Backfill or removal planning couldn't complete during a planner transition. | Retry the sync after a few minutes. If it continues, contact support. |
| `REMOVE_RETRY_CHANGED_COLUMN_TYPES` | Rejected-row removal retries are unsafe because model column types changed between runs. | Refresh your model schema and trigger a [full resync](/syncs/overview#resync-full-query) from a consistent state. |
| `SORT_RAN_OUT_OF_DISK_SPACE` | Sorting rows exhausted available disk space. | Narrow the model to reduce row volume, or contact support for infrastructure help. |
| `UNSPECIFIED` | Hightouch couldn't classify the failure into a more specific code. | Use the [live debugger](/syncs/debugger) and [warehouse sync logs](/syncs/warehouse-sync-logs) to identify the root cause. |
## Destination errors
### Generic destination errors
These errors can occur with any destination. They indicate a problem with authentication, rate limits, or the destination's availability.
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `DESTINATION_INVALID_GRANT` | The destination's authorization grant expired or was revoked. | Reconnect the destination in Hightouch to refresh authorization. |
| `DESTINATION_INVALID_CREDENTIALS` | The destination credentials are invalid. | Verify the username, password, token, or secret configured for the destination. Reconnect if needed. |
| `DESTINATION_INVALID_API_KEY` | The API key is invalid or expired. | Replace it with a current key that has the required permissions. |
| `DESTINATION_DEFINITION_NOT_FOUND` | Hightouch couldn't load the destination type definition. | Confirm you're using a supported destination type. If the connector appears broken, contact support. |
| `DESTINATION_INTERNAL_ERROR` | The destination returned a generic internal failure or timed out. | Retry the sync. If it continues, check the destination's status page for outages, or contact support. |
| `DESTINATION_RATE_LIMIT` | The destination rate-limited the sync. | Wait for the rate limit window to reset, then retry. To prevent recurrence, reduce [sync frequency](/syncs/schedule-sync-ui) or batch volume. |
| `DESTINATION_MISSING_AD_ACCOUNT` | A required ad account is missing from the destination configuration or inaccessible. | Re-check the destination setup and confirm your account has access to the selected ad account. |
| `DESTINATION_PAYLOAD_TOO_LARGE` | The destination rejected the payload as too large. | Reduce batch size in your sync configuration, or trim the number of mapped fields. |
### Salesforce
For additional Salesforce-specific errors returned by the Salesforce API, see [Salesforce: Tips and troubleshooting](/destinations/salesforce#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `SALESFORCE_INACTIVE_USER` | The Salesforce integration user is inactive. | Reconnect the destination with an active Salesforce user. |
| `SALESFORCE_EXPIRED_TOKEN` | The Salesforce access token expired. | [Reauthorize the destination](/destinations/salesforce#connect-to-salesforce) in Hightouch. |
| `SALESFORCE_INACTIVE_ORG` | The Salesforce org is inactive or inaccessible. | Confirm the org is active and the integration user can log in. |
| `SALESFORCE_RESOURCE_DOES_NOT_EXIST` | A referenced Salesforce object, field, or record doesn't exist. | Check the object name, field name, or record ID in your sync configuration. |
| `SALESFORCE_AUTH_FAILURE` | Salesforce authentication failed. | Re-check credentials, connected app settings, and any IP or login restrictions in Salesforce. |
| `SALESFORCE_REQUESTS_LIMIT_EXCEEDED` | Salesforce API limits were exceeded. | Wait for the limit to reset, or reduce sync frequency and concurrency. See [Salesforce API limits](https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_api.htm). |
| `SALESFORCE_NETWORK_DIFFICULTY` | A network or transport failure prevented the Salesforce request from completing. | Retry the sync and check for temporary Salesforce outages on [Salesforce Status](https://status.salesforce.com/). |
| `SALESFORCE_MAX_CALL_STACK` | Salesforce hit a recursion or call-stack limit during processing. | Review Apex triggers, flows, and automation rules that may be recursively updating records during the sync. See [Salesforce: UNABLE_TO_LOCK_ROW](/destinations/salesforce#unable_to_lock_row) for related guidance. |
| `SALESFORCE_UNABLE_TO_LOCK_ROW` | Salesforce row locking prevented the write because another process is modifying the same record. | Retry later, reduce concurrent updates, or lower your [parallelization setting](/destinations/salesforce#batching-and-parallelization). See [Salesforce: UNABLE_TO_LOCK_ROW](/destinations/salesforce#unable_to_lock_row) for detailed steps. |
### Salesforce Marketing Cloud (SFMC)
For additional SFMC-specific errors, see [SFMC: Tips and troubleshooting](/destinations/sfmc#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `SFMC_DATA_EXTENSION_ADD_UPDATE_ONLY` | This SFMC sync path only supports add and update operations. | Change the sync mode or target a Data Extension that supports the requested operation. |
| `SFMC_IMPORT_FAILED_NO_RECORDS` | The SFMC import failed because no records were present in the batch. | Confirm that the source model returned rows for this sync run. |
### HubSpot
For additional HubSpot-specific errors, see [HubSpot: Tips and troubleshooting](/destinations/hubspot#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `HUBSPOT_RATE_LIMIT_DAILY` | HubSpot's daily API limit was reached. | Wait for the daily limit to reset (typically midnight ET). To prevent recurrence, reduce sync frequency or volume. See [HubSpot API usage guidelines](https://developers.hubspot.com/docs/api/usage-details). |
| `HUBSPOT_RATE_LIMIT_TEN_SECONDLY` | HubSpot's rolling 10-second rate limit was reached. | Hightouch automatically retries these. If the error persists across runs, reduce concurrent syncs to HubSpot. |
| `HUBSPOT_RATE_LIMIT_SECONDLY` | HubSpot's per-second rate limit was reached. | Hightouch automatically retries these. Reduce burst volume if the error recurs. |
| `HUBSPOT_INVALID_REFRESH_TOKEN` | The HubSpot OAuth refresh token is invalid or expired. | Reconnect the HubSpot destination in Hightouch to generate a new token. |
| `HUBSPOT_SYNC_ERROR` | HubSpot returned a generic sync error. | Retry the sync. If it continues, check HubSpot-side logs and object validation rules. |
| `HUBSPOT_CONNECTION_ERROR` | A connection failure prevented the HubSpot request from completing. | Retry and check for temporary HubSpot outages on [HubSpot Status](https://www.hubspotstatusupdates.com/). |
| `HUBSPOT_CONTACT_ALREADY_EXISTS` | A contact with this email already exists in HubSpot. | Review your record matching configuration. HubSpot uses email as the primary unique identifier. See [HubSpot: 409 - Contact already exists](/destinations/hubspot#409---contact-already-exists) for detailed steps. |
| `HUBSPOT_CLOUDFLARE_BLOCKED_IP` | HubSpot's Cloudflare proxy blocked the request IP. | If you use [Hightouch's static IPs](/security/networking#ip-addresses), confirm they aren't blocked by your HubSpot account's network settings. Contact HubSpot support if needed. |
### Google Ads
For additional Google Ads-specific errors, see [Google Ads: Tips and troubleshooting](/destinations/google#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `GOOGLE_ADS_INTERNAL_SERVER_ERROR` | Google Ads returned an internal server error. | Retry the sync after a short delay. Check [Google Ads Status](https://ads.google.com/status/publisher/) for outages. |
| `GOOGLE_ADS_USER_LIST_NOT_FOUND` | The target Google Ads audience doesn't exist. | Confirm the audience still exists in Google Ads and the connected account has access. |
| `GOOGLE_ADS_UNAUTHORIZED` | Google Ads rejected the request due to missing permissions or invalid authorization. | Reauthorize the destination and confirm the connected account has the required access level. |
| `GOOGLE_ADS_USER_LIST_NAME_ALREADY_IN_USE` | An audience with this name already exists in Google Ads. | Use a unique audience name, or connect the sync to the existing audience. |
| `GOOGLE_ADS_CONCURRENT_MODIFICATION` | Google Ads rejected the update because another change is still processing. | Wait a few minutes and retry. Avoid running multiple syncs to the same audience simultaneously. |
| `GOOGLE_ADS_STORE_SALES_DIRECT_DATA_NOT_ALLOWED` | Store sales direct data isn't enabled for this Google Ads account. | Confirm account eligibility and enable the feature in Google Ads. See [Google Ads Store Sales requirements](https://support.google.com/google-ads/answer/7620302). |
| `GOOGLE_ADS_GENERIC_INVALID_ARGUMENT` | Google Ads rejected the request as invalid. | Review required fields, identifiers, and mapped values. Use the [live debugger](/syncs/debugger) to inspect the rejected payload. |
### Braze
For additional Braze-specific errors, see [Braze: Tips and troubleshooting](/destinations/braze#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `BRAZE_SERVICE_UNAVAILABLE` | Braze is temporarily unavailable. | Retry the sync after a short delay. Check [Braze Status](https://braze.statuspage.io/) for outages. |
| `BRAZE_ACCESS_DENIED` | Braze denied access to the API. | Confirm the API key has the required permissions in your Braze dashboard. |
| `BRAZE_INVALID_REQUEST` | Braze rejected the request content or structure. | Check required fields and value formats. Use the [live debugger](/syncs/debugger) to inspect the rejected payload. |
### TikTok
For additional TikTok-specific errors, see [TikTok: Tips and troubleshooting](/destinations/tiktok#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `TIKTOK_GATEWAY_TIMEOUT` | TikTok timed out while processing the request. | Retry the sync after a short delay. |
| `TIKTOK_REJECTED_REPLACE_OP` | TikTok rejected a replace operation because another audience job is still processing. | Wait for the existing job to complete before retrying. |
| `TIKTOK_AUDIENCE_ERROR` | The TikTok audience is invalid or not editable. | Confirm the audience exists in TikTok Ads Manager and supports the requested operation. |
| `TIKTOK_PERMISSION_DENIED` | The connected TikTok account lacks the required permissions. | Reauthorize with a TikTok account that can manage the target advertiser and audience. |
| `TIKTOK_INVALID_ENCRYPTION_VALUE` | TikTok rejected the configured encryption or hashing value. | Verify the expected hashing format in your TikTok sync configuration. |
### Google Sheets
For additional Google Sheets-specific errors, see [Google Sheets: Tips and troubleshooting](/destinations/google-sheets#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `GOOGLE_SHEETS_INVALID_GRANT` | The Google Sheets authorization grant is invalid. | Reauthorize the destination in Hightouch. |
| `GOOGLE_SHEETS_INVALID_DATA_TYPE` | The sync tried to write an unsupported value type to a cell. | Cast complex values (arrays, objects) to plain text or supported scalar types in your model. |
| `GOOGLE_SHEETS_INVALID_RANGE` | The configured write range is invalid. | Check the tab name, cell range, and sheet structure in the destination configuration. |
| `GOOGLE_SHEETS_EXCEEDS_GRID_LIMITS` | The write would exceed the spreadsheet's grid limits (10 million cells). | Reduce the number of rows or columns, or write to a new sheet. |
| `GOOGLE_SHEETS_MISSING_SHEET_ID` | The spreadsheet or sheet identifier is missing from the configuration. | Reopen the destination setup and confirm the selected sheet. |
| `GOOGLE_SHEETS_QUOTA_EXCEEDED` | The Google Sheets API quota was exceeded. | Retry later, or reduce sync frequency. See [Google Sheets API limits](https://developers.google.com/sheets/api/limits). |
| `GOOGLE_SHEETS_ENTITY_NOT_FOUND` | The target spreadsheet or sheet no longer exists. | Confirm the spreadsheet exists and is shared with the connected Google account. |
### Slack
For additional Slack-specific errors, see [Slack: Tips and troubleshooting](/destinations/slack#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `SLACK_NOT_IN_CHANNEL` | The Hightouch Slack app isn't a member of the target channel. | Invite the Hightouch app to the channel, or select a different channel. |
| `SLACK_IS_ARCHIVED` | The target Slack channel is archived. | Unarchive the channel or choose a different destination channel. |
| `SLACK_INVALID_BLOCKS` | Slack rejected the block payload as invalid. | Validate the [Block Kit JSON](https://api.slack.com/tools/block-kit-builder) structure and required fields. |
| `SLACK_NO_TEXT` | The Slack message is missing required text content. | Add message text to your sync configuration, even if blocks are also present. |
| `SLACK_NON_CALLABLE_ITERATOR` | An unexpected payload construction failure occurred. | Retry once. If it persists, contact support. |
| `SLACK_UNEXPECTED_TOKEN` | Slack rejected the payload due to invalid syntax. | Review templated message content and JSON formatting for syntax errors. |
### Other destinations
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `AMPLITUDE_INVALID_JSON_REQUEST` | Amplitude rejected the request payload format. | Review field mappings and ensure the JSON structure matches [Amplitude's API requirements](https://www.docs.developers.amplitude.com/analytics/apis/http-v2-api/). |
| `CITRUS_AD_INVALID_BASE_URL` | The configured CitrusAd base URL is invalid. | Update the destination with the correct API endpoint. |
| `ITERABLE_INVALID_LIST_ID` | The Iterable list ID is invalid. | Confirm the list still exists in Iterable and the configured ID is correct. |
| `ANAPLAN_BAD_MAPPINGS` | The Anaplan field mappings are invalid. | Re-check dimension, line-item, and field mapping setup. See [Anaplan: Tips and troubleshooting](/destinations/anaplan#tips-and-troubleshooting). |
| `ANAPLAN_NULL_POINTER` | The Anaplan request failed with a null-pointer error. | Retry first. If it persists, check mapped objects and source values for missing required fields. |
| `TABOOLA_MISSING_NUMERIC_ACCOUNT_ID` | The Taboola account ID is missing or not numeric. | Update the destination configuration with the correct numeric account ID. |
| `ELASTICSEARCH_INVALID_METADATA_TYPE` | The Elasticsearch metadata type is invalid. | Check mapping configuration and ensure metadata fields use supported types. |
| `FRESHDESK_CLOUDFLARE_IP_BLOCKED` | Freshdesk's Cloudflare proxy blocked the request IP. | Confirm [Hightouch's static IPs](/security/networking#ip-addresses) are allowlisted, or contact Freshdesk support. |
## Rejected-row errors
Rejected-row errors apply to individual records, not the entire sync run. You can inspect rejected rows in the [live debugger](/syncs/debugger) or [warehouse sync logs](/syncs/warehouse-sync-logs).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `NULL_EXTERNAL_ID` | A required external ID is null for one or more rows. | Ensure the mapped identifier column is populated for every record. Update your model to filter out rows with null identifiers. |
| `DUPLICATE_PRIMARY_KEY` | Multiple source rows share the same primary key value. | Deduplicate your model so each primary key appears only once per sync run. |
### LinkedIn
For additional LinkedIn-specific errors, see [LinkedIn: Tips and troubleshooting](/destinations/linkedin#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `LINKEDIN_INVALID_CONVERSION` | LinkedIn rejected the conversion payload. | Check that all required conversion fields are present and correctly formatted. |
| `LINKEDIN_EVENT_TOO_OLD` | The event timestamp is outside LinkedIn's allowed time window. | Send more recent events. LinkedIn typically requires events within the last 90 days. |
| `LINKEDIN_MISSING_USER_IDS` | Required LinkedIn user identifiers are missing. | Map at least one accepted identifier (email, Google Advertising ID, or Apple IDFA) for each row. |
| `LINKEDIN_MISSING_INVALID_USER_INFO` | LinkedIn user information is missing or invalid. | Review identifier formatting and required match fields. |
| `LINKEDIN_MISSING_IDENTIFIER_OR_NAME` | A required identifier or name value is missing from the row. | Populate the required matching fields before syncing. |
| `LINKEDIN_INVALID_SEGMENT_VALUE` | A segment value (often email) failed LinkedIn's validation. | Check email formatting and ensure values meet LinkedIn's requirements. |
| `LINKEDIN_PERMISSION_DENIED` | The connected LinkedIn account lacks the required permissions. | Reauthorize with a LinkedIn account that can manage the target audience or conversion action. |
| `LINKEDIN_API_ERROR` | LinkedIn returned a generic row-level API failure. | Retry if the issue appears transient. Use the [live debugger](/syncs/debugger) to inspect LinkedIn's response for the affected rows. |
## Predictive traits errors
These errors occur when training or running [predictive trait](/customer-studio/predictive-traits) models. They typically indicate a problem with training data, event configuration, or warehouse access.
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `NO_USERS_IN_INPUT_FILE` | The training audience returned no users. | Check that the parent audience or model query returns rows before training starts. |
| `NO_USER_ROWS_AFTER_FILTERS` | Users were present initially, but predictive filters removed all of them. | Loosen the filters or verify that event and property conditions match real data. |
| `NO_TRAINING_DATA_FILES` | Hightouch couldn't generate the training dataset. | Confirm that the configured events return data and the event model is queryable. |
| `LABEL_FREQUENCY_TOO_LOW` | There aren't enough examples of the target outcome to train the model reliably. | Increase the lookback window, reduce filters, or choose a higher-volume outcome event. |
| `NO_POSITIVE_EXAMPLES` | The training data contains zero positive examples of the target event. | Check that the outcome event is occurring for users in scope. |
| `TRAINING_MATRIX_DOWNLOAD_FAILED` | An internal failure occurred while retrieving the training artifact. | Retry the run. If it persists, contact support — this is typically not customer-fixable. |
| `NO_KEY_EVENTS` | No feature-generation events are configured. | Add at least one event input in the predictive trait configuration before retraining. |
| `EVENT_MODEL_MISCONFIGURED` | The event model configuration is invalid. | Confirm the event model exists, has a valid primary key, and includes all referenced columns. |
| `EVENT_QUERY_FAILED` | Hightouch couldn't query event data from the warehouse. | Check warehouse permissions, model SQL, and whether the referenced tables still exist. |
| `SOURCE_ACCOUNT_LOCKED` | The warehouse user account is locked. | Wait for the lockout to clear, or have your warehouse admin re-enable the account. |
| `UNKNOWN_EVENT_IN_FILTER` | A predictive filter references an event Hightouch can't resolve. | Check for renamed, deleted, or misconfigured events in the predictive trait setup. |
| `INVALID_EVENT_PROPERTY_CONDITION` | An event-property filter is malformed. | Verify the filter syntax and confirm you're using valid string property names. |
| `NO_EVENT_DATA_FILES` | No event data was produced for training or inference. | Confirm the configured events have recent data and event pulls are succeeding. |
| `TRAINING_ERROR` | An internal training failure occurred. | Retry once. If it repeats, contact support. |
| `CACHED_EVENT_PULL_DISABLED` | A required event pull is disabled. | Re-enable the event pull in your schema configuration and rerun the predictor. |
| `MALFORMED_EVENT_DATA` | Numeric event fields contain invalid values. | Clean the source data so numeric columns contain only valid numbers (no strings, nulls in numeric-only fields, etc.). |
| `MODEL_QUERY_FAILED` | Hightouch couldn't query the parent model data needed for the run. | Check that the parent model still exists and all referenced columns are present. |
---
## Lightning sync engine
**URL:** https://hightouch.com/docs/syncs/lightning-sync-engine
**Description:** Use the Lightning sync engine to speed up sync performance by storing change data capture in your data warehouse rather than on Hightouch managed infrastructure.
**Section:** Syncs
{/* */}
{/* Allow usage of "Warehouse planning" just this once */}
The Lightning sync engine was previously known as warehouse planning. Both terms describe the same feature.
{/* */}
## Overview
Hightouch identifies the incremental changes in your data model so that syncs send only the necessary updates to your destinations. For every sync, Hightouch computes what rows to send by pulling all rows within your data model and comparing them with the last sync. This process is called **change data capture** (CDC) or **diffing.** You can find more details in the [change data capture overview](/syncs/cdc).
By default, the change data capture computation and file storage happens on Hightouch managed infrastructure. This can be slow for large models that take a long time to query.
With the **Lightning sync engine**, Hightouch computes change data capture **directly in your warehouse**. It stores previously synced data into a special schema managed by Hightouch. Customers with extensive datasets requiring high performance syncs typically use the Lightning sync engine.
Hightouch recommends using the Lightning sync engine for [supported sources](#supported-sources) when syncing more than 100 thousand rows of data.
### Engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also unlocks features that are not available with the Basic engine, including [Journeys](/customer-studio/journeys), [Sampling](/customer-studio/sampling), [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Audience Snapshots](/customer-studio/audience-snapshots), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
| Criteria | Basic sync engine | Lightning sync engine |
| ---------------------------------------------------- | ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Performance | Slower | Quicker (up to 100 times faster) |
| Ideal for large data models (over 100 thousand rows) | No | Yes |
| Reliability | Normal | High |
| Resilience to sync interruptions | Normal | High |
| Extra features | None | [Journeys](/customer-studio/journeys), [Sampling](/customer-studio/sampling), [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Audience Snapshots](/customer-studio/audience-snapshots), [Match Booster](/match-booster/overview), [Identity Resolution](https://hightouch.com/platform/identity-resolution), experiment [holdout group logs](/customer-studio/experiments) |
| Ease of setup | Simpler | More involved |
| Location of change data capture | Hightouch infrastructure | Warehouse schemas managed by Hightouch |
| Required warehouse permissions | Read-only | Read and write |
| Ability to switch | You can move to the Lightning engine at any time | You can't move to the Basic engine once Lightning is configured |
### When the Lightning sync engine is required
The Basic sync engine supports standard audience syncs, but several Hightouch features require the Lightning sync engine. Enable the Lightning sync engine if you plan to use any of the following:
- [Journeys](/customer-studio/journeys)
- [Sampling](/customer-studio/sampling)
- [Warehouse Sync Logs](/syncs/warehouse-sync-logs)
- [Audience Snapshots](/customer-studio/audience-snapshots)
- Experiment [holdout group logs](/customer-studio/experiments)
- [Match Booster](/match-booster/overview)
- [Identity Resolution](https://hightouch.com/platform/identity-resolution)
If you are setting up Customer Studio for anything beyond basic audience syncs, Hightouch recommends enabling the Lightning sync engine during source setup so you don't need to revisit your sync engine choice later.
## Supported sources
The Lightning sync engine works with the following sources:
- [Amazon Athena](/sources/amazon-athena)
- [Amazon Redshift](/sources/amazon-redshift)
- [Azure Synapse](/sources/synapse)
- [Databricks](/sources/databricks)
- [Google BigQuery](/sources/google-bigquery)
- [Greenplum Database](/sources/greenplum)
- [Lakebase](/sources/lakebase)
- [Microsoft Fabric](/sources/fabric)
- [PostgreSQL](/sources/postgresql)
- [Snowflake](/sources/snowflake)
- [Snowflake Iceberg](/sources/snowflake-iceberg)
- [Trino](/sources/trino)
## Enable the Lightning sync engine
When first setting up one of the [supported sources](#supported-sources), Hightouch prompts you to **Choose your sync engine**.
You can set up the Lightning sync engine at this point or continue with the Basic sync engine and [upgrade to the Lightning engine](#migrate-from-basic-sync-engine) later. Once you've configured the Lightning engine, you can't switch back to the Basic engine.
In addition to enabling the Lightning sync engine in the Hightouch UI, you need to:
1. Grant Hightouch write permission to your data warehouse.
2. Create the warehouse schemas for Hightouch to use.
Refer to the source-specific in-app instructions on how to complete these steps.
### Warehouse schemas
Hightouch requires you to create two schemas:
- `hightouch_planner`: Hightouch uses this schema to power the Lightning sync engine.
- `hightouch_audit`: Hightouch uses this schema to power [Warehouse Sync Logs](/syncs/warehouse-sync-logs).
Whenever a sync that uses the Lightning sync engine runs, Hightouch creates two new tables in the `hightouch_planner` schema: one to log the model's query results (the `_plan` table) and one to log rows rejected during the sync (the `_rejections` table).
Hightouch uses these tables to perform [change data capture](#overview) and plan which rows to sync.
Hightouch only keeps the two most recent pairs of tables to compute the diff for the current sync.
In other words, Hightouch [doesn't maintain historical records](/syncs/cdc#faq) of these tables, and any given sync never has more than the two most recent pairs of tables.
Because these tables and their names change with every sync run, Hightouch requires permission to write to the entire `hightouch_planner` schema rather than specific tables.
You can learn more about the `hightouch_audit` schema on the [Warehouse Sync Logs](/syncs/warehouse-sync-logs#schema) page.
#### Table removal
Removing tables from the `hightouch_planner` and `hightouch_audit` schemas isn't recommended:
- `hightouch_planner`: these tables are required for [change data capture](#overview) to work correctly; if you delete these tables, you need to trigger a [full resync](/syncs/overview#resync-full-query) to fix your syncs.
- `hightouch_audit`: removing tables from the `hightouch_audit` schema would mean deleting the [Warehouse Sync Logs](/syncs/warehouse-sync-logs) related to a sync's prior runs, which are useful for historical analysis.
If your [source](#supported-sources) is located in the European Union, you might want to avoid having PII older than 30 days in your data warehouse, in accordance with the [GDPR](https://en.wikipedia.org/wiki/General_Data_Protection_Regulation).
In this case, we recommend running **all** syncs in your workspace at least once every 15 days.
This ensures that Hightouch refreshes the tables in the `hightouch_planner` schema and the **Snapshot** table in the [`hightouch_audit` schema](/syncs/warehouse-sync-logs#schema), removing all older PII before it reaches the 30-day threshold.
Historical data is never automatically cleared out from the **Changelog** table in the `hightouch_audit` schema.
If required, you need to manually remove older PII from this table.
The **Sync runs** table in the `hightouch_audit` schema doesn't contain PII.
## Migrate from Basic sync engine
You can migrate to the Lightning sync engine by going to the **Configuration** tab of any previously setup [supported source](#supported-sources).
1. From the source's **Configuration** tab, select **Lightning sync engine** under **Choose your sync engine** .
2. Follow the source-specific in-app instructions to grant Hightouch write permissions to your warehouse and create the schemas for Hightouch to use.
3. Click **Save changes**.
4. Test your connection to ensure setup is correct.
If you enable the Lightning sync engine on an existing source, Hightouch migrates any syncs on the Basic engine to the Lightning sync engine on the next run. Any previously synced data won't be unnecessarily resynced since the existing [change data capture](#overview) data is copied over to your warehouse schemas. In most cases you don't need to trigger [full resyncs](/syncs/overview#resync-full-query) or [reset change data capture (CDC)](/syncs/overview#reset-cdc), unless you [encounter errors](#migration-tips).
The first run after migrating to the Lightning sync engine can take longer than usual, as it uses a special migration sync engine to create the necessary [schemas](#warehouse-schemas) in your warehouse.
### Migration tips
Follow these tips when performing the migration:
- The Lightning sync engine requires your [primary key to be unique](#primary-key-is-not-unique) for every model using the source. If you're unsure if your primary key is unique, check before migrating.
- If you have a large number of syncs to migrate, please
so that we can help you migrate incrementally.
- If you hit any errors, you can often fix them with a [full resync](/syncs/overview#resync-full-query). Ensure your sync meets the [full resync prerequisites](/syncs/overview#full-resync-prerequisites) before doing so. Otherwise, you can [reset change data capture (CDC)](/syncs/overview#reset-cdc).
## Common errors
### Primary key is not unique
The Lightning sync engine requires that every row in your model has a unique value for its [primary key](/models/creating-models#unique-primary-key-requirement). This is because the primary key is used to uniquely identify each row, and for example, whether it failed previously.
Models with duplicate primary keys can be deduplicated in SQL with the `ROW_NUMBER()` function.
For example:
```sql
WITH your_data_model AS (
// copy/paste your model here
)
WITH your_data_model_with_rank AS (
SELECT *, ROW_NUMBER() OVER (PARTITION BY your_primary_key) as rank FROM your_data_model
)
SELECT * FROM your_data_model_with_rank WHERE rank=1
```
The preceding example de-duplicates your model by _partitioning_ on the primary key. It chooses an arbitrary record from your duplicates. You can use `ORDER BY` or `WHERE` to filter and select which data from the duplicates you want to keep.
---
## Field mapping
**URL:** https://hightouch.com/docs/syncs/mapping-data
**Description:** Mapping is the establishment of a one-to-one relationship between a column in your source and a property or field in a destination.
**Section:** Syncs
## Overview
One of the key aspects of sync configuration is **field mapping**. Field mapping defines which columns from your model query results should appear in your destination and how. In other words, it's a way of relating model columns to destination fields or properties.
**"Columns" vs. "fields" vs. "properties"**: Though some may use these terms
interchangably,
Hightouch uses the term "columns" to refer to a set of vertical data values in
a source or model and "fields" or "properties" to refer to them in a destination.
Depending on your destination, you can map your data with either the **basic mapper** or the **advanced mapper**. The basic mapper maps columns to fields in a one-to-one way. The advanced mapper gives the additional options of:
- mapping with **static** or constant values
- mapping with **variables**
- mapping with [Liquid](https://shopify.dev/api/liquid) **templates**
- **inline mapping**, which lets you create nested objects and arrays from model columns
### Usage
You can start with either the source or destination input fields when using either basic or advanced mapper.
The Hightouch UI offers this flexibility to enable you to map data the way that makes the most sense for you.
For example, you may approach mapping with an idea of all model columns you want to send to your destination.
To do so, click **Add mapping** and select each model column in the left-hand inputs.
Then select the appropriate destination fields to map them to in the corresponding right-hand dropdowns.
Conversely, you may know all the destination fields you need to sync to.
Select each destination field from the dropdowns on the right and map the appropriate model columns on the left.
You can also select to **Suggest mappings**.
This option proposes relevant destination fields for model columns.
For example, a model column named **email_address** would receive a suggested destination field like **Email**.
This autocompletion works by searching for matching and near-matching column and field names.
Before saving your configuration, you should confirm that the autocompleted suggestions have made the correct selections.
## Basic mapper
With the basic mapper, you tell Hightouch which columns in your source to feed into which fields in your destination. For example, you can use the basic mapper to relate model columns to destination fields like these:
| Column in your source | Field in your destination |
| --------------------- | ------------------------- |
| `email_address` | `Email` |
| `first_name` | `FirstName` |
| `last_name` | `LastName` |
| `account_id` | `AccountId` |
## Advanced mapper
For destinations with the **advanced mapper**, you can create mappings that include:
- a column value
- a static value
- a sync-based variable value
- values using the Liquid templating language
- nested objects and arrays using the [**inline mapper**](#inline-mapping)
You can open the advanced mapper by clicking the source field you want to map. If you click a source field and the advanced mapper doesn't appear, it isn't supported. Please if you have a use case requiring the advanced mapper on a destination that doesn't support it.
### Column values
Mapping column values is similar to the [basic mapper's](#basic-mapper) functionality. You can search for and select the model column name you want to map to a destination field.
#### Don't sync null values
By default, Hightouch syncs any `null` values found in your model query results to your destination. You can select to sync nothing instead of null values by enabling **Don't sync null values**.
### Static values
With static mapping, you can specify a constant value to use for each field value. For example, suppose you want to map the string value `EMEA` for every record's `region` field value in a destination. You can do this with static mapping:
1. Open the advanced mapper by clicking on a model column.
1. Select **Static value** and the value's data type: **String**, **Number**, **Boolean**, or **Null**.
1. Enter the desired value in the field provided.
1. Click **Apply**.
1. In the field input to the right of the static value, select the destination field name to which you want to map the static value.
### Variable values
With variable mapping, you can send sync metadata to a destination field. The available metadata variables are:
- `model-id` (string)
- `model-name` (string)
- `current-timestamp` (date/time)
- `sync-id` (string)
- `sync-run-id` (string)
- `operation` (string)
For example, you may want to send a sync's timestamp to an `updated_at` field in a destination.
1. Open the advanced mapper by clicking on a model column.
1. Select **Variable value** and the desired metadata, for example, **current-timestamp**.
1. Click **Apply**.
1. In the field input to the right of the variable value, select the destination field name to which you want to map the variable value.
With this example mapping, Hightouch sends ISO time/date value (YYYY-MM-DD) into every record's `updated_at` field:
### Template mapping
The template mapper is convenient when your destination requires data in a specific format, and you want to avoid baking this specificity into your model.
For example:
- [Webflow](/destinations/webflow) requires slugs not to have any special characters. You can use the template mapper's `regex_replace` function to remove them.
- You want to validate email addresses before sending them to Mailchimp or another email service provider.
- You want to enable lead routing in your destination using Liquid conditionals without touching your central dbt model.
To create nested objects or arrays, you need to use the [inline
mapper](#inline-mapping).
The template mapper uses the [Liquid templating language](https://shopify.dev/api/liquid) to create complex mappings.
#### Test Liquid templates
Use the **Liquid template tester** to preview how your Liquid expression evaluates against real rows from your model before saving the mapping. The tester runs your template against up to five sample rows and shows the rendered output for each, so you can catch formatting issues, missing values, or logic errors without triggering a sync.
To open the tester, click the **Test template** icon in the top-right corner of the template editor. The tester evaluates your expression, displays the output for each sample row, and indicates whether the template parsed successfully. You can edit the template directly in the tester and apply it when you're satisfied.
1. Open the advanced mapper by clicking on a model column.
1. Select **Template**, then select a **Variable** and **Function**. The function's code automatically appears in the editor window.
1. Click **Apply**.
1. In the field input to the right of the templated value, select the destination field name to which you want to map the templated value.
In the example screenshot, the template uses the function `{{ row['last_name'] | includes: "Stark" }}` to check checks if a row's `last_name` column includes the string `"Stark"`. If it does, the row is processed. If it doesn't, the row is rejected.
Beyond using row values, accessible by `{{ row['column_name'] }}`, you can also access any of the [variable values](#variable-values) present in the advanced mapper. Switch to the **Metadata** tab in the template editor to see available context variables like Model ID, Sync ID, and Operation.
These variable values are available on the `context` object.
For example, you could you use `{{ row['id'] | concat: context['sync_run_id'] }}` to concatenate a row's ID with the current sync run's ID.
The **Filters** tab lists all available Liquid functions, including built-in Liquid filters and [Hightouch-specific functions](#liquid-inspired-functions-in-the-advanced-mapper). Click any filter name to inject it into your template.
The template mapper supports all Liquid native features and also includes [Liquid-inspired](#liquid-inspired-functions-in-the-advanced-mapper) functions.
To learn more about the Liquid templating language, skip to the [Liquid reference](#liquid-reference) section in this doc or check out the [Liquid reference documentation](https://shopify.dev/api/liquid).
### Inline mapping
The inline mapper is only available for certain destinations and is currently in beta. Please {" "}if you'd like to be considered for early access.
When syncing data to a marketing automation or enterprise resource planning (ERP) tool, you may need to nest related entities. For example, you may have nested items in an invoice or nested entities for a user: associated devices, shipping address, pets, etc. The inline mappers allows you to create these nested objects and arrays from your model query results _without altering your model definition_.
Each destination has its own formatting expectations for these nested entities. For example, in Stripe, an `invoice` object supports an `items` field that expects an [array of items](/destinations/netsuite-rest#arrays).
The inline mapper lets you model your data _once_ and format it during sync configuration rather creating a separate model for each destination requiring nested objects or arrays.
#### Create an object
The object inline mapper allows you to construct objects and map them to destination fields usings columns from your model. This section walks through constructing an object for the [shipping](https://stripe.com/docs/api/customers/create#create_customer-shipping-address) field in Stripe's customer object.
1. In the destination field input on the right, select the field name to which you want to map an object, for example, **Shipping Address**.
2. The model column input on the left autopopulates to **Object** mode if the selected field is of type `object`. If the type of the field is not provided, open the advanced mapper by clicking on the source input field and select **Create an object**.
3. Select the relevant destination fields you want to map values to and map the nested fields as needed.
Given the previous example [row](#create-an-object) and the mappings shown in the screenshot above, Hightouch would sync the following payload:
```json
{
"email": "doe@test.com",
"shipping": {
"address": {
"city": "San Francisco",
"state": "California",
"postal_code": "94105"
},
"name": "John Doe",
"phone": "123-456-7890"
}
}
```
#### Create an array
The array inline mapper allows you to construct an array of objects and map them to destination fields using a model column that contains an array of objects. This section walks through constructing a [line items](https://stripe.com/docs/api/quotes/object#quote_object-line_items) array for Stripe's quote object. You would use the same steps for any other object arrays you want to create.
#### Required data format
To use the inline mapper to create an array of objects, the query results from your [model](/models/creating-models) need to have a certain format. Specifically, they should appear as an array of objects, like the example **items** field in the entity relationship diagram (ERD) above.
```json
[
{
"item_id": "542",
"quantity": 50
},
{
"item_id": "631",
"quantity": 20
}
]
```
If you don't already have a data field formatted like this, you can define your model using a SQL query that constructs an array of objects from other fields.
For example, imagine you have a `line_items` data table where each row represents an invoice item. The columns are:
- `invoice_id` for the ID of the invoice the item belongs to
- `customer_id` for the ID of the customer who purchased the item
- `amount` for the cost of item
- `description` for a short description of the item
For example:
| invoice_id | customer_id | amount | description |
| ---------- | ----------- | ------ | ---------------------- |
| in_1 | cust_abcd | 55 | Waffle Maker |
| in_1 | cust_abcd | 7 | Dog Chew Toy |
| in_2 | cust_efgh | 12 | HDMI Cable |
| in_3 | cust_abcd | 45 | Humidifier Filters |
| in_3 | cust_abcd | 109 | Electric Kettle |
| in_3 | cust_abcd | 15 | Wall Calendar |
| in_4 | cust_ijkl | 25 | Mediterranean Cookbook |
| in_4 | cust_ijkl | 10 | AAA Batteries |
You want to sync an array of items for each invoice for each customer. For example, for `customer_abcd` from the preceding table, they would have two separate invoice objects, each with their own `items` array, one for the invoice with ID `in_1` and one for the `in_3` invoice:
```json
[
{
"invoice_id": "in_1"
"customer_id": "cust_abcd",
"items": [
{
"description": "Waffle Maker",
"amount": 55,
},
{
"description": "Dog Chew Toy",
"amount": 7,
}
]
},
{
"invoice_id": "in_3"
"customer_id": "cust_abcd",
"items": [
{
"description": "Humidifier Filters",
"amount": 45,
},
{
"description": "Electric Kettle",
"amount": 109,
},
{
"description": "Wall Calendar",
"amount": 15,
}
]
}
]
```
To make these fields available in the inline mapper to construct an `items` array, you would need to include SQL like this in your model definition:
```SQL
select
inv.customer_id,
inv.invoice_id,
array_agg(object_construct(it.*)) as items
from
line_items inv
join items it on it.invoice_id = invoice._id
group by
1,
2
```
This query selects the `customer_id` and `invoice_id` as well as all properties—in this case just description and amount—on each item and then aggregates them into an array of objects. The `group by` statement groups them based on the `customer_id` and `invoice_id` so that each aggregated array is for one invoice for one customer. Without the inline mapper, you would need to write more complex SQL manually constructing each item attribute as part of the item object.
#### Map fields
Once your model query results have an array of objects to pull data from, you can use the inline mapper to configure as many syncs as you need to sync these items.
1. In the **Field Mapping** section of your sync configuration, in the destination field input on the right, select the field name to which you want to map an array, for example, **Line items**.
2. The model column input on the left autopopulates to **Array** mode if the selected field is of type `array`. If the type of the field is not provided, open the advanced mapper by clicking on the source input field and select **Create an array**.
3. You will be prompted to pick a column that contains an array of objects. Select your desired column from the list of compatible columns. If no columns appear, ensure your model query results conform to the [required data format](#required-data-format).
4. Select the relevant destination fields you want to map values to and map the nested fields as needed.
The columns available under the array inline mapper are properties of the previously selected column.
Hightouch uses the first 100 rows from your query to extract the available properties at each index of the array. To add additional properties that may be missing, type into the input field and select the custom option or press **Enter**.
Hightouch only processes properties if the mapped object property at the index of the array exists.
For example, if `[0].price` exists and `[1].price` doesn't exist, `items[0].price` will be mapped and `items[1].price` won't be mapped. Hightouch syncs `null` values if you [don't choose to ignore them](#dont-sync-null-values).
For each row in your query, the array created using the inline mapper matches the length of the array in your column.
Given the previous example [row](#create-an-array) and the mappings shown in the screenshot above, Hightouch would sync the following payload:
```json
{
"line_items": [
{
"price_data": {
"currency": "USD",
"product": "542"
},
"quantity": 50
},
{
"price_data": {
"currency": "USD",
"product": "631"
},
"quantity": 20
}
],
"customer": 231,
"description": "Two items"
}
```
#### Nesting depth
You may need to sync arrays or objects with more deeply nested objects. For example, you may need to sync a `Shipping Address` object with this format:
```json
{
"name": "Jane Doe",
"address": {
"address_line_1": "123 High Street",
"address_line_2": "",
"city": "San Francisco",
"state": "California",
"postal_code": "94105",
"country": "USA"
},
"phone_number": "4151234567"
}
```
The "address" key expects a further nested object. Hightouch supports nesting up to two levels deep. You can do this by selecting **Create an object** or **Create an array** while already using the inline mapper.
### Liquid reference
This reference goes over the following:
- Basic [liquid formatting](#liquid-template-format) rules in the advanced mapper
- Available [liquid variables](#liquid-variables-in-the-advanced-mapper) and [functions](#liquid-inspired-functions-in-the-advanced-mapper)
- [Extended use cases](#extended-liquid-use-cases)
For more detailed information, refer to the [Liquid reference documentation](https://shopify.dev/api/liquid).
#### Liquid template format
Variables follow this format:
```
{{ row['company'] }}
```
The double handlebars enclose a Liquid snippet that generates an output. The column name, `company` in the preceding example, should be in single quotes inside the square brackets.
Hightouch also provides these alternative syntaxes:
```
{{ row.company }}
```
```
{{ company }}
```
These shorter alternatives can be useful when templating messages, such as when syncing to [Slack](/destinations/slack#templating-messages), [Mattermost](/destinations/mattermost#templating-messages), [Microsoft Teams](/destinations/microsoft-teams#templating-messages), or to the [SMTP Email](/destinations/email#templating-emails) destination.
Make sure to use the `{{ row['first name'] }}` syntax if your model column name contains spaces.
Functions follow this format:
```
{{ row['last_name'] | includes : 'Stark' }}
```
A single pipe (`|`) comes after the variable and before the name of the function and any necessary parameters. See the [Liquid Reference](https://shopify.dev/api/liquid) or [Liquid Cheatsheet](https://www.shopify.com/partners/shopify-cheat-sheet?shpxid=7bd156ec-6E04-4D53-42BC-9C614D0E5E0E) for a complete list of available functions.
#### Liquid variables in the advanced mapper
You can use metadata variables in your liquid templates, such as a model's name, all model column names, and the timestamp of a sync's last run.
```
$
$ Model Name
$ Timestamp
```
You can view all available variables in the **Variables** column in the template editor.
#### Liquid-inspired functions in the advanced mapper
Hightouch provides these non-native functions for you to use out of the box:
| Function | Syntax | Description |
| ---------------- | ------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `base64_decode` | `row['email']\|b64_decode` | Base64 decode input |
| `base64_encode` | `row['email']\|b64_encode` | Base64 encode input |
| `cast` | `row['phone']\|cast:'string'` | Cast input to specified type. Allowable types are 'string', 'number', 'boolean' |
| `includes` | `row['last_name']\|includes:'Stark'` | Check input for a substring. Row will be rejected if substring isn't found |
| `json_construct` | `\| json_construct: 'city', row['city'], 'state', row['state_province']` | Construct JSON object from key/value arguments |
| `MD5` | `row['email']\| md5` | Hash input using MD5 algorithm |
| `null_if_empty` | `row['email']\| null_if_empty` | Replace empty strings with `null` |
| `parse` | `row['first_name']\|parse` | Parse a JSON-formatted string to a JSON object |
| `push` | `row['first_name']\|push : ''` | Append an item to an array |
| `regex_replace` | `row['last_name']\|regex_replace: '<[a-zA-Z]>', ''` | Search and replace substrings of input using [RegEx](https://regexr.com/) |
| `regex_test` | `row['last_name']\|regex_test: '<[a-zA-Z]>'` | Check input for a [RegEx](https://regexr.com/) match. Row will be rejected if no match is found using [RegEx](https://regexr.com/) |
| `sha256` | `row['email']\|sha256` | Hash input using SHA-256 |
| `sha512` | `row['email']\|sha512` | Hash input using SHA-512 |
| `to_date` | `row['dob']\|to_date` | Parse input and return a [Date](https://shopify.dev/api/liquid/objects/date) object ([ISO 8601](https://www.iso.org/iso-8601-date-and-time-format.html) format) |
| `to_unix` | `row['date_sent']\|to_unix` | Convert input to [Unix](https://www.unixtimestamp.com/index.php) time |
| `validate_email` | `row['email']\|validate_email` | Reject emails that aren't [RFC 2822](http://haacked.com/archive/2007/08/21/i-knew-how-to-validate-an-email-address-until-i.aspx/) compliant email addresses |
#### Liquid function chaining
You can use more than one Liquid function in a template by [chaining](https://en.wikipedia.org/wiki/Method_chaining) them. For example, you may want to put a string variable into camel case before hashing it. To chain functions, separate them with a single pipe (`|`):
Examples:
```
{{ row['full_name'] | camelcase | sha256 }}
```
```
{{ row['email'] | downcase | validate_email | sha256 }}
```
### Extended Liquid use cases
See Shopify's [Liquid docs](https://shopify.github.io/liquid/basics/introduction/), [Liquid Cheatsheet](https://www.shopify.com/partners/shopify-cheat-sheet), or [Liquid Reference](https://shopify.dev/api/liquid) for a complete list of available functions.
In this section, you can learn how to use Liquid to:
- set up [conditional outputs](#conditional-output)
- output a `null` value for [empty values](#conditional-output-for-empty-values)
- set a [default value](#default-value)
- [escape special characters](#json-function) in JSON mappings
You can also use Liquid to work with different [data types](/models/data-types-casting#data-types):
- **Numbers**: use [math functions](#math-functions) (such as `abs` and `round`) or format [currency amounts](#currency-amounts)
- **Booleans**: output a [conditional boolean value](#conditional-boolean-output)
- **Strings**: use [string functions](#string-functions) (such as `capitalize`, `strip`, and `append`) or [isolate parts of a string](#isolate-parts-of-a-string)
- **Dates**: use the [date function](#date-function)
- **Arrays**: [construct a static array](#construct-a-static-array), [sync an empty array](#sync-an-empty-array), or [convert an array to a string](#convert-an-array-to-a-string)
- **Objects**: [convert an object to a string](#json-function) or [isolate object properties](#isolate-object-properties)
#### Conditional output
You can use Liquid conditionals to specify output based on input. For example, this Liquid snippet:
```
{%- if row['discount_percent'] >= 50 -%}
Over half off!
{%- elsif row['discount_percent'] == 50 -%}
50% off!
{%- elsif row['discount_percent'] > 0 -%}
Discounted!
{%- else -%}
null
{%- endif -%}
```
Outputs the following result in a destination column:
#### Conditional output for empty values
Conditionals are also helpful to guard against missing values.
For example, you could use the following Liquid snippet to send a `null` value for empty values.
This is helpful if you need to map a column that exists for some rows but not others:
```
{%- if row['column_name'] !=nil -%}
{{ row['column_name'] }}
{%- else -%}
{{ null }}
{%- endif -%}
```
#### Conditional boolean output
You could also use the [`assign`](https://shopify.dev/docs/api/liquid/tags#assign) function to output a conditional boolean value:
```
{%- if row['column_name'] == "value" -%}
{%- assign variable = true -%}
{%- else -%}
{%- assign variable = false -%}
{%- endif -%}
{{ variable }}
```
#### Default value
You can also use Liquid to set a [default](https://shopify.github.io/liquid/filters/default/) value, in case an input value is `null`, `false`, or empty. For example, this Liquid snippet sets the `revenue` field as `0` if the value is missing or `null` in a model's row:
```
{{ row['revenue'] | default: 0 }}
```
Be sure to change `revenue` to the column name you want to set default values for.
#### Math functions
See [Liquid Math Functions](https://shopify.dev/api/liquid/filters/math-filters) for a complete list of math functions.
In Hightouch, the absolute value and rounding functions are some of the most commonly used:
- Use the `abs` function to send an absolute value:
```
{{ row['value'] | abs }}
```
- Use the `round` function to round a value to the nearest integer or a specified number of decimals:
```
{{ row['value'] | round: 2 }}
```
#### Currency amounts
Hightouch doesn't support [`money`](https://shopify.dev/docs/api/liquid/filters/money-filters) functions.
If you would like to format currency amounts, such as prices, you can set this up manually by using the following template:
```
{%- assign decimals = row['price'] | split: "" | reverse | slice: 0,2 | reverse | join: "" -%}
{%- assign units = row['price'] | split: "" | reverse | slice: 3,3 | reverse | join: "" -%}
{%- assign thousands = row['price'] | split: "" | reverse | slice: 6,3 | reverse | join: "" -%}
{%- assign millions = row['price'] | split: "" | reverse | slice: 9,3 | reverse | join: "" -%}
{%- if millions != "" -%}
${{millions}},{{thousands}},{{units}}.{{decimals}}
{%- elsif thousands != "" -%}
${{thousands}},{{units}}.{{decimals}}
{%- else -%}
${{units}}.{{decimals}}
{%- endif -%}
```
To explain this expression:
- [`split`](https://shopify.github.io/liquid/filters/split/): converts the value in the `row['price']` model column (either `number` or `string`) to an array
- [`reverse`](https://shopify.github.io/liquid/filters/reverse/): reverses the order of items in the array
- [`slice`](https://shopify.github.io/liquid/filters/slice/): isolates the items in groups of threes
- [`reverse`](https://shopify.github.io/liquid/filters/reverse/): reverses the order of items in the groups
- [`join`](https://shopify.github.io/liquid/filters/join/): transforms the single groups of items back into strings
The second part is a [conditional](#conditional-output) that displays the currency amount in the correct format.
If your model column contains currency amounts without decimal places, make sure to remove the `decimals` variable from the Liquid expression and update the `slice` functions accordingly.
#### String functions
See [Liquid String Functions](https://shopify.dev/api/liquid/filters/string-filters) for a complete list of string functions.
In Hightouch, the `capitalize`, `strip`, and `append` functions are some of the most commonly used:
- Use the `capitalize` function to capitalize the first letter of a string:
```
{{ row['title'] | capitalize }}
```
- Use the `strip` function to remove all whitespace from the left and right sides of a string:
```
{{ row['title'] | strip }}
```
- Use the `append` function to add a file extension to a string:
```
{{ row['file_name'] | append: '.png' }}
```
{/* */}
#### Isolate parts of a string
You can use the Liquid [`split` function](https://shopify.github.io/liquid/filters/split/) to isolate a certain part of a string.
For example, suppose you have a model column that returns URLs, including query parameters.
An example value may look like this: `www.example.com/?gclid-123`.
You may want to send the base URL (`www.example.com/`) as one field and one of the query parameters as another field.
Query parameters often include click IDs, such as the `gclid` or `fbclid` that you should include on events you send to [Google Ads](/destinations/google#conversion-events) or [Meta Conversions](/destinations/meta-conversions#syncing-events) (or [Meta Offline Conversions](/destinations/meta-offline#syncing-events)) respectively.
For the example value `www.example.com/?gclid-123`, this Liquid snippet would return the base URL, `www.example.com/`:
```
{{ row['column_name'] | split: "?gclid-" | first }}
```
And this Liquid snippet would return the `gclid` value, `123`:
```
{{ row['column_name'] | split: "?gclid-" | last }}
```
{/* */}
#### Date function
If your destination expects dates in a different format than your model returns them,
you can use the Liquid [`date` function](https://shopify.github.io/liquid/filters/date/) to reformat them.
For example, this Liquid snippet ensures that dates from the column are always in the `DD/MM/YYYY` format:
```
{{ row['column_name'] | date: '%d/%m/%Y' }}
```
And this snippet always returns dates in the `MM/DD/YYYY` format:
```
{{ row['column_name'] | date: '%m/%d/%Y' }}
```
You can also include other parameters to include time and timezone data, by adding `%H:%M:%S` and `%z`:
```
{{ row['column_name'] | date: '%s' | date: '%Y-%m-%dT%H:%M:%S%z' }}
```
You can use Liquid to change a timestamp's time and date values. For example, you may want to send a date one day earlier than the original. This snippet subtracts 24 hours (represented as 86,400 seconds) from the original date in the column:
```
{{ row['column_name'] | date: '%s' | minus: 86400 | date: '%Y-%m-%dT%H:%M:%S' }}
```
To add time to a date, replace `minus` with `plus`.
For more examples of how to transform dates and times, refer to this [documentation](https://support.sweethawk.com/hc/en-us/articles/4412143470221-Liquid-markup-for-date-and-time#index_3).
#### Construct a static array
You can sync a static array value to your destination by using this Liquid snippet:
```
{%- assign value = '[' | append: '"value1","value2"' | append: ']' -%}
{{ value | parse }}
```
You can replace `'"value1","value2"'` with the values you wish to insert in your static array.
#### Sync an empty array
You can sync an empty array to your destination by using this Liquid snippet:
```
{{ '[]' | parse }}
```
If you decide to use this with [Liquid conditionals](#conditional-output) or other complex syntax, make sure to [use a hyphen](https://shopify.github.io/liquid/basics/whitespace/) in your tag syntax to strip whitespace.
#### Convert an array to a string
You can convert an array to a string by using the [`join`](https://shopify.github.io/liquid/filters/join/) function:
```
{{ row['items'] | join: "" }}
```
#### JSON function
You can convert an object to a JSON-formatted string by using this Liquid snippet:
```
{{ row['invoice'] | json }}
```
You can also use this function to escape special characters when creating a JSON payload.
For example, if you attempt to map this value, `This is the "value" in the column` in the `special_characters` column, it will fail:
```
"column_mapping" : "{{ row.special_characters }}"
```
To successfully sync this value, select the `json` function and use the example format below:
```
"column_mapping" : {{ row.special_characters | json }}
```
#### Isolate object properties
If your model columns contain `object` type values, you may want to sync only one or some of the object's properties to your destination.
You can do this in the template mapper using the [dot notation](https://en.wikipedia.org/wiki/Property_(programming)#Dot_notation).
Specifically, you can write a Liquid expression containing the model column's name and the property's name: `{{ row['model_column'].property }}`
You can use dot notation for [field mapping](#usage) and [record matching](/syncs/record-matching), but only for destinations that support Liquid templating.
Furthermore, dot notation doesn't work when [testing a row](/syncs/overview#test-a-row).
To test your sync, trigger a sync run and take a look at the [live debugger](/syncs/debugger#use-the-live-debugger).
For example, suppose you had a model column named `invoice` that contains objects with this structure:
```json
{
"invoice_number": "INV-001",
"issue_date": "2023-06-08",
"due_date": "2023-06-30",
"customer": {
"name": "John Doe",
"email": "johndoe@example.com",
"address": "123 Main St, City, State, ZIP"
},
"items": [
{
"description": "Product A",
"quantity": 2,
"unit_price": 10.99,
"total_price": 21.98
},
{
"description": "Product B",
"quantity": 1,
"unit_price": 15.99,
"total_price": 15.99
}
],
"subtotal": 37.97,
"tax": 3.60,
"total": 41.57,
"currency": "GBP"
}
```
You may only want to map the `total` column to a `price` field in your destination.
You could do this by including the following Liquid in the template mapper:
```
{{ row['invoice'].total }}
```
You can also use this to map nested properties, for example, `customer.name`:
```
{{ row['invoice'].customer.name }}
```
If you want to sync multiple object properties individually, you can use these types of expressions for multiple mappings in your sync configuration.
Make sure you've set the model column's type as `Object / Array` in the [model's **Columns** tab](/models/data-types-casting#casting-from-your-model-configuration).
Otherwise, Hightouch syncs an `undefined` value to your destination.
Dot notation only works with model columns that contain single objects. For arrays of objects, you can use the [inline mapper](/syncs/mapping-data#inline-mapping).
## Field mapping updates
If you edit the field mappings in your sync configuration, the sync engine automatically reprocesses the entire [model query result set](/models/creating-models#preview-model-results) during the next sync run. Depending on the selected [sync mode](/syncs/types-and-modes#sync-modes), Hightouch syncs the model rows detected as **added** or as **changed** by [change data capture](/syncs/cdc) to your destination.
If the sync uses either **update** or **upsert** mode, Hightouch resyncs previously synced rows (with the updated mappings) as "changed" rows. If the sync uses **insert** mode, Hightouch doesn't reprocess previously synced rows. Instead, it only syncs model rows that are detected as **added** after the field mapping change was saved in your sync configuration.
### Model column changes
Most changes to your [model columns](/models/creating-models) aren't automatically detected by sync configurations.
Hightouch handles the changes differently depending on whether columns are added, renamed, or removed from your model definition, or if you change a column's data type.
If you add a new column to your model, Hightouch doesn't automatically add it as a field mapping to your sync configurations. This is because Hightouch doesn't know which columns you would like to sync to the destination, and which destination fields they should be mapped to. Follow the [field mapping](#usage) instructions to include new columns in your sync configuration.
You need to [preview and save](/models/creating-models#preview-model-results) your model to have new columns appear in the Hightouch UI.
If you rename a column in your model, all syncs that had that column mapped in their configuration fail with an `Unknown field name` error.
The error message also contains the original name of the renamed column. To resolve this, remove or update the relevant field mappings in your sync configuration.
Changes to model column names aren't detected for [**All** / **Mirror**](/syncs/types-and-modes#sync-modes) type syncs.
If you remove a column from your model, Hightouch doesn't automatically remove it from the field mappings in the sync configuration. Make sure to remove the mapping manually.
If you change a model column's [data type](/models/data-types-casting), the sync engine considers this update as a [field mapping update](#field-mapping-updates) and reprocesses all model rows during the next sync run.
As explained in the [primary key updates](/models/creating-models#updating-a-primary-key) section, if you update a model's primary key by selecting a different column, you need to trigger a [full resync](/syncs/overview#resync-full-query) or reset change data capture (CDC) for all syncs that use that model. Otherwise, [change data capture](/syncs/cdc) can't process your model data correctly, which can make your syncs fail.
### Destination field changes
If you rename a field in your destination, you need to update the mapping to this field in your sync configuration. Select the new field name from the dropdown menu and save your changes.
---
## Match Booster on syncs
**URL:** https://hightouch.com/docs/syncs/match-boosting
**Description:** Enable Match Booster in syncs to send enriched identifiers to advertising destinations.
**Section:** Syncs
This page has moved. See [Enable Match Booster in syncs](/match-booster/enable-in-syncs) for the full setup guide.
---
## Syncs
**URL:** https://hightouch.com/docs/syncs/overview
**Description:** An overview of your syncs along with their status, which model they use, the destination they write to, the schedule, last run, and when they were created.
**Section:** Syncs
**Syncs** declare how you want query results from a model to appear in your destination.
To learn more about syncs on a conceptual level, see the [core concepts page](/getting-started/concepts#syncs). Refer to destination docs for specific configuration details.
To learn more about the information the Hightouch UI displays on your syncs and what actions you can take from the UI, continue to the following sections.
## Syncs overview page
Hightouch displays all syncs, both active and inactive on the [**Syncs** overview](https://app.hightouch.com/syncs?from=docs) page.
For each sync, the overview shows:
- The sync's [**status**](#sync-status)
- The sync's associated [**model**](/getting-started/concepts#models)
- The sync's associated [**destination**](/getting-started/concepts#destinations)
- The date and time, in your timezone, of the sync's **last run**
- The date the sync was **created at**, and who created the sync
You can view more in-depth information for any sync by clicking on it. This takes you to that individual sync's [overview page](#sync-overview).
### Sync status
Your sync's status is based on the latest information available on a run:
- **Healthy**: a recent run completed
- **Pending**: no runs have been scheduled yet
- **Disabled**: the sync has been turned off
- **Querying**: the sync is querying its associated model
- **Preparing**: the sync is preparing data to send to its associated destination
- **Queued**: the sync is scheduled to begin writing to its destination
- **In Progress**: the sync is writing to its destination
- **Incomplete**: the sync had a temporary interruption and will resume in a new run
- **Aborted/Failed**: there was an error in the sync progress—check the errors on the sync to see why the sync failed
## Individual sync overview page
You can view an individual sync's information by clicking on it on the on the general [**Syncs** overview](https://app.hightouch.com/syncs?from=docs) page.
A particular sync's overview page shows general information, including:
- The sync's associated model and destination
- Whether the sync is currently **enabled** or **disabled**
- The option to manually **Run** the sync
You can also choose to move the sync to a [folder](/workspace-management/folders-filters#folders), edit its labels, [resync the full query results](#resync-full-query) of a model, [reset change data capture](#reset-cdc), or delete a sync from here.
The overview page also shows more specific information including:
- **Sync ID**: a numerical unique identifier for the sync that is necessary when setting up an [Airflow](/extensions/airflow) operator, or using the [Dagster](https://github.com/hightouchio/dagster-hightouch), [Prefect](https://github.com/PrefectHQ/prefect-hightouch), or [Mage](https://docs.mage.ai/integrations/hightouch) extensions
- **Sync schedule type**: manual, interval, custom, cron, dbt Cloud-, or Fivetran-triggered
- **Sync mode**: this only appears if there are multiple sync mode options, see the [sync mode docs](/syncs/types-and-modes#sync-modes) for more information
- **Match rate**: this only appears if the sync is to a paid advertising destination, see the [match rate docs](/customer-studio/syncs#view-match-rates) docs for more information
- **Any associated labels**: labels help organize and categorize syncs
Finally, you can click the tabs on the overview page to view and edit settings:
- **Runs**: You can see the status, time, and [more details](#run-details) from the most recent runs. Selecting a particular run brings you to the [live debugger](https://hightouch.com/docs/syncs/debugger).
- **Configuration**: You can view and edit the sync's [configuration](#configuration).
- **Schedule**: You view and edit the sync's run [schedule](#schedule).
- **Alerts**: You view and edit the sync's [alerts](/syncs/alerting).
- **Sync logs**: You can view and edit where your [sync's historical logs](/syncs/warehouse-sync-logs) are stored.
- **Activity**: You can view recent changes to the sync configuration and other settings
### Resync full query
By default, Hightouch keeps track of changes to your data between syncs and [only sends necessary updates](/getting-started/concepts#change-data-capture-cdc).
Therefore, an initial sync run [often shows](/syncs/debugger#view-run-details) that many records were **Added**. Subsequent runs often show fewer records **Added**, and instead show that many records have **Changed** or **Removed**.
If you wish to reset the sync's tracking to its initial state—as if it were syncing for the first time again—you can select **Resync full query** from the menu on the sync's main page.
Before you do, make sure that your sync meets [the prerequisites](#full-resync-prerequisites) to safely do so.
#### Full resync prerequisites
Full resyncs process all rows that currently exist in the model as **Added** rows. Check the following before triggering a full resync:
| Criteria | Explanation |
| ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| [Sync mode](/syncs/types-and-modes#sync-modes) |
Syncs set to [**update** or **upsert** mode](/syncs/types-and-modes#sync-modes) are safe to fully resync because if a row was already synced, Hightouch updates the same record.
[**Insert** mode](/syncs/types-and-modes#sync-modes) syncs, [event syncs](/syncs/types-and-modes#event-syncs), and [webhook destinations](/destinations/http-request) aren't recommended for full resyncs because reprocessing rows can create duplicates.
|
| [Delete behavior](#delete-behavior) |
If your destination allows for deleting records, and you selected to **Do nothing** when records leave the model query result set, a full resync will have no impact.
If you selected **Clear fields** or **Delete destination record**, this setting gets ignored during a full resync. This is because full resyncs only process rows that currently exist in the model and run the sync as if it were the initial run. This means that there will be no [diffing](/syncs/cdc) or comparison to the prior run to identify what has been removed. Therefore, all **Removed** rows (in comparison to the previous sync run) are ignored even if delete behavior is enabled.
|
Resyncing the full query could create duplicates or other issues in your
destination data if the prerequisites above haven't been met.
### Clear and fill
If the destination supports removing all records in a segment or audience, this option will become available. This resync mode is similar to a [full resync](#resync-full-query) mentioned above with an additional preprocessing step.
Before processing all rows that are currently in the model, the segment/audience selected in the sync configuration will be cleared of all users/identifiers.
### Reset CDC
If you want to reset [change data capture](/syncs/cdc) without resyncing records, you can select the "Reset CDC" option instead.
This instructs Hightouch to ignore the current state of your model and to track only rows that are added, changed, or removed in the future. The next sync run will be used as an opportunity to capture a snapshot of your model, but no data will be sent to the destination. Thereafter, future sync runs will track new changes only.
The "Reset CDC" option is recommended for insert-only use cases like conversion events, operational alerts, and other situations where a backfill would create undesired duplicate records. Beware that skipping the backfill may cause your destination to drift out of sync with your data model.
### Run details
When viewing a specific sync, you can inspect detailed run-level information. A run is a particular invocation of a sync, either triggered manually or scheduled.
Each run provides:
- The particular run's **status**
- The time the run **started** and its duration in seconds
- The number of **rows queried**
- The total number of **operations**, including rows added, changed, removed
- The **results**—the total number of successful operations in green and the number of rejected operations in red
If there were any errors, you can click the **View Error** button to view details related to the run.
### Configuration
The configuration section exposes destination-specific configuration options. While sync configurations vary depending on the destination, they generally include:
- **Type**: What data type to sync to in your destination; for example, objects, events, subscription lists, etc. To learn more, see sync [types and modes](/syncs/types-and-modes).
- **Mode**: What update mode to use; for example, upsert, insert, update, etc. To learn more, see sync [types and modes](/syncs/types-and-modes).
- **Record matching** If updating or upserting, how rows in the source should be matched to records in the destination. To learn more, refer to the [record matching docs](/syncs/record-matching).
- **Field mapping**: What columns from your source should be synced to the destination, and what destination fields they should appear as. To learn more, refer to the [field mapping docs](/syncs/mapping-data).
- **Batching**: How data should be batched in API requests.
- **Delete behavior**: What to do when a record leaves your source—in other words, when your model no longer returns a record that was returned in the previous sync run.
#### Delete behavior
For some sync types and destinations, you may want to delete records if they no longer appear in your model's query results.
For example, you may be syncing customer records to your CRM based on a [model](/getting-started/concepts#models) that queries all active users from your data warehouse. If some users become inactive—in other words, they no longer appear in your model's query results—you may want to delete them or some information about them from your CRM.
Depending on the destination and sync mode, these are common options for delete behavior:
| Behavior | Description |
| ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Do nothing** | Preserve the record in the destination. This option is useful for when you want your destination to continually **accrue** records from your source. |
| **Clear fields** | Keep the record, but clear the mapped fields. This option useful to retain records in your destination, but clear the fields that are mapped in your sync. |
| **Delete destination record** | Delete the record in the destination. This option is useful for when you want your destination to exactly **mirror** your source by dropping records that have dropped out of your model. |
Hightouch only acts on records as they leave your model's query results, and _after_ you apply deletion settings. Hightouch doesn't have access to historically removed records, so syncs can't delete records or clear fields that have already been removed from your source. Hightouch only deletes records that left your source in comparison to the immediately previous run. A [full resync](#resync-full-query) doesn't delete or clear historically removed records.
Regardless of the option you choose for delete behavior, a [sync run's detail
page](/syncs/debugger#view-run-details) always displays removed rows.
Check out the particular destination's documentation for its delete behavior configuration details.
#### Configuration actions
When working with a sync configuration, you can also choose to [test a row](#test-a-row) or edit the configuration [as a JSON](#edit-as-json) object.
#### Test a row
You can test a sync by sending just one row from your model to your destination. This lets you verify that the sync is working as intended before initiating a run of your entire sync.
From your sync's configuration page, click **Test** and select a row to sync.
Before testing a row, you can edit the columns’ data types and values by hovering over them.
Click the pencil icon, make your edit, and then click the check mark to save any changes.
Editing the model column used for [record matching](/syncs/record-matching#model-column-selection) is not supported since Hightouch [intentionally strigifies](/models/data-types-casting) it for enhanced performance.
The row testing window only displays certain [data
types](/models/data-types-casting#data-types), such as **string** and
**number**. If a model contains a column with a different type, such as
**date**, the UI displays the type as **string**. This only concerns the
UI—Hightouch doesn't convert the data type to string when syncing data. To
confirm a column's data type, refer to its type on the [model
configuration](/models/data-types-casting#view-model-data-types) page.
You can then decide to sync the row as an [added, changed, or removed row](/syncs/debugger#view-run-details) and observe the API requests and responses.
The UI might take a few seconds to update before displaying the results.
#### Edit as JSON
You can edit your sync's configurations as JSON code to quickly make multiple changes or copy and paste one sync's configurations to another.
For example:
```JSON
{
"mode": "upsert",
"type": "object",
"fromId": "track_visitor_id",
"object": "user",
"mappings": [
{
"to": "uuid-western",
"from": "uuid"
},
{
"to": "Trialer Status",
"from": "trialer_status"
}
],
"customMappings": [
{
"to": "unique_notification_status",
"from": "notify_status"
}
]
}
```
### Schedule
Part of sync configuration is deciding how frequently the sync should run. Besides triggering syncs manually, you can schedule syncs with these options:
- **Interval**: You can schedule your sync to run on a set interval, for example once every day.
- **Custom recurrence**: You can schedule your sync to run a specific date and time, for example every Monday at 9 AM.
- **Cron expression**: You can schedule your sync using a [cron expression](https://en.wikipedia.org/wiki/Cron).
- **dbt Cloud**: You can trigger syncs automatically via [dbt Cloud](/extensions/dbt-cloud).
- **Fivetran**: You can trigger syncs automatically via [Fivetran](/extensions/fivetran).
Though not listed as schedule options, you can also trigger syncs using [Airflow](/extensions/airflow), [Dagster](https://github.com/hightouchio/dagster-hightouch), [Prefect](https://docs-v1.prefect.io/api/latest/tasks/hightouch.html), [Mage](https://docs.mage.ai/integrations/hightouch), or the [REST API](/api-reference). To learn more about your scheduling options, refer to the [sync scheduling page](/syncs/schedule-sync-ui).
If you have dependencies between syncs, for example you require one sync to complete before another begins, sync [Sequences](/syncs/schedule-sync-with-sequences) can help you manage scheduling.
If you're planning around Daylight Saving Time changes, review the [DST guidance](/syncs/schedule-sync-ui#daylight-saving-time-dst) on the sync scheduling page.
### Alerts
You can configure [alerting](/syncs/alerting) on a per-sync basis from this section.
### Sync logs
You can configure [Warehouse Sync Logs](/syncs/warehouse-sync-logs) from this section. This feature writes information on the results of your syncs back into your warehouse so that you can perform more complex analysis.
---
## Record matching
**URL:** https://hightouch.com/docs/syncs/record-matching
**Description:** Learn how Hightouch matches records from your model rows to your destination records.
**Section:** Syncs
If you're using [update, upsert, or archive mode](/syncs/types-and-modes) to sync data, one of the first steps of your sync configuration is defining how Hightouch should match rows from your model to existing destination records.
To do this, Hightouch asks you to select the model column and destination field to match on.
**"Columns" vs. "fields" vs. "properties"**: Though some may use these terms
interchangably,
Hightouch uses the term "columns" to refer to a set of vertical data values in
a source or model and "fields" or "properties" to refer to them in a destination.
## Destination field selection
It's often best to start by selecting the destination field to match on because some destinations have requirements or recommendations on the field to use.
For example, some destinations require matching on PII fields like email or phone number. Hightouch only displays permitted fields in the dropdown when selecting a destination field to match records on.
For example, [Attentive](/destinations/attentive#matching-users) allows you to match users on several fields, including email, phone number, and various IDs.
If there is a recommended field, Hightouch displays **(recommended)** next to the field name. For example, [Salesforce](/destinations/salesforce) recommends using an external ID field like `External_ID__c` for matching.
Refer to the record matching section of each destination doc for more information.
### External IDs
Some destinations, for example, [Meta Conversions API](https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/external-id/) and [Salesforce](/destinations/salesforce), use the concept of **external IDs**. External IDs refer to unique identifiers from another system "external" to the destination platform. This allows the platform to maintain its own primary unique identifiers while enabling associations with data from other platforms.
Each platform may have different specifications for fields used as external IDs. For example, [Salesforce](https://help.salesforce.com/s/articleView?id=000385174&type=1) requires `External ID` type fields to be the explicit External ID type in Salesforce, not just an external ID field by practice. Ensure that any destination field you select for record matching complies to the platform's specifications.
## Model column selection
When selecting which model column to use, you need to consider more than the destination requirements.
Since the column you select is used to identify which records to update, it's imperative to **choose a column with unique values**. For that reason, the model column you choose is often the same as the [primary key](/models/creating-models#unique-primary-key-requirement) column you select for your model. Though the model's primary key can be used for record matching, this isn't mandatory.
For example, you may have `email` and `user_id` fields in your model. Which you choose for record matching depends on:
- Which fields in your destination are available and expected for matching
- Which field is guaranteed to be unique
If both fields are accepted by the destination for matching, you should choose the one which you are more confident is strictly unique.
---
## Retry strategy
**URL:** https://hightouch.com/docs/syncs/retries
**Section:** Syncs
Whenever a sync runs, Hightouch tracks all records that the destination rejects using their [primary key](/models/creating-models#unique-primary-key-requirement). Hightouch then automatically retries syncing these records two ways:
1. **Within an Active Sync:** If Hightouch receives any record-level errors during an active sync, it automatically retries those records at an exponential backoff until it meets a destination-dependent failure threshold. Hightouch's retry process is similar to the one [described by AWS](https://docs.aws.amazon.com/general/latest/gr/api-retries.html). Some destinations support [split retries](#split-retries) as part of the active sync retry strategy.
2. **On Subsequent Syncs:** Hightouch keeps an index of all failed records and automatically retries them on every subsequent sync until they succeed.
## Example retry scenario
Imagine you are trying to sync 10,000 records to Salesforce in a single sync, and five of the records are locked within Salesforce at the beginning of the sync.
- During that sync run, imagine two of those five records become available for update. Because Hightouch performs retries during an active sync, these two will update successfully during the sync run.
- On the subsequent sync run, Hightouch automatically retries the remaining three records. Assuming they're unlocked by that point, they will successfully update on the subsequent sync. Hightouch continues to retry the failed records until they are successfully synced.
## Split retries
Some destinations count all records in a batch request as rejected if the request contains a single invalid record.
For some of these, Hightouch provides you the option of enabling **split retries**.
If you enable split retries, Hightouch splits the rejected batch in half and retries each sub-batch separately.
If either of those sub-batches fail, Hightouch splits _it_ in half and retries each sub-batch separately.
Hightouch repeats this process until the destination has marked each record in the batch as successful or rejected.
The split retry strategy allows you to pinpoint which records are getting rejected with which errors.
It also reduces the number of valid records that get retried.
Split retries are part of the [active sync retry strategy](/syncs/retries).
Hightouch retries records that are still rejected at the end of the split retry process in the subsequent sync.
To see if a destination supports split retries, refer to its docs page.
Enabling split retries may lead to longer sync times.
## FAQ
### If part of my sync failed, does Hightouch retry every record in the sync?
Hightouch generally only retries failed records.
For example, if you try to sync 1,000 records to Salesforce, but only 600 pass, Hightouch retries only the 400 that failed.
The exception to this is when a destination marks an entire batch as failed, even if only some of the records are invalid.
In that case, Hightouch retries the entire failed batch.
Enabling [split retries](#split-retries) can help to reduce the number of valid records that get retried.
### How can I be alerted to failing records or syncs?
Hightouch retries failed records until the underlying issue is addressed.
To become aware of failed records or syncs and fix them, it's best to set up [alerting](/syncs/alerting).
### How can I avoid Hightouch retrying invalid rows?
If one or more of your records have malformed data that won't be accepted by the destination, Hightouch retries them and they will continue to be rejected.
If you don't want to sync these records to the destination, you can [edit your model's definition](/models/sql-editor) to filter out these rows.
The SQL query you use to filter out invalid rows will depend on your destination's definition of accepted values.
### Why is Hightouch retrying a removed row?
If rows are removed from your model query result and you've configured [delete behavior](/syncs/overview#delete-behavior) for your sync, Hightouch sends requests to your destination to clear or delete removed records. Hightouch either clears or deletes the removed records from the destination based on the delete behavior setting you configured.
If a sync is unable to clear or delete certain destination records, Hightouch retries the request during the following sync run. This can happen because the destination rejects the request or if the destination's API is inaccessible.
If you don't want the sync to retry removed rows in future sync runs, you can choose one of the following options:
1. Set delete behavior to **Do nothing**. Keep in mind that this affects all rows removed from the model's query results, not just the ones being retried.
2. Temporarily set delete behavior to **Do nothing**, run the sync [manually](/syncs/schedule-sync-ui#manual-runs) once,
and then reconfigure the sync to the delete behavior setting you would like. This clears out the removed rows from the [diff file created by CDC](/syncs/cdc).
3. [Reset change data capture (CDC)](/syncs/overview#reset-cdc) without a backfill.
Please keep in mind that:
- with option 2), during the run where delete behavior is set to **Do nothing**, no model rows that CDC marks as "removed" are synced to the destination
- with option 3), any changes to your model (added, changed, or removed rows) between the previous run and the Reset CDC run aren't synced to the destination
To minimize the risk of model data not being synced to the destination, we recommend triggering a regular sync run right before following through with options 2) or 3).
---
## Schedule syncs in the UI
**URL:** https://hightouch.com/docs/syncs/schedule-sync-ui
**Section:** Syncs
## Overview
Syncs declare how you want query results from a model to appear in your destination, including *how often* you want the data to be updated. This last step of sync configuration is called a sync's **schedule**, and you can configure it from the Hightouch UI.
From the [**Syncs** overview](https://app.hightouch.com/syncs?from=docs) page, select the sync you want to schedule and open the **Schedule** tab. You can then choose the sync's **Schedule type**.
### Schedule types
- **Manual**: Run your sync manually or via the [Airflow](/extensions/airflow), [Dagster](https://github.com/hightouchio/dagster-hightouch), [Prefect](https://docs-v1.prefect.io/api/latest/tasks/hightouch.html), [Mage](https://docs.mage.ai/integrations/hightouch) extensions, or the [REST API](/api-reference). You must also select this schedule type for any syncs you plan to use [in a sequence](/syncs/schedule-sync-with-sequences).
- **Interval**: Run your sync on a set interval, for example, every 12 hours.
- **Custom recurrence**: Use a visual interface to set what days and times your sync should run, for example, every Monday at 8:30 AM.
- **Cron Expression**: Use a cron expression to decide when your sync should run. This option is helpful if you need highly customized schedules, for example, every minute from 9 AM - 5 PM and every hour outside of business hours.
- **dbt Cloud**: Schedule your sync to run upon the completion of a dbt job.
- **Fivetran**: Schedule your sync to run upon the completion of a Fivetran job.
Times you configure in the UI and the respective sync schedules are in your local timezone. Cron expressions are evaluated in UTC.
### Manual runs
A sync with a **Manual** schedule type only runs when you tell it to run. Click the **Run** button in the top right corner of your sync's configuration page to kick off a manual run.
You also need to select a **Manual** schedule if you plan to:
- use the sync [in a sequence](/syncs/schedule-sync-with-sequences)
- trigger the sync with the [Airflow](/extensions/airflow), [Dagster](https://github.com/hightouchio/dagster-hightouch), [Prefect](https://docs-v1.prefect.io/api/latest/tasks/hightouch.html), or [Mage](https://docs.mage.ai/integrations/hightouch) extensions
- trigger the sync with the [REST API](/api-reference)
### Interval schedules
Use the **Interval** schedule type to configure a sync to run every *n* **minutes**, **hours**, **days**, or **weeks**.
You can choose when to apply the schedule—effective immediately or from a specific date and time—and how long the schedule should remain effective—indefinitely or until a particular date and time.
Interval scheduling starts the next sync run at approximately the specified interval after the last run. The sync start time can drift due to various reasons including [manual runs](#manual-runs) and background processes.
If you require precise sync runs start times, use [custom recurrence scheduling](#custom-recurrence-schedules) for a more stable sync schedule.
### Custom recurrence schedules
Use the **Custom recurrence** option to schedule your sync on specific days and times. The time you select in the UI is your local timezone, and your sync runs at that time in your local timezone.
You can add multiple recurrences by selecting **Add recurrence** and configuring the day and time.
The preceding screenshot shows a sync schedule that runs at midnight on weekdays and on 8 AM weekends.
You can choose when to apply the schedule—effective immediately or from a specific date and time—and how long the schedule should remain effective—indefinitely or until a particular date and time.
### Cron expression schedules
You can schedule your sync with a [cron](https://en.wikipedia.org/wiki/Cron) for even greater specificity. Your cron expression should contain five—not six—fields, since the smallest time denomination Hightouch supports is minutes not seconds. Refer to the example expressions below:
| Cron expression | Description |
| :-------------- | :-------------------------------------------------- |
| * * * * * | Every minute |
| 0 12 * * * | Every day at noon |
| 0 12 1 * * | Every month on the first day of the month at noon |
| 0 12 L * * | Every month on the last day of the month at noon |
| 0 12 * 12 1,2 | Every Monday and Tuesday at noon in December only |
Consult a [cron expression generator](https://crontab.guru/#*_*_*_*) to help build an expression that meets your needs.
Cron expressions are evaluated in UTC.
Hightouch doesn't support the following non-standard characters for Cron expressions: `W`, `LW`, `L-2`, `L-3`, `#`.
### dbt Cloud
To schedule syncs via dbt Cloud, dbt requires you to have a paid plan.
Refer to our documentation on [scheduling a sync with dbt](/extensions/dbt-cloud) for setup instructions.
### Fivetran
Refer to our documentation on [scheduling a sync with Fivetran](/extensions/fivetran) for setup instructions.
## Tips and troubleshooting
### Daylight Saving Time (DST)
Daylight Saving Time can affect schedules differently depending on the schedule type:
- **Cron expression** schedules are evaluated in UTC, so the UTC runtime stays fixed while local wall-clock time can shift by one hour when DST starts or ends.
- **Custom recurrence** schedules run at the time shown in your local timezone in the UI.
For DST-sensitive schedules, Hightouch recommends these approaches:
- If possible, define your SLA in UTC and keep the schedule fixed.
- If a one-hour local-time shift is acceptable, keep your current schedule and monitor around DST transitions.
- If you need a strict local wall-clock time (for example, "7:00 AM local" year-round), use an external timezone-aware scheduler and trigger the sync with the [REST API](/api-reference). Common options include [Airflow](/extensions/airflow), [Dagster](https://github.com/hightouchio/dagster-hightouch), [Prefect](https://docs-v1.prefect.io/api/latest/tasks/hightouch.html), cloud schedulers, or your own application scheduler.
### Disabled syncs continue to run
If you [disable](/syncs/overview#individual-sync-overview-page) a sync but it still has recent runs, check to see if it's part of a [sequence](/syncs/schedule-sync-with-sequences). Sync schedules and sync sequences run independently of each other, so a sync that is disabled still runs if it is part of a sequence. Hightouch recommends setting any syncs you plan to use in a sequence to the [manual schedule type](#manual-runs).
Disabled syncs can also run if they're [triggered manually](#manual-runs) or by an external service, such as [Airflow](/extensions/airflow).
---
## Schedule syncs with Sequences
**URL:** https://hightouch.com/docs/syncs/schedule-sync-with-sequences
**Section:** Syncs
## Overview
Sync **Sequences** allow you to execute a series of related syncs in a predefined order. Sequences are helpful when you have dependencies between syncs. For example, you may have two syncs you need to run in order:
1. Sync A creates a set of new contacts in your CRM
2. Sync B adds the new contacts to their associated accounts
Without Sequences, you would have to schedule the two syncs individually to give the first enough time to finish before the second starts. It would be challenging to guarantee the correct behavior since sometimes sync duration can vary.
Sequences make orchestration a first-class feature in Hightouch so you can control how and when to sync data to your destinations. You can schedule sequences to run on various [schedules](/syncs/schedule-sync-ui#schedule-types), such as [triggered by dbt Cloud runs](/extensions/dbt-cloud) or [cron schedules](/syncs/schedule-sync-ui#cron-expression-schedules), just as you would with syncs. You can also [initiate sequences manually](#sequence-management).
## Sequence example
Suppose you manage your staff members, their permissions, customer information, and staff customer assignments using Hightouch. To do so, you use four syncs that need to run in a specific order:
1. The table of staff members needs to be updated first.
2. Then the staff permissions can be updated.
3. Separately, a table of customer prospects needs to be updated.
4. The final sync updates staff member assignments to customer prospects.
To represent this sequence of syncs, you can create a sequence that executes these four syncs in order with the following error handling:
- The staff member sync terminates the sequence if it fatally errors or has too many rejected rows.
- The second and third syncs only terminate the sequence if they fatally error.
- The final staff member assignment sync doesn't check for errors, so the sequence completes successfully even if this sync fails.
## Sequence setup
Before you create a sequence, ensure that all syncs you plan to use in it have their schedule type set to [**Manual**](/syncs/schedule-sync-ui#schedule-types). You can configure a sync to have a set schedule and use it in a sequence simultaneously, but we recommend against it unless you have a particular reason to do so.
Then, to create a new sequence, go to the [**Sequences** page](https://app.hightouch.com/sequences?from=docs) and click the **Add a sequence**.
### Add syncs to the sequence
Click **Add syncs** to select syncs for the new sequence. You can include as many syncs as you want and add more later as needed.
### Configuring sync order and error handling
Once you've added syncs to the sequence, you can re-order them using the up and down arrows.
You can configure each sync's error-handling behavior using the **Terminate** drop-down menu.
### Set your sequence schedule
To finalize your sequence, give it a name and define its schedule type. You have all the same options for sequence scheduling as for [sync scheduling](/syncs/schedule-sync-ui#schedule-types).
Once you save your new sequence, it's visible on the main [**Sequences** page](https://app.hightouch.com/sequences?from=docs).
## Sequence management
You can click a sequence from the [**Sequences** page](https://app.hightouch.com/sequences?from=docs) to go to its details page. There you can:
1. Disable or enable it
2. Run it manually
3. See the sequence's run history
4. Or update its configuration or schedule
Clicking on a particular run displays its details in a pop-up, including the execution time and final state of each sync run belonging to the sequence run.
---
## Sync types and modes
**URL:** https://hightouch.com/docs/syncs/types-and-modes
**Description:** Learn more about different types of syncs and update modes, inluding upsert, update, insert, and more.
**Section:** Syncs
When [configuring a sync](/syncs/overview#configuration) there are two key settings that determine the basic behavior of your sync:
- Sync type
- Sync mode
The options for both vary depending on the destination. Refer to the specific destination documentation for more information about its supported sync types and mdoes.
## Sync types
The **sync type** refers to _what_ you are syncing in a destination. These are the most common types:
- **Objects**: such as customers, accounts, organizations, catalog items, and the objects' associated attributes or metadata
- **Events**, **Actions**, and **Activities**: such as conversion events
- **Segments**, **Audiences**, and **Lists**, : such as subscription or campaign lists, specifically user membership in these lists
When syncing to CRM type destinations, it's commonly to object type records. When syncing to advertising or marketing platforms, it's commonly to objects, events, or lists.
For some destinations, you can also [manage journeys](/destinations/sfmc#managing-user-journeys), [trigger campaigns](/destinations/marketo#campaign-trigger), or [merge anonymous and known users](/destinations/braze#merge-anonymous-and-known-users) by syncing data. Check out a destination's dedicated docs to learn its supported sync types.
## Sync modes
Depending on your sync type, you may have to select your **sync mode**. For example, if you're updating contacts, you might have the option to **Insert**, **Update**, or **Upsert** them.
Also called update mode, this configuration describes _how_ you want to update records in your destination based on your [model's](/models/creating-models) query results. For example, if your model queries a **Users table** and rows in that table change, your sync mode determines how those changes should translate to your destination.
Depending on the destination and sync type, the supported sync modes could include:
- **Update**: only update existing records in your destination
- **Insert**: only insert new records without touching existing records
- **Upsert**: update existing records and insert new ones
- **Add**: add records, usually user records in a segment, list, or journey
- **Remove**: remove records, usually user records in a segment, list, or journey
- **Archive**: archive all records that appear in your model's query results (only for Mailchimp, Intercom, and Salesforce destinations)
- **All**: overwrites all existing records in your destination with the results of your model query; sometimes called **Mirror** mode
- **Snapshot**: creates a new file on each sync run with the full results set of your model query
- **Diff**: creates three new files on each sync run—one for rows added, one for rows changed, and another for rows removed since the last sync
Some sync types such as triggering campaigns, merging users, or sending messages don't require you to select a mode since the sync applies an action rather than updates records. Other sync types, such as [events](#event-syncs), have only one sync mode option.
In **All** and **Snapshot** sync modes, the order of rows and columns in exported files may differ from how they appear in your model. This is normal behavior. [Learn more about row and column ordering →](/models/creating-models#row-and-column-ordering)
### Sync modes and change data capture
[Change data capture (CDC)](/syncs/cdc) is a process that captures changes made to your source data and that records those changes. Specifically, CDC tracks rows that were added to, changed, or removed from your model's query results since the last sync run.
Your sync mode determines how Hightouch acts on observed changes:
- In **update** mode, Hightouch only acts on changes to existing rows.
- In **insert** mode, Hightouch only syncs new rows—those whose [primary key](/models/creating-models#unique-primary-key-requirement) wasn't present in the previous sync run.
- In **upsert** mode, Hightouch acts on existing _and_ new rows rows.
- **Archive** and **all** modes don't perform change data capture and don't store requests in the [live debugger](/syncs/debugger).
Sync mode is separate from [delete behavior](/syncs/overview#delete-behavior), which determines how to treat removed rows—those that leave your model's query results.
## Sync type and mode combinations
Certain destination types tend to support certain sync modes.
|Destination type|Example destinations|Commonly supported sync modes |
|--|--|--|
|CRMs and other object-based destinations |Salesforce, HubSpot, Braze, Iterable | Upsert, Update, Insert |
|Advertising| Google Ads, Meta Conversions API, TikTok, Amazon Ads | Insert for events, Update or Upsert for customer segments, lists, and audiences |
|File storage| Google Drive, Amazon S3, Azure Blob,|Insert, All, Diff|
Check out Hightouch's [integration catalog](https://hightouch.com/integrations) for a complete list of destinations and categories.
### Event syncs
Event syncs are **insert** only, meaning that you can't update previously synced events or activities.
Hightouch follows this convention because events tend to be part of [fact tables](https://allthingssql.com/fact-table-vs-dimension-table/) rather than dimension tables.
Fact tables store transactions as rows that shouldn't change.
Recall that in [insert mode](#sync-modes-and-change-data-capture), Hightouch syncs rows whose [primary key](/models/creating-models#unique-primary-key-requirement) wasn't present in the previous sync run.
If the event sync's associated model has a primary key that appears in multiple events—for example, `user_id`—syncs will omit some events.
To ensure syncs include all events, select a truly unique column, for example a unique `eventID`.
If your events data doesn't contain an inherently unique column, define a [composite primary key](/models/creating-models#composite-primary-keys) during model creation.
---
## Warehouse Sync Logs
**URL:** https://hightouch.com/docs/syncs/warehouse-sync-logs
**Section:** Syncs
The **Warehouse Sync Logs** feature writes sync metadata back into your data warehouse. It makes per-row information from the [live debugger](/syncs/debugger) available in your warehouse so that you can perform complex analysis, rather than just inspect syncs row-by-row.
When you enable Warehouse Sync Logs, Hightouch logs a corresponding row for every row processed during the sync. This includes the row's **status** and any errors from processing the row. You can then explore these logs using SQL or BI tools you use on top of your warehouse.
For example, you can:
- **Categorize** all the errors in your sync using [regular expressions](https://docs.snowflake.com/en/sql-reference/functions-regexp.html) and find unexpected errors.
- **Filter** out previously failed rows from your [model](/getting-started/concepts#models) using a `JOIN`.
- **Aggregate** the sync history to see what rows are changing the most. Flapping rows can be a sign of data integrity issues.
- **Visualize** row changes in your models over time. For example, you may be interested in seeing how targeted users in an ad campaign changed over the campaign duration.
Refer to the [example queries](/syncs/warehouse-sync-logs#example-queries) section for concrete examples.
## Schema
Hightouch writes sync logs into three tables within the `hightouch_audit` schema:
- **Changelog**: This table contains a row for every operation performed by Hightouch. It includes the result of the operation and any error messages from syncing.
- **Snapshot**: This table contains each row's latest status in your model. The information is similar to the **Changelog** table, but since it contains the latest status, it's easier to query for some use cases.
- **Sync runs**: This table contains a log of all the sync runs. You can `JOIN` the changelog and snapshot tables to this table for more information on when the sync occurred and how it was configured.
Information across all syncs is written into these same three tables—you can differentiate which rows were part of which sync using the `sync_id` column.
See the [detailed schema](/syncs/warehouse-sync-logs#detailed-schema) section for a detailed description of the available columns.
You can learn more about PII in these tables on the [Lightning sync
engine](/syncs/lightning-sync-engine#table-removal) page.
## Setup
Enabling Warehouse Syncs logs requires you to enable [the Lightning sync engine](/syncs/lightning-sync-engine) first. Hightouch supports using Warehouse Sync Logs with the following sources:
- [Amazon Redshift](/sources/amazon-redshift)
- [Databricks](/sources/databricks)
- [Google BigQuery](/sources/google-bigquery)
- [Greenplum Database](/sources/greenplum)
- [PostgreSQL](/sources/postgresql)
- [Snowflake](/sources/snowflake)
### Required permissions
The user you used to connect your source to Hightouch must be able to write into the `hightouch_audit` schema. You shouldn't require any additional permissions once you've set up [the Lightning sync engine](/syncs/lightning-sync-engine#granting-required-warehouse-permissions).
## Enable Warehouse Sync Logs for a source
To enable logging for **all syncs** that use a source:
1. Go to `Integrations` → `Sources`.
2. Select your source.
3. Open the `Sync logs` tab.
4. Select the log tables you want Hightouch to write to your warehouse:
- **Sync snapshots**
- **Changelog**
- **Sync runs**
- **Audience snapshots** (optional)
- **Audience holdout group logs** (optional)
Each of the first three tables also has an **Audience syncs only** sub-option that limits logging to audience syncs.
5. Save.
Enabling a table here allows Hightouch to write logs for **any sync** that uses this source.
**Be mindful of storage usage**. Sync logs can generate a lot of data
depending on how many tables you enable and how often your syncs run. If you
only need logs for a few workflows, you can enable them at the **sync level**
instead of the source level to reduce storage. However, enabling logs at the
**source level** ensures logs are always available when debugging issues,
which can save considerable time if a sync fails unexpectedly.
### Enable Warehouse Sync Logs for a sync
Use sync-level logging when you only need logs for specific workflows or want to limit storage usage. Use source-level logging when you want logging enabled consistently across all syncs powered by the source.
Warehouse Sync Logs are off for all syncs by default. To enable them on a particular sync:
1. Ensure the [Lightning sync engine](/syncs/lightning-sync-engine#enable-the-lightning-sync-engine) is enabled for the source.
2. Go to the **Sync Logs** tab in the sync's overview page.
3. Enable your desired [tables](/syncs/warehouse-sync-logs#schema): **Snapshot**, **Changelog**, and/or **Sync runs**.
## Example queries
The following example queries are written for Snowflake, but you could create similar queries for other sources. Check out Hightouch's [dbt package](https://github.com/hightouchio/dbt-hightouch) for more use cases.
### Get the most common sync error
This SQL groups and counts rows by `failure_reason`, enabling you to find the most common sync error.
```
select
failure_reason,
count(*) as c
from hightouch_audit.sync_snapshot
where failure_reason is not null
group by failure_reason
order by c desc
```
### Track when users entered and exited a model
This SQL tracks when users enter and exit a model. It's particularly useful when used with [Customer Studio audiences](/customer-studio/overview) and visualized in a BI tool.
```
with details as (
select
model_name,
row_id,
op_type as type,
started_at as timestamp,
lag(op_type) over(partition by model_name, row_id order by started_at) as lag_type
from hightouch_audit.sync_changelog c
join hightouch_audit.sync_runs r on c.sync_id = r.sync_id
where op_type != 'changed'
order by model_name, row_id
)
select
row_id as user_id,
model_name as audience,
type,
timestamp
from details
where (lag_type != type or lag_type is null)
order by model_name, row_id, timestamp
```
### Get the current rows in all models
This SQL finds all current (most recently synced rows that didn't fail) across all models. It's particularly useful to find all members for audiences created in [Customer Studio](https://hightouch.com/platform/customer-studio).
```
with model_names as (
select distinct
sync_id,
model_name
from hightouch_audit.sync_runs
)
select
model_name,
row_id as user_id
from hightouch_audit.sync_snapshot s
join model_names r on s.sync_id = r.sync_id
where s.status != 'failed'
qualify row_number() over (partition by user_id, model_name order by null) = 1
order by user_id
```
## Detailed schema
Hightouch writes to the `sync_changelog`,`sync_snapshot`, and `sync_runs` tables after each sync.
If you've enabled **audience snapshotting**, you'll also find a `hightouch_planner.audience_membership` table in your warehouse.
Refer to the [audience snapshot docs](/customer-studio/audience-snapshots) for its detailed schema.
### Changelog table
This `hightouch_audit.sync_changelog` table is log of all changes across all sync runs. If the same row is synced in multiple sync runs, it has multiple entries in this table.
| COLUMN | DESCRIPTION |
| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `sync_id` | The ID of the sync |
| `sync_run_id` | The ID of the sync run |
| `op_type` | Whether the row was added, changed, or removed relative to the last run. This is computed by Hightouch when planning the sync run |
| `row_id` | The value of the row's primary key as defined from the model |
| `status` | Whether the row was successfully synced into destination. They value may be: succeeded - the row was successfully synced, failed - Hightouch attempted to sync the row, but it failed to sync, or aborted - Hightouch planned to sync the row, but didn't attempt to sync. This may happen if the sync may have been cancelled, or the sync encountered a fatal error that terminated the run early. |
| `failure_reason` | If the status is failed, this field contain a string describing why the row failed to sync. |
| `fields` | A JSON object of the raw data from the model that is synced into the destination. Note that this is the raw data from the warehouse, not the payload that Hightouch sent to the destination. This column has limitations in Redshift (see FAQ for details, below). |
| `split_group` | (Optional) The [experiment group](/customer-studio/experiments) name. If no syncs are using experiments, this column isn't created. |
### Snapshot table
This `hightouch_audit.sync_snapshot` table stores the current status of each row in the most recent sync run, even if the row wasn't synced in the most recent run.
After each run, the old statuses for the sync are dropped and replaced with updated statuses.
| COLUMN | DESCRIPTION |
| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| `sync_id` | The ID of the sync |
| `op_type` | Whether the row was added, changed, or unchanged relative to the last run |
| `row_id` | The value of the row's primary key as defined from the model |
| `status` | The status of the row. See the sync_changelog.status description for a list of possible statuses |
| `failure_reason` | If the status is failed, this contains a string describing why the row failed to sync |
| `fields` | The fields from the model for this row. See the sync_changelog.fields description for more information |
| `split_group` | (Optional) The [experiment group](/customer-studio/experiments) name. If no syncs are using experiments, this column isn't created. |
### Sync runs table
This `hightouch_audit.sync_runs` table stores general metadata information about each sync run. You can join the `sync_changelog` and `sync_snapshot` tables using the `sync_id` column.
| COLUMN | DESCRIPTION |
| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| `sync_id` | The ID of the sync |
| `sync_run_id` | The ID of the sync run |
| `primary_key` | The primary key column of your sync as defined on the model attached to the sync. |
| `destination` | The destination type, for example, Salesforce or Braze |
| `model_name` | The name of the model attached to the sync |
| `model_id` | The ID of the model attached to the sync |
| `status` | The status of the sync run. This will be either succeeded or failed. In general, the per-row results of the sync are a better indication of status. |
| `error` | The sync-level error if the sync terminated early |
| `started_at` | When the sync run started |
| `finished_at` | When the sync run finished |
| `num_planned_add` | The number of planned adds. |
| `num_planned_change` | The number of planned changes. |
| `num_planned_remove` | The number of planned removes. |
| `num_attempted_add` | The number of planned adds that were actually attempted. |
| `num_attempted_change` | The number of planned changes that were actually attempted. |
| `num_attempted_remove` | The number of planned removes that were actually attempted. |
| `num_succeeded_add` | The number of planned adds that were successfully synced to the destination |
| `num_succeeded_change` | The number of planned changes that were successfully synced to the destination |
| `num_succeeded_remove` | The number of planned removes that were successfully synced to the destination |
| `num_failed_add` | The number of planned adds that were attempted, but failed to get synced into destination |
| `num_failed_change` | The number of planned changes that were attempted, but failed to get synced into destination |
| `num_failed_remove` | The number of planned removes that were attempted, but failed to get synced into destination |
## FAQ
### What's the performance impact of enabling Warehouse Sync Logs?
The performance impact of enabling Warehouse Sync Logs is low since it reuses data already present from the Lightning sync engine. Hightouch only writes the rows after Hightouch syncs to the destination, meaning there is no effect on destination throughput.
Pruning entries in the history tables is safe. Doing so doesn't affect future syncs, though the deleted rows won't be rewritten into the history tables.
### What are the limitations in Redshift?
{/* */}
Because [Redshift does not support strings longer than 65,535 bytes](https://docs.aws.amazon.com/redshift/latest/dg/r_Character_types.html#r_Character_types-storage-and-ranges),
it is not possible to store JSON representations of arbitrary length in the warehouse sync logs. Hightouch attempts
to store model data in the `fields` column of `sync_changelog` and `sync_snapshot`, but if the model contains either
long strings or a large number of fields, some fields may be truncated.
Truncated values in `sync_changelog.fields` and `sync_snapshot.fields` do _not_ mean that values were truncated
when Hightouch synced data to the destination; it's purely a limitation of the warehouse logs.
{/* */}
---
## Acoustic
**URL:** https://hightouch.com/docs/destinations/acoustic
**Description:** Market better with all of your up-to-date customer data in Acoustic
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
---------------------|-----------------------------------------------------------|----------------------|------------------------------------------------------------------------------------------------------
Any data set | Sync data from any source to an Acoustic relational table or database | Upsert | [Relational table docs](https://help.goacoustic.com/hc/en-us/articles/360043347953-Relational-tables), [Database docs](https://help.goacoustic.com/hc/en-us/articles/360042858454-Import-a-database)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Acoustic
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Acoustic** and click **Continue**. You can then authenticate Hightouch to **Acoustic** by entering the following required fields into Hightouch:
- **Pod**
- **Client Token**
- **Client Secret**
- **Refresh Token**
You can find your Pod, for example, `6`, in your [Acoustic Account Settings](https://cloud.goacoustic.com/settings).
If you know the region but are unsure of your pod, you can use the table below.
| Campaign URL | Pod |
| ------------ | --- |
|api-campaign-us-1.goacoustic.com| 1|
|api-campaign-us-2.goacoustic.com| 2|
|api-campaign-us-3.goacoustic.com| 3|
|api-campaign-us-4.goacoustic.com| 4|
|api-campaign-us-5.goacoustic.com| 5|
|api-campaign-us-6.goacoustic.com| 6|
|api-campaign-eu-1.goacoustic.com| 7|
|api-campaign-ap-2.goacoustic.com| 8|
|api-campaign-us-6.goacoustic.com| 9|
|api-campaign-ap-1.goacoustic.com| A|
|api-campaign-ap-3.goacoustic.com| B|
|api-campaign-pilot.goacoustic.com| PILOT|
You can generate the other credentials in the **Application Account Access** section of your [Acoustic Organization Settings](https://cloud.goacoustic.com/campaign-automation/Settings/Administration/Organization_settings).
See [these instructions](https://developer.goacoustic.com/acoustic-campaign/reference/getting-started-with-oauth#step-one-defining-your-api-application) for more details.
## Sync configuration
Once you've set up your Acoustic destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Acoustic destination you want to sync to.
### Syncing relational tables or databases
Hightouch requires an existing [database](https://help.goacoustic.com/hc/en-us/articles/360042858454-Import-a-database) or [relational table](https://help.goacoustic.com/hc/en-us/articles/360043347953-Relational-tables) in Acoustic to sync to.
The only supported [sync mode](/syncs/types-and-modes#sync-modes) is **Upsert**.
In Upsert mode, Hightouch inserts new records and keeps [mapped attributes](#field-mapping) up-to-date on existing records in Acoustic.
#### Record matching
You can match rows from your model to rows in Acoustic on any column in your model and any [key](https://help.goacoustic.com/hc/en-us/articles/360042859634-Unique-identifiers) fields in Acoustic.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync model columns via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to any existing columns in your Acoustic database or relational table.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|-----------------------------------------------------------------
**Do nothing** | Keep the record in Acoustic with all its synced fields
**Delete** | Delete the synced records from Acoustic
If you enable deletion, Hightouch provides all columns to Acoustic in the deletion API request.
This enables Hightouch to uniquely identify a single record, rather than a single key column, which could match multiple records.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## ActiveCampaign
**URL:** https://hightouch.com/docs/destinations/active-campaign
**Description:** Enrich customer experiences by using ActiveCampaign's automation platform with rich customer and product data
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
-------------|-------------|---------------------
**Contacts** | Sync data from any source to Contacts in ActiveCampaign | Upsert
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to ActiveCampaign
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **ActiveCampaign** and click **Continue**. You can then authenticate Hightouch to **ActiveCampaign** by entering the following required fields into Hightouch:
- **API URL**
- **API Key**
You can find them in ActiveCampain under **Settings** > **Developer** under **API Access**.
You can find more information about these credentials in the [ActiveCampaign docs](https://help.activecampaign.com/hc/en-us/articles/207317590-Getting-started-with-the-API#what-is-an-api--0-1).
## Sync configuration
Once you've set up your ActiveCampaign destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the ActiveCampaign destination you want to sync to.
### Syncing contacts
This integration only supports the **Upsert** mode. In this mode, Hightouch inserts new contacts into ActiveCampaign and keeps all user attributes up-to-date.
#### Record matching
To match rows from your model to records in ActiveCampaign, you need to select the model column that contains values that match the **ActiveCampaign Email** field.
#### Field mapping
You can map data from any of your model columns to native and custom fields in ActiveCampaign. Ensure the data type of your model column matches the data type of the field you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Adestra
**URL:** https://hightouch.com/docs/destinations/adestra
**Description:** Sync contacts and manage list subscriptions in Adestra (MessageFocus) to power your email marketing campaigns.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes |
| ----------------- | ------------------------------------------------------------ | -------------------- |
| **Contact lists** | Sync data from any source to contact lists in Adestra | Add, Remove |
| **Contacts** | Sync records to contacts in Adestra | Upsert, Update |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Adestra
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Adestra** and click **Continue**. You can then authenticate Hightouch to **Adestra**.
Enter the following fields into Hightouch:
- **API URL**: The base URL of your Adestra API instance.
- **API token**: Your Adestra API token for authenticating requests.
- **Core table ID**: The ID of the core table you want to sync data to.
## Sync configuration
Once you've set up your Adestra destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Adestra destination you want to sync to.
### Syncing contacts
Sync records to contacts in Adestra.
#### Record matching
To match rows from your model to contacts in Adestra, you need to select a model column and corresponding Adestra field. You can match on any of the following Adestra fields:
- **Contact ID**
- **Email**
#### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default and custom contact fields in Adestra.
Ensure your model's columns have the same data types as the fields you want to sync to.
### Syncing contact lists
Sync data from any source to contact lists in Adestra.
#### Record matching
To match rows from your model to contacts in Adestra, you need to select a model column and corresponding Adestra field. You can match on any of the following Adestra fields:
- **Contact ID**
- **Email**
#### Field mapping
Hightouch lets you sync contact list fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default contact list fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Adform
**URL:** https://hightouch.com/docs/destinations/adform
**Description:** Adform empowers your marketing team to analyze conversion events and develop real-time feedback on the effectiveness of your campaigns.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API reference |
| -------------------- | ---------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------- |
| **Conversion Event** | Sync conversion event data to Adform | Insert | [Conversions docs](https://www.adformhelp.com/hc/en-us/articles/9740579489041-Use-Server-Side-Tracking) |
| **Segment** | Sync users from your model to Adform DMP | Add | [Audience base (DMP) docs](https://api.adform.com/help/references/audience-base) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Adform
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs), select **Add destination**, select **Adform**, and then select **Continue**.
Adform supports two sync types with different credentials. Enter the fields for the sync types you plan to use:
- For conversion syncs:
- **Tracking Setup ID**
- **Tracking Domain**
- For segment syncs:
- **Client ID**
- **Client Secret**
You can enter one set of credentials or both if you plan to run both sync types from the same destination.
For segment syncs, ask Adform Support to grant your OAuth client these scopes:
- `https://api.adform.com/scope/dmp.segments`
- `https://api.adform.com/scope/dmp.categories`
- `https://api.adform.com/scope/dmp.authenticated`
- `https://api.adform.com/scope/dmp.dataproviders.readonly`
## Sync configuration
Once you've set up your Adform destination and have a [model](/getting-started/concepts#models) to pull data from, go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs), select **Add sync**, and then select the relevant model and Adform destination.
For each sync, select **Conversion Event** or **Segment** for **Sync type**.
### Syncing conversion events
Sync conversion event data to Adform.
#### Record matching
You can match rows from your model to conversions in Adform on any column in your model and any field in Adform. Ensure the data types of the model column and Adform field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
### Syncing audience segments
Sync users from your model to Adform DMP segments.
You can sync to an existing segment or create a new one on the first sync.
Adform segment syncs are add-only.
If records leave your model, Hightouch doesn't remove them from the segment.
#### Select a data provider
Select the Adform data provider that owns the segments you want to sync.
Hightouch fetches the available data providers using the destination's **Client ID** and **Client Secret**.
#### Select an existing segment or create a new one
You can sync to an existing segment or create a new segment in Hightouch.
If you create a new segment, you also need to configure these fields:
- **Category**: Every Adform segment must belong to one category. You can select an existing category or create a new one on the first sync.
- **Data consumer ID**: Optional. Enter an agency ID or publisher ID if another Adform entity should have access to the segment.
- **TTL (days)**: Optional. Controls how long a user remains in the segment after their last upload. The default is `30`.
- **Frequency**: Optional. Controls the minimum number of occurrences required for a user to qualify for the segment. The default is `1`.
#### Select a user identity type
Select the Adform identity type for the user IDs you upload.
Depending on your Adform setup, supported identifier types can include:
- First-party IDs
- Mobile advertising IDs (MAIDs)
- Adform cookies
- ID5 identifiers
After you select an identity type, map the column from your model that contains that identifier to **User ID**.
#### Batch size
You can optionally set a batch size for segment uploads.
The default is `10000`, and Adform limits each upload file to 100 MB.
## Tips and troubleshooting
### Common errors
#### Adform DMP token request failed with invalid_scope
This error means the OAuth client configured on your Adform destination is missing one or more required DMP scopes.
Ask Adform Support to grant these scopes to your OAuth client:
- `https://api.adform.com/scope/dmp.segments`
- `https://api.adform.com/scope/dmp.categories`
- `https://api.adform.com/scope/dmp.authenticated`
- `https://api.adform.com/scope/dmp.dataproviders.readonly`
After Adform updates the client, test the destination connection again.
### Live debugger
### Sync alerts
---
## Adikteev
**URL:** https://hightouch.com/docs/destinations/adikteev
**Description:** Upload mobile advertising identifiers to Adikteev
**Section:** Destinations
## Overview
With Hightouch and the Adikteev API, you can send data from your warehouse to audiences in Adikteev.
Deliver relevant ads by utilizing data from various sources within your data warehouse.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------ | ----------------------------------------------------------- | -------------------- |
| **Audience** | Create audience lists and keep them up-to-date in Adikteev. | All (file upload) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Adikteev
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Adikteev** and click **Continue**. You can then authenticate Hightouch to Adikteev via username and password.
Enter your Adikteev credentials and hit **Test Connection** to verify credentials.
If successful, click **Continue**. To finish connecting, give your Adikteev destination a descriptive name.
## Sync configuration
Once you've connected your Adikteev destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Adikteev destination you want to sync to.
### Syncing custom audiences
Hightouch lets you create and maintain custom audiences via the [Custom Audiences API](https://help.adikteev.com/hc/en-us/articles/360003819660-Custom-Audiences-API).
#### User identifiers
To identify which users to add or update in an audience, select the model column that represents the user's device ID.
Adikteev allows two different device ID types for iOS devices: IDFA and IDFV.
Be sure to select the correct option in the ID type dropdown.
#### Custom Audiences API mode
Adikteev lets you upload an entire audience through their [Custom Audiences API](https://help.adikteev.com/hc/en-us/articles/360003819660-Custom-Audiences-API#SendingCustomAudiencesToAdikteev-Step3-Uploading/UpdatingadeviceIDlistforasegment). Hightouch uploads the entire model query results for each sync run.
These results effectively replaces the previous upload.
#### Custom audience list
Hightouch lets you use an existing audience in Adikteev or create a new one. Note that if you create a new audience, you can't
change the name once your sync has run. Instead, you have to create a new sync.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Adjust
**URL:** https://hightouch.com/docs/destinations/adjust
**Description:** Streamline your app growth with Adjust’s mobile marketing platform.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
------------|--------------------------------------------------------------------------------|---------------------
**S2S Events** | Sync records as events to Adjust. This is often in the form of a track call. | Insert
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Adjust
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination**
button. Select **Adjust** and click **Continue**. You can then authenticate Hightouch to **Adjust** by entering the event token and app token:
If you do not have an event or app token, use the following steps from [Adjust](https://help.adjust.com/en/article/server-to-server-s2s-security#s2s-authentication) to set up S2S authentication and generate them:
1. Under **AppView**, select **All apps**.
2. Select your app.
3. Select the **Protection** tab.
4. Under the **S2S Security** section, select **Open S2S Security**.
5. Select **Add token**.
6. Enter a token name, and select **Add token**.
You must also activate S2S Authentication with the following steps in [Adjust](https://help.adjust.com/en/article/server-to-server-s2s-security#activate-s2s):
1. Under **AppView**, select **All apps**.
2. Select your app.
3. Select the **Protection** tab.
4. Under the **S2S Security** section, select **Open S2S Security**.
5. Ensure you have at least one token configured.
6. Switch the toggle **ON**. A warning message appears to inform you that only requests sent with configured tokens will work.
7. Select **Turn on** to proceed.
Once you've entered the event and app, click **Continue**. Give your destination a descriptive name and **Finish** the destination connection setup.
## Sync configuration
Once you've set up your Adjust destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and
the Adjust destination you want to sync to.
#### Record matching
Adjust requires all S2S events to have at least one device identifier. The device identifiers accepted for S2S events are as follows:
- Raw IDFA (IOS only) (`idfa`)
- Raw Google advertising ID (`gps_adid`)
- Raw Amazon Fire advertising ID (`fire_adid`)
- Open Advertising ID (OAID, Huawei only) (`oaid`)
- Raw IDFV (`idfv`)
- Android ID (`android_id`)
#### Field mapping
Adjust recommends including the following parameters with events:
- Adjust ID (`adid`)
- IP address (`ip_address`)
- Timestamp (`created_at` or `created_at_unix`)
- User agent (`user_agent`)
If you do not include the event timestamp, Adjust will use the time the event was received.
Hightouch accepts a normal date time format for the timestamp, and
automatically converts it to the format required by Adjust.
Adjust does not accept events more than 58 days old.
#### Custom Data
Hightouch allows mapping fields as `callback_params` or `partner_params` to send with Adjust events.
**Callback parameters** are available in your raw data exports and **partner parameters** are shared with your network partners.
In Hightouch, type in the name of the field you would like included in these parameters, and choose a column to sync to it.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Adobe Analytics
**URL:** https://hightouch.com/docs/destinations/adobe-analytics
**Description:** Sync your analytics data from your warehouse to Adobe Analytics
**Section:** Destinations
## Overview
Adobe Analytics is a web analytics service that provides tools to understand customer behavior, measure marketing campaigns, and optimize the user experience. With Hightouch, you can send data from your data warehouse to Adobe Analytics to enrich your analytics with valuable customer data from various sources. This helps you build a more complete view of the customer journey and improve your marketing and product strategies.
This destination uses the [Adobe Analytics Batch Data Insertion (BDIA) API](https://developer.adobe.com/analytics-apis/docs/2.0/guides/endpoints/bulk-data-insertion/), which allows you to upload large amounts of data in batches.
## Supported syncing
| Type | Description | Supported Sync Modes |
| ---------- | --------------------------------------------------------- | -------------------- |
| **Events** | Sync event data to Adobe Analytics for detailed analysis. | Insert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Adobe Analytics
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Adobe Analytics** and click **Continue**. You can then authenticate Hightouch to **Adobe Analytics** by entering the following required fields:
- **API Key (Client ID)**
- **Client Secret**
These credentials are not your Adobe Analytics login details. They are obtained by creating an OAuth app in the Adobe Developer Console.
For more information on how to create an OAuth app and obtain these credentials, see [Adobe's documentation](https://developer.adobe.com/developer-console/docs/guides/authentication/ServerToServerAuthentication/implementation#setting-up-the-oauth-server-to-server-credential)
## Sync configuration
Once you've set up your **Adobe Analytics** destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Adobe Analytics destination you want to sync to.
This destination allows you to sync event data to Adobe Analytics. Each row in your model should represent a single event or interaction.
### Visitor identification
You can match rows from your model to visitors in Adobe Analytics using one of the following identifiers. Ensure the data types of the model column and the Adobe Analytics field you select match.
- **Visitor ID**: A unique identifier for the visitor.
- **Marketing Cloud Visitor ID**: The Adobe Marketing Cloud visitor identifier.
- **IP Address**: The visitor's IP address.
- **Customer ID**: Your internal customer identifier.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Adobe Analytics automatically generates an identifier for
every new record synced, so there is no need to match an existing record if
you are only sending new events. However, providing a stable visitor
identifier is crucial for tying events to the correct user profiles.
### Field mapping
Hightouch lets you sync event fields via [field mapping](/syncs/mapping-data). You can map data from any of your model columns to standard and custom Adobe Analytics fields. The following fields are required:
- **Report Suite ID**: Your Adobe Analytics report suite identifier.
- **Timestamp**: The time of the interaction, in Unix timestamp or ISO-8601 format.
- **User Agent**: The device's user agent string.
For a full list of available fields, refer to the [Adobe Analytics BDIA API documentation](https://developer.adobe.com/analytics-apis/docs/2.0/guides/endpoints/bulk-data-insertion/file-format/#required-columns).
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Adobe Campaign Classic
**URL:** https://hightouch.com/docs/destinations/adobe-campaign-classic
**Description:** Adobe Campaign pulls together cross-channel customer data into a single view and then puts it to work to create personalized cross-channel campaigns that meet customers where they’re at.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|----------------|--------------------------------------------------------|------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Recipients** | Sync data from any source to Adobe Campaign recipients | Upsert, Update, Insert | [Recipients docs](https://experienceleague.adobe.com/en/docs/campaign-classic/using/configuring-campaign-classic/api/data-oriented-apis#write---writecollection--xtk-session-) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Adobe Campaign Classic
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Adobe Campaign Classic** and click **Continue**. You can then authenticate Hightouch to **Adobe Campaign Classic**.
Enter the following fields into Hightouch:
- **Server URL**: Enter your server URL.
- **Client ID**: Enter your client ID.
- **Client secret**: Enter your client secret.
## Sync configuration
Once you've set up your Adobe Campaign Classic destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Adobe Campaign Classic destination you want to sync to.
### Syncing recipients
Sync data from any source to Adobe Campaign recipients.
#### Record matching
You can match rows from your model to recipients in Adobe Campaign Classic on any column in your model and any field in Adobe Campaign Classic. Ensure the data types of the model column and Adobe Campaign Classic field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Adobe Campaign Classic automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync recipient fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to any recipient attribute.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
|----------------|-------------------------------------------------------------------------------|
| **Do nothing** | Keep the recipient in Adobe Campaign Classic with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the recipient in Adobe Campaign Classic |
| **Delete** | Delete the synced recipients from Adobe Campaign Classic |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Adobe Experience Platform
**URL:** https://hightouch.com/docs/destinations/adobe-experience-platform
**Description:** Deliver personalized experiences at scale requires a centralized and connected data foundation with the Adobe Experience Platform.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| -------------------------------------------- | -------------------------------------------- | -------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| **Datasets** | Sync data from any source to dataset records | Upsert, Insert | [Datasets docs](https://developer.adobe.com/experience-platform-apis/references/batch-ingestion/#operation/createBatch) |
| **Adobe Journey Optimizer campaign trigger** | Trigger campaigns in Adobe Journey Optimizer | -- | [Campaign trigger docs](https://experienceleague.adobe.com/en/docs/journey-optimizer/using/campaigns/api-triggered-campaigns/api-triggered-campaigns) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Adobe Experience Platform
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Adobe Experience Platform** and click **Continue**. You can then authenticate Hightouch to **Adobe Experience Platform**.
Enter the following fields into Hightouch:
- **Organization ID**
- **Sandbox name**
- **Client ID**
- **Client secret**
For more information on how to obtain these values, refer to [Adobe's docs](https://experienceleague.adobe.com/en/docs/experience-platform/landing/platform-apis/api-authentication).
## Sync configuration
Once you've set up your Adobe Experience Platform destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Adobe Experience Platform destination you want to sync to.
### Syncing datasets
Sync data from any source to dataset records. Upsert mode is only available if your dataset has unified profile and upsert enabled. Refer to [Adobe's docs](https://experienceleague.adobe.com/en/docs/experience-platform/catalog/datasets/enable-upsert) to enable these features.
#### Record matching
You can match rows from your model to datasets in Adobe Experience Platform on any column in your model and any field in Adobe Experience Platform. Ensure the data types of the model column and Adobe Experience Platform field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Adobe Experience Platform automatically generates an
identifier for every new record synced, so there is no need to match an
existing record.
#### Field mapping
Hightouch lets you sync dataset fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the fields defined in the schema of your dataset.
Ensure your model's columns have the same data types as the fields you want to sync to.
### Trigger campaigns via Adobe Journey Optimizer
Trigger push, sms, or emails to your Adobe Journey Optimizer campaigns.
#### Campaign ID
Enter the ID of the campaign you want to trigger. Each sync can trigger sends to a single campaign.
#### Field mapping
You must specify which AEP profile namespace (e.g. `email`) you want to use to map your recipients in AEP and map the
`userId` to your source column to identify that namespace. You may also include other user attributes in the `profile` field.
You can add dynamic content to your campaign through the `context` field.
The `context` field is an object of property-value pairs (e.g. `{ "subject": "Subject line" }`).
To use dynamic content in your campaign properly, you need to set up the
template to reference the context object in Adobe Journey Optimizer. Refer to
this
[tutorial](https://experienceleague.adobe.com/en/docs/journey-optimizer-learn/tutorials/create-campaigns/api-triggered-campaigns)
on how to do so.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Adobe Target
**URL:** https://hightouch.com/docs/destinations/adobe-target
**Description:** Sync customer segments to Adobe Target
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|--------------|----------------------------------------------|----------------------|-----------------------------------------------------------------------|
| **Profiles** | Create and update visitor profile attributes | Upsert | [Profiles](https://experienceleague.adobe.com/en/docs/target-dev/developer/api/profile-apis/profile-bulk-api) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Getting started
To generate the required authentication credentials to connect Adobe Target to Hightouch, follow these instructions:
1. Go to your [Adobe admin console](https://developer.adobe.com/console/home).
2. Select **Create new project** > **Add API** > **Adobe Target** and click **Next**.
3. Select the **OAuth Server-to-Server** authentication method.
4. Once the authentication method has been created, select the credentials.
5. Click **Retrieve client secret**.
## Connect to Adobe Target
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button.
Select **Adobe Target** and click **Continue**.
You can then authenticate Hightouch to **Adobe Target** by entering the following required fields into Hightouch:
- **Client code**: Your client code can be found in the [Adobe Experience Cloud](https://experience.adobe.com/) under **Target** > **Administration** > **Implementation**.
- **Tenant name**: Tenant name can be found in the URL when you log into Experience Cloud (`https://experience.adobe.com/#/@/target/setup/implementation`).
- **Client ID**: This is created along with the client secret from the steps above
- **Client secret**
Make sure to access the **Profile API** slide in your Adobe Target settings and enable the **Require Authentication** toggle.
Then click **Generate New Profile Authentication Token**.
You don't need to use the profile authentication token when setting up an Adobe Target destination in Hightouch, but generating a new token is required by Adobe.
You can find more details about this in [Adobe Target's documentation](https://experienceleague.adobe.com/docs/target-dev/developer/implementation/methods/profile-api-settings.html).
## Sync configuration
Once you've set up your Adobe Target destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Adobe Target destination you want to sync to.
### Syncing profiles
Syncing to Adobe Target's [Bulk Profile API](https://experienceleague.adobe.com/en/docs/target-dev/developer/api/profile-apis/profile-bulk-api) lets you manage visitor profile attributes with data typically stored in cookies or third-party apps.
#### Syncing to audiences
Audiences in Adobe Target are condition-based, so syncing profiles to audiences is handled by setting a `__hightouch_audience_{model name}` attribute to match on. You can only choose to create a new audience, and the condition of the audience will be set to include all user profiles whose `__hightouch_audience_{model name}` attribute is `true`.
#### Record matching
To match rows from your model to profiles in Adobe Target, you need to select a model column and corresponding Adobe Target field. You can match on any of the following Adobe Target fields:
- thirdPartyId (recommended)
- pcID
#### Field mapping
You can map data from any of your model columns to fields in Adobe Target. Ensure the data type of your model column matches the data type of the field you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the profile in Target |
| **Clear** | Keep the profile in Target and clear fields managed by Hightouch |
## Tips and troubleshooting
#### Non-unique IDs
Profiles can be matched from your warehouse to your Adobe Target destination using the pcID or a third party ID. If you selected a model column that isn't unique, the incorrect profile may be updated.
### Common errors
#### Failed to authenticate to Adobe Target: 400 - Invalid request: authentication is disabled for current client
As explained in the [Connect to Adobe Target](#connect-to-adobe-target) section, make sure to access the **Profile API** slide in your Adobe Target settings and enable the **Require Authentication** toggle.
Then click **Generate New Profile Authentication Token**.
You don't need to use the profile authentication token when setting up an Adobe Target destination in Hightouch, but generating a new token is required by Adobe.
You can find more details about this in [Adobe Target's documentation](https://experienceleague.adobe.com/docs/target-dev/developer/implementation/methods/profile-api-settings.html).
### Live debugger
### Sync alerts
---
## Agillic
**URL:** https://hightouch.com/docs/destinations/agillic
**Description:** Agillic is the trusted Nordic marketing automation platform, enabling personalisation that performs with scalability, operational efficiency, and full GDPR compliance.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
----------------|------------------------------------------------------------------|----------------------|---------------------------------------------------------------------------------------------
**Recipients** | Sync data from any source to Agillic recipients | Upsert | [Recipients docs](https://developers.agillic.com/getting-started-with-apis/recipients-api/)
**One-to-Many** | Sync data from any source to Agillic one-to-many tables | Upsert | [One-to-Many docs](https://developers.agillic.com/getting-started-with-apis/recipients-api/)
**Events** | Sync data from any source to achieve recipient events in Agillic | Insert | [Events docs](https://developers.agillic.com/getting-started-with-apis/recipients-api/)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Agillic
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Agillic** and click **Continue**. You can then authenticate Hightouch to **Agillic**.
Enter the following fields into Hightouch:
- **Client ID**
- **Client secret**
## Sync configuration
Once you've set up your Agillic destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Agillic destination you want to sync to.
### Syncing recipients
Sync data from any source to Agillic recipients.
#### Record matching
To match rows from your model to recipients in Agillic, you need to select the model column that contains values that match any unique recipient field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync recipient fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default recipient fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|---------------------------------------------------------
**Do nothing** | Keep the recipient in Agillic with all its synced fields
**Delete** | Delete the synced recipients from Agillic
### Syncing one-to-many
Sync data from any source to Agillic one-to-many tables.
#### Record matching
To match rows from your model to one-to-many records in Agillic, you need to select a model column that contains a unique recipient field
and a unique field for the one-to-many table you are syncing to.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync one-to-many fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default one-to-many fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
### Syncing events
Sync data from any source to achieve recipient events in Agillic.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Airship
**URL:** https://hightouch.com/docs/destinations/airship
**Description:** Build better campaigns on Airship with up-to-date customer data from your data warehouse
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ---------------------- | ------------------------------------------------------------- | -------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| **Named users** | Sync tags and attributes to named users in Airship | Upsert, Update | [Named users docs](https://docs.airship.com/api/ua/?openapi=http#operation-api-named_users-named_user_id-post) |
| **Subscription lists** | Subscribe and unsubscribe named users from subscription lists | Add, Remove | [Subscription lists docs](https://docs.airship.com/api/ua/?openapi=http#operation-api-named_users-scoped-named_user_id-post) |
| **Push** | Send push notifications and rich messages across channels | Insert | [Push endpoint docs](https://docs.airship.com/developer/rest-api/ua/operations/push/#sendpush) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Airship
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Airship** and click **Continue**.
You can then authenticate Hightouch to Airship using either **basic authentication** or **bearer authentication**.
For both types, you need to enter your Airship **Cloud Site**. See Airship's [docs](https://docs.airship.com/api/ua/?openapi=http#servers) for more info.
Then, for basic authentication, enter these fields:
- **App Key**: This is an Airship-generated string identifying your application. You can find it in your Airship dashboard.
- **Master Secret**: Airship-generated string used for server-to-server API access. You can find it in your Airship dashboard.
Follow the instructions in this [article](https://support.airship.com/hc/en-us/articles/213491663-How-do-I-find-my-App-Key-App-Secret-or-Master-Secret-#:~:text=Open%20the%20app%20in%20the,right%20side%20of%20the%20screen.)
to retrieve your **App Key** and **Master Secret**
For bearer authentication:
- **Access Token**: This is an access token used to grant control permissions to the Airship API.
Follow the instructions in this [article](https://docs.airship.com/guides/messaging/user-guide/project/bearer-tokens/) to retrieve your **Access token**.
## Sync configuration
Once you've set up your Airship destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Airship destination you want to sync to.
### Syncing named users
Sync tags and attributes to named users in Airship. Begin by selecting **Object** as your sync type.
#### Record matching
To match rows from your model to named users in Airship, you need to select the model column that contains values that match the **Named User ID** field.
#### Field mapping
You can sync columns from your model to Airship attributes and tags.
Hightouch provides a dropdown for [default attributes](https://docs.airship.com/guides/messaging/user-guide/audience/segmentation/attributes/reference/#default-attributes) and [predefined attributes](https://docs.airship.com/guides/messaging/user-guide/audience/segmentation/attributes/reference/#predefined-attributes).
If you want to map custom attributes, you have to manually enter the [attribute ID](https://docs.airship.com/guides/messaging/user-guide/audience/segmentation/attributes/about/#attribute-components-and-variations), _not attribute name_ in the mapper.
Although [predefined
attributes](https://docs.airship.com/guides/messaging/user-guide/audience/segmentation/attributes/reference/#predefined-attributes)
are avaliable in the dropdown by default, you must first define them in Airship
for them to be valid. Airship skips invalid attributes by default.
Hightouch allows you to sync values to existing tag groups in Airship. Enter the **Group key** in the right side of the tags mappings section to map values to an existing tag group.
See Airship's [docs](https://docs.airship.com/guides/messaging/user-guide/audience/segmentation/tags/#creating-custom-tag-groups) for more information on tag groups.
{/* */}
You can map `string` or `array` values to tags.
If you provide a string, Hightouch automatically transforms it into an `array` using commas as the delimiters for different elements.
For example, Hightouch transforms the string `"dogs, cats, parrots"` into this array: `["dogs, "cats", "parrots"]`.
{/* */}
### Syncing subscription lists
Syncs can subscribe and unsubscribe named users from subscription lists.
#### Record matching
To match rows from your model to named users in Airship, you need to select the model column that contains values that match the **Named User ID** field.
#### Field mapping
You can map data from any of your model columns to fields in Airship. Airship requires the **Scopes** field, so you must map it to complete your configuration. Refer to Airship's [scopes docs](https://docs.airship.com/api/ua/?openapi=http#schemas-scopes) to see valid values.
Ensure the data types of your model columns match the data types of the fields you want to sync to.
### Sending push notifications
Send push notifications and rich messages to your users across multiple channels including iOS, Android, email, and SMS.
#### Field mapping
Configure your push notifications by mapping model columns to the following fields:
- **Audience**: Specify recipients using channels, tags, groups, or other audience selectors. This field is required.
- **Notification**: The message content to be sent. This field is required.
- **Device Types**: Target platform(s) for the notification (e.g., iOS, Android, email). This field is required.
- **Expiry** (optional): Expiration time for time-sensitive messages to eliminate the need for manual removal.
For detailed information about payload structure and additional options, refer to Airship's [Push API documentation](https://docs.airship.com/developer/rest-api/ua/operations/push/#sendpush).
## Tip and troubleshooting
### Common errors
### Live debugger
### Sync alerts
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Airtable
**URL:** https://hightouch.com/docs/destinations/airtable
**Description:** Sync your database or data warehouse to your Airtable bases for DataOps, CRM, and much more
**Section:** Destinations
## Supported syncing
|Sync Type |Description |Supported Sync Modes |
|--------------|----------------------------------------------------------------------------------------------|---------------------|
|Objects | Insert new objects without updating others or push new objects and update fields that change |Insert, Upsert |
|Object fields | Update particular fields on existing objects without adding new objects |Update |
## Connect to Airtable
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Airtable** and click **Continue**. You can then authenticate Hightouch to Airtable using an Airtable personal access token.
### Create Airtable personal access token
Navigate to your [Airtable Personal access tokens page](https://airtable.com/create/tokens) and click **Create token**.
Give your token a descriptive name, for example, "Hightouch Integration Key," and grant your token the following scopes:
- `data.records:read`
- `data.records:write`
- `schema.bases:read`
- `schema.bases:write`
Grant your token access to the bases and workspaces you need Hightouch to access, and then copy your token to use it in Hightouch.
## Sync configuration
Once you've set up your Airtable destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Airtable destination you want to sync to.
Choose the **Airtable base** and **table** you want to sync to using the two drop-down lists.
Then select the [mode](/syncs/types-and-modes) for how to alter Airtable objects:
- **Insert**: pushes new objects to Airtable. It doesn't update any objects.
- **Upsert**: pushes new objects to Airtable and updates fields that change in your warehouse.
- **Update**: updates particular fields on existing objects in Airtable. It doesn't add new objects.
### Record matching
If you are using **Upsert** or **Update** mode, you must select a record matching field.
You can match records from your source to your Airtable workspace by the table's primary key or any other unique fields in the table.
You can choose the field you would like to use from the dropdown.
You should match the record using a unique identifier with a `string` type, otherwise the sync may not update the intended records properly.
### Field mapping
You can sync columns from your source to any of your Airtable properties. The Airtable API performs best-effort
automatic data conversion from string values. You can choose the field names from the drop-down lists.
When syncing data to [linked](https://support.airtable.com/docs/linking-records-in-airtable) or [lookup](https://support.airtable.com/docs/lookup-field-overview) Airtable fields, make sure to set your model column to [`string`](/models/data-types-casting#data-types) type. Otherwise, the values synced to these fields won't appear in Airtable.
## Tips and troubleshooting
### Common errors
#### Duplicated records
Duplicated records in Airtable may occur if a data type mismatch between the [model column](/syncs/record-matching#model-column-selection) and the Airtable field selected for [record matching](#record-matching) occurs.
Hightouch [automatically stringifies](/models/data-types-casting#view-model-data-types) the model column you select for your [primary key](/models/creating-models#unique-primary-key-requirement). If you choose your model's primary key for record matching, you need to select a `string` type Airtable field for record matching as well. If you select a `number` type Airtable field, duplicate records could occur because, for example, `"1234"` doesn't match `1234`.
If you want to use the primary key column for record matching, but need to match it to a `number` type Airtable field, you can use [SQL aliasing](/models/data-types-casting#sql-aliasing) to create a new column. The new column is a "copy" of the primary key column, but because Hightouch won't automatically stringify it, you can cast it to [`number` type](/models/data-types-casting#data-types) and use it for record matching.
### Live debugger
### Sync alerts
---
## Algolia
**URL:** https://hightouch.com/docs/destinations/algolia
**Description:** Build, power, and optimize digital experiences
**Section:** Destinations
## Overview
You want to provide relevant search and discovery results and doing so heavily relies on data in your Algolia account to be accurate and fresh. By automatically indexing records and events from your data warehouse into Algolia, you no longer need to worry about data consistency because data is no longer flowing in from multiple sources in different formats, and you can focus on building world-class customer experiences.
## Supported syncing
Type | Description | Supported Sync Modes
---------------------|----------------------------------------------------------|-----------------------
**Index records** | Sync data from any source to indices in Algolia | Upsert, Update, Insert
**Insight events** | Sync data from any source to insight events in Algolia | Insert
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Algolia
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination**
button. Select **Algolia** and click **Continue**. You can then authenticate Hightouch to **Algolia** with an API key.
Enter the following required fields into Hightouch:
- **Application ID**
- **API Key**
Follow [these instructions](https://www.algolia.com/doc/guides/security/api-keys/#creating-and-managing-api-keys) to create your API key and provide at least these [permissions](https://www.algolia.com/doc/guides/security/api-keys/#rights-and-restrictions):
Permissions | Required?
---------------------|------------
**listIndexes** | YES
**addObject** | YES
**deleteObject** | If you choose to [delete records](#delete-behavior)
## Sync configuration
Once you've set up your Algolia destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and
the Algolia destination you want to sync to.
### Indices
An [Algolia index](https://support.algolia.com/hc/en-us/articles/4406981910289-What-is-an-index-) is the place where the data used by a search engine is stored. You can use Hightouch to update individual records in an index.
#### Record matching
You must match rows in your model with objects in Algolia on the [**Object ID** field](https://www.algolia.com/doc/guides/sending-and-managing-data/send-and-update-your-data/#using-unique-object-identifiers).
In **Insert** mode, Algolia automatically generates a **Object ID** for every new record synced, so there is no need to match an existing record.
#### Field mapping
Records in Algolia index don't need to follow a schema so you can sync any custom fields to them.
Visit Algolia's docs for more information about [Algolia records](https://www.algolia.com/doc/guides/sending-and-managing-data/prepare-your-data/#algolia-records).
#### Delete behavior
**Clear** is only available for update mode. **Delete** is only available for upsert mode. All sync modes use the **Do nothing** behavior by default.
Behavior | Description
-------------- | -------------------------------------------------------------------------------------------------
**Do nothing** | Keep the customer record in Algolia with all its synced fields
**Clear** | Unset all the mapped fields from Algolia
**Delete** | Delete the synced record from your Algolia index. Your [API key needs the **deleteObject** ACL](#connect-to-algolia) to enable this
### Insight events
Insights events lets you capture click, conversion, and view events to help you understand how your users interact with your digital experience.
#### Field mapping
When syncing insight events to Algolia, you must provide the following fields:
- **Event Name**: Name of the event. Typically the action performed.
- **Event Type**: Type of the event. The allowed values are `"click"`, `"conversion"`, or `"view"`.
- **Index**: Name of the targeted index.
- **User Token**: A user identifier. Depending if the user is logged-in or not, several strategies can be used from a `sessionId` to another technical identifier. You should always send pseudonymous or anonymous userTokens.
YOu can also provide these optional fields:
- **Timestamp**: Time of the event expressed in milliseconds since the [Unix epoch](https://en.wikipedia.org/wiki/Unix_time).
- **Query ID**: Algolia `queryID`. This is required when an event is tied to a search.
- **Object IDs**: An array of index `objectIDs`. Limited to 20 objects. An event can't have both `objectIDs` and filters at the same time.
- **Filters**: An array of filters. Limited to 10 filters. An event can't have both `objectIDs` and filters at the same time.
- **Positions**: Position of the click in the list of Algolia search results. This field is required if a `queryID` is provided. One position must be provided for each `objectID`.
When an event is tied to an Algolia search, it must also provide a queryID. If that event is a click, their absolute positions should also be passed. Consult [Algolia's documentation](https://www.algolia.com/doc/api-reference/api-methods/send-events/#parameters) for more information on event fields.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## AlloyDB
**URL:** https://hightouch.com/docs/destinations/alloydb
**Description:** AlloyDB combines the best of Google with one of the most popular open-source database engines, PostgreSQL, for superior performance, scale, and availability
**Section:** Destinations
## Setup
### Networking
AlloyDB supports network connectivity through private, internal IP addresses configured for private services access.
To connect to AlloyDB from Hightouch, configure an SSH connection tunnel to go through a bastion host in your VPC.
Read about more options here: [https://cloud.google.com/alloydb/docs/connection-overview](https://cloud.google.com/alloydb/docs/connection-overview)
### SSH tunneling
Hightouch can connect directly to AlloyDB over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your AlloyDB instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
### Required permissions
The user credentials must have one of the following IAM roles in the Cloud project you are using. If you don't have any of these roles, contact your Organization Administrator to request access.
- `roles/alloydb.admin` (the AlloyDB Admin predefined IAM role)
- `roles/owner` (the Owner basic IAM role)
- `roles/editor` (the Editor basic IAM role)
To create private services access configuration, you also must have these IAM permissions:
- `compute.networks.list`
- `compute.addresses.create`
- `compute.addresses.list`
- `servicenetworking.services.addPeering`
To ensure your credentials are correct, click **Test connection**. This confirms
if Hightouch is able to properly connect to your database by running a simple `SELECT` query.
## Syncing
Hightouch supports **Upsert** mode, with the option to delete removed rows, and **Insert** mode
using the `COPY FROM` statement.
### Record matching
Hightouch requires a unique identifier in the table you are syncing to in order to add, remove, and update rows.
The main identifier column in your sync configuration must match on a `UNIQUE` or `PRIMARY KEY` column within your database.
You should see those columns as available options under the records matching section. If there are no fields in the dropdown,
you need to add a unique column type to your table.
### Column types
Hightouch works out of the box with all standard column types, including:
- `BIGINT`
- `INT`
- `TEXT`
- `VARCHAR`
- `TIMESTAMPTZ`
- `BOOLEAN`
- `DECIMAL`
If you see type errors, it may be because your SQL query is producing the wrong format.
Hightouch doesn't support arrays of `USER-DEFINED` types that are JSON
objects. We do support arrays of regular JSON objects (JSON or JSONB).
right now.
### Batch size
You can tune the number of rows that Hightouch upserts or inserts per query based on your needs and database threshold.
The default is 1k rows per batch.
If you want to improve the sync's speed performance, you can increase
the batch size to a higher number like 10k. However, keep in mind that AlloyDB will fail the entire batch
of rows if it detects any erroneous row. If you suspect that you will have many bad rows, you should
not have a high batch size. Also make sure you account for your database's capacity when increasing the
batch size to avoid any locks.
### Insert mode
Hightouch supports insert mode through the `COPY FROM STDIN` AlloyDB statement. This will load the
queried rows into your table. However, AlloyDB will throw an error and reject the entire batch if any row
already exists or the row contains a primary key or unique value that already exists.
The [`COPY FROM`](https://www.postgresql.org/docs/current/sql-copy.html) statement should be faster than the regular statement we use for Upsert mode so we recommend
selecting Insert mode if you are only inserting rows and want to sync your data faster.
The `COPY FROM STDIN` statement should support the same column types as Upsert
mode, but the array of JSONB type (`jsonb[]`) column. Instead, you will need
to use a JSONB column type, which supports an array format.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Amazon Ads DSP and AMC
**URL:** https://hightouch.com/docs/destinations/amazon-ads
**Description:** Ensure that you never serve another irrelevant advertisement with Amazon Ads
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| ------------- | ---------------------------------------------------------------------------------------------- | -------------------- | -------------------------------------------------------------------------------------------- |
| [**Events**](#syncing-events) | Sync data from any source to Amazon Ads DSP as offline conversion events | Insert | [Events docs](https://advertising.amazon.com/API/docs/en-us/dsp-conversion-builder) |
| [**Audiences**](#syncing-audiences) | Sync first-party data and third-party data from any source to Amazon Ads DSP and AMC audiences | Add | [Audiences docs](https://advertising.amazon.com/API/docs/en-us/dsp-audiences/#tag/Audiences) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Amazon Ads
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Amazon Ads** and click **Continue**. You can then authenticate Hightouch to **Amazon Ads**.
For the **Authentication method**, select **Log in to Amazon Ads** and log into your Amazon Ads account. Once successful, you will be redirected back to Hightouch to enter an advertiser ID and a descriptive name for your destination to complete setup.
## Sync configuration
Once you've set up your Amazon Ads destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Amazon Ads destination you want to sync to.
### Syncing events
Sync data from any source to Amazon Ads as offline conversion events.
#### Create a conversion definition in Amazon Ads
Before syncing events, you need to create a conversion definition in Amazon Ads.
See the [Amazon Ads API documentation](https://advertising.amazon.com/API/docs/en-us/dsp-conversion-builder) for more information.
You can create a conversion definition using the Amazon Ads UI or API.
To create a conversion definition using the Amazon Ads API, use the endpoint documented [here](https://advertising.amazon.com/API/docs/en-us/dsp-conversion-builder#tag/Amazon-Conversion-Definitions/operation/dspAmazonCreateConversionDefinitions).
To create a conversion definition using the Amazon Ads UI, follow these steps:
1. [Sign in](https://advertising.amazon.com/en-us/sign-in) to Amazon Ads.
1. Navigate to **Amazon DSP > Campaign manager**.
1. Navigate to **Advertisers** and select an advertiser.
1. Navigate to the **Events manager** for that advertiser.
1. Go to the **Conversions** tab and click the `Add conversion` button.
After creating a conversion definition, you can select it in the sync configuration.
#### Record matching
For events, you can use any of the following fields as identifiers in Amazon Ads:
- **Email**
- **Phone**
- **City**
- **State**
- **Address**
- **Postal**
- **MAID**
- **First name**
- **Last name**
Please see the [Amazon Ads API documentation](https://advertising.amazon.com/API/docs/en-us/dsp-conversion-builder) for more information on the required fields for event syncing.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
### Syncing audiences
Sync data from any source to Amazon Ads audiences.
#### Audience type
You can sync to first-party data or third-party data based on the type of audience.
- **First-party data audience**: Sync hashed user data to audiences
- **Third-party data audience**: Sync external identifiers such as MAIDs and cookie IDs
#### Record matching
To match rows from your model to audience in Amazon Ads, you need to select a model column and corresponding Amazon Ads field. For third-party data audiences, you can match on any of the following fields:
- **Cookie ID**
- **Mobile Advertising ID**
- **External ID**
For first-party data audiences, you can match on any of the following fields:
- **Email**
- **Phone**
- **City**
- **State**
- **First name**
- **Last name**
- **Address**
- **External ID**
#### Handling PII and hashing
By default, Hightouch automatically hashes first-party user data fields before sending requests to Amazon Ads.
You can turn off this behavior in the sync configuration. When off, the data from the model should be appropriately normalized and hashed according to Amazon Ads' hashing requirements.
## Tips and troubleshooting
### Matched users count
Below only applies to the audiences sync type.
### Common errors
#### Invalid conversion definition ID
If you receive an error that the conversion definition ID is invalid, ensure that you have created a conversion definition in Amazon Ads and selected it in the sync configuration.
If you are using a column for the conversion definition ID, ensure that the column values match the numeric ID of the conversion definition in Amazon Ads, rather than its name.
### Live debugger
### Sync alerts
---
## Amazon EventBridge
**URL:** https://hightouch.com/docs/destinations/amazon-eventbridge
**Description:** Amazon EventBridge is a serverless, fully managed, and scalable event bus that enables integrations between AWS services, Software as a services (SaaS), and your applications.
**Section:** Destinations
## Overview
Hightouch integrates directly with Amazon EventBridge to support high-throughput, distributed, or asynchronous workloads, letting you build a custom connector to your internal systems.
This destination was designed to be as flexible as possible. You can publish into different buses, have custom sources attributes and detail types for each message trigger allowing you to enrich your events.
## Getting started
### Connect to your AWS account
When setting up the Amazon EventBridge destination for the first time, you need to enter your **AWS Credentials** to give Hightouch access to your AWS account. Hightouch needs permission to send events to Amazon EventBridge on your behalf.
Hightouch believes in the principle of least privilege. We ask for no more permissions than necessary. For the Amazon EventBridge destination, Hightouch requires `events:PutEvents` and `events:ListEventBuses`.
When configuring your credentials in Hightouch, you have two options:
- Setting up a cross-account role (recommended)
- Providing an access key
#### Set up cross-account role
Cross-account roles are the most secure method for granting Hightouch access to Amazon EventBridge in your AWS account.
You need the **Account ID** and **External ID** in the Hightouch UI to set up a cross-account role.
1. In the **AWS Console**, navigate to **IAM** → **Roles**.
1. Click **Create Role**.
1. Under **Trusted entity type**, choose **AWS account**.
1. Select **Another AWS account** and also click **Require external ID**.
1. Copy/paste the **Account ID** from Hightouch into the **Account ID** field in AWS.
1. Copy/paste the **External ID** from Hightouch into the **External ID** field in AWS.
1. On the **Add permissions** screen, proceed to attach permissions to the role using any policy that includes at least `events:PutEvents` and `events:ListEventBuses`.
1. Copy/paste the **Role ARN** from AWS into Hightouch. Click **Create** to finish setting up your cross-account role.
#### Provide access key
If you don't want to create a cross-account role, you can create a regular IAM user and share a **Access Key ID** and **Secret Access Key** with Hightouch.
Be sure to attach the user to a permission policy that include `events:PutEvents` and `events:ListEventBuses` for the specific resource you want to use via a Hightouch sync.
An example policy looks like this:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "events:PutEvents",
"Resource": ["YOUR_EVENTBRIDGE_BUS_ARN_1", "YOUR_EVENTBRIDGE_BUS_ARN_2", "YOUR_EVENTBRIDGE_BUS_ARN_3"]
},
{
"Effect": "Allow",
"Action": "events:ListEventBuses",
"Resource": "*"
}
]
}
```
### Select your AWS region
In this step, you specify which region has the resources you want to sync to.
## Syncing data
Once you've authenticated your AWS account in Hightouch and selected an AWS Region, you've completed setup for an Amazon EventBridge destination in your Hightouch workspace. The next step is to configure a sync that send events whenever rows are added, changed, or removed in your model.
### Configure your events trigger
Hightouch monitors your data model for added, changed, and removed rows. In this step, you specify which of these events should trigger message publishing.
### Choose your event bus
In this step, you choose to send the events to a specific event bus. Hightouch allows you to sync to existing buses in your AWS EventBridge.
## Customize your source and detail type attributes
In this step, you tell Hightouch what value to put for the `source` and `detail-type` attribute in EventBridge for the event. More information can be found here on what these fields mean can be found in the [AWS docs](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-events.html).
#### Use JSON editor
With the JSON editor, you can compose any JSON object using the [Liquid template language](https://shopify.github.io/liquid/). This is particularly useful for complex message data bodies containing nested objects and arrays, which can sometimes be difficult to model entirely in SQL.
Suppose your data model looks like this:
| full_name | age | email_address | phone_number |
| --------- | --- | ------------------ | ------------ |
| John Doe | 30 | `john@example.com` | +14158675309 |
And you want your message data like this:
```json
{
"name": "John Doe",
"age": 30,
"contact_info": [
{
"type": "email",
"value": "john@example.com"
},
{
"type": "phone",
"value": "+14158675309"
}
]
}
```
Your Liquid template should look like this:
```txt
{
"name": "{{row.full_name}}",
"age": {{row.age}},
"contact_info": [
{
"type": "email",
"value": "{{row.email_address}}"
},
{
"type": "phone",
"value": "{{row.phone_number}}"
}
]
}
```
This makes it so you can reference any column using the syntax `{{row.column_name}}`. You can also use advanced Liquid features to incorporate control flow and loops into your dynamic message data.
When injecting strings into your JSON request body, be sure to surround the
Liquid tag in double quotes.
#### Use one column from model
If you're already storing JSON data in your source, or if you have the ability to construct a JSON object using SQL, you can select one column in your model that already contains the full message data.
This setting is commonly used when syncing web events that have already been collected and stored as JSON objects in your database.
#### Use multiple columns from model
For the simplest use cases, Hightouch can construct a JSON object with key/value pairs based on multiple columns in your model.
Suppose your model looks like this:
| email | first_name | last_name |
| ----------------------- | ---------- | --------- |
| `alice.doe@example.com` | Alice | Doe |
| `bob.doe@example.com` | Bob | Doe |
| `carol.doe@example.com` | Carol | Doe |
The field mapping in the preceding screenshot would generate the following message data for the first row:
```json
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "alice.doe@example.com"
}
```
You can use the field mapper to rename fields. For example, `first_name` can
be mapped to `customer_first_name`.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows may require performing a backfill of all rows during the initial sync. For other use cases, you might only want to send messages in response to future data changes.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Amazon Kinesis
**URL:** https://hightouch.com/docs/destinations/amazon-kinesis
**Description:** Ingest and process your data via a data stream
**Section:** Destinations
## Overview
Hightouch integrates directly with Amazon Kinesis to support high-throughput, distributed, or asynchronous workloads, letting you build a custom connector to your internal systems.
## Getting started
### Connect to your AWS account
When setting up the Amazon Kinesis destination for the first time, you need to enter your **AWS Credentials** to give Hightouch access to your AWS account. Hightouch needs permission to send streaming data to Amazon Kinesis on your behalf.
Hightouch believes in the principle of least privilege. We ask for no more permissions than necessary. For the Amazon Kinesis destination, Hightouch requires `kinesis:PutRecords` and `kinesis:ListStreams`.
When configuring your credentials in Hightouch, you have two options:
- Setting up a cross-account role (recommended)
- Providing an access key
#### Set up cross-account role
Cross-account roles are the most secure method for granting Hightouch access to Amazon Kinesis in your AWS account.
You need the **Account ID** and **External ID** in the Hightouch UI to set up a cross-account role.
1. In the **AWS Console**, navigate to **IAM** → **Roles**.
1. Click **Create Role**.
1. Under **Trusted entity type**, choose **AWS account**.
1. Select **Another AWS account** and also click **Require external ID**.
1. Copy/paste the **Account ID** from Hightouch into the **Account ID** field in AWS.
1. Copy/paste the **External ID** from Hightouch into the **External ID** field in AWS.
1. On the **Add permissions** screen, proceed to attach permissions to the role using any policy that includes at least `kinesis:PutRecords` and `kinesis:ListStreams`. Then create the role.
1. Copy/paste the **Role ARN** from AWS into Hightouch. Click **Create** to finish setting up your cross-account role.
#### Provide access key
If you don't want to create a cross-account role, you can create a regular IAM user and share a **Access Key ID** and **Secret Access Key** with Hightouch.
Be sure to attach the user to a permission policy that includes `kinesis:PutRecords` and `kinesis:ListStreams` for the specific resource you want to use via a Hightouch sync.
An example policy looks like this:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["kinesis:PutRecords", "kinesis:ListStreams"],
"Resource": [
"YOUR_KINESIS_ARN_1",
"YOUR_KINESIS_ARN_2",
"YOUR_KINESIS_ARN_3"
]
}
]
}
```
### Select your AWS region
In this step, you specify which region has the resources you want to sync to.
## Syncing data
Once you've authenticated your AWS account in Hightouch and selected an AWS Region, you've completed setup for a Amazon Kinesis destination in your Hightouch workspace. The next step is to configure a sync that send messages whenever rows are added, changed, or removed in your model.
### Configure your events trigger
Hightouch monitors your data model for added, changed, and removed rows. In this step, you specify which of these events should trigger message publishing.
### Choose your data stream
In this step, you choose which data stream to send messages to. Hightouch allows you to sync to existing data streams that are already in your Amazon Kinesis.
### Customize your streaming data
In this step, you tell Hightouch how to build the JSON streaming data object using data from your model.
#### Configure partition key
This field is sent along with the streaming data and determines which shard in the stream the data record is assigned to. Amazon Kinesis Data Streams uses the partition key as input to a hash function that maps the partition key and associated data to a specific shard. Specifically, an MD5 hash function is used to map partition keys to 128-bit integer values and to map associated data records to shards. As a result of this hashing mechanism, all data records with the same partition key map to the same shard within the stream.
This destination offers three methods of composing a JSON object:
- [Using a JSON editor](#use-json-editor)
- [Using one column from a model](#use-one-column-from-model)
- [Using multiple columns from a model](#use-multiple-columns-from-model)
#### Use JSON editor
With the JSON editor, you can compose any JSON object using the [Liquid template language](https://shopify.github.io/liquid/). This is particularly useful for complex streaming data bodies containing nested objects and arrays, which can sometimes be difficult to model entirely in SQL.
Suppose your data model looks like this:
| full_name | age | email_address | phone_number |
| --------- | --- | ------------------ | ------------ |
| John Doe | 30 | `john@example.com` | +14158675309 |
And you want your streaming data like this:
```json
{
"name": "John Doe",
"age": 30,
"contact_info": [
{
"type": "email",
"value": "john@example.com"
},
{
"type": "phone",
"value": "+14158675309"
}
]
}
```
Your Liquid template should look like this:
```txt
{
"name": "{{row.full_name}}",
"age": {{row.age}},
"contact_info": [
{
"type": "email",
"value": "{{row.email_address}}"
},
{
"type": "phone",
"value": "{{row.phone_number}}"
}
]
}
```
This makes it so you can reference any column using the syntax `{{row.column_name}}`. You can also use advanced Liquid features to incorporate control flow and loops into your dynamic streaming data.
When injecting strings into your JSON body, be sure to surround the Liquid tag
in double quotes.
#### Use one column from model
If you're already storing JSON data in your source, or if you have the ability to construct a JSON object using SQL, you can select one column in your model that already contains the full streaming data.
This setting is commonly used when syncing web events that have already been collected and stored as JSON objects in your database.
#### Use multiple columns from model
For the simplest use cases, Hightouch can construct a JSON object with key/value pairs based on multiple columns in your model.
Suppose your model looks like this:
| email | first_name | last_name |
| ----------------------- | ---------- | --------- |
| `alice.doe@example.com` | Alice | Doe |
| `bob.doe@example.com` | Bob | Doe |
| `carol.doe@example.com` | Carol | Doe |
The field mapping in the screenshot above would generate the following streaming data for the first row:
```json
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "alice.doe@example.com"
}
```
You can use the field mapper to rename fields. For example, `first_name` can
be mapped to `customer_first_name`.
### Configure optional message properties
Along with your row data in JSON format, you can also optionally include an explicit hash key to explicitly determine the shard the data record is assigned to.
##### `ExplicitHashKey`
This is a string field that should be a hash value and will override the partition key hash.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows may require performing a backfill of all rows during the initial sync. For other use cases, you might only want to send messages in response to future data changes.
## Tips and troubleshooting
### Common errors
#### ProvisionedThroughputExceededException: Rate exceeded for shard.
This error refers to Amazon Kinesis’ [quotas and limits](https://docs.aws.amazon.com/streams/latest/dev/service-sizes-and-limits.html). Manual intervention isn't required since our sync engine [retries rejected rows](/syncs/retries) during the following sync run.
### Live debugger
### Sync alerts
---
## Amazon SQS
**URL:** https://hightouch.com/docs/destinations/amazon-sqs
**Description:** Connect to your internal microservices via a message queue
**Section:** Destinations
## Overview
Hightouch integrates directly with Amazon SQS to support high-throughput, distributed, or asynchronous workloads, letting you build a custom connector to your internal systems.
This destination was designed to be as flexible as possible. You can publish into different queues including standard and FIFO queues for each message trigger and define custom ordering keys and metadata fields to enrich your messages.
## Getting started
### Connect to your AWS account
When setting up the Amazon SQS destination for the first time, you need to enter your **AWS Credentials** to give Hightouch access to your AWS account. Hightouch needs permission to send messages to Amazon SQS on your behalf.
Hightouch believes in the principle of least privilege. We ask for no more permissions than necessary. For the Amazon SQS destination, Hightouch requires `sqs:SendMessage` and `sqs:ListQueues`.
When configuring your credentials in Hightouch, you have two options:
- Setting up a cross-account role (recommended)
- Providing an access key
#### Set up cross-account role
Cross-account roles are the most secure method for granting Hightouch access to Amazon SQS in your AWS account.
You need the **Account ID** and **External ID** in the Hightouch UI to set up a cross-account role.
1. In the **AWS Console**, navigate to **IAM** → **Roles**.
1. Click **Create Role**.
1. Under **Trusted entity type**, choose **AWS account**.
1. Select **Another AWS account** and also click **Require external ID**.
1. Copy/paste the **Account ID** from Hightouch into the **Account ID** field in AWS.
1. Copy/paste the **External ID** from Hightouch into the **External ID** field in AWS.
1. On the **Add permissions** screen, proceed to attach permissions to the role using any policy that includes at least `sqs:SendMessage` and `sqs:ListQueues`. Then create the role.
1. Copy/paste the **Role ARN** from AWS into Hightouch. Click **Create** to finish setting up your cross-account role.
#### Provide access key
If you don't want to create a cross-account role, you can create a regular IAM user and share a **Access Key ID** and **Secret Access Key** with Hightouch.
Be sure to attach the user to a permission policy that includes `sqs:SendMessage` and `sqs:ListQueues` for the specific resource you want to use via a Hightouch sync.
An example policy looks like this:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["sqs:SendMessage", "sqs:ListQueues"],
"Resource": ["YOUR_SQS_ARN_1", "YOUR_SQS_ARN_2", "YOUR_SQS_ARN_3"]
}
]
}
```
### Select your AWS region
In this step, you specify which region has the resources you want to sync to.
## Syncing data
Once you've authenticated your AWS account in Hightouch and selected an AWS Region, you've completed setup for a Amazon SQS destination in your Hightouch workspace. The next step is to configure a sync that send messages whenever rows are added, changed, or removed in your model.
### Configure your events trigger
Hightouch monitors your data model for added, changed, and removed rows. In this step, you specify which of these events should trigger message publishing.
### Choose your queue
In this step, you choose to send the messages to either Standard or FIFO queues. Hightouch allows you to sync to existing queues that are already in your Amazon SQS.
### Customize your message
In this step, you tell Hightouch which column in your model contains the unique message ID and how to build the JSON message data object using data from your model.
#### Configure unique message ID
This unique ID field is sent along with the message data and is an identifier for a message in this batch used to communicate the result.
#### Configure message group ID (for FIFO queues only)
If you are syncing to a FIFO queue, you need to provide a message group ID to specify that a message belongs to a specific message group. Messages that belong to the same message group are processed in a FIFO manner however, messages in different message groups might be processed out of order.
You can toggle between a column in your model or a static value to set your message group ID.
This destination offers three methods of composing a JSON object:
- [Using a JSON editor](#use-json-editor)
- [Using one column from a model](#use-one-column-from-model)
- [Using multiple columns from a model](#use-multiple-columns-from-model)
#### Use JSON editor
With the JSON editor, you can compose any JSON object using the [Liquid template language](https://shopify.github.io/liquid/). This is particularly useful for complex message data bodies containing nested objects and arrays, which can sometimes be difficult to model entirely in SQL.
Suppose your data model looks like this:
| full_name | age | email_address | phone_number |
| --------- | --- | ------------------ | ------------ |
| John Doe | 30 | `john@example.com` | +14158675309 |
And you want your message data like this:
```json
{
"name": "John Doe",
"age": 30,
"contact_info": [
{
"type": "email",
"value": "john@example.com"
},
{
"type": "phone",
"value": "+14158675309"
}
]
}
```
Your Liquid template should look like this:
```txt
{
"name": "{{row.full_name}}",
"age": {{row.age}},
"contact_info": [
{
"type": "email",
"value": "{{row.email_address}}"
},
{
"type": "phone",
"value": "{{row.phone_number}}"
}
]
}
```
This makes it so you can reference any column using the syntax `{{row.column_name}}`. You can also use advanced Liquid features to incorporate control flow and loops into your dynamic message data.
When injecting strings into your JSON request body, be sure to surround the
Liquid tag in double quotes.
#### Use one column from model
If you're already storing JSON data in your source, or if you have the ability to construct a JSON object using SQL, you can select one column in your model that already contains the full message data.
This setting is commonly used when syncing web events that have already been collected and stored as JSON objects in your database.
#### Use multiple columns from model
For the simplest use cases, Hightouch can construct a JSON object with key/value pairs based on multiple columns in your model.
Suppose your model looks like this:
| email | first_name | last_name |
| ----------------------- | ---------- | --------- |
| `alice.doe@example.com` | Alice | Doe |
| `bob.doe@example.com` | Bob | Doe |
| `carol.doe@example.com` | Carol | Doe |
The field mapping in the preceding screenshot would generate the following message data for the first row:
```json
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "alice.doe@example.com"
}
```
You can use the field mapper to rename fields. For example, `first_name` can
be mapped to `customer_first_name`.
### Configure optional message properties
Along with your row data in JSON format, you can optionally include ordering keys to configure the order your queue receives your message and metadata fields as headers.
##### `DelaySeconds` (for standard queues only)
A number field that postpone the delivery of new messages to consumers for a number of seconds. Any messages that you send to the queue will remain invisible to consumers for the duration of the delay period. The maximum length of time you can delay for is 15 minutes.
##### `MessageDeduplicationId` (for FIFO queues only)
This is a string field. If a message with a particular Message Deduplication ID is sent successfully, subsequent messages with the same Message Deduplication ID are accepted successfully but aren't delivered. If you enabled Content-based deduplication when configuring your FIFO queue then you may optionally embed a deduplication message ID for your message. If you turned Content-based deduplication off, then this section is required for the messages to be published successfully.
##### `MessageAttributes`
This is an object containing key/value pairs of custom mapping fields. Values can be text or byte strings.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows may require performing a backfill of all rows during the initial sync. For other use cases, you might only want to send messages in response to future data changes.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Amobee
**URL:** https://hightouch.com/docs/destinations/amobee
**Description:** Transfer data from your data warehouse to Amobee via sFTP file drop.
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|Any data set|Sync data from a source to a tab-delimited Amobee file|Insert only|
This destination ignores the concept of additions, changes, removals. Syncs always send the full results of your model. For example, if your model has 1,000 records, your Amobee data file will be 1,000 records.
## Getting started
### Setup on Amobee
The initial setup process involves getting assigned a `market ID` from Amobee. If you don't know this ID, reach out to your Amobee representative.
## Setup on Hightouch
To get started with the Amobee destination on Hightouch, you must provide the market ID.
## Syncing data
In addition to the data file, Hightouch sends configuration and taxonomy files during the sync. Amobee's system picks up the files hourly.
Amobee also emails a success or failure notification to the address you provide in the sync configuration.
### Configuration file
To create the configuration file, enter the following information into the Hightouch during sync setup:
- Advertiser ID
- Email address to receive success notifications
- Name of the data file you want to send to Amobee
- Classification symbol
- Data file unique identifier ID type
- Data file start date
Hightouch uses this information to construct the configuration file included in the sync.
### Taxonomy file
You also need to upload a taxonomy file as part of the sync configuration form. An Amobee taxonomy file is an Excel document that helps categorize data into a meaningful format.
The accepted file types include: `.xls` and `.xlsx`. For reference, you can check out this [taxonomy file example](https://bit.ly/3UzkGP5).
### Data file format and encoding
The data file must be delimited, with fields and categories separated. Categories need to be
separated using a comma (`,`).
### Columns to sync
For this destination, you can export all columns represented in your model.
If you need to remap the fields you're exporting, you can do so manually.
In that case, the sync only exports fields you've mapped. This example exports `ID` and `CategoryID` and ignores all other fields in the model.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Amplitude
**URL:** https://hightouch.com/docs/destinations/amplitude
**Description:** Empower your product teams with richer data from your warehouse to perform deeper analysis and better understand user behavior
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-------------|--------------------------------------------------------------------------------|----------------------|---------------
**Objects** | Sync records to objects such as users or organizations in Amplitude | Upsert | [Objects docs](https://amplitude.com/docs/analytics/apis/identify-api)
**Events** | Sync records as events to Amplitude. This is often in the form of a track call | Insert | [Events docs](https://amplitude.com/docs/analytics/apis/batch-event-upload-api)
**Cohorts** | Sync records to cohorts in Amplitude | Add, Remove |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Amplitude
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination**
button. Select **Amplitude** and click **Continue**. You can then authenticate Hightouch to **Amplitude** by entering these fields into Hightouch:
- **API Key**: You can find your API key by following [these instructions in Amplitude's docs](https://www.docs.developers.amplitude.com/analytics/find-api-credentials/).
- **Region**: This is either a **Standard server** or **EU residency server**.
Once you've entered these fields, click **Continue**. Give your destination a descriptive name and **Finish** the destination connection setup.
## Sync configuration
Once you've set up your Amplitude destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and
the Amplitude destination you want to sync to.
### Objects
Hightouch can sync data to either **[User](https://help.amplitude.com/hc/en-us/articles/115002380567-User-properties-and-event-properties)** and **[Group](https://developers.amplitude.com/docs/group-identify-api)** objects. A dropdown is available to select between **Users** and **Groups** when configuring an Amplitude sync.
This integration only supports the **Upsert** mode. In this mode, Hightouch inserts new users or groups into Amplitude and keeps all user attributes up-to-date.
#### Record matching
Hightouch matches rows from your model to Amplitude users by **Amplitude User ID** or **Amplitude Device ID**. Select the model column that contains these values if you are syncing user objects. See [Amplitude's docs](https://help.amplitude.com/hc/en-us/articles/206404628-Step-2-Identifying-your-users) for more on identifying users.
Hightouch matches rows from your model to Amplitude groups by **Group Value**. Select the model column that contains these values if you are syncing groups.
#### Field mapping
You can sync columns from your source to Amplitude's default and custom fields. User properties only apply on the next event **after** the property has been changed, and don't apply retrospectively. For more information, check out [Amplitude's FAQ](https://help.amplitude.com/hc/en-us/articles/115002380567-User-Properties-Event-Properties#applying-user-properties-to-events).
#### Delete behavior
The delete behavior you select dictates what to do when a row no longer appears in your model's query results. You have the following options for Amplitude objects:
Behavior | Description
---------------|-----------------------------------------------------------------
**Do nothing** | Keep the customer or group record in Amplitude with all its synced fields
**Clear** | Remove all the mapped fields, but keep the record in Amplitude
### Events
Hightouch supports sending events of a given name, for example, `Signed Up`. Enter this name and choose the model column containing the either user ID or device ID. You can also select the model column containing the event timestamp. If the timesamp field is empty, Hightouch uses the time the event arrives at the server.
Hightouch accepts a normal date time format for the timestamp, and
automatically converts it to the format required by Amplitude.
#### Field mapping
Hightouch allows you to pass data to the [default properties](https://developers.amplitude.com/docs/http-api-v2#keys-for-the-event-argument) and the custom event properties of an Amplitude event.
### Cohorts
Hightouch can sync data to Behavioral Cohorts in Amplitude.
#### Record matching
Hightouch matches rows from your model to Amplitude users by **User ID** or **Amplitude ID**. Select the model column that contains these values if you are syncing to cohorts. See [Amplitude's docs](https://amplitude.com/docs/apis/analytics/behavioral-cohorts) for more on behavioral cohorts.
#### Creating new Cohorts
Hightouch can **Create a new cohort** when syncing to Amplitude cohorts. Hightouch will create a single placeholder user when creating a new cohort, and then sync the mapped identifiers to your newly created cohort.
## Tips and troubleshooting
For Hightouch platform error codes related to Amplitude, see [Error codes: Amplitude](/developer-tools/error-codes#other-destinations).
### Common errors
### Live debugger
### Sync alerts
---
## NCR Advanced Marketing Solution
**URL:** https://hightouch.com/docs/destinations/ams
**Description:** Manage targeted promotions and loyalty programs for all the stores in your chain, from a central point.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes |
| ------------------ | ---------------------------------------------------------------------- | -------------------- |
| **Customer group** | Sync customer groups by external ID to NCR Advanced Marketing Solution | Insert, All |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to NCR Advanced Marketing Solution
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **NCR Advanced Marketing Solution** (AMS) and click **Continue**. You can then authenticate Hightouch to **NCR Advanced Marketing Solution** by entering the following fields into Hightouch:
- **Host**: The hostname or IP address of your NCR AMS instance
- **Username**: This can be your personal NCR AMS login or a dedicated user for Hightouch
- (Optional) **Password**: The password for the specified user
- (Optional) **Private Key**: An SSH private key
- (Optional) **Private Key Passphrase**: The passphrase associated with the private key
## Sync configuration
Once you've set up your NCR AMS destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the NCR AMS destination you want to sync to.
### Syncing customer groups
Hightouch syncs customer groups to NCR AMS as `.cgi` files.
Hightouch automatically batches your query results into up to two million rows per file.
Each `.cgi` file contains `ID`,`Name`, and `BulkData` XML elements.
The `BulkData` element contains the external IDs of the customer groups you are syncing.
Refer to the [example `.cgi` file](#example-file) for details.
#### File path
The file path defines where Hightouch should send the `.cgi` files.
The file naming convention is `<>_<>_<>.cgi`.
Hightouch automatically adds the file number so you only need to provide the `<>_<>.cgi`.
For example, suppose you input `TestGroup_AMSID_73237_UPC_53395004106.cgi`, the first two million rows of data would appear in a `.cgi` file with the name `TestGroup_AMSID_73237_UPC_53395004106_01`.
The second two million rows would appear in a file with the name `TestGroup_AMSID_73237_UPC_53395004106_02`, etc.
#### Batch ID
Hightouch uses your input in the **Batch ID** for the `Name` and `ID` elements in the `.cgi` file and automatically suffixes a `_<>`.
For example, suppose you input `37293_US` as the batch ID, the first two million rows would use `37293_US_01` for both `Name` and `ID` elements.
The second two million rows would use `37293_US_02` for `Name` and `ID` elements, etc.
#### External ID column
Use this field to specify which column from your model contains the external IDs to sync.
In the `.cgi` file, these values have the suffix `,0\n` (comma, zero digit, and new line) and are sent as the `BulkData` element.
#### Example file
Hightouch syncs `.cgi` files with contents like this:
```
37293_US_0137293_US_01
405247281386905506,0
213550969606979806,0
497647364788236008,0
914660244431276315,0
637947282525555405,0
525462241744844516,0
494763443801427416,0
augment
```
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Anaplan
**URL:** https://hightouch.com/docs/destinations/anaplan
**Description:** Learn how to sync data from your warehouse to Anaplan using Hightouch.
**Section:** Destinations
## Overview
With the Anaplan destination, you can automate file uploads and trigger import actions in Anaplan. You can sync a variety of data from your warehouse, such as purchase orders, customer data, and more.
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| ----------------------- | -------------------------------------------- | -------------------- | -------------------------------------------------------------------------------------------------------------------- |
| [**Records**](#records) | Sync data from any source to Anaplan records | Insert, All | [Perform an import action ↗](https://help.anaplan.com/5ef6eaab-72d9-43de-91b3-3f4dcc4711e2-Perform-an-import-action) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Anaplan
Select an authentication method: OAuth or basic auth (username and password).
### OAuth
#### Setup Anaplan OAuth client
Hightouch supports Anaplan OAuth clients that have the type `Authorization code grant` with both non-rotatable and rotatable refresh tokens.
- Non-rotatable: The refresh token used for getting access tokens do not expire for as long as you allow the connection.
- Rotatable: The refresh token used for getting access tokens changes every time a new access token is retrieved.
Rotatable tokens require setting a `Refresh token lifespan` in Anaplan. This
setting is the total time in seconds before the connection will need to be
re-authenticated in Hightouch.
**Ex:** Setting `Refresh token lifespan` token to 604800 (7 days) will rotate the
refresh token every time an access token expires but will require you to re-authenticate
your account at the end of that time. Until you re-authenticate all syncs will
continue to fail. [More detail on refresh tokens](https://help.anaplan.com/rotate-client-secret-357f0086-5752-435b-a5bd-5eb6797ffc3f)
Use Non-rotatable refresh tokens for syncs to minimize chances of sync
failures.
#### Connecting with Hightouch
Select the region you want to connect to and then enter your client ID and client secret.
Your client ID and secret are provided by Anaplan when you setup your OAuth client.
Continue through the auth flow to connect your account.
### Basic auth
Select the region you want to connect to and enter your username and password.
If your Anaplan workspace uses an IP allow list, you must add Hightouch's
server IP addresses before connecting. See [Anaplan's IP allow list
documentation](https://help.anaplan.com/enable-ip-allow-list-9fcc01f4-fcb9-4384-8568-4ebb1bdfe9f9)
and [Hightouch's IP addresses](/security/networking#ip-addresses).
## Prerequisites
Before creating a sync, set up the following in Anaplan:
### Create an import data source
You must create a data source in Anaplan for Hightouch to upload data into. Go to your model's **Model settings → Actions → Import Data Source** and create a new import data source by uploading a placeholder CSV.
Hightouch overwrites this CSV with data from your warehouse based on the mappings in your sync configuration.
### Create an import (optional)
You can optionally create imports based on the data source and trigger them after the upload completes. Go to a list and select **Import**. After selecting the placeholder data source, you can create an import that can then be selected in Hightouch.
## Sync configuration
### Records
Use this sync type to upload data from your warehouse to an Anaplan import data source and optionally trigger imports.
#### Configure record matching
Select the model and data source that you created (or an existing one). Hightouch overwrites the import data source on each sync run. You can choose between two modes:
- **Insert**: The data source is overwritten with only the rows that have been added since the last run.
- **All**: The data source is overwritten with the full results of the query in this run.
#### Configure triggered imports
You can select one or more imports to trigger after the data source upload completes. By default, Hightouch triggers these imports in parallel and waits for all imports to finish before completing the sync.
#### Map fields
Map the columns in your Hightouch model to columns in your Anaplan data source.
The Anaplan column names you map to in Hightouch must exactly match the column
names configured in your Anaplan data source. The columns must also be in the
same order as they appear in your data source. Incorrect column names or order
mismatches cause your sync to error.
## Tips and troubleshooting
For Hightouch platform error codes related to Anaplan, see [Error codes:
Anaplan](/developer-tools/error-codes#other-destinations).
### Validate your setup
After your first sync completes, verify the data in Anaplan:
- Go to your model's **Model settings → Actions → Import Data Source** and open the data source you configured. Confirm the file contents match the rows and columns from your Hightouch model.
- If you configured triggered imports, go to the [imports page](https://help.anaplan.com/5ef6eaab-72d9-43de-91b3-3f4dcc4711e2-Perform-an-import-action) and check that the import status shows success. Failed imports typically indicate a column mismatch between the file and the target list.
- Open the target list or module in Anaplan and spot-check that values landed in the correct fields. Column name or ordering mismatches are the most common cause of sync failures — if data appears in the wrong columns, compare your Hightouch field mapping order against the import data source column order.
### Common errors
#### Connection failures
If your Anaplan connection fails during setup or stops working after initial configuration:
- **Basic auth:** Verify that your username and password are correct and that the account hasn't been locked. If your organization enforces SSO, confirm that the Anaplan user account has SSO disabled for API access — SSO-only accounts can't authenticate via basic auth. If your Anaplan workspace uses an IP allow list, confirm that [Hightouch's IP addresses](/security/networking#ip-addresses) are included.
- **OAuth:** Confirm that your client ID and secret match the [OAuth client configured in Anaplan](https://help.anaplan.com/rotate-client-secret-357f0086-5752-435b-a5bd-5eb6797ffc3f). If you're using rotatable refresh tokens, the connection expires after the `Refresh token lifespan` period you set in Anaplan — re-authenticate the destination in Hightouch when this happens.
- **Region mismatch:** Confirm that the region selected in Hightouch matches your Anaplan workspace region. Using the wrong region causes authentication to fail silently.
#### Column name or order mismatch
If your sync errors with a message about column mapping, verify that the column names in your Hightouch field mapping **exactly match** the column names in your Anaplan import data source, and that the columns are in the **same order**. Even small differences in capitalization or spacing cause sync failures.
### Live debugger
### Sync alerts
---
## Antavo
**URL:** https://hightouch.com/docs/destinations/antavo
**Description:** Attract customers, scope loyalty, and build your dream customer loyalty program with Antavo.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-------------------|----------------------------------------------------|----------------------|----------------------------------------------------------------------------------
**Events** | Sync data from any source to Antavo events | | [Events docs](https://developers.antavo.com/reference/about-events-api)
**Customer Lists** | Sync data from any source to Antavo customer lists | Add, Remove | [Customer Lists docs](https://developers.antavo.com/reference/about-entities-api)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Antavo
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Antavo** and click **Continue**. You can then authenticate Hightouch to **Antavo**.
Enter the following fields into Hightouch:
- **API key**
- **API secret**
- **Region**
## Prerequisite steps in Antavo
There are prerequisite steps in Antavo to sync Events. Custom event creation is required before syncing to Antavo.
For Customer Lists, you have the option to create a customer list in Antavo directly and use the existing list in Hightouch, or create a new customer list.
If you run into any issues, you can reach out to the Antavo CSM team.
## Sync configuration
Once you've set up your Antavo destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Antavo destination you want to sync to.
### Syncing events
Sync data from any source to Antavo events. You can either:
- sync one [event type](https://developers.antavo.com/docs/api-events) for the entire batch by specifying an `Event type`, and map the corresponding event's attributes or
- sync multiple event types but pass in the `data` objet directly in your mappings.
#### Record matching
You can match rows from your model to events in Antavo on any column in your model and any field in Antavo. Ensure the data types of the model column and Antavo field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
### Syncing customer lists
Sync data from any source to Antavo customer lists.
#### Record matching
To match rows from your model to customer lists in Antavo, you need to select the model column that contains values that match the **ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync customer list fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default customer list fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Apache Kafka
**URL:** https://hightouch.com/docs/destinations/apache-kafka
**Description:** Connect to your internal microservices via a message queue
**Section:** Destinations
## Overview
Hightouch integrates directly with Apache Kafka to support high-throughput, distributed, or asynchronous workloads, letting you build a custom connector to your internal systems.
This destination was designed to be as flexible as possible. Some of its capabilities include:
- connecting to multiple brokers
- authenticating with Simple Authentication and Security Layer (SASL)
- using your own certificate authority
- publishing different topics for each message trigger
- defining custom ordering and partition keys
Hightouch supports all managed Kafka services (Amazon MSK, Confluent Cloud, etc.) and can also connect to self-hosted instances.
## Getting started
### Connect to your Kafka server
For Hightouch to connect to your Kafka server, you need to enter:
- a client ID
- your broker information
- authentication method used in your Kafka server configuration
#### Client ID
The [**Client ID**](https://kafka.apache.org/documentation/#design_quotasgroups) is a logical identifier of a client application—in this case it's the Hightouch Apache Kafka destination. It's used to distinguish each running application of your Kafka server. You can choose to name this anything you want that fits your use case.
#### Brokers
For Hightouch to sync data to the right Kafka Brokers, you need to provide the **host** and **port** number in the format `{host}:{port}`. If you want to configure the destination to connect to multiple brokers, you can input the details separated by a comma. For example, you could enter `{host1}:{port1},{host2}:{port2}`.
#### Authentication
Hightouch can connect to your Kafka server either with SASL or directly without authentication. For security purposes, it's best to configure your Kafka server to require SASL authentication when syncing production data, and to only omit authentication requirements for testing purposes.
When configuring your SASL mechanism in Hightouch, you have four options:
- PLAIN
- SCRAM SHA256
- SCRAM SHA512
- AWS IAM
All four provide the option to include your own self-signed certificate authority.
#### PLAIN/SCRAM
For **Username** and **Password**, enter your username and password configured in your Kafka server. If you are using a managed Kafka service, your details can usually be found in your environment's settings or as an API key and secret.
#### AWS IAM
If you want to authenticate via AWS IAM, we assume your Kafka server is configured to use AWS IAM as an authentication method, that is [STACK's Kafka AWS IAM LoginModule](https://github.com/STACK-Fintech/kafka-auth-aws-iam) or a compatible alternative is installed on all target brokers.
- **Authorization Identity** must be the `aws:userid` of the AWS IAM identity. Typically, you can retrieve this value using the `aws iam get-user` or `aws iam get-role` commands of the AWS CLI toolkit. The `aws:userid` is usually listed as the `UserId` or `RoleId` property of the response.
- You can find your **Access Key ID**, **Secret Access Key**, and **Session Token** in your AWS account. For more information on AWS IAM credentials and authentication, refer to the official [AWS docs](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html).
## Syncing data
Once you've connected your Kafka server to Hightouch, you've completed setup for a Apache Kafka destination in your Hightouch workspace. The next step is to configure a sync that send messages whenever rows are added, changed, or removed in your model.
### Configure your events trigger
Hightouch monitors your data model for added, changed, and removed rows. In this step, you specify which of these events should trigger message publishing.
### Choose your topic
In this step, you choose which topics to publish the messages to. Hightouch allows you to sync to existing topics that are already in your Kafka cluster.
Suppose you want to sync to multiple existing topics but don't want to create a new sync for every topic. As long as your model has a column associated to topic names in your Kafka cluster, Hightouch can sync to multiple Apache Kafka topics in just one sync.
To enable this feature, toggle **USE COLUMN**, and select a column in your model containing the topic name rows.
When syncing to multiple topics, if a topic name in the selected column of
your model doesn't exist in the Kafka cluster, then the entire batch of
messages will fail to sync.
### Customize your message
In this step, you tell Hightouch how to build the JSON message data object using data from your model.
This destination offers three methods of composing a JSON object:
- [Using a JSON editor](#use-json-editor)
- [Using one column from a model](#use-one-column-from-model)
- [Using multiple columns from a model](#use-multiple-columns-from-model)
#### Use JSON editor
With the JSON editor, you can compose any JSON object using the [Liquid template language](https://shopify.github.io/liquid/). This is useful for complex message data bodies containing nested objects and arrays, which can sometimes be difficult to model entirely in SQL.
Suppose your data model looks like this:
| full_name | age | email_address | phone_number |
| --------- | --- | ------------------ | ------------ |
| John Doe | 30 | `john@example.com` | +14158675309 |
And you want your message data like this:
```json
{
"name": "John Doe",
"age": 30,
"contact_info": [
{
"type": "email",
"value": "john@example.com"
},
{
"type": "phone",
"value": "+14158675309"
}
]
}
```
Your Liquid template should look like this:
```txt
{
"name": "{{row.full_name}}",
"age": {{row.age}},
"contact_info": [
{
"type": "email",
"value": "{{row.email_address}}"
},
{
"type": "phone",
"value": "{{row.phone_number}}"
}
]
}
```
This makes it so you can reference any column using the syntax `{{row.column_name}}`. You can also use advanced Liquid features to incorporate control flow and loops into your dynamic message data.
When injecting strings into your JSON object, be sure to surround the Liquid
tag in double quotes.
#### Use one column from model
If you're already storing JSON data in your source, or if you have the ability to construct a JSON object using SQL, you can select one column in your model that already contains the full message data.
This setting is commonly used when syncing web events that have already been collected and stored as JSON objects in your database.
#### Use multiple columns from model
For the simplest use cases, Hightouch can construct a JSON object with key/value pairs based on multiple columns in your model.
Suppose your model looks like this:
| email | first_name | last_name |
| ----------------------- | ---------- | --------- |
| `alice.doe@example.com` | Alice | Doe |
| `bob.doe@example.com` | Bob | Doe |
| `carol.doe@example.com` | Carol | Doe |
The field mapping in the preceding screenshot would generate the following message data for the first row:
```json
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "alice.doe@example.com"
}
```
You can use the field mapper to rename fields. For example, `first_name` can
be mapped to `customer_first_name`.
### Configure optional message properties
Along with your row data in JSON format, you can optionally include ordering keys to configure the order your Kafka cluster receives message and metadata fields as headers.
##### `partition`
A number field that determines which partition to send the message to. This field takes precedence over the `key` field. That is if you provided `partition` and `key`, the message will be sent to the partition stated in the `partition` field and not the `key` field. Hightouch automatically tries to cast the value to a number. If we can't cast the value to a number then it is sent as `null`.
##### `key`
If no `partition` column is selected but a `key` of string type is selected, then Kafka chooses a partition to send the message to based on a murmur2 hash of the key. For example, if you use an `orderId` as the key, you can ensure that all messages regarding that order will be processed in order.
If no `partition` or `key` is included, then the message will be sent to a partition in a round-robin fashion.
##### `headers`
This is an object containing key/value pairs of custom mapping fields.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows may require performing a backfill of all rows during the initial sync. For other use cases, you might only want to send messages in response to future data changes.
## Validate your setup
After you configure your sync, run a test to confirm data flows correctly:
1. Start with a small test audience or a single record.
2. Trigger a manual sync from the [Syncs overview](https://app.hightouch.com/syncs?from=docs) page.
3. After the sync completes, verify the messages were received by your Kafka consumer. You can use a CLI consumer like `kafka-console-consumer` or your existing consumer application to confirm messages appear on the expected topic with the correct payload structure.
4. If the sync shows errors, check the sync run log in Hightouch for specific error messages.
---
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Apollo.io
**URL:** https://hightouch.com/docs/destinations/apollo
**Description:** Apollo’s comprehensive solution provides the most accurate B2B contact data, built-in engagement tools, and intelligence features that help companies find and convert quality prospects.
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes | API Reference|
|--|--|--|--|
|**Accounts**|Sync data from any source to an account |Upsert, Update, Insert|[Account docs](https://apolloio.github.io/apollo-api-docs/?shell#accounts-api)|
|**Contacts**|Sync data from any source to a contact|Upsert, Update, Insert|[Contact docs](https://apolloio.github.io/apollo-api-docs/?shell#contacts-api)|
|**Sequences**|Sync data from any source to update membership in Apollo sequences|Insert, Remove|[Sequence docs](https://apolloio.github.io/apollo-api-docs/?shell#sequences-api)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Apollo.io
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Apollo.io** and click **Continue**.
You can then authenticate Hightouch to Apollo.io with by entering an Apollo.io **API Key**.
Follow [these instructions](https://knowledge.apollo.io/hc/en-us/articles/4415734629773-Configure-Access-to-the-Apollo-REST-API) to create one.
Because Apollo.io API key scopes are based on endpoints and not permissions, you must create a master key to use the Hightouch integration.
## Sync configuration
Once you've set up your Apollo.io destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Apollo.io destination you want to sync to.
### Syncing accounts
Account objects represent companies in Apollo.io and can be used as leads or opportunities.
#### Record matching
To match source rows to contacts in Apollo.io, you need to select a source column and corresponding Apollo.io field. You can match on any of the following Apollo.io fields:
- Name
- ID
#### Field mapping
You can sync columns from your source to Apollo.io default and custom fields. For custom picklist fields, you can send display values, and Hightouch remaps the value to Apollo.io's internal ID.
### Syncing contacts
Contact objects are people in Apollo.io and are used as the point-of-contact for leads and opportunities.
#### Record matching
To match source rows to contacts in Apollo.io, you need to select a source column and corresponding Apollo.io field. You can match on any of the following Apollo.io fields:
- Email (recommended)
- ID
- Name
- Title
- Company
#### Field mapping
You can sync columns from your source to Apollo.io default and custom fields. For custom picklist fields, you can send display values, and Hightouch remaps the value to Apollo.io's internal ID.
### Syncing sequences
Sequences are outreach campaigns with any number of sequential contact points and tasks that you can customize to personally engage your target audiences at scale.
#### Record matching
To match rows from your model to records in Apollo.io, you need to select the model column that contains values that match the **ID** Apollo.io field.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Appcues
**URL:** https://hightouch.com/docs/destinations/appcues
**Description:** Push user behavior data from your warehouse into Appcues
**Section:** Destinations
## Setup
After selecting Appcues from our destinations catalog, you will be required to enter your Appcues _Account ID_ (found in _Appcues_ > _Settings_ > [_Account_](https://studio.appcues.com/settings/account)) and _API Key / API Secret_ (found in _Appcues_ > _Settings_ > [_API Keys_](https://studio.appcues.com/settings/keys)).
Hightouch supports syncing both `Objects` and `Events` to Appcues.
## Syncing objects
Hightouch supports syncing to the following Appcues objects:
- `Users`
### Sync modes
Hightouch supports the following sync modes:
- **Upsert** - pushes new `User` objects to Appcues and updates properties that change in your warehouse.
### Record matching
`User` records can be matched from your source to your Appcues workspace by User ID (`user_id`), which is a unique identifier
in Appcues for each `User`.
### Field mapping
You may also sync columns from your source to Appcues's `User` properties. Appcues doesn't provide information on properties set for `User` objects,
so you will need to type the exact name of the property you would like to map to in Appcues. These property names must be spelled correctly and are case-sensitive.
## Sending events to Appcues
The Appcues integration also allows you to send events to Appcues with custom attributes.
The three required elements of any Appcues event are the _Event Name_ (which can be a single string used to name all events, or pulled from your warehouse per event), a _Timestamp_ (in ISO 1806 format, or a Unix timestamp in **milliseconds**), and a **User ID** (the unique ID of the user corresponding to the event).
You can also map properties that will become event `attributes` in Appcues.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## AppLovin
**URL:** https://hightouch.com/docs/destinations/applovin
**Description:** AppLovin is a mobile technology company that provides app developers with tools to grow, monetize, and analyze their apps through user acquisition, in-app bidding, and ad mediation. Its platform uses machine learning to optimize ad delivery and maximize revenue across a global network.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-----------------|----------------------------------------------------|----------------------|-------------------------------------------------------------------------------------------
**Suppressions** | Sync data from any source to AppLovin suppressions | Insert | [Suppressions docs](https://developers.applovin.com/en/app-discovery/api/suppression-api/)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to AppLovin
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **AppLovin** and click **Continue**. You can then authenticate Hightouch to **AppLovin**.
Enter the following fields into Hightouch:
- **API key**: API key (or Management key) for authentication. Find it in the [Keys tab](https://www.applovin.com/analytics#keys) of your account page. If the Management Key is not visible, [open a ticket](https://support.applovin.com/hc/en-us/requests/new) with AppLovin and request one be generated for you.
## Sync configuration
Once you've set up your AppLovin destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the AppLovin destination you want to sync to.
### Syncing suppressions
Sync data from any source to AppLovin suppressions.
#### Package Name
Specify the package name you would like to use with the AppLovin suppression API. The package name can be a bundle ID (iOS) or a general package name (Android) of your app.
#### Record matching
You can match rows from your model to suppressions in AppLovin on any column in your model. Ensure the data types of the model column and the AppLovin Device UUID field match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
The AppLovin suppressions API only accepts UUID format device IDs. Ensure your device IDs are the correct format.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
------------------------|------------------------------------------------------------
**Do nothing** | Keep the suppression in AppLovin
**Remove from list** | Remove the suppression from AppLovin
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## AppsFlyer
**URL:** https://hightouch.com/docs/destinations/appsflyer
**Description:** Give your team visibility into campaign performance, scale app installs, and maximize customer LTV with a new standard of privacy-enhancing measurement and deep linking solutions.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
|-----------------------------|-------------------------------------------------------------|----------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|
| **Server-to-server events** | Sync mobile events that occur outside the app to AppsFlyer | Insert | [Server-to-server events docs](https://dev.appsflyer.com/hc/reference/post_s2s_inappevent) |
| **Audiences** | Import predefined audience segments as lists into AppsFlyer | Add, Remove | [Import audience docs](https://support.appsflyer.com/hc/en-us/articles/360013291857-Audiences-import-an-audience#use-the-import-audience-api) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to AppsFlyer
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **AppsFlyer** and click **Continue**. You can then authenticate Hightouch to AppsFlyer by entering the following fields depending on your use case:
To sync S2S events:
- **AppsFlyer V2 S2S key** to use AppsFlyer's `api3` events endpoint. Follow [these instructions in AppsFlyer's docs](https://support.appsflyer.com/hc/en-us/articles/360004562377-Managing-AppsFlyer-tokens) to create a S2S key.
- **Dev key** *(legacy)* to use AppsFlyer's `api2` events endpoint. As of December 2023, AppsFlyer has deprecated `api2`. We recommend updating your configuration to use the S2S key for best performance. If you choose to use a dev key, follow [these instructions in AppsFlyer's docs](https://support.appsflyer.com/hc/en-us/articles/211719806-App-settings-#sdk-authentication-dev-key)
For more information about upgrading from `api2` to `api3`, see AppsFlyer's announcement [here](https://support.appsflyer.com/hc/en-us/articles/20509378973457-Bulletin-Upgrading-the-AppsFlyer-S2S-API).
To sync to audiences:
- **AppsFlyer V2 API key**: You can create a V2 API key by following [these instructions](https://support.appsflyer.com/hc/en-us/articles/360004562377-Managing-API-and-Server-to-server-S2S-tokens#manage-your-tokens).
You only need to provide credentials for the sync type you intend to use. For example, if you would only like to sync S2S events, you do not need to provide an API key but are required to provide a S2S key.
## Sync configuration
Once you've set up your AppsFlyer destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the AppsFlyer destination you want to sync to.
### Syncing server-to-server events
You can use server-to-server events to send mobile events that occur outside the app from your servers to AppsFlyer.
Appsflyer's API requires the following event parameters:
- `app_id`
- `eventName`
- `eventValue`
The sync configuration form ensures all these are set and provides some additional options.
#### Appsflyer app ID
The app ID is the identifier used in the AppsFlyer dashboard. Insert it exactly as it appears on the dashboard. You can either provide a static value or select to **use a column** from your model. Hightouch syncs the static or column value as the [`app_id` path parameter](https://dev.appsflyer.com/hc/reference/post_s2s_inappevent#path-params) the API requires.
#### Event name
Providing an event name is required to send an event to the Appsflyer API. You can either provide a static value or select to **use a column** from your model. Hightouch syncs the static or column value as the [`eventName` body parameter](https://dev.appsflyer.com/hc/reference/post_s2s_inappevent#path-params) the API requires.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Appsflyer uses the time the event arrives at the server.
#### Field mapping
Hightouch lets you map model columns to event metadata. Some of the metadata fields you can map to include:
- AppsFlyer ID
- IDFV
- IP
- Operating system
The AppsFlyer ID is a required field, so ensure you include that in your mapping.
#### Event value mapping
Each event _must_ contain a JSON `eventValue`, for example:
```json
{
"af_revenue": "6",
"af_currency": "USD",
"af_content_type": "wallets",
"af_content_id": "15854",
"your_custom_attribute" :"456.123"
}
```
Hightouch lets you map these fields to create the JSON object the `eventValue` field expects.
#### Custom mappings
You can also map additional custom data that Hightouch sends to the AppsFlyer platform as JSON using the `custom_data` field.
### Syncing audiences
This sync type lets you update which users belong to an audience.
You must use an existing audience in AppsFlyer.
#### Record matching
To identify which users to add or update in an audience, select model columns and the corresponding AppsFlyer fields. You can match on the following fields:
- App ID
- IDFV
- IDFA
- GAID
- OAID
- IMEI
- CUID
- Email
- Phone
- Braze ID
- Amplitude ID
You must include the App ID for all audience types.
Depending on whether you're syncing an Android or iOS audience, you need to include at least one additional identifier.
For Android audiences, you must also include at least one of the following:
- GAID
- OAID
- IMEI
- CUID
- Braze ID
- Amplitude ID
For iOS audiences, you must also include at least one of the following:
- IDFV
- CUID
- Braze ID
- Amplitude ID
See [AppsFlyer's docs](https://support.appsflyer.com/hc/en-us/articles/360013291857#identifier-hash) on identifiers for more information.
#### Select an existing audience or create a new one
You can create a new audience or use an existing one. When creating a new audience, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model. To use an existing audience, you need to enter the audience's `import_key`.
To get an audience's `import_key` follow these instructions:
1. In the AppsFlyer left-side panel, go to **Integration > Audiences**.
2. Click on the name of the audience you want to update.
3. Select the **Import** tab.
4. In the **Import using API** section, choose **Add new devices/users**
5. Copy the `import_key` in the **API request body**.
For more information about the **Import Audience API**, refer to [AppsFlyer's docs](https://support.appsflyer.com/hc/en-us/articles/360013291857-Audiences-import-an-audience?query=import%20key).
#### Handling PII and hashing
By default, Hightouch automatically hashes fields that require it before sending them to Appsflyer.
You can disable this behavior in the sync configuration. If disabled, the data from the model should be appropriately normalized and hashed according to [Appsflyers's hashing requirements](https://support.appsflyer.com/hc/en-us/articles/360017807278-Audiences-additional-identifiers).
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when an audience member no longer appears in your model's query results. You have the following options:
| Behavior | Description |
|----------------|-----------------------------------|
| **Do nothing** | Keep the user in the audience |
| **Delete** | Remove the user from the audience |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Asana
**URL:** https://hightouch.com/docs/destinations/asana
**Description:** Keep your team's projects and tasks up-to-date with fresh customer data
**Section:** Destinations
## Overview
With the Asana destination, Hightouch can upsert or update tasks, or update projects.
For the task object, the destination supports:
- **Upsert**: Upsert mode creates new tasks and updates existing tasks based on an external ID.
- **Update**: Update mode updates existing or newly created tasks based on the Asana GID. The update mode can also update tasks created with Hightouch using the external ID.
For the projects object, the destination supports:
- **Update**: Update mode updates existing projects based on the Asana GID.
## Setup
Connect Asana via OAuth by granting permission to the Hightouch Asana Connect app.
## Tasks
### Upsert
Hightouch can upsert tasks based on an external ID in Asana. This means it will update the task in Asana if it was previously created by Hightouch, otherwise it will create a new task. The external ID is unique to Hightouch thus in this mode it will only update tasks created via Hightouch.
| Name | Description |
| :--------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Hightouch to Asana Merge Rule | The column to use as the external ID in Asana. This should be a unique identifier and will most likely be some form of ID. |
| Hightouch to Asana Field Mappings | A list of columns that will be pushed to tasks in Asana. Each time a sync occurs in Hightouch these columns will be updated. For array properties such as projects or followers, [refer to notes on array properties.](#array-properties) |
| Hightouch to Asana Custom Field Mappings | A list of custom fields that will be pushed to tasks in Asana. [Refer the notes on custom fields in Asana.](#custom-fields) |
### Update
Hightouch can update tasks based on the Asana GID. Hightouch can also update tasks based on an external ID if the task was created using Hightouch.
If upserting or updating via an external ID, Hightouch can only update tasks
created with such an external ID through Hightouch, due to the way Asana
[external IDs are limited on a per application
basis](https://developers.asana.com/docs/custom-external-data).
| Name | Description |
| :--------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Hightouch to Asana Merge Rule | The column that matches either the Asana GID or external ID in Asana. |
| Hightouch to Asana Field Mappings | A list of columns that will be pushed to tasks in Asana. Each time a sync occurs in Hightouch these columns will be updated. For array properties such as projects or followers, [refer to notes on array properties.](#array-properties) |
| Hightouch to Asana Custom Field Mappings | A list of custom fields that will be pushed to tasks in Asana. [Refer the notes on custom fields in Asana.](#custom-fields) |
## Projects
Hightouch can update projects on the Asana GID. Unfortunately, Hightouch can't upsert projects or update based on external ID because Asana doesn't yet support external metadata for objects other than projects.
| Name | Description |
| :--------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Hightouch to Asana Merge Rule | The column that matches the Asana Project GID. |
| Hightouch to Asana Field Mappings | A list of columns that will be pushed projects in Asana. Each time a sync occurs in Hightouch these columns will be updated. For array properties such as followers, [refer to notes on array properties.](#array-properties) |
| Hightouch to Asana Custom Field Mappings | A list of custom fields that will be pushed to projects in Asana. [Refer the notes on custom fields in Asana.](#custom-fields) |
## Array properties
Array properties can be passed to Asana by returning the data as either a comma delimited string or an array in your SQL query. Examples of array properties are:
- Tasks: projects, followers, tags
- Projects: followers
In the SQL query, a column `taskFollowers`containing a string such as `'1199948419683571,1185085322202311'` can be mapped to the followers field of tasks in Asana.
Using the example of followers, keep in mind that every type Hightouch runs a sync, it will check the followers assigned to the task in Asana, and:
- Add all followers that are present in `taskFollowers` and not present in Asana
- Remove all followers that are present in Asana and not present in `taskFollowers`.
## Subtasks
A `subtask` is an Asana task object that is associated with a `parent` task object.
To create a `subtask`, select the column from your source that references `parent` tasks, then map it to the `parent` field in
the Mappings section.
In your source, you can identify the `parent` task either by the external ID or the Asana GID. The external ID or the Asana GID should
be a string. Not passing any value in for the `parent` field will remove an existing parent-subtask association - the task will simply be considered a normal `task` instead of a `subtask`.
## Custom fields
Custom fields are complex in Asana because they can be associated with both tasks and projects, and they can be restricted to only tasks in certain projects. To start, Hightouch supports custom fields that are available on a workspace level, which can be done by adding the field to the workspace's field library in Asana.
Secondly, custom fields may cause errors if the field is not enabled in the project you are inserting the task into.
## Delete behavior
For the task object, you can choose what Hightouch's behavior is when rows leave your query results. The default is doing nothing.
In **Upsert** mode, you have the following options:
| Behavior | Description |
| ----------------------------- | ------------------------------------------ |
| **Do nothing** | Keep the task in Asana |
| **Mark As Complete** | Keep the task but mark as complete |
| **Delete Destination Record** | Delete the task from Asana completely |
In **Update** mode, you have the following options:
| Behavior | Description |
| -------------------- | ----------------------------------------- |
| **Do nothing** | Keep the task in Asana |
| **Mark As Complete** | Keep the task but mark as complete |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Attentive
**URL:** https://hightouch.com/docs/destinations/attentive
**Description:** Attentive empowers the most innovative brands to create meaningful interactions through personalized text messaging
**Section:** Destinations
## Overview
This destination lets you push data from various sources into Attentive to maintain an up-to-date subscriber list for mobile messaging campaigns. Add attributes to subscribers in Attentive to build better segments for
personalized experiences and create user-associated events to trigger targeted workflows.
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| --------------------- | -------------------------------------------------- | -------------------- | ---------------------------------------------------------------------------------------- |
| **Identity** | Associate user IDs with other identifiers | Update | [Identity](https://docs.attentive.com/openapi/reference/tag/Identity/) |
| **Custom Events** | Send any event-based data associated with a user | Insert | [Custom Events](https://docs.attentive.com/openapi/reference/tag/Custom-Events/) |
| **Custom Attributes** | Add customizable data or characteristics to a user | Upsert | [Custom Attributes](https://docs.attentive.com/openapi/reference/tag/Custom-Attributes/) |
| **Subscribers** | Subscribe and unsubscribe users from subscriptions | Update | [Subscribers](https://docs.attentive.com/openapi/reference/tag/Subscribers/) |
## Getting started
To use the Attentive destination in Hightouch, you will need to generate an API key within Attentive.
To generate the API key, follow these [steps](https://docs.attentive.com/pages/authentication/).
Ensure that the generated key has the correct scopes for the endpoints you wish to use:
- **Identity:** `identity:write`
- **Custom Events:** `events:write`
- **Custom Attributes:** `attributes:write`
- **Product Catalog:** `product_catalogs:write`
Additionally, within Attentive, set **Event webhook status** to _Enabled_ and **Subscriber to event notifications** to _Subscription webhook_ for your app.
## Syncing data
### Identity
Syncing to Attentive identities lets you associate a client user identifier or custom identifier with other identifiers. A client user or custom identifier must be accompanied by at least one of the following identifiers:
`phone`, `email`, `Shopify ID`, `Klaviyo ID`, `client user ID`, or `custom identifier`.
Refer to the [Attentive Identity API docs](https://docs.attentive.com/openapi/reference/tag/Identity/) to learn more.
#### Matching users
You can match rows in your model with users in Attentive on the following fields:
- Email
- Phone number
- Client User ID
- Shopify ID
- Klaviyo ID
#### Field mapping
You can map other identifiers to associate with the primary identifier, including external IDs not natively supported by Attentive.
### Custom events
Syncing custom events lets you send user actions to use in the Attentive Segment Builder and the Journey Builder.
This data can't contain any sensitive or special categories of information as defined in applicable data
protection and privacy laws, including the California Consumer Privacy Act (CCPA) and California Privacy Rights Act (CPRA).
Refer to the [Attentive Custom Events API docs](https://docs.attentive.com/openapi/reference/tag/Custom-Events/) to learn more.
#### Associating users
You can use the following fields to map associated events with users:
- Email
- Phone number
- Client User ID
- Shopify ID
- Klaviyo ID
- Custom Identifiers
#### Event types
The event type can be a static value or a field from the model to send multiple different event types.
The event type name is case-sensitive. For example, "Order shipped" and "Order Shipped" are different event types.
#### Event universal unique identifier
Hightouch allows you to send an event universal unique identifier (UUID) for each event. Though optional, using a UUID is [recommended by Attentive](https://docs.attentive.com/openapi/reference/tag/Custom-Events/#tag/Custom-Events/operation/postCustomEvents).
#### Timestamps
Hightouch allows you to send a timestamp of when a custom event occurred in ISO 8601 format.
If the timestamp is older than 12 hours, relevant Journeys aren't triggered in Attentive.
If an event doesn't include a timestamp, the time of the API request is used.
#### Event properties
You can use properties to send metadata associated with the event.
Object keys are expected to be strings and can't contain any of the following special characters:
- Double quote (`"`)
- Curly braces (`{ }` )
- Square brackets (`[ ]`)
- Backslash (`\`)
- Vertical bar (`|`)
### Custom attributes
Syncing custom attributes lets you send attribute-based user data to Attentive. You can create as many custom attributes as needed. Note that you can create net-new properties with this API, however,
it can't be used to create new values for an existing UI-created property name. Whenever you create a property name on the Attentive platform, you must also define all possible property values there. Refer to the [Attentive Attributes API docs](https://docs.attentive.com/openapi/reference/tag/Custom-Attributes/) to learn more.
#### Associating users
You can associate custom attributes with users on the following fields:
- Email
- Phone number
- Client User ID
- Shopify ID
- Klaviyo ID
- Custom Identifiers
#### User attributes
User attributes are any metadata associated with a user. Attribute object keys should be strings, and object values can be any type. Both object keys and object values are case-sensitive. "Favorite color" and "Favorite Color" are considered different custom attributes.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Attio
**URL:** https://hightouch.com/docs/destinations/attio
**Description:** Push enriched data from your warehouse into Attio
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ------------------ | ------------------------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------ |
| **Companies** | Sync data from any source to companies in Attio | Upsert | [Companies docs](https://developers.attio.com/reference/put_v2-objects-companies-records) |
| **Custom objects** | Sync data from any source to custom objects in Attio | Upsert | [Custom objects docs](https://attio.com/developers/rest/record#assert-a-record) |
| **Deals** | Sync data from any source to deal objects in Attio | Upsert | [Deal docs](https://developers.attio.com/reference/put_v2-objects-deals-records) |
| **People** | Sync data from any source to people in Attio | Upsert | [People docs](https://developers.attio.com/reference/put_v2-objects-people-records) |
| **Users** | Sync data from any source to user objects in Attio | Upsert | [User docs](https://developers.attio.com/reference/put_v2-objects-users-records) |
| **Workspaces** | Sync data from any source to workspace objects in Attio | Upsert | [Workspace docs](https://developers.attio.com/reference/put_v2-objects-workspaces-records) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Attio
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Attio** and click **Continue**.
You will be required to enter your Attio API key. You can locate this API key within your Attio account by navigating to Workspace Settings > Developers:
## Sync configuration
Hightouch supports syncing to the following Attio objects:
- `Companies`
- `Custom objects`
- `Deals`
- `People`
- `Users`
- `Workspaces`
### Sync modes
Hightouch supports the following sync modes:
- **Upsert** - pushes new objects to Attio and updates fields that change in your warehouse.
### Record matching
Depending on the object you are syncing, records can be matched from your source to your Attio workspace by `domains` for companies, `email_addresses` for people, or other read-only fields for custom objects.
### Field mapping
You may also sync columns from your source to Attio's object properties.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Auth0
**URL:** https://hightouch.com/docs/destinations/auth0
**Description:** Sync user lists to Auth0 and automate and manage all of your passwords, grants, and permissions in a single platform
**Section:** Destinations
## Setup
After selecting Auth0 from our Destinations catalog, you will be prompted to enter
your _Auth0 domain_, _Client ID_, and _Client secret_ from your **registered
machine-to-machine (M2M) application for Hightouch**. You should find this by navigating
to your Auth0 dashboard -> Applications dropdown -> Applications -> select corresponding
Hightouch M2M. This particular M2M will give Hightouch permission to connect with your
Management API and update your users.
### Registering your M2M for Hightouch
If you do not have a registered M2M for Hightouch, refer to
[this Auth0 doc](https://auth0.com/docs/get-started/create-apps/machine-to-machine-apps) on
how to get started. You will then need to select the following configuration:
- Select the **Auth0 Management API**
- Check the **update:users** permission.
After you authorize, you should go to the M2M's settings to obtain the required keys.
Keep in mind you might have to go back and add to the permissions for
potential future features.
## Syncing rows
Hightouch supports syncing to the following Auth0 objects:
- `Users`
User objects in Auth0 will use the external ID as its own ID. Therefore, the user's source ID
(that is, external ID) must be present to properly match the user.
### Sync modes
This integration only supports **Update** mode.
In this mode, the destination will update the mapped attributes and metadata to the user objects.
It will not add new users.
### Record matching
Records can be matched from your source to Auth0 workspace by the external user's ID only.
### Field mapping
You can sync columns from your source to Auth0's user attributes, including the app and user's metadata.
You can view this [list for the supported Auth0 user attributes](https://auth0.com/docs/users/user-profiles/user-profile-structure).
The app and user metadata holds custom data so can take any shape or form.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Awin
**URL:** https://hightouch.com/docs/destinations/awin
**Description:** Empower advertisers and publishers with Awin's affiliate marketing platform to grow their businesses online.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-----------|------------------------------------------|----------------------|-----------------------------------------------------------------
**Orders** | Sync data from any source to Awin orders | Insert | [Orders docs](https://developer.awin.com/apidocs/conversion-api)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Required Permissions
Your AWIN account should have `Full Access` or `Admin` level [permissions](https://success.awin.com/s/article/how-can-i-manage-user-permissions?language=en_US) to use the Conversion API to manage orders. If you don't have either of these permission levels, contact your Organization Administrator to request access.
## Connect to Awin
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Awin** and click **Continue**. You can then authenticate Hightouch to **Awin**.
Enter the following fields into Hightouch:
- **OAuth 2.0 Token**: Find your OAuth 2.0 Token under API Credentials in the User dropdown.
- **Advertiser Id**: Find your Advertiser Id on the Overview page for your Profile.
## Sync configuration
Once you've set up your Awin destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Awin destination you want to sync to.
### Syncing orders
Sync data from any source to Awin orders.
#### Field mapping
Hightouch lets you sync order properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## AWS Lambda
**URL:** https://hightouch.com/docs/destinations/aws-lambda
**Description:** Build your own custom destination using a serverless function
**Section:** Destinations
## Overview
This destination invokes an [AWS Lambda](https://aws.amazon.com/lambda/) function whenever data changes in your model. It makes it possible to build your own custom integrations that execute code written in Python, Java, Go, PowerShell, Node.js, C#, or Ruby.
This destination was designed to be as flexible as possible. You can exercise granular control over function triggers, batching, rate limits, concurrency limits, and even error handling. Together, these features let you integrate Hightouch with any internal tool or third-party API.
Under the hood, the AWS Lambda destination uses Hightouch's powerful sync engine, so you continue to benefit from the security and observability features available in our native SaaS destinations.
Example use cases include:
- Syncing data to web APIs not yet natively supported
- Enriching data using external sources like Clearbit and ZoomInfo
- Transforming and filtering data using code instead of SQL
## Getting started
### Connect to your AWS account
When setting up the AWS Lambda destination for the first time, you need to enter your **AWS Credentials** to give Hightouch access to your AWS account. Hightouch needs permission to invoke Lambda functions on your behalf.
Hightouch believes in the principle of least privilege. We ask for no more permissions than absolutely necessary. For the AWS Lambda destination, Hightouch requires `lambda:InvokeFunction` for the specific function or functions you want to invoke.
When configuring your credentials in Hightouch, you have two options:
- Setting up a cross-account role (recommended)
- Providing an access key
#### Set up cross-account role
Cross-account roles are the most secure method for granting Hightouch access to specific Lambda functions in your AWS account.
You need the **Account ID** and **External ID** in the Hightouch UI to set up a cross-account role.
1. In the **AWS Console**, navigate to **IAM** → **Roles**.
1. Click **Create Role**.
1. Under **Trusted entity type**, choose **AWS account**.
1. Select **Another AWS account** and also click **Require external ID**.
1. Copy/paste the **Account ID** from Hightouch into the **Account ID** field in AWS.
1. Copy/paste the **External ID** from Hightouch into the **External ID** field in AWS.
1. On the **Add permissions** screen, proceed to attach permissions to the role using any policy that includes at least `lambda:InvokeFunction`. Then create the role.
1. Copy/paste the **Role ARN** from AWS into Hightouch. Click **Create** to finish setting up your cross-account role.
#### Provide access key
If you don't want to create a cross-account role, you can create a regular IAM user and share a **Access Key ID** and **Secret Access Key** with Hightouch.
Be sure to attach the user to a permission policy that includes `lambda:InvokeFunction` for the specific function or functions you want to invoke via a Hightouch sync.
An example policy looks like this:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["lambda:InvokeFunction"],
"Resource": [
"YOUR_LAMBDA_ARN_1",
"YOUR_LAMBDA_ARN_2",
"YOUR_LAMBDA_ARN_3"
]
}
]
}
```
## Syncing data
Once you've authenticated your AWS account in Hightouch, you've completed setup for a Lambda destination in your Hightouch workspace. The next step is to configure a sync that invokes your function whenever rows are added, changed, or removed in your model.
### Specify your Lambda function
To get started, you need the ARN of the Lambda function you want to invoke. You can find the ARN in the AWS web console.
When providing an ARN, you can use either a qualified ARN or an unqualified ARN. A qualified ARN has a version suffix, for example, `arn:aws:lambda:aws-region:acct-id:function:helloworld:42`. Unqualified ARNs don't have a version suffix, for example, `arn:aws:lambda:aws-region:acct-id:function:helloworld`.
Enter your ARN into your sync configuration in Hightouch.
### Choose your function triggers
Hightouch monitors your data model for added, changed, and/or removed rows. In this step, you specify which of these events should invoke your function.
Suppose you want to use AWS Lambda to send a confirmation email whenever a customer places a new order or updates an existing order. Your model might look something like `SELECT * FROM orders`. The Lambda function should be triggered whenever rows are added—for example, when customers place new orders—or whenever rows change—when orders are updated. Therefore, you would want to enable the **Rows added** and **Rows changed** triggers. When invoking your function, Hightouch passes along metadata about *why* the function was invoked.
All function invocations are synchronous, meaning that Hightouch invokes your Lambda function and waits for its response.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows, such as hydrating a CRM for the first time, may require performing a backfill of all rows during the initial sync. For other use cases, such as sending confirmation emails, you might only want to invoke your function in response to future data changes.
### Configure batch size
By default, Hightouch separately invokes your Lambda function for each added, changed, or removed row.
Certain high-throughput use cases may require batching together multiple rows in the payload. Hightouch supports batches of up to 1000 rows per function invocation.
Batching isn't recommended unless absolutely necessary for performance reasons. Enabling this feature requires your Lambda function to be either idempotent or significantly more fault tolerant in the event of a partial batch failure.
### Configure rate and concurrency limits
In this step, you declare whether function invocations should be throttled.
To determine the ideal limits, you should consider whether your function interacts with any downstream services that have limits of their own.
Most modern web APIs enforce rate limits, which set a maximum allowed number of requests per second, minute, or hour. Occasionally, APIs may also have concurrency limits, which set a maximum allowed number of requests that can be processed simultaneously. Rate limits and concurrency limits both affect overall sync speed.
### Configure error handling
Lambda functions can fail for many reasons. Hightouch's retry logic for this destination doesn't discriminate between invocation errors, for example, missing permissions, runtime errors, for example, due to syntax issues, or function errors, for example, due to uncaught exceptions. Hightouch retries all errors eventually.
In this step, you decide whether errors should be retried immediately or during the next sync run. If you choose to retry immediately, you can specify how many retries should be attempted during the sync run. If all these retries fail, Hightouch retires the request during subsequent sync runs until it succeeds.
If an error is gracefully caught in your function, you can flag it as a rejected row and provide a custom error message using the response format specified in the [error handling section](#configure-error-handling).
You can also include retry logic inside of your Lambda function. After exhausting all retry attempts during a function invocation, respond with an error so that Hightouch knows to reinvoke the function later.
### Invocation payload schema
The payload, also known as the "event document," contains your row data in JSON format, along with additional metadata. When writing your Lambda function, you can assume that all function invocations include a payload following this schema:
```json
{
"operation": XXXXX,
"primary_key_column": XXXXX,
"rows": [
{
XXXXXX
}
],
"metadata": {
"api_version": XXXXX,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
#### Invocation payload properties
##### `operation`
The row operation, also known as the trigger, explains *why* your function was invoked.
Its possible values are these strings: `"add"`, `"change"`, and `"remove"`.
##### `primary_key_column`
This string refers to the column name of the primary key for your Hightouch model.
##### `rows`
This is an array of objects, with each object containing row data from your Hightouch model.
##### `metadata`
This is an object containing three key/value pairs:
- `api_version` (integer): currently set to `1`, this identifier only changes if the payload schema is modified in the future
- `sync_id` (integer): this is the unique identifier for the Hightouch sync configuration associated with the function invocation
- `sync_run_id` (integer): this is the unique identifier for the specific sync run associated with the function invocation
#### Example invocation payload
Suppose a new row is added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `john.doe@example.com` | John | Doe |
The payload for the function invocation would look like this:
```json
{
"operation": "add",
"primary_key_column": "customer_id",
"rows": [
{
"customer_id": "928713",
"email": "john.doe@example.com",
"first_name": "John",
"last_name": "Doe"
}
],
"metadata": {
"api_version": 1,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
Note that `rows` is an **array** containing a single element.
#### Example invocation payload for batching
Suppose three new rows are added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `alice.doe@example.com` | Alice | Doe |
| 283743 | `bob.doe@example.com` | Bob | Doe |
| 162352 | `carol.doe@example.com` | Carol | Doe |
The payload for the function invocation would look like this:
```json
{
"operation": "add",
"primary_key_column": "customer_id",
"rows": [
{
"customer_id": "928713",
"email": "alice.doe@example.com",
"first_name": "Alice",
"last_name": "Doe"
},
{
"customer_id": "283743",
"email": "bob.doe@example.com",
"first_name": "Bob",
"last_name": "Doe"
},
{
"customer_id": "162352",
"email": "carol.doe@example.com",
"first_name": "Carol",
"last_name": "Doe"
}
],
"metadata": {
"api_version": 1,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
When batching is enabled, rows sharing a common operation type—`add`, `change`, `remove`: are batched together. Function invocations always represent exactly one operation type.
### Response payload schema
If your Lambda function fails to process any rows, it can respond with an array of "rejected rows" that encountered errors. Each rejected row should be identified by its primary key/value and may be associated with an optional error message. Hightouch retries these rows.
If there is an error, the response payload looks like this:
```json
{
"errors": [
{
"primary_key_value": XXXXX,
"reason": XXXXX
}
]
}
```
#### Response payload properties
Errors in the `errors` array have these properties:
##### `primary_key_value`
This identifies the rejected row that needs to be retried.
##### `reason` (optional)
This string may represent an error message or any other information that would be helpful for debugging and monitoring sync health.
#### Example success response payload
If all rows are processed successfully, your function responds with an empty payload.
```json
{}
```
#### Example response payload for rejected rows
Suppose three new rows are added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `alice.doe@example.com` | Alice | Doe |
| 283743 | `bob.doe@example.com` | Bob | Doe |
| 162352 | `carol.doe@example.com` | Carol | Doe |
While processing these rows, your Lambda function encounters two errors:
- Alice's email can't be found in the downstream service
- Bob's email already exists in the downstream service
To surface these errors in Hightouch, your function should respond like this:
```json
{
"errors": [
{
"primary_key_value": "928713",
"reason": "Email not found"
},
{
"primary_key_value": "283743",
"reason": "Email already exists"
}
]
}
```
The primary key values (`928713` and `283743`) refer to `customer_id` values. The `customer_id` column was designated the `primary_key_column` in the function invocation payload.
#### Example response payload for other errors
If your function experiences an invocation error, for example, exceeding AWS Lambda rate limits, runtime error, for example, due to syntax issues, or function error, for example, due to uncaught exceptions, all rows in the batch are automatically marked as rejected rows.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Azure Blob Storage
**URL:** https://hightouch.com/docs/destinations/azure-blob
**Description:** Transfer data at scale from your warehouse to Azure Blob Storage
**Section:** Destinations
## Overview
With Hightouch, you can export all the data in your warehouse to JSON, CSV, or Parquet files in any of your Azure Blob Storage containers. Whether it's for internal processes, analytics, or product data, our Azure Blob Storage integration allows you to securely and easily transfer data at web-scale.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------ | ---------------------------------------------------------------------------- | -------------------- |
| Any data set | Sync data from a source to Azure Blob Storage as JSON, CSV, or Parquet files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Diff mode creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Azure Blob Storage
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Azure Blob Storage** and click **Continue**. You can then authenticate Hightouch to **Azure Blob Storage** using Azure credentials. You can select credentials you previously created or click **New credentials** to set them up.
See the guide for [configuring Azure credentials](/security/azure) to learn how you can set up a service account and use its credentials. The IAM user whose credentials you use must have programmatic access enabled and permission to write to the Azure Blob Storage container you want to use.
Next, enter your Blob Storage **Account name** and **Container name** to complete setup.
You can find these in the [Azure Portal](https://portal.azure.com/).
Go to **Storage accounts** and open the storage account you want to use.
The account name is in the top left above **Storage account**.
You can see your container names once you open **Containers** in the left side menu.
If you want to create a new container for Hightouch to sync to, you need [create it](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-portal#create-a-container) before you can proceed.
## Sync configuration
Once you've set up your Azure Blob Storage destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Azure Blob Storage destination you want to sync to.
### Select file format
Hightouch supports syncing JSON, CSV, and Parquet files to Azure Blob Storage.
### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it. To view different versions of the same file in Azure Blob Storage, you can right-click the file and select **View versions**.
If you are using an audience and would like to include the audience name, you will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter `-86400` (`24 hours * 60 minutes * 60 seconds`). If you want the filename's data to be one hour after the sync takes place, you would enter `3600` (`60 minutes * 60 seconds`).
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `first_name`, `email`, and `last_name` fields. Hightouch exports these fields to new fields in the file and ignores all other columns from your results.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Azure Functions
**URL:** https://hightouch.com/docs/destinations/azure-functions
**Description:** Build your own custom destination using a serverless function
**Section:** Destinations
## Overview
This destination invokes a [Microsoft Azure](https://azure.microsoft.com/en-us/products/functions/) function whenever data changes in your model. It makes it possible to build your own custom integrations that execute code written in any language that Azure functions [supports](https://learn.microsoft.com/en-us/azure/azure-functions/supported-languages).
This destination was designed to be as flexible as possible. You can exercise granular control over function triggers, batching, rate limits, concurrency limits, and even error handling. Together, these features let you integrate Hightouch with any internal tool or third-party API.
Under the hood, the Azure functions destination gives you Hightouch's powerful sync engine, so you continue to benefit from the security and observability features available in our native SaaS destinations.
Example use cases include:
- Syncing data to web APIs not yet natively supported
- Enriching data using external sources like Clearbit and ZoomInfo
- Transforming and filtering data using code instead of SQL
## Getting started
### Connect to your Azure functions
When setting up the Azure functions destination for the first time, you need to enter your function's [URL](https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-http-webhook-trigger?tabs=in-process%2Cfunctionsv2&pivots=programming-language-javascript#customize-the-http-endpoint) and it's [access key](https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-http-webhook-trigger?tabs=in-process%2Cfunctionsv2&pivots=programming-language-javascript#authorization-keys) for authorizing invocations.
## Syncing data
Once you've connected to your Azure function in your Hightouch workspace, the next step is to configure a sync that invokes your function whenever rows are added, changed, or removed in your model.
### Choose your function triggers
Hightouch monitors your data model for added, changed, and/or removed rows. In this step, you specify which of these events should invoke your function.
Suppose you want to use an Azure function to send a confirmation email whenever a customer places a new order or updates an existing order. Your model might look something like `SELECT * FROM orders`. The Azure function should be triggered whenever rows are added—for example, when customers place new orders—or whenever rows change—when orders are updated. Therefore, you would want to enable the **Rows added** and **Rows changed** triggers. When invoking your function, Hightouch passes along metadata about *why* the function was invoked.
All function invocations are synchronous, meaning that Hightouch invokes your Azure function and waits for its response.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows, such as hydrating a CRM for the first time, may require performing a backfill of all rows during the initial sync. For other use cases, such as sending confirmation emails, you might only want to invoke your function in response to future data changes.
### Configure batch size
By default, Hightouch separately invokes your Azure function for each added, changed, or removed row.
Certain high-throughput use cases may require batching together multiple rows in the payload. Hightouch supports batches of up to 1000 rows per function invocation.
Batching isn't recommended unless absolutely necessary for performance reasons. Enabling this feature requires your Azure function to be either idempotent or significantly more fault tolerant in the event of a partial batch failure.
### Configure rate and concurrency limits
In this step, you declare whether function invocations should be throttled.
To determine the ideal limits, you should consider whether your function interacts with any downstream services that have limits of their own.
Most modern web APIs enforce rate limits, which set a maximum allowed number of requests per second, minute, or hour. Occasionally, APIs may also have concurrency limits, which set a maximum allowed number of requests that can be processed simultaneously. Rate limits and concurrency limits both affect overall sync speed.
### Configure error handling
Azure functions can fail for many reasons. Hightouch's retry logic for this destination doesn't discriminate between invocation errors, for example, missing permissions, runtime errors, for example, due to syntax issues, or function errors, for example, due to uncaught exceptions. Hightouch retries all errors eventually.
In this step, you decide whether errors should be retried immediately or during the next sync run. If you choose to retry immediately, you can specify how many retries should be attempted during the sync run. If all these retries fail, Hightouch retires the request during subsequent sync runs until it succeeds.
If an error is gracefully caught in your function, you can flag it as a rejected row and provide a custom error message using the response format specified in the [error handling section](#configure-error-handling).
You can also include retry logic inside of your Azure function. After exhausting all retry attempts during a function invocation, respond with an error so that Hightouch knows to reinvoke the function later.
### Invocation payload schema
The payload, also known as the "event document," contains your row data in JSON format, along with additional metadata. When writing your Azure function, you can assume that all function invocations include a payload following this schema:
```json
{
"operation": XXXXX,
"primary_key_column": XXXXX,
"rows": [
{
XXXXXX
}
],
"metadata": {
"api_version": XXXXX,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
#### Invocation payload properties
##### `operation`
The row operation, also known as the trigger, explains *why* your function was invoked.
Its possible values are these strings: `"add"`, `"change"`, and `"remove"`.
##### `primary_key_column`
This string refers to the column name of the primary key for your Hightouch model.
##### `rows`
This is an array of objects, with each object containing row data from your Hightouch model.
##### `metadata`
This is an object containing three key/value pairs:
- `api_version` (integer): currently set to `1`, this identifier only changes if the payload schema is modified in the future
- `sync_id` (integer): this is the unique identifier for the Hightouch sync configuration associated with the function invocation
- `sync_run_id` (integer): this is the unique identifier for the specific sync run associated with the function invocation
#### Example invocation payload
Suppose a new row is added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `john.doe@example.com` | John | Doe |
The payload for the function invocation would look like this:
```json
{
"operation": "add",
"primary_key_column": "customer_id",
"rows": [
{
"customer_id": "928713",
"email": "john.doe@example.com",
"first_name": "John",
"last_name": "Doe"
}
],
"metadata": {
"api_version": 1,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
Note that `rows` is an **array** containing a single element.
#### Example invocation payload for batching
Suppose three new rows are added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `alice.doe@example.com` | Alice | Doe |
| 283743 | `bob.doe@example.com` | Bob | Doe |
| 162352 | `carol.doe@example.com` | Carol | Doe |
The payload for the function invocation would look like this:
```json
{
"operation": "add",
"primary_key_column": "customer_id",
"rows": [
{
"customer_id": "928713",
"email": "alice.doe@example.com",
"first_name": "Alice",
"last_name": "Doe"
},
{
"customer_id": "283743",
"email": "bob.doe@example.com",
"first_name": "Bob",
"last_name": "Doe"
},
{
"customer_id": "162352",
"email": "carol.doe@example.com",
"first_name": "Carol",
"last_name": "Doe"
}
],
"metadata": {
"api_version": 1,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
When batching is enabled, rows sharing a common operation type—`add`, `change`, `remove`—are batched together. Function invocations always represent exactly one operation type.
### Response payload schema
If your Azure function fails to process any rows, it can respond with an array of "rejected rows" that encountered errors. Each rejected row should be identified by its primary key/value and may be associated with an optional error message. Hightouch retries these rows.
If there is an error, the response payload looks like this:
```json
{
"errors": [
{
"primary_key_value": XXXXX,
"reason": XXXXX
}
]
}
```
#### Response payload properties
Errors in the `errors` array have these properties:
##### `primary_key_value`
This identifies the rejected row that needs to be retried.
##### `reason` (optional)
This string may represent an error message or any other information that would be helpful for debugging and monitoring sync health.
#### Example success response payload
If all rows are processed successfully, your function responds with an empty payload.
```json
{}
```
#### Example response payload for rejected rows
Suppose three new rows are added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `alice.doe@example.com` | Alice | Doe |
| 283743 | `bob.doe@example.com` | Bob | Doe |
| 162352 | `carol.doe@example.com` | Carol | Doe |
While processing these rows, your Azure function encounters two errors:
- Alice's email can't be found in the downstream service
- Bob's email already exists in the downstream service
To surface these errors in Hightouch, your function should respond like this:
```json
{
"errors": [
{
"primary_key_value": "928713",
"reason": "Email not found"
},
{
"primary_key_value": "283743",
"reason": "Email already exists"
}
]
}
```
The primary key values (`928713` and `283743`) refer to `customer_id` values. The `customer_id` column was designated the `primary_key_column` in the function invocation payload.
#### Example response payload for other errors
If your function experiences an invocation error, for example, exceeding Azure function rate limits, runtime error, for example, due to syntax issues, or function error, for example due to uncaught exceptions, all rows in the batch are automatically marked as rejected rows.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Batch
**URL:** https://hightouch.com/docs/destinations/batch
**Description:** Build smarter CRM campaigns on Batch with real-time customer data from your Data Warehouse
**Section:** Destinations
## Overview
This integration lets you send server-to-server events, attributes, and custom audiences to Batch without writing a custom pipeline.
## Supported syncing
| Type | Description | Supported Sync Modes |
| ------------------- | --------------------------------------------- | -------------------- |
| **Attributes** | Sync data from any source to your Batch users | Upsert |
| **Events** | Sync records as events to Batch | Insert |
| **Custom Audience** | Sync users to a custom audience in Batch | Add, Remove |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Batch
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Batch** and click **Continue**. You can then authenticate Hightouch to **Batch**.
To get your API and project keys, navigate to `Settings` and copy your REST API key and Project key into Hightouch.
If you plan on using custom audience syncing, you also need to provide your **SDK Key**. You can find this in your Batch settings page.
## Sync configuration
Once you've set up your **Batch** destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Batch destination you want to sync to.
### Events
Any records added to [your source](/getting-started/concepts) are treated as new events and are sent to Batch when your sync runs.
`Event name` and`Custom ID` fields are the minimum required fields to make the API call to Batch. For richer analytics, we recommend that you provide all relevant fields in your use case.
The `Custom ID` field is used to deduplicate events sent to Batch.
If you don't map the `timestamp` field, it will default to the time Hightouch processes the event.
### Attributes
In this mode each record in [your source](/getting-started/concepts) is treated as a user and sent to Batch when your sync runs.
#### Record matching
You can match rows from your model to users in Batch on any column in your model and any field in Batch. Ensure the data types of the model column and Batch field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync user fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to any user attribute. For a reference of natively supported user fields see [Batch's docs](https://doc.batch.com/api/profile/update/#native-attributes).
If you send data for a custom field, Hightouch adds the field and automatically detects its type.
### Custom Audiences
In this mode, you can add or remove users from a custom audience in Batch.
#### Record matching
You can match rows from your model to users in Batch on any column in your model and any field in Batch. Ensure the data types of the model column and Batch field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Audience Selection
You must select the custom audience you want to sync users to. You can select an existing audience from the dropdown or create a new one.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Bazaarvoice
**URL:** https://hightouch.com/docs/destinations/bazaarvoice
**Description:** Drive sales with the most trusted network of authentic product reviews.
**Section:** Destinations
## Overview
This destination lets you send customer transaction data from your data warehouse to Bazaarvoice. This enables you to automatically submit transaction data for review and rating collection, helping you gather authentic customer feedback and display reviews on your products.
## Supported syncing
|Sync Type|Description|Supported Sync Modes|
|--|--|--|
|**Objects**|Sync transaction data from any source to Bazaarvoice transactions|Insert|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Bazaarvoice
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Bazaarvoice** and click **Continue**.
You need to provide the following credentials:
- **Client ID**: Your Bazaarvoice OAuth Client ID. You can obtain this from the Bazaarvoice Developer Portal or from your Bazaarvoice representative.
- **Client Secret**: Your Bazaarvoice OAuth Client Secret. You can obtain this from the Bazaarvoice Developer Portal or from your Bazaarvoice representative.
- **API Environment**: Select whether to use the Production or Staging environment.
### Create API credentials
1. Log in to the [Bazaarvoice Developer Portal](https://developer.bazaarvoice.com/).
2. Navigate to your application settings or contact your Bazaarvoice representative to obtain API credentials.
3. Create or access your OAuth Client ID and Client Secret credentials.
4. Copy your **Client ID** and **Client Secret**.
5. Ensure your credentials are activated. If you see a "Developer Inactive" error, contact Bazaarvoice support to activate your credentials.
## Sync configuration
Once you've set up your Bazaarvoice destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Bazaarvoice destination you want to sync to.
### Objects
Hightouch supports syncing to Bazaarvoice transactions with the following sync mode:
|Sync Type|Supported Sync Modes|
|--|--|
|**Transactions**|Insert|
#### Configuration
Enter the following field into Hightouch:
- **Client Name**: Your Bazaarvoice client instance name (e.g., `mycompany-en_us`, `mycompany-fr_fr`). This identifies which Bazaarvoice deployment zone to send transactions to.
#### Field mapping
Hightouch lets you sync transaction fields via [field mapping](/syncs/mapping-data). You can map data from any of your model columns to the Bazaarvoice transaction fields.
Bazaarvoice requires the following fields, so you must map them to complete your configuration:
**Required fields:**
- **Transaction ID**: A unique identifier for the transaction
- **Email Address**: The customer's email address
- **User ID**: A unique identifier for the user
- **Customer Name** (nickname): The customer's display name
- **Locale**: The locale code (e.g., `en_US`, `fr_FR`)
- **Transaction Date**: The date and time of the transaction
- **Deployment Zone**: The Bazaarvoice deployment zone identifier
- **Products**: An array of product objects. Each product must include:
- `id`: Product identifier (required)
- `name`: Product name (required)
- `imageUrl`: Product image URL (required)
- `money`: Object with `amount` (number) and `currency` (string)
- `incentivizedReview`: Optional string field
**Optional fields:**
- **Transaction Channel**: The channel through which the transaction occurred
Use the [inline mapper](/syncs/mapping-data#inline-mapping) to structure the products array. The products field requires an array of objects, so you'll need to use the inline mapper to transform your source data into the required format.
#### Batch processing
Bazaarvoice processes transactions in batches of up to 200 transactions per request. Hightouch automatically batches your transactions to optimize API usage and ensure efficient data transfer.
#### Error handling
The Bazaarvoice API returns a 207 Multi-Status response, which means some transactions in a batch may succeed while others fail. Hightouch automatically handles these partial failures and reports which specific transactions failed, allowing you to troubleshoot individual issues without affecting successful transactions.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Beeswax
**URL:** https://hightouch.com/docs/destinations/beeswax
**Description:** Combine best-in-breed solutions from across the programmatic ecosystem with your proprietary data, algorithms, and strategies into a seamless whole
**Section:** Destinations
## Setup
After selecting Beeswax from our destinations catalog, you will be required to enter your _Beeswax Subdomain_,
_Email_, and _Password_ from your Beeswax account. Your _Beeswax Subdomain_ is the text before `.beeswax.com` in
the URL, _Username_ and _Password_ are the credentials you use to login to Beeswax.
Optionally you can also add a salt that is provided by Beeswax that will be used when hashing data used for [HEM](#hem) identifiers.
## Syncing segments
Hightouch supports automatically creating a new segment for your sync. You can optionally assign a custom name to your segment.
If no name is provided, Hightouch will default to using your model name.
Here, the model name `User Segment 1` will be used as the segment name:
Additionally, instead of creating a new segment, Hightouch can also sync users to existing segments that are already populated in your Beeswax account.
### Sync modes
This integration supports Segment mode. New records will be inserted into a Beeswax segment. Records that are removed from the model will be removed from the
segment.
### Select Beeswax sync options
Hightouch allows you to choose which Beeswax account and data center you wish to sync data to. This can be selected on the sync configuration page.
### Record mappings
Records can be matched from your source to your segment by `Beeswax ID`, `Customer ID`, `HEM`, `IDFA`, `IDFA_MD5`, `IDFA_SHA1`, `AD_ID`, `AD_ID_MD5`, `AD_ID_SHA1`, or `IP_ADDRESS`.
#### HEM
When using HEM as the matching type, the mapped value will automatically be hashed based on the [docs](https://hub.freewheel.tv/display/BW/Hashed+Email+Addresses+in+FreeWheel+Identity+Network).
You can turn this off using the checkbox if your IDs are already hashed.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## BigCommerce
**URL:** https://hightouch.com/docs/destinations/bigcommerce
**Description:** Seamlessly create unique storefronts for each facet of your business, all managed through one BigCommerce dashboard.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes |
| ------------- | ----------------------------------------------------------- | -------------------- |
| **Orders** | Create and update orders, including updating their statuses | Update, Insert |
| **Customers** | Create and update customers with data from your warehouse | Upsert, Update, Insert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to BigCommerce
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination**
button. Select **BigCommerce** and click **Continue**. You can then authenticate Hightouch to **BigCommerce** by entering the following required fields into Hightouch:
- **Access token**: This is the access token for your BigCommerce API account. It should have read/write access to orders and customers.
- **Store hash**: This is the store hash for your BigCommerce store. You can find it in your control panel URL after `store-` or in your API path after `stores/`..
You can find more information on how to generate an access token [in BigCommerce's docs](https://developer.bigcommerce.com/api-docs/getting-started/authentication/rest-api-authentication#obtaining-store-api-credentials).
## Sync configuration
Once you've set up your BigCommerce destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and
the BigCommerce destination you want to sync to.
### Syncing orders
Create and update orders, including updating their statuses.
#### Record matching
To match rows from your model to records in BigCommerce, you need to select the model column that contains values that match the BigCommerce **ID** field.
In **Insert** mode, BigCommerce automatically generates an identifier for
every new record synced, so there's no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to native and custom fields in BigCommerce. Ensure the data type of your model column matches the data type of the field you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ------------------------------------------------------------------ |
| **Do nothing** | Keep the customer record in BigCommerce with all its synced fields |
| **Clear** | Unset all the mapped fields from BigCommerce |
| **Delete** | Delete the synced record from your BigCommerce order |
### Syncing customers
Create and update customers with data from your warehouse.
#### Record matching
To match rows from your model to profiles in BigCommerce, you need to select a model column and corresponding BigCommerce field. You can match on any of the following BigCommerce fields:
- **ID**
- **Email**
In **Insert** mode, BigCommerce automatically generates an identifier for
every new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to native and custom fields in BigCommerce. Ensure the data type of your model column matches the data type of the field you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ------------------------------------------------------------------ |
| **Do nothing** | Keep the customer record in BigCommerce with all its synced fields |
| **Clear** | Unset all the mapped fields from BigCommerce |
| **Delete** | Delete the synced record from your BigCommerce customer |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Microsoft Ads (Bing Ads)
**URL:** https://hightouch.com/docs/destinations/bingads
**Description:** Ensure that you never serve another irrelevant advertisement with Bing Ads
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| ------------- | -------------------------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------- |
| [**Events**](#syncing-events) | Sync offline conversion events with data from any source | Insert | [Offline conversion reference](https://learn.microsoft.com/en-us/advertising/bulk-service/offline-conversion) |
| [**Audiences**](#syncing-audiences) | Create new or use an existing audience to sync users | Add, Remove | [Customer list item reference](https://learn.microsoft.com/en-us/advertising/bulk-service/customer-list-item) |
## Connect to Bing Ads
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Bing Ads** and click **Continue**. You can then authenticate Hightouch to Bing Ads using OAuth.
Click **Log in to Microsoft Bing Ads** , log in with your Microsoft Ads account, and click **Allow**.
Hightouch requires access to manage your ads account and see your email address.
Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Bing Ads destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Bing Ads destination you want to sync to.
Whether you're syncing events or audiences, begin your configuration by selecting the Microsoft Ads account you want to sync data to.
### Syncing events
When syncing offline conversion events, Hightouch treats any records added to [your source](/getting-started/concepts) as new events and sends them to Bing Ads when your sync runs.
#### Conversion goal name
Select the [conversion goal name](https://help.ads.microsoft.com/#apex/ads/en/56689/2) to associate with the offline conversions.
Your options populate depending on the Microsoft Ads account you selected at the beginning of the sync configuration.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server. Hightouch syncs the timestamp as the `ConversionTime` parameter the Microsoft Advertising API requires.
The timestamp field must also:
- be in Coordinated Universal Time (UTC)
- be later than the recorded **Microsoft Click ID**
- be within the conversion goal window
- be within the last 90 days
For more information, see [Microsoft Bing Ads' official documentation](https://learn.microsoft.com/en-us/advertising/bulk-service/offline-conversion).
#### Field mapping
Hightouch lets you sync event fields via field mapping.
You can map data from any of your model columns to [supported event fields](https://learn.microsoft.com/en-us/advertising/bulk-service/offline-conversion?view=bingads-13).
The Microsoft Ads API requires the **Microsoft Click ID** field, so you must map it to complete your configuration.
Ensure your model columns data types match the data types of the fields you want to sync to.
### Syncing audiences
You can use Hightouch to insert and update audience members in a the Bing Ads customer match list audience.
Users removed from the model get removed from the audience.
To sync to a customer match list audience, you need to first create a customer
list and accept terms and conditions in the Microsoft Advertising UI.
#### Select an existing audience or create a new one
You can create a new audience or use an existing one. To use an existing audience, select the desired audience from the dropdown. When creating a new audience, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model.
By default, the audience will be shareable across all accounts, but you can choose to limit access to the audience to only the account creating the audience.
#### User identifiers
Bing Ads requires you to identify which users to add or update in an audience on a valid **Email**.
Select the model column that contains user emails.
Bing Ads doesn't support mapping any other fields.
For more information, see [Microsoft Bing Ads' official documentation](https://learn.microsoft.com/en-us/advertising/bulk-service/customer-list-item?view=bingads-13#subtype).
#### Handling PII
By default, Hightouch automatically detects if your **Email** field requires hashing and hashes the value before sending requests to Bing Ads if needed.
## Tips and troubleshooting
### Matched users count
Below only applies to the audiences sync type.
### Common errors
{/* */}
#### 8003: CustomerListTermsAndConditionsNotAccepted
To sync to a customer match list audience, you need to first create a customer list and accept terms and conditions in the Microsoft Advertising UI.
For more information, read the [Bing Ads documentation](https://learn.microsoft.com/en-us/advertising/bulk-service/customer-list-item?view=bingads-13).
#### Error uploading audience XXX, bulk operation failed: - (4835) - InvalidAudienceId in path
Check the Bing Ads UI to make sure the audience from your sync configuration exists. If the audience no longer exists, you need to create a new sync.
For additional errors, read the Bing Ads [error codes documentation](https://learn.microsoft.com/en-us/advertising/guides/operation-error-codes?view=bingads-13).
{/* */}
### Live debugger
### Sync alerts
---
## Bird
**URL:** https://hightouch.com/docs/destinations/bird
**Description:** The complete platform for enterprise brands to acquire, convert, and retain customers across every channel—powered by AI that actually understands your business.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-------------|--------------------------------------------|----------------------|------------------------------------------------------------------------------------------------
**Contacts** | Sync data from any source to Bird contacts | Upsert, Update | [Contacts docs](https://docs.bird.com/api/contacts-api/api-reference/manage-workspace-contacts)
**Segments** | Sync data from any source to Bird segments | Add | [Segments docs](https://docs.bird.com/api/contacts-api/api-reference/manage-contact-lists)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Bird
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Bird** and click **Continue**. You can then authenticate Hightouch to **Bird**.
Enter the following fields into Hightouch:
- **Workspace ID**
- **Access key**
## Sync configuration
Once you've set up your Bird destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Bird destination you want to sync to.
### Syncing contacts
Sync data from any source to Bird contacts.
#### Record matching
You can match rows from your model to contacts in Bird on any column in your model and any contact identifier in Bird. By default, Bird supports `contactId` and `emailaddress` as identifiers, and you can add new identifier types as well. Hightouch supports matching by `contactId` and `emailaddress`, and if you have an existing custom contact identifier, you can set it in the id mapping section.
Ensure the data types of the model column and Bird field you select match. Refer to the Bird [contact identifier docs](https://docs.bird.com/api/contacts-api/api-reference/manage-contact-identifiers) and the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom contact fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
Contact attributes are mapped in the second section.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|----------------------------------------------------------
**Do nothing** | Keep the contact in Bird with all its synced fields
**Clear** | Clear all the mapped fields, but keep the contact in Bird
**Delete** | Delete the synced contacts from Bird
### Syncing segments
Sync data from any source to Bird segments.
#### Record matching
To match rows from your model to segments in Bird, you need to select the model column that contains values that match the **Contact ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync segment fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default segment fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Split retries
Bird counts all segments in a batch as rejected if the request contains a single invalid segment. To pinpoint which segments are getting rejected with which errors and reduce the number of valid segments that get retried, you can enable [split retries](/syncs/retries#split-retries).
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Bloomreach
**URL:** https://hightouch.com/docs/destinations/bloomreach
**Description:** Bloomreach helps companies personalize customer experiences by targeting the right customers with the right experience at the right time.
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|**Events**|Sync data from any source to any events|Insert|
|**Customers**|Sync data from any source to the customer objects|Upsert|
|**Catalogs**|Sync data from any source to catalog items|Upsert|
## Getting started
To use the Bloomreach destination in Hightouch, you need to provide a Bloomreach API Key, API Secret, API URL, and Project Token. Follow these steps to retrieve your credentials:
1. In Bloomreach, click the wheel icon in the top right corner. Select **Project Settings** from the dropdown.
2. Click on **Access management** in the side navigation. Select **API** from the options.
3. Your **Project token** and **API Base URL** are here. You can either use an existing **API Key ID** and **API Secret**, or you can click **+ Add key** to create another one.
## Syncing data
### Events
When syncing events, Hightouch treats any records added to [your source](/getting-started/concepts) as new events and sends them to Bloomreach when your sync runs.
You need to select the type of event and one or more hard and/or soft customer IDs to make the API call to Bloomreach.
The most common customer IDs are `registered`.
#### Field mapping
One or more of the event properties, also known as attributes, are required. The event properties show up as custom mappings and attach custom data relevant to your event.
Custom mappings are included as the [`properties`](https://documentation.bloomreach.com/engagement/reference/add-event-2#body-parameters)
parameter of the API request payload.
For more information about event properties, refer to [Bloomreach's documentation](https://documentation.bloomreach.com/engagement/reference/add-event-2).
### Customers
You can also sync to the **Customers** object. This integration supports **Upsert** mode, meaning you can insert new customers and update existing ones in Bloomreach.
#### Record matching
You can match rows in your model with customers in Bloomreach on one or more of the following hard and/or soft customer IDs:
- Cookie
- Email
- Registered
ID mappings are included as the [`customer_ids`](https://documentation.bloomreach.com/engagement/reference/update-customer-properties-2)
parameter of the API request payload.
#### Field mapping
You need to include at least customer property you wish to update. The customer properties show up as custom mappings and attach custom data relevant to your customer.
Custom mappings are included as the [`properties`](https://documentation.bloomreach.com/engagement/reference/update-customer-properties-2)
parameter of the API request payload.
For more information about customer properties, refer to [Bloomreach's documentation](https://documentation.bloomreach.com/engagement/reference/update-customer-properties-2).
### Catalogs
You can sync data to Bloomreach **Catalogs** to manage product or item data. This integration supports **Upsert** mode, allowing you to insert new catalog items and update existing ones.
#### Catalog selection
You can either create a new catalog or sync to an existing catalog in Bloomreach. When creating a new catalog, Hightouch automatically creates the catalog with fields based on your mappings.
#### Record matching
You must match rows in your model with catalog items using the **Item ID** field. This uniquely identifies each item in the catalog.
#### Field mapping
Map columns from your source to catalog item properties. These properties contain the data for each item in your catalog, such as product name, price, or description.
For more information about catalogs, refer to [Bloomreach's documentation](https://documentation.bloomreach.com/engagement/reference/catalog-api-overview-2).
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Bluecore
**URL:** https://hightouch.com/docs/destinations/bluecore
**Description:** Empower e-commerce brands to personalize customer communications with Bluecore, the retail marketing platform.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
------------------------|-----------------------------------------|---------------------
**Batch File Delivery** | Upload batch files to Bluecore via SFTP | Insert, All, Diff
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Bluecore
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Bluecore** and click **Continue**. You can then authenticate Hightouch to **Bluecore**.
Enter the following fields into Hightouch:
- **Host**
- **Port**
- **Username**
- **Password**
- (Optional) **Private Key**
- (Optional) **Private Key Passphrase**
## Sync configuration
Once you've set up your Bluecore destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Bluecore destination you want to sync to.
### Syncing via batch file delivery
Upload batch files to Bluecore via SFTP
#### Field mapping
Hightouch lets you sync batch file delivery fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default batch file delivery fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Box
**URL:** https://hightouch.com/docs/destinations/box
**Description:** Transfer data at scale to Box
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
| ------------ | --------------------------------------------------- | -------------------- |
| Any data set | Sync data from a source to Box as JSON or CSV files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Diff mode creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Box
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Box** and click **Continue**. You can then authenticate Hightouch to Box via OAuth.
After logging into your Box account, Hightouch prompts you to grant access to your Box account:
After authorizing, you've successfully connected Hightouch to Box.
## Sync configuration
Once you've set up your Box destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Box destination you want to sync to.
### Select file format
Hightouch supports syncing JSON and CSV files to Box.
### Select folder
Next, enter the folder ID to where the file will be synced to. You can find the folder ID by visiting the folder in the web application and copying the ID from the URL. For example, for the URL https://app.box.com/folders/123, the folder ID is 123.
### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it. To view different versions of the same file, you can right-click the file in Box. Click **Properties** and then **Version History**.
If you are using an audience and would like to include the audience name, you will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter `-86400` (`24 hours * 60 minutes * 60 seconds`). If you want the filename's data to be one hour after the sync takes place, you would enter `3600` (`60 minutes * 60 seconds`).
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `company`, `email`, `name`, and `primaryKey` fields. Hightouch exports these fields to new fields in the file and ignores all other columns from your results.
### CSV options
If you're syncing to a CSV file, you have additional configuration options:
- **Delimiter**: Your options are comma (`,`), semicolon (`;`), pipe (`|`), tilde (`~`), and tab
- (Optional) Whether to **include a CSV header row** in the exported files
- (Optional) Whether to **include a [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark)** in the exported files, the BOM is ``
### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Branch.io
**URL:** https://hightouch.com/docs/destinations/branchio
**Description:** Event tracking helps you understand how a user moves through your app and what actions they are taking along the way. Commerce, Content, and User Lifecycle events can all be tracked this way.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
|-----------------------------|-------------------------------------------------------------|----------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|
| **Events** | Sync data from any source to Branch.io events | Insert | [GitHub docs](https://github.com/BranchMetrics/branch-deep-linking-public-api?tab=readme-ov-file#logging-custom-events) and [Event docs](https://help.branch.io/developers-hub/reference/logstandardevents) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Branch.io
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Branch.io** and click **Continue**. You can then authenticate Hightouch to **Branch.io**.
Enter the following fields into Hightouch:
- **App ID**
- **Branch Key**
- **Branch Secret**
All of these fields can be found by managing the your [Branch.io profile settings](https://help.branch.io/using-branch/docs/profile-settings). The Branch Key and Branch Secret are considered sensitive information.
## Sync configuration
Once you've set up your Branch.io destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Branch.io destination you want to sync to.
### Syncing events
Sync data from any source to Branch.io events.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
At least one of the following fields must be mapped so that events can be uniquely identified with a user:
- Developer Identity
- Browser Fingerprint ID
- IDFA
- IDFV
- AAID
- Android ID
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Braze
**URL:** https://hightouch.com/docs/destinations/braze
**Description:** Build better campaigns on Braze with up-to-date customer data from your data warehouse
**Section:** Destinations
## Overview
Braze is a customer engagement platform that supports real-time and batch messaging across email, SMS, mobile, and web. Use Hightouch to sync user data, events, subscription status, and catalog information so Braze can run accurate, data-driven messaging and lifecycle campaigns.
---
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| ----------------------- | ---------------------------------------------------------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| [**Event**](#events) | Send track events and track purchase type events to Braze | Insert | [Track events ↗](https://www.braze.com/docs/api/objects_filters/event_object/) |
| [**Object**](#users) | Sync records to Braze objects like `User` or `Organization`. | Upsert, Update | [Update user data ↗](https://www.braze.com/docs/api/endpoints/user_data/) |
| [**Subscription status**](#subscription-status)| Sync SMS or email subscription groups. | Upsert | [Manage subscription groups ↗](https://www.braze.com/docs/api/endpoints/subscription_groups/) |
| [**User identity**](#merge-anonymous-and-known-users)| Merge alias-only users with an identified profile and remove the alias-only profile | Insert | [Identify users ↗](https://www.braze.com/docs/api/endpoints/user_data) |
| [**Catalog items**](#catalog-items) | Sync catalog items such as products or offers. | Upsert, Update | [Manage catalog items ↗](https://www.braze.com/docs/api/endpoints/catalogs) |
| [**Campaign triggers**](#trigger-campaigns) | Trigger campaigns with context from your records. | Insert, Update, Upsert | [Send triggered campaigns ↗](https://www.braze.com/docs/api/endpoints/messaging/send_messages/post_send_triggered_campaigns/) |
| [**Canvas triggers**](#trigger-canvases) | Trigger Canvases with context from your records. | Insert, Update, Upsert | [Send triggered Canvases ↗](https://www.braze.com/docs/api/endpoints/messaging/send_messages/post_send_triggered_canvases/) |
### How Braze uses the data you sync
Different sync types enable different workflows in Braze:
- **User attributes** update profiles and improve segmentation.
- **Events** can trigger campaigns and Canvas entries or provide context for message logic.
- **Subscription status** controls channel eligibility and opt-in choices.
- **Catalog items** populate dynamic content blocks in messages and templates.
- **User identity data** helps consolidate user profiles across devices.
- **Trigger properties** add context when activating Campaign or Canvas triggers.
---
## Connect to Braze
Connecting Braze to Hightouch requires an **API key** and **region**.
### API key
Generate or locate a REST API key in your Braze dashboard under **Settings → APIs and Identifiers → API Keys**. Create a dedicated REST API key for Hightouch rather than reusing a Partner Integration key — Partner Integration keys don't include the permissions that Hightouch requires.
#### Required permissions
The API key must include permissions for each sync type you plan to use.
| Sync Type | Required permissions |
| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Event** | `users.track`, `custom_attributes.get`, `users.export.ids` (only required if using the **Retry** [user existence behavior](#events)) |
| **Users** | `users.track`, `users.export.ids`, `users.delete` (optional if you want to rely on Hightouch for user deletions), `users.alias.update` (if you are updating existing users), `custom_attributes.get` |
| **Subscription status** | `subscription.status.set` |
| **User identification** | `users.identify` |
| **Catalog items** | `catalogs.add_items`, `catalogs.update_items`, `catalogs.get_item`, `catalogs.create`, `catalogs.get`, `catalogs.delete_items`, `catalogs.replace_items` |
| **Campaign triggers** | `campaigns.trigger.send`, `campaigns.list`, `campaigns.details` |
| **Canvas triggers** | `canvas.trigger.send`, `canvas.list`, `canvas.details` |
| **Segments** | `segments.list`, `segments.details` (only required if you use Braze with Hightouch [Agents](/agents/overview)) |
### Region
To find your Braze region, check the domain in your Braze dashboard URL and match it to the table below.
| URL | Region |
| -------------------------------- | ------------------ |
| `https://dashboard-01.braze.com` | `iad-01.braze.com` |
| `https://dashboard-02.braze.com` | `iad-02.braze.com` |
| `https://dashboard-03.braze.com` | `iad-03.braze.com` |
| `https://dashboard-04.braze.com` | `iad-04.braze.com` |
| `https://dashboard-05.braze.com` | `iad-05.braze.com` |
| `https://dashboard-06.braze.com` | `iad-06.braze.com` |
| `https://dashboard-07.braze.com` | `iad-07.braze.com` |
| `https://dashboard-08.braze.com` | `iad-08.braze.com` |
| `https://dashboard-01.braze.eu` | `fra-01.braze.eu` |
| `https://dashboard-02.braze.eu` | `fra-02.braze.eu` |
Refer to the [Braze endpoint
documentation](https://www.braze.com/docs/api/basics/?redirected=true#api-definitions)
to view additional regions. To derive the region, remove the prefix
"`https://rest`" from the respective value in the **REST ENDPOINT** column.
---
## Sync configuration
Once you've set up your Braze destination and have a [model](/getting-started/concepts#models) to pull data from, you can configure syncs to begin sending data:
1. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs).
2. Click **Add sync**.
3. Select the relevant model and the Braze destination you want to sync to.
Destination fields appear greyed out until you select a sync type and configure record matching. This is expected behavior during initial setup, not an error.
---
### Events
Hightouch supports syncing events to Braze and assigning them to a user. You can send two event types to the Braze destination:
- **Track Event:** Sends events of a particular name when a record is added
- **Track Purchase:** Track when a customer makes a purchase
When syncing events, you can configure what happens when the associated user doesn't exist in Braze yet. This is controlled by the **user existence behavior** setting:
- **Create user**: Braze automatically creates a new user profile when the event is received.
- **Skip event**: Braze won't create a new profile or track the event.
- **Retry until user exists**: Hightouch polls Braze to check whether the user exists before sending the event. If the user hasn't been created after 2 minutes, the event is rejected as retriable and retried later.
#### Record matching
Hightouch can match events to users on different Braze fields. If user existence behavior is set to **Create user**, you can only match on **Braze External ID** or **User Alias Object**. If set to **Skip event** or **Retry until user exists**, you can also match on **Braze ID**.
For `track event` type events you can also send the event name and event timestamp. `Track purchases` don't require an event name.
Hightouch accepts a standard date time format for the timestamp and
automatically converts it to the format Braze requires.
#### Field mapping
Hightouch lets you pass data to the event properties of a Braze event.
#### Event property configuration
Hightouch lets you send all your custom properties as a single object column or as multiple columns from your model.
When using a single column, make sure that the model column's [data
type](/models/data-types-casting#data-types) is **Object / Array**.
---
### Users
Hightouch supports syncing rows from your model to Braze users. This integration supports both **Upsert** and **Update** modes. In Upsert mode, Hightouch inserts new users into Braze, with all attributes kept up-to-date. In Update mode, Hightouch updates attributes on users that already exist in Braze.
Hightouch automatically normalizes some user data fields to meet Braze's specifications.
| Field name | Hightouch normalization |
| ------------- | ------------------------------------------------ |
| Date of birth | Convert from timestamps to year-month-date |
| Email | Convert to all lowercase letters |
| Phone numbers | Remove leading plus signs (`+`) and white spaces |
#### Record matching
In Upsert mode, you can match records from your source to your Braze workspace using the Braze External User ID. In Update mode, you can also match records based on Braze ID or a user alias object.
If you match records with a column **other** than the primary key of your
model, changes in the model column used for record matching will not be
reflected in Braze. This is to ensure duplicate records are not created in
Braze.
When syncing a large amount of rows, make sure to match the records using
Braze External User ID instead of Braze ID. Using the Braze ID requires more
requests which often results in slower sync speeds. See the [slow initial sync
section](#slow-initial-sync) for more details.
#### Field mapping
You can sync columns from your source to Braze's default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
You can manage your custom fields in **Manage App Group → Custom Attributes** in Braze.
You can also map your model columns to **user aliases** in Braze.
This configuration only appears if you select **Braze External ID** for [record matching](/destinations/braze#record-matching).
#### Nested objects and object arrays
Braze supports user attributes containing JSON objects or [object arrays](https://www.braze.com/docs/user_guide/data_and_analytics/custom_data/custom_attributes/array_of_objects/). These objects and object arrays are commonly used to support "entities" related to each user—for example, devices, medications, pets, etc.—especially when marketing communication needs to include metadata about these related entities.
Hightouch is a great tool to sync to these types of attributes because you can leverage your data warehouse to manage these associations. To do so, you must build the full JSON object into a single column within your Hightouch data model.
Modeling nested attributes into JSON can be done directly in your warehouse with something like [dbt](https://www.getdbt.com/), or you can use SQL functions appropriate for your data warehouse and build these objects within the [SQL Editor](/models/sql-editor) in Hightouch.
#### Nested object array example
Let's say you want an array of a customer's pets to be an attribute named `pets` in Braze. You would need a data model with a column named `pets` and would then build the full JSON pets object array to populate that column. This array might look something like this:
```json
[
{
"id": 0,
"type": "dog",
"breed": "Mutt",
"name": "Rae"
},
{
"id": 1,
"type": "cat",
"breed": "Tabby",
"name": "Garfield"
}
]
```
This full object array would need to be the value of a column in your Hightouch data model:
Then you would map this column to the `pets` custom attribute in Braze in your Hightouch sync configuration:
After scheduling your sync, you'd have relevant pet data on each user in Braze.
---
### Subscription status
Hightouch supports syncing users to both Email and SMS subscription groups.
Braze has a designated set of fields to manage subscription group opt-ins independent of user attributes.
Hightouch supports syncing to existing subscription groups within Braze. You can enter a static subscription group ID or select a column from your model containing the subscription group ID.
Read [Braze's
docs](https://www.braze.com/docs/user_guide/message_building_by_channel/email/managing_user_subscriptions/)
for information on creating a subscription group.
#### Record matching
You can match records from your source to your Braze subscription groups by Braze External User ID, Phone Number, or Email.
#### Field mapping
You can map any source fields to user fields in Braze, for example name, email, etc.
#### Delete behavior
When a user is removed from a Subscription status sync, Hightouch makes an API request to set the user's [`subscription_state`](https://www.braze.com/docs/api/endpoints/subscription_groups/post_update_user_subscription_group_status_v2/#request-body) to `unsubscribed`.
---
### Merge anonymous and known users
Hightouch supports merging alias-only "anonymous" users with "known" Braze users with an external ID.
It's common to have “soft-signup” workflows where all you collect from a user is an email or phone without forcing the user to create an account. This workflow results in users where all you may have is PII. When they create an account, you need to use this endpoint to “identify” these users with a new login ID.
For example, your workflow might look like this:
1. A user visits your website or app. The user doesn't have a Braze ID to start with—they only enter an email address, phone number, or some other PII. You create a user in you customer database, and an alias-only user profile in Braze via the Braze SDK. Hightouch isn't involved at this point.
2. You can use Hightouch to send user attributes or user-related events to Braze via Hightouch by [matching on a user alias object](#record-matching), since there isn't an external ID yet. You must use Update mode.
3. Once you have a unique user ID in your customer database, you can use Hightouch to call Braze's [Identify endpoint](https://www.braze.com/docs/api/endpoints/user_data/post_user_identify/) to "identify" the alias-only user using your unique user ID as the external ID in Braze. This merges the alias-only user in Braze with a known Braze user. If an external ID doesn't exist when using the Identify endpoint, Braze creates the external ID and the alias-only user is considered “identified.”
4. All subsequent updates to the user should be done in Hightouch using the external user ID, not the user alias.
#### Record matching
You can match and merge records using Braze user external IDs.
#### Field mapping
Hightouch lets you map your Braze aliases to known users using the alias label and value.
If you're adding a JSON object directly to your model, each [`user_alias`](https://www.braze.com/docs/api/objects_filters/user_alias_object) object should consist of `alias_label` and `alias_name` as top-level attributes, rather than nested in another object.
```javascript
# CORRECT FORMAT
"user_aliases": [
{
"alias_label": "label",
"alias_name": "name_1"
},
...
]
# INCORRECT FORMAT
"user_aliases": [
{
"user_alias" : {
"alias_name" : "label",
"alias_label" : "name_1"
}
},
,
...
]
```
---
### Catalog items
Hightouch supports syncing rows from your models to your existing catalog through the [Braze Catalogs API](https://www.braze.com/docs/api/endpoints/catalogs).
Braze's Catalog endpoint is in early access. Contact your Braze account
manager if you are interested in participating.
You can choose to create a new [Braze catalog](https://www.braze.com/docs/user_guide/personalization_and_dynamic_content/catalogs/catalog/) the first time the
sync runs or use an existing catalog. If creating a new catalog, you can name the catalog from Hightouch. If you leave this field blank, the name defaults to the
model name.
With the **Create a new catalog** option enabled, Hightouch only creates a
catalog the first time the sync runs. Hightouch doesn't modify the catalog
name after the first run. If you want to change the name or fields, you have
to create a new catalog using a new sync.
Syncing to catalog items allows either **Upsert** and **Update** modes. In Upsert mode, Hightouch inserts new catalog items into the Braze. In Update mode, Hightouch updates fields on catalog items that already exist in Braze.
#### Record matching
In Upsert and Update modes, you can match records from your source to your Braze workspace using the catalog item ID.
#### Field mapping
You can map data from your source to any existing field in your catalog.
If automatically creating a catalog, you can't add new fields or change data
types after the initial sync.
#### Testing
If you are using an existing catalog, you can test your sync by syncing a single row. You can verify that a row was successfully synced by going to your Catalog and finding the synced row by its catalog item ID. If you are automatically creating a catalog, it's not possible to test by row.
---
### Trigger campaigns
Hightouch lets you use records from your models to trigger campaigns through the [Braze Campaign API](https://www.braze.com/docs/api/endpoints/messaging/send_messages/post_send_triggered_campaigns/).
Once configured, your Hightouch sync triggers your campaign once rows appear in your model. Hightouch triggers your campaign for every row in your model the first time you run the sync, whenever you do a full resync, and for every new row added to your query results in subsequent syncs.
#### Sync modes
The sync mode controls which rows re-trigger the campaign on subsequent runs ([CDC behavior](/syncs/cdc#cdc-by-sync-mode)). It doesn't change the underlying Braze API call. To control whether messages are only sent to users that already exist in Braze, use [Send to existing only](#send-to-existing-only).
- **Insert** (default): triggers the campaign for every row the first time you run the sync and on every full resync, and for every new row added to your query results in subsequent syncs.
- **Update**: triggers the campaign only for rows that change in your model.
- **Upsert**: triggers the campaign for new rows and for rows that change in your model.
In Update and Upsert modes, a row that changes re-triggers the campaign, so
the same user can receive duplicate messages. Set [delivery
controls](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns/delivery_types/reeligibility/#canvas)
in Braze to prevent duplicates.
#### Prerequisites
Before you can set up a Hightouch sync to trigger a campaign, you need set up an [API-triggered](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns/delivery_types/api_triggered_delivery/) campaign in Braze. See [how to create an email campaign in Braze](https://www.braze.com/docs/user_guide/message_building_by_channel/email/creating_an_email_campaign/) for more information.
Hightouch can only trigger campaigns that are
[API-triggered](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns/delivery_types/api_triggered_delivery/).
These are different from [API
campaigns](https://www.braze.com/docs/api/api_campaigns/).
#### Record matching
This destination lets you trigger a campaign for your Braze users based on their external user ID.
This destination doesn't check for duplicates. Hightouch recommends that you
set [delivery
controls](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns/delivery_types/reeligibility/#canvas)
in Braze to prevent duplicated campaign messages in case your source has
duplicates.
#### Field mapping
This destination allows you to customize your messages via [API trigger properties](https://www.braze.com/docs/api/objects_filters/trigger_properties_object). Hightouch automatically lists trigger properties you can map values with. See [using the templated content included with an API request](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns/delivery_types/api_triggered_delivery/#setting-up-an-api-triggered-campaign) for more information.
#### Send to existing only
This destination allows you to create or update users when you trigger a campaign. To create new users, just uncheck the setting in the sync and map the user properties.
Note: Mapping user properties when using this setting is *required*
#### Testing
Hightouch recommends that you test with your email before running an actual sync for your users.
You can test by following these steps:
1. Create a Google Sheet with a valid `external_id` and your email. You will use this to sync yourself as user in Braze. See [Braze's user ID convention](https://www.braze.com/docs/developer_guide/platform_integration_guides/web/analytics/setting_user_ids/#suggested-user-id-naming-convention) if you are unfamiliar with `external_id`.
2. Use the Google Sheet as a source to [add yourself to Braze as a user](/destinations/braze#users).
You can use the same sync to remove yourself from Braze once your testing
is complete. Set the sync configuration's delete mode to "Delete
destination record associated with source record," remove yourself from the
source sheet and rerun the sync.
3. Build your campaign in Braze. See Braze's guide on [how to build a campaign](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns) for more information. When setting up your campaign schedule, make sure to select [API-triggered Delivery](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns/delivery_types/api_triggered_delivery/).
4. Set up your campaign trigger sync and run it on Hightouch.
5. Verify that you are able to receive your campaign message. If you need to make corrections and thus want to re-run the sync, click **Resync full query** next to the **Run** button. Also note that your [re-eligibility settings](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns/delivery_types/reeligibility/#canvas) in Braze campaign must be loosely set for you to re-trigger the sync and actually receive the email.
---
### Trigger Canvases
Hightouch lets you use records from your models to trigger Canvases through the [Braze Canvas API](https://www.braze.com/docs/api/endpoints/messaging/send_messages/post_send_triggered_canvases/).
Once configured, your Hightouch sync triggers your Canvases once rows appear in your model. Hightouch triggers your Canvases for every row in your model the first time you run the sync, whenever you do a full resync, and for every new row added to your query results in subsequent syncs.
#### Sync modes
The sync mode controls which rows re-trigger the Canvas on subsequent runs ([CDC behavior](/syncs/cdc#cdc-by-sync-mode)). It doesn't change the underlying Braze API call. To control whether messages are only sent to users that already exist in Braze, use [Send to existing only](#send-to-existing-only).
- **Insert** (default): triggers the Canvas for every row the first time you run the sync and on every full resync, and for every new row added to your query results in subsequent syncs.
- **Update**: triggers the Canvas only for rows that change in your model.
- **Upsert**: triggers the Canvas for new rows and for rows that change in your model.
In Update and Upsert modes, a row that changes re-triggers the Canvas, so the
same user can receive duplicate messages. Set [delivery
controls](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns/delivery_types/reeligibility/#canvas)
in Braze to prevent duplicates.
#### Prerequisites
Before you can set up a Hightouch sync to trigger a Canvas, you need set up an [API-triggered](https://www.braze.com/docs/user_guide/engagement_tools/canvas/create_a_canvas/create_a_canvas/#step-2b-set-your-canvas-entry-schedule) Canvas in Braze. See [Braze Ideas & Strategies](https://www.braze.com/docs/user_guide/engagement_tools/canvas/ideas_and_strategies) for more information and best practices.
Hightouch can only trigger Canvases that are
[API-triggered](https://www.braze.com/docs/user_guide/engagement_tools/canvas/create_a_canvas/create_a_canvas/#step-2b-set-your-canvas-entry-schedule).
Only non-archived, active canvases are available in sync configuration.
#### Record matching
This destination lets you trigger a Canvas for your Braze users based on their external user ID.
#### Field mapping
This destination allows you to customize your messages via [Canvas entry properties](https://www.braze.com/docs/api/objects_filters/canvas_entry_properties_object). Hightouch automatically lists Canvas entry properties you can map values to.
See [Braze's docs](https://www.braze.com/docs/user_guide/engagement_tools/canvas/create_a_canvas/canvas_entry_properties_event_properties) on using the templated content included with an API request for more information.
#### Send to existing only
This destination allows you to create or update users when you trigger a Canvas. To create new users, just uncheck the setting in the sync and map the user properties.
Note: Mapping user properties when using this setting is *required*
---
## Validate your setup
After you configure your sync, run a test to confirm data flows correctly:
1. Start with a small test audience or a single record.
2. Trigger a manual sync from the [Syncs overview](https://app.hightouch.com/syncs?from=docs) page.
3. After the sync completes, verify the data in Braze:
- For user syncs, search for the test user in the Braze dashboard under **Audience → Search Users** and confirm the mapped attributes appear with expected values.
- For event syncs, check the user's profile activity feed to confirm events were received.
- For catalog syncs, go to **Data Settings → Catalogs** and verify items were created or updated.
4. If the sync shows errors, check the sync run log in Hightouch for specific API error messages.
---
## Tips and troubleshooting
For Hightouch platform error codes related to Braze, see [Error codes: Braze](/developer-tools/error-codes#braze).
### Syncing and data point consumption
Hightouch [only syncs data that needs updating](/getting-started/concepts#change-data-capture-cdc) to minimize the number of API requests made to your destinations.
Braze offers different pricing models, and how Hightouch syncs impact your usage depends on which model you're using.
To optimize for speed instead of data point updates, you can disable data point optimization in your Hightouch sync configuration.
#### Data points based pricing
In this model, pricing is based on data points, which count the number of updates made by you (and Hightouch). Each update contributes to your total data point usage, so efficient syncing can help reduce consumption.
#### Monthly Active Users (MAU) based pricing (CY24–25 SKU)
This pricing model emphasizes data flexibility for customers. There is no concept of "data points" in this SKU. However, overall data consumption is still constrained by product limits such as API rate limits and other usage thresholds, as detailed in Braze's documentation.
#### How data points are consumed
These are some key points to understand how Hightouch syncs affect your Braze data point consumption.
When syncing arrays without nested objects, you consume one Braze [data point](https://www.braze.com/docs/user_guide/data_and_analytics/data_points/) per API call:
- Creating an array consumes one data point.
- Updating arrays consumes one data point per value updated.
When syncing [arrays of objects](https://www.braze.com/docs/user_guide/data_and_analytics/custom_data/custom_attributes/array_of_objects/), you may consume multiple data points with one API call:
- Creating an array of objects consumes one data point _per attribute added_ within each object. For example, creating two `pet` objects with four attributes each (`id`, `type`, `color`, and `size`) consumes eight data points.
- Updating an array of objects consumes one data point _per attribute changed_.
- Removals consume one data point _per removal criteria_. For example, removing objects where `id = 4` removes one object and costs one data point. Removing objects where `type = dog` may remove many objects but still only consumes one data point.
When syncing records that don't exist in Braze, Hightouch updates **all** mapped attributes with the values from your query results. This means you consume one data point per attribute per created record.
When syncing new records in the model that already exist in Braze, Hightouch first gets the user attributes from Braze, calculates which columns actually changed, and then only sends the changes for those columns. Each updated attribute equals one data point consumed.
When syncing records that have been previously synced to Braze, Hightouch compares the new record's columns to the previously sent record's columns, calculates which columns actually changed, and then only sends the changes for those columns. Each updated attribute equals one data point consumed.
Hightouch sync strategy aims for the lowest Braze data point consumption
possible. Because Hightouch makes a `GET` request to first see what user attributes
haven't changed since the last sync, sync speed is slower than other methods
which overwrite all data in Braze and consume more data points.
---
### Required permissions
If your syncs are failing, check that the REST API key you use to [connect Braze to Hightouch](#api-key) has the [correct permissions](#required-permissions) for your sync type or types.
### Slow initial sync
If you're using a Braze ID to [match user records](#record-matching), your initial sync may be slow. When Hightouch updates Braze with new records or fields, Hightouch queries users in Braze for [change data capture](/syncs/cdc).
Braze allows batch searches on external IDs, but only allows querying for one user at a time if using Braze IDs. That means Hightouch needs to make a separate request for each user if using Braze IDs.
It's best to match records on external ID rather than Braze ID. This speeds up the initial sync and limits the number of requests to make to Braze.
### NULL values clear profile attributes
When syncing user attributes to Braze, any mapped field containing a NULL value [clears the corresponding attribute](https://www.braze.com/docs/api/objects_filters/user_attributes_object#braze-user-profile-fields) on the Braze user profile. If you don't want to overwrite existing attribute values with empty data, exclude NULL rows from your model using a SQL `WHERE` clause, or select the **Don't sync NULL values** option in your sync's [field mapping configuration](/syncs/mapping-data).
### Syncing nested or array attributes
To sync nested or array attributes to Braze, structure your model data as JSON objects. In the sync configuration, map the JSON column to the corresponding Braze custom attribute. For arrays of objects, see the Braze documentation on [nested custom attributes](https://www.braze.com/docs/user_guide/data_and_analytics/custom_data/custom_attributes/nested_custom_attribute_support/).
### Audience count drops to 0 when adding Braze as a destination
If your audience count drops to 0 after you add Braze as a destination, the audience's parent model is likely missing the identifier field that Braze requires (for example, `external_id` or `braze_id`). The destination filter needs matching identifiers to calculate the sendable count. Verify that your parent model includes the correct Braze identifier column and that values are populated for the users you expect to sync.
### Multiple requests per user
When updating an attribute's value in a nested object to `null`, Braze's API doesn't let you combine this change with other non`-null` updates. Therefore, when updating values to `null`, Hightouch sends the full nested object with the `_merge_objects` parameter set to `false`.
Setting `_merge_objects` to `true` deep merges your update with the existing object data. See the Braze [API documentation](https://www.braze.com/docs/user_guide/data_and_analytics/custom_data/custom_attributes/nested_custom_attribute_support/#api-request-body) for more details.
This means the same user may require two update actions:
- Updates to nested objects with non-null values with `_merge_objects:true`
- Updates setting one or more attributes in nested objects to `null` with `_merge_objects:false`
---
## Common errors
**400 – The value provided for the phone number field is not a valid phone number**
Braze marks some phone numbers as invalid if they are deactivated, rejected by providers, on an [opt-out list](https://www.braze.com/docs/user_guide/message_building_by_channel/sms/keywords/optin_optout), or already associated with another user. See Braze’s guidance on [importing phone numbers](https://www.braze.com/docs/user_guide/message_building_by_channel/sms/phone_numbers/user_phone_numbers#importing-phone-numbers).
**Fixes:**
- Validate phone number formatting in your source.
- Remove numbers that are known to be deactivated or opted out.
- Check for conflicting assignments of the same number to multiple users.
**alias objects must be a hash**
This error indicates that the alias object is not in the expected format with both `alias_label` and `alias_name`, for example:
```
{
"alias_label": "label",
"alias_name": "name_1"
}
```
See Braze’s [User Alias Object](https://www.braze.com/docs/api/objects_filters/user_alias_object) documentation for the required structure.
A common cause is mapping a string column (for example, a JSON string) to the User Alias Object field instead of a JSON object.
**Fixes:**
- Ensure the model column mapped to the User Alias Object is of type **Object**, not string.
- Build the alias object in the correct shape before syncing.
**Batch too large or payload exceeds Braze limits**
Braze returns errors when event payloads, user attributes, or catalog items exceed size limits.
**Symptoms:** `Max input length exceeded`, `Request body too large`
**Fixes:**
- Reduce the number of mapped fields.
- Remove large nested objects or unnecessary attributes.
- Split very large catalogs or audiences across multiple syncs.
**Invalid or missing user identifiers**
Braze cannot update or target users unless identifiers are valid and non-null.
**Symptoms:** `Missing matching row`, dropped rows, partial updates
**Fixes:**
- Prefer **external_id** as the primary identifier when possible.
- Ensure identifier columns are populated and stable in your model.
- For alias-based syncs, include both `alias_name` and `alias_label`.
**Event attribute type errors**
Braze enforces strict types for event and purchase properties.
**Symptoms:** `TypeError`, `Invalid event attribute`
**Fixes:**
- Convert timestamps to ISO 8601 or Unix milliseconds.
- Ensure numeric fields (such as price or quantity) are not strings.
- Avoid nested objects or arrays in event attributes unless Braze explicitly supports them.
**Catalog field type or schema mismatches**
Catalog syncs fail when field types change or when new attributes do not match the existing catalog schema.
**Symptoms:** `Field type mismatch`, partial item failures
**Fixes:**
- Keep catalog field names and types consistent across syncs.
- When schema changes are required, consider creating a new catalog (Braze catalog deletions are permanent).
**Subscription group ID or permission errors**
Subscription syncs fail when the subscription group ID is invalid or its API key lacks required scopes.
**Symptoms:** `Invalid subscription group ID`, subscription state not updating
**Fixes:**
- Use the subscription group’s **GUID** (not the display name).
- Ensure the API key includes `subscription.status.set` or an equivalent permission.
**Campaign or Canvas trigger failures**
Triggers do not fire when user identifiers or required trigger properties are missing.
**Symptoms:** Trigger fails silently or returns a `400` error
**Fixes:**
- Confirm record matching uses a valid Braze identifier (for example, `external_id`).
- Verify that all required trigger properties exist in the model and are mapped in the sync configuration.
**rowsPerBatch must be less than or equal to 50**
This error occurs when the advanced mapper’s batch size exceeds the Braze API limit of 50 rows per batch. Reduce the `rowsPerBatch` value in your sync’s advanced mapper configuration to 50 or fewer.
---
## Live debugger
---
## Sync alerts
---
## Braze Cohorts
**URL:** https://hightouch.com/docs/destinations/braze-cohorts
**Description:** Build better campaigns on Braze with up-to-date customer data from your data warehouse
**Section:** Destinations
This destination is specific to the [Braze Cohort Import](https://www.braze.com/docs/partners/data_and_analytics/reverse_etl/hightouch/hightouch_cohort_import) feature, which lets partners like Hightouch manage user cohorts within Braze automatically. The Braze Cohorts feature is an excellent alternative to storing audience subscriptions into user-level attributes in Braze since it can reduce your Braze [data point](https://www.braze.com/docs/user_guide/onboarding_with_braze/data_points/) consumption when segmenting audiences.
### Get Braze data import key
In Braze, navigate to **Technology Partners** and select **Hightouch**. Here, you will find your REST Endpoint and generate your Braze data import key. Once generated, you can create a new key or invalidate an existing one.
### Add Braze Cohorts as a destination in Hightouch
Navigate to the **Destination** page in your Hightouch workspace, search for **Braze Cohorts**, and click **Continue**. From there, take your REST endpoint and data import key and click **Continue**.
### Sync a model or audience into Braze Cohorts
In Hightouch, using your created [model](/syncs/create-your-first-sync#set-up-a-model) or [audience](/customer-studio/usage), create a new sync. Next, select the Braze Cohorts destination you created in the previous step. Lastly, in the Braze Cohorts destination configuration, select the identifier you want to match against and decide whether or not you want Hightouch to create a new Braze Cohort or update an existing one.
#### Sync behavior
Braze Cohorts supports an **Add, Remove** sync pattern:
- **Add**: Adds users present in your model or audience to the selected Braze Cohort.
- **Remove**: Removes users who are no longer present in your model or audience from the Braze Cohort. When Hightouch detects via [CDC](/syncs/cdc) that a user has dropped out of the model or audience, it sends a remove operation for that user so they are taken out of the Braze Cohort on the next sync run. This only removes them from the cohort--it does **not** delete the underlying Braze user profile.
Hightouch handles these changes automatically on each sync run so your Braze Cohort always reflects the current membership of your model or audience.
### Create and use a Braze Segment from the Hightouch custom audience
In Braze, navigate to **Segments**, create a new segment, and select **Hightouch Cohorts** as your filter. From here, you can choose which Hightouch cohort you wish to include. Once created, you can select your Hightouch cohort segment as an audience filter when creating a campaign or Canvas.
To use your Hightouch segment, create a Braze campaign or Canvas and select the segment as your target audience.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Brevo
**URL:** https://hightouch.com/docs/destinations/brevo
**Description:** Streamline your marketing efforts by syncing data into Brevo
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-------------|---------------------------------------------|------------------------|----------------------------------------------------------------------
**Contacts** | Sync data from any source to Brevo contacts | Upsert, Update, Insert | [Contacts docs](https://developers.brevo.com/reference/createcontact)
**Contact Lists** | Sync data from any source to a Brevo contact list | Upsert | [Contact lists docs](https://developers.brevo.com/reference/createlist-1)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Brevo
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Brevo** and click **Continue**. You can then authenticate Hightouch to **Brevo**.
Enter the following required fields into Hightouch:
- **API key**:Enter a valid API key.
## Sync configuration
Once you've set up your Brevo destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Brevo destination you want to sync to.
### Syncing contacts
Sync data from any source to Brevo contacts.
### Syncing contact lists
#### List selection
Select which list in your Brevo account you'd like to upsert contacts to. If you would like to create a new list, enter the name of the list, and choose the folder to create the list in.
### Record matching
For **upsert** mode, you can match contacts by email.
For **update** mode, you can match by email, sms, or the contact's ID. Ensure the data types of the model column and Brevo field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
Brevo requires each email, SMS, and contact ID in your list to be unique. Brevo rejects updates that contain information that conflicts with existing contacts.
In **Insert** mode, Brevo automatically generates an identifier for every new record synced, so there is no need to match an existing record.
### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default contact fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|-----------------------------------------------------------
**Do nothing** | Keep the contact in Brevo with all its synced fields
**Clear** | Clear all the mapped fields, but keep the contact in Brevo
**Delete** | Delete the synced contacts from Brevo
For Brevo Contact Lists, when records leave your query result, they are removed from your audience by default. If you would not like to delete any records from your audience, you can choose to do nothing when records leave your query result.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Cadent
**URL:** https://hightouch.com/docs/destinations/cadent
**Description:** Activate your data as Conversion events in Cadent
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes |
----------------|------------------------------------|----------------------|
**Conversions** | Sync conversion event data to Cadent | Insert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Cadent
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Cadent** and click **Continue**. You can then authenticate Hightouch to **Cadent** by entering your API key.
If you do not have an API key, contact Cadent support to set one up.
## Sync configuration
Once you've set up your Cadent destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Cadent destination you want to sync to.
### Syncing conversions
Sync event conversions data to Cadent.
#### Dataset selection
Enter the dataset ID you would like to sync events to. This ID can be found on the Dataset Details page in Cadent.
#### Field mapping
Each Cadent dataset is set up with a match key. Include this match key in your mappings to successfully sync events to Cadent.
Provide the event name, and optionally provide the event timestamp. If no timestamp is provided, Cadent will use the time the event was received.
Hightouch lets you sync event properties and user properties via field mappings. Select the column in your model you'd like to map to each field in Cadent you'd like to sync.
The following are valid values for the event conversion type in Cadent:
- `email_clickthrough`
- `site_clickthrough`
- `online_purchase`
- `app_install`
- `in_app_purchase`
- `physical_store_visit`
- `offline_purchase`
- `phone_call`
- `chat_event`
- `system_generated`
#### Field hashing
The following fields are required to be hashed, and hashed by default:
- Email (`em`)
- First name (`fn`)
- Last name (`ln`)
- Date of birth (`db`)
- Gender (`ge`)
- City (`ct`)
- State (`st`)
- Zip code (`zp`)
- Country (`country`)
- External ID (`external-id`)
If the data in your model is already hashed, you can disable field hashing in your sync.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Calixa
**URL:** https://hightouch.com/docs/destinations/calixa
**Description:** Accelerate product-led sales with data directly from your data warehouse
**Section:** Destinations
## Setup
To set up Calixa, you will need your Calixa **Account ID** and **Auth Token**
These can be found in Calixa by clicking the dropdown menu on the Calixa console home page and select **API Keys**.
## Supported types
Hightouch supports syncing to the following Calixa types:
- **Events**
- **Objects**
## Syncing objects
### Object types
Hightouch supports syncing to the following Calixa objects:
- **Users**
- **Accounts**
### Event types
Hightouch supports syncing to the following Calixa events:
- **Custom User Events**
- **Custom Account Events**
- **Published Metric Events**
### Object sync modes
- **Upsert** - Upsert mode is supported for Users and Accounts. In upsert mode, Hightouch adds new rows and keeps existing rows up to date.
### Record matching
Records can be matched from your source to your Calixa workspace by an external ID.
### Field mapping
You can sync columns from your source to Calixa's default and custom fields.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Campaign Monitor
**URL:** https://hightouch.com/docs/destinations/campaign-monitor
**Description:** Run better email campaigns on Campaign Monitor with up-to-date customer data from your data warehouse
**Section:** Destinations
## Setup
To find your API keys, navigate to your Profile (top right of the page) > Account Settings > API Key and create a private API key.
## List subscriptions
Hightouch supports automatically creating a new List for your sync. You can optionally assign a custom name to your list.
If no name is provided, Hightouch will default to using your model name.
Here, the model name `Subscription List 1` will be used as the list name:
Additionally, instead of creating a new list, Hightouch can also sync subscribers to existing lists that are already populated in your Campaign Monitor account.
### Sync modes
This integration supports Segment mode. New records will be inserted into the Campaign Monitor
List. Records that are removed from the model will be removed from the
segment.
### Record mappings
Records can be matched from your source to your Campaign Monitor Subscriptions by email.
### Field mappings
You can sync columns from your source to Campaign Monitor's default fields.
Note: You must provide a mapping for the "Consent To Track" field. More details can be found in the [Campaign
Monitor API docs.](https://www.campaignmonitor.com/api/v3-3/subscribers/#importing-many-subscribers)
### Custom mappings
You can also sync custom fields to Campaign Monitor's Subscription Lists. If you are syncing custom fields of
data type `Multiple options - select many`, ensure your column data is an array.
Note: Ensure the Campaign Monitor List has custom fields already defined in your account before syncing custom fields.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Campaigner
**URL:** https://hightouch.com/docs/destinations/campaigner
**Description:** Enrich Campaigner with data from your warehouse to build robust email marketing campaigns to drive higher ROI and grow your audience.
**Section:** Destinations
## Overview
Providing relevant and timely campaigns relies on fresh and accurate data in your Campaigner account. By automatically syncing customer data from your data warehouse into Campaigner, you don't need to worry about data consistency or staleness. Instead, you can focus on building world-class customer experiences.
## Getting started
### Connect to Campaigner
**Create an API user and retrieve the API Token from Campaigner.**
1. Log in to Campaigner.
2. Visit [Campaigner's Users Settings Page](https://secure.campaigner.com/CSB/Account/AccountUsers.aspx).
3. Click on **Create New User**.
4. Select **API user** from the **User Role** dropdown select, and fill out the information required in the **User Profile** tab.
5. Click on **Create API Key**.
6. Add the [Hightouch IP Addresses](/security/networking#ip-addresses) to the IP Allowlist and click **Save**.
7. Click on the newly created user and head to the **API Keys** tab and copy the API key.
#### Add Campaigner as a destination in Hightouch
1. Log in to Hightouch or create a free Hightouch account
2. Click on **Destinations** in the left sidebar
3. Click on **Add destination** in the top right corner
4. Select **Campaigner** in the destination catalog and click **Continue** to proceed
5. Paste your **API Key** and click **Continue** to proceed
6. Give your destination a name and unique slug (for example, Campaigner Production” and `Campaigner-production`)
7. Click **Finish** to create your Campaigner destination.
### Connect to your data source
Hightouch sits on top of your data warehouse and can read from Snowflake, Redshift, BigQuery, Databricks, Postgres, MySQL, and many other data sources. You can even fetch data from Google Sheets and Airtable. Hightouch can help activate your data wherever it resides.
We've written dedicated guides for each supported data source. Take your pick:
- Connect to [Airtable](/sources/airtable)
- Connect to [Amazon Athena](/sources/amazon-athena)
- Connect to [Amazon Redshift](/sources/amazon-redshift)
- Connect to [ClickHouse](/sources/clickhouse)
- Connect to [Databricks](/sources/databricks)
- Connect to [Elasticsearch](/sources/elasticsearch)
- Connect to [Firebolt](/sources/firebolt)
- Connect to [Google BigQuery](/sources/google-bigquery)
- Connect to [Google Sheets](/sources/google-sheets)
- Connect to [Looker](/sources/looker)
- Connect to [Metabase](/sources/metabase)
- Connect to [MySQL](/sources/mysql)
- Connect to [Palantir Foundry](/sources/foundry)
- Connect to [PostgreSQL](/sources/postgresql)
- Connect to [Rockset](/sources/rockset)
- Connect to [SFTP](/sources/sftp)
- Connect to [Snowflake](/sources/snowflake)
- Connect to [SQL Server](/sources/sqlserver)
- Connect to [Trino](/sources/trino)
If you're missing the credentials necessary to connect to your data source,
try inviting your teammates to Hightouch. You can have unlimited collaborators
in your workspace.
#### Connect to a sample database instead
Waiting for credentials to your data warehouse? No problem. You can connect to one of our [sample datasets](/sources/sample-data) to get started right away. For the rest of this tutorial, we'll use the **B2B SaaS** sample source, which contains a `users` table with names, emails, locations, and other user attributes. This sample source is available when you log in for the first time—no setup required.
Next, we'll write a SQL query to define which users should be synced to our Campaigner customers resource. You can follow the same pattern to sync to other support Campaigner resources,
### Create a model
In Hightouch, a model represents a query that filters or transforms the records in your data source. Models are used to determine exactly which records to sync between sources and destinations.
When you connect to a data warehouse, your models will probably be written in SQL. You can also import models from tools like dbt and Looker.
In this example, we'll keep it simple. Suppose you want to sync all your customers to Campaigner. To accomplish this, you'll want to create a model that looks like this:
```sql
SELECT * FROM users;
```
1. Click on **Models** in the left sidebar.
2. Click on **Add model** in the top right corner.
3. Select **Demo Database** as your source and click **Continue** to proceed.
4. Select **SQL Editor** as your modeling method.
5. Paste the SQL query from above into the editor on the left side of the page.
6. Click **Preview** to execute the query.
7. Click **Continue** to proceed.
8. Give your model a name and unique slug (for example, "All Users" and `all-users`).
9. Select **ID** as the primary key for your model.
10. Click **Finish** to create your model.
We just created a model using a SQL query that retrieves all our current users. Next, we'll configure our sync to add these users to Campaigner.
### Create a sync
1. Click on **Syncs** in the left sidebar.
2. Click on **Add sync** in the top right corner.
3. Select **All Users** as your model and click **Continue** to proceed.
4. Select **Campaigner** as your destination and click **Continue** to proceed.
## Sync configuration
### Syncing subscribers object
#### Object selection
Hightouch supports syncing to the following Campaigner resources:
- **Subscribers**. Also referred to as **Contacts** on Campaigner.
#### Sync modes
This integration supports **Upsert** and **Update**.
- In **Upsert** mode, records that aren't found in Campaigner given the provided identifier will be inserted. And existing records will be updated.
- In **Update** mode, only records that are found in Campaigner given the provided identifier will be updated.
#### Record matching
- Records can be matched from your source to your Campaigner workspace by `Email`.
#### Field mapping
Hightouch allows you to sync columns from your source to the supported Campaigner fields. You can map fields in Campaigner that aren't in the table below using [custom field mapping](#custom-field-mapping).
| Mapping | Type | Description |
| :------------------- | :-----: | --------------------------------------------------------------------------------------------------------------------------------------------: |
| `Source ID` | Number | The ID of a valid Source this subscriber should be associated with. |
| `Publications` | Array | An array of Publication IDs to be associated with the subscriber. |
| `Lists` | Array | An array of List IDs to be associated with the subscriber. |
| `Orders` | Array | An array of [order objects](https://media.campaigner.com/apidocs/edapi/#OrderAdd). |
| `Force` | Boolean | If the Force parameter is **true**, "Removed" or "Bounced" subscriber statuses will be ignored and set back to "Active" |
| `Force Publications` | Boolean | Set this property to **true** if you want to force this subscriber back into a publication they might have previously requested removal from. |
#### Custom field mapping
Hightouch allows you to set custom attributes. Any mappings from this section will be synced as custom fields. Campaigner expects custom properties to follow a `PascalCase` format for both _new_ and _existing_ custom fields. For example, if `First Name` exists as a field in Campaigner, enter `FirstName` in our mapper. Use the same format for custom fields you want Hightouch to create in Campaigner.
#### Delete records
When the Hightouch sync engine recognizes that a record has been removed from your source, you can either:
In **Upsert** mode:
- Do Nothing.
- Clear fields that are being synced in the destination.
- Delete record in the destination.
In **Update** mode:
- Do Nothing.
- Clear fields that are being synced in the destination.
Clearing fields in Campaigner through Hightouch will not clear existing
Publications and Lists from the Subscriber's record. However, It will reset
the Source ID, and return the boolean mappings:`Force Publications` and
`Force` to their default if they exist in your mappings.
### Syncing mailing lists
- Create a mailing list and/or add subscribers with the **Mailing Lists** integration. Note that contacts must already exist in your Campaigner account to be synced to a mailing list.
#### Record matching
- Records can be matched from your source to your Campaigner workspace by `Email`.
#### Create a new mailing list
- To create a new mailing list, select `Create a new segment`.
- You can provide a name for this mailing list, or Hightouch will use the name of the model used to create the sync.
#### Use an existing mailing list
- To use an existing mailing list, select `Use existing segment`.
- You can select any of your **active** Campaigner mailing lists from the drop-down select menu.
- If you'd like to sync to segments dynamically, click `Use Column` and pick a column from your model that contains the Campaigner `Mailing List ID`.
### Syncing suppression lists
- Add subscribers to suppression lists with **Suppression Lists** integration.
#### Record matching
- Records can be matched from your source to your Campaigner workspace by `Email`.
#### Use an existing suppression list
- Campaigner doesn't provide the ability to create suppression lists programmatically.
- You can select any of your **active** Campaigner suppression lists from the drop-down select menu.
## Tips and troubleshooting
### Common errors
#### Error: 101 Email address has been screened out
You may see this error on records that were rejected during an otherwise successful sync run. This is frequently due to improper formatting (`@com.com`) or other common email validation checks.
### Live debugger
### Sync alerts
---
## Cendyn
**URL:** https://hightouch.com/docs/destinations/cendyn
**Description:** Capture demand and drive direct bookings by managing your digital ads and metasearch campaigns from one single platform.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
-------------------|--------------------------------------------------|---------------------
**Profiles** | Sync data from any source to Cendyn profiles | Upsert
**Reservations** | Sync data from any source to Cendyn reservations | Upsert
**Transactions** | Sync data from any source to Cendyn transactions | Upsert
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Cendyn
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Cendyn** and click **Continue**. You can then authenticate Hightouch to **Cendyn**.
Enter the following fields into Hightouch:
- **Base URL**
- **Property ID**
- **Subscription ID**
## Sync configuration
Once you've set up your Cendyn destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Cendyn destination you want to sync to.
### Syncing objects
Sync data from any source to Cendyn profiles, reservations, or transactions.
#### Record matching
To match rows from your model to objects in Cendyn, you need to select the model column that contains values that match IDs in the Cendyn objects' **Unique IDs** array.
The value entered as the `Destination field` will be used as the unique ID type. After entering the type, you will be prompted to enter the context of the unique ID.
In the image below, the unique ID's type is `PMS` and its context is `true`.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync object fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default fields for each object.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Chameleon
**URL:** https://hightouch.com/docs/destinations/chameleon
**Description:** Boost product adoption and reduce churn
**Section:** Destinations
## Overview
Power your Chameleon workspace with up-to-date user data straight from your data warehouse. You can sync enriched user data and events from your warehouse, specifically to augment product experiences. Moreover, you can ensure that the data in Chameleon is always up-to-date and in the expected format for your Product and Growth teams.
## Supported syncing
Type | Description | Supported Sync Modes
---------------------|--------------------------------------------------|-----------------------
**Users** | Sync data from any source to any user object | Upsert
**Companies** | Sync data from any source to any company object | Upsert
**Events** | Sync data from any source to any event | Upsert
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Chameleon
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination**
button. Select **Chameleon** and click **Continue**. You can then authenticate Hightouch to **Chameleon**.
### Authenticate with credentials
Enter the following required fields into Hightouch:
- **Chameleon API Key**.
## Sync configuration
Once you've set up your Chameleon destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and
the Chameleon destination you want to sync to.
### Users
Sync data from any source into your Chameleon users.
#### Record matching
You can match rows in your model with users in Chameleon on the following fields:
- User uid
#### Field mapping
You can sync columns from your source to Chameleon custom fields.
### Companies
Sync data from any source into your Chameleon companies.
#### Record matching
You can match rows in your model with users in Chameleon on the following fields:
- Company Uid
#### Field mapping
You can sync columns from your source to Chameleon custom fields.
### Events
Insert events into Chameleon by user id.
#### Field mapping
When sending events to Chameleon you can only specify the event name and user id.
## Tips and troubleshooting
### Common errors
### Live debugger
Hightouch provides complete visibility into the API calls made during each of your sync runs. We recommend reading our
[article on debugging tips and tricks](/syncs/debugger) to learn more.
### Live debugger
### Sync alerts
---
## Chargebee
**URL:** https://hightouch.com/docs/destinations/chargebee
**Description:** Enrich Chargebee data with customer data from your warehouse to drive revenue analysis
**Section:** Destinations
## Overview
To get the most out of a tools like Chargebee, you'll need to update customers and custom fields.
Chargebee has a tedious process of having to update each customer individually. Hightouch solves this problem by letting you define your customers and it's custom fields on top of your data warehouse or production database. This ensures that your Chargebee will always remain in sync with your entire data stack.
### Why subscription billing should be based on top of the warehouse
- You can sync analytics as custom fields to your Chargebee customers. This allows you to prioritize a subset of certain customers more easily.
- You no longer have to check and compare your data between Chargebee and your business tools.
## Supported sync methods
Hightouch supports syncing data to a Chargebee customer.
### Sync customers to Chargebee
#### Matching source to destination records
When matching records, it's preferred to match it through Chargebee's customer ID. However, if your source has no access to the ID, it may match with other fields, such as email, last name, etc.
If there are multiple rows with same matching value, such as two customers
having the same email, it will only select the first customer returned from
Chargebee.
#### Modes
- **Upsert**: This mode will update existing customers or insert the customer if they do not already exist.
- **Update**: This mode will just update existing customers.
#### Custom fields
You are able to update Chargebee's custom fields through our destination custom field mapping. When referencing the Chargebee custom field within Hightouch, **use the API name** of the custom field
#### Rate limits
Something to keep in mind, as you process all your rows of data is that Chargebee limits the amount of times we can update the rows within their product per minute. This is completely dependent on which tier plan you have with Chargebee and if you're using live/test site.
[More Information On Chargebee's Rate Limit](https://support.chargebee.com/support/solutions/articles/250853-js-api-rate-limit-chargebee)
## Setup
### Connect to Chargebee
#### Creating API token
1. Log in to Chargebee
2. Click on **Settings**
3. Click on **Configure Chargebee** tab
4. Click on the **API keys** section
5. Click on **Add API key**
6. Select the **Full-Access Key** type, then **All** permissions, then give a name
7. Click **Create Key** and copy the newly created token
#### Adding destination in HT
1. Log in to Hightouch or create a free Hightouch account
2. Click on **Destinations** in the left sidebar
3. Click on **Add destination** in the top right corner
4. Select **Chargebee** in the destination catalog and click **Continue** to proceed
5. Type in your site name into the **Site** field (Hint: It's the subdomain when you're on the Chargebee app)
6. Paste your API token into the **API Access Token** field and click **Continue** to proceed
7. Give your destination a name and unique slug (for example, "Chargebee Production” and `chargebee-production`)
8. Click **Finish** to create your Chargebee destination.
### Connect to your data source
Hightouch sits on top of your data warehouse and can read from Snowflake, Redshift, BigQuery, Databricks, Postgres, MySQL, and many other data sources. You can even fetch data from Google Sheets and Airtable. Hightouch can help activate your data wherever it resides.
We've written dedicated guides for each supported data source. Take your pick:
- Connect to [Airtable](/sources/airtable)
- Connect to [Amazon Athena](/sources/amazon-athena)
- Connect to [Amazon Redshift](/sources/amazon-redshift)
- Connect to [ClickHouse](/sources/clickhouse)
- Connect to [Databricks](/sources/databricks)
- Connect to [Elasticsearch](/sources/elasticsearch)
- Connect to [Firebolt](/sources/firebolt)
- Connect to [Google BigQuery](/sources/google-bigquery)
- Connect to [Google Sheets](/sources/google-sheets)
- Connect to [Looker](/sources/looker)
- Connect to [Metabase](/sources/metabase)
- Connect to [MySQL](/sources/mysql)
- Connect to [Palantir Foundry](/sources/foundry)
- Connect to [PostgreSQL](/sources/postgresql)
- Connect to [Rockset](/sources/rockset)
- Connect to [SFTP](/sources/sftp)
- Connect to [Snowflake](/sources/snowflake)
- Connect to [SQL Server](/sources/sqlserver)
- Connect to [Trino](/sources/trino)
If you're missing the credentials necessary to connect to your data source,
try inviting your teammates to Hightouch. You can have unlimited collaborators
in your workspace.
#### Connect to a sample database instead
Waiting for credentials to your data warehouse? No problem. You can connect to one of our [sample datasets](/sources/sample-data) to get started right away. For the rest of this tutorial, we'll use the **B2B SaaS** sample source, which contains a `users` table with names, emails, locations, and other user attributes. This sample source is available when you log in for the first time—no setup required.
### Create a model
### Create a sync
## Tips and troubleshooting
### Common errors
#### Don't see your data in Chargebee?
Make sure you've configured your sync to use the correct Chargebee environment. You might've accidentally pointed to a different environment, such as using test site instead of live site. If that doesn't solve your issue, . We're happy to help.
### Live debugger
### Sync alerts
---
## ChartMogul
**URL:** https://hightouch.com/docs/destinations/chartmogul
**Description:** Empower your product teams with richer data from your warehouse to perform deeper analysis on subscription revenue and better understand business performance.
**Section:** Destinations
## Setup
To find your API keys, navigate to your profile and create a private API key.
## Syncing
Hightouch supports syncing to the following ChartMogul Objects:
- `Customers`
- `Invoices`
- `Transactions`
### Sync modes
This integration supports **Upsert**, **Update**, and **Insert** modes for `Customers` objects and **Insert** mode for `Invoices` and `Transactions` objects.
- **Upsert**: Upsert mode pushes new objects to ChartMogul and updates fields that change in your warehouse.
- **Update**: Update mode updates particular fields on existing objects in ChartMogul.
- **Insert**: Insert mode pushes new objects to ChartMogul, and doesn't update the objects as they change in your warehouse.
## Record matching
### Customers
In **Upsert** mode, records can be matched from your source to ChartMogul by `External ID` or `Email`.
In **Update** mode, records can be matched from your source to ChartMogul by `External ID` or ChartMogul's `Customer UUID`.
ChartMogul doesn't consider `Email` as a unique identifier, so upserting
records will apply to all matching emails.
## Field mapping
### Customers
You can sync columns from your source to ChartMogul's default and custom attributes. When syncing with **Upsert** or **Insert** mode, `Data Source UUID`, `Name`, and `External ID`
(depending on if `External ID` was selected in the Record Matching section) are required fields in order for the sync to run successfully. When syncing with **Update** mode, `Name`
is a required field in order for the sync to run successfully. To know more about `Customer` fields, refer to the [ChartMogul docs](https://dev.chartmogul.com/reference/customers).
Additionally when syncing custom attributes with **Upsert** or **Insert** mode, Hightouch automatically transforms your data to match the required
format that ChartMogul expects for the `custom` field. For example, say you want to enrich a customer profile by inserting `age` as a custom attribute, where the object is:
```JSON
{
"attributes": {
"custom": {
"age": 25,
},
}
}
```
Hightouch will automatically detect the `type` of the record value and transform the object for you to the expected `custom` field format:
```JSON
{
"attributes": {
"custom": [
{
"type": Integer,
"key": "age",
"value": 25
},
]
}
}
```
### Invoices
You can sync columns from your source to ChartMogul's default and custom attributes. `Customer UUID`, `External ID`, `Date`, `Currency`, `Line Items`, and
`Customer External ID` are required fields in order for the sync to run successfully. To know more about `Invoice` fields, refer to the [ChartMogul docs](https://dev.chartmogul.com/reference/import-customers-invoices).
### Transactions
You can sync columns from your source to ChartMogul's default and custom attributes. `Invoice UUID`, `Date`, `Type`, and `Result` are required fields in order
for the sync to run successfully. To know more about `Transaction` fields, refer to the [ChartMogul docs](https://dev.chartmogul.com/reference/import-invoice-transaction).
## Delete mode
### Customers
You can choose what Hightouch's behavior is when records leave the query result set. The default is doing nothing, but you can also set Hightouch to delete the `Customer` record.
ChartMogul doesn't allow custom attributes to be cleared, so Hightouch will
sync `"null"` to the field instead.
## Tips and troubleshooting
### Common errors
{/* */}
#### write ECONNRESET
{/* */}
You may receive the `write ECONNRESET` error if the model columns used for [record matching](#record-matching) or [mapping](#field-mapping) don't have an associated [data type](/models/data-types-casting#data-types) in Hightouch.
To resolve the error, ensure that each model column has the data type ChartMogul expects.
Refer to the [data types and casting docs](/models/data-types-casting) for more information.
### Live debugger
### Sync alerts
---
## ChurnZero
**URL:** https://hightouch.com/docs/destinations/churnzero
**Description:** Fight customer churn with fresh customer data in ChurnZero
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference |
----------------|-----------------------------------------------------------------------------------|----------------------| ------------- |
**Events** | Sync records as events to Churnzero. This is often in the form of a track call | Insert | [Event schema](https://app.churnzero.net/developers#schemas-event) |
**Enrichment** | Sync records to a Churnzero custom table | All | [Batch import API](https://support.churnzero.com/hc/en-us/articles/360004339852-Batch-Imports-via-CSV) |
**Objects** | Sync records to objects such as accounts and contacts in Churnzero | Upsert | [Account schema](https://app.churnzero.net/developers#schemas-account) and [contact schema](https://app.churnzero.net/developers#schemas-contact)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to ChurnZero
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **ChurnZero** and click **Continue**. You can then authenticate Hightouch to **ChurnZero** by entering a ChurnZero **App Key**.
To find your app key, go to **Admin > Application Keys > New App Key** in ChurnZero.
Create a new key with a descriptive name, for example, "Hightouch Integration".
Though optional, it's best to enter your **App URL** into Hightouch.
You can find it below the Application Keys in ChurnZero.
Do not include the `/i` in the URL you enter into Hightouch.
If you don't provide your App URL, Hightouch defaults to `https://analytics.churnzero.net/`.
## Sync configuration
Once you've set up your ChurnZero destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the ChurnZero destination you want to sync to.
### Syncing events
ChurnZero's events API requires the following event parameters:
- `eventDate`: The date/time the event occurred. ISO 8601 formatted string.
- `AccountId`: The ID of the Account associated with the event
- `ContactId`: The ID of the Contact associated with the event
The sync configuration form ensures all these are set and provides some additional options.
#### Event name
Providing an event name is best practice while event tracking. You can either provide a static value or select to **use a column** from your model. Hightouch syncs the static or column value as the `eventName` parameter.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server. Hightouch syncs the timestamp as the `eventTime` parameter the events API requires.
If you select a column, it should be in [ISO 8601](https://www.iso.org/iso-8601-date-and-time-format.html) format.
#### Field mapping
Hightouch lets you sync default and custom event properties via [field mapping](/syncs/mapping-data).
You must include the **Account External ID** and **Contact External ID** of the account and contact associated with the event.
### Syncing enrichments
Hightouch lets you sync data to custom tables through ChurnZero's [batch import API](https://support.churnzero.com/hc/en-us/articles/360004339852-Batch-Imports-via-CSV).
To use this API, you must first create the custom table within ChurnZero.
You can create custom tables by going to the admin tab in ChurnZero and selecting **Custom Fields**.
To locate the custom table to update, Hightouch requires these two fields in your sync configuration:
- **Custom table ID**: You can find the table ID by going into **Admin/Imports** and selecting the custom table name from the dropdown there.
- **Account ID**: This is the user account the custom table is associated with. You can find it by clicking on your name at the top of the ChurnZero dashboard and selecting **My Account**. Your ID is at the end of the URL.
#### Record matching
You can match rows from your model to enrichment in ChurnZero on any column in your model and any field in ChurnZero.
Ensure the data types of the model column and ChurnZero field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync custom table columns via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the columns you've already created in your custom table.
Ensure your model columns' data types match the data types of the fields you want to sync to.
Hightouch can only sync data to fields you've already created in your custom table in ChurnZero on the **Custom Fields** page.
### Syncing objects
Hightouch supports syncing to the following ChurnZero objects:
- **Contacts**
- **Accounts**
#### Record matching
To match rows from your model to contacts in ChurnZero, you need to select the model column that contains values that match the **Contact External ID** field.
To match rows from your model to accounts in ChurnZero, you need to select the model column that contains values that match the **Account External ID** field.
You can find these identifiers in the top left of your contact or account page.
External identifiers are not the same as the ID in the URL of a contact or account page.
#### Field mapping
Hightouch lets you sync account and contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
You can find the custom fields for your ChurnZero workspace in **Admin > Custom Fields**.
Go to either the contact or account table to see the custom fields for that object type.
When syncing custom fields, use the **CZ API Name** as the destination field name in the Hightouch mapper.
ChurnZero requires that you map at least one default field.
If you use primarily custom fields, make sure to also map at least one default field.
Visit the ChurnZero's [API documentation](https://app.churnzero.net/developers#schemas) to see objects' default fields.
When syncing contacts, you must map a field to the contact's **Account Exernal ID**.
Please make sure that the contact's associated account exists in ChurnZero before attempting to sync contact information.
## Tips and troubleshooting
### Common errors
#### 404 - "null"
You may receive a `404 - "null"` error because there is a trailing `/` at the end of the App URL within the Destination configurations. Remove the trailing `/` from the App URL to clear the error.
### Live debugger
ChurnZero's API doesn't return detailed error messages.
For example, it doesn't state whether an account ID or a custom field you are attempting to sync to exists.
Therefore, Hightouch is unable to display this type of information in the debugger.
### Sync alerts
---
## Epsilon Retail Media (CitrusAd)
**URL:** https://hightouch.com/docs/destinations/citrusad
**Description:** Reach customers across the open web and drive increased sales both on-site and in-store.
**Section:** Destinations
## Overview
Now deliver even more relevant ads both onsite and across the web by combining data from various sources within your data warehouse to build custom audiences for CitrusAd. By keeping your custom audiences updated automatically, you'll never show an irrelevant ad to anyone.
## Supported syncing
| Type | Description | Supported Sync Modes | API documenation |
| ------------- | ----------------------------------------------- | ---------------------- | ---------------- |
| **Customers** | Sync customer data from any source to CitrusAd customers | Upsert, Update, Insert | [Create or update a customer](https://developers.citrusad.com/integration/reference/create-or-update-a-customer)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to CitrusAd
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **CitrusAd** and click **Continue**. You can then authenticate Hightouch to CitrusAd by entering the following required fields into Hightouch:
- **Base URL**: The base URL for staging or production environment. Visit [CitrusAd's docs](https://developers.citrusad.com/integration/reference/api-overview#base-urls) for information on retrieving your base URL.
- **API key**: The API key for authenticating the API requests. Visit [CitrusAd's docs](https://developers.citrusad.com/integration/reference/api-overview#basic-authentication) for information on retrieving your API key.
## Sync configuration
Once you've set up your CitrusAd destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and
the CitrusAd destination you want to sync to.
### Customers
Customer data is required to generate personalized ads with CitrusAd. See CitrusAd's [docs](https://developers.citrusad.com/integration/reference/customer-data-1) for more details.
#### Record matching
You must match rows in your model with customers in CitrusAd on the **Customer ID** field.
In **Insert** mode, CitrusAd automatically generates a customer ID for every new record synced, so there is no need to match an existing record.
#### Field mapping
You can sync any model columns to CitrusAd's default customer fields including `targetingData`.
When creating new customer records in CitrusAd via insert or upsert mode, CitrusAd sets a default value to all fields without a value, whether you've mapped them or not.
You can find CitrusAd's default values in the table below:
| Field name | Default value |
| ----------------- | ----------------|
| **Year of Birth** | `0` |
| **Gender** | `"Other"` |
| **Postcode** | `""` |
| **Suburb** | `""` |
The **Gender** field accepts the following values:
`"Male"`,`"Female"`,`"Other"` case insensitive. If you send any other values, CitrusAd changes them to `"Other"`.
When updating existing customer data via update or upsert mode, Hightouch pulls the existing customer data from CitrusAd and merges it with your updates.
## Tips and troubleshooting
For Hightouch platform error codes related to CitrusAd, see [Error codes: CitrusAd](/developer-tools/error-codes#other-destinations).
#### Delete behavior
**Clear** is only available for update mode. All sync modes use the **Do nothing** behavior by default.
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the customer record in CitrusAd with all its synced fields |
| **Clear** | Unset all the mapped fields from CitrusAd |
### Common errors
### Live debugger
### Sync alerts
---
## CleverTap
**URL:** https://hightouch.com/docs/destinations/clevertap
**Description:** Build better campaigns on CleverTap with up-to-date customer data from your data warehouse
**Section:** Destinations
## Setup
Navigate to Setting > Project in CleverTap to view credentials.
From the Project tab, copy `Project Id` as `Account Id` and `Passcode` into Hightouch. Use `CleverTap Region` for `Localization`.
## Supported types
Hightouch supports syncing to the following CleverTap types:
- **Objects**
- **Events**
- **Segments**
## Syncing objects
Hightouch supports syncing to the following CleverTap objects:
- **Profile** Hightouch allows you to sync user profiles from a source into CleverTap.
### Sync modes
- **Upsert** - In upsert mode, Hightouchs adds new rows and keeps existing rows up to date.
### Record matching
For each user profile, a user identifier is required. This is the key that CleverTap uses to find the user whose profile needs to be updated. You have to set a value for one of these parameters to identify the user: Identity, CleverTap Global Object ID, Meta ID or Google Plus ID. If this identity is not found a new user profile will be created.
You may also provide additional identifiers through Field Mapping. Profiles
are automatically unified by CleverTap.
### Field mapping
You can sync columns from your source to CleverTap.
CleverTap **highly recommends** you provide common fields like `Name, Email, Phone, etc.`. You can see additional details in the [CleverTap docs](https://developer.clevertap.com/docs/concepts-user-profiles#manually-updating-predefined-user-profile-properties).
Phone numbers must use E164 formatting. CleverTap will reject rows that aren't in this format.
Additional fields that fits your unique use case can be mapped through custom mappings.
CleverTap automatically generates schema for custom fields you provide. You
can see existing schemas by navigating to Setting > Schema > User
Properties in CleverTap.
## Syncing events
When syncing events, Hightouch treats any records added to [your source](/getting-started/concepts) as new events and sends them to CleverTap when your sync runs.
### Event name
You can provide a fixed event name or use a value from your source as event name by toggling the switch button.
New event names are automatically added to CleverTap schema.
### Record matching
For each event, a user identifier is required. This is the key that CleverTap uses to find the user whose profile needs to be updated. You have to set a value for one of these parameters to identify the user: Identity, CleverTap Global Object ID, Meta ID or Google Plus ID. If this identity is not found a new user profile will be created.
### Field mapping
You can sync any column from your source to into any event field in CleverTap.
## Syncing segments
When syncing segments or custom lists, CleverTap requires the users in the segment to have already been synced to CleverTap. You can use [sequences](/syncs/schedule-sync-with-sequences) to ensure that users are synced before syncing segments.
To start syncing, choose a field to identify users (Identity, CleverTap Global Object ID, Meta ID, or Google Plus ID), name the segment, and assign an admin's email for ownership.
CleverTap automatically generates schema for custom fields you provide. You
can see generated and existing schemas by navigating to Setting > Schema
> Events in CleverTap.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## ClickUp
**URL:** https://hightouch.com/docs/destinations/clickup
**Description:** A productivity team app for storing all essentials such as tasks, documents, goals, chat, and more.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
----------|---------------------------------------------------------|------------------------|-----------------------------------------------------------------------------
**Tasks** | Sync data from any source to ClickUp tasks and subtasks | Upsert, Update, Insert | [Tasks docs](https://clickup.com/api/clickupreference/operation/CreateTask/)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to ClickUp
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **ClickUp** and click **Continue**. You can then authenticate Hightouch to ClickUp using OAuth.
Select **Log in to ClickUp** and log into your ClickUp account.
You then need to select the ClickUp workspace or workspaces you want to connect to.
Once done, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your ClickUp destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the ClickUp destination you want to sync to.
### Syncing tasks
In ClickUp, a task is a specific action or activity that needs to be completed to achieve a particular goal or objective.
Hightouch supports syncing tasks and subtasks.
To get started, select the ClickUp **workspace**, **space**, **folder**, and **list** for your tasks.
You can then choose to **insert** new tasks, **update** existing ones, or do both in **upsert** mode.
If you want to sync subtasks, you need to map the **Parent** field. Refer to the [references section](#references) to learn more.
Hightouch doesn't update [closed or archived](https://help.clickup.com/hc/en-us/articles/6309452618647-Create-and-manage-custom-statuses#h_3e1a44262c) ClickUp tasks.
If your model contains data related to a closed or archived task, the sync creates a new task with the provided data.
#### Record matching
You can match rows from your model to tasks in ClickUp on identifer fields in ClickUp.
The identifer fields you can use depend on your sync mode:
- In **Upsert** mode, you can match on [custom fields](https://help.clickup.com/hc/en-us/articles/6303536766231-Custom-Fields-overview), which are displayed as **id** in the dropdown menu.
- In **Update** mode, you can match on any of the following ClickUp fields:
- [Task ID](https://help.clickup.com/hc/en-us/articles/6309890818071-Task-IDs)
- [Custom task ID](https://help.clickup.com/hc/en-us/articles/6304361756951-Custom-Task-IDs-ClickApp-)
- [Custom fields](https://help.clickup.com/hc/en-us/articles/6303536766231-Custom-Fields-overview)
- In **Insert** mode, ClickUp automatically generates an identifier for every new record synced, so there is no need to match an existing record.
To avoid unexpected duplicates or other potential issues, make sure you select a ClickUp field with unique values. Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
When creating a new task, ClickUp requires a name for it.
Therefore, in upsert and insert modes, you must map the **Name** field to complete your configuration.
You can also map data from any of your model columns to default fields in ClickUp.
The following default fields have some special guidelines:
- **Status**: The values you sync must be a valid statuses for your task. Status is case insensitive; for example, `in progress` is the same as `IN PROGRESS`. See [ClickUp's custom status docs](https://docs.clickup.com/en/articles/831184-create-and-manage-custom-statuses) for customization information.
- **Priority**: This is a number that corresponds to the priorities available in the ClickUp UI. **1** is the must urgent priority while **4** is the least urgent.
- **Time Estimate**: This field must be in milliseconds.
#### Arrays
You can sync array fields to ClickUp by selecting model columns that contain either comma-delimited strings or arrays.
If you sync a comma-delimited string, for example `"tag1, tag2, tag3"`, Hightouch transforms it into the array (`["tag1", "tag2", "tag3"]`) ClickUp expects.
Hightouch overwrites array values rather than aggregating them.
For example, suppose you sync the following array `["tag1", "tag2"]` to the tags field of a particular task.
Then, a later sync sends another array `["tag1", "tag3"]` to the same task's tags.
The tasks tags will be `["tag1", "tag3"]`, not `["tag1", "tag2", "tag3"]` after the later sync runs.
#### References
You can relate tasks to other ClickUp objects while field mapping.
To do so, select a ClickUp [reference](https://help.clickup.com/hc/en-us/articles/6309152582935-Create-and-view-Reference-Relationships) field.
Hightouch denotes these with a chainlink icon.
Begin by selecting one of these fields, for example **Links to**.
The Links To field lets you include a task ID to create a linked dependency for the task you're syncing.
Then, select a field on the object you're referencing to match on, for example, a task's ID, and the corresponding column from your model.
Hightouch uses this column to look up the related object's ID and create the relationship with the task you're syncing.
Generally, it's best to match on ID fields, but Hightouch also supports matching on meaningful properties like **email** for users.
A common reference field is **Parent**.
Mapping to the **Parent** field lets you sync subtasks.
The preceding screenshot shows associating a parent task to a subtask based on the parent's task ID.
The parent cannot be a subtask itself and it must be part of the ClickUp list you selected in the sync configuration.
Hightouch syncs only one value to reference fields. This includes reference fields like **assignees** that allow multiple values.
#### Custom field mapping
Hightouch supports syncing data to these [custom field types](https://docs.clickup.com/en/articles/2310503-custom-fields-overview):
|Custom field type|Expected data type or description|Example|
|--|--|--|
|**Checkbox**|Boolean value|`true` or `false`|
|**Date**|Millisecond [Unix](https://en.wikipedia.org/wiki/Unix_time) timestamp; See the [date formatting](#formatting-dates) tip. |`1682452269219`|
|**Dropdown**|A series of options in a menu|-|
|**Email**|Valid email address|`lana@clickup.com`|
|**Labels**|A flexible list of options (invalid labels will be omitted)|-|
|**Money**|A numerical value|`9.99`|
|**People**|People or teams, see (#references)|`Sam`|
|**Phone**|A valid phone number with country and area code|`"+1 123 456 7890"`|
|**Rating**|A number value from `1` through `5`|`5`|
|**Tasks**|Tasks linked together without relationships|`Task A`|
|**Text**|Single-line text|`"This is a single-line text"`|
|**Text Area**|Multi-line text|`"This is a multi-line text..."`|
|**Website**|A valid URL|`https://clickup.com`|
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options for upsert mode:
Behavior | Description
---------------|------------------------------------------------------------
**Do nothing** | Keep the task in ClickUp with all its synced fields
**Clear** | Clear all the mapped fields, but keep the task in ClickUp
**Delete** | Delete the synced tasks from ClickUp
Update mode has these options:
Behavior | Description
---------------|------------------------------------------------------------
**Do nothing** | Keep the task in ClickUp with all its synced fields
Insert mode doesn't support any delete behavior.
## Tips and troubleshooting
### Formatting dates
ClickUp expects millisecond [Unix](https://en.wikipedia.org/wiki/Unix_time) timestamps for all date fields.
A millisecond timestamp is numerical value with 13 digits, for example, `1620170285617`, as compared to a traditional [Unix timestamp](https://en.wikipedia.org/wiki/Unix_time) which has ten.
### Common errors
### Live debugger
### Sync alerts
---
## ClientSuccess
**URL:** https://hightouch.com/docs/destinations/client-success
**Description:** Improve product adoption and reduce churn by enriching ClientSuccess with rich, up-to-date customer data from your data warehouse
**Section:** Destinations
## Overview
This integration allows you to sync Clients, Contacts and Subscriptions from your warehouse to ClientSuccess.
## Setup
To enable syncing with [ClientSuccess](https://www.clientsuccess.com/) first you need a username and a password for
a user to perform the API calls on your behalf.
Then copy these values, and create a new Destination in Hightouch, selecting Client Success as the destination type. Lastly, paste
the values into the configuration form. Save the destination and it's ready to start syncing data.
## Syncing
When creating a sync, first you need to select the type of object you want your data to be transformed into. Following that, you
could define the ID column and the rest of the mappings.
If you have custom-defined fields in your account, they will appear as mappings, with the appropriate data type.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Close
**URL:** https://hightouch.com/docs/destinations/closeio
**Description:** Close more deals by keeping your prospect data up-to-date and in sync with data in your warehouse
**Section:** Destinations
## Overview
Your leads and prospects expect you to have the necessary context before engaging with them. You can accomplish this by syncing lead and prospect data from your data warehouse to Close. You no longer need to worry about executives chasing the wrong leads or engaging with warm ones without proper context.
## Supported syncing
|Sync Type|Description|Supported Sync Modes | API Reference|
|--|--|--|--|
|**Lead**|Sync data from any source to Leads|Upsert, Update| [Lead docs](https://developer.close.com/resources/leads/)|
|**Contact**|Sync data from any source to Contacts|Upsert, Update|[Contact docs](https://developer.close.com/resources/contacts/)|
|**Opportunity**|Sync data from any source to Opportunities|Upsert, Update|[Opportunity docs](https://developer.close.com/resources/opportunities/)|
|**Note Activity**|Sync data from any source to Note activities|Insert, Update| [Note docs](https://developer.close.com/resources/activities/note/)|
|**Custom Activity**|Sync data from any source to custom activities|Upsert, Update|[Custom Activities docs](https://developer.close.com/resources/custom-activities/)|
When inserting Note activities, Hightouch creates and attaches new Note activities to the Lead object you specify in your sync configuration.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Close
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Close** and click **Continue**. You can then authenticate Hightouch to Close with a Close API key.
You can generate a Close API Key in your Close account under [**Settings > API Keys**](https://app.close.com/settings/api/).
For more information, refer to [Close's docs](https://help.close.com/docs/api-keys).
## Sync configuration
Once you've set up your Close destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Close destination you want to sync to.
### Record matching
You can match rows from your model to Close records on any unique property.
You must match Contacts by either email or the contact's ID.
### Field mapping
Use field mapping to sync any model columns to Close's default and custom fields.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## CockroachDB
**URL:** https://hightouch.com/docs/destinations/cockroachdb
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Unique identifiers
Hightouch requires a unique identifier in the destination Cockroach table to add, remove, and update rows. The column must be either a `UNIQUE` or `PRIMARY KEY` column. At least one Hightouch field must map to a unique column.
If you're unsure whether your mapping includes a compatible column, just try it. Hightouch will error if you need to change your mappings.
## Column types
Hightouch works out of the box with all standard column types, including:
- `BIGINT`
- `INT`
- `TEXT`
- `VARCHAR`
- `TIMESTAMPTZ`
- `BOOLEAN`
- `DECIMAL`
If you see type errors, it may be because your SQL query is producing the wrong format. Message us on Intercom if you need help.
## Required permissions
Your Cockroach credentials must be able to:
- `SELECT`, `INSERT`, `UPDATE` and `DELETE` rows from the destination table.
- Perform the following queries:
- `SHOW SCHEMAS`
- `SHOW TABLES FROM `
- `SHOW CONSTRAINTS FROM .
`
- `SHOW COLUMNS FROM .
`
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## CJ Affiliate
**URL:** https://hightouch.com/docs/destinations/commission-junction
**Description:** CJ Affiliate's technology powers a partnership ecosystem where over 167K publishers and brands engage billions of consumers worldwide.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|------------|--------------------------------------------------|------------------------|------------------------------------------------------------------------------------|
| **Orders** | Sync data from any source to CJ Affiliate orders | Upsert, Update, Insert | [Orders docs](https://developers.cj.com/docs/advertiser-api-tracking/api-overview) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to CJ Affiliate
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **CJ Affiliate** and click **Continue**. You can then authenticate Hightouch to **CJ Affiliate**.
Enter the following fields into Hightouch:
- **Advertiser ID**: Enter your advertiser ID.
- **Access token**: Enter your personal access token.
## Sync configuration
Once you've set up your CJ Affiliate destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the CJ Affiliate destination you want to sync to.
### Syncing orders
Sync data from any source to CJ Affiliate orders.
#### Record matching
To match rows from your model to orders in CJ Affiliate, you need to select the model column that contains values that match the **Order ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, CJ Affiliate automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync order fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom order fields.
CJ Affiliate requires the following fields, so you must map them to complete your configuration:
- **Order ID**
- **Enterprise ID**
- **Action tracker ID**
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Common Room
**URL:** https://hightouch.com/docs/destinations/common-room
**Description:** Get a unified view and automate using every signal across social, community, product, CRM, and more. Initiate conversations, grow adoption, and accelerate pipeline faster than ever before.
**Section:** Destinations
## Supported syncing
| Sync Type | Supported Sync Modes |
| --------------------------------------- | -------------------- |
| **Community Members Information** | Upsert |
| **Community Organizations Information** | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Getting started
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Common Room** and click **Continue**. Then paste in your customer user Common Room URL.
## Sync configuration
Once you've set up your Common Room destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Common Room destination you want to sync to.
#### Field mapping
You can sync columns from your source to Common Room default and custom fields.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Contentstack
**URL:** https://hightouch.com/docs/destinations/contentstack
**Description:** Contentstack is a modern, headless CMS built for teams that want to ship content faster everywhere. Plug in any front-end and use real-time collaboration and workflow automation to keep content fresh and consistent across all platforms.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
----------|-------------------------------------------------|----------------------|----------------------------------------------------------------
**Users** | Sync data from any source to Contentstack users | Update | [Users docs](https://docs.lytics.com/reference/put_attributes-table-key-value)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Contentstack
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Contentstack** and click **Continue**. You can then authenticate Hightouch to **Contentstack** using a Data Activation Layer (DAL) access token.
### Create a Data Activation Layer access token
1) From your Contentstack account, create a new Data Activation Layer (DAL).
2) Access your DAL through **Data and Insights**, and under **Account** and **Access Tokens**, create a new access token. For token roles, select **Data Manager** under **Custom Roles** and generate your token.
If you set an expiration for your access token, ensure that your access token has not expired if you notice syncs failing due to authorization issues.
## Sync configuration
Once you've set up your Contentstack destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Contentstack destination you want to sync to.
### Syncing users
Sync data from any source to Contentstack users.
#### Record matching
To match rows from your model to users in Contentstack, you need to select a model column and corresponding Contentstack field. You can match on any of the following Contentstack fields:
- **External ID**
- **Email**
- **Hashed email**
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync user attributes via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom attributes that are defined in Contentstack.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Adding users to audiences
Contentstack allows you to pass a list of audience IDs to **External Audiences** for users to add them to audiences. The user will be added to every audience that is included in the list passed in. The unique IDs of audiences can be found under **Personalize** > Select a personalize project > **Audiences**.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Cordial
**URL:** https://hightouch.com/docs/destinations/cordial
**Description:** Sync data to Cordial through Contacts API
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
| ---------- | -----------------------------------------------------| ---------------------- | ------------------------------------------------------------------------------------------------------- |
| **Contacts** | Sync data from any source to Cordial as contacts | Upsert, Update, Insert | [Contacts](https://support.cordial.com/hc/en-us/articles/203885958-Contacts-API)|
| **Supplements** | Sync data from any source to Cordial's supplemental data sets | Upsert, Update, Insert | [Supplements](https://support.cordial.com/hc/en-us/articles/204570687-Supplements-API)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Cordial
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Cordial** and click **Continue**.
Enter your API key from your Cordial account.
If you don't have an API key, follow the [instructions on **Cordial** to create API keys](https://support.cordial.com/hc/en-us/articles/115005365087). Ensure the API key has write access for the object you are syncing.
Based on the sync object, ensure your API key has access to the following API endpoints:
| Object | Required API endpoints |
|--------------|------------------------------------|
| **Contacts** |
`GET /accountcontactattributes`
`GET/POST/PUT /contacts`
|
| **Supplements** |
`GET /supplements`
`GET/POST/PUT /supplements//records`
|
Next, select which Cordial domain your account uses. This will either be `admin.cordial.io` or `usw2.admin.cordial.io`. You can determine which one your account uses by navigating to your Cordial dashboard and checking the website URL.
## Sync configuration
Once you've set up your Cordial destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Cordial destination you want to sync to.
Select which object you'd like to sync.
### Sync mode
- **Upsert**: pushes new objects to Cordial and updates mapped fields that change in your source.
- **Update**: updates mapped fields on existing objects in Cordial. New objects are not added.
- **Insert**: pushes new objects to Cordial. Existing objects are not updated.
### Field mapping
#### Contacts
Cordial has the following [contact attribute types](https://support.cordial.com/hc/en-us/articles/204570347-Accountcontactattributes):
- string
- number
- date (ISO 8601 standard)
- geo
- array
Hightouch supports syncing with all of these types. You may use Hightouch's object inline mapping to construct Cordial's object values from your source.
You **must** create a contact attribute on Cordial before syncing to that field. Follow the [instructions on **Cordial**](https://support.cordial.com/hc/en-us/articles/115005528368-Contact-attributes#addAttributes) either via UI or API to create contact attributes.
#### Supplements
Start syncing supplements by [creating a supplement collection on Cordial](https://support.cordial.com/hc/en-us/articles/115005528468-Supplements#ui). Indexes on supplement collections are **required** mapping fields on Hightouch.
If a field you'd like to map to is not available in the mapping dropdown, you can enter the field name manually:
### Record matching
#### Contacts
In **Upsert** and **Update** mode, records must be matched by either Cordial ID or Email. Records without unique IDs will fail to sync.
In **Insert** mode, records must have an **email field** to successfully be added to Cordial. Records without an email identifier will fail to sync.
#### Supplements
Since Cordial does not create ID fields for supplements, **id** is required in **Insert** mode. In **Upsert** and **Update** mode, **id** is also required to match records. Records without unique IDs will fail to sync.
Cordial [does not allow](https://support.cordial.com/hc/en-us/articles/204570687-Supplements-API#postSuppRecords) 24 character hex strings as ID values for supplements.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Courier
**URL:** https://hightouch.com/docs/destinations/courier
**Description:** Power product notifications with your warehouse data in Courier
**Section:** Destinations
## Supported sync methods
Hightouch can sync data to Courier Profiles and List Subscriptions
### Courier profiles
#### Sync modes
Hightouch supports Upsert and Update sync modes:
- In the Upsert mode, new rows will be inserted into Courier and Profile attributes will be kept up-to-date within Courier
- In the Update mode, Profile attributes will be kept up-to-date within Courier.
#### Record matching
Database records can be matched from your source to your Courier workspace by a unique property. Profiles can be matched by a `Recipient ID`.
#### Field mapping
You can sync columns from your source to Courier's Profile default and custom fields.
#### Delete mode
You can choose what Hightouch's behavior is when records leave the query result set. The default is doing nothing, but you can also set Hightouch to delete or clear the Courier profile on record exit.
### Creating a new list
In your sync configuration, you can tell Hightouch to create a new Courier list for you.
You can optionally assign a custom name to your list. If no name is provided, Hightouch will default to using your model name.
### Syncing to existing lists
Instead of creating a new list, Hightouch can also sync recipients to existing lists that are already populated in your Courier workspace.
Suppose you want to sync to multiple existing lists but do not want to create a new sync for every list. As long as your model has a `list_id` column associated to a row, Hightouch can sync to multiple Courier lists in just one sync.
## Getting started
If you're new to Hightouch, be sure to [sign up for a free account](https://app.hightouch.com/signup?from=docs). Your first destination is free, and Courier is a great way to get started.
Before diving into this tutorial, we also recommend reading [this article on our core concepts](/getting-started/concepts). It's a quick overview of how Hightouch works.
### Connect to Courier
#### Create an API token
1. Log in to Courier
2. Visit your **Settings** page
3. Click on the **API Keys** tab
4. If you already have an API key set up, copy the **Published** token, otherwise click **Generate New API Key**
#### Add Courier as a destination in Hightouch
1. Log in to Hightouch or create a free Hightouch account
2. Click on **Destinations** in the left sidebar
3. Click on **Add destination** in the top right corner
4. Select **Courier** in the destination catalog and click **Continue** to proceed
5. Paste your API token into the **API Key** field and click **Continue** to proceed
6. Give your destination a name and unique slug (for example, “Courier Production” and `courier-production`)
7. Click **Finish** to create your Courier destination.
Now that we've linked Hightouch to your Courier account, it's time to connect to build profiles and list subscriptions using the data that resides in your data warehouse.
### Connect to your data source
Hightouch sits on top of your data warehouse and can read from Snowflake, Redshift, BigQuery, Databricks, Postgres, MySQL, and many other data sources. You can even fetch data from Google Sheets and Airtable. Hightouch can help activate your data wherever it resides.
We've written dedicated guides for each supported data source. Take your pick:
- Connect to [Airtable](/sources/airtable)
- Connect to [Amazon Athena](/sources/amazon-athena)
- Connect to [Amazon Redshift](/sources/amazon-redshift)
- Connect to [ClickHouse](/sources/clickhouse)
- Connect to [Databricks](/sources/databricks)
- Connect to [Elasticsearch](/sources/elasticsearch)
- Connect to [Firebolt](/sources/firebolt)
- Connect to [Google BigQuery](/sources/google-bigquery)
- Connect to [Google Sheets](/sources/google-sheets)
- Connect to [Looker](/sources/looker)
- Connect to [Metabase](/sources/metabase)
- Connect to [MySQL](/sources/mysql)
- Connect to [Palantir Foundry](/sources/foundry)
- Connect to [PostgreSQL](/sources/postgresql)
- Connect to [Rockset](/sources/rockset)
- Connect to [SFTP](/sources/sftp)
- Connect to [Snowflake](/sources/snowflake)
- Connect to [SQL Server](/sources/sqlserver)
- Connect to [Trino](/sources/trino)
If you're missing the credentials necessary to connect to your data source,
try inviting your teammates to Hightouch. You can have unlimited collaborators
in your workspace.
#### Connect to a sample database instead
Waiting for credentials to your data warehouse? No problem. You can connect to one of our [sample datasets](/sources/sample-data) to get started right away. For the rest of this tutorial, we'll use the **B2B SaaS** sample source, which contains a `users` table with names, emails, locations, and other user attributes. This sample source is available when you log in for the first time—no setup required.
Next, we'll write a SQL query to define which profiles and list subscriptions should be synced to Courier.
### Create a model
In Hightouch, a model represents a query that filters or transforms the records in your data source. Models are used to determine exactly which records to sync between sources and destinations.
When you connect to a data warehouse, your models will probably be written in SQL. You can also import models from tools like dbt and Looker.
In this example, we'll keep it simple. Suppose you want to use Courier to send notifications only for your customers located in the city of Nashville. To accomplish this, you'll want to create a model that looks like this:
```sql
SELECT * FROM users WHERE location = 'Nashville';
```
1. Click on **Models** in the left sidebar
2. Click on **Add model** in the top right corner
3. Select **Demo Database** as your source and click **Continue** to proceed
4. Select **SQL Editor** as your modeling method
5. Paste the SQL query from above into the editor on the left side of the page
6. Click **Preview** to execute the query
7. Click **Continue** to proceed
8. Give your model a name and unique slug (for example, "Users in Nashville" and `users-in-nashville`)
9. Select **ID** as the primary key for your model
10. Click **Finish** to create your model.
We just created a model using a SQL query that filters our `users` table down to only those located in Nashville. Next, we'll configure our sync to add these users to Courier.
### Create a sync to Courier Profiles
1. Click on **Syncs** in the left sidebar
2. Click on **Add sync** in the top right corner
3. Select **Users in Nashville** as your model and click **Continue** to proceed
4. Select **Courier Production** as your destination and click **Continue** to proceed
5. Select **Profile** as the object you wish to update in Courier
6. Select which update mode you want to use
7. Identify which field from your model represents the identifier known to Courier
8. Identify which fields from your model you want to sync to a Courier Profile
9. Select how you want to handle records leaving your source
10. Give a name to your segment (for example, "My beta test")
11. Click **Continue** to proceed to the last step
12. Select **Interval** to tell your sync to run on a set interval
13. Configure your sync schedule to run every **5 minutes**
14. Click **Finish** to create your sync.
Your Courier sync configuration would look something like this.
And your sync schedule configuration would resemble this.
### Create a sync to Courier Lists
1. Click on **Syncs** in the left sidebar
2. Click on **Add sync** in the top right corner
3. Select **Users in Nashville** as your model and click **Continue** to proceed
4. Select **Courier Production** as your destination and click **Continue** to proceed
5. Identify which field from your model represents the identifier known to Courier
6. Select Create a new segment
7. Give a name to your segment (for example, "My beta test")
8. Click Continue to proceed to the last step
9. Select Interval to tell your sync to run on a set interval
10. Configure your sync schedule to run every 5 minutes
11. Click Finish to create your sync.
Your Courier sync configuration would look something like this.
And your sync schedule configuration would resemble this.
## Tips and troubleshooting
### Common errors
#### Don't see your data in Courier?
Make sure you've configured your destination to use the correct Courier environment API key. There may be quite a few so you might've accidentally copied the wrong one. If that doesn't solve your issue, . We're happy to help.
### Live debugger
### Sync alerts
---
## Criteo
**URL:** https://hightouch.com/docs/destinations/criteo
**Description:** Reach the people who will care most about you, whether they’re just discovering you, or have engaged with you before.
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| ------------- | --------------------------------------------------- | -------------------- | --------------------------------------------------------------------------------------- |
| [**Audiences**](#syncing-audiences) | Create audiences and update contact list membership | Add, Remove | [Audiences reference](https://developers.criteo.com/marketing-solutions/docs/audiences) |
## Connect to Criteo
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Criteo** and click **Continue**. You can then authenticate Hightouch to Criteo using OAuth.
Click **Log in to Criteo**, log in with your Criteo account, and click **Approve**.
Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Criteo destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Criteo destination you want to sync to.
Whether you're syncing events or audiences, begin your configuration by selecting the advertiser ownign the audience you want to sync to.
### Syncing audiences
Syncing to Criteo's [Audience API](https://developers.criteo.com/marketing-solutions/docs/audiences) lets you manage
your contact lists by creating, deleting, and updating audience data used for advertising campaigns.
#### Select an existing audience or create a new one
You can create a new audience or use an existing one. When creating a new audience, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model. To use an existing audience, select the desired audience from the dropdown.
#### User identifiers
To identify which users to add or update in an audience, select model columns and the corresponding Criteo fields.
You can match on any of the following Criteo fields:
- Email
- Mobile ad identifier
- LiveRamp identity link
- Criteo GUM cookie identifier
When using the Criteo GUM cookie identifier, you must also enter your
advertiser's GUM Caller ID. To get the GUM Caller ID, reach out to your Criteo
account team.
#### Delete behavior
You can choose how to handle user records in Criteo when the corresponding rows are deleted in your source.
| Behavior | Description |
| -------------------- | -------------------------------------------- |
| **Do nothing** | Keep the contact in your Criteo audience |
| **Remove from list** | Remove the contact from your Criteo audience |
## Tips and troubleshooting
### Matched users count
### Common errors
### Live debugger
### Sync alerts
---
## CrowdTwist
**URL:** https://hightouch.com/docs/destinations/crowdtwist
**Description:** Oracle CrowdTwist is a SaaS based omni-channel loyalty and analytics platform that is designed to allow marketers to acquire, engage and retain customers
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
| ---------- | -----------------------------------------------------| ---------------------- | ------------------------------------------------------------------------------------------------------- |
| **Users** | Sync data from any source to CrowdTwist as users | Update | [User Update](https://docs.oracle.com/en/cloud/saas/marketing/crowdtwist-develop/Developers/UserUpdate.html)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to CrowdTwist
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **CrowdTwist** and click **Continue**.
Enter your API key. If you do not have a CrowdTwist API key, follow the instructions [here on CrowdTwist](https://docs.oracle.com/en/cloud/saas/marketing/crowdtwist-develop/Developers/GeneratingAPIKeys.htm) to generate a Client API key.
## Sync configuration
Once you've set up your CrowdTwist destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the CrowdTwist destination you want to sync to.
### Record matching
CrowdTwist uses the following ID types to identify users:
- Email address
- Meta user ID
- X user ID
- CrowdTwist ID
- Third party ID
- Username
- Mobile phone number
Select which ID you'd like to use to identify users in your model.
### Field mapping
Sync any column in your model to map to any of the following user attributes in CrowdTwist:
- First name
- Last name
- Date of birth
- Email address
- Email is verified
- Is active
- Remove points
- Language preference
- Postal code
- Username
- Password
- Third party ID
- City ID
- City name
- Country code
- Meta access token
- Meta user ID
- Instagram access token
- Instagram username
- Instagram user ID
- Foursquare access token
- Foursquare user ID
- Extra data
- Gender ID
- Middle name
- Mobile phone number
- Receive email updates
- Custom data
- Tier override
- Password generated
Reference [CrowdTwist's documentation](https://docs.oracle.com/en/cloud/saas/marketing/crowdtwist-develop/Developers/UserUpdate.html) for more detail on the types and requirements for each attribute to reduce the likelihood of rejected rows in your sync.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Embedded Destination
**URL:** https://hightouch.com/docs/destinations/custom
**Description:** Build a custom connector that looks and feels just like a native destination
**Section:** Destinations
If you need to connect to a destination Hightouch doesn't support yet, we recommend using the [HTTP Request destination](/destinations/http-request). Please also —we're always gauging interest in new destinations.
## Overview
The Embedded Destination framework allows Hightouch users to develop their own custom data connector and use it as an extension of Hightouch's existing library.
If you are looking to send data to a private API or a SaaS tool that Hightouch doesn't support, you can use Hightouch to send data to your own connector in a specific format, while continuing to use all of our alerting, reliability, and error handling systems.
To implement the destination, Hightouch will need users to provide information on what objects, modes, and fields are available in the destination, as well as how data is synced.
## Example destinations
Our team has produced some [example destinations](https://github.com/hightouchio/custom-destinations) for reference.
- Slack: Sends messages formatted with Liquid to a Slack channel as rows are added to the query results
- Mailchimp: Adds and updates contacts in a Mailchimp audience list
## Setup
Embedded destinations in Hightouch are implemented by providing information of two steps:
- Sync configuration: Hightouch needs information on what objects, modes, and fields are available, as well as custom forms
- Sync execution: Hightouch sends data that needs to be synced to a method on your connector, which should format and send the data to the destination and return a success or error response
### Sync configuration
Hightouch uses the following methods to render the configuration form.
These methods can be optionally implemented and used in any combination:
- `list_objects`: Retrieves a list of objects available in the destination (for example, users, organizations, etc.)
- `list_modes`: Retrieves a list of modes available in your implementation (for example, upsert, update, mirror, etc.)
- `list_fields`: Retrieves a list of object fields, including identifiers and custom fields
- `list_options`: Retrieves a JSON schema of custom configuration fields, which will be rendered into a form based on [react-jsonschema-form](https://rjsf-team.github.io/react-jsonschema-form)
- `validate`: This method is called when a user attempts to save the configuration form, and is used to display errors that aren't handled automatically by default or by JSON schema
#### Example 1: Objects, modes, fields, and options
When all the methods are implemented and all the concepts are used in conjunction, there is a hierarchy where objects must be selected first before modes are shown, and modes must be selected before fields and options are shown.
This is because the mode options shown are conditional on the object selected, for example, "upsert" and "update" can be enabled for the "user" object, but only "update" is enabled for the "organizations" object.
This will render a form similar to the example above.
#### Example 2: Options only
When only `list_options` is implemented, Hightouch will render a form based on the JSON schema. Note that all formats such as dropdowns, arrays, text areas are available. In this case, only the `list_options` method is implemented.
The other methods such as `list_objects` aren't implemented, and therefore selecting an object is not required.
### Sync execution
The Embedded Destination acts as a transformation and parsing layer between Hightouch and any API. Hightouch calls the following methods when a sync is run, either by schedule or manual run.
When a sync is run, Hightouch first calls for information on sync behavior, then sends sync requests of each operation (add, change, remove) with the data and configuration in batches of a specified size:
- `behavior`: Retrieves behavior settings such as batch size, which operations to send, etc. for the sync run
- `add`: This method is called with a batch of raw data from rows that have been added since the last sync run, as well as configuration details
- `change`: This method is called with a batch of raw data from rows that have changed since the last sync run, as well as configuration details
- `remove`: This method is called with a batch of raw data from rows that have been removed since the last sync run, as well as configuration details
Each of the `add`, `change`, `remove` methods should return a response with any errors encountered when sending the data to the destination API.
## API reference
Implement the following methods in a JSON-RPC server available via a public URL.
The server can be implemented with packages such as:
- [Jayson](https://github.com/pavlov99/json-rpc): A JSON-RPC server for node.js
- [json-rpc](https://github.com/pavlov99/json-rpc): A JSON-RPC implementation written on Python
### `list_objects`
This method should return the objects that are available to sync to in the destination API.
#### Request body
`list_objects` has no request body since it's the highest level method.
#### Response sample
```json
{
"objects": [
{ "label": "User", "id": "user", "description": "A user." },
{
"label": "Organization",
"id": "organization",
"description": "An organization."
}
]
}
```
#### Properties of an `object`
| Name | Description |
| ---------------------- | ------------------------------------------------------------------ |
| `id` | ID of object option will be referenced in the configuration |
| `label` | Label of object option shown in the dropdown |
| `description` (optional) | Description of object option (not shown in the dropdown currently) |
### `list_modes`
This method should return the sync modes that are available for a specific object (or no object).
#### Request body
| Name | Description |
| ------ | ------------------------------------------------------------------------------------------------------------------------------ |
| `object` | ID of object selected in the configuration (can be undefined if `list_object` not implemented or returned an empty object array) |
#### Response sample
```json
{
"modes": [
{
"id": "upsert",
"label": "Upsert",
"description": "Pushes new records and updates records that change in your source."
}
]
}
```
#### Properties of a `mode`
| Name | Description |
| ---------------------- | --------------------------------------------------------- |
| `id` | ID of mode option will be referenced in the configuration |
| `label` | Label of mode option shown in the radio selector |
| `description` (optional) | Description of mode option shown in the radio selector |
### `list_fields`
This method should return the mapping fields that are available for a specific object and mode (or no object and mode).
#### Request body
| Name | Description |
| ------ | ------------------------------------------------------------------------------------------------------------------------------ |
| `object` | ID of object selected in the configuration (can be undefined if `list_object` not implemented or returned an empty object array) |
| `mode` | ID of mode selected in the configuration (can be undefined if `list_mode` not implemented or returned an empty mode array) |
#### Response sample
```json
{
"fields": [
{
"id": "email",
"label": "Email Address",
"type": "STRING",
"identifier": true
},
{
"id": "first_name",
"label": "First Name",
"type": "STRING",
"required": true
},
{
"id": "last_name",
"label": "Last Name",
"type": "STRING",
"required": true
}
]
}
```
#### Properties of a `field`
| Name | Description |
| ---------------------- | ------------------------------------------------------------------------- |
| `id` | ID of the field option will be referenced in the configuration |
| `label` | Label of field option shown in the dropdown |
| `description` (optional) | Description of field option (not shown in the dropdown currently) |
| `required` (optional) | Whether the field is required to be mapped |
| `type` (optional) | The type of the field (check the field types section for accepted values) |
#### Field types
Hightouch supports the following field types. Hightouch will not automatically cast the value to this type, nor will it validate whether the data in the row is of that type. it's generally just for suggestion and display purposes.
```jsx
STRING;
ENUM;
NUMBER;
BOOLEAN;
DATE;
DATETIME;
EMAIL;
OBJECT;
ARRAY;
REFERENCE;
UNKNOWN;
```
### `list_options`
This method should return the `schema` and `uiSchema` of the form that you would like to render. Hightouch uses [React JSON Schema Form](https://react-jsonschema-form.readthedocs.io/en/latest/) to build a form out of a semi-standard JSON Schema.
If your schema at the highest level is an object, we will spread the properties into the configuration, how
| Name | Description |
| ------ | ------------------------------------------------------------------------------------------------------------------------------ |
| `object` | ID of object selected in the configuration (can be undefined if `list_object` not implemented or returned an empty object array) |
| `mode` | ID of mode selected in the configuration (can be undefined if `list_mode` not implemented or returned an empty mode array) |
#### Response sample
```json
{
"options": {
"schema": {
"type": "object",
"properties": {
"channel": {
"title": "Channel",
"description": "Channel that messages are sent to.",
"type": "string",
"enum": [1, 2, 3],
"enumNames": ["#general", "#engineering", "#marketing"]
},
"message": {
"label": "Message",
"description": "Template for messages in Liquid format.",
"type": "string"
}
}
},
"uiSchema": {
"message": {
"ui:widget": "textarea"
}
}
}
```
#### Properties of `options`
| Name | Description |
| -------- | --------------------------------------------------------------------------------------------- |
| `schema` | This property defines which fields will appear in the custom configuration form. |
| `uiSchema` | This property defines the appearance of fields, such as converting text inputs to text areas. |
### `validate`
This method receives a JSON of the configuration and should return an array of errors. If there is no error, it should return an empty array.
#### Request body
| Name | Description |
| ------------- | ------------------------------------------------------------------------------------- |
| configuration | The sync configuration as a JSON, including properties such as object, mode, mappings |
#### Request sample
```json
{
"configuration": {
"object": "user",
"mode": "upsert",
"identifierMapping": [{ "from": "user_email", "to": "email" }],
"mappings": [
{ "from": "user_fn", "to": "first_name" },
{ "from": "user_ln", "to": "last_name" }
],
"customMappings": [{ "from": "user_ltv", "to": "ltv" }],
// the following properties come from custom configuration from list_options
"archiveOnRemove": true
}
}
```
#### Response sample
```json
{
"errors": ["If first name is mapped, last name must also be mapped."]
}
```
### `behavior`
This method should return the operations that Hightouch should send, as well as the batch size Hightouch should send at.
#### Request body
`behavior` has no request body.
#### Response sample
```json
{
"batchSize": 100,
"operations": {
"add": true,
"change": true,
"remove": false
},
"skipInitialRun": false
}
```
### `add` `change` `remove`
This method should send the batch of data forward to the destination API and return an array of rejected rows if any errors occurred.
#### Request body
| Name | Description |
| ------------- | ------------------------------------------------------------------------------------- |
| `idColumn` | The column name of the ID selected as the primary key |
| `configuration` | The sync configuration as a JSON, including properties such as `object`, `mode`, `mappings` |
| `rows` | An array of rows from the query results |
#### Request sample
```json
{
"idColumn": "user_email",
"configuration": {
"object": "user",
"mode": "upsert",
"identifierMapping": [{ "from": "user_email", "to": "email" }],
"mappings": [
{ "from": "user_fn", "to": "first_name" },
{ "from": "user_ln", "to": "last_name" }
],
"customMappings": [{ "from": "user_ltv", "to": "ltv" }],
// the following properties come from custom configuration from list_options
"archiveOnRemove": true
},
"rows": [
{
"user_email": "alice@hightouch.com",
"user_fn": "Alice",
"user_ln": "Doe",
"user_ltv": 0
},
{
"user_email": "bob@hightouch.com",
"user_fn": "Bob",
"user_ln": "Doe",
"user_ltv": 10
}
]
}
```
#### Response sample
```json
{
"rejectedRows": [
{ "id": "alice@hightouch.com", "reason": "Couldn't insert user" }
]
}
```
#### Properties of a `rejectedRow`
| Name | Description |
| ------ | ---------------------------------------- |
| `id` | The ID of the row referenced by `idColumn` |
| `reason` | Reason for failure |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Customer.io
**URL:** https://hightouch.com/docs/destinations/customerio
**Description:** Deliver exceptional engagement campaigns using the freshest customer data from your warehouse
**Section:** Destinations
## Overview
You want to provide relevant, timely interactions to your customers. Doing so means that data in your Customer.io account needs to be as accurate and fresh as possible. By syncing customer data directly from your data warehouse into Customer.io, you no longer need to worry about data consistency.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
|---------------|-------------------------------------------------------|----------------------|
| **Customers** | Sync data from any source to customer objects | Upsert, Update |
| **Events** | Sync data from any source as customer events | Insert |
| **Segments** | Sync data from any source to new or existing segments | Insert |
| **Objects** | Sync data from any source to custom objects | Upsert, Update |
## Getting started
To give Hightouch access to Customer.io, you need to enter:
- tracking site ID
- tracking API key
- app API key
- the region of your Customer.io account
Follow the instructions in the [following sections](#retrieve-tracking-site-id-and-api-key) to retrieve your site ID and API keys.
If you restrict access to your Customer.io workspaces by IP address, you need to add the [Hightouch IP Addresses](/security/networking#ip-addresses) to your **IP Address Allowlist**.
If you see "**Any IP address is currently able to access the App API for all workspaces.**" in your Customer.io configuration, it means you can skip this step.
### Retrieve tracking site ID and API key
1. Navigate to your [Account Settings](https://fly.customer.io/settings) and [Manage API Credentials](https://fly.customer.io/settings/api_credentials?keyType=tracking) page.
2. Under **Tracking API Keys**, copy a pre-created **Site ID** and **API Key**, or create a new one to use for Hightouch.
### Retrieve app API key
1. Navigate to your [Account Settings](https://fly.customer.io/settings) and to the App API Keys tab under the [Manage API Credentials](https://fly.customer.io/settings/api_credentials?keyType=app) page.
2. Create an **App API Key** and assign it to the appropriate workspace, for example "Hightouch."
## Syncing data
### Customers
Sync data from any source to [customer objects](https://www.customer.io/docs/api/#tag/Track-Customers).
#### Record matching
You can match rows in your model with users in Customer.io on the following fields:
- Customer.io External ID
- Email
- CIO ID
#### Field mapping
You can sync columns from your source to Customer.io customer attributes. If the customer attribute doesn't exist, Hightouch creates it.
#### Delete behavior
In **Upsert** mode, you have the following options for how Hightouch should handle rows leaving your query results:
| Behavior | Description |
| ----------------------------- | ------------------------------------------------------ |
| **Do nothing** | Keep the customer in Customer.io |
| **Clear Fields** | Clear the mapped fields but keep the customer record |
| **Delete Destination Record** | Delete the customer record from Customer.io completely |
In **Update** mode, you have the following options for how Hightouch should handle rows leaving your query results:
| Behavior | Description |
| ---------------- | ---------------------------------------------------- |
| **Do nothing** | Keep the customer in Customer.io |
| **Clear Fields** | Clear the mapped fields but keep the customer record |
### Objects
Sync data from any source to [custom objects](https://docs.customer.io/api/track/#section/Identify-object).
#### Record matching
You can match rows in your model with objects in Customer.io on the `Object ID` field
#### Field mapping
You can sync columns from your source to Customer.io object attributes. If the object attribute doesn't exist, Hightouch creates it.
#### Relationships
You can create relationships from each object synced to a customer in Customer.io. You can create relationships to the following fields:
- Customer.io External ID
- Email
- CIO ID
#### Delete behavior
You have the following options for how Hightouch should handle rows leaving your query results:
| Behavior | Description |
|-------------------------------|------------------------------------------------------|
| **Do nothing** | Keep the object in Customer.io |
| **Delete destination record** | Delete the object record from Customer.io completely |
#### Record matching
You can match rows in your model with users in Customer.io on the following fields:
- Customer.io External ID
- Email
- CIO ID
### Events
You can sync the following [customer event](https://www.customer.io/docs/api/#operation/track) types:
- **Track event**
- **Track page views**
- **Track mobile screen views**
#### Event name
Provide an event name. This is how you can reference the event in Customer.io campaigns or segments.
#### Event ID (optional)
Event ID is an identifier Customer.io uses to de-duplicate events. If provided and an event has the same value as previously synced event, Hightouch doesn't process the duplicate event.
#### User ID
A User ID is a unique value representing a user. You may identify a user by ID, email address, or Customer.io ID.
If you wish to sync the events as [_anonymous_](https://www.customer.io/docs/api/#operation/trackAnonymous), you can skip providing a User ID and provide an Anonymous ID instead.
#### Event timestamp
Provide a timestamp of when the event occurred. If this field is empty, Hightouch uses the time the event arrives at the server.
Any `DateTime` value provided from your source is converted to Unix timestamp as expected by Customer.io.
#### Field mapping
You can sync columns from your source to Customer.io customer attributes. If the customer attribute doesn't exist, Hightouch creates it.
### Segments
Sync data from any source to [new or existing segments](https://www.customer.io/docs/api/#tag/Track-Segments).
#### User identifiers
You can match rows in your model with users in Customer.io on the following fields:
- Customer.io External ID
- Email
- CIO ID
#### Create a segment
- To create a new mailing list, select `Create a new segment`.
- You can provide a name for this segment, or Hightouch use the name of the model used to create the sync.
#### Use an existing segment
- To use an existing mailing list, select `Use existing segment`.
- You can select any of your existing Customer.io segments from the drop-down select menu.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Delighted
**URL:** https://hightouch.com/docs/destinations/delighted
**Description:** Create or update your customers' information while sending surveys with rich segmentation data directly from your warehouse.
**Section:** Destinations
## Overview
The Delighted integration allows you to sync customer data and send surveys from your warehouse based on models that you define.
### Why sending surveys should be based on the warehouse
- You can do more complex segmentation by leveraging all the data in your warehouse, including event data and third-party data. Your warehouse is the only place where all of this data coexists.
- You can keep your Delighted segments in sync with the rest of your business tools. For example, you might want to send a survey to users who've been invited to a beta program. You also might want to sync these attributes to a CRM like Salesforce or HubSpot. Hightouch integrates with SaaS tools used for email marketing, customer support, sales, and more. Define your audience once and activate your data everywhere.
- Segments can be defined using the tools that your teams are most comfortable with. You can filter and transform data from your warehouse using SQL or our [visual query builder](/customer-studio/usage#setup)—whichever you prefer. (You can even sync customers to or send surveys through Delighted segments based on work you've already done in dbt or Looker!)
## Getting started
### Connect to Delighted
**Retrieve your API Token from Delighted.**
1. Log in to Delighted.
2. Visit [Delighted's API Introduction Page](https://app.delighted.com/docs/api).
3. Copy your **API key**.
#### Add Delighted as a destination in Hightouch
1. Log in to Hightouch or create a free Hightouch account
2. Click on **Destinations** in the left sidebar
3. Click on **Add destination** in the top right corner
4. Select **Delighted** in the destination catalog and click **Continue** to proceed
5. Paste your **API key** into the **API key** field and click **Continue** to proceed
6. Give your destination a name and unique slug (for example, Delighted Production” and `Delighted-production`)
7. Click **Finish** to create your Delighted destination
### Connect to your data source
Hightouch sits on top of your data warehouse and can read from Snowflake, Redshift, BigQuery, Databricks, Postgres, MySQL, and many other data sources. You can even fetch data from Google Sheets and Airtable. Hightouch can help activate your data wherever it resides.
We've written dedicated guides for each supported data source. Take your pick:
- Connect to [Airtable](/sources/airtable)
- Connect to [Amazon Athena](/sources/amazon-athena)
- Connect to [Amazon Redshift](/sources/amazon-redshift)
- Connect to [ClickHouse](/sources/clickhouse)
- Connect to [Databricks](/sources/databricks)
- Connect to [Elasticsearch](/sources/elasticsearch)
- Connect to [Firebolt](/sources/firebolt)
- Connect to [Google BigQuery](/sources/google-bigquery)
- Connect to [Google Sheets](/sources/google-sheets)
- Connect to [Looker](/sources/looker)
- Connect to [Metabase](/sources/metabase)
- Connect to [MySQL](/sources/mysql)
- Connect to [Palantir Foundry](/sources/foundry)
- Connect to [PostgreSQL](/sources/postgresql)
- Connect to [Rockset](/sources/rockset)
- Connect to [SFTP](/sources/sftp)
- Connect to [Snowflake](/sources/snowflake)
- Connect to [SQL Server](/sources/sqlserver)
- Connect to [Trino](/sources/trino)
If you're missing the credentials necessary to connect to your data source,
try inviting your teammates to Hightouch. You can have unlimited collaborators
in your workspace.
#### Connect to a sample database instead
Waiting for credentials to your data warehouse? No problem. You can connect to one of our [sample datasets](/sources/sample-data) to get started right away. For the rest of this tutorial, we'll use the **B2B SaaS** sample source, which contains a `users` table with names, emails, locations, and other user attributes. This sample source is available when you log in for the first time—no setup required.
Next, we'll write a SQL query to define which users should be synced to our Delighted customers resource. You can follow the same pattern to sync to other support Delighted resources,
### Create a model
In Hightouch, a model represents a query that filters or transforms the records in your data source. Models are used to determine exactly which records to sync between sources and destinations.
When you connect to a data warehouse, your models will probably be written in SQL. You can also import models from tools like dbt and Looker.
In this example, we'll keep it simple. Suppose you want to sync all your customers to Delighted. To accomplish this, you'll want to create a model that looks like this:
```sql
SELECT * FROM users;
```
1. Click on **Models** in the left sidebar.
2. Click on **Add model** in the top right corner.
3. Select **Demo Database** as your source and click **Continue** to proceed.
4. Select **SQL Editor** as your modeling method.
5. Paste the SQL query from above into the editor on the left side of the page.
6. Click **Preview** to execute the query.
7. Click **Continue** to proceed.
8. Give your model a name and unique slug (for example, "All Users" and `all-users`).
9. Select **ID** as the primary key for your model.
10. Click **Finish** to create your model.
We just created a model using a SQL query that retrieves all our current users. Next, we'll configure our sync to add these users to Delighted.
### Create a sync
1. Click on **Syncs** in the left sidebar.
2. Click on **Add sync** in the top right corner.
3. Select **All Users** as your model and click **Continue** to proceed.
4. Select **Delighted** as your destination and click **Continue** to proceed.
5. Give your destination a name and unique slug (for example, "Delighted All Customers Sync" and `Delighted-all-customers-sync`) and click **Finish** to proceed.
6. Select **People** from the object dropdown.
7. Select **Upsert** from the mode radio options.
8. Select `email` and we will map it to `Email` from the external ID mapping field.
9. Select `first_name` and we will map it to `Name` in the fields that we'd like to sync to Delighted.
10. Select **Yes** in the Sending Survey section.
11. Select **Do Nothing** in the Delete section.
12. Click **Continue** to proceed to the last step.
13. Select **Interval** to tell your sync to run on a set interval.
14. Configure your sync schedule to run every **5 minutes**.
15. Click **Finish** to create your sync.
And We're done. Your Delighted sync configuration is complete.
## Sync configuration
### Object selection
Hightouch supports syncing to the following Delighted resources:
- **People**
### Sync modes
This integration supports **Upsert**.
In **Upsert** mode, records that aren't found in Delighted given the provided identifier will be inserted. And existing records will be updated.
### Record matching
Records can be matched from your source to your Delighted workspace by `Email`.
### Field mapping
Hightouch allows you to sync columns from your source to the supported Delighted resources:
| Mapping | Type | Description |
| :----------------------- | :----: | -------------------------------------------------------------------------------------------------------: |
| `Phone Number` | String | Phone number of customer. Format must be E.164 (for example, +17132746524). |
| `Survey Channel` | String | Defaults to `email`. Specify `sms` and provide a `Phone Number` to have the survey request sent via SMS. |
| `Name` | String | Name of person. |
| `Delay To Send Email` | Number | Delay in `seconds` before sending the survey. |
| `Update Email To` | String | Field to update the person's email. |
| `Update Phone Number To` | String | Field to update the person's phone number. |
### Custom field mapping
Hightouch allows you to set custom attributes. Any mappings from this section will be synced to the custom properties of each associated survey sent. You can add as many properties as you need.
Properties are a powerful way to analyze survey results on Delighted. To learn more about it, visit [Properties Overview](https://help.delighted.com/article/575-properties-overview).
### Sending surveys
When configuring your Delighted sync form, there are a couple of choices provided to you when it comes to sending surveys. You can either:
- **Not** send surveys to all synced records.
- **Send surveys** to all synced records. Records that are created will be sent a survey. Records that are updated and are out of Delighted's survey throttling interval will be sent a survey.
- **Send surveys** based on each record **dynamically** by providing a `Boolean` column. To use this feature, toggle **Use Column**, and select that respective column from the dropdown.
### Delete records
When the Hightouch sync engine recognizes that a record has been removed from your source, you can either:
- Do Nothing.
- Delete the record from Delighted.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Digital Turbine
**URL:** https://hightouch.com/docs/destinations/digital-turbine
**Description:** Empower advertisers to grow the bottom line by targeting your first-party audiences with in-app ad experiences across mobile devices.
**Section:** Destinations
## Overview
Empower advertisers to grow the bottom line by targeting your first-party audiences
with in-app ad experiences across mobile devices.
Deliver more relevant ads by combining data from various sources within your data
warehouse to target and suppress your first-party audiences with in-app ad
experiences across mobile devices. By keeping your custom audiences updated
automatically, never show an ad to someone who shouldn't see it.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------- | ---------------------------------- | -------------------- |
| **Audiences** | Create and update custom audiences | All |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Digital Turbine
This integration requires prior approval from the Digital Turbine team and the
Hightouch team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your Digital Turbine rep who will verify approval. Once we have confirmed,
we will unlock the integration for you to connect.
To connect, input the `Advertiser Name` provided by your Digital Turbine rep.
### Syncing custom audiences
You can input the name of the audience you plan on delivering to Digital Turbine.
#### User identifiers
To identify which users to add or update in an audience, select model columns
and the corresponding Digital Turbine fields. You can match on any of the following user
fields:
- MAID
#### Delete behavior
The only supported sync mode is `all` mode, meaning that each sync run will refresh
the entire audience. Users removed from your audience will automatically be dropped
with each refresh - only present users will be in Digital Turbine's audience.
## Tips and troubleshooting
### Refresh rate
Digital Turbine has a max refresh rate of daily. You should not have your Digital
Turbine syncs running more than daily.
### Matched device count
Match rates are not available to preview in the Hightouch platform. Contact your
Digital Turbine rep for insights into match rates and audience reach.
Keep in mind that audience counts reflect the number of devices match, not the
number of users matched. Also keep in mind that matching occurs at the household
level, and that a single household may own multiple devices.
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Live debugger
### Sync alerts
---
## Discord
**URL:** https://hightouch.com/docs/destinations/discord
**Description:** Send events so Discord can attribute them to ad campaigns or track player progress on in-game quests.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
-----------|---------------------------------------------|---------------------
**Events** | Sync data from any source to Discord events | Insert
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Discord
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Discord** and click **Continue**. You can then authenticate Hightouch to **Discord**.
Enter the following fields into Hightouch:
- **Bot token**: The bot token for your Discord Event Source Application. This is provided by Discord during onboarding.
## Sync configuration
Once you've set up your Discord destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Discord destination you want to sync to.
### Syncing events
Sync data from any source to Discord events.
#### Record matching
You can match rows from your model to events in Discord on any column in your model and any user field in Discord.
At leat one user identifier must be mapped. Ensure the data types of the model column and Discord field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Google Display & Video 360
**URL:** https://hightouch.com/docs/destinations/display-video
**Description:** Sync customer segments to Display & Video 360
**Section:** Destinations
## Overview
With the Google Display & Video 360 integration, Hightouch can sync first and third party audiences directly from your warehouse.
This allows you to:
1. Create ad target groups for your own users, using email, physical address or mobile device ID.
2. Create target groups from users shared with a partner.
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| ----------------------------------- | ------------------------------------------------------------------------------ | -------------------- | -------------------------------------------------------------------------------------- |
| [**Audiences**](#syncing-audiences) | Create and update first and third party audiences as customer match user lists | Add | [Audiences](https://developers.google.com/display-video/api/guides/audiences/overview) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Display & Video 360
Select the **Google Display & Video 360** destination option. Hightouch will prompt you to authorize the app with your Google account with the required scope to create and update audiences on your DV360 accounts.
After authorizing, you will have successfully connected Hightouch to DV360.
## Sync configuration
Hightouch allows you to create and update first and third party audiences as customer match user lists. These lists support adding members using the mobile device ID, or an array of email, phone numbers, and physical addresses.
When creating the audience, make sure the source of the data is properly selected between first or third party sources.
### Syncing audiences
#### User consent
Starting in March 2024, Display & Video 360 introduced the [consent object](https://developers.google.com/display-video/api/guides/audiences/upload-customer-match?sjid=10202302840984618943-NA#set_user_consent) for uploading consent with customer match data. The consent object specifies two distinct types of consent and users must grant both types of consent to use their data for customer match lists. Reference the [DV360 docs](https://support.google.com/displayvideo/answer/14673239?hl=en) for more information.
If you don't set the consent status or if the status is denied, the sync
results in an error.
#### PII data and hashing
Hightouch will automatically hash all the PII data before it's sent to Google. If the data is already
hashed, this option can be disabled. Reference the [DV360 docs](https://developers.google.com/display-video/api/reference/rest/v1/firstAndThirdPartyAudiences#ContactInfo) for more information.
Google's Personalized Ads Policy, Customer Match Policy and Platforms program
policies include requirements for advertisers to show sufficiently established
Google Ads or Display & Video 360 account history before accessing certain
features. To enable uploading data to DV360, please fill [this
form](https://support.google.com/displayvideo/answer/9176010) before you
create the sync.
#### User identifiers
Mobile device IDs and other user identifiers can not be mixed. Also, if providing the user's physical address, the zip code,
country code, first and last name must be provided.
Phone numbers must be in [E.164](https://en.wikipedia.org/wiki/E.164) format.
## Validate your setup
After you configure your sync, run a test to confirm data flows correctly:
1. Start with a small test audience or a single record.
2. Trigger a manual sync from the [Syncs overview](https://app.hightouch.com/syncs?from=docs) page.
3. After the sync completes, verify the audience in DV360:
- Go to **Audiences** in the DV360 UI and search for the audience you configured in Hightouch.
- Google may take up to 48 hours to fully process audience data and reflect the audience size.
4. If the sync shows errors, check the sync run log in Hightouch for specific API error messages.
## Tips and troubleshooting
### Common errors
#### Can not use an UNKNOWN or UNSPECIFIED as enum value
Make sure to set the [consent status](#user-consent) because Google rejects any users where the consent status is `Unspecified` or `Denied`.
#### Advertiser ID not appearing in sync configuration
If you connected DV360 successfully but the advertiser ID doesn't appear in the sync configuration dropdown, the Google account used to authenticate may not have access to the specific DV360 advertiser. Verify that the authorized user has at least advertiser-level access for the advertiser in the DV360 partner settings. If you manage multiple partners or advertisers, confirm you selected the correct partner during the connection setup.
### Live debugger
### Sync alerts
---
## Drift
**URL:** https://hightouch.com/docs/destinations/drift
**Description:** Engage with your leads at the right time with the right context via Drift
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-------------|---------------------------------------------|------------------------|--------------------------------------------------------------
**Accounts** | Sync data from any source to Drift accounts | Upsert, Update, Insert | [Accounts docs](https://devdocs.drift.com/docs/account-model)
**Contacts** | Sync data from any source to Drift contacts | Upsert, Update, Insert | [Contacts docs](https://devdocs.drift.com/docs/contact-model)
**Users** | Sync data from any source to Drift users | Update | [Users docs](https://devdocs.drift.com/docs/user-model)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Drift
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Drift** and click **Continue**. You can then authenticate Hightouch to **Drift** using OAuth.
For the **Authentication method**, select **Log in to Drift** and log into your Drift account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Drift destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Drift destination you want to sync to.
### Syncing accounts
Accounts can be used for ABM (account based marketing) in Drift, and can be tied to the display conditions of particular playbooks-enabling account specific marketing strategies.
#### Record matching
To match rows from your model to accounts in Drift, you need to select a model column and corresponding Drift field. You can match on any of the following Drift fields:
- **Domain**
- **Account ID**
In **Insert** mode, Drift automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync account fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom account fields.
If you send data for a custom field that doesn't exist, Hightouch adds the field.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available on **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
Behavior | Description
---------------|-----------------------------------------------------------
**Do nothing** | Keep the account in Drift with all its synced fields
**Clear** | Clear all the mapped fields, but keep the account in Drift
**Delete** | Delete the synced accounts from Drift
### Syncing contacts
Contacts are your site visitors who interact with Drift. Contacts may provide you with their email address, or their information may have been updated in the past 90 days. Contacts belong to an account, and may have conversations, composed of messages, with your users.
#### Record matching
You can match rows from your model to contacts in Drift on any column in your model and any field in Drift. Ensure the data types of the model column and Drift field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Drift automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom contact fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available on **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
Behavior | Description
---------------|-----------------------------------------------------------
**Do nothing** | Keep the contact in Drift with all its synced fields
**Clear** | Clear all the mapped fields, but keep the contact in Drift
**Delete** | Delete the synced contacts from Drift
### Syncing users
Drift users are internal agents to your organization or account-while site visitors or contacts are those that are interacting from your website.
Hightouch syncs can only update existing users.
#### Record matching
You can match rows from your model to users in Drift on any column in your model and any field in Drift. Ensure the data types of the model column and Drift field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync user fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default user fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|--------------------------------------------------------
**Do nothing** | Keep the user in Drift with all its synced fields
**Clear** | Clear all the mapped fields, but keep the user in Drift
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Dropbox
**URL:** https://hightouch.com/docs/destinations/dropbox
**Description:** Easy to use, reliable, private, and secure. It’s no wonder Dropbox is the choice for storing and sharing your most important files.
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
| ------------ | ------------------------------------------------------- | -------------------- |
| Any data set | Sync data from a source to Dropbox as JSON or CSV files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Diff mode creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Dropbox
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Dropbox** and click **Continue**. You can then authenticate Hightouch to Dropbox via OAuth.
After logging into your Dropbox account, Hightouch prompts you to grant access to your Dropbox account:
After authorizing, you will have successfully connected Hightouch to Dropbox.
## Sync configuration
Once you've set up your Dropbox destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Dropbox destination you want to sync to.
### Select file format
Hightouch supports syncing JSON and CSV files to Dropbox.
### Select folder
Next, select the folder in Dropbox to which you want to sync your files.
### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it. To view different versions of the same file, you can right-click the file in Dropbox. Click **Activity** and then **Version history**.
If you are using an audience and would like to include the audience name, you will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter `-86400` (`24 hours * 60 minutes * 60 seconds`). If you want the filename's data to be one hour after the sync takes place, you would enter `3600` (`60 minutes * 60 seconds`).
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `company`, `email`, `name`, and `primaryKey` fields. Hightouch exports these fields to new fields in the file and ignores all other columns from your results.
### CSV options
If you're syncing to a CSV file, you have additional configuration options:
- **Delimiter**: Your options are comma (`,`), semicolon (`;`), pipe (`|`), tilde (`~`), and tab
- (Optional) Whether to **include a CSV header row** in the exported files
- (Optional) Whether to **include a [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark)** in the exported files, the BOM is ``
### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Dynamic Yield
**URL:** https://hightouch.com/docs/destinations/dynamic-yield
**Description:** Use Dynamic Yield to update product data.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ------------------ | ------------------------------------------------------------ | -------------------- | ----------------------------------------------------------------- |
| **Product Feed** | Sync data from any source to product feed in Dynamic Yield | Upsert | [Product Feed docs](https://dy.dev/reference/update-product-feed) |
| **User Data Feed** | Sync data from any source to user data feed in Dynamic Yield | Upsert | [User Data Feed docs](https://dy.dev/reference/user-data-api) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Dynamic Yield
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Dynamic Yield** and click **Continue**.
You’ll need to provide a Dynamic Yield API key.
### Create an API key in Dynamic Yield
1. In your Dynamic Yield account, navigate to **Settings › API Keys** and click **New Key**.
2. Fill out the form:
- **Name**: Enter a descriptive name, such as `Hightouch Sync Key`.
- **Notes**: (Optional) Add details for your team.
- **Source**: Select `Server-side` (required for Hightouch).
3. Under **ACL (Access Control List)**, check all the following boxes:
- [**Experience API**](https://dy.dev/docs/experience-api-basics) – Grants Hightouch the ability to update the product catalog and other assets Dynamic Yield uses to run campaigns. This is what lets marketers surface the right products or offers in recommendation modules, banners, and emails.
- [**User Data Profile**](https://dy.dev/docs/intro-to-user-data-enrichment) – Allows Hightouch to sync customer attributes (like loyalty tier, geography, or purchase history). Marketers use this profile data to build audiences and target campaigns.
- [**User Data Events**](https://dy.dev/docs/intro-to-user-data-enrichment) – Enables Hightouch to send customer activity signals (like product views, sign-ups, or cart actions) in real time. Marketers use these events for triggering in-session personalization and measuring engagement.
4. Click **Save** and copy the generated API key.
### Connect the API key in Hightouch
5. In Hightouch, paste the API key in the **Connect destination** step.
6. Select your data center (**United States** or **Europe**). See the [Experience API Basics guide](https://dy.dev/docs/experience-api-basics#data-centers) for details on region-specific endpoints.
## Sync configuration
Hightouch supports syncing to the following Dynamic Yield objects:
- [`Product Feed`](#product-feed)
- [`User Data Feed`](#user-data-feed)
To sync to Product Feed or User Data Feed, your Dynamic Yield API key must
have **all three ACL permissions** enabled (`Experience API`, `User Data
Profile`, and `User Data Events`).
### Sync modes
Hightouch supports the following sync modes:
- **Upsert** - pushes new products to Dynamic Yield Product feed or User Data API and updates fields that change in your warehouse.
### Product Feed
#### Feed ID
You can find the feed's ID by navigating to Assets > Data Feeds and looking for the ID column in the table of available feeds.
#### Record matching
Dynamic Yield uses product `SKU` as the product ID for matching records in the feed.
#### Field mapping
There are several required properties when syncing products to the feed. You can add additional fields in the custom mapping section.
#### Deleting
You may also opt to remove products from your feed when they are removed from your source.
You can do that by selecting `Delete the Dynamic Yield record associated with source`
### User Data Feed
You can find the feed's ID by navigating to Assets > Data Feeds and looking for the Unique ID column in the table of available feeds.
#### Record matching
Select the ID for the record from the source. You will also need to enter or select the type of ID used, this would be defined in Dynamic Yield when you setup the feed, i.e., hashed email.
#### Field mapping
To map fields for the user object, copy the feed schema from the Dynamic Yield admin into Hightouch. The fields will automatically be generated for mapping.
#### Deleting
You may also opt to remove products from your feed when they are removed from your source.
You can do that by selecting `Delete the Dynamic Yield record associated with source`
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## DynamoDB
**URL:** https://hightouch.com/docs/destinations/dynamodb
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Overview
This destination combines the analytical power of your data warehouse to power your high-performance applications at any scale.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| --------- | ------------------------------------------------------------------------------------------------------------------------------------------ | -------------------- |
| **Items** | Sync rows from any source as [items](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/WorkingWithItems.html) into DynamoDB | Upsert |
## Getting started
To connect to your DynamoDB instance, you need to provide Hightouch access to your AWS account. You can do this via access key or cross-account role.
For either method, you must give permission for the following actions:
```
"dynamodb:BatchGetItem",
"dynamodb:BatchWriteItem",
"dynamodb:PutItem",
"dynamodb:DescribeTable",
"dynamodb:DeleteItem",
"dynamodb:UpdateItem",
```
### Authenticating with a cross-account role
Hightouch recommends using a cross-account role for cleaner access. See our [AWS security page](/security/aws#cross-account-role) to learn more about how to use the cross-account role.
The cross-account role must also allow `dynamodb:ListTables` on `Resource: "*"`. Hightouch uses this permission to discover available tables during destination setup and connection testing. Without it, destination configuration in the UI fails with an access denied error.
### Authenticating with an access key
You can give Hightouch access to your AWS account with an access key. Copy your Access key ID and Secret access key into Hightouch. See the [AWS docs](https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html) for best practices.
## Syncing data
Once you've authenticated DynamoDB in Hightouch, you need to set up your sync configuration.
### Table selection
In the sync configuration page, select a DynamoDB table to sync to. If your table doesn't exist yet, create it in AWS.
### Record matching
match rows in your model with records in DynamoDB via DynamoDB's primary key. See AWS's guide on [how to choose a primary key](https://aws.amazon.com/blogs/database/choosing-the-right-dynamodb-partition-key/) to learn more.
### Field mapping
Hightouch infers field types from the fields values of rows.
Hightouch sends the `timestamp` field to DynamoDB as an ISO string. If you
want to format it differently, format the value in your model and set the type
to `String`.
#### Delete behavior
When rows are deleted from your source, you can `do nothing`, `clear`, or `delete` them in DynamoDB.
| Type | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the synced record in your DynamoDB |
| **Clear** | Remove the synced fields of the synced record from your DynamoDB |
| **Delete** | Delete the synced record from your DynamoDB |
Deletions are destructive and irreversible. Be sure this is the behavior you
want when selecting `clear` or `delete`.
### Testing a sync
Before running your sync, Hightouch recommends that you test your sync configuration by syncing a single row. You can verify that a row was successfully synced by going to your DynamoDB instance table and finding the synced row by its primary key.
## Tips and troubleshooting
### Rate limits
DynamoDB has quotas that may limit your syncs. If that happens, Hightouch [retries failed rows](https://hightouch.com/docs/syncs/retries). For more information, refer to DynamoDB's [throughput quotas](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ServiceQuotas.html#default-limits-throughput-capacity-modes).
### Empty field mappings
DynamoDB is a key/value NoSQL database. That means the database doesn't store metadata on its columns and column names and you won't see any suggested fields during field mapping in Hightouch.
### Common errors
### Using the Hightouch debugger
### Setting up sync alerts
Hightouch can alert you of sync issues via Slack, PagerDuty, SMS, or email. For details, please visit our [article on alerting](/syncs/alerting).
### Sync alerts
---
## Dynata
**URL:** https://hightouch.com/docs/destinations/dynata
**Description:** Empower your brand, marketing, and advertising teams to make data-driven decisions with Dynata’s data and insight solutions.
**Section:** Destinations
## Overview
Sync your first-party audiences with Dynata data and insights solutions to
better measure, evaluate, and optimize advertising effectiveness in critical
areas of your media to see how a campaign influences consumers at all levels of
the purchase funnel.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------- | ---------------------------------- | -------------------- |
| **Audiences** | Create and update custom audiences | All |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Dynata
This integration requires prior approval from the Dynata team and the Hightouch
team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your Dynata rep who will verify approval. Once we have confirmed, we will
unlock the integration for you to connect.
To connect, input the following provided by your Dynata rep:
- Access key
- Secret key
- Bucket name
### Syncing custom audiences
You can input the name of the audience you plan on delivering to Dynata.
#### User identifiers
To identify which users to add or update in an audience, select model columns
and the corresponding Dynata fields. You can match on any of the following user
fields:
- Unique ID (required for de-duping)
- Email (SHA256)
- Phone (SHA256)
- MAID
- IP address
- First name
- Last name
- Address1
- Address2
- City
- State
- Postal code
- Zip4
- Country
#### Delete behavior
The only supported sync mode is `all` mode, meaning that each sync run will refresh
the entire audience. Users removed from your audience will automatically be dropped
with each refresh - only present users will be in Dynata's audience.
## Tips and troubleshooting
### Refresh rate
Dynata has a max refresh rate of monthly. You should not have your Dynata syncs
running more than every 4 weeks.
### Matched users count
Match rates are not available to preview in the Hightouch platform. Contact your
Dynata rep for insights into match rates and audience reach.
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Live debugger
### Sync alerts
---
## Eagle Eye
**URL:** https://hightouch.com/docs/destinations/eagle-eye
**Description:** Eagle Eye's AI-powered platform unifies loyalty management and 1-to-1 personalization. Capture first-party data, deliver personalized experiences at scale, and measure results.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| --------------------------- | -------------------------------------------------------------------- | -------------------- | ---------------------------------------------------------------------------------------------------------- |
| [**Campaigns**](#campaigns) | Issue coupons for an Eagle Eye campaign to audience members' wallets | Add, Remove | [Create coupon →](https://developer.eagleeye.com/eagleeye-developer/reference/createwalletcampaignaccount) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Eagle Eye
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Eagle Eye** and click **Continue**. You can then authenticate Hightouch to **Eagle Eye**.
Enter the following fields into Hightouch:
- **Client ID**: your Eagle Eye API client ID.
- **Client secret**: your Eagle Eye API client secret. Hightouch uses it to sign every API request with Eagle Eye's SHA-256 hash authentication.
- **Region**: the region your Eagle Eye account is hosted in: `us`, `us2`, `uk`, `au`, or `sandbox.uk`. Hightouch uses the region to build the API URL, for example `https://wallet.us.eagleeye.com`.
To get your client ID and client secret, contact your Eagle Eye account team.
## Sync configuration
Once you've set up your Eagle Eye destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Eagle Eye destination you want to sync to.
### Campaigns
Syncing to Eagle Eye issues coupons for a campaign you select. When a member enters the audience, Hightouch creates a coupon account for the campaign in the member's Eagle Eye wallet. When a member leaves the audience, Hightouch can optionally cancel the coupons that the member hasn't activated yet.
Select the Eagle Eye campaign to issue coupons for. The dropdown lists campaigns from your Eagle Eye account that are in `ACTIVE`, `PENDING`, `DRAFT`, or `READY` status. You can't create campaigns from Hightouch, so create the campaign in Eagle Eye first.
Each sync issues coupons for one campaign. To give the same audience multiple offers, create one sync per campaign from the same audience.
Eagle Eye coupons have a state that controls whether they apply at checkout:
`UNLOADED` coupons are issued but not yet activated by the member, and
`LOADED` coupons are activated. The state a coupon starts in is configured on
the campaign in Eagle Eye. Hightouch doesn't set or change coupon states.
#### Configure record matching
To match rows from your model to wallets in Eagle Eye, select the model column that contains the member's Eagle Eye **Wallet ID**. Eagle Eye coupons are issued directly to wallets, so every row must include a wallet ID. Hightouch rejects rows with a missing or empty wallet ID.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Configure delete behavior
When a member leaves the audience, you can choose how Hightouch handles their coupons:
| Behavior | Description |
| --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| **Keep all coupons** | Leave all of the member's coupons in place. This is the default. |
| **Cancel unloaded coupons** | Cancel the member's unactivated (`UNLOADED`) coupons for this campaign. Coupons the member has activated (`LOADED`) are never canceled. |
Canceling a coupon is irreversible. If a member re-enters the audience later,
Hightouch issues a new coupon.
## Tips and troubleshooting
For Hightouch platform error codes related to Eagle Eye, see [Error codes:
Other destinations](/developer-tools/error-codes#other-destinations).
### Common errors
When Eagle Eye rejects a coupon issuance, Hightouch rejects the row and includes the error code and message returned by Eagle Eye. Refer to [Eagle Eye's API documentation](https://developer.eagleeye.com/eagleeye-developer/reference/createwalletcampaignaccount) for details on specific error codes.
### Live debugger
### Sync alerts
---
## Eesii
**URL:** https://hightouch.com/docs/destinations/eesii
**Description:** Activate your data as campaign records in Eesii
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes |
----------------|------------------------------------|----------------------|
**Campaign records** | Sync data as campaign records to Eesii | Insert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Eesii
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Eesii** and click **Continue**.
You can then authenticate Hightouch to **Eesii** by entering your API key.
To find your API key, navigate to the detail page of one of your campaigns. From there, click "Generate Webhook" and you will see your API key displayed.
## Sync configuration
Once you've set up your Eesii destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Eesii destination you want to sync to.
### Syncing campaign records
#### Campaign and layout selection
Begin setting up your sync by selecting which campaign you'd like to sync records to.
Select which layout you'd like to use to sync records. If you selected a **variant** campaign, you will have multiple layouts to choose from. If you selected a **regular** campaign, you will only have one.
#### Field mapping
Once you have selected a layout, you will see the attributes of your layout as possible mapping options.
Select which columns in your model you'd like to map to record fields in Eesii.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Elasticsearch
**URL:** https://hightouch.com/docs/destinations/elasticsearch
**Description:** Elasticsearch is a distributed, free and open search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured.
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes | API Reference|
|--|--|--|--|
|**Index documents**|Sync data from any source to an Elasticsearch index|Upsert, Update|[Index API](https://www.elastic.co/guide/en/elasticsearch/reference/8.7/docs-index_.html)|
## Connect to Elasticsearch
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Elasticsearch** and click **Continue**. You can then authenticate Hightouch to Elasticsearch either with an Elasticsearch **Cloud ID** and **API key** or via optional [basic authentication](https://www.elastic.co/guide/en/elasticsearch/reference/current/http-clients.html).
### Authenticate with an Elasticsearch Cloud ID and API key
You can find your [Cloud ID](https://www.elastic.co/guide/en/cloud/current/ec-cloud-id.html) in your Elasticsearch deployment. To generate an API key, follow these steps in the [Elasticsearch docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html).
### Authenticate with basic authentication
Enter the following fields into Hightouch:
- **Protocol**: Select the protocol used for connecting to Elasticsearch. This is either `HTTP` or `HTTPS`.
- **Host**: The hostname or IP address of your Elasticsearch server.
- **Port**: The port number of your Elasticsearch server's HTTP API. The default is 9200, but yours may be different.
- **Username**: This can be your personal Elasticsearch login or a dedicated user for Hightouch.
- **Password**: The password for the user specified above.
## Sync configuration
Once you've set up your Elasticsearch destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Elasticsearch destination you want to sync to.
### Syncing to an index
An Elasticsearch [index](https://www.elastic.co/blog/what-is-an-elasticsearch-index) is the place where the data used by a search engine is stored. You can use Hightouch to update individual [documents](https://www.elastic.co/guide/en/elasticsearch/reference/current/documents-indices.html#:~:text=Elasticsearch%20is%20a%20distributed%20document,been%20serialized%20as%20JSON%20documents.) in an index.
#### Record matching
To match rows from your model to documents in Elasticsearch, you need to select a model column and corresponding Elasticsearch field. You can match on any Elasticsearch field.
#### Field mapping
You can sync data to both existing fields and fields that don't yet exist in Elasticsearch. If you send data to a field that doesn't exist yet, Hightouch adds the field and automatically detects its type.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ------------------------------------------------------------------ |
| **Do nothing** | Keep the document in Elasticsearch |
| **Clear** | Keep the document in Elasticsearch, but set mapped fields to empty |
| **Delete** | Remove the document from Elasticsearch |
## Tips and troubleshooting
For Hightouch platform error codes related to Elasticsearch, see [Error codes: Elasticsearch](/developer-tools/error-codes#other-destinations).
**Connect to Elasticsearch on AWS**
To connect to an AWS hosted Elasticsearch, connect to Elastic Cloud where the instance has a Cloud ID, then generate an API key. If you can't connect via this method, use the basic authentication method to connect with username and password.
There may be compatibility issues across different versions of Elasticsearch. If you receive an error when you test the connection, please for support for your specific version.
If you are running Elasticsearch via Amazon OpenSearch Service, connect directly via the [OpenSearch destination](/destinations/opensearch)
### Common errors
### Live debugger
### Sync alerts
---
## Eloqua
**URL:** https://hightouch.com/docs/destinations/eloqua
**Description:** Enhance Eloqua marketing automations for mobile, email, video and search results by bringing enriched customer data into Eloqua from your data warehouse.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- |
| **Contacts** | Sync data from any source to [Eloqua contacts](https://docs.oracle.com/en/cloud/saas/marketing/eloqua-user/Help/Contacts/Contacts.htm) | Upsert, Update |
| **Accounts** | Sync data from any source to [Eloqua accounts](https://docs.oracle.com/en/cloud/saas/marketing/eloqua-user/Help/Accounts/Accounts.htm) | Upsert, Update |
| **Custom Objects** | Sync data from any source to any [Eloqua custom object](https://docs.oracle.com/en/cloud/saas/marketing/eloqua-user/Help/CustomObjects/CustomObjects.htm) | Upsert, Update |
| **Contact Lists** | Sync data from any source to [Eloqua contact lists](https://docs.oracle.com/en/cloud/saas/marketing/eloqua-user/Help/SharedLists/SharedContactLists.htm) | Add |
| **Email Trigger** | Sync data from any source to [trigger](https://docs.oracle.com/en/cloud/saas/marketing/eloqua-develop/Developers/RESTAPI/2.0/email-deployment/email-deployment-API.htm) emails in Eloqua | -- |
## Connect to Eloqua
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Eloqua**.
Then **Authorize connection** by clicking **Log in to Eloqua**. Clicking the button leads you through your Eloqua login and asks for Hightouch to have authorization to your Eloqua instance.
Once you've finished, Hightouch displays an **Authorization successful** message.
To finish connecting, give your Eloqua destination a descriptive name.
## Sync configuration
Once you've connected your Eloqua destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Eloqua destination you want to sync to.
### Contacts
Hightouch supports creating new contacts and updating existing ones.
#### Record matching
You can match rows in your model with contacts in Eloqua on any contact field. We recommend matching on a unique field, such as **Email address**.
#### Field mapping
You can sync any columns from your source to Eloqua contact fields.
### Accounts
Hightouch supports creating new accounts and updating existing ones.
#### Record matching
You can match rows in your model with accounts in Eloqua on any account field. We recommend matching on a unique field, such as **Eloqua Company ID** or a unique custom field.
#### Field mapping
You can sync any columns from your source to Eloqua account fields.
### Custom objects
Hightouch supports creating new instances of custom objects and updating existing ones.
Custom objects and their fields must already be defined on your Eloqua UI
before Hightouch can sync to them.
#### Record matching
You can match rows in your model with custom objects in Eloqua on any custom object field.
#### Field mapping
You can sync any columns from your source to Eloqua any existing field in your custom object.
### Contact lists
Hightouch supports syncing data to Eloqua contact lists.
You can create a new contact list or use an existing one. When creating a new list, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model.
#### Record matching
You can match rows in your model with contacts in Eloqua on any contact field. We recommend matching on a unique field, such as **Email address**.
#### Field mapping
You can sync any columns from your source to Eloqua contact fields.
### Email trigger
Hightouch supports triggering emails to contacts in Eloqua.
#### Email
For a given trigger sync, you need to select an email asset to sync to. The options come from the email assets in your Eloqua environment.
Optionally, you can also enter a specific user ID to specify the sender of each email. By default, the email will be sent by the user
the destination is connected to.
#### Contact matching
You can match rows in your model with contacts in Eloqua on by email or Eloqua's contact ID. We recommend matching on the **contact ID** because matching on email
will require a lookup for each record since Eloqua only allows triggering emails by contact IDs. Performing the lookups will impact the performance of your sync.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## SMTP Email
**URL:** https://hightouch.com/docs/destinations/email
**Description:** Sync your warehouse with Email notifications and push marketing campaigns and product updates to customers in real time
**Section:** Destinations
## Overview
With the SMTP Email destination, Hightouch can serve as a notifier for various changes. This is a very flexible integration, but here are a few example use cases:
1. Notify your customer success teams when product usage for a key account suddenly drops
2. Notify an account executive when product usage surpasses a certain threshold
3. Send a daily summary of the number of sign ups, compared to the same day last month
4. Send a daily summary on the adoption of a core feature
## Sending emails \(added / changed / removed\)
Hightouch will not send any emails on the initial run to prevent a flood of
emails to your channel. Emails will start after the this initial sync.
Hightouch supports sending emails based on rows that have been added, changed, or removed since the last sync run. Each row is sent as an individual email and can be sent to different channels and templated with Liquid.
Email formatting is best when the following conditions are met:
1. You want to write a query, and have Hightouch detect the added and removed rows
2. You want a custom email sent per each added and removed rows
### Recipient selection
Hightouch supports both sending emails to specific static recipients, or an email each row. Enable the "Use column" toggle to dynamically set the target email addresses.
Hightouch also supports CC and BCC so that multiple recipients can be included on the email chain.
The recipient, CC, and BCC sections all support a single email or a comma-separated list of emails
### Added, changed, and removed emails
Hightouch allows you to template the content you would like to send in your email.
- When a row is added in your query results, Hightouch will send the row added content.
- When a row is changed in your query results, Hightouch will send the row changed content.
- When a row is deleted in your query results, Hightouch will send the row deleted content.
This content can be [templated](#liquid-templating).
## Sending emails \(all results\)
The batch message mode is similar to the normal message mode, except it sends all rows in the query results rather than only added, changed, and removed rows since the last sync.
This mode also allows you to specify how many rows are sent per email, which can range from 1 to 20. With 1 row per email, it would be similar to the message mode, whereas 20 rows per email would be similar to a tabular representation.
This format is good for the following scenarios.
1. You want to write a query, and have Hightouch send an email containing **all** rows
2. You don't want Hightouch to detect added, changed, and removed rows, but instead want to send all rows on each query run
### Recipient selection
The batch message only supports sending your emails to a static email or list of emails.
Hightouch also supports CC and BCC so that multiple recipients can be included on the email chain.
The recipient, CC, and BCC sections all support a single email or a comma-separated list of emails
### Subject and body content
For each email, Hightouch allows setting the subject and row content. Only the row content is required. The row content is repeated per row in the email and has access to the row values via templating.
This content can be [templated](#liquid-templating).
## Templating emails
### Liquid templating
For both the message and batch message modes, [Liquid templating ](https://shopify.github.io/liquid/)is supported. The most important implication of this is that columns from the added or removed row can be accessed as part of the message body. Here is an example message making use of Liquid to insert a column into the message.
```text
User with name {{ row["name"] }} and email {{ row["email"] }} has signed up
```
## Table
Table formatting sends all of your query results formatted as a text table. This mode is recommended only for small sets of data.
This format is good for the following scenarios.
1. You want to write a query, and have Hightouch send an email containing **all** rows, formatted as a table
2. You don't want Hightouch to detect added and removed rows, but instead want to send all rows on each query run
### Recipient selection
The batch message only supports sending your emails to a static email or list of emails.
Hightouch also supports CC and BCC so that multiple recipients can be included on the email chain.
The recipient, CC, and BCC sections all support a single email or a comma-separated list of emails
## CSV
The CSV option allows you to send a CSV file or a zipped CSV file of all records in the query results for each sync attached to a single email. You can customize the subject and body of the email, as well as the name of the CSV file attachment.
This format is useful for reporting purposes, such as sending weekly or monthly reports.
If sending a zipped CSV file, make sure that your email service has the appropriate permissions to send .zip files to recipients. This may require unblocking IP addresses that your service uses.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Emarsys
**URL:** https://hightouch.com/docs/destinations/emarsys
**Description:** Emarsys platform helps marketers deliver personalized, 1:1 experiences across multiple channels and drive business outcomes.
**Section:** Destinations
## Setup
You will need your username, usually in the format of `account_name00X` where `X` is a digit, and the API secret.
To get the API secret, you can create a new API user for Hightouch under **Admin** > **Security Settings**.
Make sure to copy and save the secret key here since the entire key will no
longer display after you close the confirmation dialog box.
You will need to enable the necessary endpoints for this API key. The endpoints
Hightouch requires depend on how you configure your syncs.
#### Always required
- [`field.get`](https://dev.emarsys.com/docs/core-api-reference/a0l7f9tviiuiv-list-available-fields) — Used during the connection test and to load available contact fields for mapping. Without this endpoint enabled, the connection test will fail with a 403 error.
#### Contact syncs
- [`contact.create`](https://dev.emarsys.com/docs/core-api-reference/g617t4kfs4y69-create-contact) — Required for upsert sync mode (creates new contacts).
- [`contact.update`](https://dev.emarsys.com/docs/core-api-reference/f8ljhut3ac2i1-update-contacts) — Required for upsert and update sync modes.
- [`contact.delete`](https://dev.emarsys.com/docs/core-api-reference/szmq945esac90-delete-contacts) — Only required if you plan to remove contacts when data leave your warehouse.
#### Contact list syncs
- [`contactlist.list`](https://dev.emarsys.com/docs/core-api-reference/paz2wputeebqx-list-contact-lists) — Required to load available contact lists for selection.
- [`contactlist.contact.add`](https://dev.emarsys.com/docs/core-api-reference/1m0m70hy3tuov-add-contacts-to-a-contact-list) — Required to add contacts to a list.
- [`contactlist.contact.delete`](https://dev.emarsys.com/docs/emarsys-api/b3A6MjQ4OTk4MzI-remove-contacts-from-a-contact-list) — Only required if you plan to remove contacts from a list.
- [`contactlist.create`](https://dev.emarsys.com/docs/emarsys-api/b3A6MjQ4OTk4MjI-create-a-contact-list) — Only required if you plan to create new contact lists from Hightouch.
The following screenshot shows an example of a working permissions configuration in the Emarsys UI.
It is also safe to enable all **contacts**, **fields**, and **contact lists** endpoints.
API users and permissions are described in the [Emarsys docs](https://dev.emarsys.com/docs/core-api-reference/ef41493bd7812-endpoint-permission-settings).
## Syncing
Hightouch supports syncing **contacts** and **contact lists**.
### Sync modes
Here are the possible modes for the Emarsys contacts:
- **Upsert**: pushes new contacts to Emarsys and updates existing contacts with the fields that change in your warehouse.
- **Update**: updates particular fields on existing contacts in Emarsys. It doesn't add new objects.
This integration also supports **Segment** mode. New users will be inserted into the list.
Users that are removed from the model will be removed from the list.
### Record matching
Records can be matched from your source to Emarsys by _any_ default and custom fields. However, it's important to make sure
that the field you are using is unique to ensure the best results. For example, Emarsys will throw an error if it detects duplicate
values in an attempt to update or remove a batch of contacts.
We recommend using `email` or creating a custom unique field, like an `external_id` (make sure to set it as _unique_).
### Fields mapping
You can also map to any updatable fields in Emarsys, including your custom fields.
However, there are some things to keep in mind:
- The Emarsys API uses the field's ID when making requests so if you need to use the JSON editor, you will not see any named fields.
Instead, you will only see the field ids. You can use this [table](https://dev.emarsys.com/docs/emarsys-api/ZG9jOjI0ODk5NzY3-contact-system-fields) to cross reference the field ids.
You can also view your settings to reference your custom fields.
- Any fields with a date type are formatted: _YYYY-MM-DD_. You can have your date values be a string in this format (for example, "2020-11-30").
- You must use the Emarsys accepted values for any choice fields (that is, `gender`, `title`, etc.).
## Debugging
Similarly to the fields mapping in the sync configuration, the request will only show the fields' ids unless it's Emarsys' internal `id` and `uid`.
Please refer to this [table](https://dev.emarsys.com/docs/emarsys-api/ZG9jOjI0ODk5NzY3-contact-system-fields) and your custom fields settings to cross reference
the ID to the field's name.
The Emarsys API lets us upsert/update/remove contacts in batches per API call. Most cases, the API will let us know which specific rows fail, instead of failing
the entire batch. However, the error message may not be very descriptive but it will reference their [internal error code](https://dev.emarsys.com/docs/emarsys-api/ZG9jOjI0ODk5NzY4-http-200-errors).
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## EnjoyHQ
**URL:** https://hightouch.com/docs/destinations/enjoy-hq
**Description:** Sync data from your warehouse to EnjoyHQ to consolidate all customer feedback and research data into a single platform, providing you with better insights for informed decision-making and enhancing customer experiences
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
--------------|---------------------------------------------------|----------------------|---------------------------------------------------------------------------
**Documents** | Sync data from any source to the Documents object | Upsert | [Documents docs](https://developers.enjoyhq.com/reference/document-import)
**Users** | Sync data from any source to the Users object | Upsert | [Users docs](https://developers.enjoyhq.com/reference/user-import)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to EnjoyHQ
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **EnjoyHQ** and click **Continue**. You can then authenticate Hightouch to **EnjoyHQ** by entering an EnjoyHQ **API key**.
To retrieve an API key in EnjoyHQ, go to [Manage integrations](https://app.enjoyhq.com/account/integrations) and select the API card in the **Other** section.
You can then use an existing API key or generate a new one by clicking on the **Add New API Key** button.
## Sync configuration
Once you've set up your EnjoyHQ destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the EnjoyHQ destination you want to sync to.
### Syncing documents
Create and update documents.
#### Record matching
To match rows from your model to documents in EnjoyHQ, you need to select the model column that contains values that match the [**Foreign ID** field](https://developers.enjoyhq.com/reference/document-import#updating-documents).
#### Field mapping
You can map data from any of your model columns to fields in EnjoyHQ. EnjoyHQ requires the following fields, so you must map them in order to complete your configuration:
- **Content**
- **Source**
Ensure the data types of your model columns match the data types of the fields you want to sync to.
### Syncing users
Create and update users.
#### Record matching
To match rows from your model to users in EnjoyHQ, you need to select the model column that contains values that match the **Email** field.
#### Field mapping
You can sync columns from your model to EnjoyHQ default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|------------------------------------------------------
**Do nothing** | Keep the record in EnjoyHQ with all its synced fields
**Delete** | Delete the synced record from your EnjoyHQ user
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Expedia Group Advertising
**URL:** https://hightouch.com/docs/destinations/expedia
**Description:** Get a full picture of your Expedia ad performance with attribution reports that track their impact on direct bookings—both online and offline.
**Section:** Destinations
## Expedia Data Clean Rooms
Expedia uses AWS Clean Room technology to build secure, tailored data collaborations that let partners share insights without exposing sensitive information.
## Supported syncing
Type | Description | Supported Sync Modes
--------------------|------------------------------------------------------------------------------------------------------------|---------------------
**Conversion Data** | Sync data from any source to AWS Data Clean Room to use with Expedia Group Media Solutions conversion data | Mirror
Mirror mode means that the current table will be cleared and replaced with all the rows in the new query result, every time the sync runs.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Prerequisites
To get started, you need:
- to contact Expedia Group to set up an AWS Data Clean Room collaboration with them. This will involve sharing an AWS account ID with Expedia Group.
- AWS [credentials configured in Hightouch](/security/aws) with programmatic access enabled and permission to write to the S3 path, use Glue databases and tables, and associate tables to data clean rooms.
- the following AWS resources **(these AWS resources must reside in `us-east-1`, consistent with Expedia's Data Clean Room)**:
- an [S3 bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html) to store your synced data in. **Make sure that this bucket is only used for your data that you are syncing to use with Expedia**.
- an S3 bucket for Data Clean Room analysis results. **This must be a different bucket than the above bucket used for storing synced data**.
- your AWS Data Clean Room collaboration membership ID with Expedia Group. This can be found in your AWS Data Clean Room collaboration with Expedia under Details > Membership Details > Membership ID after creating a collaboration with Expedia.
Ensure that the configured user has the following IAM permission policy:
```yaml
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:*",
"s3-object-lambda:*"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"sts:AssumeRole",
"iam:PassRole"
],
"Resource": "arn:aws:iam::${accountId}:role/${IAMRoleName}"
}
]
}
```
And a separate IAM Role with the following permission policy:
```yaml
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "CloudWatchLogsAccess",
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "*"
},
{
"Sid": "GlueCatalogAccess",
"Effect": "Allow",
"Action": [
"glue:CreateDatabase",
"glue:GetDatabase",
"glue:CreateTable",
"glue:DeleteTable",
"glue:GetTables",
"glue:GetTable"
],
"Resource": [
"arn:aws:glue:us-east-1:${accountId}:catalog",
"arn:aws:glue:us-east-1:${accountId}:database/ht_expedia__glue_database_sync_*",
"arn:aws:glue:us-east-1:${accountId}:table/ht_expedia__glue_database_sync_*/*"
]
},
{
"Sid": "S3AccessForGlue",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetObject",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::{bucketName}",
"arn:aws:s3:::${bucketName}/*"
]
},
{
"Sid": "CleanRoomsAccess",
"Effect": "Allow",
"Action": [
"cleanrooms:ListConfiguredTables",
"cleanrooms:CreateConfiguredTable",
"cleanrooms:GetConfiguredTable",
"cleanrooms:ListConfiguredTableAssociations",
"cleanrooms:CreateConfiguredTableAssociation",
"cleanrooms:GetConfiguredTableAssociation",
"cleanrooms:GetConfiguredTableAnalysisRule",
"cleanrooms:CreateConfiguredTableAnalysisRule",
"cleanrooms:GetConfiguredTableAssociationAnalysisRule",
"cleanrooms:CreateConfiguredTableAssociationAnalysisRule"
],
"Resource": "*"
},
{
"Sid": "AllowPassRole",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "arn:aws:iam::${accountId}:role/${IAMRoleName}"
}
]
}
```
and Trust policy:
```yaml
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowUserAssumeRole",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${accountId}:user/${IAMUserName}"
},
"Action": "sts:AssumeRole"
},
{
"Sid": "AllowGlueAndCleanRoomsService",
"Effect": "Allow",
"Principal": {
"Service": [
"cleanrooms.amazonaws.com",
"glue.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}
```
## Connect to AWS DCR + Expedia Group Media Solutions
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Expedia Group Media Solutions** and click **Continue**. You can then authenticate Hightouch to **Expedia Group Media Solutions**.
Enter the following fields into Hightouch:
- **Bucket Name**
- **AWS Glue and Data Clean Room IAM Role Arn**: For the custom role outlined above, the Role ARN can be found under the custom role's Summary > Arn
- **Data Clean Room Collaboration Membership ID**
## Sync configuration
Once you've set up your Expedia Group Media Solutions destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Expedia Group Media Solutions destination you want to sync to.
### Syncing conversion data
Sync data from any source to Expedia Group Media Solutions conversion data for analysis in AWS Data Clean Room.
Your data will be:
1) Sent to the specified S3 bucket with the path `ht_expedia__dcr/upload.csv`. **Do not put any other files in the folder.**
2) Converted to an S3 table with the name `ht_expedia__glue_table_{sync_id}` inside the S3 database with the name `ht_expedia__glue_database_{sync_id}`. **Do not append any other data to the table**.
3) Associated to the entered Data Clean Room collaboration.
#### Field mapping
Hightouch allows you to sync columns from your source to Expedia's AWS Data Clean Room with this schema:
| Mapping | Type | Required |
| :---------------------------| :-----: | :------: |
| `Event ID` | string | No |
| `Timestamp (ISO8601)` | string | **Yes** |
| `Transaction value (USD)` | string | No |
| `Transaction category` | string | No |
| `Email` | string | No |
| `Email (SHA256 Hex)` | string | No |
| `Phone number` | string | No |
| `Phone number (SHA256 Hex)` | string | No |
Expedia recommends to share both email and phone numbers (either hashed or unhashed) to increase the match rate of the collaboration.
#### Adding your table to the collaboration
Hightouch handles associating a table with your data to the Expedia Data Clean Room collaboration. Additional analysis rules should be added to the Hightouch table that was added to the Expedia Data Clean Room collaboration to use your data with Expedia's.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Firestore
**URL:** https://hightouch.com/docs/destinations/firestore
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Supported syncing
| Supported Sync Modes | Description |
| -------------------- | -------------------------------------------------------------- |
| Upsert | Push new records and update records that change in your source |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Firestore
You can decide whether to set up Firestore with [Firebase](#firebase) or with [Google Cloud Platform](#google-cloud-platform).
The following sections outline the necessary setup for either service.
### Firebase
To find the credentials, navigate to [Firestore](https://console.firebase.google.com) and select your project.
In the left-hand sidebar, click the wheel icon next to **Project Overview** and select **Project settings** from the menu.
Once on the Project setting screen, select the **Service accounts** tab. Make sure you are on the **Firebase Admin SDK**.
You will need to click the **Generate new private key** button to download a JSON file of the credentials.
In Hightouch, paste your **private key** including the header and footer in the following format:
```bash
-----BEGIN PRIVATE KEY-----
-----END PRIVATE KEY-----\n
```
Make sure to include the final `\n` character as well, as that is still part of the private key.
### Google Cloud Platform
To find the credentials, navigate to [Google Cloud Platform](https://console.cloud.google.com/firestore/welcome). In the header, next to Google Cloud Platform, click the **Select a project** dropdown and select a project.
If you do not have Firestore set up within this project, you will be taken to the **Get started** screen. Click the **Select Native Mode** button.
Select a location from the dropdown, and then click the **Create Database** button.
Once you are redirected to the Firestore dashboard screen, click the **Security Rules** in the sidebar. Click on the **Enable Firebase** button, and confirm by clicking the **Enable** link within the modal.
Click on the hamburger menu in the top left corner. Select **APIs & Services** from the sidebar, and then select **Credentials** from the dropdown menu.
Once on the Credentials screen, scroll to the **Service Accounts** table. Click on the edit icon in the row with the **Name** as **App Engine default service account**.
Click on the **Keys** tab, and then click the **Add Key** button.
Select the **Create new key** option. Keep the **Key type** as **JSON**, and click the **Create** button in the modal. This will download a JSON with the needed credentials.
In Hightouch, paste your **private key** including the header and footer in the following format:
```bash
-----BEGIN PRIVATE KEY-----
-----END PRIVATE KEY-----\n
```
Make sure to include the final `\n` character as well, as that is still part of the private key.
{/* */}
### Custom database name
Firestore defaults to the `(default)` database. If your setup requires a custom database name enter it in the **Database Name** field in Hightouch during setup.
## Sync configuration
### Sync modes
Hightouch supports the Upsert sync mode.
During the Upsert mode, new rows will be inserted into Firestore and existing rows will be kept up-to-date within Firestore.
### Record matching
Records can be matched from your source to Firestore by any unique property.
### Columns to sync
For Firestore, we give you the ability to send all columns as they are represented in your model.
You can also choose to manually map fields. Only the fields that you map will be exported. All other columns from your results are ignored.
### Delete mode
The default is to do nothing, but you can also set Hightouch to delete the record when it leaves the query result.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## FreeWheel
**URL:** https://hightouch.com/docs/destinations/free-wheel
**Description:** FreeWheel gives advertisers direct connections to streaming video inventory from the world's leading publishers.
**Section:** Destinations
## Overview
FreeWheel is a Comcast-owned advertising technology platform for CTV/OTT video advertising and monetization. Hightouch's FreeWheel destination allows you to upload audience segment files to FreeWheel's infrastructure via S3.
## Supported syncing
Type | Description | Supported Sync Modes
------------------------|----------------------------------------------------------|---------------------
**S3 file upload** | Upload audience segment files to FreeWheel via S3 | All, Insert, Diff
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to FreeWheel
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **FreeWheel** and click **Continue**.
Enter the following fields into Hightouch:
- **AWS Credentials**: Select or create [AWS credentials](../security/aws)
- **Bucket name**: The FreeWheel bucket name you would like to sync to
## Sync configuration
Once you've set up your FreeWheel destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button. Then, select the relevant model and the FreeWheel destination you want to sync to.
### Sync settings
Configure the following settings for your sync:
- **Sync mode**: Choose from All, Insert, or Diff modes
- **File path**: The S3 object key (file path) where the file will be uploaded, i.e. `aud-sgmt-01.csv`
- **Timestamp offset**: Optional timezone offset for timestamp formatting
### Field mapping
Map columns from your model to the fields you want to include in the uploaded CSV file. Ensure your model's columns have the same data types as the fields you want to sync to.
### File format
FreeWheel syncs automatically use the following CSV settings:
| Setting | Value |
|---------|-------|
| File type | CSV |
| Delimiter | Comma (`,`) |
| Include header | Yes |
| Batching | Enabled |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Freshdesk
**URL:** https://hightouch.com/docs/destinations/freshdesk
**Description:** Deliver better support by making contacts, companies, and tickets available in Freshdesk.
**Section:** Destinations
## Setup
Navigate to your Profile Settings and select the API Key on the right most column.
Copy the API key and fill in the fields in Hightouch destination setup:
- **API Key:** token from Freshdesk
- **Freshdesk Subdomain:** subdomain for your Freshdesk support site, https://hightouch.freshdesk.com
## Syncing
This integration support syncing the following objects:
- **Contacts**
- **Companies**
- **Tickets**
### Sync modes
This integration supports both Update, Upsert, and Insert mode.
In Update mode, users and organizations can be matched based on the Freshdesk ID or the external ID and updated by Hightouch with the latest sync data.
In Upsert mode, users and organizations can be matched based on the Freshdesk ID or the external ID and updated or inserted by Hightouch with the latest sync data.
In Insert mode, any new tickets will be inserted by Hightouch with the latest sync data.
### Field mapping
You can sync columns from your source to Freshdesk's default fields, as well as custom fields.
## Tips and troubleshooting
For Hightouch platform error codes related to Freshdesk, see [Error codes: Freshdesk](/developer-tools/error-codes#other-destinations).
### Common errors
### Live debugger
### Sync alerts
---
## Freshsales
**URL:** https://hightouch.com/docs/destinations/freshsales
**Description:** Transform your selling experience with Freshsales
**Section:** Destinations
## Setup
In order for Hightouch to sync data to Freshsales, you need to give Hightouch an API key and subdomain so that Hightouch can update your records.
You can retrieve your API key by navigating to Settings > API Settings:
For subdomain, check the domain that you are logged in, it should have the pattern of `https://yoursubdomain.myfreshworks.com`. The subdomain used to configure the destination is `yoursubdomain`.
## Syncing
Hightouch supports syncing the following Freshsales objects:
- `Contacts`
- `Accounts`
### Sync modes
- **Upsert** - pushes new objects to Freshsales _and_ updates fields that change in your warehouse.
- **Update**: only updates particular fields on existing objects in Freshsales. It does _not_ add new objects.
### Record matching
Records can be matched from your source to your Freshsales's account's Name or contact's email.
### Field mapping
You can sync columns from your source to Freshsales's Accounts or Contacts properties.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Front
**URL:** https://hightouch.com/docs/destinations/front
**Description:** Empower your teams to deliver better support and experiences by making customer data available in the tools they live in
**Section:** Destinations
## Setup
To get an API Token, in the Front app, navigate to Settings > Plugins & API > API and click the New Token button.
## Modes
### Upsert contacts
In this mode, new contacts will be inserted into Front and existing contacts will be updated.
**User Identification**
To determine whether a contact should be created or updated, you will need to provide a column mapping that can be associated with a unique [Handle](https://dev.frontapp.com/reference/contacts) in Front.
A user can have multiple handles but we will only use one to determine the existence of a contact in Front. Please provide this mapping in this section of the sync configuration:
Please ensure that the column you are mapping to is unique and non-null. Also
keep in mind that making changes to this mapping later on can result in
unexpected behavior.
### Update contacts
In this mode, existing contacts will be updated in Front but new contacts will not be inserted.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## FullStory
**URL:** https://hightouch.com/docs/destinations/fullstory
**Description:** Sync your user properties into FullStory on your schedule, not only when users are interacting with your properties. This allows you to integrate these platforms and gain a more complete view of your customer experience.
**Section:** Destinations
## Overview
You've recently built a great digital experience but need to gather information to improve upon the current iteration. How can you know what works and what doesn't? With Hightouch's FullStory integration, you can sync your most recent user properties data to better analyze who is interacting with your product and how. This allows you to make product improvements driven by meaningful data and optimized to enhance retention rates.
## Supported syncing
| Type | Description | Supported Sync Modes |
| --------- | ----------------------------------------------- | -------------------- |
| **Users** | Sync data from any source to Users in FullStory | Update |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to FullStory
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **FullStory** and click **Continue**. You can then authenticate Hightouch to **FullStory** with your FullStory **API key**.
You can find your API key within your FullStory account by clicking the dropdown button on the top left of the page, and opening **Settings** > **API Keys** under the **Integrations** section.
For more information on FullStory API keys, refer to [FullStory's documentation](https://help.fullstory.com/hc/en-us/articles/360052021773-Managing-API-Keys).
## Sync configuration
Once you've set up your FullStory destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and
the FullStory destination you want to sync to.
### Users
Sync data from any source to Users in FullStory.
#### Record matching
Hightouch matches rows from your model to FullStory users by **User ID**. Select the model column that contains these values.
#### Field mapping
You can sync columns from your source to the following FullStory user fields:
- **Display Name**
- **Email**
#### Custom mapping
You can sync to existing FullStory custom fields or create them via field mapping. When entering new custom fields, ensure the field is named using the pattern `ident_type`,
where `ident` is a sequence of alphanumeric characters `A-Z`, `a-z`, or `0-9` and `type` is a short suffix that denotes the field type. Custom field names must start with an alphabetic character (`A-Z` or `a-z`).
Example acceptable field names includes `nickname_str`, `PasswordResets_int`, and `amtAbandonedInCart_real`.
Refer to the FullStory [docs](https://help.fullstory.com/hc/en-us/articles/4446290296599-Setting-custom-API-properties) for more details.
#### Delete behavior
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the customer record in FullStory with all its synced fields |
| **Clear** | Unset some mapped fields from FullStory |
Hightouch can only clear fields of type `_str`, `_real`, and `_int`. Hightouch sets the
cleared result of `_str` field type to an empty string (`""`) and the
cleared result of `_real` and `_int` to `0`.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Gainsight
**URL:** https://hightouch.com/docs/destinations/gainsight
**Description:** Improve product adoption and reduce churn by enriching Gainsight with rich, up-to-date customer data from your data warehouse
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes | API reference|
|--|--|--|--|
|**Company**|Sync data from any source to Company Objects|Upsert, Update|[Company endpoint](https://support.gainsight.com/Gainsight_NXT/API_and_Developer_Docs/Company_and_Relationship_API/Company_API_Documentation)|
|**Person**|Sync data from any source to Person Objects|Upsert|[Person endpoint](https://support.gainsight.com/Gainsight_NXT/API_and_Developer_Docs/Person_API/Person_API_Documentation)|
|**Custom Objects**|Sync data from any source to Custom Objects|Upsert, Update|[Custom Object endpoint](https://support.gainsight.com/Gainsight_NXT/API_and_Developer_Docs/Custom_Object_API/Gainsight_Custom_Object_API_Documentation)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Gainsight
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Gainsight** and click **Continue**. You can then authenticate Hightouch to Gainsight by entering the following fields into Hightouch:
- **Subdomain**: You can create domains and subdomains by following [these instructions in Gainsight's docs](https://support.gainsight.com/Gainsight_NXT/06Surveys/02Admin_Guides/Set_up_a_Gainsight_Domain).
- **API Key**: You can generate an API key by following [these instructions in Gainsight's docs](https://support.gainsight.com/Gainsight_NXT/Connectors/Connectors/API_Integrations/Generate_API_Access_Key#Generate_Access_Key).
## Sync configuration
Once you've set up your Gainsight destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Gainsight destination you want to sync to.
### Syncing companies
The [Company object](https://support.gainsight.com/Gainsight_NXT/API_and_Developer_Docs/Company_and_Relationship_API/Company_API_Documentation) is one of the core objects in Gainsight and is used to store information about your customers, including individual customer records.
#### Record matching
To match rows from your model to companies in Gainsight, you need to select a model column and corresponding Gainsight field. You can match on any [filterable, unique](https://support.gainsight.com/Gainsight_NXT/API_and_Developer_Docs/Data_Management_APIs/Data_Management_APIs#Get_Describe_OMD) field in Gainsight.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync company fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to any non read-only company fields.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
|Behavior| Description|
|--|--|
|**Do nothing**| Keep the company in Gainsight|
|**Clear** | Remove all the mapped fields, but keep the record in Gainsight|
|**Delete**| Delete the synced company record from Gainsight|
### Syncing people
A Gainsight [Person](https://support.gainsight.com/Gainsight_NXT/API_and_Developer_Docs/Person_API/Person_API_Documentation) is a standard object that stores unique records of people in the real world.
#### Record matching
To match rows from your model to people in Gainsight, you need to select the model column that contains values that match the **Email** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync people fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to any non read-only people fields.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
|Behavior| Description|
|--|--|
|**Do nothing**| Keep the person in Gainsight|
|**Clear** | Remove all the mapped fields, but keep the record in Gainsight|
|**Delete**| Delete the synced person record from Gainsight|
### Syncing custom objects
Hightouch lets you update [Low Volume Custom Objects](https://support.gainsight.com/SFDC_Edition/Data_Management/Managing_Data_In_Gainsight/Low_Volume_Custom_Objects) in Gainsight.
You can use these objects to store low volume data that needs to be edited frequently.
Hightouch diplays Low Volume custom objects in your Gainsight instance in a picklist at the beginning of your sync configuration.
To create custom objects, refer to [Gainsight's docs](https://support.gainsight.com/Gainsight_NXT/02Data_Management/Objects/Gainsight_Objects#Create_Low_Volume_Custom_Objects).
Gainsight doesn't support creating or updating High Volume Custom Objects via [their API](https://support.gainsight.com/Gainsight_NXT/API_and_Developer_Docs/Custom_Object_API/Gainsight_Custom_Object_API_Documentation).
As a workaround, you can sync your data to [S3](/destinations/s3) and use Gainsight's [S3 Connector](https://support.gainsight.com/SFDC_Edition/Connectors/Connectors/File-Based_Integration/Gainsight_S3_Connector) to import your data into a High Volume Custom Object.
#### Record matching
To match rows from your model to custom objects in Gainsight, you need to select a model column and corresponding Gainsight field. You can match on any [filterable, unique](https://support.gainsight.com/Gainsight_NXT/API_and_Developer_Docs/Data_Management_APIs/Data_Management_APIs#Get_Describe_OMD) field in Gainsight.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync custom object fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to any non read-only custom object fields.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
|Behavior| Description|
|--|--|
|**Do nothing**| Keep the custom object in Gainsight|
|**Clear** | Remove all the mapped fields, but keep the record in Gainsight|
|**Delete**| Delete the synced custom object from Gainsight|
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Gainsight PX
**URL:** https://hightouch.com/docs/destinations/gainsightpx
**Description:** Powerful data analytics and product engagement platform
**Section:** Destinations
## Setup
Gainsight PX requires an API key. To find the API key, navigate to the [Gainsight PX Dashboard Settings](https://app.aptrinsic.com/settings/api-keys).
In the administrative settings, select the REST API page under the An API Key can be created on this page.
If an API key has never been created, you can create one here by clicking the New API Key button.
## Syncing users and accounts
Hightouch supports syncing to the following Gainsight PX resources:
- Users
- Accounts
### Sync modes
This integration supports Upsert mode.
In the Upsert mode, new users or accounts will be inserted into Gainsight PX and all attributes will be kept up-to-date within Gainsight PX.
### Record matching
User records can be matched from your source to your Gainsight PX workspace by your Gainsight PX User Email in the Upsert mode.
Account records can be matched from your source to your Gainsight PX workspace by your Gainsight PX Account ID in the Upsert mode.
### Tag keys
Gainsight PX requires a product to sync data to. These products can be accessed by finding the tag key of the product.
Navigating to [Gainsight PX Dashboard Products Page](https://app.aptrinsic.com/settings/products)
All your products can be found here.
The tag key can be copied and entered into the Hightouch sync configuration.
### Field mapping
You can sync columns from your source to Gainsight's default and custom fields. Hightouch automatically detects existing custom fields from your Gainsight PX account.
You can add custom attributes for users and accounts in your Gainsight PX account.
### Delete mode
You can choose what Hightouch's behavior is when records leave the query result set. The default is doing nothing, but you can also set Hightouch to delete the user or account data on record exit.
- `DELETE` : Remove a user or account record and delete the information.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Google Ad Manager 360
**URL:** https://hightouch.com/docs/destinations/gam360
**Description:** Optimize ad delivery with targeted in-sync audiences
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| -------------------------------------- | --------------------------------------------------------------------------- | -------------------- | ----------------------------------------------------------------------------------------------------------------- |
| [**Any data set**](#syncing-data-sets) | Sync data from a source to an Ad Manager cloud storage bucket as a CSV file | Insert, All | [Segment Population API](https://developers.google.com/ad-manager/api/reference/v202411/SegmentPopulationService) |
The Google Ad Manager 360 (GAM 360) destination allows you to batch upload identifiers to audience segments in bulk by sending your desired segments to an Ad Manager cloud storage bucket. The segments appear as [a CSV file](#example-exported-file), which you can then use in GAM 360.
The integration supports these sync modes:
- **Insert**: Insert mode creates a CSV file with the rows that were added since the last sync.
- **All**: All mode creates a CSV file with all the rows in the query results, every time the sync runs.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Prerequisites
Before you can use Hightouch to batch upload your identifiers to GAM 360, ensure you've completed these steps, taken from [Google Ad Manager Help](https://support.google.com/admanager/answer/4349785/):
1. Locate and submit your network code. Find your network code in Ad Manager under **Admin > Global settings > Network code**.
2. Ensure that you've created an active first-party audience segment with which you want to associate the identifiers. If you don't want this segment to collect users using your own inventory, select "Publisher Managed" as the Population method.
Expect to wait up for 24 hours from the time of segment creation to the time
when you can perform batch upload.
3. [Create a Google Group](https://support.google.com/groups/answer/2464926) that contains all Google Accounts that will have access to upload and view files.
Once your account manager has completed your setup, you can create the file of identifiers you want to upload.
## Connect to Google
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Google Ad Manager 360** and click **Continue**. You can then authenticate Hightouch to Google Ad Manager 360 by entering your:
- **GCP Credentials**: if you need to create a cloud credential, consult [our documentation](/security/gcp)
- **GCP Project ID**: you can follow [these steps](https://support.google.com/googleapi/answer/7014113?hl=en) to find it
- **GCS Bucket Name**: this be should just be the name of the bucket, not a URL
Once you've done this, ensure you run the `gcloud` command to grant Hightouch's service account the [Storage Object Admin](https://cloud.google.com/storage/docs/access-control/iam-roles#standard-roles) role. To run this command, you need to install the [gcloud CLI tool](https://cloud.google.com/sdk/docs/install). Otherwise, if you use [Google Groups](https://groups.google.com/), you can also [assign this role to a group member](https://support.google.com/a/answer/167094) by using the Admin console or Google Groups.
## Sync configuration
Once you've set up your Google Ad Manager 360 destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Google Ad Manager 360 destination you want to sync to.
### Syncing data sets
#### List of identifiers
The GAM 360 destination supports several different identifier types.
In the first mapping section, select the column corresponding to the list of identifiers you want to batch upload on the left. On the right, select the type of identifiers that are being uploaded to each segment.
You can find the full list of identifiers in [Google's docs](https://support.google.com/admanager/answer/4349785?hl=en&sjid=15048174647173915590-NA).
#### Audience segment identifiers
In the next section, you need to denote the audience segment IDs with which the identifiers from the previous section should be associated. You can either specify a column from your model to associate each identifier with a different segment, or a static value to associate all identifiers with the same segment. Use the toggle on the right of the field to flip between these two options.
#### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync,
Hightouch overwrites it.
#### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter '-86400' (`24 hours * 60 minutes * 60 seconds`).
#### Empty file results
In insert mode, you can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
## Example exported file
_Example file content for encrypted cookie IDs:_
```
cookie_encrypted,list_id
ScpJKu-yV8je93qkd32MOA,3153490
w2gsrUcwxF-OiJTRmQswQA,3153490
```
_Example file content for raw AdIDs:_
```
cookie_adid,list_id
38400000-8cf0-11bd-b23e-10b96e40000d,3153490
38400001-8cf1-11be-b23f-10b96e40000e,3153490
```
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Google Cloud Storage
**URL:** https://hightouch.com/docs/destinations/gcs
**Description:** Transfer data at scale from your warehouse to Google Cloud Storage
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
| ------------ | ---------------------------------------------------------------------- | -------------------- |
| Any data set | Sync data from a source to Google Drive as JSON, CSV, or Parquet files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Diff mode creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Google
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Google Cloud Storage** and click **Continue**. You can then authenticate Hightouch to Google Cloud Storage by entering your:
- **GCP Credentials**: if you need to create a cloud credential, consult [our documentation](/security/gcp)
- **GCP Project ID**: you can follow [these steps](https://support.google.com/googleapi/answer/7014113?hl=en) to find it
- **GCS Bucket Name**: this be should just be the name of the bucket, not a URL
## Sync configuration
Once you've set up your Google Cloud Storage destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Google Cloud Storage destination you want to sync to.
### Select file format
Hightouch supports syncing JSON, CSV, Parquet, and XML files to Google Cloud Storage.
### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it.
If you are using an audience and would like to include the audience name, you will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter '-86400' (`24 hours * 60 minutes * 60 seconds`).
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `company`, `email`, `name`, and `price` fields. Hightouch exports these fields to new fields in the file and ignores all other columns from your results.
### CSV options
If you're syncing to a CSV file, you have additional configuration options:
- **Delimiter**: Your options are comma (`,`), semicolon (`;`), pipe (`|`), tilde (`~`), and tab
- (Optional) Whether to **include a CSV header row** in the exported files
- (Optional) Whether to **include a [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark)** in the exported files, the BOM is ``
### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Genesys
**URL:** https://hightouch.com/docs/destinations/genesys
**Description:** Connect conversations across channels and deliver seamless customer experience with Genesys Cloud.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|-------------------|--------------------------------------------------------|------------------------|----------------------------------------------------------------------------------------------------------|
| **External contacts** | Sync data from any source to Genesys external contacts | Upsert, Update, Insert | [Contacts docs](https://developer.genesys.cloud/commdigital/externalcontacts/externalcontacts-apis) |
| **Organizations** | Sync data from any source to Genesys organizations | Upsert, Update, Insert | [Organizations docs](https://developer.genesys.cloud/commdigital/externalcontacts/externalcontacts-apis) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Genesys
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Genesys** and click **Continue**. You can then authenticate Hightouch to **Genesys**.
To create your Genesys OAuth client credentials, follow this [Genesys guide](https://developer.genesys.cloud/authorization/platform-auth/guides/oauth/module-1-client-credentials#:~:text=Create%20Client%20Credentials,Client%20Credentials%20as%20Grant%20Type.).
Enter the following fields into Hightouch:
- **Region** (Required): Find which region to use in the [Genesys API docs](https://developer.genesys.cloud/platform/api/)
- **Client ID** (Required)
- **Client secret** (Required)
## Sync configuration
Once you've set up your Genesys destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Genesys destination you want to sync to.
### Syncing contacts
Sync data from any source to Genesys external contacts
#### Record matching
You can match rows from your model to contacts in Genesys on any column in your model and any field in Genesys. Ensure the data types of the model column and Genesys field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Genesys automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom contact fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
|----------------|--------------------------------------------------------------|
| **Do nothing** | Keep the contact in Genesys with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the contact in Genesys |
| **Delete** | Delete the synced contacts from Genesys |
### Syncing organizations
Sync data from any source to Genesys organizations
#### Record matching
You can match rows from your model to organizations in Genesys on any column in your model and any field in Genesys. Ensure the data types of the model column and Genesys field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Genesys automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync organization fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom organization fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
|----------------|-------------------------------------------------------------------|
| **Do nothing** | Keep the organization in Genesys with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the organization in Genesys |
| **Delete** | Delete the synced organizations from Genesys |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Gladly
**URL:** https://hightouch.com/docs/destinations/gladly
**Description:** Streamline multi-channel customer communication
**Section:** Destinations
## Overview
Gladly is a customer service platform that enables businesses to communicate with their customers across various channels, such as phone, email, chat, social media, and more. Syncing data to Gladly means integrating your customer data with the Gladly platform so that your customer service team can access and use this data to provide better and more personalized support to your customers.
## Supported syncing
|Sync Type|Description|Supported Sync Modes |API Reference |
|--|--|--|--|
|**Customers**|Sync data from any source to Gladly customers|Insert, Upsert, Update| [Customer docs](https://developer.gladly.com/rest/#tag/Customers)|
|**Tasks**|Sync data from any source to any Gladly tasks|Insert, Update|[Task docs](https://developer.gladly.com/rest/#tag/Tasks)|
## Connect to Gladly
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Gladly**. You can then authenticate Hightouch to Gladly by entering:
- A Gladly API Token
- Email on your Gladly account
- Gladly domain
To retrieve a Gladly **API Token**, open the hamburger menu in the top left corner of Gladly. Select the **Settings** option. Under the **App Developer Tools** section, click the **API Tokens** option. You can use an existing token or click the **Create Token** button to generate a new one.
We recommend having the following roles on the account: `Administrator`, `Agent`, `API User`, and `Developer`.
To check the roles, you select **Users** under the **People** section. Hover over the row with the **Email** you provided to Hightouch. Click on the icon with three dots that appear and select **Edit**.
To get the **Email** on your Gladly account, click the user icon in the top right corner. Select the **Profile** option. You should see the account's associated **Email** there.
The **Domain** is the Gladly domain you are logged into. It should be similiar to `https://{organization}.gladly.com`. You only need to provide this portion: `{organization}.gladly.com`.
## Sync configuration
Once you've connected your Gladly destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Gladly destination you want to sync to.
### Customers
Hightouch supports inserting, upserting and updating customers.
#### Record matching
You can match rows in your model with customers in Gladly on **Email**, **External Customer ID**, or **Phone Number**.
#### Field and custom mappings
You can sync any columns from your source to both default properties or custom attributes.
Make sure that if you are creating a new custom attribute, please inform [Gladly Customer Support](https://help.gladly.com/docs/contact-gladly-support) before or after running the sync in order for this attribute to be seen/used on Gladly. If the custom attribute already exists, then you are all set.
### Tasks
Hightouch supports creating new tasks and updating existing ones.
#### Record matching
You can match rows in your model with tasks in Gladly on **Task ID**.
#### Field mapping
You can sync any columns from your source to default properties in Gladly.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Gong
**URL:** https://hightouch.com/docs/destinations/gong
**Section:** Destinations
## Setup
To get started with Gong, create an integration with Gong, and use your Access Token and Secret to create a new destination within Hightouch.
The Gong API page can be found here: \([https://app.gong.io/company/api](https://app.gong.io/company/api)\).
## Select CRM integration
Hightouch allows you to choose which CRM integration you wish to sync data to. This can be selected on the sync configuration page.
## Syncing CRM data
### Upload CRM data
The Gong integration allows users to upsert the following object types:
- CRM Users
- Accounts
- Deals
- Contacts
- Leads
### Field mapping
You can sync columns from your source to Gong's fields.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Google Ads
**URL:** https://hightouch.com/docs/destinations/google
**Description:** Empower your marketing team to run highly granular retargeting and lookalike campaigns on Google
**Section:** Destinations
Google is centralizing data delivery on the Data Manager API for better match
quality, simpler maintenance, and built-in privacy features. You can migrate
existing Customer Match syncs without rebuilding anything. See [Migrate to
Data Manager API](#migrate-to-data-manager-api).
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| [**Customer Match User List**](#customer-match-user-lists) | Creates a customer list for the Customer Match feature of Google Ads. Segments are automatically created in Google Ads based on the name of the model | Add, Update, Remove | [Upload customer lists ↗](https://developers.google.com/google-ads/api/docs/remarketing/audience-types/customer-match) |
| [**Click Conversions**](#click-conversions) | Sends click conversion tracking information to Google Ads for each record in the query results | Insert | [Upload click conversions ↗](https://developers.google.com/google-ads/api/docs/conversions/upload-clicks) |
| [**Click Conversion Adjustments**](#click-conversion-adjustments)| Sends click conversion adjustments to Google Ads for each record in the query results | Insert | [Upload conversion adjustments ↗](https://developers.google.com/google-ads/api/docs/conversions/upload-adjustments) |
| [**Call Conversions**](#call-conversions) | Sends call conversion tracking information to Google Ads for each record in the query results | Insert | [Upload call conversions ↗](https://developers.google.com/google-ads/api/docs/conversions/upload-calls) |
| [**Offline Store Conversions**](#offline-store-conversions) | Sends offline store sales conversion tracking information to Google Ads for each record in the query results | Insert | [Upload store sales ↗](https://developers.google.com/google-ads/api/docs/conversions/upload-store-sales-transactions) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
API version in use: [v16](https://developers.google.com/google-ads/api/reference/rpc/v16/overview)
## Connect to Google Ads
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button.
Select **Google Ads** and click **Continue**.
Hightouch supports three authentication methods for Google Ads:
| Method | Description |
| -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Data Manager API OAuth (Recommended)** | OAuth-based authentication that uses Google's [Data Manager API](https://developers.google.com/data-manager/api). Recommended for all new destinations. |
| **OAuth (Legacy)** | OAuth-based authentication that uses the Google Ads API directly. Existing destinations configured with this method continue to work without changes to their sync configurations. |
| **Data Manager API (Existing Partner Link)** | A non-OAuth method for customers who can't complete the OAuth flow (for example, due to two-factor authentication restrictions). Requires a partner link that already exists in your Google Ads account. |
### Data Manager API OAuth (Recommended)
Select **Data Manager API OAuth (Recommended)**, click **Log in to Google Ads**, log in with your Google Account, and click **Allow**.
You should then see a success notification in the Hightouch UI.
With this method, Hightouch creates a partner link between Hightouch and your Google Ads account during the first sync run. Creating the partner link requires the authenticating user to have **Admin** access on the account. If the user doesn't have Admin access, the sync fails with a `PERMISSION_DENIED` error. See [PERMISSION_DENIED from Data Manager API](#permission_denied-from-data-manager-api) to verify and fix the access level.
### OAuth (Legacy)
Select **OAuth (Legacy)**, click **Log in to Google Ads**, log in with your Google Account, and click **Allow**.
You should then see a success notification in the Hightouch UI.
If you already have a Google Ads destination configured with this method, you
can re-authorize it using the **Data Manager API OAuth (Recommended)** method.
No changes are required to existing sync configurations.
### Data Manager API (Existing Partner Link)
Use this method if you can't complete the OAuth flow (for example, due to two-factor authentication restrictions on your Google account) and you already have a Hightouch partner link set up in your Google Ads account.
Unlike the OAuth method, this method doesn't create the partner link for you. Before connecting, create the partner link in your Google Ads account by linking **Hightouch** as a data provider. See Google's [data partner linking](https://developers.google.com/data-manager/api/devguides/concepts/destinations#ads-credentials) documentation for details.
Select **Data Manager API (Existing Partner Link)** and enter your Google Ads account ID. You can find this 10-digit value at the top-right corner of your Google Ads dashboard. Remove any dashes or spaces.
Because this method doesn't authenticate as a specific Google user, Hightouch
can't query your Google Ads account on your behalf. As a result, existing user
lists and ad accounts don't auto-populate in dropdowns during sync
configuration—you'll need to provide these values manually.
## Sync configuration
Once you've set up your Google Ads destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data.
Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin.
Then, select the relevant model and the Google Ads destination you want to sync to.
### Select account
To begin, select the account you want to sync data to.
If an account appears at the top level, it means you have direct access to that ad account or [manager account](https://support.google.com/google-ads/answer/7459399).
Hightouch accesses nested accounts via the root account that you have direct access to.
In the preceding example screenshot, the authenticated account has direct access to the **Hightouch** (5138398925) ad account and the **Hightouch Manager Account** (5277198417).
The authenticated account also has access to the manager account's three nested accounts, including the Hightouch (5138398925) ad account.
Selecting either account at the top level syncs directly to those accounts without logging in via another manager account.
Selecting a nested account requires Hightouch to login via the root manager account.
### Customer match user lists
For Customer Match user lists, Google recommends using the [Data Manager
API](/destinations/google-data-manager). If your destination already uses
**Data Manager API OAuth**, your Customer Match syncs are already using the
new API. If you're still on **OAuth (Legacy)**, see [Migrate to Data Manager
API](#migrate-to-data-manager-api) to switch.
Google's [Customer Match](https://support.google.com/google-ads/answer/10550383) tool lets you use the online and offline data your customers have shared with you to deliver targeted ads.
Hightouch lets you create and maintain customer match lists via the Google Ads API.
It can take 6 to 48 hours for a list to be populated with members once Hightouch has sent the data to Google.
You can find more information about this on the API reference page linked in the [supported syncing](#supported-syncing) section.
#### User list type
To begin, select the [upload key type](https://developers.google.com/google-ads/api/reference/rpc/v13/CustomerMatchUploadKeyTypeEnum.CustomerMatchUploadKeyType) for the customer match list.
It can be one of these values:
- **CONTACT_INFO**
- **MOBILE_ADVERTISING_ID**
- **CRM_ID**
Choose the type that aligns with the data fields you are sending.
For example, if you are sending email or address data, select **CONTACT_INFO**.
#### Select an existing list or create a new one
You can create a new [Customer match user list](https://support.google.com/google-ads/answer/6276125) or use an existing one.
When creating a new list, you can optionally enter a name.
Otherwise, Hightouch defaults to the name of the associated model.
To use an existing list, select the desired list from the dropdown.
If you choose to create a new list, you can't update the
[account](#select-account) to which you're syncing data after the initial sync
run. If you need to change the account, configure a new sync.
Optionally, you can also set the **membership lifespan** for the list.
The value must be between 0 and 540. If you do not set this value, the default value is 540.
Starting April 7th, 2025, the maximum membership duration for a Google Ads customer list is **540 days**.
From this date, existing lists configured to have no membership expiration set or membership expiration greater than 540 days will be updated to use the maximum membership duration of 540 days, and a membership duration of 540 days will be retroactively applied to all those existing members that have a longer duration.
See [Google's announcement](https://ads-developers.googleblog.com/2025/02/update-to-customer-match-membership.html) for more information.
#### User identifiers
To identify which users to add or update in an audience, select model columns and the corresponding Google Ads fields.
You must map at least one identifier.
#### Handling PII and hashing
You can enable PII hashing if you wish for Hightouch to detect whether your data requires normalization and hashing based on the chosen record mapping.
For example, if you select **Email** and the model column with the email data is unhashed, Hightouch hashes the value for you.
By default, Hightouch automatically detects if the following fields require hashing:
- **Email**
- **Phone**
- **First Name**
- **Last Name**
#### Cleaning your data
Optionally, you can decide if Hightouch should clean your phone numbers and remove `null` or incorrectly formatted values from your data.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results.
You have the following options:
| Behavior | Description |
| ----------------- | -------------------------------------------------------- |
| **Do nothing** | Keep the record in the audience |
| **Remove record** | Remove the Google Ads record from the specified audience |
#### EU user consent policy
If you are uploading data for users in the European Economic Area ([EEA]()),
you need to ensure you are passing the required consent signals to [Google](https://www.google.com/about/company/user-consent-policy/?sjid=2196795394224792925-NA).
For each sync, you can specify the consent status for these types:
- `ad_user_data`: consent for sending user data to Google for advertising purposes
- `ad_personalization`: consent for ad personalization
Check out Google's support [doc](https://support.google.com/google-ads/answer/14310715?hl=en) to learn more.
### Conversion events
Hightouch supports syncing the following conversion events:
- [Click conversions](#click-conversions) and [Offline click conversions](#offline-click-conversions)
- [Click conversion adjustments](#click-conversion-adjustments)
- [Call conversions](#call-conversions)
- [Offline store conversions](#offline-store-conversions)
When syncing conversion events, Hightouch treats any records added to [your source](/getting-started/concepts) as new events and sends them to Google Ads when your sync runs.
#### Custom conversion variables
You can [map](/syncs/mapping-data) custom variables when syncing conversion events to Google Ads.
First, you need to set up [custom conversion variables](https://support.google.com/google-ads/answer/9962082) in Google. Custom conversion variables are only available for the following:
- Conversion tracking using [tags on your website](https://support.google.com/google-ads/answer/6331314), if you're syncing [click conversions](#click-conversions) or [call conversions](#call-conversions)
- [Offline conversion import](https://support.google.com/google-ads/answer/2998031), if you're syncing [offline click conversions](#offline-click-conversions)
You can read more about this in [Google Ads Help](https://support.google.com/google-ads/answer/9964350) and also in [Google Ads' API documentation](https://developers.google.com/google-ads/api/docs/conversions/conversion-custom-variables).
#### Currency codes
Google requires all conversion events to include a currency code in a separate column in addition to the amount.
This applies to [click conversions](#click-conversions), [click conversion adjustments](#click-conversion-adjustments), and [offline store conversions](#offline-store-conversions).
Consult Google's chart of [supported currency codes](https://developers.google.com/adsense/management/appendix/currencies) to ensure your data meets their specifications.
#### Enhanced conversions
You can sync both types of [**Enhanced conversions**](https://support.google.com/google-ads/answer/9888656) to Google Ads using Hightouch:
1. [**Enhanced conversions for leads**](https://support.google.com/google-ads/answer/9888656#leads): set up an [Offline click conversion](#offline-click-conversions) in Google Ads; then select **Click Conversions** as the [sync type](#supported-syncing) and [set up your field mappings](#user-identifiers-1) accordingly
2. [**Enhanced conversions for web**](https://support.google.com/google-ads/answer/13261987): select [Click Conversion Adjustments](#click-conversion-adjustments) as the [sync type](#supported-syncing) and **Enhancement** as the [adjustment type](#select-conversion-action-and-adjustment-type)
#### EU user consent policy
If you are uploading data for users in the European Economic Area ([EEA]()),
you need to ensure you are passing the required consent signals to [Google](https://www.google.com/about/company/user-consent-policy/?sjid=2196795394224792925-NA).
For [click conversions](#click-conversions), [call conversions](#call-conversions) and [offline store conversions](#offline-store-conversions), you can specify the Google consent status for advertising purposes, by mapping these fields in the sync configuration:
- `Ad user data consent` (for the `ad_user_data` property)
- `Ad personalization consent` (for the `ad_personalization` property)
As per [Google's API documentation](https://developers.google.com/google-ads/api/reference/rpc/v17/Consent), these fields only accept the following values: `GRANTED`, `DENIED`, `UNSPECIFIED`, `UNKNOWN`.
Check out Google's support [doc](https://support.google.com/google-ads/answer/14310715?hl=en) to learn more.
### Click conversions
Google Ads allows you to [import conversions from ad clicks](https://support.google.com/google-ads/answer/12216424) to see how your online ads are performing.
Before setting up your sync, you need to [set up a conversion action](https://support.google.com/google-ads/answer/12216226) in Google Ads.
After clicking **New conversion action**, remember to select **Website**.
It can take over 24 hours for click conversions to be processed by Google.
You can find more information about this and other requirements on the API reference page linked in the [supported syncing](#supported-syncing) section.
#### Offline click conversions
**Offline click conversions** are not [offline store
conversions](#offline-store-conversions).
You can sync [offline click conversion imports](https://support.google.com/google-ads/answer/2998031) which allow you to measure what happens in the offline world after an ad results in a click or call to your business.
For these events, you need to [set up offline conversion imports](https://support.google.com/google-ads/answer/7012522).
After clicking **New conversion action**, remember to select **Import**.
Then select **CRMs, files, or other data sources** and **Track conversions from clicks** and continue the offline conversion import setup. You can safely skip setting up a data source.
Check out the [blog
post](https://hightouch.com/blog/google-offline-conversions) and
[playbook](https://hightouch.com/playbooks/automate-offline-conversion-uploads-to-maximize-roas)
to learn how to sync **offline click conversions** to Google Ads.
#### Select conversion action
Select the **conversion action** you previously set up in your Google Ads account.
The dropdown menu displays the conversion action IDs present in your Google Ads account.
#### User identifiers
Select which columns to sync from your source to Google Ads' fields. When syncing click conversions, at least one identifier is required:
- **GCLID**
- **GBRAID**
- **WBRAID**
- **User hashed email(s)**
- **User hashed phone(s)**
Furthermore, **Conversion time** is a mandatory field.
You can send [enhancements as click conversions](https://support.google.com/google-ads/answer/9888656#leads) by mapping the **User hashed email(s)** and **User hashed phone(s)** fields instead of **GCLID**.
The [model columns](/models/data-types-casting) mapped to these fields can contain either single values or arrays.
Read through Google Ads' [API documentation](https://developers.google.com/google-ads/api/docs/conversions/upload-identifiers) for more details.
If you opt to send user emails and/or phone numbers instead of **GCLID**, keep in mind that each click conversion can only have up to five user identifiers in total.
For example, a click conversion can have three emails and two phone numbers, or any combination that adds up to five.
These enhancements are also referred to as **Enhanced conversions for leads**.
They are not **Enhanced conversions for web**, which are described in the
[Syncing click conversion
adjustments](#select-conversion-action-and-adjustment-type) section.
Avoid storing your **GCLID**, **GBRAID**, and **WBRAID** data in the same model column since this can cause issues when syncing your data.
[Google Ads' limitations](https://developers.google.com/google-ads/api/docs/conversions/upload-clicks#id_parameters) determine which values can and cannot be synced together.
If this data is in the same model column and you try mapping this column to multiple Google Ads fields (such as **GCLID** and **WBRAID**), you might receive a `VALUE_MUST_BE_UNSET` or `GBRAID_WBRAID_BOTH_SET` error.
Check out the [common errors](#common-errors) section to learn more about these errors.
If a model row contains a `null` value in a model column mapped to the
**GCLID**, **GBRAID**, or **WBRAID** fields, Hightouch ignores that value when
syncing the row Google Ads. This process helps reduce
[`VALUE_MUST_BE_UNSET`](#value_must_be_unset) or
[`GBRAID_WBRAID_BOTH_SET`](#gbraid_wbraid_both_set) errors.
#### EU user consent policy
Learn more about the EU consent policy [in this section](#eu-user-consent-policy-1).
### Click conversion adjustments
After sending a click conversion to Google Ads, you can [adjust the conversion](https://support.google.com/google-ads/answer/7686447) later by sending a click conversion adjustment.
Wait at least 24 hours after a click conversion is reported to Google Ads before adjusting it to avoid a `CONVERSION_NOT_FOUND` or `TOO_RECENT_CONVERSION` error.
You can find more information about this on the API reference page linked in the [supported syncing](#supported-syncing) section.
#### Select conversion action and adjustment type
Select the **conversion action** you previously set up in your Google Ads account.
The dropdown menu displays the conversion action IDs present in your Google Ads account.
You also need to select the adjustment type. You can choose from:
- [Retraction](https://support.google.com/google-ads/answer/7686447)
- [Restatement](https://support.google.com/google-ads/answer/7686447)
- [Enhancement](https://support.google.com/google-ads/answer/13261987)
Enhancements are also referred to as **Enhanced conversions for web**. They
are not **Enhanced conversions for leads**, which are described in the
[Syncing click conversions](#user-identifiers-1) section.
If you select the **Enhancement** adjustment type, you need to select if this is a first party or third party data upload.
#### Field mapping
Select which columns to sync from your source to Google Ads' fields:
- **GCLID**
- **Conversion time**
- **Adjustment time**
- **Order ID**
If you select the **Retraction** adjustment type, you can only map those fields.
If you select the **Restatement** adjustment type, you can map those four fields plus:
- **Restatement value**, which is mandatory
- **Restatement currency**
If you select the **Enhancement** adjustment type, you can map those four fields plus:
- **Enhancement User Agent**
- various hashed fields: **Email**, **Phone**, **First Name**, **Last Name**, **Street Address**
- **Country**
- **Postal Code**
- **State**
- **City**
#### Handling PII and hashing
You can find more information about this in the [PII and hashing](#handling-pii-and-hashing) section.
### Call conversions
Google Ads allows you to import phone call conversions to keep track of how your calls impact your ad campaigns.
After setting up [phone call conversions](https://support.google.com/google-ads/answer/6275629), you can use Hightouch to sync your data to Google Ads.
You can find more information about this on the API reference page linked in the [supported syncing](#supported-syncing) section.
#### Select conversion action
Select the **conversion action** you previously set up in your Google Ads account.
The dropdown menu displays the conversion action IDs present in your Google Ads account.
#### Field mapping
Select which columns to sync from your source to Google Ads' fields.
**Conversion time**, **Caller ID**, and **Call start time** are mandatory fields.
#### EU user consent policy
Learn more about the EU consent policy [in this section](#eu-user-consent-policy-1).
### Offline store conversions
**Offline store conversions** are not [offline click
conversions](#offline-click-conversions).
Google Ads offers enhanced store sales measurement and reporting data to better optimize your online ads.
After setting up [offline store sales conversions](https://support.google.com/google-ads/answer/9995886), you can use Hightouch to sync your data to Google Ads.
You can find more information about this on the API reference page linked in the [supported syncing](#supported-syncing) section.
#### Select conversion action
Select the **conversion action** you previously set up in your Google Ads account.
The dropdown menu displays the conversion action IDs present in your Google Ads account.
#### Optional settings
You can decide if you would like to include the following information:
- **custom key**, to segment store sales conversions
- **loyalty fraction**, which is a decimal value that sets the percentage of overall sales that can be associated with a customer
- **transaction upload fraction**, which is a decimal value that sets the percentage of overall sales with customer information that are already uploaded to Google Ads
#### First or third party data
You need to select if this is a first party or third party data upload.
#### Field mapping
Select which columns to sync from your source to Google Ads' fields.
**Transaction time** and **Transaction amount (micros)** are mandatory fields.
Any transaction amount here should be in the unit **micros**. For example,
$200 = 200000000.
#### EU user consent policy
Learn more about the EU consent policy [in this section](#eu-user-consent-policy-1).
#### Handling PII and hashing
You can find more information about this in the [PII and hashing](#handling-pii-and-hashing) section.
#### Cleaning your data
You can find more information about this in the [Cleaning your data](#cleaning-your-data) section.
## Migrate to Data Manager API
Google is centralizing data delivery on the Data Manager API. You can migrate supported syncs to the Data Manager API without replacing destinations, recreating audiences, or rebuilding syncs. Read the [launch article](https://hightouch.com/blog/introducing-google-data-manager-api) for more background.
### Why migrate
Google is shifting investment from the legacy Google Ads and DV360 APIs to the Data Manager API. Migrating gives you:
- **Unified endpoint**: One delivery path instead of multiple specialized connections.
- **Simpler maintenance**: Less setup and fewer ongoing configuration changes.
- **Better match quality**: Richer customer match data improves attribution and audience reach.
- **Privacy-enhancing technology**: Confidential matching and encryption built into the API.
### What you can migrate today
Hightouch currently supports migrating Google Ads Customer Match user list syncs. Support for additional Data Manager use cases is expanding as Google updates the API. Hightouch continues to support legacy Google Ads and DV360 integrations while coverage expands.
### Migration steps
Migration doesn't require a new destination or new syncs. When you reauthenticate an existing Google Ads destination with the Data Manager API OAuth method, Hightouch moves your supported syncs to the new API automatically. Sync behavior remains the same after migration.
1. In Hightouch, go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and select your existing Google Ads destination.
1. Under **Authentication method**, select **Data Manager API OAuth**.
1. Click **Reauthorize Google Ads**, log in with your Google account, and click **Allow**.
1. Click **Save Changes**.
After you save, your existing Customer Match user list syncs use the Data Manager API. You don't need to update any sync configurations.
## Tips and troubleshooting
For Hightouch platform error codes related to Google Ads, see [Error codes: Google Ads](/developer-tools/error-codes#google-ads).
### Matched users count
Below only applies to the customer match user list sync type.
Google Ads contain a few different matched users count in their platform.
Hightouch refers to the [display network
size](https://support.google.com/google-ads/answer/117120?ctx=glossary&sjid=7733029462129080931-NA)
and the [search network
size](https://support.google.com/google-ads/answer/90956?ctx=glossary&sjid=7733029462129080931-NA).
If the display network size is available, Hightouch displays that value, else the
search network size is used. Google Ads also have a concept of a match rate
but this is different than what Hightouch is calculating, which is the % of
targetable users from your model in this GoogleAds audience. Per Google, ”Your
Customer Match match rate is the percentage of your uploaded customer list
that could be connected to Google users, so you can see how much of your list
is usable. Keep in mind that your match rate is not an indicator of how your
lists will perform, but rather an indicator of whether you're uploading the
right data in the right way”
([reference](https://support.google.com/google-ads/answer/10534785?hl=en)).
### Audience uploads showing size 0
After a successful Customer Match sync, your audience in Google Ads may show a size of 0 for up to 24–48 hours. This is expected — Google Ads processes uploaded audience data asynchronously and doesn't reflect audience sizes immediately. A successful sync in Hightouch means the data was accepted by Google's API, but it takes time for Google to match and populate the audience.
### Common errors
#### At least one click identifier ('gclid', 'gbraid' or 'wbraid') is required
Make sure that you mapped at least one identifier when syncing [Click conversions](#field-mapping).
#### CONVERSION_PRECEDES_EVENT
As described in [Google Ads' documentation](https://developers.google.com/google-ads/api/reference/rpc/v13/ConversionUploadErrorEnum.ConversionUploadError#conversion_precedes_event), this error occurs if the imported event has a `conversion_date_time` that precedes the click.
To avoid this error, make sure that the timestamps in your source data and your Google Ads account have the same timezone configured.
#### EXPIRED_EVENT
As explained in [Google Ads' documentation](https://developers.google.com/google-ads/api/reference/rpc/v13/ConversionUploadErrorEnum.ConversionUploadError#expired_event), this error happens when a [click conversion](#click-conversions) can't be accepted since its click occurred before this conversion's [click-through window](https://support.google.com/google-ads/answer/3123169).
#### GBRAID_WBRAID_BOTH_SET
This error happens if you sync data both to the **GBRAID** and **WBRAID** fields.
You can read more about this in [Google Ads' API documentation](https://developers.google.com/google-ads/api/docs/conversions/upload-clicks).
#### INCOMPATIBLE_UPLOAD_KEY_TYPE
When syncing Customer Match user lists, make sure to select the correct [user list type](#user-list-type). If you selected the wrong type when setting up your sync, create a new sync with the correct type.
#### INVALID_CONVERSION_ACTION
When syncing Click conversions, ensure that you've chosen the correct conversion type within the Google Ads UI.
For normal [Click conversions](#click-conversions), choose the **Website** type. For [Offline click conversions](#offline-click-conversions), choose the **Import** type.
Furthermore, make sure that:
- your data satisfies the requirements outlined in [this section](https://developers.google.com/google-ads/api/docs/conversions/upload-clicks#upload_clickconversion) of Google Ads' API documentation
- you selected the correct [Google Ads account](#select-account) in your sync configuration
Make sure to not confuse [Offline click
conversions](#offline-click-conversions) with [Offline store
conversions](#offline-store-conversions).
#### INVALID_CUSTOMER_ID
You may receive a `400 - {"error":{"code":400,"message":"Request contains an invalid argument."..."message":"Invalid customer ID...}` error message because you are attempting to sync to a Google account that is different from the initial sync.
If there has been a successful initial sync with sync configurations of `Customer Match User List` and `Create a new user list`, changing the Google account will result in sync failure on the next run. You can resolve the error by either:
- Selecting the original Google account that was used on the initial sync
- Creating a new sync and selecting the new Google Account
#### INVALID_GRANT
The token for the user that connected the Google Ads destination has expired or revoked. Please re-auth within the destination settings.
#### No error message provided
This error can happen if you [rename a model column](/syncs/mapping-data#model-column-changes) after adding it as a field mapping in your sync configuration.
To resolve this, update your field mappings to contain the updated model column names.
#### REQUIRED_FIELD_MISSING
[Customer Match User Lists](#record-matching) require at least one personal identifier. Check the required fields relevant for the match list type you've chosen.
#### PERMISSION_DENIED from Data Manager API
You may receive a `403 - {"error":{"code":403,"message":"The caller does not have permission","status":"PERMISSION_DENIED","domain":"datamanager.googleapis.com"}}` error when syncing to Google Ads.
This occurs when Hightouch creates the partner link during a sync run but the authenticating user doesn't have **Admin** access on the Google Ads account. Google Data Manager checks access against the login account, which in manager-account setups is the manager (MCC) account. A user with a lower access level, such as Standard, can sign in successfully but can't create the partner link.
To verify the access level:
1. Go to your Google Ads account.
2. Open **Access and security**.
3. Open the **Users** tab.
4. Turn off the **Show users in full hierarchy** toggle.
5. Confirm the authenticating user is listed directly on the manager (MCC) account.
6. Confirm the user's access level is **Admin**. For more on access levels, see Google's [access levels](https://support.google.com/google-ads/answer/9978556) documentation.
If the user isn't an Admin, upgrade the access level and rerun the sync, or re-authorize the destination with a different user who has Admin access. If you can't grant Admin access, create the partner link manually and switch to the [**Data Manager API (Existing Partner Link)**](#data-manager-api-existing-partner-link) authentication method instead.
#### UNAUTHORIZED_CUSTOMER
This error occurs when you try syncing a [click conversion](#click-conversions) to Google Ads that is associated with a different ad account. This commonly happens when importing click conversions from campaigns running under a different sub-account within the same manager account hierarchy.
To resolve this error, verify that the Google Ads customer ID in your sync configuration matches the account that owns the click. If you need to sync click conversions from different ad accounts without using separate syncs, set up a manager account and enable [cross-account conversion tracking](https://support.google.com/google-ads/answer/3030657).
#### VALUE_MUST_BE_UNSET
This error happens if you:
- sync data to the **GCLID** field and also to the **GBRAID** or **WBRAID** fields
- sync data to [custom conversion variables](#custom-conversion-variables) and also to the **GBRAID** or **WBRAID** fields
You can read more about this in [Google Ads' API documentation](https://developers.google.com/google-ads/api/docs/conversions/upload-clicks).
### Live debugger
### Sync alerts
---
## Google ADH (BigQuery)
**URL:** https://hightouch.com/docs/destinations/google-adh-bigquery
**Description:** Google Ads Data Hub enables customized analysis that aligns with your specific business objectives, while respecting user privacy and upholding Google’s high standards of data security.
**Section:** Destinations
## Overview
Hightouch lets you send data from your warehouse into Google BigQuery to join it with Google's Ads Data Hub.
Connecting Hightouch to BigQuery requires some setup in both platforms. It's recommended to set up a service account with the correct permissions in BigQuery before configuring the connection in Hightouch.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------ | ---------------------------------------------- | -------------------- |
| Any data set | Sync data from any source to a Big Query table | Insert |
## BigQuery credential setup
Setup in BigQuery has three main steps:
1. Enable BigQuery for your Google Cloud project
2. Create a service account
3. Grant the Hightouch service account access to your project
### Create a project and enable BigQuery API
1. Login to the [Google Developers Console](https://console.developers.google.com/).
2. Configure the [Cloud Platform](https://console.cloud.google.com/):
- If you don't have a project already, [create one](https://support.google.com/cloud/answer/6251787?hl=en&ref_topic=6158848).
- Once you have a project, [enable the BigQuery API](https://console.cloud.google.com/flows/enableapi?apiid=bigquery&_ga=2.71379221.724057513.1673650275-1611021579.1664923822&_gac=1.213641504.1673650813.EAIaIQobChMIt9GagtPF_AIVkgB9Ch331QRREAAYASAAEgJfrfD_BwE) for it.
3. Copy your **Project ID** for later use.
4. Find the **location** of your BigQuery dataset or sets. You can find this by querying the `INFORMATION_SCHEMA.SCHEMATA` view or by visiting the [Google Cloud web console](https://console.cloud.google.com/) and clicking on a BigQuery dataset in the Explorer panel. You need both **Project ID** and **Data location** when connecting Hightouch to BigQuery.
Make sure [billing is
enabled](https://support.google.com/cloud/answer/6293499#enable-billing) on
your project, otherwise Hightouch can't write into the cluster.
### Create a service account
To create a service account, follow the setup instructions in our [Google Cloud Platform (GCP) documentation](/security/gcp).
### Grant access
Your GCP service account will need [permission](https://cloud.google.com/bigquery/docs/access-control#permissions-predefined-roles) to read and write data to BigQuery.
You can set up your service account to have [full access](#grant-full-access) to your project using a [predefined role](https://cloud.google.com/iam/docs/understanding-roles#predefined_roles).
Otherwise, you can create a [custom role](https://cloud.google.com/iam/docs/roles-overview#custom) and provide [limited access](#grant-limited-access) according to a user-specified list of permissions.
#### Grant full access
You can grant full access by assigning the `bigquery.user`, `bigquery.dataViewer`, and `bigquery.dataEditor` roles to your service account.
You can do this in the [Google Cloud web console](https://cloud.google.com/iam/docs/granting-changing-revoking-access#grant-single-role) or by running these snippets in the [Cloud Shell](https://cloud.google.com/shell/docs/run-gcloud-commands).
Grant permission to **read metadata and list tables**:
```bash
gcloud projects add-iam-policy-binding \
--member serviceAccount: \
--role roles/bigquery.user
```
Grant permission to **read data from tables and views**:
```bash
gcloud projects add-iam-policy-binding \
--member serviceAccount: \
--role roles/bigquery.dataViewer
```
Grant permission to **write data to tables and views**:
```bash
gcloud projects add-iam-policy-binding \
--member serviceAccount: \
--role roles/bigquery.dataEditor
```
#### Grant limited access
If you don't want to grant full access to your BigQuery service account, you can opt to grant limited access instead.
You can do this by assigning the `bigquery.dataViewer` and `bigquery.dataEditor` roles only to the specific datasets, tables, or views you want to use in Hightouch.
Since you are assigning these roles only to specific resources, you need to assign the `bigquery.user` role **and** grant the `bigquery.tables.get` permission at the project level.
For this, you can create a [custom role](https://cloud.google.com/iam/docs/creating-custom-roles#creating) in the Google Cloud web console based on an [existing predefined role](https://cloud.google.com/bigquery/docs/access-control#bigquery.user) (`bigquery.user`), which you can name `custom.bigquery.user`.
When setting up the custom role, click **Add permissions** to add the `bigquery.tables.get` permission to this custom role.
Then assign this role to your service account at the project level.
You can do this in the [Google Cloud web console](https://cloud.google.com/iam/docs/granting-changing-revoking-access#grant-single-role) or by running this snippet in the [Cloud Shell](https://cloud.google.com/shell/docs/run-gcloud-commands):
```bash
gcloud projects add-iam-policy-binding \
--member serviceAccount: \
--role roles/custom.bigquery.user
```
If this `custom.bigquery.user` role still isn't limited enough, you can try assigning the [`bigquery.jobUser`](https://cloud.google.com/bigquery/docs/access-control#bigquery.jobUser) role **and** granting the `bigquery.dataset.get`, `bigquery.tables.get`, and `bigquery.tables.list` permissions at the project level.
Hightouch needs to be able to list the schemas and tables in your BigQuery
project when setting up your sync configuration.
## Connect to Google ADH via BigQuery
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Google ADH (Big Query)** and click **Continue**.
### Configure your service account
Select the **GCP credentials** you [previously created](#create-a-service-account) or click **Create new**. To learn more about these credentials, see the [Google Cloud Provider (GCP) documentation](/security/gcp).
### Configure your destination
Enter the **Project ID** for the project you [enabled the BigQuery API](#create-a-project-and-enable-bigquery-api) for and the **Dataset location**.
## Sync configuration
Once you've set up your Google ADH (BigQuery) destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Snowflake destination you want to sync to.
### Sync mode
Hightouch supports **Insert** mode with the option to send boosted data with [Match Booster](../match-booster/overview).
### Syncing with match booster
You can choose to send boosted emails and/or phones to BigQuery. Hightouch will clean and hash the data accordingly.
## Tips and troubleshooting
### Common errors
### Compatible versions
The supported Oracle DB version is `19c`. Other versions aren't officially supported but may work regardless.
### Live debugger
### Sync alerts
---
## Google Analytics
**URL:** https://hightouch.com/docs/destinations/google-analytics
**Description:** Send events directly to Google Analytics from your data warehouse
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-----------|-----------------------------------|----------------------|------------------------------------------------------------------------------------------------------------------------
**Events** | Sync events to GA4 via the Measurement Protocol API | Insert | [Measurement Protocol Reference](https://developers.google.com/analytics/devguides/collection/protocol/ga4/reference?client_type=firebase)
**Anonymous Audiences** | Sync events and user properties to GA4 to create anonymous audiences | Upsert | [Audiences reference](https://support.google.com/analytics/answer/12799087?hl=en)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Google Analytics
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Google Analytics** and click **Continue**. You can then authenticate Hightouch to **Google Analytics** by entering a Google Analytics **API Secret**.
To generate an API secret, navigate to **Admin > Data Streams > choose your stream > Measurement Protocol > Create**. If you don't have any existing data streams you need to [create one](https://support.google.com/analytics/answer/9304153?sjid=1280583774983537157-NA#stream) first.
If you are syncing anonymous audiences, you must additionally authenticate via OAuth to grant Hightouch the `analytics.edit` scope. This is needed to create new audiences in GA4.
## Sync configuration
Once you've set up your Google Analytics destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Google Analytics destination you want to sync to.
### Syncing events
Hightouch supports sending following Google Analytics event types:
- [**Firebase App Events**](https://developers.google.com/analytics/devguides/collection/protocol/ga4/reference?client_type=firebase)
- [**Gtag.js Events**](https://developers.google.com/analytics/devguides/collection/protocol/ga4/reference?client_type=gtag#payload)
Firebase app events require an associated **Firebase app ID** (`firebase_app_id`) and **app instance ID** (`app_instance_id`). You can find the `firebase_app_id` in the Firebase console in **Project Settings > General > Your Apps > App ID**. You can retrieve the `app_instance_id` through the [Firebase SDK](https://firebase.google.com/docs/reference/android/com/google/firebase/analytics/FirebaseAnalytics#public-taskstring-getappinstanceid).
If you are having issues syncing your Firebase app events, check out [Google's troubleshooting docs](https://developers.google.com/analytics/devguides/collection/protocol/ga4/troubleshooting?client_type=firebase).
The `app_instance_id` isn't the same as your `firebase_app_id`. The `firebase_app_id` identifies your app uniquely, whereas `app_instance_id` identifies a single installation of the app uniquely.
Gtag.js events require an associated **measurement ID** (`measurement_id`) and **client ID** (`client_id`). You can find the `measurement_id` in the Google Analytics UI in **Admin > Data Streams > choose your stream > Measurement ID**. You can retrieve the `client_id` using [these examples](https://developers.google.com/tag-platform/gtagjs/reference#get) from Google.
If you are having issues syncing your Gtag.js events, check out [Google's troubleshooting docs](https://developers.google.com/analytics/devguides/collection/protocol/ga4/troubleshooting?client_type=gtag).
All users of your website have the same `measurement_id`, but the `client_id` value is different for every user of your app.
Refer to Google Analytics [required parameters](https://developers.google.com/analytics/devguides/collection/protocol/ga4/sending-events?client_type=firebase#required_parameters) for more information.
#### Measurement ID
Your **measurement ID** in Google Analytics is an identifier for a web data stream. It typically starts with `UA`, `MO`, or `G` followed by 10-20 characters, for example `UA-12345678-9`. You can find your measurement ID by following the **Find your Measurement ID** link in [Google's docs](https://support.google.com/analytics/answer/12270356?hl=en#:~:text=A%20Measurement%20ID%20is%20an,same%20as%20your%20destination%20ID).
You can also use the steps outlined in the preceding section.
#### Event name
Google Analtyics requries an event name for all events. You can either provide a static value or select to **use a column** from your model data. Hightouch syncs the static or column value as the `events[].name` parameter Google Analytics requires. See [Google's events reference](https://developers.google.com/analytics/devguides/collection/protocol/ga4/reference/events) for all event name options.
#### Event timestamp
Optionally, you can decide to select a model column that contains your event timestamps. Timestamps should be in [ISO 8601](https://www.iso.org/iso-8601-date-and-time-format.html) format or [Unix timestamps](https://en.wikipedia.org/wiki/Unix_time) in milliseconds, which are automatically converted to the format required by Google Analytics.
If this is selection is empty, Google Analytics creates an event timestamp by using the time the event arrives at their server.
#### Field mapping
Field mapping is where you select which event parameters you want to send to Google Analytics. You can map data from any of your model columns to event parameters in Google Analytics. Ensure your model column data type matches the data type of the parameter you want to sync to.
Different event types require different parameters. For example, `purchase` events require the following parameters:
- `currency` (string)
- `value` (number)
- `transaction_id` (string)
- `items` (array with additional [required parameters](https://developers.google.com/analytics/devguides/collection/protocol/ga4/reference/events#purchase_item))
You can use Hightouch's [advanced mapper](https://hightouch.com/docs/syncs/mapping-data#advanced-mapper) to map [column](/syncs/mapping-data#column-values), [variable](/syncs/mapping-data#variable-values), and [static](/syncs/mapping-data#static-values) values, for example, "USD" for `currency`.
See [Google's events reference](https://developers.google.com/analytics/devguides/collection/protocol/ga4/reference/events) for the suggested parameters for each event.
### Syncing audiences
Create anonymous audiences based on user properties in GA4.
Hightouch creates anonymous audiences in GA4 by first creating a new [custom dimension](https://support.google.com/analytics/answer/14240153?hl=en) based on the name of your audience. The custom dimension will be in the format `HT_`.
Once the dimension is created, Hightouch creates an audience using the new custom dimension as a filter. Hightouch will send events with the custom dimension value set to `1` in user properties, and filter based on that value. Anonymous users with the custom dimension property set will be included in your anonymous audience.
When a record is removed from your audience, the custom audience value is set to `-1` and therefore removed from your audience.
#### Audience population
There may be a lag between syncing events and user properties to GA4 and seeing audiences being populated.
GA4 recommends sending an event with user properties to see your audience populated with the least lag time.
## Tips and troubleshooting
### Validating events
According to the [Google Analytics documentation](https://developers.google.com/analytics/devguides/collection/protocol/ga4/validating-events?client_type=gtag), the Google Analytics Measurement Protocol for Google Analytics 4 does not return HTTP error codes, even if an event is malformed or missing required parameters. To ensure your events are valid, you should test them against the Measurement Protocol Validation Server before deploying them to production.
You can run syncs using the Measurement Protocol Validation Server, if selected in your Hightouch sync config. Alternatively, before running your sync in Hightouch, use this [tool from Google Analytics](https://ga-dev-tools.google/ga4/event-builder/) to validate events and make sure you're not missing any parameters.
If you don't validate the event and are missing parameters in the [field mapping](#field-mapping) section, the sync will look healthy in Hightouch because the response from GA4 won't contain errors, but you won't see events in GA4.
### Common errors
### Live debugger
### Sync alerts
---
## Google Campaign Manager 360
**URL:** https://hightouch.com/docs/destinations/google-campaign-manager
**Description:** Empower your marketing team to dynamically and intelligently serve and view of all of their ad campaigns across all channels
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
|----------------------------|------------------------------------------------------------|----------------------|----------------------------------------------------------------------------------------------------------------------|
| [**Conversions**](#syncing-conversion-events) | Sync records from your model as new conversion events | Insert | [Conversions docs](https://developers.google.com/doubleclick-advertisers/rest/v4/conversions/batchinsert) |
| [**Conversion Adjustments**](#syncing-conversion-events) | Sync records from your model to existing conversion events | Insert | [Conversion Adjustments docs](https://developers.google.com/doubleclick-advertisers/rest/v4/conversions/batchupdate) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Google Campaign Manager 360
You need to have access to at least one [user profile](https://support.google.com/campaignmanager/answer/6098287?hl=en&sjid=13478745075182123470-NA) and one [advertiser](https://support.google.com/campaignmanager/answer/2829356?hl=en&sjid=13478745075182123470-NA) in Google Marketing Platform to sync data.
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Google Campaign Manager 360** and click **Continue**. You can then authenticate Hightouch to **Google Campaign Manager 360** via OAuth.
For the **Authentication method**, select **Log in to Google Campaign Manager 360** and log into your Google Campaign Manager 360 account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Google Campaign Manager 360 destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Google Campaign Manager 360 destination you want to sync to.
### Syncing conversion events
You can choose to sync new conversion events or update existing ones via the **Conversion Adjustments** sync type.
Then, select the Google Campaign Manager profile to sync conversion data to.
#### Floodlight activities and configuration
Google Campaign Manager requires each conversion to be associated with a [Floodlight activity](https://support.google.com/campaignmanager/answer/3027419?hl=en&sjid=13478745075182123470-NA) and [configuration](https://support.google.com/campaignmanager/answer/2893922).
Hightouch provides a dropdown to select the available activities and configurations based on your Google Campaign Manager profile.
You can either select a static value for all events or select to **use a column** from your model.
#### User advertising identifiers
You can opt to include encrypted user advertising IDs in your conversion event payload.
Hightouch obtains encrypted user advertising IDs from the [%m match macro](https://developers.google.com/doubleclick-advertisers/guides/conversions_overview#using_the_m_match_macro) or [Data Transfer](https://developers.google.com/doubleclick-advertisers/guides/conversions_overview#using_data_transfer).
If you are including, `dclid`, `gclid`, `matchId`, or a mobile device ID in your [field mapping](#field-mapping), you don't need to enable this.
#### Field mapping
Field mapping is where you select which customer information parameters or other event parameters you want to send to the Conversions API.
You can map data from any of your model columns to default and custom Google fields.
The Conversions API requires you to include at least one of the user identifier parameters below in your event payload, so be sure to map one here.
The identifiers options are:
- Mobile device ID
- Google Click ID
- Match ID
- Display Click ID
- Encrypted user ID
- Encrypted user ID candidates
#### Enhanced conversions mappings
The CM360 offline conversion API supports [enhanced conversions](https://developers.google.com/doubleclick-advertisers/guides/conversions_ec) with additional user identifiers. To send enhanced conversions, you can map data from any of your model columns to the following user identifiers:
- Hashed email
- Hashed phone number
- Hashed first name
- Hashed last name
- Hashed street address
- City
- State
- Country code
- Postal code
You should provide **only one** of these user identifiers for matching.
For more information, please check the [Google Campaign Manager 360 documentation](https://developers.google.com/doubleclick-advertisers/rest/v3.5/Conversion).
## Tips and troubleshooting
### Common errors
When uploading offline conversions CM360 enforces a 28 day window, therefore, any conversion timestamp older than 28 days will be rejected with errors like "The timestamp is too old". If you receive this error then you should check to ensure that the timestamp of your conversions is at most 28 days old from the time of submission.
This information, along with a few other FAQs, can be found on the CM360 Conversions FAQ page [here](https://developers.google.com/doubleclick-advertisers/guides/conversions_faq).
### Live debugger
### Sync alerts
---
## Google Cloud Functions
**URL:** https://hightouch.com/docs/destinations/google-cloud-functions
**Description:** Build your own custom destination using a serverless function
**Section:** Destinations
## Overview
This destination invokes a [Google Cloud Function](https://cloud.google.com/functions) whenever data changes in your model. It makes it possible to build your own custom integrations that execute code written in Javascrip, Java, Python, Go, Ruby and PHP. See [how to write HTTP function in Google Cloud Functions](https://cloud.google.com/functions/docs/writing/write-http-functions) for more details.
This destination was designed to be as flexible as possible. You can exercise granular control over function triggers, batching, rate limits, concurrency limits, and even error handling. Together, these features let you integrate Hightouch with any internal tool or third-party API.
Under the hood, the Google Cloud Functions destination gives you Hightouch's powerful sync engine, so you continue to benefit from the security and observability features available in our native SaaS destinations.
Example use cases include:
- Syncing data to web APIs not yet natively supported
- Enriching data using external sources like Clearbit and ZoomInfo
- Transforming and filtering data using code instead of SQL
## Getting started
### Connect to your Google Cloud Functions
When setting up the Google Cloud Functions destination for the first time, you need to fill out the following:
- [**GCP credentials**](/security/gcp) for authenticating you function invocations.
- [**ProjectID**](https://cloud.google.com/resource-manager/docs/creating-managing-projects) of the GCP project where your function lives.
- [**Region**](https://cloud.google.com/resource-manager/docs/creating-managing-projects) for the location of your function.
- The **function name** of the function you want to invoke when sync runs.
Hightouch recommends that you follow [the principle of least privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege) when providing permissions with your service accounts.
This destination only requires a `cloudfunctions.invoker` role for the resource pointing to the function.
## Syncing data
Once you've completed setup for the Google Cloud Functions destination in your Hightouch workspace, the next step is to configure a sync that invokes your function whenever rows are added, changed, or removed in your model.
### Choose your function triggers
Hightouch monitors your data model for added, changed, and/or removed rows. In this step, you specify which of these events should invoke your function.
Suppose you want to use Google Cloud Functions to send a confirmation email whenever a customer places a new order or updates an existing order. Your model might look something like `SELECT * FROM orders`. The Google Cloud Functions should be triggered whenever rows are added—for example, when customers place new orders—or whenever rows change—when orders are updated. Therefore, you would want to enable the **Rows added** and **Rows changed** triggers. When invoking your function, Hightouch passes along metadata about *why* the function was invoked.
All function invocations are synchronous, meaning that Hightouch invokes your Google Cloud Function and waits for its response.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows, such as hydrating a CRM for the first time, may require performing a backfill of all rows during the initial sync. For other use cases, such as sending confirmation emails, you might only want to invoke your function in response to future data changes.
### Configure batch size
By default, Hightouch separately invokes your Google Cloud Function for each added, changed, or removed row.
Certain high-throughput use cases may require batching together multiple rows in the payload. Hightouch supports batches of up to 1000 rows per function invocation.
Batching isn't recommended unless absolutely necessary for performance reasons. Enabling this feature requires your Google Cloud Function to be either idempotent or significantly more fault tolerant in the event of a partial batch failure.
### Configure rate and concurrency limits
In this step, you declare whether function invocations should be throttled.
To determine the ideal limits, you should consider whether your function interacts with any downstream services that have limits of their own.
Most modern web APIs enforce rate limits, which set a maximum allowed number of requests per second, minute, or hour. Occasionally, APIs may also have concurrency limits, which set a maximum allowed number of requests that can be processed simultaneously. Rate limits and concurrency limits both affect overall sync speed.
### Configure error handling
Google Cloud Functions can fail for many reasons. Hightouch's retry logic for this destination doesn't discriminate between invocation errors, for example, missing permissions, runtime errors, for example, due to syntax issues, or function errors, for example, due to uncaught exceptions. Hightouch retries all errors eventually.
In this step, you decide whether errors should be retried immediately or during the next sync run. If you choose to retry immediately, you can specify how many retries should be attempted during the sync run. If all these retries fail, Hightouch retires the request during subsequent sync runs until it succeeds.
If an error is gracefully caught in your function, you can flag it as a rejected row and provide a custom error message using the response format specified in the [error handling section](#configure-error-handling).
You can also include retry logic inside of your Google Cloud Function. After exhausting all retry attempts during a function invocation, respond with an error so that Hightouch knows to reinvoke the function later.
### Invocation payload schema
The payload, also known as the "event document," contains your row data in JSON format, along with additional metadata. When writing your Google Cloud Function, you can assume that all function invocations include a payload following this schema:
```json
{
"operation": XXXXX,
"primary_key_column": XXXXX,
"rows": [
{
XXXXXX
}
],
"metadata": {
"api_version": XXXXX,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
#### Invocation payload properties
##### `operation`
The row operation, also known as the trigger, explains *why* your function was invoked.
Its possible values are these strings: `"add"`, `"change"`, and `"remove"`.
##### `primary_key_column`
This string refers to the column name of the primary key for your Hightouch model.
##### `rows`
This is an array of objects, with each object containing row data from your Hightouch model.
##### `metadata`
This is an object containing three key/value pairs:
- `api_version` (integer): currently set to `1`, this identifier only changes if the payload schema is modified in the future
- `sync_id` (integer): this is the unique identifier for the Hightouch sync configuration associated with the function invocation
- `sync_run_id` (integer): this is the unique identifier for the specific sync run associated with the function invocation
#### Example invocation payload
Suppose a new row is added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `john.doe@example.com` | John | Doe |
The payload for the function invocation would look like this:
```json
{
"operation": "add",
"primary_key_column": "customer_id",
"rows": [
{
"customer_id": "928713",
"email": "john.doe@example.com",
"first_name": "John",
"last_name": "Doe"
}
],
"metadata": {
"api_version": 1,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
Note that `rows` is an **array** containing a single element.
#### Example invocation payload for batching
Suppose three new rows are added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `alice.doe@example.com` | Alice | Doe |
| 283743 | `bob.doe@example.com` | Bob | Doe |
| 162352 | `carol.doe@example.com` | Carol | Doe |
The payload for the function invocation would look like this:
```json
{
"operation": "add",
"primary_key_column": "customer_id",
"rows": [
{
"customer_id": "928713",
"email": "alice.doe@example.com",
"first_name": "Alice",
"last_name": "Doe"
},
{
"customer_id": "283743",
"email": "bob.doe@example.com",
"first_name": "Bob",
"last_name": "Doe"
},
{
"customer_id": "162352",
"email": "carol.doe@example.com",
"first_name": "Carol",
"last_name": "Doe"
}
],
"metadata": {
"api_version": 1,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
When batching is enabled, rows sharing a common operation type—`add`, `change`, `remove`—are batched together. Function invocations always represent exactly one operation type.
### Response payload schema
If your Google Cloud Function fails to process any rows, it can respond with an array of "rejected rows" that encountered errors. Each rejected row should be identified by its primary key/value and may be associated with an optional error message. Hightouch retries these rows.
If there is an error, the response payload looks like this:
```json
{
"errors": [
{
"primary_key_value": XXXXX,
"reason": XXXXX
}
]
}
```
#### Response payload properties
Errors in the `errors` array have these properties:
##### `primary_key_value`
This identifies the rejected row that needs to be retried.
##### `reason` (optional)
This string may represent an error message or any other information that would be helpful for debugging and monitoring sync health.
#### Example success response payload
If all rows are processed successfully, your function responds with an empty payload.
```json
{}
```
#### Example response payload for rejected rows
Suppose three new rows are added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `alice.doe@example.com` | Alice | Doe |
| 283743 | `bob.doe@example.com` | Bob | Doe |
| 162352 | `carol.doe@example.com` | Carol | Doe |
While processing these rows, your Google Cloud Function encounters two errors:
- Alice's email can't be found in the downstream service
- Bob's email already exists in the downstream service
To surface these errors in Hightouch, your function should respond like this:
```json
{
"errors": [
{
"primary_key_value": "928713",
"reason": "Email not found"
},
{
"primary_key_value": "283743",
"reason": "Email already exists"
}
]
}
```
The primary key values (`928713` and `283743`) refer to `customer_id` values. The `customer_id` column was designated the `primary_key_column` in the function invocation payload.
#### Example response payload for other errors
If your function experiences an invocation error, for example, exceeding Google Cloud Functions rate limits, runtime error, for example, due to syntax issues, or function error, for example due to uncaught exceptions, all rows in the batch are automatically marked as rejected rows.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Google Data Manager API
**URL:** https://hightouch.com/docs/destinations/google-data-manager
**Description:** Fuel AI-powered personalization and support advanced measurement use-cases by connecting first-party audience and conversion data to Google Ads, Google Analytics, and Google Marketing Platform through a single unified API.
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| -------------------------------------------------------------------------- | -------------------------------------------------------------------------------- | -------------------- | --------------------------------------------------------------------------------------------------------------- |
| [**Google Ads Audiences**](#syncing-google-ads-audience-members) | Sync records as audience members in Google Data Manager via the Data Manager API | Add | [Audience Members](https://developers.google.com/data-manager/api/get-started/quickstart/send-audience-members) |
| [**Display & Video 360 Audiences**](#syncing-display--video-360-audiences) | Create and update first and third party audiences in Google Display & Video 360 | Add | [Display & Video 360 Audiences](https://developers.google.com/display-video/api/guides/audiences/overview) |
| [**Google Ads Conversion Events**](#syncing-google-ads-events) | Send Online and Offline events to Google Data Manager via the Data Manager API | Insert | [Events](https://developers.google.com/data-manager/api/get-started/quickstart/send-events) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Google Data Manager
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button.
Select **Google Data Manager** and click **Continue**. Select whether you'd like to send data to Google Ads or Display & Video 360.
For Google Ads, Hightouch supports two authentication methods:
| Method | Description |
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Sync data to Google Ads** | OAuth-based authentication. Hightouch creates a partner link between Hightouch and your Google Ads account during the first sync run. Recommended for most destinations. |
| **Existing Partner Link** | A method for accounts that already have a Hightouch partner link. Use this when the OAuth method can't create the partner link for you. |
### Sync data to Google Ads
Select **Sync data to Google Ads**, click **Log in to Google Data Manager**, log in with your Google Account, and click **Allow**.
You should then see a success notification in the Hightouch UI.
Hightouch creates the partner link during the first sync run. Creating the partner link requires the authenticating user to have **Admin** access on the Google Ads account. If the user doesn't have Admin access, the sync fails with a `PERMISSION_DENIED` error. See [PERMISSION_DENIED from Data Manager API](#permission_denied-from-data-manager-api) to verify and fix the access level.
### Existing Partner Link
Use this method if the OAuth method can't create the partner link for you and you already have a Hightouch partner link set up in your Google Ads account.
Unlike the OAuth method, this method doesn't create the partner link for you. Before connecting, create the partner link in your Google Ads account by linking **Hightouch** as a data provider. See Google's [data partner linking](https://developers.google.com/data-manager/api/devguides/concepts/destinations#ads-credentials) documentation for details.
### Enter your account ID
Enter the Google Ads or Display & Video 360 account ID you'd like to use with this destination. You can find this value in Google, at the top-right corner with your account information. This should be a 10 digit value. Remove any dashes that may be present in your account ID.
## Sync configuration
Once you've set up your Google Data Manager destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data.
Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin.
Then, select the relevant model and the Google Data Manager destination you want to sync to.
### Syncing Google Ads audience members
Hightouch lets you create and maintain customer match lists via the Google Data Manager API.
It can take 6 to 48 hours for a list to be populated with members once Hightouch has sent the data to Google.
#### User list type
To begin, select which type of Customer Match User List you'd like to create or sync to. The available options are:
- **CONTACT_INFO**
- **MOBILE_ADVERTISING_ID**
#### Select an existing list or create a new one
You can create a new [Customer match user list](https://support.google.com/google-ads/answer/6276125) or use an existing one.
When creating a new list, you can optionally enter a name.
Otherwise, Hightouch defaults to the name of the associated model.
To use an existing list, enter the list's ID in the following format: `products/{product}/customers/{customer_id}/userLists/{user_list_id}`. Example: `products/GOOGLE_ADS/customers/12345678/userLists/12345678`
If you choose to create a new list, you can't update the
[account](#select-account) to which you're syncing data after the initial sync
run. If you need to change the account, configure a new sync.
#### User identifiers
Select which columns in your model you'd like to send to Google Ads as user identifiers.
For user lists with the Contact Info type, you must select _either_ hashed email addresses or hashed phone numbers. If you choose to sync addresses, all fields of the address are required.
For user lists with the Mobile Advertising ID type, the mobile advertising ID is required.
#### Handling PII and hashing
You can enable PII hashing if you wish for Hightouch to detect whether your data requires normalization and hashing based on the chosen record mapping.
For example, if you select **Email** and the model column with the email data is unhashed, Hightouch hashes the value for you.
By default, Hightouch automatically detects if the following fields require hashing:
- **Email**
- **Phone**
- **Given name**
- **Family name**
#### EU user consent policy
If you are uploading data for users in the European Economic Area ([EEA]()),
you need to ensure you are passing the required consent signals to [Google](https://www.google.com/about/company/user-consent-policy/?sjid=2196795394224792925-NA).
For each sync, you can specify the consent status for these types:
- `ad_user_data`: consent for sending user data to Google for advertising purposes
- `ad_personalization`: consent for ad personalization
Check out Google's support [doc](https://support.google.com/google-ads/answer/14310715?hl=en) to learn more.
### Syncing Display & Video 360 audiences
Hightouch allows you to create and update first and third party audiences in Google Display & Video 360 as customer match user lists. These lists support adding members using mobile device ID or an array of email, phone numbers, and physical addresses.
When creating the audience, make sure the source of the data is properly selected between first or third party sources.
#### User identifiers
Mobile device IDs and other user identifiers cannot be mixed. Also, if providing the user's physical address, the zip code, country code, first and last name must be provided.
Phone numbers must be in [E.164](https://en.wikipedia.org/wiki/E.164) format.
#### User consent
Starting in March 2024, Display & Video 360 introduced the [consent object](https://developers.google.com/display-video/api/guides/audiences/upload-customer-match?sjid=10202302840984618943-NA#set_user_consent) for uploading consent with customer match data. The consent object specifies two distinct types of consent and users must grant both types of consent to use their data for customer match lists. Reference the [DV360 docs](https://support.google.com/displayvideo/answer/14673239?hl=en) for more information.
If you don't set the consent status or if the status is denied, the sync
results in an error.
#### PII data and hashing
Hightouch automatically hashes all the PII data before it's sent to Google. If the data is already hashed, this option can be disabled. Reference the [DV360 docs](https://developers.google.com/display-video/api/reference/rest/v1/firstAndThirdPartyAudiences#ContactInfo) for more information.
Google's Personalized Ads Policy, Customer Match Policy and Platforms program
policies include requirements for advertisers to show sufficiently established
Google Ads or Display & Video 360 account history before accessing certain
features. To enable uploading data to DV360, please fill [this
form](https://support.google.com/displayvideo/answer/9176010) before you
create the sync.
### Syncing Google Ads events
Hightouch supports syncing:
- Online events: Send event data as an additional data source for your tag conversions, to maximize ad interaction signals and strengthen your data and overall performance.
- Offline events: Send event data for [offline conversions](https://support.google.com/google-ads/answer/2998031) or [enhanced conversions for leads](https://support.google.com/google-ads/answer/15713840).
When syncing conversion events, Hightouch treats any records added to [your source](/getting-started/concepts) as new events and sends them to Google Data Manager when your sync runs.
#### Field mapping
Select which columns in your model you'd like to sync as attributes of conversion events.
Google Data Manager requires the following fields for each event:
- Event timestamp (event_timestamp): The time the event occurred.
- Transaction ID (transaction_id): The unique identifier for the event.
- Event source (event_source): The source of the event. Required for offline events. Optional for online events. If specified for an online event, must be WEB.
- Ad identifiers (GCLID, GBRAID, or WBRAID) or user data (email, phone, or address): The event must have either an ad identifier or user data. Send both if you have both for the event.
The accepted values for `event_source` are:
- `EVENT_SOURCE_UNSPECIFIED`
- `WEB`
- `APP`
- `IN_STORE`
- `PHONE`
- `OTHER`
Additional user properties mappings include:
- Customer type
- Accepted values: `CUSTOMER_TYPE_UNSPECIFIED`, `NEW`, `RETURNING`, `REENGAGED`
- Customer value bucket
- Accepted values: `CUSTOMER_VALUE_BUCKET_UNSPECIFIED`, `LOW`, `MEDIUM`, `HIGH`
#### Landing page device info (recommended for online events)
For online events, we recommend collecting and sending information about the device being used (if any) when the event happened. This will improve the likelihood that conversion events can be accurately attributed back to ad interactions, resulting in more accurate reporting and better performance as Google's AI systems learn and improve from each attributed conversion.
To map device information to your conversion events, select `Landing page device info` and map the following fields:
- `userAgent` - user-agent string of the device for the given context.
- `ipAddress` - The IP address of the device for the given context.
Google Ads does **not** support IP address matching for end users in the
European Economic Area (EEA), United Kingdom (UK), or Switzerland (CH). We
recommend adding logic directly in your model to conditionally exclude sharing
IP addresses from users from these regions and ensure that you provide users
with clear and comprehensive information about the data you collect on your
sites, apps, and other properties and get consent where required by law or any
applicable Google policies. See Google's [**About offline conversion
imports**](https://support.google.com/google-ads/answer/2998031) page for more
details.
#### EU user consent policy
If you are uploading data for users in the European Economic Area ([EEA]()),
you need to ensure you are passing the required consent signals to [Google](https://www.google.com/about/company/user-consent-policy/?sjid=2196795394224792925-NA).
For [click conversions](#syncing-click-conversions), [call conversions](#syncing-call-conversions) and [offline store conversions](#syncing-offline-store-conversions), you can specify the Google consent status for advertising purposes, by mapping these fields in the sync configuration:
- `Ad user data consent` (for the `ad_user_data` property)
- `Ad personalization consent` (for the `ad_personalization` property)
## Tips and troubleshooting
### Common errors
#### PERMISSION_DENIED from Data Manager API
You may receive a `403 - {"error":{"code":403,"message":"The caller does not have permission","status":"PERMISSION_DENIED","domain":"datamanager.googleapis.com"}}` error when syncing to Google Ads.
This occurs when Hightouch creates the partner link during a sync run but the authenticating user doesn't have **Admin** access on the Google Ads account. Google Data Manager checks access against the login account, which in manager-account setups is the manager (MCC) account. A user with a lower access level, such as Standard, can sign in successfully but can't create the partner link.
To verify the access level:
1. Go to your Google Ads account.
2. Open **Access and security**.
3. Open the **Users** tab.
4. Turn off the **Show users in full hierarchy** toggle.
5. Confirm the authenticating user is listed directly on the manager (MCC) account.
6. Confirm the user's access level is **Admin**. For more on access levels, see Google's [access levels](https://support.google.com/google-ads/answer/9978556) documentation.
If the user isn't an Admin, upgrade the access level and rerun the sync, or re-authorize the destination with a different user who has Admin access. If you can't grant Admin access, create the partner link manually and switch to the [**Existing Partner Link**](#existing-partner-link) authentication method instead.
### Live debugger
### Sync alerts
---
## Google Drive
**URL:** https://hightouch.com/docs/destinations/google-drive
**Description:** Transfer data at scale from your warehouse to Google Drive
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
| ------------ | ------------------------------------------------------------ | -------------------- |
| Any data set | Sync data from a source to Google Drive as JSON or CSV files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Diff mode creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Google Drive
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Google Drive** and click **Continue**. You can then connect to Google Drive via OAuth.
After authorizing, you will have successfully connected Hightouch to Google Drive.
## Sync configuration
Once you've set up your Google Drive destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Google Drive destination you want to sync to.
### Select file format
Hightouch supports syncing JSON and CSV files to Google Drive.
### Enter folder
Enter the URL of the folder in Google Drive you want to sync to. You can find a folder's URL k by right-clicking on it and then selecting **Get Link**.
### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it. To view different versions of the same file, you can right-click the file in Google Drive and then click **Manage versions**.
If you are using an audience and would like to include the audience name, you will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter '-86400' (`24 hours * 60 minutes * 60 seconds`).
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `company`, `email`, `name`, and `primaryKey` fields. Hightouch exports these fields to new fields in the file and ignores all other columns from your results.
### CSV options
If you're syncing to a CSV file, you have additional configuration options:
- **Delimiter**: Your options are comma (`,`), semicolon (`;`), pipe (`|`), tilde (`~`), and tab
- (Optional) Whether to **include a CSV header row** in the exported files
- (Optional) Whether to **include a [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark)** in the exported files, the BOM is ``
### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Google Cloud Pub/Sub
**URL:** https://hightouch.com/docs/destinations/google-pubsub
**Description:** Connect to your internal microservices via a message queue
**Section:** Destinations
## Overview
Hightouch integrates directly with Google Cloud Pub/Sub to support high-throughput, distributed, or asynchronous workloads, letting you build a custom connector to your internal systems.
## Getting started
### Connect to your Google Cloud account
When setting up the Google Cloud Pub/Sub destination for the first time, you need to enter your **Google Cloud Credentials** to give Hightouch access to your Google Cloud account. Hightouch needs permission to send messages to Google Cloud Pub/Sub on your behalf. Consult [our GCP documentation](/security/gcp) for guidance on how to create a Google Cloud credential.
### Project details
First, you need to enter your **GCP Project ID** and **[Regional API Endpoint](https://cloud.google.com/pubsub/docs/reference/service_apis_overview#list_of_regional_endpoints)**.
### Grant permissions
Hightouch believes in the principle of least privilege. We ask for no more permissions than necessary. For the Google Cloud Pub/Sub destination, Hightouch requires `roles/pubsub.publisher` and `roles/pubsub.viewer`. Ensure you run the command in your Google Cloud Console to grant Hightouch's service account the correct permissions.
## Syncing data
Once you've authenticated your Google Cloud account in Hightouch, entered project details, and granted the correct permissions, you can configure a sync that sends messages whenever rows are added, changed, or removed in your model.
### Configure your events trigger
Hightouch monitors your data model for added, changed, and removed rows. In this step, you specify which of these events should trigger message publishing.
### Choose your topic
In this step, you choose which topics to publish the messages to. Hightouch allows you to sync to existing topics that are already in your Google Cloud Pub/Sub.
### Customize your message
In this step, you tell Hightouch how to build the JSON message data object using data from your model.
This destination offers three methods of composing a JSON object:
- [Using a JSON editor](#use-json-editor)
- [Using one column from a model](#use-one-column-from-model)
- [Using multiple columns from a model](#use-multiple-columns-from-model)
#### Use JSON editor
With the JSON editor, you can compose any JSON object using the [Liquid template language](https://shopify.github.io/liquid/). This is particularly useful for complex message data bodies containing nested objects and arrays, which can sometimes be difficult to model entirely in SQL.
Suppose your data model looks like this:
| full_name | age | email_address | phone_number |
| --------- | --- | ------------------ | ------------ |
| John Doe | 30 | `john@example.com` | +14158675309 |
And you want your message data like this:
```json
{
"name": "John Doe",
"age": 30,
"contact_info": [
{
"type": "email",
"value": "john@example.com"
},
{
"type": "phone",
"value": "+14158675309"
}
]
}
```
Your Liquid template should look like this:
```txt
{
"name": "{{row.full_name}}",
"age": {{row.age}},
"contact_info": [
{
"type": "email",
"value": "{{row.email_address}}"
},
{
"type": "phone",
"value": "{{row.phone_number}}"
}
]
}
```
This makes it so you can reference any column using the syntax `{{row.column_name}}`. You can also use advanced Liquid features to incorporate control flow and loops into your dynamic message data.
When injecting strings into your JSON object, be sure to surround the Liquid
tag in double quotes.
#### Use one column from model
If you're already storing JSON data in your source, or if you have the ability to construct a JSON object using SQL, you can select one column in your model that already contains the full message data.
This setting is commonly used when syncing web events that have already been collected and stored as JSON objects in your database.
#### Use multiple columns from model
For the simplest use cases, Hightouch can construct a JSON object with key/value pairs based on multiple columns in your model.
Suppose your model looks like this:
| email | first_name | last_name |
| ----------------------- | ---------- | --------- |
| `alice.doe@example.com` | Alice | Doe |
| `bob.doe@example.com` | Bob | Doe |
| `carol.doe@example.com` | Carol | Doe |
The field mapping in the screenshot above would generate the following message data for the first row:
```json
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "alice.doe@example.com"
}
```
You can use the field mapper to rename fields. For example, `first_name` can
be mapped to `customer_first_name`.
### Configure optional message properties
Along with your row data in JSON format, you can also optionally include an ordering key to configure the order your topic receives your message and metadata fields as additional attributes.
##### `orderingKey`
This is a string field that ensure subscribers receive messages [in a specific order](https://cloud.google.com/pubsub/docs/ordering). Hightouch automatically tries to cast the value to a string. If we can't cast the value to a string then it is sent as `null`.
##### `attributes`
This is an object containing key/value pairs of custom mapping fields. Values can be text or byte strings. Attributes can be text strings or byte strings. You can have at most 100 attributes per message. Attribute keys should not start with `goog` and should not exceed 256 bytes.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows may require performing a backfill of all rows during the initial sync. For other use cases, you might only want to send messages in response to future data changes.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Google Retail
**URL:** https://hightouch.com/docs/destinations/google-retail
**Description:** Ensure all of your Google Retail listings match your inventory
**Section:** Destinations
## Supported content
| Type | Description | Supported Sync Modes |
|--------------|--------------------------------------------------|----------------------|
| **Products** | Sync records from your model as product listings | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Google Retail
You need to have access to a [merchant account](https://www.google.com/retail/) to sync listings.
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Google Retail** and click **Continue**. You can then authenticate Hightouch to **Google Retail** via OAuth.
For the **Authentication method**, select **Log in to Google Retail** and log into your Google Retail account. Once successful, you will be redirected back to Hightouch to enter your Merchant ID and complete setup.
## Sync configuration
Once you've set up your Google Retail destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Google Retail destination you want to sync to.
### Product category
Each sync must have a product category. This is the category that the products will be listed under in Google Retail.
The product category option can also be specified from a column in your model if your model contains products across categories.
### Field mapping
Field mapping is where you select which customer information parameters or other event parameters you want to send to the Content API.
You can map data from any of your model columns to default and custom Google fields.
Go to the [Product Specification](https://support.google.com/merchants/answer/7052112?visit_id=638422392791129370-3944862796&rd=1) to see the mappings available for all Google Retail products.
#### Vehicle mapping
Cars, Trucks & Vans (916) have an additional collection of fields available to map data to. Read more about them [here](https://support.google.com/merchants/answer/11190480?sjid=6359241266768509981-EU).
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Google Search Ads 360
**URL:** https://hightouch.com/docs/destinations/google-search-ads-360
**Description:** Search Ads 360 helps you respond to an ever-changing market in real time and at scale.
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
|-----------------|----------------------------------------------------------------|----------------------|---------------------------------------------------------------------------------------------|
| [**Conversions**](#syncing-conversions) | Sync data from any source to Google Search Ads 360 conversions | Insert | [Conversions docs](https://developers.google.com/search-ads/v2/reference/conversion/insert) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Google Search Ads 360
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Google Search Ads 360** and click **Continue**. You can then authenticate Hightouch to **Google Search Ads 360**.
For the **Authentication method**, select **Log in to Google Search Ads 360** and log into your Google Search Ads 360 account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Google Search Ads 360 destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Google Search Ads 360 destination you want to sync to.
### Syncing conversions
Sync data from any source to Google Search Ads 360 conversions.
#### Record matching
You can match rows from your model to conversion in Google Search Ads 360 on any column in your model and any field in Google Search Ads 360. Ensure the data types of the model column and Google Search Ads 360 field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
The `segmentationType` property is sent with all conversions with the value 'FLOODLIGHT' which is the only acceptable value for this field.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Google Sheets - User Account
**URL:** https://hightouch.com/docs/destinations/google-sheets
**Description:** Sync warehouse data to spreadsheets used by sales, ops, and marketing teams
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| --------------------------------- | -------------------------------------------------------- | ------------------------ | -------------------------------------------------------------------------------------------------------------------- |
| [**Sheets**](#sync-configuration) | Sync data from any source to a Google Sheets spreadsheet | Mirror, Snapshot, Append | [Update spreadsheet values ↗](https://developers.google.com/sheets/api/reference/rest/v4/spreadsheets.values/update) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Google Sheets
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button.
Hightouch provides two Google Sheets options:
- **Google Sheets - User Account**: Use this option if you want to authenticate access to Hightouch with a particular user's login. This gives Hightouch access to any Google Sheet the user has Editor access to. This option uses OAuth for authentication.
- **Google Sheets - Service account (Recommended)**: This option provides fine-grained control over access to your data.
Using a service account is best practice for a few reasons:
- You can grant service accounts access to specific sheets. This means that only the necessary permissions are granted to the service account, instead of allowing access to all files belonging to a user. This reduces the risk of unauthorized access to sensitive data or functionality.
- Service accounts are designed to represent applications or services, not individuals. You don't have to worry about disruptions to your syncs if the user who authenticated has their permissions changed or if they leave the organization.
### Authenticate with a user account
Select the **Google Sheets - User Account** destination option and click **Continue**.
Then select to **Log in to Google Sheets**, select your Google account, and click **Allow** to the requested permissions.
Hightouch uses the same OAuth mechanism for all Google destinations. If you've
set up other Google destinations, such as Google Ads, Campaign Manager, or
Display & Video 360, they appear here.
After authorizing, Hightouch connects to Google Sheets.
Click **Continue** and complete setup by giving the destination a descriptive name.
### Authenticate with a service account
Select the **Google Sheets \(Service Account\)** destination option and click **Continue**.
Then, click **Generate a service account**.
Hightouch then generates a **Google Service Account Email**.
Click **Continue** and complete setup by giving the destination a descriptive name.
To give Hightouch access to a particular spreadsheet, share the spreadsheet with the service account email with **Editor** permissions.
You can always access the service account email from the destination's overview page, should you need to share additional sheets.
## Sync configuration
Once you've set up your Google Sheets destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Google Sheets destination you want to sync to.
The Google Sheets destination allows you to **Mirror** your data to a spreadsheet or create a new sheet on every sync with **Snapshot** mode.
You can also add data to an existing sheet with **Append** mode. In this mode, rows added to your model are added to the sheet. Rows changed and removed in your model are ignored. This may result in faster syncs.
**Mirror** mode replaces the entire selected sheet's contents with the rows
from your model. Ensure that your sheet doesn't contain data that shouldn't be
overwritten if you select this mode.
### Sheet selection
You can choose to select the Google Sheets file you want to update either by its **name** using the sheets picker (recommended) or inputting the Google Sheet's **ID**.
You can find your spreadsheet's ID in its URL: `https://docs.google.com/spreadsheets/d//edit#gid=0`
If you don't see your spreadsheet's name in the dropdown, make sure you authorized the right account or invited the Hightouch service account email to your spreadsheet,
depending on your [setup mode](#connect-to-google-sheets).
If you updated a sheet's permissions, click the refresh button to see newly authorized sheets.
Then, select the sheet within the spreadsheet that you would like to sync to.
Colons (`:`) in spreadsheet and sheet titles can cause errors. Ensure your
spreadsheet and sheet titles don't include any.
Avoid setting up multiple syncs to the same sheet, as this can corrupt your
data.
### Data format
Google Sheets accepts inputted data in two ways:
- **Raw** Won't parse the values the user has entered and stores values as-is. Keep in mind that Hightouch does some data casting and may send numbers and dates as strings. This process can add a leading apostrophe to values in Google Sheets cells.
- **User entered** Parses values as if the user typed them into the UI. Numbers stay as numbers, but strings may convert to numbers, dates, etc. following the same rules that apply when entering text into a cell via the Google Sheets UI.
If you want to sync numbers, dates, or other non-string values in their
original data type and not converted to strings, use **User entered** mode.
If you are inserting null values using the **Raw** data format, these won't
register as blank fields though they may appear to be. Keep this in mind if
you are using formulas that are looking for null values. If any of your
formulas rely on blank fields, it's best to use **User entered** as the input
mode.
For more information, read the [Google documentation](https://developers.google.com/sheets/api/reference/rest/v4/ValueInputOption).
### Enable new sheet creation
Google Sheets has a limit of 10 million cells in a sheet.
If your sync contains more data than this, you can enable Hightouch to create new sheets with the additional data.
For each additional 10 million cells, Hightouch creates a new sheet with the original sheet's name suffixed with an underscore and a number.
For example, "sheetname_1" for the first additional 10 million cells, "sheetname_2" for the second 10 million cells, etc.
Ensure these names don't conflict with existing sheets' names to avoid them being overwritten.
To use this configuration option, ensure the [user you authenticated with](#connect-to-google-sheets) has access to the spreadsheet's folder, not just the spreadsheet itself.
Hightouch released enabling new sheet creation on June 22, 2023. If you set up
your Google Sheets destination before this date and want to use this option,
you need to [reauthorize your connection](#connect-to-google-sheets).
### Data range selection
Hightouch supports selecting a manual range in your Google Sheets to write data to.
This allows you to preserve formulas, notes, or other data in your spreadsheets' margins.
To use a manual range enter a range into the **Custom Cell Range** input in the format `A1:Z1000`.
If you leave this field blank, Hightouch mirrors data to the entire spreadsheet, dependent on your model's query results.
## Tips and troubleshooting
For Hightouch platform error codes related to Google Sheets, see [Error codes:
Google Sheets](/developer-tools/error-codes#google-sheets).
### Rate limits
You may receive an error from Google if you exceed your quota for write requests during a sync: `Quota exceeded for quota metric`.
For more information, read the Google [usage limit documentation](https://developers.google.com/sheets/api/limits).
To avoid this error:
- stagger or schedule your syncs to Google Sheets to [run in sequence](/syncs/schedule-sync-with-sequences) to avoid simultaneous write requests between syncs
- set up multiple Google Sheets destinations in your workspace so as to reduce the number of syncs that use the same destination
- set up [Google Sheets (Service Account)](#authenticate-with-a-service-account) destinations rather than [authenticating with a user account](#authenticate-with-a-user-account)
- reach out to Google to request a [quota increase](https://developers.google.com/sheets/api/limits#increase)
### Find sheets originating from an Excel file upload
You may have trouble selecting that Google Sheet that was originally uploaded to Google as an Excel file.
If you are the owner of the file, but can't find the spreadsheet in the [sheet selection dropdown](#sheet-selection), confirm that it's not an `.XLS` or `.XLSX` file.
You can see the file type of a file next to its name in the Google Sheets UI.
Hightouch can only update Google Sheets files, not `.XLS` or `.XLSX` files. To convert a file to a Google Sheets file, go to **File → Save as Google Sheets**.
After converting the file to a Google sheet, it becomes available in the dropdown selector.
You may need to click the refresh button next to the dropdown selector for
spreadsheets after you've converted a sheet, or updated its access.
If you want to work with Excel files, consider using the [Microsoft Excel destination](/destinations/microsoft-excel).
### Resolve "This app is blocked" errors
If you see a "This app is blocked" error when trying to authorize Hightouch with your Google account, your Google Workspace admin may need to re-approve Hightouch's access. This can happen after Google API verification policy changes.
To resolve the error:
1. Go to your [Google Account](https://myaccount.google.com/) → **Data & Privacy** → **Third-party apps & services**.
2. Find **Hightouch** in the list and select **See details**.
3. Select **Remove access** to revoke the existing authorization.
4. Return to Hightouch and re-authorize the Google Sheets destination.
Removing access disconnects all existing Google integrations that use this
account. Any syncs using these integrations stop running until you
re-authorize. Before removing access, confirm which syncs rely on this Google
account.
This issue can affect any Google destination that uses OAuth, including [Google Drive](/destinations/google-drive), [Google Ads](/destinations/google), and [Google Analytics](/destinations/google-analytics).
### Validate your setup
After your first sync completes, open the target spreadsheet in Google Sheets and confirm:
- The expected columns appear in the correct order.
- Row counts roughly match the number of records in your model.
- If using **Mirror** mode, verify that previous sheet contents were replaced as expected.
- If using **Snapshot** mode, confirm a new sheet was created with the correct data.
### Common errors
| Error message | Description |
| :-------------------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Action would increase the number of cells in the workbook above the limit of 10000000 cells` | Spreadsheet exceeds the 10 million cell limit. [More detail.](#action-would-increase-the-number-of-cells-in-the-workbook-above-the-limit-of-10000000-cells) |
| `Cannot read properties of null (reading 'properties')` | The selected sheet was renamed or deleted. [More detail.](#cannot-read-properties-of-null-reading-properties) |
| `Internal error encountered` | Charts or pivot tables in the spreadsheet cause timeouts. [More detail.](#internal-error-encountered) |
| `Invalid values: list_value` | Object or array columns are not supported. [More detail.](#invalid-values-list_value) |
| `Range {...} exceeds grid limits` | The sheet or range is protected. [More detail.](#range--exceeds-grid-limits) |
| `Requested writing within range...but tried writing to column` | Sheet name contains a colon. [More detail.](#requested-writing-within-rangebut-tried-writing-to-column) |
| `The caller does not have permission` | The sheet is not shared with the authenticated account. [More detail.](#the-caller-does-not-have-permission) |
| `Unable to parse range` | The selected sheet was renamed or deleted. [More detail.](#unable-to-parse-range) |
| Permission errors using a sheet ID | Sheet must be selected in the picker before using its ID. [More detail.](#permission-errors-using-a-sheet-id) |
#### Action would increase the number of cells in the workbook above the limit of 10,000,000 cells
```
Action would increase the number of cells in the workbook above the limit of 10000000 cells
```
**Cause:** Your model's data exceeds the Google Sheets 10 million cell limit for a single spreadsheet.
**Solution:** Enable Hightouch to [create new sheets](#enable-new-sheet-creation). When enabled, Hightouch automatically creates additional sheets if your data surpasses the limit.
#### Cannot read properties of null (reading 'properties')
```
Cannot read properties of null (reading 'properties')
```
**Cause:** The sheet within the spreadsheet selected in the sync configuration was renamed or deleted in Google Sheets.
**Solution:** [Select a new sheet](#sheet-selection) from the dropdown menu in your sync configuration.
#### Internal error encountered
```
Internal error encountered
```
**Cause:** The spreadsheet contains sheets with charts or pivot tables, which can cause Google API timeouts.
**Solution:** Move the data you want Hightouch to update to a separate spreadsheet. Reference this data from your original spreadsheet using a formula like [`IMPORTRANGE`](https://support.google.com/docs/answer/3093340?hl=en).
#### Invalid values: list_value
```
Invalid values [1][2]: list_value
```
**Cause:** The Google Sheets destination doesn't support syncing object or array type columns.
**Solution:** Change the type of any object or array columns to **String** [in your model configuration](/models/data-types-casting#casting-from-your-model-configuration).
#### Range \{...\} exceeds grid limits
```
Range {...} exceeds grid limits
```
**Cause:** The [selected sheet](#sheet-selection) or a data range within it is [protected](https://support.google.com/docs/answer/1218656).
**Solution:** Remove the protection from the sheet or range in Google Sheets. See [Google's documentation](https://support.google.com/docs/answer/1218656) for details.
#### Requested writing within range...but tried writing to column
```
Requested writing within range ['Example:Sheet'!A1], but tried writing to column [B]
```
**Cause:** The sheet name contains a colon, which Google Sheets can't parse in range references.
**Solution:** Replace colons in the sheet name with an underscore or remove them entirely.
#### The caller does not have permission
```
The caller does not have permission
```
**Cause:** The sheet is private and not shared with the authenticated account.
**Solution:** Share the sheet with the [user or service account you used to set up your Google Sheets destination](#connect-to-google-sheets).
#### Unable to parse range
```
Unable to parse range
```
**Cause:** The sheet selected in the sync configuration was renamed or deleted.
**Solution:** Select an existing sheet in the [sheet selection](#sheet-selection) dropdown.
#### Permission errors using a sheet ID
```
Permission denied when writing to sheet by ID
```
**Cause:** Google Sheets requires the sheet to be selected through the picker before it can be referenced by ID.
**Solution:** Select the sheet in the sheets picker first, then use its ID.
### Live debugger
### Sync alerts
---
## Great Question
**URL:** https://hightouch.com/docs/destinations/great-question
**Description:** Maintain your candidates list to reach the right audience for your Great Question studies.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
---------------|--------------------------------------------------------|----------------------|-------------------------------------------------------------------------
**Candidates** | Sync data from any source to Great Question candidates | Upsert | [Candidates docs](https://developer.greatquestion.co/reference/overview)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Great Question
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Great Question** and click **Continue**. You can then authenticate Hightouch to **Great Question**.
Enter the following fields into Hightouch:
- **Base URL**: If left empty, the default base URL (`https://api.greatquestion.co/`) will be used.
- **API key** (Required): Refer to the [Great Question docs](https://developer.greatquestion.co/reference/authentication-1) to generate your API key.
## Sync configuration
Once you've set up your Great Question destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Great Question destination you want to sync to.
### Syncing candidates
Sync data from any source to Great Question candidates.
#### Record matching
You can match rows from your model to candidates in Great Question on any column in your model and any field in Great Question. Ensure the data types of the model column and Great Question field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync candidate fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom candidate fields.
If you send data for a custom field that doesn't exist, Hightouch adds the field.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|----------------------------------------------------------------
**Do nothing** | Keep the candidate in Great Question with all its synced fields
**Delete** | Delete the synced candidates from Great Question
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Heap
**URL:** https://hightouch.com/docs/destinations/heap
**Description:** Sync your most recently captured event data to Heap to both measure and dramatically improve your digital experience
**Section:** Destinations
## Setup
Upon selecting Heap as your destination you will be prompted to
input your Heap Environment ID, which correspond to the specific project environment
you wish to sync your data to.
You can locate the appropriate Environment ID within your Heap account by navigating to
_Account > Manage > Projects_ and clicking on your desired project:
## Heap sync types
Hightouch supports syncing the following Heap sync types:
- `Events`
- `Objects`:
- `User Properties`
- `Account Properties`
## Syncing objects
### Object sync modes
- **Upsert**: pushes new objects to Heap _and_ updates fields that change in your warehouse.
### Record matching
Records can be matched from your source to your Userflow workspace by the `identity` (typically a unique user property)
or `account_id`, depending on the object you are syncing.
### A note on syncing account properties
To sync account properties you must first have users synced into your Heap workspace.
The reason for this is that to sync account properties you must first configure Account IDs.
This may be done by navigating to _Account > Manage > Features_ and reading the following:
> _"To analyze accounts in Heap, you need to specify a user property that can be used to group users into accounts (for example, Account ID)._"
## Syncing events
When syncing events, Hightouch treats any records added to [your source](/getting-started/concepts) as new events and sends them to Heap when your sync runs.
### Event sync modes
- **Insert**: _only_ pushes new events to Heap.
### Event name
To specify the name of the event you wish to sync to Heap you have the option of either
inputting the name of a particular event or using a column from your model.
### Timestamp
You may also specify the timestamp of the event in your source. However, be aware that this value
may be overridden by its respective mapping and is completely optional. If you choose not to explicitly specify
a timestamp then Heap will use the time the event arrives to the server.
### Identity
The identity of an event is a required field and is typically a unique user property. If the specified identity
doesn't exist, a new user will be created with that identity in Heap.
## Field mapping (events and objects)
You may also sync columns from your source to Heap as event, user, or account `properties`.
Any property previously created will be updated and any new properties will be created.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Help Scout
**URL:** https://hightouch.com/docs/destinations/help-scout
**Description:** Help Scout is a dedicated customer support platform built for growing teams to deliver best-in-class customer service.
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes | API Reference |
|--|--|--|--|
|**Customers**|Sync data from any source to Help Scout customers|Insert, Upsert, Update| [Customer docs](https://developer.helpscout.com/mailbox-api/endpoints/customers/list/)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Help Scout
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Help Scout** and click **Continue**. You can then authenticate Hightouch to Help Scout via OAuth.
Click **Log in to Help Scout**, log in to your Help Scout account, and then click **Authorize**.
Once you've finished, Hightouch displays an **Authorization successful** message. To finish connecting, give your Help Scout destination a descriptive name.
## Sync configuration
Once you've connected your Help Scout destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Help Scout destination you want to sync to.
### Syncing customers
Hightouch supports creating new customers and updating existing ones.
#### Record matching
To match rows from your model to customers in Help Scout, you need to select a model column and corresponding Help Scout field. You can match on any of the following Help Scout fields:
- Email
- Customer ID (only with update mode)
#### Field mapping
You can sync any model columns to both native and custom Help Scout customer fields.
In insert mode, you can add new custom fields to the customer. Update and upsert mode only allow replacing existing fields on the customer. In other words, the fields you sync to must already exist in Help Scout.
For information on creating or editing customer fields, please refer to [Help Scout's docs](https://docs.helpscout.com/article/1385-customer-properties#create).
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| ----------------------------- | ----------------------------------------------------------------- |
| **Do nothing** | Keep the customer record in Help Scout with all its synced fields |
| **Delete Destination Record** | Delete the record from the Help Scout completely |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## HTTP Request
**URL:** https://hightouch.com/docs/destinations/http-request
**Description:** Connect to any internal system, third-party API, or other web service.
**Section:** Destinations
## Overview
We designed the HTTP Request destination to be as flexible as possible. We want our customers to build their own integrations with third-party APIs, internal tools, and other services not yet supported in our growing catalog of native destinations.
This destination empowers you to integrate Hightouch with almost any API without writing code. Using our visual interface, you can exercise granular control over HTTP endpoints, request payloads, rate limits, concurrency limits, error handling, and more.
Under the hood, the HTTP Request destination leverages Hightouch's powerful sync engine, so you continue to benefit from the security and observability features available in all our native destinations.
This article provides a technical overview of how the HTTP Request destination works. Skip to the [guided tutorial](#tutorial) showing how to create a custom integration with Algolia's REST API if you want to follow along with an example.
It's strictly prohibited to use the HTTP request destination to replicate
functionality already supported by a native destination.
## Setup
Hightouch queries your data source and monitors the query results for added rows, changed rows, and/or removed rows. When configuring the HTTP Request destination, you specify which of these events should trigger an HTTP request. For each trigger, you then specify an endpoint, payload, and other parameters.
To help Hightouch play nicely with other services, you can also specify constraints such as rate limits, concurrency limits, and error handling logic. You can configure all this in the Hightouch app—no code required.
Let's walk through the process to see how it works.
## Create a new destination
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **HTTP Request** and click **Continue**.
Next, the UI prompts you to enter a **base URL**, **HTTP headers**, and **certificate**. Only the base URL is required.
You'll configure specific endpoints, payloads, rate limits, and other logic during your [sync configuration](#sync-configuration).
That means you can create one HTTP destination per service and multiple sync configurations for different endpoints you want to make requests to.
### Add base URL
The **base URL** is the static part of the API endpoint you plan on making requests to.
For example, if you plan on using the [Algolia REST API](https://www.algolia.com/doc/rest-api/search/), the base URL would be `https://{Application-ID}-dsn.algolia.net`.
During [sync configuration](#sync-configuration), you provide additional path details and query parameters for the specific operations you want to perform.
### Add HTTP headers
You can optionally provide headers to include in each outbound HTTP request. These headers often include access tokens used for authentication.
To enter a header, enter its key name in the input. If you want a header property to be obscured in the UI and encrypted in storage, select the **Secret** checkbox. Click **Add key**, then enter the key's value in the input that appears to the right. Repeat this for as many headers as you want to add.
If you need to fetch a new token during each sync, consider using
[OAuth](#oauth) for authentication. You can also consider using one of our
serverless function destinations, for example, [AWS
Lambda](/destinations/aws-lambda).
### Add certificates
You can optionally add a custom CA, SSL, or TLS certificate.
Enable the relevant toggle and upload your certificate files.
Certificates can have a maximum size of 10 MB.
### OAuth
We also support OAuth authentication. OAuth implementations can vary between providers, so we recommend using a tool like [Postman](https://www.postman.com/) to test the token refresh process end-to-end. Once you've confirmed that you're able to fetch access tokens and authorize requests using [Postman](https://www.postman.com/), you can copy your [Postman](https://www.postman.com/) configuration into Hightouch verbatim.
Hightouch supports the following grant types:
- **[Authorization code](https://oauth.net/2/grant-types/authorization-code/)**
- **[Client credentials](https://oauth.net/2/grant-types/client-credentials/)**
- **[Password credentials](https://oauth.net/2/grant-types/password/)**
When setting up OAuth, you'll need to provide the following information:
- **Authorization header prefix (optional):** Specify the prefix for the Authorization header (for example, "Bearer").
- **Auth URL:** Required for authorization code grant type. The URL used to get the authorization code.
- **Access token URL:** The URL used to get an access token.
- **Refresh token URL (optional):** If different from `Access token URL`, provide the `Refresh token URL`.
- **Client ID:** The client identifier issued during the registration process.
- **Client Secret:** The client secret issued during the registration process.
- **Scopes (optional):** If your OAuth provider requires specific scopes.
- **Client authentication:** Choose to include the credentials as a header or in the request body.
To complete the OAuth setup, make sure to add the following `Callback URL` to your OAuth app: `https://api.hightouch.com/api/webhookV2/oauth/callback`.
#### Advanced configurations
During the OAuth flow, multiple requests are made to retrieve or refresh access tokens. Configure the following
options to include additional headers, URL parameters, or values in the request body.
- **Authorization request URL parameters**: URL parameters sent in the **Auth URL** for the authorization code grant type.
- **Token request**: Additional parameters and values sent with the request to the **Access token URL**.
- **Refresh request**: Additional parameters and values sent with the refresh token request to the **Refresh token URL** or
**Access token URL** if no refresh URL is provided.
## Sync configuration
Once you've created your HTTP Request destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the HTTP Request destination you previously set up.
### Choose request triggers
In this first step, you tell Hightouch when to trigger HTTP requests. You then configure each trigger separately. This lets your integration have different behaviors depending on whether a row is added, changed, and/or removed. Most APIs use different endpoints and HTTP methods for creating, updating, and deleting resources.
For example, suppose you're syncing customer data to an email marketing tool. You might want to create a contact when a row is added, update an existing contact when a row changes, and delete an existing contact when a row is removed. Having separate configuration for each trigger offers the flexibility required to support this use case.
A valid sync configuration requires at least one trigger.
The number of operations displayed on the [run details
page](/syncs/debugger#view-run-details) only counts the operations for the
request triggers you configured. For example, if you only configured the
**Rows added** trigger, the run details page doesn't display information
related to changed or removed rows.
### Batch configuration
You can send a single row or batch of multiple rows in each HTTP request.
To enable batching, select **A batch of multiple rows** in your sync configuration. The default batch size is 100, but you can enter a different value if needed.
When using the [JSON editor](#use-json-editor), make sure to correctly
[configure your payload](#batching-with-the-json-editor) for batch requests.
### Specify webhook endpoint
In this step, you tell Hightouch how to send HTTP requests to your internal system, third-party API, or other web service.
Hightouch supports `GET`, `POST`, `PUT`, `PATCH`, and `DELETE` requests to any endpoint.
When specifying the endpoint URL, you may either provide a static URL or a dynamic, templated URL.
Recall that you provided the [base URL](#add-base-url) in the initial destination setup, so you only need to provide the specific endpoint path.
If you provide a static URL, Hightouch uses that URL for all HTTP requests triggered by the sync. For example, Stripe's REST API uses a single, static URL for creating new customer objects: `POST https://api.stripe.com/v1/customers`.
If you provide a dynamic URL, Hightouch generates a different URL for each HTTP request by incorporating data from your model. For example, Stripe's REST API requires you to reference an ID when deleting specific objects. You could incorporate the `objectID` into the URL template like this: `DELETE https://api.stripe.com/v1/customers/{{customer_id}}`.
Hightouch uses the [Liquid templating language](https://shopify.github.io/liquid/) to generate dynamic content.
#### How to use Liquid templates
The [Liquid templating language](https://shopify.github.io/liquid/) supports variable injection, control flow, iteration, filters, and more. Details are available in the [Liquid docs](https://shopify.github.io/liquid/).
To insert a value from your model, use `{{row.column_name}}`, where `column_name` references any column in your model. For example, `{{row.first_name}}` or `{{row.email_address}}`.
To change the output of a Liquid object, you can append a filter with a single pipe (`|`):
`{{ row.column_name | filter }}`.
For example, let's say the value of the `first_name` column in a row is `"Alice"`. The Liquid template `{{ row.first_name | upcase }}` would generate `{{ "Alice" | upcase }}` → `ALICE`.
You can also chain multiple filters together like this: `{{ row.product_id | remove: "0" | prepend: "SKU_"}}` → `{{ "000123" | remove: "0" | prepend: "SKU_"}}` → `SKU_123`.
### Customize request payload
In this step, you tell Hightouch how to build the request payload using data from your model. Hightouch supports JSON and XML payloads, as well as URL encoded forms. You can also send an empty body, if needed.
Most modern web APIs use JSON payloads. This destination offers three methods of composing a JSON request body:
- [Using a JSON editor](#use-json-editor)
- [Using one column from a model](#use-one-column-from-model)
- [Using multiple columns from a model](#use-multiple-columns-from-model)
#### Use JSON editor
With the JSON editor, you can compose any JSON object using the [Liquid template language](https://shopify.github.io/liquid/). This is particularly useful for complex payloads containing nested objects and arrays, which can sometimes be difficult to model entirely in SQL.
Suppose your data model looks like this:
| `full_name` | `age` | `email_address` | `phone_number` |
| ----------- | ----- | ------------------ | -------------- |
| John Doe | 30 | `john@example.com` | +14158675309 |
And you want your HTTP request to have a payload like this:
```json
{
"name": "John Doe",
"age": 30,
"contact_info": [
{
"type": "email",
"value": "john@example.com"
},
{
"type": "phone",
"value": "+14158675309"
}
]
}
```
The Liquid template you enter under **Define JSON payload** should look like this:
```txt
{
"name": "{{row.full_name}}",
"age": {{row.age}},
"contact_info": [
{
"type": "email",
"value": "{{row.email_address}}"
},
{
"type": "phone",
"value": "{{row.phone_number}}"
}
]
}
```
You can also use advanced Liquid features to incorporate control flow and loops into your dynamic payloads.
When injecting strings into your JSON request body, be sure to surround the
Liquid tag in double quotes.
#### Batching with the JSON editor
When batching is enabled, the context is changed from a single record (`row`) to multiple records (`rows`).
With multiple rows, you likely want to use [loops](https://shopify.github.io/liquid/tags/iteration/) to build your payload.
Suppose your data model looks like this:
| `full_name` | `age` | `email_address` | `phone_number` |
| ----------- | ----- | ------------------ | -------------- |
| John Doe | 30 | `john@example.com` | +14158675309 |
| Jane Doe | 29 | `jane@example.com` | +14158674319 |
And you want your HTTP request to have a payload like this:
```json
{
"users": [
{
"name": "John Doe",
"age": 30,
"contact_info": [
{
"type": "email",
"value": "john@example.com"
},
{
"type": "phone",
"value": "+14158675309"
}
]
},
{
"name": "Jane Doe",
"age": 29,
"contact_info": [
{
"type": "email",
"value": "jane@example.com"
},
{
"type": "phone",
"value": "+14158674319"
}
]
}
]
}
```
The Liquid template you enter under **Define JSON payload** should look like this:
```txt
{
"users": [
{% for row in rows %}
{
"name": "{{row.full_name}}",
"age": {{row.age}},
"contact_info": [
{
"type": "email",
"value": "{{row.email_address}}"
},
{
"type": "phone",
"value": "{{row.phone_number}}"
}
]
},
{% endfor %}
]
}
```
Don't worry about unnecessary trailing commas when evaluating loops in your
payload template. Hightouch automatically strips trailing commas to conform
with the JSON specification.
#### Using Hightouch metadata inside a Liquid template
In addition to using the `{{row.column_name}}` syntax to access row-level data, you can also tap into metadata about your Hightouch syncs and models.
This metadata is provided via the `context` variable. The following table lists the available metadata options.
| Variable | Description | Example |
| ------------------------- | ------------------------------------------------------------ | ----------------- |
| `{{context.model_id}}` | The ID of the model (or audience) associated with the sync | `12345` |
| `{{context.model_name}}` | The name of the model (or audience) associated with the sync | `"VIP Customers"` |
| `{{context.sync_id}}` | The ID of the sync | `67890` |
| `{{context.sync_run_id}}` | The ID of the sync run during which the request is made | `1234567890` |
#### Use one column from model
If you're already storing JSON data in your source, or if you have the ability to construct a JSON object using SQL, you can select one column in your model that already contains the full request payload.
This setting is commonly used when syncing web events that have already been collected and stored as JSON objects in your database.
#### Batching with one column from model
When batching is enabled, the payload is an array where each element is a row value from the selected column. For example, if you select a column that includes a JSON object as each row value, the batch payload would be an array of those JSON objects.
#### Use multiple columns from model
For the simplest use cases, Hightouch can construct a JSON object with key/value pairs based on multiple columns in your model.
Suppose your model looks like this:
| `email` | `first_name` | `last_name` |
| ----------------------- | ------------ | ----------- |
| `alice.doe@example.com` | Alice | Doe |
| `bob.doe@example.com` | Bob | Doe |
| `carol.doe@example.com` | Carol | Doe |
The field mapping in the preceding screenshot would generate the following payload for the first row:
```json
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "alice.doe@example.com"
}
```
You can use the [field mapper](/syncs/mapping-data) to rename fields. For
example, you can map `first_name` to `customer_first_name`. The field mapper
offers additional capabilities, such as [inline
mapping](/syncs/mapping-data#inline-mapping) to create objects and arrays.
#### Batching with multiple columns from model
When batching is enabled, the payload is sent as an array.
Suppose your model looks like this:
| `email` | `first_name` | `last_name` |
| ----------------------- | ------------ | ----------- |
| `alice.doe@example.com` | Alice | Doe |
| `bob.doe@example.com` | Bob | Doe |
| `carol.doe@example.com` | Carol | Doe |
The field mapping in the preceding screenshot would generate the following payload for the first batch:
```json
[
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "alice.doe@example.com"
},
{
"customer_first_name": "Bob",
"customer_last_name": "Doe",
"customer_email": "bob.doe@example.com"
},
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "carol.doe@example.com"
}
]
```
### Configure rate limiting and concurrency
In this step, you configure rate limits and concurrency. In other words, you tell Hightouch how to avoid overwhelming your HTTP endpoint with requests.
Most modern web APIs enforce rate limits, which set a maximum allowed number of requests per second, minute, or hour. Occasionally, APIs may also have concurrency limits, which set a maximum allowed number of requests that can be processed simultaneously.
Hightouch defaults to 1000 requests per second or 100 concurrent requests, whichever limit is reached first. You can override these defaults to meet the requirements of your web service. Keep in mind that rate limits and concurrency limits both affect sync speed.
To ensure that Hightouch never exceeds your chosen limits, we apply a small offset to the values provided in the sync configuration form.
Hightouch waits out the entire rate limit when a sync finishes, to make sure the next run has the full number of requests and doesn't violate the rate limit configuration.
For example, if you set the rate limit window to **Requests per hour**, a sync run waits one full hour before triggering a new run.
If you want to avoid this behavior, you can set the window to **Requests per second** or **Requests per minute**.
Concurrency and rate limit controls aren't shared between syncs and don't
persist across sync runs.
### Configure error handling
In this step, you tell Hightouch how to handle errors, such as HTTP request timeouts and error responses.
These errors will be retried indefinitely until they succeed. You have a choice between retrying immediately or waiting until the next sync run. If you elect to retry immediately, you can specify how many retries should be attempted during the sync run. If all of these retries fail, the request will be retried during subsequent sync runs until it succeeds. Setting up [alerts for row and sync errors](/syncs/alerting) is always recommended.
Any HTTP response with a 400- or 500-level status code is considered an error.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows, such as hydrating a CRM for the first time, may require performing a **backfill of all rows** during the initial sync. For other use cases, such as sending product notifications, you may want to **skip existing rows** and only make HTTP requests for future data changes.
## Limitations
This destination can't perform complex operations that involve lookups or multiple requests chained together.
If you need to perform such complex operations, consider using one of our serverless function destinations: [AWS Lambda](/destinations/aws-lambda), [Google Cloud Functions](/destinations/google-cloud-functions), or [Azure Functions](/destinations/azure-functions). You could also build your own integration using our [Embedded Destination framework](/destinations/custom).
## Tutorial
This guided tutorial uses the HTTP Request destination to integrate Hightouch with [Algolia's REST API](https://www.algolia.com/doc/rest-api/search/). Specifically, it gives instructions for how to sync [synonyms](https://www.algolia.com/doc/guides/managing-results/optimize-search-results/adding-synonyms/) in a search index for each row added to a data model.
Hightouch provides an [Algolia integration](/destinations/algolia) that allows you to index records and events, but not synonyms, rules, or other Algolia objects.
### Destination configuration
First, we go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Then, we select **HTTP Request** and click **Continue**. Next, the UI prompts us to enter a **base URL**, **HTTP headers**, and **certificate**.
Algolia's Search API has the same host URL for searching, adding objects, and adding settings like synonyms.
We can enter this URL as the **base URL** in Hightouch.
According to [Algolia's docs](https://www.algolia.com/doc/rest-api/search/#authentication), you should authenticate API requests with these headers:
- `X-Algolia-Application-Id`: the ID of your Algolia application
- `X-Algolia-API-Key`: an [API key](https://www.algolia.com/doc/guides/security/api-keys/)
In Hightouch, we would provide these as the **HTTP headers**.
Be sure to replace the example headers with your actual Algolia application ID and an API Key with the write permission.
Since we don't need to use any security certificates, we can leave those options turned off and click **Continue**.
The last step of the initial configuration is giving our destination a descriptive name.
In this case, "HTTP Request - Algolia REST API" lets us know what we can use this destination for.
### Create synonyms
Next, we need to understand Algolia's API for adding synonyms to a search index. According to [Algolia's API reference](https://www.algolia.com/doc/rest-api/search/#synonyms-endpoints), we can add multiple synonyms by making a `POST` request to the [`/1/indexes/{indexName}/synonyms/batch` endpoint](https://www.algolia.com/doc/rest-api/search/#save-synonyms).
Here's how we can configure the **rows added** trigger in Hightouch to match this:
- Select to add a **batch of multiple rows** in each request, we can leave the default batch size of 100
- Select **POST** as the HTTP method
- Enter `/1/indexes/{indexName}/synonyms/batch` as the **URL**
In the preceding screenshot, the application ID and index name we want to add synonyms to have been filled out; you need to include your own Algolia application ID and index name.
[Algolia's docs](https://www.algolia.com/doc/rest-api/search/#format) say that the entire API uses JSON, so we select **JSON** as the payload type.
To construct the request payload, we need to reference our Hightouch model, which might look like this:
{/* */}
| `id` | `synonyms` |
| ----- | ----------------------------------------- |
| "sso" | "sso, single sign on, single sign-on" |
| "cdc" | "diff, diffing, change data capture, cdc" |
{/* */}
[Algolia's synonym batch endpoint reference](https://www.algolia.com/doc/rest-api/search/#save-synonyms) tells us that the synonym batch endpoint expects a JSON array of synonyms with a specific format.
```json
[
{
"objectID": "synonymID1",
"type": "synonym",
"synonyms": ["iphone", "ephone", "aphone", "yphone", "apple phone"]
},
{
"objectID": "synonymID2",
"type": "onewaysynonym",
"input": "iphone",
"synonyms": ["ephone", "aphone", "yphone", "apple phone"]
}
]
```
We can construct this expected JSON payload by using the **JSON editor** and inputting this as the **JSON payload** definition:
```json
[
{% for row in rows %}
{
"objectID": "{{ row.id }}",
"type": "synonym",
"synonyms": [ "{{ row.synonyms | replace: ', ', '", "' }}" ],
},
{% endfor %}
]
```
This example uses the Liquid [`replace`
function](https://shopify.github.io/liquid/filters/replace/) to create a
properly formatted array from a comma-separated string. If the data type in
the model results is already an array, this transformation isn't necessary.
Next, we reference [Algolia's docs regarding rate limits](https://www.algolia.com/doc/guides/sending-and-managing-data/prepare-your-data/in-depth/index-and-records-size-and-usage-limitations/#indexing-rate-limit).
Algolia starts throttling indexing operations when you have over 100 pending requests, so we set a rate limit of 100 requests per second and a concurrency limit of 100 requests at a time.
In the unlikely scenario that Algolia's API is experiencing downtime, we would want to wait a while before retrying failed rows. Therefore, we configure our sync to retry errors during the next sync run and give Algolia 30 seconds to respond to each of our HTTP requests.
Lastly, we want to make sure that our first sync run backfills our Algolia index with synonyms for all rows present in the initial model results.
That's it. Using Hightouch's UI, we've built a custom integration with Algolia to create new synonyms whenever new rows appear in our Hightouch model.
Once we've run the sync, we should check our Algolia dashboard to confirm the synonyms appear as we expect.
We can also use the Hightouch [debugger](/syncs/debugger) to inspect sync runs.
### Updating or deleting synonyms
In the future, we might want to update and delete existing synonyms so that our Algolia index always stays in sync with the latest data in our source.
To do this, we would enable the **rows changed** and **rows removed** triggers for the HTTP Request destination. The configuration for each of these triggers would be slightly different from the tutorial above, but the same concepts apply. Reference the Algolia API docs to determine the appropriate HTTP method, endpoint, and payload for each operation.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## HubDB
**URL:** https://hightouch.com/docs/destinations/hubdb
**Description:** Easily sync data and publish tables to always keep your website or landing page up to date.
**Section:** Destinations
## Overview
With Hightouch's HubDB integration, you no longer need to rely on others to build and maintain databases. Push and store data to tables that you can embed within websites and landing pages. Integrated into HubSpot, data is easily accessible, editable and shareable across multiple tables. You can even publish table updates in one sync.
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|**Tables**|Sync data to create or update tables in a HubDB. |Upsert, Update|
## Connect to HubDB
You must have one of the following HubSpot account types to use the Hightouch integration:
- CMS Hub Professional
- Enterprise
- Marketing Hub Professional
- Enterprise with Website Add-on
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **HubDB**. You can then authenticate Hightouch to HubDB via **OAuth**.
### Authenticate with OAuth
Click **Log in to HubDB** and log into your HubSpot account.
Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your HubDB destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the HubDB destination you want to sync to.
In the HubDB sync configuration form, select the HubDB table you want to sync to.
#### Record matching
Hightouch requires choosing one column to define how rows in your source should be matched to records in HubDB. The field to match should be unique and the data type of the field should match the data type of the corresponding column in your model.
#### Field mapping
You can choose which specific fields you want to sync to your HubDB table. Only the fields that you map will be exported. All other columns from your results are ignored. If you don't have a HubDB field to sync data to, you can add a new field to the table directly within HubDB. Then, refresh the HubDB fields and the new field should be available to map.
#### Delete behavior
You can choose how to handle data points in HubDB when the corresponding rows are deleted in your source.
| Behavior | Description |
| ----------------------------- | -------------------------------------------------------------- |
| **Do nothing** | Keep the data in HubDB |
| **Clear Fields** | Clear the mapped fields but keep the record in the HubDB table |
| **Delete Destination Record** | Delete the record from the HubDB table completely |
## Tips and troubleshooting
### Common errors
#### 400 - the value for column date was of type STRING, but we're expecting NUMERIC
The full error message looks something like this:
```json
{
"message":"Error when batch creating datatable rows in table 6058806 for portal 2661677: the value {type=string, value=2021/03/20 13:30} for column date was of type STRING, but we're expecting NUMERIC",
}
```
One possible cause for this error is sending incorrectly formatted date values to HubDB.
[HubDB's `date`](https://developers.hubspot.com/docs/api/cms/hubdb) data type must be a millisecond timestamp set to midnight UTC.
A millisecond timestamp is numerical value with 13 digits, for example `1620170285617`, as compared to a traditional [Unix timestamp](https://en.wikipedia.org/wiki/Unix_time) which has ten.
### Live debugger
### Sync alerts
---
## HubSpot
**URL:** https://hightouch.com/docs/destinations/hubspot
**Description:** Prioritize & personalize your marketing & sales outreach by adding customer data into HubSpot
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| ----------------------------------- | ----------------------------------------------------------------- | ---------------------- | ----------------------------------------------------------------------------------------- |
| [**Objects**](#objects) | Sync records to objects such as contacts or companies in HubSpot. | Upsert, Update, Insert | [Manage CRM records ↗](https://developers.hubspot.com/docs/api/crm/understanding-the-crm) |
| [**Events**](#events) | Sync records as custom behavioral events in HubSpot. | Insert | [Track behavioral events ↗](https://developers.hubspot.com/docs/api/analytics/events) |
| [**Contact lists**](#contact-lists) | Create, sync, and keep contact lists up-to-date in HubSpot. | Add, Remove | [Manage lists ↗](https://developers.hubspot.com/docs/api/crm/lists) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
---
## Understand your HubSpot setup
Before you begin your Hightouch to HubSpot sync configuration, you should clarify your business goals and understand your HubSpot setup. Walk through the following considerations to align your team and confirm that your sync plan matches how HubSpot is configured. If you prefer, you can download this resource as a PDF to share with your team.
### Business goals
Before configuring your Hightouch to HubSpot sync, clarify what data you want to sync, how it should be updated, and how often it needs to stay in sync. Align on these decisions with your team so your sync configuration reflects your business goals.
The table below shows how these decisions influence your sync setup.
|
Consideration
| Effect on sync configuration |
| ------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| What standard and custom objects do you want to sync to HubSpot? Contacts, companies, etc.? | The response affects your [sync types](#supported-syncing)—in other words, which objects you select to sync to. You need to set up a separate sync configuration for each object you want to sync to. |
| What standard and custom properties do these objects have that you want to sync to? | The response affects the [field mapping](#field-mapping) on each of your syncs. |
| Do you have new data to add, old data that needs updating, or a combination of both? | The response affects your [sync modes](#supported-syncing)—for example, insert mode for new data, update mode for old, and upsert mode for both. |
| How frequently do you need to update the data? | The response affects your [sync schedule](/syncs/schedule-sync-ui). This is the last step of your sync configuration. |
### Required data specifications
Once you’ve defined what you want to sync, verify that your HubSpot setup supports your plan. Use the table below to validate that your objects, properties, and permissions are configured correctly before creating your syncs.
|
Consideration
| Effect on sync configuration |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Is the property that you would like to use for [record matching](#record-matching) unique? | Non-unique values can lead to duplicates and sync errors. |
| What are the expected data types of the properties you want to sync to? | The response affects your [field mapping](#field-mapping); if the expected type differs from your model column type, you must [type cast your data](/models/data-types-casting) in your model configuration. |
| Are the properties you want to sync editable? | Hightouch only displays editable fields during [field mapping](#field-mapping). |
| Are any of the properties you want to sync [calculated](https://community.hubspot.com/t5/Tips-Tricks-Best-Practices/Using-calculated-properties/m-p/372525) properties? | Calculation properties are non-editable and read-only. Change the field type if you want to sync data to it. |
| Are any of the properties you want to sync [enumeration](https://knowledge.hubspot.com/account/property-field-types-in-hubspot) properties? | Enumeration properties only accept specific values. Ensure your data matches these values or create a new property. |
| Are there any associations you want to make between objects? | You can set up [associations](#associations) while field mapping, assuming the association exists in HubSpot first. |
| Does the user you are authenticating to Hightouch have edit access? | The sync will error if the user doesn't have the correct permissions. |
Permissions depend on both the HubSpot user’s role and the OAuth scopes granted when connecting Hightouch. If objects or properties don’t appear during [record matching](#record-matching) or [field mapping](#field-mapping), re-check permissions and reconnect the destination with the appropriate scopes.
Use the steps in [Inspect objects, properties, and associations](#inspect-objects-properties-and-associations) to validate your HubSpot configuration before creating your syncs.
### Inspect objects, properties, and associations
1. In HubSpot, click the settings icon in the top navigation bar. In the left sidebar, go to **Objects** (under **Data Management**), then select the object you want to inspect (for example, **Contacts**, **Companies**, **Leads**, or a **Custom Object**).
2. In the object's overview page, under **Setup**, click **Manage [object] properties**.
3. Review both standard and custom properties you want to sync data to. Make note of their:
- **Internal name**: This is the name that appears in the Hightouch UI for [record matching](#record-matching) and [field mapping](#field-mapping). You can access it by clicking on the property name and then clicking **\** in the property inspector.
- **Property access**: The HubSpot user you use to [authenticate to Hightouch](#connect-to-hubspot) should have access to edit the property.
- **Data type**: For example, **Single-line text** or **Number**—you can view these under the property **Name**. When selecting model columns for [field mapping](#field-mapping), you must ensure the data type matches. For example, a **Single-line text** property requires text data and a **Number** requires numerical data.
4. Check whether any of the properties you plan on syncing to are an enumeration type, such as **Dropdown select**, **Radio select**, or **Multiple checkboxes**. For example, the native **Lead rating** property for contacts is a **radio select** property.
If you plan on syncing to enumeration properties, the data you sync must match the **internal value** of the options. Lead rating, for example, can accept the values "bucket_1," "bucket_2," "bucket_3," and "bucket_4," though the labels are "1 Star," "2 Stars," etc.
Since internal values can't be updated once created, you may need to create a new custom property to accept data from your model. Refer to HubSpot's guide on [property field types](https://knowledge.hubspot.com/account/property-field-types-in-hubspot) to select the appropriate field type when creating new properties.
5. Check if the property requires unique values: Click on its name, then go to **Rules**. Under **Validation rules**, see if **Require unique values for this property** is checked. Properties you want to use for [record matching](#record-matching) must have this validation.
6. Check if the property is sensitive or highly sensitive. To sync sensitive data, you must be using an Enterprise account and have permissions from your admin to read and write to sensitive and highly sensitive properties.
7. Return to the object's overview page and select **Associations**. Look for associated child objects like contacts, companies, or deals linked to the records. You can use these to map [associations](#associations) in Hightouch. Add, remove, or correct links between objects as needed.
Once you've validated the properties and associations of the objects you want to sync to, you're ready to configure your sync in Hightouch.
---
## Connect to HubSpot
1. Go to **`Integrations`** → [**`Destinations`**](https://app.hightouch.com/destinations?from=docs).
2. Click **`Add destination`**.
3. Select **`HubSpot`** and click **`Continue`**.
4. Choose how to authenticate Hightouch to HubSpot:
- **`OAuth`** (recommended): Use this option to sync CRM objects (for example, contacts and companies) and update standard properties.
- **`OAuth with enterprise scopes`**: Required if you plan to sync **custom behavioral events** or use Enterprise-only features such as sensitive properties.
- Enterprise users can also enable scopes for [sensitive data](https://knowledge.hubspot.com/properties/store-sensitive-data). Select **`Sensitive`** or **`Highly sensitive`** only if your sync requires access to those properties.
For the smoothest setup, authenticate using a HubSpot user with admin
permissions (typically **Super Admin**). HubSpot permissions and the OAuth
scopes you approve both affect which objects and properties appear in
Hightouch and whether syncs can write updates.
Only enable **`Sensitive`** or **`Highly sensitive`** scopes if your sync
needs to read or write those properties.
5. Select the **HubSpot account** you want to connect to Hightouch.
6. Review and accept the scopes that the Hightouch app requests from HubSpot.
7. After authorization succeeds, you’ll be redirected back to Hightouch. Click **`Continue`**.
8. (Optional) Configure alert settings and recipients. You can edit these later from the destination settings.
9. Enter a descriptive **destination name** and configure permissions for:
- Triggered syncs
- Configuring models and syncs
- Managing the destination
10. Click **`Finish`** to complete setup.
---
At this point, you should know which objects, properties, identifiers, and associations you plan to sync, and have verified that your HubSpot permissions and property types support your configuration.
## Sync configuration
Once you've set up your HubSpot destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the HubSpot destination you want to sync to.
### Objects
Hightouch can create and update standard HubSpot objects such as contacts, companies, deals, etc. Hightouch can also sync data to custom HubSpot objects. To sync to a custom object, you must [create it in HubSpot](https://knowledge.hubspot.com/crm-setup/use-custom-objects) first.
#### Record matching
To match rows from your model to objects in HubSpot, select a HubSpot property and the corresponding column from your model.
You can match on any of the available properties in the dropdown. It's best to select a property that [HubSpot automatically deduplicates on](https://knowledge.hubspot.com/crm-setup/deduplication-of-contacts-companies-deals-tickets#deduplicate-companies-by-company-domain). These include **email** (Contacts), **company domain** (Companies), or a **HubSpot record ID** (for example, `hs_object_id`) for other object types.
If you select a model column with duplicates to send to one of these properties, HubSpot [returns a 409 error](#409---contact-already-exists) and the sync errors.
If record matching isn’t unique (for example, multiple HubSpot records match the same value), HubSpot may reject updates and Hightouch may need extra lookups. This can slow syncs and cause partial failures.
The `id` property is only available for record matching when in update mode.
It is a read-only property, and HubSpot sets its value upon object creation.
The HubSpot property you select to match on should be **unique**, and its data type must match the data type of the corresponding column in your model. If the property to match on isn't unique, record matching will be slower and duplicate records may occur. You can check a property's uniqueness validation by [inspecting the property in HubSpot](#inspect-objects-properties-and-associations).
If the object doesn't have a native unique property, consider creating a
unique custom property. Refer to [HubSpot's
instructions](https://community.hubspot.com/t5/Tips-Tricks-Best-Practices/Custom-unique-property/td-p/683119#)
on how to do so.
#### Field mapping
Field mapping lets you choose which properties you want to update on your HubSpot objects. If the HubSpot object you're syncing to doesn't have the specific property you want to send your data to, you need to [create a new property on the object](https://knowledge.hubspot.com/crm-setup/manage-your-properties) in HubSpot first.
You can also sync to [enumeration
properties](https://knowledge.hubspot.com/account/property-field-types-in-hubspot),
such as **Dropdown select**, **Radio select**, and **Multiple checkboxes**.
Make sure to create your property's internal values before syncing them, as
explained in the [Common errors](#400---property-values-were-not-valid)
section.
Check out [HubSpot's documentation](https://knowledge.hubspot.com/crm-setup/import-data-to-checkbox-properties)
to learn how to sync **Multiple checkboxes**.
After creating or editing properties, you can refresh the available properties for mapping by clicking the refresh button next to **`Field from HubSpot`**. The new property should then be available.
Hightouch displays the property's **Internal name**, not label, in the dropdown. You can find a property's internal name by [inspecting the property in HubSpot](#inspect-objects-properties-and-associations).
Read-only fields in HubSpot aren’t available for mapping in Hightouch.
Field mapping columns may appear greyed out until you select a sync type and
configure record matching. Once record matching is set, the available fields
for that object type populate and become editable.
If a model column’s data type doesn’t match the HubSpot property type (for example, sending text to a number property), HubSpot may reject the row. If needed, type cast your data in your model before syncing.
#### Field sync behavior
Each field mapping has a sync behavior setting that controls how Hightouch handles the value for that field. Click the gear icon on a field mapping row to choose one of the following options:
| Behavior | Description |
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Always sync** | Syncs the value from your model to HubSpot on every sync run, including null values. Use caution, as this can erase existing production data. Does not apply to nested object or array properties. |
| **Don’t sync null values** | Skips syncing any null values from your model to HubSpot. If the model value is null, the existing HubSpot value is left unchanged. |
| **Only write to empty fields** | Only writes the value if the HubSpot property is currently empty. Existing non-empty values are not overwritten. |
**Only write to empty fields** is useful when you want to populate missing data without overwriting values that have already been set in HubSpot—for example, backfilling phone numbers or other contact details without replacing manually entered data.
#### Associations
**Associations** let you express a relationship between the object you're syncing to (the **main object**) and a different HubSpot object (the **associated object**). For example, you may be syncing contacts to HubSpot and want to associate a company with each one.
Before mapping an association in Hightouch, confirm the association exists for the object pair in HubSpot (**Settings → Objects → [Object] → Associations**).
1. Click **`Add mapping`**, then select the association label under **`Field from HubSpot`**. The list of supported association labels for the object appears in the dropdown. You can find an object's supported associations and add others by [inspecting the object in HubSpot](#inspect-objects-properties-and-associations).
2. Search for the object you want to associate to (the **associated object**). For example, if you want to associate a company with each contact you're syncing, search for "company," then select the appropriate association. In this example, you would select **CONTACT_TO_COMPANY**.
The association should only be set up in the main object's sync configuration.
In this example, you would only add it to the **Contact** sync and not to the
**Company** one. You can learn more about this in the [Common
errors](#conflicting_one_to_ones_specified) section.
Make sure that the sync to the associated object always runs before the one to
the main object. After creating your syncs, you can [set up a
sequence](/syncs/schedule-sync-with-sequences) to schedule them in the correct
order.
Next, select the properties to match on. For example, if you're associating companies with contacts and a company's `name` matches the `COMPANY_NAME` in your contact data, you would select those properties.
If your data contains the `id` of the object you want to associate to, it's best to use this column for matching. Since the sync doesn't have to perform a lookup, sync performance is significantly faster.
The example below shows using contact data that includes the associated company's id in a column called `ACCOUNT_ID`.
You can create multiple HubSpot associations in a single sync. Click **`Add mapping`** for each association you want to make.
For more information on associations, check out [HubSpot's association docs](https://knowledge.hubspot.com/crm-setup/associate-records).
#### Association modes
##### **Create added associations**
Associations will only be appended to the record. This setting will not remove previous associations if the associated object is updated or removed.
##### **Overwrite existing associations**
When there are changes to associations in the source, remove all associations in HubSpot before creating the associations present in the source. If there are no changes to associations in the source, do not remove any associations.
This setting will also remove any associations that were created outside of
Hightouch.
##### **Delete removed associations and create added associations**
This mode will find the differences between sync runs to delete associations in HubSpot that have been removed from the model, then create associations that have been added to the model.
#### One-to-many relationships
You might want to associate the object you're syncing to with multiple objects of the same type. For example, if you're syncing company objects, you may want to link several contacts to each company.
For Hightouch to associate multiple objects, the model column you select for the association must be in one of these formats:
- an array of string or number values, for example, `["id1", "id2", "id3"]`
- a comma-separated string, for example, `"id1, id2, id3"`
Confirm that the association cardinality in your HubSpot settings is
compatible with the data you're syncing to HubSpot. You can learn more about
this in the [Common errors](#conflicting_one_to_ones_specified) section.
#### Association labels
If you have a Professional or Enterprise HubSpot account, you can also use HubSpot's [association labels feature](https://knowledge.hubspot.com/crm-setup/create-and-use-association-labels) to describe relationships between objects.
Association labels let you add descriptive text to an association that provides more context about the relationship. For example, instead of just linking a contact to a company, you can use an association label like "Employee at" or "Customer of" to clarify the nature of the relationship.
Before configuring a sync to use association labels, check out HubSpot's
association label limitations at the bottom of this [HubSpot
doc](https://developers.hubspot.com/docs/api/crm/associations/v4).
To add association labels to a Hightouch sync, you must first [set up your labels in your HubSpot account](https://knowledge.hubspot.com/crm-setup/create-and-use-association-labels). Then, you need to update the [model](/getting-started/concepts#models) column you use for associations to contain an object with this schema:
```
{ "value": string | number, "label": string | Array }
```
Make sure to set the model column's [data
type](/models/data-types-casting#view-model-data-types) to **Object / Array**.
Hightouch uses the **value** to match the associated object and the **label** as the label on the association in HubSpot. For example, if you're associating companies with contacts, you may set the company name as the value and the contact's role as the label.
```
{ "value": "ABC Corp", "label": "Employee at" }
```
If you want to add more than one label, the label can be an array of strings or a comma-separated list. For example, if you're associating companies with a contact and the contact has multiple roles, a row from your model might look like this:
```
{ "value": "ABC Corp", "label": ["Employee at", "Decision maker"] }
```
If you want to add more than one association label to an object, you can sync an array of association label objects.
For example, if you're associating billing data with multiple contacts, a row from your model might look like this:
```
[{ "value": "John Doe", "label": "Buyer" }, { "value": "Jane Doe", "label": "Seller" }]
```
Ensure **all** rows follow this schema to avoid inconsistent behavior with HubSpot's API. If a row doesn't have a label, you can set the label as `null`.
For example, suppose you want to associate a company with a contact but don't have any information about the contact's role there. The model row should look like this:
```
{ "value": "ABC Corp", "label": null }
```
You can use some SQL helper functions to create an object or array of objects with this schema. Below is an example of a SQL query that creates an object for an association label:
```sql
SELECT
id,
first_name,
last_name,
email,
JSONB_BUILD_OBJECT('value', company_name, 'label', role) as company_obj
```
The query creates a column called `company_obj` and sets the `company_name` name as the value in the object and the `role` as the label in the object.
Once you've formatted the data correctly, ensure the association in your sync configuration uses the column with the properly formatted object.
#### Sensitive data
HubSpot allows for Enterprise users to store [sensitive data](https://knowledge.hubspot.com/properties/store-sensitive-data). Prior to creating Sensitive Data properties, a user with super admin permissions must turn on Sensitive Data in your Privacy & Consent
settings and accept the related terms and conditions.
A property can be defined as sensitive or highly sensitive upon creation in the UI or through the API. If syncing to these sensitive data properties, ensure that you have the correct scopes selected when you authorize your account.
Currently, Hightouch supports using the following sensitive data scopes:
- `crm.objects.custom.sensitive.read`
- `crm.objects.custom.sensitive.write`
- `crm.objects.custom.highly_sensitive.read`
- `crm.objects.custom.highly_sensitive.write`
- `crm.objects.appointments.sensitive.read`
- `crm.objects.appointments.sensitive.write`
Sensitive scopes are object-specific. If you don’t see the object you need covered by a sensitive scope, confirm HubSpot supports sensitive properties for that object type and reconnect with the required scopes.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. Depending on the [sync mode](#objects) you're using, you can select from some or all of these options:
| Behavior | Description |
| --------------------------------- | ---------------------------------------------------- |
| **Do nothing** | Keep the record in HubSpot |
| **Clear fields** | Keep the record, but clear the mapped fields |
| **Archive record** | Archive the record in HubSpot |
| **Delete record (Contacts only)** | GDPR Delete the record in HubSpot (Permanent delete) |
In HubSpot, **archived** records are removed from active views but are not permanently deleted. **GDPR delete** is permanent and only available for Contacts.
In Upsert mode, you can select from all of the above (**Delete record** for contacts only, performs a permanent delete using the [**GDPR delete**](https://developers.hubspot.com/docs/reference/api/crm/objects/contacts/v3#post-%2Fcrm%2Fv3%2Fobjects%2Fcontacts%2Fgdpr-delete) endpoint). In Update mode, you can only select between **Do nothing** and **Clear fields**. Insert mode doesn't support delete behavior.
#### Split retries
HubSpot counts all objects in a batch as rejected if the request contains a single invalid record. To pinpoint which records are getting rejected with which errors and reduce the number of valid records that get retried, you can enable [**split retries**](/syncs/retries#split-retries).
### Events
Hightouch supports syncing [**custom behavioral events**](https://knowledge.hubspot.com/analytics-tools/create-codeless-custom-behavioral-events) to HubSpot with custom properties. To sync events, ensure you logged in via **OAuth with enterprise scopes** when you [connected Hightouch to HubSpot](#connect-to-hubspot).
#### Event name
Providing an event name is required to log an event in HubSpot. You can either provide a static value or select to **use a column** from your model data.
The event name should match the event's **internal name**. HubSpot assigns the internal name automatically when you create the event. You can find an event's internal name either in [HubSpot](https://knowledge.hubspot.com/analytics-tools/create-custom-behavioral-events#define-the-api-call) or by using the HubSpot [events API](https://legacydocs.hubspot.com/docs/methods/events/get_events).
Follow the instructions in HubSpot's docs to learn how to [find the internal name](https://knowledge.hubspot.com/analytics-tools/create-custom-behavioral-events#define-the-api-call) for an event.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server.
#### Assigning events to contacts
Hightouch lets you choose how you assign events to contacts. Select a column in your events data that matches one of these HubSpot contacts identifiers:
- email
- object (contact) ID
- user token
If the identifier column is null or doesn’t match an existing HubSpot contact,
HubSpot can’t attribute the event to a record.
#### Field mapping
In the field mapping section, you can select columns from your model to sync to the default or custom properties of a HubSpot custom behavioral event.
Custom properties must exist on the event before you can sync to them. See
HubSpot's docs on [how to create a custom
property](https://knowledge.hubspot.com/analytics-tools/create-custom-behavioral-events-with-the-code-wizard)
on an event.
### Contact lists
Hightouch supports syncing contacts to [contact lists](https://knowledge.hubspot.com/lists/create-active-or-static-lists) in HubSpot. You can sync data to an existing contact list or create a new one on the first sync run. Select **Segment** as the sync type.
#### User identifiers
To identify which contacts to add or update in a contact list, select a model column and the corresponding HubSpot field.
You can match on either **Email** or **HubSpot Contact ID**.
#### List creation or selection
Hightouch supports automatically creating a new list for your sync. You can specify a custom name for this list; if left empty, Hightouch defaults to the model name. The example below shows defaulting to the model name "Newsletter subscribers."
If you opt to sync to an existing list, you can select the list name from the dropdown.
#### Migrating from v1 to v3
If you had existing contact list syncs that use HubSpot's v1 API before April 30th, 2026, you need to migrate them to v3. The v1 and v3 APIs use different list IDs, so you must re-select your list after switching.
To migrate a sync:
1. Open your sync configuration.
2. Under **Object type**, select **Contacts**.
3. Select **Use an existing list** and **re-select your list from the dropdown.**
4. Save the sync.
Your list contents and membership are unchanged by this migration. Only the API version is updated.
---
## Disconnecting HubSpot
### Disconnect HubSpot from Hightouch
To disconnect HubSpot from Hightouch:
1. Open [**`Destinations`**](https://app.hightouch.com/destinations?from=docs).
2. Select your HubSpot destination.
3. Choose **`Actions`** → **`Delete`**.
Deleting the destination also deletes any syncs that use that destination. Previously synced data in HubSpot is not changed.
### Uninstall Hightouch from HubSpot
You can uninstall the Hightouch app from your HubSpot account by referring to these [instructions](https://knowledge.hubspot.com/integrations/connect-apps-to-hubspot#uninstall-an-app). No data previously synced from Hightouch to HubSpot will be impacted.
---
## Tips and troubleshooting
For Hightouch platform error codes related to HubSpot, see [Error codes:
HubSpot](/developer-tools/error-codes#hubspot).
### Date values
When syncing date or timestamp values to HubSpot, you need to consider which data type the HubSpot field should use: `date` or `datetime`.
HubSpot `date` fields support date values which store a date without a time. `datetime` fields support timestamps. For more information, check out [HubSpot's timestamp FAQ](https://developers.hubspot.com/docs/api/faq#timestamps).
Values you send to a `date` field must follow one of these formats:
- date-only ISO 8601 strings (`YYYY-MM-DD`), for example, `2020-02-29`
- EPOCH-millisecond [timestamps](https://en.wikipedia.org/wiki/Unix_time), **with the time set to midnight UTC**: for example, `1430438400000`
If you're syncing a timestamp to a date field, Hightouch automatically sets its time to midnight UTC, as required by HubSpot.
Values you send to a `datetime` field must follow one of these formats:
- ISO 8601 formatted strings (`YYYY-MM-DDThh:mm:ss.sTZD`), for example, `2020-02-29T03:30:17.000Z`
- EPOCH-millisecond [timestamps](https://en.wikipedia.org/wiki/Unix_time): for example, `1427997766000`
HubSpot displays `datetime` values in the timezone of the user viewing the record.
If you notice the timestamp from your Hightouch model column isn't appearing in HubSpot for your `datetime` field, edit the JSON mapping in your sync configuration to include `"skipDateParse": true`. See example below:
```json
mappings: [
{
"from": "hightouch_datetime_column",
"to": "hubspot_datetime_field",
"skipDateParse": true
}
]
```
### Validate your setup
After your first sync completes, verify the data in HubSpot:
- For object syncs, open the target object in HubSpot (for example, **Contacts** or **Companies**) and search for a test record. Confirm that mapped properties appear with expected values.
- For event syncs, open a contact record and check the **Activity** timeline for the custom behavioral event.
- For contact list syncs, open the target list in **Contacts → Lists** and confirm membership matches your model.
If properties or objects don't appear in Hightouch during sync configuration, re-check the connected user's permissions and OAuth scopes, then reconnect the destination.
### Common errors
| Error message | Description |
| :------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------- |
| `402 - Cannot create another PORTAL_SPECIFIC_OBJECT` | Reached HubSpot's object creation limit for a specific type. [More detail.](#402---cannot-create-another-portal_specific_object) |
| `403 (1006) - Firewall blocked request` | HubSpot's firewall is blocking Hightouch's IP addresses. [More detail.](#403-1006---firewall-blocked-request) |
| `429 - You have reached your secondly limit` | Rate limit exceeded due to concurrent syncs or association constraints. [More detail.](#429---you-have-reached-your-secondly-limit) |
| `Missing permissions or OAuth scopes` | Objects or properties don't appear, or syncs fail after connecting. [More detail.](#missing-permissions-or-oauth-scopes) |
| `409 - Contact already exists` | Duplicate email or record matching values. [More detail.](#409---contact-already-exists) |
| `Property does not exist or is not writable` | Property missing, read-only, or insufficient scopes. [More detail.](#property-does-not-exist-or-is-not-writable) |
| `400 - Cannot update object...the following properties were cleared` | No data sent to a required HubSpot property. [More detail.](#400---cannot-update-object-with-type--the-following-properties-were-cleared-) |
| `400 - Property values were not valid` | Value doesn't match a valid option in an enumeration property. [More detail.](#400---property-values-were-not-valid) |
| `400 - BAD_REFRESH_TOKEN` | OAuth token expired or revoked. [More detail.](#400---bad_refresh_token) |
| `400 - Duplicate unique property value` | Another record already uses the value for a unique field. [More detail.](#400---duplicate-unique-property-value) |
| `400 - One or more associations are invalid` | Association limit exceeded or self-referencing association. [More detail.](#400---one-or-more-associations-are-invalid) |
| `Invalid property value due to type mismatch` | Data type in model doesn't match the HubSpot property type. [More detail.](#invalid-property-value-due-to-type-mismatch) |
| `CONFLICTING_ONE_TO_ONES_SPECIFIED` | Association set up in both objects' syncs or cardinality mismatch. [More detail.](#conflicting_one_to_ones_specified) |
| `Row with ID email...could not be added to the list` | Contact doesn't exist in HubSpot before list sync. [More detail.](#row-with-id-emailcould-not-be-added-to-the-list) |
| `Association label missing or misformatted` | Association payload missing the required label key. [More detail.](#association-label-missing-or-misformatted) |
#### 402 - Cannot create another PORTAL_SPECIFIC_OBJECT
```
402 - Cannot create another PORTAL_SPECIFIC_OBJECT
```
**Cause:** Your HubSpot portal has reached the maximum number of records for a specific object type. For example, the default limit for contacts is 1,500,000.
**Solution:**
1. Archive or delete unused records in HubSpot to free up capacity.
2. If you need a higher limit, upgrade your HubSpot subscription or contact HubSpot support.
#### 403 (1006) - Firewall blocked request
```
403 - {"status":"error","category":"BLOCKED","errors":[{"errorType":"BLOCKED","id":"1006"}]}
```
**Cause:** HubSpot's web application firewall (WAF) is blocking requests from Hightouch's IP addresses.
**Solution:**
1. Contact HubSpot support and ask them to allowlist Hightouch's IP addresses. You can find the current IPs in [Hightouch's networking documentation](/security/networking).
2. Try disconnecting and reconnecting your HubSpot destination in Hightouch to reauthorize.
#### 429 - You have reached your secondly limit
```
429 - You have reached your secondly limit
```
**Cause:** HubSpot's per-second API rate limit has been exceeded. This typically happens when another sync to HubSpot is running simultaneously, or when [associations](#associations) are configured (the HubSpot API has constraints that slow syncs using associations).
**Solution:**
1. Ensure you're following [record matching configuration suggestions](#record-matching).
2. Stagger sync schedules so multiple HubSpot syncs don't run at the same time.
3. Enable [split retries](#split-retries) so Hightouch retries only the failed records instead of the full batch.
4. If you're on a lower-tier HubSpot plan, check whether your plan's API rate limits are sufficient for your sync volume.
For more information about HubSpot's API limits, see [HubSpot's API usage guidelines](https://developers.hubspot.com/docs/api/usage-details).
{/* */}
#### Missing permissions or OAuth scopes
Objects, properties, or events don’t appear in Hightouch, or syncs fail immediately after connecting. This isn’t a specific error message — it’s a pattern that indicates the HubSpot user may not have the required permissions, or the connection may be missing required scopes.
**Cause:** The HubSpot user who authorized the connection doesn’t have sufficient role permissions, or the OAuth connection wasn’t granted the necessary scopes.
**Solution:**
1. Reconnect HubSpot using a user with admin permissions (typically Super Admin).
2. Approve the required scopes (use **OAuth with enterprise scopes** for events and sensitive properties).
#### 409 - Contact already exists
{/* */}
```
409 - Contact already exists
400 - Duplicate IDs found in batch input
```
**Cause:** The model column used for [record matching](#record-matching) contains duplicate values, the column [mapped](#field-mapping) to HubSpot's `email` field contains duplicate values, or you're creating a new contact with the same email as an existing one but a different record-matching value. HubSpot uses email as the primary unique identifier — two contacts can't share the same email. See [HubSpot's Contacts API Reference](https://developers.hubspot.com/docs/api/crm/contacts#properties) for more information.
**Solution:**
1. Ensure the column you selected for record matching doesn't contain duplicate values.
2. Ensure any email information you're syncing doesn't include duplicate values.
3. Confirm you aren't creating a new HubSpot contact with the same email as an existing one but a different value in the record-matching field.
The `409 - Contact already exists` error message contains an `Existing ID`
value returned by HubSpot. This value refers to HubSpot's ID for the contact
and doesn't necessarily correspond to any values in your model data.
{/* */}
#### Property does not exist or is not writable
```
Property does not exist or is not writable
```
**Cause:** The HubSpot property doesn’t exist on the target object, is read-only (for example, a calculated property), or the connected HubSpot user/scopes don’t have permission to write to it.
**Solution:**
1. Confirm the property exists on the correct HubSpot object (**Settings → Objects → [Object] → Properties**).
2. Confirm the property is editable (calculated properties are read-only).
3. Reconnect the destination if you recently added custom objects or changed permissions/scopes.
#### 400 - Cannot update object with type ... the following properties were cleared ...
{/* */}
```
400 - Cannot update object with type: ... the following properties were cleared: [property_name]
```
**Cause:** No data is being sent to a required property in HubSpot. The properties listed after "the following properties were cleared" are the required properties.
**Solution:**
1. Remove records with empty values in the required property by updating your [model definition](/models/creating-models) to exclude those rows.
2. Alternatively, add a value to the required property in your model data.
#### 400 - Property values were not valid
```
400 - {"status":"error","message":"Property values were not valid: [{"isValid":false,...}]"}
```
**Cause:** You're sending a value to an [enumeration property](https://knowledge.hubspot.com/account/property-field-types-in-hubspot) that isn't one of the valid options. Enumeration properties include **Dropdown select**, **Radio select**, and **Multiple checkboxes**. The error message may contain diagnostic lines like `"error":"INVALID_OPTION","name":"custom_dropdown_field"` or `"error":"INVALID_OPTION","message": {...} was not one of the allowed options` to help identify the field.
**Solution:**
1. Ensure values exactly match HubSpot's **Internal values**, not the **Labels**. These values are case sensitive.
2. Refer to step 4 of [Inspect objects, properties, and associations](#inspect-objects-properties-and-associations) to find the **Internal Values** of your HubSpot field.
As explained in the [field mapping](#field-mapping) section, make sure to
follow [HubSpot's
guidelines](https://knowledge.hubspot.com/crm-setup/import-data-to-checkbox-properties)
when syncing to **Multiple checkboxes** fields.
#### 400 - BAD_REFRESH_TOKEN
```
400 - BAD_REFRESH_TOKEN
```
**Cause:** The OAuth refresh token for your HubSpot connection has expired or been revoked.
**Solution:** Disconnect and reconnect HubSpot in **Destinations → [your HubSpot destination] → Configuration**. Use a HubSpot user with admin permissions to reauthorize.
#### 400 - Duplicate unique property value
```
400 - Cannot set PropertyValueCoordinates: ... A record with this value already exists
```
**Cause:** You’re syncing a value to a field that has a unique constraint in HubSpot, but another record already uses that value.
**Solution:**
1. Check the HubSpot property’s settings to confirm it’s marked as unique.
2. Deduplicate the values in your model, or update your model query to exclude rows that conflict with existing HubSpot records.
#### 400 - One or more associations are invalid
```
400 - One or more associations are invalid
```
**Cause:** The sync is attempting to create an association that exceeds HubSpot’s [association limits](https://developers.hubspot.com/docs/api/crm/associations#association-limits), or is trying to associate a record with itself.
**Solution:**
1. Check HubSpot’s association limits for the object types involved.
2. Ensure your model data doesn’t include rows that would associate a record with its own ID.
3. If you’re hitting limits, review your data model to reduce the number of associations per record.
#### Invalid property value due to type mismatch
```
Invalid property value due to type mismatch
```
**Cause:** The data type in your model doesn’t match the HubSpot property type (for example, sending text to a number property).
**Solution:**
1. Check the property’s field type in HubSpot.
2. Type cast the model column to the expected type before syncing.
3. Re-run the sync after updating the model or mapping.
#### CONFLICTING_ONE_TO_ONES_SPECIFIED
```
CONFLICTING_ONE_TO_ONES_SPECIFIED
```
**Cause:** An [association](#associations) is set up in both the main object's and the associated object's sync configuration, or the cardinality of your association or [association label](#association-labels) in HubSpot isn't compatible with the data you're syncing. For example, a `1-to-1` association can't [associate multiple records](#one-to-many-relationships) on the main object with one record on the associated object.
**Solution:**
1. If the association exists in both sync configurations, remove it from the associated object's sync.
2. Access the main object's HubSpot settings, open the **Associations** tab, and check your associations' cardinalities.
3. In your Hightouch sync configuration, use an association or association label that has the correct cardinality.
#### Row with ID email...could not be added to the list.
{/* */}
```
Row with ID email...could not be added to the list
```
**Cause:** When syncing [contact lists](#contact-lists) and [matching on **Email**](#user-identifiers), the contact you're trying to add doesn't exist in your HubSpot workspace.
**Solution:** Ensure the contact exists in HubSpot before syncing it to a contact list. Sync contacts first, then sync list membership.
#### Association label missing or misformatted
```
Associating 0-2 object with {…} was not found
```
**Cause:** An association requires a label but the payload doesn’t include one, or the label format doesn’t match the expected schema.
**Solution:**
1. Ensure the model column for association labels uses the required object schema described in [Association labels](#association-labels).
2. Ensure the `label` key is present (set to a string, array of strings, or `null`).
### Live debugger
### Sync alerts
---
## Impact.com
**URL:** https://hightouch.com/docs/destinations/impact-com
**Description:** Impact.com - One platform. All partnerships.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| -------------------------------- | ----------------------------------------------- | -------------------- | -------------------------------------------------------------------------------------------- |
| **Conversion Events** | Create and send conversion events to Impact.com | Insert | [Conversions docs](https://integrations.impact.com/impact-brand/reference/conversions-model) |
| **Objects (Catalogs / Actions)** | Sync catalog items or action records | Insert, Update | [Catalog docs](https://integrations.impact.com/impact-brand/reference/catalogs-overview) |
See [sync modes](/syncs/types-and-modes#sync-modes) for more details.
---
## Add Impact.com as a Destination
1. Go to the [**Destinations**](https://app.hightouch.com/destinations?from=docs) page and click **Add destination**.
2. From the destination list, search for **Impact.com** and select it.
3. Click **Continue**.
---
### 1. Connect to Impact.com
1. Enter your **Account SID** and **Auth Token**.
You can authenticate with either a legacy account token or a scoped token. Both are available in your Impact.com account under **Settings → API → API Access**.
- **Legacy account token** — find under **Legacy Account Tokens**.
- **Scoped token** — find under **Scoped Tokens**. When creating the token, grant the following scopes:
- `Accounts`
- `Actions`
- `Campaigns`
- `Catalogs`
- `Conversions`
- `Jobs Management`
- `Submissions`
From July 1, 2026, Impact.com legacy read/write and read-only tokens can no
longer be copied. Existing integrations will continue to work, but new
integrations or token resets will require scoped tokens.
2. Click **Test connection** to confirm the connection is valid.
3. Click **Continue**.
---
### 2. Configure alerts (optional)
You can configure default alerting rules for all syncs using this destination.
You may add recipients and enable alerts for:
- Fatal errors
- Rejected rows
- Sync throughput
- Model or audience size
- Sync duration
- Excessive idle time
Click **Continue**.
---
### 3. Finalize destination
Name your destination and assign permissions.
You can choose which user groups can:
- Trigger syncs
- Configure models & syncs
- Manage the destination
Click **Finish**.
---
## Create a sync
After creating and connecting your Impact.com destination, navigate to:
**`Integrations → Destinations → Impact.com → Syncs`** and click **Add a sync**.
You’ll follow the standard Hightouch sync wizard:
1. **Select model**
2. **Select destination**
3. **Configure sync**
4. **Finalize sync**
The following steps describe the **Configure sync** settings unique to Impact.com.
---
## Configure sync
At the top of the configuration page, choose what you want Hightouch to send:
- **Conversions** — send conversion events
- **Objects** — send catalogs or actions
This choice determines which mapping sections appear.
---
## Conversions sync configuration
Choose **Conversions** when sending conversion events to Impact.com.
#### 1. Campaign ID
Select a value that identifies the program/campaign on Impact.com. You can:
- Select a campaign ID returned from Impact.com, or
- Toggle **Use column** to map a value from your model
This ID is required for all conversion event syncs.
#### 2. Event attribution keys
These fields identify **who or what** generated the conversion.
At least one attribute is required.
Common attribution keys include:
- Email (may require automatic hashing)
- Customer ID
- Click ID
- Partner or affiliate ID
- Custom fields configured in Impact.com
You can add multiple attribution fields.
#### 3. Event ID mapping
This mapping determines the **type of event** being sent.
Event types are created and managed in Impact.com under:
**Event Types → Actions → View/Edit → “Codes”**
Map at least one column that identifies the event type.
Examples include: `purchase`, `lead`, `signup`, or custom codes defined in Impact.com.
#### 4. Event timestamp
Choose a column that contains the timestamp of the event.
If you leave this unmapped, Impact.com will use the server arrival time.
Hightouch converts timestamps automatically to the format required by Impact.com (ISO 8601 or Unix milliseconds).
#### 5. Additional event information
Use this section to include extra metadata with your event, such as:
- Order amount
- Currency
- Product identifiers
- SKU or item lists
- Discounts
- Customer metadata
- Device or session information
If you need to include multiple items in the same conversion, manually create
additional fields by appending the index (e.g. `ItemName1`, `ItemName2`,
`ItemName3`).
Each mapping section requires at least one field. Providing rich event
metadata leads to better reporting in Impact.com.
Learn more about the Impact.com Conversion object in their [API documentation](https://integrations.impact.com/impact-brand/reference/the-conversion-object).
#### 6. Automatic PII hashing
Impact.com often requires that PII such as email addresses be **SHA-1 hashed**.
Toggle **Yes** to allow Hightouch to automatically detect and hash fields when required.
Fields commonly hashed include:
- Customer Email
- Partner identifiers that contain PII
Only fields required by Impact.com will be hashed.
---
## Object sync configuration
Choose **Objects** when syncing **catalogs** or **actions**.
After selecting **Objects**, choose the Impact.com object:
- **Catalog**
- **Action**
Each object type has its own configuration sections.
---
### Catalog object configuration
Choose **Catalog** when syncing product catalog data such as SKUs, descriptions, or prices.
#### 1. Record update mode
Choose how Hightouch should write records:
- **Update** — update existing catalog items
- **Insert** — create new items without updating existing ones
Use **Update** if you want ongoing sync behavior.
#### 2. Catalog ID
Choose which catalog to sync to.
You can:
- Select a catalog returned by Impact.com
- Enter a static ID
- Toggle **Use column** to map a dynamic catalog ID
If your catalog does not appear, ensure it is activated and available in Impact.com.
#### 3. Record matching
Define how Hightouch should match records in your model to the records in Impact.com.
Impact.com typically matches catalog entries using an identifier such as:
- `Item ID`
- `SKU`
- `Product Code`
The left side of your mapping is your model field; the right side is the corresponding Impact.com identifier field.
Example:
| Model field | Impact.com field |
| ----------- | ---------------- |
| `sku` | `Item ID` |
Incorrect identifiers will cause mismatched updates or unintended new record creation.
#### 4. Catalog field mapping
Map your model columns to Impact.com catalog fields.
Common catalog attributes include:
- Product name
- Category
- Price
- Brand
- Description
- Primary image URL
- Inventory status
- Custom metadata fields
You may add as many mappings as needed.
#### 5. Handling deleted records
Choose what happens when a row disappears from your query result:
- **Do nothing** — Hightouch leaves the Impact.com record unchanged
- **Delete the Impact.com record**
- This permanently removes the catalog entry from Impact.com
- This cannot be undone
Choose deletion only if you want your catalog in Impact.com to stay in sync with your dataset.
---
### Action object configuration
Choose **Action** when syncing Impact.com Actions (non-conversion activities).
#### 1. Sync modes
The following modes are available:
**Operation**
- **Update** — update existing action records
- **Reverse** — reverse previously submitted action data
- **Insert** — create new action records (if supported)
**Match mode**
Choose which identifiers are used to match action records, such as:
- `Action ID`
- Custom external IDs
These must correspond to identifiers configured in your Impact.com action settings.
#### 2. Action field mapping
Map model fields to the relevant Impact.com action fields.
Common fields include:
- Action ID
- Action amount
- Status
- Event/action code
- Custom metadata
- Any additional fields your Impact.com account is configured to accept
#### 3. Disposition code (metadata)
You can optionally map a Disposition Code, which determines how Impact.com categorizes the action (e.g., approved, rejected, pending).
You can:
- Select a static value
- Toggle **Use column** to map a value from your model
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## InMobi
**URL:** https://hightouch.com/docs/destinations/inmobi
**Description:** InMobi help brands understand, identify, engage and acquire consumers.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------- | ---------------------------------- | -------------------- |
| **Audiences** | Create and update custom audiences | All |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to InMobi
This integration requires prior approval from the InMobi team and the Hightouch
team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your InMobi rep who will verify approval. Once we have confirmed, we will
unlock the integration for you to connect.
To connect, input the following provided by your InMobi rep:
- Access key
- Secret key
- Bucket name
### Syncing custom audiences
Hightouch will upload a file to the specified bucket with a naming pattern agreed upon by Hightouch and InMobi for your use case.
#### User identifiers
To identify which users to add or update in an audience, select model columns
and the corresponding InMobi fields. You can match on any of the following user
fields:
- MAID (preferred)
- Email
- IP Address
- Ramp ID
- ID5
#### Handling PII and hashing
InMobi expects **email** values to be hashed. For audience syncs, by default, Hightouch automatically hashes first-party user data fields before sending requests to Reddit.
If the data in your model is already hashed, you can disable PII hashing to successfully send data to Reddit.
#### Delete behavior
The only supported sync mode is `all` mode, meaning that each sync run will refresh
the entire audience. Users removed from your audience will automatically be dropped
with each refresh - only present users will be in InMobi's audience.
## Tips and troubleshooting
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Live debugger
### Sync alerts
---
## Insider InOne
**URL:** https://hightouch.com/docs/destinations/insider
**Description:** InOne brings together all technologies marketers need to deliver personalized experiences under a single, highly usable platform. InOne makes the technology so seamless that it truly blends into the daily lives of marketers, fully empowering them behind the scenes.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes |
| ----------------- | -------------------------------------------------- | ---------------------- |
| **Users** | Sync data from any source to Insider users | Upsert, Update, Insert |
| **Events** | Sync data from any source to Insider events | Insert |
| **Segments** | Sync data from any source to Insider segments | Add, Remove |
| **Subscriptions** | Sync data from any source to Insider subscriptions | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Insider InOne
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Insider** and click **Continue**. You can then authenticate Hightouch to **Insider**.
Enter the following required fields into Hightouch:
- **Partner name**:This is your partner name. Navigate to InOne > InOne Settings > [Account Preferences](https://academy.useinsider.com/docs/account-preferences) to copy your partner name. The partner name should be lowercase.
- **Request token**:This key is required to authorize your request. Refer to [API Authentication Tokens](https://academy.useinsider.com/docs/api-authentication-tokens#generate-api-key) to generate your token.
## Sync configuration
Once you've set up your Insider destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Insider destination you want to sync to.
### Syncing users
Sync data from any source to Insider users.
#### Record matching
To match rows from your model to user in Insider, you need to select a model column and corresponding Insider field. You can match on any of the following Insider fields:
- **Email**
- **Phone number**
- **UUID**
In **Insert** mode, Insider automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync user fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom user fields.
If you send data for a custom field that doesn't exist, Hightouch adds the field.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | --------------------------------------------------------- |
| **Do nothing** | Keep the user in Insider with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the user in Insider |
### Syncing events
Sync data from any source to Insider events.
#### Record matching
You can match rows from your model to event in Insider on any column in your model and any field in Insider. Ensure the data types of the model column and Insider field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
### Syncing segments
Sync data from any source to Insider segments.
#### List field
The name of an array field that has already been created in Insider InOne. This field will contain a list of segments the user belongs to.
#### Segment name
The name of the segment the sync will handle subscriptions for. This segment name will be added to or removed from the list field entered above as users' subscription statuses change.
#### Record matching
To match rows from your model to segment in Insider, you need to select a model column and corresponding Insider field. You can match on any of the following Insider fields:
- **Phone number**
- **UUID**
#### Field mapping
Hightouch lets you sync segment fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default segment fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
### Syncing subscriptions
Sync data from any source to Insider subscriptions.
#### Record matching
To match rows from your model to subscription in Insider, you need to select a model column and corresponding Insider field. You can match on any of the following Insider fields:
- **Email**
- **Phone**
#### Field mapping
Hightouch lets you sync subscription fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default subscription fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Intercom
**URL:** https://hightouch.com/docs/destinations/intercom
**Description:** Sync product and CRM data to Intercom so that your support team has all the information they need about customers
**Section:** Destinations
## Supported syncing
Hightouch can create and update both contacts and company objects within the [Intercom Contacts Model](https://developers.intercom.com/intercom-api-reference/reference#contacts-model) and [Intercom Company Model](https://developers.intercom.com/intercom-api-reference/reference#company-model) respectively.
|Sync Type|Description|Supported Sync Modes | API Reference|
|--|--|--|--|
|**Contact objects**|Sync data from any source to Intercom contacts.|Upsert, Update, Archive|[Contact docs](https://developers.intercom.com/docs/references/rest-api/api.intercom.io/Contacts/CreateContact)|
|**Company objects**|Sync data from any source to Intercom company objects.|Upsert, Update|[Company docs](https://developers.intercom.com/intercom-api-reference/reference/createorupdatecompany)|
|**Events**|Sync records as events to Intercom. This is often in the form of a track call.|Insert|[Event docs](https://developers.intercom.com/intercom-api-reference/reference/createdataevent)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Intercom
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Intercom** and click **Continue**. You can then authenticate Hightouch to Intercom with OAuth. Click **Log in to Intercom** and log into your Intercom account.
Make sure you select the region that corresponds with your Intercom workspace from the *Data host region:* dropdown:
Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Intercom destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Intercom destination you want to sync to.
### Syncing contacts
Contact objects allow you to represent and provide information on individual users. All contact objects are represented as users or leads as marked by the role attribute.
When looking at the [People list](https://www.intercom.com/help/en/articles/318-the-people-company-and-account-list-explained#the-people-list) or when [creating an audience](https://www.intercom.com/help/en/articles/2848161-create-more-targeted-audiences-with-and-or-rules), the number of contacts (users) displayed might differ from the number of rows processed by your sync. This is because Intercom [automatically filters out unsubscribed contacts](https://www.intercom.com/help/en/articles/5537581-include-unsubscribed-contacts-in-your-email-audience#h_c8fcd619b4) from your contact list.
#### Record matching
To match rows from your model to objects in Intercom, you need to select a model column and corresponding Intercom field. You can match on any of the following Intercom fields:
- Email
- External User ID
#### Field mapping
You can map data from any of your model columns to contact fields in Intercom. Ensure the data type of your model column matches the data type of the field you want to sync to.
#### Associations
You can associate companies to contacts using this configuration section. To do so, select a column from your model to match the associated object. Hightouch uses this column to look up the associated object's ID and make the connection between the two objects.
For example, if you wanted to associate a company to your contact, you select **Company** from the right dropdown. Then, select a field to match the company on, for example Company ID.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the contact in Intercom |
| **Clear fields** | Keep the contact in Intercom but clear the synced fields |
| **Remove from list** | Remove the contact from Intercom |
### Syncing companies
Company objects allow you to represent and provide information on organizations using your product. Each company has its own description and can be associated with contacts.
Company objects' Company ID can't be updated. If you need to change a company object's Company ID you need to delete it and recreate it with a new Company ID.
When syncing company objects, make sure they have at least one [associated contact](#associations-1). Otherwise, the company won't be visible in Intercom.
#### Record matching
To match rows from your model to records in Intercom, you need to select the model column that contains values that match the **Company ID** field. If there isn't a company with the matching Company ID, Hightouch creates a new company.
#### Field mapping
You can map data from any of your model columns to company fields in Intercom. Ensure the data type of your model column matches the data type of the field you want to sync to.
#### Associations
You can associate contacts to companies using this configuration section. To do so, select a column from your model to match to the associated object. Hightouch uses this column to look up the associated object's ID and make the connection between the two objects.
For example, if you wanted to associate a company to your contact, you select **Contact** from the right dropdown. Then, select a field to match the contact on, for example `user_id`.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the company in Intercom |
| **Clear fields** | Keep the company in Intercom but clear the synced fields |
| **Remove from list** | Remove the company from Intercom |
### Syncing events
When syncing events, Hightouch treats any records added to [your source](/getting-started/concepts) as new events and sends them to Intercom when your sync runs.
#### Event name
Providing an event name is required to log an event in Intercom. You can either provide a static value or select to **use a column** from your model.
Refer to [Intercom's event docs](https://developers.intercom.com/intercom-api-reference/reference/createdataevent) for guidance on event naming.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server.
#### Field mapping
You can select model columns to sync to native and custom event parameters in the field mapping section.
## Tags
You can manage tags on companies and contacts in Intercom using Hightouch.
The data in your source should contain the name of one or more tags for each row.
For multiple tags, your source can contain an array or a comma-separated string of tag names.
#### Tag Modes
Syncs can have different tag modes based on how you would like to handle tags.
##### **Create added tags**
When there are tags added to the source, create the tags in Intercom. If the tag does not already exist, it will be created in Intercom. If the tag already exists, it will be added to the contact or company. Do nothing for tags removed from the source.
##### **Overwrite existing tags**
When there are changes to tags in the source, **remove all tags** in Intercom before creating the tags present in the source. If there are no changes to the tags, do nothing. Because this removes all tags in Intercom, this may delete data not authored by Hightouch.
##### **Delete removed tags and create added tags**
When there are changes to tags in the source, removes tags in Intercom that have been removed from the source, and creates tags that have been added to the source.
Read more about how Intercom handles tags [here](https://developers.intercom.com/docs/references/rest-api/api.intercom.io/tags).
## Tips and troubleshooting
### Common errors
{/* */}
#### 429 Rate limit exceeded
The full error message looks something like this:
```
Custom data attributes could not be created: StatusCodeError: 429 -
{
"type": "error.list",
"request_id": "003drrt76xkssahyhgd41",
"errors":[{
"code": "rate_limit_exceeded",
"message": "Exceeded rate limit of 167 in 10_seconds"
}]
}
```
This error means you've exceeded [Intercom's rate limit](https://developers.intercom.com/intercom-api-reference/reference/rate-limiting).
One reason for this could be running two Hightouch syncs to Intercom simultaneously.
Try staggering the syncs so that they don't overlap.
{/* */}
#### Missing companies
If your Hightouch sync processes successfully, but you don't see the companies
in Intercom, check if there is at least one contact [associated to the company](#associations-1). Intercom only displays companies with at least one associated contact. For more information, read the Intercom [company API documentation](https://developers.intercom.com/intercom-api-reference/reference/create-or-update-a-company).
### Live debugger
### Sync alerts
---
## iSpot.tv
**URL:** https://hightouch.com/docs/destinations/ispot-tv
**Description:** Empower your brand, marketing, and advertising teams to make data-driven decisions with iSpot.tv’s TV measurement solutions.
**Section:** Destinations
## Overview
Onboard your first-party audiences to iSpot.tv to provide visibility into tune-in
metrics across your audience segments and illuminate unique media buying
opportunities.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| --------------- | ---------------------------------- | -------------------- |
| **Audiences** | Create and update custom audiences | All |
| **Conversions** | Upload TV conversion data | Insert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to iSpot.tv
This integration requires prior approval from the iSpot.tv team and the Hightouch
team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your iSpot rep who will verify approval. Once we have confirmed, we will unlock
the integration for you to connect.
To connect, input the `Advertiser Name` provided by your iSpot rep.
### Syncing custom audiences
You can input the name of the audience you plan on delivering to iSpot.
#### User identifiers
To identify which users to add or update in an audience, select model columns
and the corresponding iSpot fields. You can match on any of the following user
fields:
- Email (Sha256 hashed)
- MAID
- IP Address
#### Delete behavior
The only supported sync mode is `all` mode, meaning that each sync run will refresh
the entire audience. Users removed from your audience will automatically be dropped
with each refresh - only present users will be in iSpot's audience.
### Syncing conversions
Sync data from any source to iSpot.tv as conversion events.
#### Field mapping
Hightouch lets you sync event properties via field mappings.
## Tips and troubleshooting
### Refresh rate
iSpot.tv has a max refresh rate of monthly. You should not have your iSpot syncs
running more than every 4 weeks.
### Matched device count
Match rates are not available to preview in the Hightouch platform. Contact your
iSpot rep for insights into match rates and audience reach.
Keep in mind that audience counts reflect the number of devices match, not
the number of users matched. Also keep in mind that matching occurs at the
household level, and that a single household may own multiple devices.
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Live debugger
### Sync alerts
---
## Iterable
**URL:** https://hightouch.com/docs/destinations/iterable
**Description:** Run better email, push, and SMS campaigns on Iterable with up-to-date customer data from your data warehouse
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API reference |
| ------------------------------------------------------------- | ------------------------------------------------------------------------------ | -------------------- | ------------------------------------------------------------------------------------------------------------ |
| [**Users**](#syncing-users) | Sync data from any source to your Iterable users | Upsert, Update | [User docs](https://api.iterable.com/api/docs#users_bulkUpdateUser) |
| [**Events**](#syncing-events) | Sync records as events to Iterable. This is often in the form of a track call. | Upsert | [Event docs](https://api.iterable.com/api/docs#events_track) |
| [**User shopping cart items**](#updating-shopping-cart-items) | Update a user's shopping cart and create an event for it | Insert | [Commerce docs](https://api.iterable.com/api/docs#commerce_updateCart) |
| [**Catalog items**](#syncing-catalogs) | Sync records as catalog items in Iterable. | Upsert | [Catalog docs](https://api.iterable.com/api/docs#catalogs_partialUpdateCatalogItem) |
| [**Static list**](#syncing-segments) | Subscribe and unsubscribe users from lists | Add, Remove | [List docs](https://api.iterable.com/api/docs#lists_subscribe) |
| [**Update email address**](#updating-user-email-addresses) | Update a user's email address | Update | [User email update docs](https://api.iterable.com/api/docs#users_updateEmail) |
| [**Other user actions**](#user-api-actions) | Perform API actions like delete, forget and unforget users. | -- | [User docs](https://api.iterable.com/api/docs#users_delete) |
| [**Subscriptions**](#subscribing-users) | Subscribe and unsubscribe users from lists, message types and message channels | Upsert, Remove | [Subscription docs](https://api.iterable.com/api/docs#subscriptions_Bulk_subscription_action) |
| [**Trigger email**](#trigger-email) | Send an email to a user | -- | [Email docs](https://api.iterable.com/api/docs#email_target) |
| [**Trigger push**](#trigger-push) | Send a push to a user | -- | [Push notification docs](https://api.iterable.com/api/docs#push_target) |
| [**Trigger in-app**](#trigger-in-app) | Send an in-app message to a user | -- | [In-app docs](https://api.iterable.com/api/docs#In-app_target) |
| [**Trigger SMS**](#trigger-sms) | Send an SMS message to a user | -- | [SMS docs](https://api.iterable.com/api/docs#SMS_target) |
| [**Trigger journey (formerly workflow)**](#trigger-journey) | Trigger a journey for a user | -- | [Journey docs](https://api.iterable.com/api/docs#workflows_triggerWorkflow) |
| [**Merge users**](#merge-users) | Merge two Iterable user profiles into one. | Insert | [Merging user profiles](https://support.iterable.com/hc/en-us/articles/33744314673172-Merging-User-Profiles) |
| [**Register device tokens**](#register-device-tokens) | Register a device to a user using a token | Insert | [Device token docs](https://api.iterable.com/api/docs#users_registerDeviceToken) |
## Connect to Iterable
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Iterable** and click **Continue**. You can then authenticate Hightouch to Iterable with an Iterable **API key** and [**Project Type Identifier**](https://support.iterable.com/hc/en-us/articles/9216719179796#checking-the-unique-identifier-for-a-project).
You can follow [Iterable's instructions](https://support.iterable.com/hc/en-us/articles/360043464871-API-Keys-#creating-api-keys) for how to create an API key. Your API key needs to have [server-side permissions](https://support.iterable.com/hc/en-us/articles/360043464871-API-Keys-#server-side-keys) for all sync types above, except for **Register device tokens** which needs to have [mobile client-side permissions](https://support.iterable.com/hc/en-us/articles/360043464871-API-Keys#client-side-keys). Hightouch does not support mobile client-side permissions using JSON web tokens (JWT).
To see your project type identifier, go to **Settings > Project Settings** and see the identifier(s) underneath the project's cluster ID.
Check out Iterable's [docs](https://support.iterable.com/hc/en-us/articles/9216719179796#checking-the-unique-identifier-for-a-project) for more information.
## Sync configuration
Once you've set up your Iterable destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Iterable destination you want to sync to.
### Syncing users
Sync Iterable users and user properties with data from any source.
#### Record matching
You can either use **email** or **user ID** to match rows from your model to users in Iterable.
In general, it's best to select the one that corresponds to the [project type](https://support.iterable.com/hc/en-us/articles/9216719179796) you selected when first setting up your Iterable instance.
There's one exception when you _shouldn't_ match your record matching column to your project type:
If you have an **email-based** Iterable project and you're using **update** mode, selecting **user ID** for record matching can increase sync speed.
You should do this if you're confident that your user IDs are unique. (In email-based projects, Iterable doesn't require uniqueness for user IDs.)
Ensure you [set your project identifier](#unset-project-identifier) appropriately.
If you select user ID for record matching in email-based projects, Hightouch uses the [bulk user update endpoint](https://api.iterable.com/api/docs#users_bulkUpdateUser) with the `preferUserId` parameter.
The `preferUserId` parameter dictates whether new users should be created if the request includes a `userId` that doesn't yet exist in the Iterable project.
This allows Hightouch to update only existing users without first having to perform a [lookup](https://api.iterable.com/api/docs#users_getUserById_0).
Iterable only respects the `preferUserId` parameter in API calls for email-based projects.
For more information on how Iterable's API works differently according to project-type, refer to [Iterable's docs](https://support.iterable.com/hc/en-us/articles/9216719179796#email-based-projects).
**Upsert** mode is performant regardless of the project-type and column used
for record matching.
#### Field mapping
You can sync columns from your model to Iterable user's data fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
When creating new fields, Iterable automatically generates a schema for that
field based on the first data sent. You cannot change the schema once it's
set. See [Iterable's
article](https://support.iterable.com/hc/en-us/articles/206430145-User-Profile-Fields-#data-types-for-user-profile-fields)
for more information.
#### Nested objects and object arrays
Iterable supports user and event fields containing [JSON objects](https://support.iterable.com/hc/en-us/articles/208183076-Field-Data-Types#objects) or [arrays of objects](https://support.iterable.com/hc/en-us/articles/208183076-Field-Data-Types#arrays).
These objects and object arrays are commonly used to support "entities" related to each user—for example, devices, medications, pets, etc.—especially when marketing communication needs to include metadata about these related entities.
For example, say you want to sync an array of a customer's pets to be an attribute named `pets` in Iterable:
```json
{
"email": "docs@hightouch.com",
"dataFields": {
"pets": [
{
"name": "Nacho",
"color": "Ginger",
"age": 1,
"favoriteToy": "Fuzzy spring"
},
{
"name": "Rami",
"color": "Brown Tabby",
"age": 2,
"favoriteToy": "Feather wand"
}
]
}
}
```
Hightouch's [inline mapper](/syncs/mapping-data#inline-mapping) is purpose-built to create nested JSON objects and arrays like this.
Refer to the [data format requirements](/syncs/mapping-data#required-data-format) and [array creation](/syncs/mapping-data#create-an-array) instructions for details.
#### Date fields
Iterable's API expects date and datetime values in a specific format (`YYYY-MM-DDTHH:MM:SS.sss-00:00`). When you map a model column to an Iterable field that Iterable declares as a `date` or `datetime` type, Hightouch automatically converts the value, including string-typed values from your warehouse (for example, ISO strings such as `2026-04-07T00:00:00.000Z` from Snowflake). You don't need a [Liquid template](/syncs/mapping-data#template-mapping) to reformat date columns.
This applies wherever Iterable user data fields are mapped, including [purchase events](#syncing-events) and [updating shopping cart items](#updating-shopping-cart-items).
#### Delete behavior
You have these options for how to treat records that leave your model's query results:
| Behavior | Description |
| -------------- | -------------------------------------------------------------------- |
| **Do nothing** | Keep the users in Iterable unchanged |
| **Clear** | Keep the user in Iterable and clear data fields managed by Hightouch |
| **Delete** | Delete the user from Iterable |
### Syncing events
The integration lets you sync [custom events](https://api.iterable.com/api/docs#commerce_trackPurchase), [purchase events](https://api.iterable.com/api/docs#commerce_trackPurchase), and `updateCart` events to Iterable.
This section describes custom and purchase events. Skip to the [syncing update shopping cart items](#updating-shopping-cart-items) for more information on this sync type.
When syncing custom and purchase events, the integration inserts new events into Iterable and lets you modify existing events as long as you [provide their event ID](#additional-track-configuration).
#### Event name
Providing an event name is required to send an event to the Iterable API.
You can either provide a static value or select to **use a column** from your model.
Hightouch syncs the static or column value as the [`eventName` body parameter](https://api.iterable.com/api/docs#events_trackBulk) the API requires.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred.
If this field is empty, Iterable uses the time the event arrives at the server.
#### Additional track configuration
You can also optionally include this information as part of your event information:
- Name of the campaign associated with the event: choose a static value or select to **use a column** from your model.
- Event template: choose a static value or select to **use a column** from your model.
- Event ID: select a column from your model. You **must** include this configuration if you want to modify existing events.
If you don't select a column from your model to map as the event ID,
Hightouch considers it a new event and Iterable generates a new event ID.
#### Field mapping
You can use field mapping to add context to your events.
You must include either an **email** or **user ID** in your mappings.
Which you include depends on [how your project identifies users](https://support.iterable.com/hc/articles/204780579#identifying-users).
If you provide both email and user ID, your project type takes precedence.
The integration uses these fields for identifying the user associated with this event.
For purchase events, you also need to provide the shopping cart items as **Items** and the **Total** order dollar amount in your mappings.
You can also include event data fields, including custom ones. Hightouch sends any custom fields you include as part of the in the `dataFields` property of event's body.
### Updating shopping cart items
This sync mode both updates the [`shoppingCartItems`](https://api.iterable.com/api/docs#commerce_updateCart) field on a user's profile and tracks `updateCart` events in Iterable.
The integration inserts new events into Iterable when they first appear in your query results.
#### Field mapping
You must include shopping cart items as **Items** and either an **email** or **user ID** in your mappings.
Which you include depends on [how your project identifies users](https://support.iterable.com/hc/articles/204780579#identifying-users).
If you provide both email and user ID, your project type takes precedence.
The integration uses these fields for identifying the user associated with this event.
You can also include user data fields for the user who updated their cart.
### Syncing catalogs
You can keep your Iterable catalogs up-to-date with **Upsert** mode.
Begin by selecting the Iterable catalog you want to sync items to.
#### Record matching
To match rows from your model to catalog items, you need to select the model column that contains values that match the **Unique Product ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
The Unique Product ID must contain only alphanumeric characters and dashes and
has a maximum length of 255 characters. See Iterable's [API
documentation](https://api.iterable.com/api/docs#catalogs_bulkUpdateCatalogItems)
for more information.
#### Field mapping
Once you've selected a catalog, Hightouch automatically pulls existing fields from your Iterable catalog for you to map data into.
#### Delete behavior
You have these options for how to treat records that leave your model's query results:
| Behavior | Description |
| -------------- | -------------------------------------------- |
| **Do nothing** | Keep the catalog items in Iterable unchanged |
| **Delete** | Delete the catalog item in Iterable |
### Syncing segments
Sync audiences into your Iterable [static lists](https://support.iterable.com/hc/en-us/articles/204780549-Static-and-Dynamic-Lists-Overview-) of users. Hightouch automatically removes users from the Iterable list when they leave the query result.
#### User identifiers
To identify which contacts to add or update in a segment, select a model column and the corresponding Iterable field.
You can match on either **Email** or **User ID**.
It's best to select the one that corresponds to the [project type](https://support.iterable.com/hc/en-us/articles/9216719179796) you selected when first setting up your Iterable instance.
If you provide both email and user ID, your project type takes precedence.
#### Field mapping
You can sync columns from your model to user's default and custom data fields.
If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
#### Create or use an existing list
Your can create a new list or select an existing one.
If you choose to create a new list, Hightouch creates it on the initial sync run and then keeps it updated on existing sync runs.
You can provide a name for the new list, or Hightouch defaults to the name of the model used to create the sync.
### Updating user email addresses
Use this sync mode to update existing users' email addresses in [email-based projects](https://support.iterable.com/hc/articles/9216719179796).
To update an `email` in a userID-based or hybrid project, use the [user sync](#syncing-users) type.
#### Record matching
You can use either **email** or **user ID** to match rows from your model to users in Iterable, regardless of your Iterable project type.
If you match by email, the update call is only made the first time the record appears in your query results.
Matching by user ID updates the email the first time the record appears in your query results and every time changes are detected.
#### Field mapping
The only field you can include in field mapping is the new email.
If you want to update other user data fields, use the [user sync type](#syncing-users).
### User API actions
Hightouch lets you perform the following actions on your Iterable users:
- [Delete](https://support.iterable.com/hc/en-us/articles/360029174171-Responding-to-GDPR-Requests-#deleting-users)
- [Forget](https://support.iterable.com/hc/en-us/articles/360029174171-Responding-to-GDPR-Requests-#forgetting-users)
- [Unforget](https://support.iterable.com/hc/en-us/articles/360029174171-Responding-to-GDPR-Requests-#unforgetting-users)
This destination only triggers delete, forget, or unforget when records
*first* appear in your model. In other words, if a row with the same [primary
key](/models/creating-models#unique-primary-key-requirement) appears in the
model's query results in multiple syncs, Hightouch only triggers the selected
API action on it the first time.
#### Record matching
You can either use **email** or **user ID** to match rows from your model to users in Iterable.
It's best to select the one that corresponds to the [project type](https://support.iterable.com/hc/en-us/articles/9216719179796) you selected when first setting up your Iterable instance.
### Subscribing users
Set all existing users from a query to be subscribed to a specific list, message type or message channel.
Hightouch automatically unsubscribes users from the target when they leave the query result.
### Trigger email
Send an email to a user. Use an existing campaign that is triggerable and running.
### Trigger push
Send a push notification to a user. Use an existing campaign that is triggerable and running.
### Trigger in-app
Send an in-app message to a user. Use an existing campaign that is triggerable and running.
### Trigger SMS
Send an SMS message to a user. Use an existing campaign that is triggerable and running.
### Trigger journey
Iterable previously called these flows **Workflows**, and the API still uses
workflow terminology (for example, `workflowId` and the `triggerWorkflow`
endpoint).
Trigger a Journey for a user in Iterable. This behaves similarly to triggering an email or push campaign. You must supply a user identifier (email or userId), and optionally include Journey-specific trigger properties if configured in Iterable.
### Merge users
Use this sync type to merge one Iterable user profile into another. Each row in your model represents a single merge operation.
Iterable removes the source user after the merge and retains the destination user.
See Iterable’s documentation on [Merging User Profiles](https://support.iterable.com/hc/en-us/articles/33744314673172-Merging-User-Profiles) for details on how merges are handled.
Merges in Iterable are irreversible.
#### Record matching
This sync type requires two identifiers:
- **Source user** — the user to merge _from_
- **Destination user** — the user to merge _into_
For each, select a model column and map it to either **email** or **userId** in Iterable. Both identifiers must be present in every row.
#### Field mapping
This sync type does not support field mapping.
The identifiers you configure are sent to Iterable as the `fromUser` and `intoUser` values.
### Register device tokens
Register mobile device tokens to enable push notifications. Use either `email` or `userId` (but not both) and a `device`
object that contains a valid `token`, `platform` and `applicationName`.
#### User identifiers
To identify which contacts to add or update in a segment, select a model column and the corresponding Iterable field.
You can match on either **Email** or **User ID**, regardless of your project type.
## Tips and troubleshooting
For Hightouch platform error codes related to Iterable, see [Error codes:
Iterable](/developer-tools/error-codes#other-destinations).
### Unset project identifier
Before March 2023, Hightouch didn't require you to set your Iterable project identifier [when connecting to Iterable](#connect-to-iterable).
Setting it may enhance sync performance if you're using **update** mode with the **users** sync type for an email-based project.
See the [record matching section](#record-matching) for more information.
You can set the project-type from the configuration tab of your Iterable destination in Hightouch.
### Common errors
#### Fewer records in Iterable than synced
If you see a discrepancy between the number of records that have passed through your sync and the number of records displaying in Iterable, check your data set for duplicates. Iterable automatically dedupes records based on primary key so the number of records in Iterable might be less than the number of records in your Hightouch sync if your sync unintentionally contains duplicates.
{/* */}
#### 413 - Request Entity Too Large
If you see this error or notice that your syncs are slow, you can try lowering the batch size in your sync configuration.
#### 400 - Request does not have the same data types as the data previously submitted
{/* */}
The full error message looks something like this:
```json
{
"msg": "Project 13691: The request does not have the same data types as the data previously submitted, or the input value is not within the correct range. Field 'date_field' already exists for type 'user' and has a data type of 'date' but possible types 'string, keyword' in the request.",
"code": "RequestFieldsTypesMismatched",
"params": {
"validationErrors": {
"date_field": {
"incomingTypes": ["string", "keyword"],
"expectedType": "date",
"category": "user",
"offendingValue": ["2023-04-19 12:15:30", "0000-00-00 00:00:00"],
"_type": "UnexpectedType"
}
}
}
}
```
The first value in the `offendingValue` array is associated with the first record in the batch sent to Iterable. The second value in the array is the value causing the issue. One instance of this value (`"0000-00-00 00:00:00"`) causes the entire batch of records being sent to Iterable to fail.
To resolve the error, check for instances of (`"0000-00-00 00:00:00"`) in the `date_field` column of your model and replace these values with `null` or a valid date.
### Live debugger
### Sync alerts
---
## Jira
**URL:** https://hightouch.com/docs/destinations/jira
**Description:** Keep your team's projects and tickets up-to-date with fresh customer data
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
| ---------- | ------------------------------------------------------------------- | ---------------------- | ------------------------------------------------------------------------------------------------------- |
| **Issues** | Sync data from any source to Jira issues or Product Discovery ideas | Upsert, Update, Insert | [Issues](https://developer.atlassian.com/cloud/jira/platform/rest/v2/api-group-issues/#api-group-issues)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Jira
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Jira** and click **Continue**. You can then authenticate Hightouch to Jira by entering your **Jira Domain**, **Username**, and **API Token** from your Jira account:
- the **Jira Domain** is the text before `.atlassian.net` in the URL,
- **Username** is the email you use to login to Jira,
- and if you don't already have an **API Token**, you can create one in your Jira account by navigating to **Account Settings** > **Security** > **Create and manage API tokens**.
## Sync configuration
Once you've set up your Jira destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Jira destination you want to sync to.
### Syncing issues and ideas
Hightouch supports syncing to Jira [issues](https://support.atlassian.com/jira-software-cloud/docs/what-is-an-issue/) and [Product Discovery ideas](https://support.atlassian.com/jira-product-discovery/docs/what-is-an-idea/). If you want to sync to Product Discovery ideas, select a [**Product Discovery Project**](https://www.atlassian.com/software/jira/product-discovery) in your sync configuration and use [Insert](#supported-syncing) mode.
#### Record matching
Upsert and Update mode are only supported for Jira issues. They require you to create a **custom
field** in Jira to use for record matching in Hightouch. This field needs to be set to the [**Text Field (read only)**](https://support.atlassian.com/jira-cloud-administration/docs/custom-fields-types-in-company-managed-projects/#Advanced-custom-field-types) type.
To create a **custom field** in Jira:
- Navigate to **Settings** > **Issues** > **Custom Fields**.
- Click **Create custom field** and select the [**Text Field (read only)**](https://support.atlassian.com/jira-cloud-administration/docs/custom-fields-types-in-company-managed-projects/#Advanced-custom-field-types) field type.
Ensure all issue fields and custom fields that you want to sync to are
included on your project's screen. To add custom fields to the
screen, navigate to the custom field, click the three-dot menu and then **Associate to Screens**.
#### Field mapping
You can map model columns to all Jira issue fields that are available to the **Project** and **Issue Type** you selected in your sync configuration. To sync successfully, ensure the **Description** field is in [Atlassian Document Format](https://developer.atlassian.com/cloud/jira/platform/apis/document/structure/) and that these other fields are of these types:
- `parent` > `id`
- `priority` > `name` (for example, Highest, High, Medium, Low, or Lowest)
- `creator` > `accountId`
- `assignee` > `accountId`
- `reporter` > `accountId`
Hightouch doesn't currently support syncing to [Status fields](https://confluence.atlassian.com/adminjiraserver/defining-status-field-values-938847108.html), so this needs to be done manually in the Jira dashboard. Please {" "} if you're interested in this feature.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Kameleoon
**URL:** https://hightouch.com/docs/destinations/kameleoon
**Description:** Empower rich personalization
**Section:** Destinations
## Overview
In Hightouch, you can easily sync data from any source to visitors in Kameleoon, much like with other platforms such as Salesforce and Meta Ads. Simply connect your data source, specify the syncing models, and create them directly within Hightouch using SQL, our user-friendly audience builder, or compatible tools like dbt or Looker.
## Supported syncing
| Type | Description | Supported Sync Modes |
| ---------------------- | -------------------------------------------------- | -------------------- |
| **Visitor properties** | Sync data from any source to visitors in Kameleoon | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Kameleoon
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination**
button. Select **Kameleoon** and click **Continue**.
Enter the following required fields into Hightouch:
- **Project ID** (Site Key)
- **Client ID**
- **Client Secret**
Follow [these instructions](https://help.kameleoon.com/api-credentials/) to retrieve your API keys.
## Sync configuration
Once you've set up your Kameleoon destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and
the Kameleoon destination you want to sync to.
#### Record matching
You must identify a unique key from your model to use as a property key in Kameleoon. This is often a visitor code from
Kameleoon, but any other unique external ID is supported.
#### Field mapping
Select any properties to sync to Kameleoon. These properties can then be used in Kameleoon to create custom segments.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Kayzen
**URL:** https://hightouch.com/docs/destinations/kayzen
**Description:** Kayzen is a mobile DSP that enables app developers, agencies, media buyers, and brands to run programmatic acquisition, retargeting, and branding campaigns.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
--------------|-----------------------------------------------|----------------------|-----------------------------------------------------------------------------------
**Audiences** | Sync data from any source to Kayzen audiences | Add, Remove | [Audiences docs](https://developers.kayzen.io/reference/audience-creation-via-api)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Kayzen
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Kayzen** and click **Continue**. You can then authenticate Hightouch to **Kayzen**.
Enter the following fields into Hightouch:
- **API key**
## Sync configuration
Once you've set up your Kayzen destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Kayzen destination you want to sync to.
### Syncing audiences
Sync data from any source to Kayzen audiences.
#### Record matching
To match rows from your model to audiences in Kayzen, you need to select a model column and corresponding Kayzen field. You can match on any of the following Kayzen fields:
- **IDFA**
- **GAID**
#### Field mapping
Hightouch lets you sync audience fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default audience fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Kevel
**URL:** https://hightouch.com/docs/destinations/kevel
**Description:** Onboard your first-party audiences to Kevel Audience to power onsite and retail media campaigns with segments built from your own data.
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes |
| --------------------------------------- | --------------------------------------------------------------------------------------------------- | -------------------- |
| [**Segment members**](#segment-members) | Upload a segment's full membership to Kevel as a CSV, with the segment's metadata in a leading row. | All |
Kevel replaces a segment's membership in full on every import, so this destination always sends the complete audience. For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Kevel
Kevel provides a pre-created S3 bucket for each Kevel Audience system. Configure a folder source in Kevel to import from that bucket, then point Hightouch at the same bucket.
1. Go to the [Destinations overview](https://app.hightouch.com/destinations?from=docs) page and select **Add destination**.
2. Select **Kevel** from the destination catalog.
3. Enter the following, then select **Continue**:
- **AWS Credentials**: Select or create [AWS credentials](/security/aws) with write access to the bucket.
- **Bucket name**: The Kevel-provided S3 bucket for your Kevel Audience system, or your own bucket registered as a Kevel folder source.
Set up a folder source in Kevel pointing at this bucket so Kevel imports each
file Hightouch uploads. Schedule the import, or configure it to run when new
files are detected.
## Sync configuration
### Segment members
Each sync run uploads one CSV to the configured object key. The file begins with a segment-definition row that carries the segment's metadata, followed by one row per member. The sync mode is **All**: every run writes the segment's complete membership, which Kevel imports as a full replacement.
#### Configure record matching
Kevel matches members by exact identifier value and type, and never matches across types. Configure:
- **Identifier type** — the type Kevel matches on. Hightouch writes it to the file's `userIdType` column:
- **Email**, **Email (SHA-256)**, or **Email (MD5)**
- **Phone (E.164)** or **Phone (E.164, SHA-256)**
- **Android advertising ID (GAID)** or **iOS advertising ID (IDFA)**
- **CRM ID**
- **Identifier column** — the source column that holds the identifier value.
#### Configure identifier hashing
When you select a hashed identifier type (such as **Email (SHA-256)**), Hightouch asks whether to detect and hash your data:
- **Yes** — Hightouch hashes the mapped column to the selected format and leaves values that are already hashed untouched.
- **No** — your column already contains hashed values, so Hightouch sends them as-is.
Plaintext identifier types are always sent unchanged.
#### Describe the segment
Kevel creates the segment on first import and updates its metadata on later imports:
| Field | Required | Notes |
| ----------------------- | -------- | ----------------------------------------------------------------------------------------- |
| **Segment name** | No | Defaults to your model's name. Up to 200 characters. Kevel keys the segment on this name. |
| **Segment category** | No | Up to 20 characters. Cannot start with "Kevel". |
| **Segment description** | No | Up to 500 characters. |
| **Static segment** | No | Mark the segment as static in Kevel. Defaults to unchecked. |
#### Set the object key
Set the **object key** (file path) in the bucket where Hightouch uploads the CSV, for example `segments/high-value-customers.csv`. You can include timestamp variables such as `{YYYY}`, `{MM}`, and `{DD}` in the path.
## Tips and troubleshooting
### Validate your setup
After your first sync completes, confirm the import in Kevel:
- Check that the CSV landed at the configured object key in your bucket.
- In Kevel, confirm the folder source imported the file and that the segment shows the expected member count. Files uploaded to the Kevel-provided bucket are deleted automatically after one month.
### Common errors
### Live debugger
### Sync alerts
---
## Klaviyo
**URL:** https://hightouch.com/docs/destinations/klaviyo
**Description:** Power e-commerce focused email marketing with your warehouse data in Klaviyo
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|------------------------|------------------------------------------------------|----------------------|--------------------------------------------------------------------------------------------|
| **Profiles** | Sync data from any source to user profiles | Upsert | [Profiles docs](https://developers.klaviyo.com/en/reference/spawn_bulk_profile_import_job) |
| **Events** | Sync data from any source to events in Klaviyo | Insert | [Events docs](https://developers.klaviyo.com/en/reference/create_event) |
| **Lists** | Add or remove user profiles from lists in Klaviyo | Add, Remove | [Lists docs](https://developers.klaviyo.com/en/reference/create_list_relationships) |
| **Subscription Lists** | Subscribe and unscribe users from marketing channels | Add, Remove | [Subscription Lists docs](https://developers.klaviyo.com/en/reference/subscribe_profiles) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Klaviyo
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Klaviyo** and click **Continue**. You can then authenticate Hightouch to **Klaviyo**.
### Authenticate with OAuth
For the **Authentication method**, select **Log in to Klaviyo** and log into your Klaviyo account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
### Authenticate with Basic Auth (deprecated)
Enter the following fields into Hightouch:
- **Public API key**
- **Private API key**
## Sync configuration
Once you've set up your Klaviyo destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Klaviyo destination you want to sync to.
### Syncing profiles
Sync data from any source to user profiles.
#### Record matching
To match rows from your model to profiles in Klaviyo, you need to select a model column and corresponding Klaviyo field. You can match on any of the following Klaviyo fields:
- **Email**
- **Phone Number**
- **Klaviyo ID**
### Secondary emails
You may also sync up to five emails to multi-email profiles in Klaviyo. Learn more about multi-email profiles in the Klaviyo [documentation](https://help.klaviyo.com/hc/en-us/articles/48274807144475).
If secondary emails are mapped, then Hightouch will add and remove emails from the profile as columns in your model change.
#### Field mapping
Hightouch lets you sync profile fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom profile fields.
If you send data for a custom field that doesn't exist, Hightouch adds the field.
Ensure your model's columns have the same data types as the fields you want to sync to.
### Syncing events
Sync data from any source to events in Klaviyo.
#### Record matching
You can match rows from your model to events in Klaviyo on any column in your model and any field in Klaviyo. Ensure the data types of the model column and Klaviyo field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
### Syncing lists
Add or remove user profiles from lists in Klaviyo.
#### Record matching
To match rows from your model to lists in Klaviyo, you need to select a model column and corresponding Klaviyo field. You can match on any of the following Klaviyo fields:
- **Email**
- **Phone Number**
- **Klaviyo ID**
#### Field mapping
Hightouch lets you sync list fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default list fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
### Syncing subscription lists
Subscribe and unscribe users from marketing channels.
#### Record matching
To match rows from your model to subscription lists in Klaviyo, you need to select a model column and corresponding Klaviyo field. You can match on any of the following Klaviyo fields:
- **Email**
- **Phone Number**
- **Klaviyo ID**
#### Field mapping
Hightouch lets you sync subscription list fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default subscription list fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Knock
**URL:** https://hightouch.com/docs/destinations/knock
**Description:** Knock lets you send and manage notifications across email, push, SMS, and in-app from one API—without building the infrastructure yourself.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|---------------|-------------------------------------------------------------------------------------------|----------------------|------------------------------------------------------------------|
| **Users** | Sync data from any source to Knock [users](https://docs.knock.app/concepts/users) | Upsert | [Users docs](https://docs.knock.app/api-reference/users) |
| **Objects** | Sync data from any source to Knock [objects](https://docs.knock.app/concepts/objects) | Upsert | [Objects docs](https://docs.knock.app/api-reference/objects) |
| **Audiences** | Sync data from any source to Knock [audiences](https://docs.knock.app/concepts/audiences) | Add, Remove | [Audiences docs](https://docs.knock.app/api-reference/audiences) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Knock
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Knock** and click **Continue**. You can then authenticate Hightouch to **Knock**.
Enter the following fields into Hightouch:
- **Secret API key**: You can find this in the Knock dashboard under `API keys`.
- (Optional) **Service token**: This is only required if you plan to sync audiences. You can find this in the Knock dashboard under `Service tokens`.
## Sync configuration
Once you've set up your Knock destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Knock destination you want to sync to.
### Syncing users
Sync data from any source to Knock [users](https://docs.knock.app/concepts/users).
#### Record matching
To match rows from your model to users in Knock, you need to select the model column that contains values that match the **ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync user fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom user fields.
If you send data for a custom field that doesn't exist, Hightouch adds the field.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
|----------------|---------------------------------------------------|
| **Do nothing** | Keep the user in Knock with all its synced fields |
| **Delete** | Delete the synced users from Knock |
### Syncing objects
Sync data from any source to Knock [objects](https://docs.knock.app/concepts/objects).
#### Record matching
To match rows from your model to objects in Knock, you need to select the model column that contains values that match the **ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
You can also optionally map the following fields:
- **Email**
- **Phone number**
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
|----------------|-----------------------------------------------------|
| **Do nothing** | Keep the object in Knock with all its synced fields |
| **Delete** | Delete the synced objects from Knock |
### Syncing audiences
Sync data from any source to Knock [audiences](https://docs.knock.app/concepts/audiences).
#### Record matching
To match rows from your model to audiences in Knock, you need to select the model column that contains values that match the **ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync user fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom user fields.
If you send data for a custom field that doesn't exist, Hightouch adds the field.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Kognitiv
**URL:** https://hightouch.com/docs/destinations/kognitiv
**Description:** Sync data from your warehouse to Kognitiv's loyalty platform
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|Any data set|Sync data from a source to a file-import specification in Kognitiv|Mirror|
Mirror mode means one file with all the rows in the query results will be created, every time the sync runs.
## Getting started
### Connect to Kognitiv
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Kognitiv** and click **Continue**.
Hightouch syncs to Kognitiv via their UploadFile Web Service, which uses a security token. Enter your security token.
If you do not have a security token, contact Kognitiv Loyalty Support to set one up.
## Syncing data
### File import specifications
Choose which file import specification you'd like to import to Kognitiv. Hightouch offers templates for:
- Member descriptor file
- Loyalty member custom fields
Access the complete list of Kognitiv's file import options [here](https://info.kognitivloyalty.com/File_Import_Specifications.html).
If you'd like to sync to a file import specification that isn't listed, select **Custom** and enter in the column fields as mappings.
### Field mapping
If you selected one of the file import specifications listed above, column mapping options will be available to select from. You can add additional mappings by creating a new field.
If you selected **Custom** file import specification, enter the field that you'd like to map data to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Kustomer
**URL:** https://hightouch.com/docs/destinations/kustomer
**Description:** Manage and contextualize data to deliver hyper-personalized support and drive smarter processes that scale your business.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
---------------|-------------------------------------------------|----------------------|-------------------------------------------------------------------------------------------------
**Companies** | Sync data from any source to Kustomer companies | Upsert, Update | [Companies docs](https://developer.kustomer.com/kustomer-api-docs/reference/bulkcreatecompanies)
**Customers** | Sync data from any source to Kustomer customers | Upsert, Update | [Customers docs](https://developer.kustomer.com/kustomer-api-docs/reference/bulkcreatecustomers)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Kustomer
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Kustomer** and click **Continue**. You can then authenticate Hightouch to **Kustomer** by entering the following required fields into Hightouch:
- **Region**
- **API Key**
You can generate an API key by navigating to **Settings > Security > API Keys > Add API Key** in your Kustomer settings.
For more information, check out the [Kustomer's API key documentation](https://help.kustomer.com/api-keys-SJs5YTIWX).
You need to grant the following roles to your API key:
- `org.permission.bulk`
- `org.permission.company`
- `org.permission.customer`
## Sync configuration
Once you've set up your Kustomer destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Kustomer destination you want to sync to.
### Syncing companies
Sync data from any source to Kustomer companies.
#### Record matching
To match rows from your model to companies in Kustomer, you need to select a model column and corresponding Kustomer field. You can match on any of the following Kustomer fields:
- **External ID**
- **Company Name**
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync company fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom company fields.
Kustomer requires the **Name** field, so you must map it to complete your configuration.
Ensure your model columns data types match the data types of the fields you want to sync to.
You must suffix custom field names with their data type.
For example, if you were syncing to a custom field called **Age** that accepted numerical values, the Kustomer field name would be **AgeNum**.
For more information, check out the [Kustomer documentation](https://developer.kustomer.com/kustomer-api-docs/reference/createacompany).
#### Split retries
Kustomer counts all companies in a batch as rejected if the request contains a single invalid company. To pinpoint which companys are getting rejected with which errors and reduce the number of valid companys that get retried, you can enable [split retries](/syncs/retries#split-retries).
### Syncing customers
Sync data from any source to Kustomer customers.
#### Record matching
To match rows from your model to customers in Kustomer, you need to select a model column and corresponding Kustomer field. You can match on any of the following Kustomer fields:
- **External ID**
- **Home Email**
- **Work Email**
- **Other Email**
- **Work Phone**
- **Mobile Phone**
- **Home Phone**
- **Fax Phone**
- **Other Phone**
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync customer fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom customer fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
You must suffix custom field names with their data type. For example, if you were syncing to a custom field called **Age** that accepted numerical values, the Kustomer field name would be **AgeNum**.
For more information, check out the [Kustomer documentation](https://developer.kustomer.com/kustomer-api-docs/reference/createacustomer).
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|--------------------------------------------------------
**Do nothing** | Keep the customer in Kustomer with all its synced fields
**Delete** | Delete the synced customers from Kustomer
#### Split retries
Kustomer counts all customers in a batch as rejected if the request contains a single invalid customer. To pinpoint which customers are getting rejected with which errors and reduce the number of valid customers that get retried, you can enable [split retries](/syncs/retries#split-retries).
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## La Growth Machine
**URL:** https://hightouch.com/docs/destinations/la-growth-machine
**Description:** Automate your outreach campaigns to send messages through LinkedIn, Email, Voice Messages, and more with La Growth Machine.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
----------|---------------------------------------------------------|----------------------|-----------------------------------------------------------------------------------------------------------
**Leads** | Sync data from any source to Leads in La Growth Machine | Upsert, Update | [Leads docs](https://documenter.getpostman.com/view/2071164/TVCmSkH2#45f9a866-2afe-4e3d-9e0f-4a63c1b32612)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to La Growth Machine
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **La Growth Machine** and click **Continue**. You can then authenticate Hightouch to **La Growth Machine**.
Enter the following fields into Hightouch:
- **API key**
## Sync configuration
Once you've set up your La Growth Machine destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the La Growth Machine destination you want to sync to.
### Syncing leads
Sync data from any source to Leads in La Growth Machine.
#### Record matching
To match rows from your model to leads in La Growth Machine, you need to select a model column and corresponding La Growth Machine field. You can match on any of the following La Growth Machine fields:
- **Professional email**
- **Personal email**
- **LinkedIn Url**
- **Company name**
- **Company URL**
When matching on `Company name` or `Company URL`, La Growth Machine requires the following fields, so you must map them to complete your configuration:
- **First name**
- **Last name**
#### Field mapping
Hightouch lets you sync lead fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom lead fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------------|--------------------------------------------------------------------
**Do nothing** | Keep the lead in La Growth Machine with all its synced fields
**Clear** | Clear all the mapped fields, but keep the lead in La Growth Machine
**Remove from list** | Remove the lead from the audience, but keep all lead fields unchanged
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## LaunchDarkly
**URL:** https://hightouch.com/docs/destinations/launchdarkly
**Description:** Keep your team's feature flags up-to-date with fresh targeting rules and lists of users
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|---------------------|-------------------------------------------------------------------|----------------------|-------------------------------------------------------------------------------------------------|
| **Segment rules** | Update inclusion or exclusion rules in an existing segment | Add, Remove | [Segments docs](https://apidocs.launchdarkly.com/tag/Segments) |
| **Synced segments** | Sync a full segment, or set of contexts, to LaunchDarkly | Add, Remove | [Synced segments docs](https://docs.launchdarkly.com/guides/integrations/build-synced-segments) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to LaunchDarkly
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **LaunchDarkly** and click **Continue**. You can then authenticate Hightouch to **LaunchDarkly**.
Enter the following required fields into Hightouch:
- **Access Token**
#### Create an access token
1. Log in to LaunchDarkly
2. Visit your **Account settings** page
3. Click on the **Authorization** tab
4. Click on **Create token**
5. Give your token a name and assign it the **Writer** role
6. Click **Save token** and copy the new token that appears in the **Authorization** page
## Sync configuration
Once you've set up your LaunchDarkly destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the LaunchDarkly destination you want to sync to.
### Syncing segment rules
Sync data from any source to LaunchDarkly segments by updating inclusion or exclusion rules for an existing segment.
Hightouch can sync data to a LaunchDarkly segment in three ways:
1. Target users for inclusion
2. Target users for exclusion
3. Target users who match a custom rule
#### Record matching
You can match rows from your model to a segment in LaunchDarkly on any column in your model and any field in LaunchDarkly. Ensure the data types of the model column and LaunchDarkly field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Targeting users for inclusion or exclusion
The most common way of using our LaunchDarkly destination is to sync a model containing a list of user IDs for inclusion or exclusion in a particular segment.
#### Targeting users who match a custom rule
Hightouch can also sync your data into a LaunchDarkly rule clause.
### Syncing synced segments
Hightouch can directly sync a segment or set of contexts to LaunchDarkly.
#### Record matching
To match rows from your model to a synced segment in LaunchDarkly, you need to select the model column that contains values that match the **id** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
Users removed from the model get removed from the segment.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## LG Ad Solutions
**URL:** https://hightouch.com/docs/destinations/lg-ad-solutions
**Description:** Empower your advertising team to run targeted campaigns across LG’s premium Smart TV inventory.
**Section:** Destinations
## Overview
Deliver more relevant ads across LG’s premium, brand-safe inventory. Reach
addressable audiences across LG Smart TV Native and CTV ad experiences by
matching your first-party identifiers with LG IDs using Match Booster.
## Supported syncing
| Sync type | Description | Supported sync modes |
| ------------------------------------------ | ---------------------------------- | -------------------- |
| [**Audiences**](#syncing-custom-audiences) | Create and update custom audiences | All |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to LG Ad Solutions
This integration requires prior approval from the LG Ad Solutions team and the
Hightouch team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your LG Ad Solutions rep who will verify approval. Once we have confirmed,
we will unlock the integration for you to connect.
To connect, input the `Advertiser Name` provided by your LG rep.
### Syncing custom audiences
You can input the name of the audience you plan on delivering to LG.
Additionally, you can specify other properties for your audience.
#### User identifiers
LG only accepts LG Ad IDs to match on. We recommend using Match Booster to
resolve your identifiers to these LG Ad IDs if you do not have them.
#### Delete behavior
The only supported sync mode is `all` mode, meaning that each sync run will refresh
the entire audience. Users removed from your audience will automatically be dropped
with each refresh - only present users will remain in the LG audience.
## Tips and troubleshooting
### Refresh rate
LG Ad Solutions has a max refresh rate of daily. You should not have your LG Ad
Solutions syncs running more than daily.
### Matched device count
Match rates are not available to preview in the Hightouch platform. Contact your
LG rep for insights into match rates and audience reach.
Keep in mind that audience counts reflect the number of LG devices match, not
the number of users matched. Also keep in mind that matching occurs at the
household level, and that a single household may own multiple devices.
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Live debugger
### Sync alerts
---
## Liftoff
**URL:** https://hightouch.com/docs/destinations/liftoff
**Description:** Empower your marketing team to run advertising on Liftoff
**Section:** Destinations
## Overview
[Liftoff](https://liftoff.io/) is a mobile app marketing platform that helps companies acquire and retain high-quality app users at scale. Hightouch's Liftoff destination allows you to sync audiences to Liftoff for ad targeting.
## Supported Syncing
| Type | Description | Supported Sync Modes | API Reference |
| ------------- | ------------------------------------------ | -------------------- | -------------- |
| **Audiences** | Sync audiences from any source to Liftoff. | Add and Remove | Not applicable |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Liftoff
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Liftoff** and click **Continue**. You can then authenticate Hightouch to **Liftoff**.
You need to enter the following required fields:
- **API key**: Ask your Customer Support Representative at Liftoff for your API key.
## Sync Configuration
Once you've set up your **Liftoff** destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Liftoff destination you want to sync to.
This integration supports syncing audiences.
### Record Matching
You can match rows from your model to users in Liftoff on their **Device ID**. Refer to the [record matching docs](/syncs/record-matching) for more information.
### Delete Behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| ------------------------ | ------------------------------------------ |
| **Do nothing** | Keep the user in the Liftoff audience. |
| **Remove from audience** | Remove the user from the Liftoff audience. |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## LINE Ads
**URL:** https://hightouch.com/docs/destinations/line-ads
**Description:** Boost your business growth by finding the right customers on LINE. Reach more audiences and achieve better results by managing campaign objectives, targeting, budget, and measurement anytime through LINE Ads
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
|--------------|-------------------------------------------------|----------------------|------------------------------------------------------------------------------------------------------|
| [**Segments**](#syncing-segments) | Sync data from any source to LINE Ads audiences | Add | [Audience docs](https://ads.line.me/public-docs/v3/3.9.7/data-general-partner#_uploadcustomaudience) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to LINE Ads
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **LINE Ads** and click **Continue**. You can then authenticate Hightouch to **LINE Ads**.
Enter the following required fields into Hightouch:
- **Ad Account ID**: Enter your ad account ID.
- **API access key**: Enter a valid API access key.
- **API secret key**: Enter a valid API secret key.
For more information about LINE's Ads Management API and requesting credentials, go to LINE's [authentication docs](https://ads.line.me/public-docs/v3/3.9.7/data-general-partner#_authentication).
## Sync configuration
Once you've set up your LINE Ads destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the LINE Ads destination you want to sync to.
### Syncing segments
Sync data from any source to LINE Ads custom audiences.
#### Record matching
To match rows from your model to segment in LINE Ads, you need to select a model column and corresponding LINE Ads field. You can match on any of the following LINE Ads fields:
- **Email**
- **Phone number**
- **IFA**
If you are syncing to an existing segment, only the ID type for that segment is available.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Linear
**URL:** https://hightouch.com/docs/destinations/linear
**Description:** Enrich and create tickets in Linear
**Section:** Destinations
## Overview
Linear is a popular issue-tracking tool designed for high-performance teams. It helps streamline workflows for software projects, sprints, and tasks.
Hightouch's Linear destination allows you to sync data from your warehouse to create and update issues in Linear. This can be useful for automating bug reports, feature requests, and other tasks based on events or data in your own systems.
## Supported Syncing
| Type | Description | Supported Sync Modes |
| ---------- | ------------------------------------------- | ------------------------------ |
| **Issues** | Sync data from any source to Linear issues. | Upsert, Update, Insert, Delete |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Linear
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Linear** and click **Continue**. You can then authenticate Hightouch to **Linear** by providing your API Key.
To get your API Key:
1. Log in to your Linear account.
2. Navigate to **Settings** > **Security & access**.
3. Under **Personal API keys**, create a new key.
4. Copy the key and paste it into the Hightouch configuration.
## Sync Configuration
Once you've set up your **Linear** destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Linear destination you want to sync to.
### Syncing Issues
Hightouch allows you to create, update, or delete issues in Linear based on your model data.
#### Record Matching
You can match rows from your model to issues in Linear using the **Issue ID**. **Issue ID** must be a UUIDv4.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Linear automatically generates an identifier for every new
record synced if one is missing.
#### Field Mapping
You can map data from any of your model columns to Linear issue fields. Hightouch only requires you to map the **Title** field.
These are the additional fields you can map:
- Description
- Priority
- Assignee ID
- Project ID
- State ID
- Label IDs
- Due Date
- Estimate
#### Delete Behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | --------------------------------------------------- |
| **Do nothing** | Keep the issue in Linear with all its synced fields |
| **Delete** | Delete the synced issues from Linear |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## LinkedIn Ads
**URL:** https://hightouch.com/docs/destinations/linkedin
**Description:** Run campaigns with custom audiences, retargeting, and lookalikes in LinkedIn
**Section:** Destinations
## Overview
Deliver even more relevant ads by combining data from various sources within your
data warehouse to build custom audiences for LinkedIn Ads. By keeping your
custom audiences updated automatically, you can avoid showing irrelevant ads.
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| --------------- | ------------------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| [**Audiences**](#syncing-audiences) | Create new audience to sync users or companies | Upsert | [Audiences docs](https://learn.microsoft.com/en-us/linkedin/marketing/integrations/matched-audiences/matched-audiences?view=li-lms-2023-11) |
| [**Conversions**](#syncing-conversions) | Sync data from any source to LinkedIn conversions | Insert | [Conversions docs](https://learn.microsoft.com/en-us/linkedin/marketing/integrations/ads-reporting/conversions-api?view=li-lms-2023-11&tabs=http) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to LinkedIn Ads
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs)
and click the **Add destination** button. Select **LinkedIn Ads** and click **Continue**.
You can then authenticate Hightouch to **LinkedIn Ads** using OAuth.
### Authenticate with OAuth
For the **Authentication method**, select **Log in to LinkedIn Ads** and log into
your LinkedIn Ads account.
Once successful, you will be redirected back to Hightouch to enter a descriptive name
for your destination and complete setup.
## Sync configuration
Once you've set up your LinkedIn Ads destination and have a
[model](/getting-started/concepts#models) to pull data from,
you can set up your sync configuration to begin syncing data. Go to the
[**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the
**Add sync** button to begin. Then, select the relevant model and the LinkedIn
Ads destination you want to sync to.
### Syncing audiences
To get started, select an ad account to sync your data to, then select
which audience object you want to sync data to: Company or User.
Hightouch supports automatically creating a new audience for your sync. You can
specify a custom name for this audience, or Hightouch will use the name of the
model.
After the initial run of the sync, you can't update the ad account, audience
object, or name of the audience. To make these changes, create a new sync.
#### Sync mode
This integration supports Upsert mode. New users will be inserted into the LinkedIn
audience. Users that are removed from the model will be removed from the
audience.
#### Field mapping
The sync configuration has two field mapping sections:
- **ID fields**: These are the primary identity keys that LinkedIn uses to match users, such as hashed email or Google Advertising ID (GAID). These values become `userIds` in the LinkedIn API payload. You must map at least one ID field.
- **Additional fields**: These are supplemental fields—such as first name, last name, title, company, and country—that improve LinkedIn's match rate but aren't primary identifiers.
LinkedIn requires emails to be hashed using SHA256 or SHA512. Hightouch handles this automatically on the backend.
LinkedIn enforces the following input data requirements: - All IDs provided by
`userId` or `userIds` fields must have a [supported type and valid
value](https://learn.microsoft.com/en-us/linkedin/marketing/integrations/matched-audiences/create-and-manage-segment-users?view=li-lms-unversioned&tabs=http#input-data-requirements).
- Each row of data must provide: (1) at least one valid ID; or (2) A valid
`firstName` and `lastName`. See LinkedIn's [user data
schema](https://learn.microsoft.com/en-us/linkedin/marketing/integrations/matched-audiences/create-and-manage-segment-users?view=li-lms-unversioned&tabs=http#schema)
for more information.
### Syncing conversions
Sync data from any source to LinkedIn as conversion events.
Create a [conversion rule](https://www.linkedin.com/help/lms/answer/a1657171) in LinkedIn Campaign Manager.
Select "Conversions" in your sync in Hightouch.
Select your LinkedIn account.
Choose your event timestamp or let LinkedIn use the timestamp the event arrives at the server.
#### Record matching
You can match rows from your model to conversions in LinkedIn on the following user fields:
- Email (SHA256)
- LinkedIn tracking UUID
- Acxiom ID
- Oracle moat ID
- IP address (IPv4)
- Google Advertising ID (GAID)
## Handling PII and hashing
You can enable PII hashing if you want Hightouch to automatically detect, normalize, and hash fields that require it. For example, if you map to a `Email (SHA256)` field and the model column with the email data is unhashed, Hightouch hashes the value for you.
By default, Hightouch automatically detects if the following fields require hashing:
- **Email (SHA256)**
- **Email (SHA512)**
#### View conversion tracking and reporting metrics
Once your sync is set up and published, you can view conversions tracking and reporting metrics in your LinkedIn Campaign Manager.
## Tips and troubleshooting
For Hightouch platform error codes related to LinkedIn, see [Error codes: LinkedIn](/developer-tools/error-codes#linkedin).
### Matched users count
Below only applies to the audiences sync type.
### Common errors
#### 400 - MISSING_REQUIRED_FIELD
You may receive a `400 - type=MISSING_REQUIRED_FIELD` because the sync is sending a required field `null` values.
You can resolve this error by either:
- [Removing rows containing `null` values](/models/sql-editor#exclude-rows-with-null-values) from your model's query results
- Replacing any `null` values in your model's query results with another value
- Using the [advanced mapper](/syncs/mapping-data#advanced-mapper) to [ignore any `null` values](/syncs/mapping-data#dont-sync-null-values)
### Live debugger
### Sync alerts
---
## LiveIntent
**URL:** https://hightouch.com/docs/destinations/liveintent
**Description:** Empower your marketing team to match audiences across LiveIntent's publisher network for targeting and suppression.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
| ---------- | -----------------------------------------------------| ---------------------- | ------------------------------------------------------------------------------------------------------- |
| **Custom Audiences** | Sync data from any source to an LiveIntent list as user identifiers | Upsert | [Custom Audience](https://audiences.liveintent.com/api-guide#tag/What-is-a-Custom-Audience)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to LiveIntent
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **LiveIntent** and click **Continue**.
You can connect to LiveIntent with either an API Token, or a username and password. Contact your account team at LiveIntent for credentials.
## Sync configuration
Once you've set up your LiveIntent destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the LiveIntent destination you want to sync to.
### Audience selection
Select whether you'd like to create a new audience or sync to an existing audience. If you would like to create a new audience, enter the name of the audience in the sync configuration form.
### Field mapping
LiveIntent uses email, IDFA, and AAID to identify users. Select which of these identifiers you would like to map your records as and send to LiveIntent.
#### Handling PII and hashing
LiveIntent expects **email** values to be hashed.
For audience syncs, by default, Hightouch automatically hashes first-party user data fields before sending requests. If the data in your model is already hashed, you can disable PII hashing to successfully send data to LiveIntent.
### Delete behavior
When records leave your query result, they are removed from your audience by default. If you would not like to delete any records from your audience, you can choose to do nothing when records leave your query result.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## LiveRamp
**URL:** https://hightouch.com/docs/destinations/liveramp
**Description:** LiveRamp helps you unify customer data across platforms, activate it securely everywhere you need it, and measure performance with precision—without compromising privacy. It turns fragmented identifiers into actionable insights so you can target smarter and drive higher ROI.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
------------------------|--------------------------------------------|---------------------
**SFTP file upload** | Upload audience files to LiveRamp via SFTP | Insert, All
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to the LiveRamp SFTP server
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **LiveRamp** and click **Continue**. You can then authenticate Hightouch to **LiveRamp**.
Enter the following fields into Hightouch:
- **Host**
- **Port**
- **Username**
- **Password**
- (Optional) **Private Key**
- (Optional) **Private Key Passphrase**
## Sync configuration
Once you've set up your LiveRamp destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the LiveRamp destination you want to sync to.
### Syncing audiences via file upload
Upload audience files to LiveRamp via SFTP
#### Field mapping
Hightouch lets you sync batch file delivery fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default batch file delivery fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
For more information on file formatting, refer to LiveRamp's [docs](https://docs.liveramp.com/connect/en/formatting-column-based-files.html).
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Loops
**URL:** https://hightouch.com/docs/destinations/loops
**Description:** Send your product, marketing, and transactional emails for your app with Loops, the email platform built for SaaS. Empower your marketing teams to create, send and track beautiful email campaigns.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-------------|---------------------------------------------|------------------------|-----------------------------------------------------------
**Contacts** | Sync data from any source to Loops contacts | Upsert | [Contacts docs](https://loops.so/docs/api-reference/intro#contacts)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Loops
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Loops** and click **Continue**. You can then authenticate Hightouch to **Loops**.
Enter the following fields into Hightouch:
- **API key**
## Sync configuration
Once you've set up your Loops destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Loops destination you want to sync to.
### Syncing contacts
Sync data from any source to Loops contacts.
#### Record matching
To match rows from your model to contacts in Loops, you need to select a model column and corresponding Loops field. You can match on any of the following Loops fields:
- **Email**
- **User Id**
#### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom contact fields.
Loops requires the **Email** field, so you must map it to complete your configuration.
If you send data for a custom field that doesn't exist, Hightouch adds the field.
Ensure your model's columns have the same data types as the fields you want to sync to.
For the **mailingLists** object, the API expects JSON objects with key-value pairs where the key is a mailing list ID and the value is a `boolean` denoting if the contact should be added (`true`) or removed (`false`) from the list. Go to [API Docs for Mailing Lists](https://loops.so/docs/api-reference/create-contact#param-mailing-lists) to learn more.
#### Trigger Loops
Hightouch can trigger a Loop to start sending emails to your contacts. Build a Loop using the [Loop builder](https://loops.so/docs/loop-builder), and configure whether you'd like to trigger Loops in the sync form.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|-----------------------------------------------------------
**Do nothing** | Keep the contact in Loops with all its synced fields
**Clear** | Clear all the mapped fields, but keep the contact in Loops
**Delete** | Delete the synced contacts from Loops
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Magnite
**URL:** https://hightouch.com/docs/destinations/magnite
**Description:** Magnite is the world’s largest independent sell-side advertising company. We’ll help you win across every channel, every format and on your terms.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
--------------|------------------------------------------------|---------------------
**Access Data Audiences** | Sync data from any source to Magnite Data audiences (`ctvid`, `ip`, `maid`, `cookie`) | All
**Match Audiences** | Sync data from any source to Magnite Match audiences (`hem`) | All
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Magnite
This integration requires prior approval from the Magnite team and the Hightouch team.
To gain approval, please contact the Hightouch support team and connect us directly with your Magnite rep who will verify approval. Once we have confirmed, we will unlock the integration for you to connect.
To use the Magnite Data integration, enter the following fields into Hightouch:
- **Client subfolder**
- **Owner**
To use the Magnite Match integration, enter the following fields into Hightouch:
- **Client name**
- **Account ID**
Once you've set up your Magnite destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin.
## Magnite Data integration
### Sync configuration
Select the relevant model and the Magnite Data destination you want to sync to. Then, enter the following fields into Hightouch:
- **Segment name**
- **Price type** (acceptable values are `cpm` and `revshare`)
- **Charge for use of the segment**
- **Segment owner**
- **Description for the segment** (optional)
### Syncing audiences
Sync data from any source to Magnite Data audiences.
#### Field mapping
Hightouch lets you sync audience fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default audience fields.
Magnite requires the following fields, so you must map them to complete your configuration:
- **User ID**
- **User ID type**
- **Update type**
Ensure your model's columns have the same data types as the fields you want to sync to.
## Magnite Match integration
### Sync configuration
Select the relevant model and the Magnite Match destination you want to sync to. Then, enter the following fields into Hightouch:
- **Segment name**
- **Description for the segment** (optional)
- **CPM for the segment** (optional)
- **Platform for the segment**
- **Membership timestamp column** (optional)
- **Full resync interval (in days)**
### Syncing audiences
Sync data from any source to Magnite Match audiences.
#### Field mapping
Hightouch lets you sync audience fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default audience fields.
Magnite requires the following fields, so you must map them to complete your configuration:
- **Email (SHA256)**
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Mailchimp
**URL:** https://hightouch.com/docs/destinations/mailchimp
**Description:** Run better email campaigns on Mailchimp with up-to-date customer data from your data warehouse
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| ------------ | ------------------------------------------------------------------------------------- | ---------------------- |----------------------------------------------------------------------------------------------------------------------------------- |
| **Object** | Sync members to audiences in Mailchimp. | Upsert | [Members Docs](https://mailchimp.com/developer/marketing/api/list-members/add-or-update-list-member/) |
| **Tag** | Create and apply a new tag or apply an existing tag to contacts within your audience. | Add, Remove | [Static Segments (Tags) Docs](https://mailchimp.com/developer/marketing/api/list-segments/batch-add-or-remove-members/) |
## Setup
To setup the Mailchimp destination, you will need your data center and an active API Key.
To find the data center and API key, navigate to the [Mailchimp Dashboard](https://admin.mailchimp.com).
### Data center
To find the data center, look at the prefix for your admin dashboard URL. In this example, the URL is `us1.admin.mailchimp.com`. Your prefix would be `us1`.
### API key
To find your API key, navigate to the [API Keys section](https://admin.mailchimp.com/account/api/) of your Mailchimp account.
Go down to the section Your API Keys, and press the Create A Key button if none exist. Otherwise, you'll see your API key in the table.
Mailchimp has API Keys and Mobile SDK API Keys. The Mobile SDK API Key will
not work with this integration. You will need to create a regular API Key.
## How Hightouch connects
Hightouch connects to Mailchimp through the [Mailchimp Marketing API](https://mailchimp.com/developer/marketing/api/), specifically the [Lists API](https://mailchimp.com/developer/marketing/api/lists/) to read, update, and manage audience data.
This integration handles:
- Retrieving your available audiences for setup and testing
- Syncing audience members and custom merge fields
- Managing segment memberships
- Archiving or permanently deleting members when records leave your query
## Syncing Mailchimp contacts
Hightouch supports syncing to the following Mailchimp resources:
- Audience members
Hightouch requires an existing audience in Mailchimp to sync audience members.
### Sync modes
This integration supports Upsert mode.
In the Upsert mode, new users will be inserted into Mailchimp and all attributes will be kept up-to-date within Mailchimp.
### Record matching
Records can be matched from your source to your Mailchimp workspace by your Mailchimp Audience Member Email in the Upsert mode.
### Field mapping
You can sync columns from your source to Mailchimp's default and custom merge fields. Hightouch automatically detects existing merge fields from your Mailchimp audience.
You can add custom merge fields in your Mailchimp dashboard from the Settings page dropdown and selecting Audience fields and \*\|MERGE\|\* tags
Once on the Audience fields page you can drop to the bottom of the page and hit the Add A Field button to create a custom merge field.
### Delete mode
You can choose what Hightouch's behavior is when records leave the query result set. The default is doing nothing, but you can also set Hightouch to archive or permanently delete the Mailchimp audience member on record exit.
- `ARCHIVE` : Unsubscribe a user from your Mailchimp audience, but do not delete the record. Archived members can be re-added at a later date.
- `DELETE` : **Permanently** remove an audience member and delete their information. This action can't be undone, and the user **can't** be re-added to the list at a later date.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## MariaDB
**URL:** https://hightouch.com/docs/destinations/mariadb
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Overview
This destination combines the analytical power of your data warehouse with the low-latency performance of a transactional database.
## Supported syncing
|Sync Type|Description |Supported Sync Modes |
|-----------|----------------------------------------------|---------------------|
| **Table** |Sync data from any source to a MariaDB table | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to MariaDB
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **MariaDB** and click **Continue**.
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your MariaDB.
- **Port**: The port number of your MariaDB. The default is 3306, but yours may be different.
- **Database**: The name of the database in your MariaDB.
- **User**: This can be your personal MariaDB login or a dedicated user for Hightouch.
- **Password**: The password for the user specified above.
You can optionally create or select a [previously configured tunnel](/security/ssh-tunneling) or upload a `.pem` CA certificate.
### SSH tunneling
Hightouch can connect directly to MariaDB over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your MariaDB instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
## Sync configuration
Once you've set up your MariaDB destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the MariaDB destination you want to sync to.
### Select table
Select the existing table in MariaDB you want to sync to.
### Record matching
You can match rows from your model to rows in MariaDB on any column as long as it is a [primary key or unique key](https://mariadb.com/kb/en/create-table/#index-categories).
### Field mapping
Select which model columns you want to sync to your MariaDB table columns. Hightouch automatically pulls the columns from your table to make them available for you to map.
When syncing to [binary](https://dev.mysql.com/doc/refman/8.0/en/binary-varbinary.html) destination fields, Hightouch assumes the model column values to be [base64](https://en.wikipedia.org/wiki/Base64) strings. If they aren't, you can use [template mapping](/syncs/mapping-data#template-mapping) to [cast](/models/data-types-casting#casting) them if needed.
### Upsert query optimization
You can choose to use the [`LOAD DATA` statement](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_local_infile) in the upsert query during your sync. This option loads data from a text file into your database, which can be up to twenty times faster than the default `INSERT INTO` query Hightouch uses.
If you enable the `LOAD DATA` option, Hightouch replaces databases rows with the mapped model row, if it exists. This means that if any MariaDB fields aren't mapped in Hightouch they receive `NULL` values.
For Hightouch to successfully sync with the `LOAD DATA` option, you need to enable the [`local_infile`](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_local_infile) system variable.
Review the [security considerations](https://dev.mysql.com/doc/refman/8.0/en/load-data-local-security.html) before enabling `local_infile`.
### Delete behavior
When rows no longer shows up from your model's query results, you can configure the sync to **Do nothing** or **Delete** those rows from the MariaDB.
| Behavior | Description |
| -------------- | ----------------------------------------------- |
| **Do nothing** | Keep the synced rows in the table in MariaDB |
| **Delete** | Delete the synced rows from the table in MariaDB|
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Marigold Engage (Selligent)
**URL:** https://hightouch.com/docs/destinations/marigold-engage
**Description:** Targeting marketing to personalize customer experiences.
**Section:** Destinations
## Supported content
| Type | Description | Supported Sync Modes |
|-----------|----------------------------------------------------|------------------------|
| **Users** | Sync records from your model to lists and segments | Upsert, Update, Insert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Marigold Engage
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Marigold Engage** and click **Continue**. You can then authenticate Hightouch to **Marigold Engage** via a Service Account.
To create a Service Account, click the settings gear in the bottom left corner of the Marigold Engage dashboard. Then, navigate to **Access Management > Service Accounts** and click **New**. Make sure to set an expiration date and configure the following endpoints:
- `/organizations/{organization}/lists`
- `/organizations/{organization}/lists/{list}/fields`
- `/organizations/{organization}/lists/{list}/data/search`
- `/organizations/{organization}/lists/{list}/data/load`
- `/organizations/{organization}/lists/{list}/segments`
- `/organizations/{organization}/lists/{list}/segments/{segment}`
- `/organizations/{organization}/lists/{list}/segments/{segment}/data/load`
Then in Hightouch:
1. Enter the Marigold hostname used to access the Marigold control panel, this is usually `{organization}.slgnt.eu`
2. Enter the API and Secret keys from the Service Account you created
3. Enter the organization the lists should be created in
## Sync configuration
Once you've set up your Marigold Engage destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Marigold Engage destination you want to sync to.
### Field mapping
Field mapping is where you select which customer information parameters or other record parameters you want to send to Marigold.
You can map data from any of your model columns to the list's preconfigured fields.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Marigold Engage+
**URL:** https://hightouch.com/docs/destinations/marigold-engage-plus
**Description:** Sync data and run campaigns on Marigold Engage+
**Section:** Destinations
## Supported content
| Type | Description | Supported Sync Modes |
|--------------|----------------------------------------------------------------|----------------------|
| **Audience** | Sync records from your model to audiences in real-time or bulk | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
Marigold Engage+ supports both real-time and bulk sync modes.
Real-time syncs are recommended only for small data sets,
while bulk syncs are recommended for any data set over 1 million.
## Connect to Marigold Engage+
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Marigold Engage+** and click **Continue**. You can then authenticate Hightouch to **Marigold Engage+** with help from your Marigold agent.
Then in Hightouch:
1. Enter the Customer ID that your Marigold agent provides
2. Enter the region your Marigold account is in
3. Enter the API and Secret keys from Marigold
4. If you would like to sync in bulk, enter the SFTP credentials your Marigold agent provides
## Advanced configuration
If you are syncing in real-time to joined tables you must specify which fields should be sent to which forms in the destination config.
If you are syncing in bulk you must specify the available mapping options in the destination config.
Your Marigold agent will help you configure the available options for your use case.
## Sync configuration
Once you've set up your Marigold Engage+ destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Marigold Engage+ destination you want to sync to.
### Field mapping
Field mapping is where you select which customer information parameters or other record parameters you want to send to Marigold.
You can map data from any of your model columns to the list's preconfigured fields.
### Form IDs
When syncing in real-time mode you must also create an API post form in Marigold and enter the form's ID in Hightouch.
The form will control which campaigns are triggered as Highouch sends data to Marigold.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Marketo
**URL:** https://hightouch.com/docs/destinations/marketo
**Description:** Build robust marketing campaigns on Marketo using rich customer data from your data warehouse
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| ------------------------------- | ------------------------------------------------------------------------------------- | -------------------- | ------------------------------------------------------------------------------------ |
| **Standard and custom objects** | Sync data to custom or standard objects including leads, companies, and opportunities | Upsert, Update | [Object docs](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/lead-database/custom-objects) |
| **Custom activities** | Sync data to custom activities | Insert | [Activities docs](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/lead-database/activities) |
| **Static lists** | Update membership in static lists | Add, Update | [Static lists docs](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/assets/static-lists) |
| **Campaign triggers** | Trigger campaigns using source data | Insert | [Campaign docs](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/assets/smart-campaigns) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Marketo setup
Giving Hightouch access to Marketo's API requires you to create a [**Custom Service**](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/custom-services) and [**API Only User**](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/custom-services) for that service in Marketo.
Even if you already have a Marketo API User, it's best to create one that's solely for the Hightouch service you'll create.
Having a dedicated user for the Hightouch integration lets you more easily track changes from Hightouch syncs.
### Create Marketo service
Before you can create a Custom Service, you need to create a Marketo [role](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/custom-services) with API Access and a user with that role.
None of the default Marketo Roles have API access, so if this is your first API integration in Marketo, you first need to create an API access role.
#### Create Marketo API access role
1. In Marketo, go to **Admin > Users & Roles > Roles** and click **New Role**.
2. Enter a **Role Name** (for example, "Hightouch Integration Role") and **Description**. Then assign **all** permissions listed in this table to the role:
| Sync Type | Necessary permissions |
| ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Standard and custom objects** | **Read-Write Company** (for company objects), **Read-Write Opportunity** (for opportunities), **Read-Write Person** (for leads), and **Read-Write Custom Object** and **Read-Write Custom Object Type** (for any custom objects) |
| **Custom activities** | **Read-Write Activity**, **Read-Write Activity Metadata** |
| **Static lists** | **Access Marketing Activities > List Import** |
| **Campaign triggers** | **Read-Write Assets**, **Activate Campaign** |
3. Click **Create**.
If you try setting up a Marketo destination with an API access role that wasn't assigned **all** permissions in the table above, the destination setup fails with a [`603 - Access denied`](#code-603-access-denied) error.
#### Create Marketo API user
After creating a role, you need to create an API-Only user.
API-Only users are a special type of user in Marketo, as they are administrated by other users and cannot be used to log in to Marketo.
1. In Marketo, go to **Admin > Users & Roles > Users** and click **Invite New User**.
2. Enter an **email** address, **first name**, and **last name** for the user. (These don't need to be valid.) Click **Next**.
3. Assign the [previously created role](#create-marketo-api-access-role), click the **API Only** checkbox, and click **Next**.
4. The last screen for user creation says "An invitation is not required for API only." Click **Send** to create the API-Only user.
Ensure that this user has access to the [workspaces and partitions](https://experienceleague.adobe.com/en/docs/marketo/using/product-docs/administration/workspaces-and-person-partitions/understanding-workspaces-and-person-partitions) you want to use in Hightouch. You can read more about this in the [`603 - Access denied`](#code-603-access-denied) section.
#### Create a custom service
Custom Services provide the actual credentials required to authenticate Marketo to Hightouch.
To provision one, follow these instructions:
1. In Marketo, go to **Admin > LaunchPoint** and select **New Service**.
2. Give your service a descriptive name. From the **Service** list, select the **Custom**. Give your service a description, for example, "Hightouch Integration" and select [the API Only user you previously created](#create-marketo-api-user), then click **Create**.
3. This adds a new service to your list of LaunchPoint services, and the option to **View Details**. Click on **View Details** and save the **Client Id** and **Client Secret** for use in Hightouch.
You're now ready to provide Hightouch access to Marketo.
## Connect to Marketo
If you created an [IP Allowlist](https://experienceleague.adobe.com/docs/marketo/using/product-docs/administration/additional-integrations/create-an-allowlist-for-ip-based-api-access.html) in your Marketo settings, make sure to allowlist Hightouch IP addresses so Hightouch can reach your cluster. Refer to our
[IP address docs](/security/networking#ip-addresses) to find the relevant IP addresses for your Hightouch region.
In Hightouch, go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Marketo** and click **Continue**. You can then authenticate Hightouch to Marketo by entering the following fields:
- Client ID
- Client Secret
- Base URL
To find the Client ID and Secret, navigate to **Admin > LaunchPoint** in your Marketo account and select [the service you created for Hightouch](#create-marketo-service).
You can find the Base URL in **Admin > Integration > Web Services** under **REST API**.
Only use the part after `https://` and before `/rest` as your base URL, for example, `663-XSQ-200.mktorest.com`.
## Sync configuration
Once you've set up your Marketo destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Marketo destination you want to sync to.
### Syncing objects
Hightouch supports syncing to both standard and custom Marketo objects.
The standard objects that Hightouch supports are:
- **Leads**
- **Companies**
- **Opportunities**
#### Record matching
You can match rows from your model to objects in Marketo on any column in your model and certain fields in Marketo. To avoid duplications or other data issues, it's imperative to choose a column with unique values. Refer to the [record matching docs](/syncs/record-matching) for more information.
{/* */}
{/* */}
The Marketo fields available for record matching depend on the object:
- for the [Lead object](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/lead-database/lead-database), Hightouch lists all `person` type [custom fields](https://experienceleague.adobe.com/docs/marketo/using/product-docs/administration/field-management/create-a-custom-field-in-marketo.html) for record matching,
- for [custom objects](https://experienceleague.adobe.com/docs/marketo/using/product-docs/administration/marketo-custom-objects/understanding-marketo-custom-objects.html), [Upsert](#supported-syncing) mode lists all [dedupe fields](https://experienceleague.adobe.com/docs/marketo/using/product-docs/administration/marketo-custom-objects/add-marketo-custom-object-fields.html), whereas [Update](#supported-syncing) mode lists all dedupe fields and `MarketoGUID`.
{/* */}
{/* */}
#### Field mapping
Hightouch lets you sync object fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to an object's default fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
-------------------|-----------------------------------------------------------------
**Do nothing** | Keep the record in Marketo with all its synced fields
**Delete record** | Delete the record from Marketo
**Clear fields** | Keep the record, but clear the mapped fields
In Upsert mode, you can select between **Do nothing** and **Delete record**. In Update mode, you can select between **Do nothing** and **Clear fields**.
#### Batch size
By default, Hightouch sends 300 objects in a batch.
You can lower the batch size to mitigate [413 Request Entity Too Large](#code-413-request-entity-too-large) errors.
### Syncing custom activities
When syncing custom activities, Hightouch treats any records added to your [model](/getting-started/concepts#models) as new activities and sends them to Marketo when your sync runs.
To sync custom activities, the custom activity type must first be created and approved in Marketo.
You can learn more about creating custom activities in [Marketo's docs](https://experienceleague.adobe.com/docs/marketo/using/product-docs/administration/marketo-custom-activities/create-a-custom-activity.html).
#### Activity selection
To start, select a custom activity type you've already created in Marketo.
If you don't see it in the dropdown, make sure it's approved in Marketo and then click **Refresh** in Hightouch.
#### Event timestamp
You can optionally select a column that contains timestamps of when activities occurred. If this field is empty, Marketo uses the time the activity arrives at the server.
#### Lead matching
To associate activities to leads, select the model column that contains either the leads' **Marketo GUID** or **Email**.
#### Activity primary attribute
Select the model column to map to the [primary attribute field](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/lead-database/activities) for your custom activity.
#### Field mapping
You can also map data from any of your model columns to a custom activity's non-primary attributes.
Ensure your model columns data types match the data types of the fields you want to sync to.
### Syncing static lists
Static lists are a list of names in Marketo.
You can configure Hightouch syncs to manage these lists by adding and updating users.
#### User identifiers
To identify which users to add or update in an static list, select a model column and the corresponding Marketo attribute, for example **Lead ID**.
#### Select an existing static list or create a new one
You can create a new static list or use an existing one. When creating a new static list, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model. To use an existing static list, select the desired static list from the dropdown.
#### Folder selection
If you're creating a new static list in Marketo, you need to select the folder/program to create the list in. Static lists can only be created in
folders and programs of the following types:
- Email Batch Program
- Marketing Program
- Nurture Program
- List
In the Marketo UI, you can tell whether the folder is one of the above types by right clicking the folder in the Marketing Activities sidebar. If the "New local asset"
or "Create list" options are available, then the folder supports static lists.
#### Static list description
If you're creating a new static list in Marketo, you can optionally Hightouch add a description for the static list.
#### Split retries
By default, Hightouch retries all records in a rejected batch. To pinpoint which users are getting rejected with which errors and reduce the number of valid users that get retried, you can enable [split retries](/syncs/retries#split-retries).
### Trigger campaigns
Hightouch lets you use records from your models to trigger campaigns through the [Marketo Campaign API](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/endpoint-reference).
#### Prerequisites
Before you can set up a Hightouch sync to trigger a campaign, you need to have an active [trigger campaign](https://experienceleague.adobe.com/en/docs/marketo/using/product-docs/core-marketo-concepts/smart-campaigns/creating-a-smart-campaign/understanding-batch-and-trigger-smart-campaigns) with **Web Service API** as the source in Marketo.
For details, follow the setup instructions below.
#### Trigger campaign setup
1. If you haven't already, go to **Design Studio** to [draft an email template](https://experienceleague.adobe.com/docs/marketo/using/product-docs/email-marketing/general/email-editor-2/create-an-email-template.html) in your Marketo instance. Before your email template you can use the email template in Hightouch, your draft must be approved.
You can use custom [tokens](https://experienceleague.adobe.com/docs/marketo/using/product-docs/demand-generation/landing-pages/personalizing-landing-pages/tokens-overview.html) to populate your email template with data from your model.
In the example screenshot above, a custom token makes it possible to dynamically populate the email subject.
2. Next, you need a **Smart Campaign** to trigger. Follow [these steps](https://experienceleague.adobe.com/docs/marketo/using/product-docs/core-marketo-concepts/smart-campaigns/creating-a-smart-campaign/create-a-new-smart-campaign.html?lang=en) if you need to create a new one. Make sure that your campaign belongs to a program.
3. Open the **Smart List** tab in the top nav to configure the trigger. Drag and drop **Campaign is Requested** from the triggers list on the right.
4. Select that the source **is** **Web Service API**.
5. Open the **Flow** tab in the top nav to configure your flow. Drag and drop **Send Email** from the actions on the right.
6. Select the email template you want to send.
7. Open the **Schedule** tab in the top nav to configure your qualifications. Click on **Edit** to edit your qualifications.
Hightouch doesn't check if an email has been sent to
a distinct lead before trigger a campaign. For example, if a lead with email `janedoe@example.com`
appears in two of the queried rows from your model, then Hightouch triggers
an email for each of these rows. Two emails would be sent to `janedoe@example.com`. If you want to only send
one email per lead, set your qualifications in Marketo to **only
once**. 
8. Click **Activate** when you're finished.
For additional reference, see [Marketo's how-to article](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/assets/transactional-email).
#### Sync configuration
Each new row in your model triggers your Marketo Smart Campaign.
That means that Hightouch triggers the campaign for every row in your model on your first sync run, or if you trigger a [full resync](/syncs/overview#resync-full-query).
In subsequent sync runs, only new rows in your model's query results trigger the campaign.
#### Campaign selection
To start, select a campaign you've created in Marketo.
Only campaigns triggerable via web service API are listed here.
If you don't see the campaign you want to trigger in the dropdown, make sure it's follows the [prerequisites](#prerequisites) and then click **Refresh** in Hightouch.
#### Lead matching
Marketo Smart Campaigns can only be triggered for leads that already exist in your Marketo instance.
To trigger an email for the intended leads, select the model column that contains either the leads' **Marketo GUID** or **Email**.
If you use email for your lead matching and your leads aren't
de-duplicated by email, Hightouch uses the latest lead based on the
`createdAt` timestamp.
#### Token mappings
Use this section to map custom token values.
You only need to add mappings for tokens used in your email template.
#### Testing
Before activating your campaign trigger sync, it's best to test it with with an example lead whose email inbox you have access to.
This lets you inspect the test email.
Click **Test** from the sync configuration page.
You can manually overwrite columns of a test row.
In the example below, since the sync uses the **Email** column to match leads, you can manually overwrite the **Email** field to test lead's email address.
If you're using the **Marketo GUID** column to match leads, you need to manually overwrite the **Marketo GUID** field to the test lead's **Marketo GUID** to send a test email.
Once you've altered the email or Marketo GUID, press **Sync as added row**. This triggers the campaign for the test lead and sends an email to their inbox.
Check the test lead's email inbox to verify that your configuration sent the intended email with custom tokens populated as expected.
## Tips and troubleshooting
### Common errors
{/* */}
#### Code 413 "Request Entity Too Large"
This error means that size of Hightouch's API request to Marketo was too large.
Lower the [batch size](#batch-size) to mitigate it.
#### Code 603 "Access denied"
This error means that authentication to Marketo was successful but the account you authenticated with doesn't have sufficient permission to call the Marketo API.
For more information, see [Marketo's help article](https://experienceleague.adobe.com/en/docs/experience-cloud-kcs/kbarticles/ka-29391).
As explained in the [Create Marketo service](#create-marketo-service) section, ensure that:
- you assigned all permissions listed in the table to the API access role (especially the [**Read-Write Activity** permission](https://experienceleague.adobe.com/docs/marketo/using/product-docs/administration/users-and-roles/descriptions-of-role-permissions.html#access-api)),
- and that the Marketo API user has access to the Marketo [workspaces and partitions](https://experienceleague.adobe.com/en/docs/marketo/using/product-docs/administration/workspaces-and-person-partitions/understanding-workspaces-and-person-partitions) you want to use in Hightouch.
#### Code 1003 "Campaign is not part of a program"
To resolve this error, move your campaign into a program.
#### Code 1007 "Multiple lead match lookup criteria"
To resolve this error, remove any duplicate rows with the same primary key in your model.
#### Code 1013 "Record not found"
This error can happen if you select [Update mode](#supported-syncing) when syncing to standard and custom objects. If a record doesn't exist in Marketo yet, you need to select [Upsert mode](/syncs/types-and-modes#sync-modes) instead, which updates existing records and creates new ones, if needed.
{/* */}
### Live debugger
### Sync alerts
---
## Mattermost
**URL:** https://hightouch.com/docs/destinations/mattermost
**Description:** Sync your warehouse with Mattermost and push business metrics and user activity data to Mattermost in real-time
**Section:** Destinations
## Overview
With the Mattermost destination, Hightouch can serve as a notifier for various changes. This is a very flexible integration, but here are a few example use cases:
1. Notify your customer success teams when product usage for a key account suddenly drops
2. Notify an account executive when product usage surpasses a certain threshold
3. Send a daily summary of the number of sign ups, compared to the same day last month
4. Send a daily summary on the adoption of a core feature
## Setup
Use personal access tokens to connect to Mattermost. Check with your Mattermost administrator that personal access tokens are enabled.
1. In Mattermost, click your icon in the top right and go to Profile to access your personal settings
2. Click the Security tab in the modal that appears
3. Click Personal Access Tokens and create the token
## Sending Mattermost messages \(added / changed / removed\)
Hightouch will not send any messages on the initial run to prevent a flood of
messages to your channel. Messages will start after the this initial sync.
Hightouch supports sending messages based on rows that have been added, changed, or removed since the last sync run. Each row is sent as an individual message and can be sent to different channels and templated with Liquid.
Message formatting is best when the following conditions are met:
1. You want to write a query, and have Hightouch detect the added and removed rows
2. You want a custom message sent per each added and removed rows
### Channel selection
Hightouch supports both sending messages to a specific channel that the Hightouch bot is part of, or by pulling the channel ID each row. Enable the "Use column" toggle to dynamically set the target channel.
### Added, changed, and removed messages
Hightouch allows you to template the content you would like to send in your message.
- When a row is added in your query results, Hightouch will send the row added content.
- When a row is changed in your query results, Hightouch will send the row changed content.
- When a row is deleted in your query results, Hightouch will send the row deleted content.
This content can be [templated](#liquid-templating).
## Sending Mattermost messages \(all results\)
The batch message mode is similar to the normal message mode, except it sends all rows in the query results rather than only added / changed / removed rows since the last sync.
This mode also allows you to specify how many rows are sent per message, which can range from 1 to 20. With 1 row per message, it would be similar to the message mode, whereas 20 rows per message would be similar to a tabular representation.
This format is good for the following scenarios.
1. You want to write a query, and have Hightouch push a message containing **all** rows, formatted as a message, to Mattermost
2. You don't want Hightouch to detect added and removed rows, but instead want to send all rows on each query run
### Channel selection
The batch message only supports sending your messages to a single channel.
### Header, body, and footer content
For each message, Hightouch allows setting the header, footer, and row content. Only the row content is required. The row content is repeated per row in the message and has access to the row values via templating.
This content can be [templated](#liquid-templating).
## Templating messages
### Liquid templating
For both the message and batch message modes, [Liquid templating ](https://shopify.github.io/liquid/)is supported. The most important implication of this is that columns from the added or removed row can be accessed as part of the message body. Here is an example message making use of Liquid to insert a column into the message.
```text
User with name {{ row["name"] }} and email {{ row["email"] }} has signed up
```
### Tagging Mattermost users \(@...\)
Hightouch supports tagging users by either their Mattermost username \(if you have it in your warehouse\) or by email.
- **Tagging by username:** tag by username by writing `<@{{ row["username"] }}>` where username is the column that contains the user's Mattermost username, just as using Liquid templating. If the username is a static value, you can write something like `<@exampleuser>`.
- **Tagging by Email:** tag by email by writing `<@{{ row["email"] | mattermost_user_lookup }}>` where email is the column that contains the user's email used in Mattermost. If the email is a static value, you can write something like `<@{{ "example@gmail.com" | mattermost_user_lookup }}>`.
To tag by email, we require the permission users:read.email.
### Unfurling links and blocking unfurl
By default, Mattermost automatically unfurls (displays a preview) URL links in your messages. To prevent automatic unfurling, surround the URL with backticks, for example:
`https://www.hightouch.com/`
(Note: The backtick is usually located above the tab key on the left-hand side of the keyboard).
## Table
Table formatting sends all of your query results formatted as a text table as messages to Mattermost. This mode is recommended only for small sets of data.
This format is good for the following scenarios.
1. You want to write a query, and have Hightouch push a message containing **all** rows, formatted as a table, to Mattermost
2. You don't want Hightouch to detect added and removed rows, but instead want to send all rows on each query run
### Channel selection
The batch message only supports sending your messages to a single channel.
## Using webhooks
If you enable the option to use webhooks for sending messages, ensure the user whose access token was used has permissions to create and get webhooks. Using webhooks allows you to set how the sender is displayed when sending the message. This can be configured with the following fields
| Name | Description |
| :------------- | :---------------------------------------------------------------------------- |
| Icon Image URL | URL of 48x48 image of the icon that you wish to use |
| Icon Emoji | Mattermost emoji \(for example, `:sweat_smile:`\), this will overwrite icon image URL |
| Bot Name | Display name of bot |
(Note: Using webhooks for the CSV mode is not supported since files can't be uploaded using webhooks)
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Maxio
**URL:** https://hightouch.com/docs/destinations/maxio
**Description:** Unlock the full potential of your subscription business by syncing data from your data warehouse to Maxio
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
------------------|-----------------------------------------------------------------|----------------------|------------------------------------------------------------------------------------------------------
**Customers** | Sync data from any source to Customers | Upsert, Update, Insert | [Customers docs](https://developers.chargify.com/docs/api-docs/18237bcfe5cbb-create-customer)
**Subscriptions** | Sync data from any source to Subscriptions | Upsert, Update, Insert | [Subscriptions docs](https://developers.chargify.com/docs/api-docs/d571659cf0f24-create-subscription)
**Usage** | Sync data from any source to Usage | Insert | [Usage docs](https://developers.chargify.com/docs/api-docs/9954fb40d117d-create-usage)
**Events** | Sync data from any source to Events-Based Billing | Insert | [Event docs](https://developers.chargify.com/docs/api-docs/b8ffb590f236a-event-ingestion)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Maxio
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Maxio** and click **Continue**. You can then authenticate Hightouch to **Maxio** by entering the following required fields into Hightouch:
- **Subdomain**: Part of your URL, for example, `example-subdomain` in `https://example-subdomain.chargify.com`
- **API key**: A valid Maxio API key, follow [Maxio's instructions](https://maxio-chargify.zendesk.com/hc/en-us/articles/5405281550477#api) to view or create a new API key
- **Password**: The same one you used to log into your account
## Sync configuration
Once you've set up your Maxio destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Maxio destination you want to sync to.
### Syncing customers
When syncing customers, you can choose from upsert, update, or insert modes.
#### Record matching
To match rows from your model to customers in Maxio, you need to select a model column and corresponding Maxio field. You can match on any of the following Maxio fields:
- **email**
- **id**
- **reference**
In **Insert** mode, Maxio automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in Maxio. Maxio requires the following fields, so you must map them to complete your configuration:
- **first_name**
- **last_name**
- **email**
Ensure the data types of your model columns match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|----------------------------------------------------------
**Do nothing** | Keep the record in Maxio with all its synced fields
**Clear** | Unset all the mapped fields, but keep the record in Maxio
**Delete** | Delete the synced record from your Maxio customer
### Syncing subscriptions
When syncing subscriptions, you can choose from upsert, update, or insert modes.
#### Record matching
To match rows from your model to records in Maxio, you need to select the model column that contains values that match the **reference** field.
In **Insert** mode, Maxio automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in Maxio. Maxio requires the following fields, so you must map them to complete your configuration:
- **customer_attributes**, **customer_id** or **customer_reference**
- **payment_profile_id**
- **product_handle** or **product_id**
Ensure the data types of your model columns match the data types of the fields you want to sync to.
### Syncing usage data
Hightouch lets you report usage events to Maxio. To do so, you must select the Maxio subscription and component you would like to sync to.
#### Record matching
Maxio automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
You can sync columns from your model to Maxio default usage fields. Check out Maxio's [usage docs](https://developers.chargify.com/docs/api-docs/9954fb40d117d-create-usage) to learn more.
### Syncing events
Maxio requires the following event parameters:
- `api-handle`: You can either provide a static value or select to **use a column** from your model.
- `subdomain`: Hightouch uses the subdomain you entered when [connecting to Maxio](#connect-to-maxio)
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server. Hightouch syncs the timestamp as the `event_time` parameter the Maxio requires.
#### Record matching
Maxio automatically generates an identifier for every new event synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync event properties via default and custom mappings.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Medallia
**URL:** https://hightouch.com/docs/destinations/medallia
**Description:** The leading experience management platform used by companies around the world to deliver exceptional experiences
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|-----------|------------------------------------------------------|----------------------|--------------------------------------------------------------------------------|
| **Users** | Sync data from any source to Medallia users | Upsert | [Users docs](https://developer.medallia.com/medallia-apis/reference/listusers) |
| **Data** | Sync data from any source to the Medallia Import API | Insert, All | [Import docs](https://developer.medallia.com/medallia-apis/reference/import-api) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Medallia
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Medallia** and click **Continue**. You can then authenticate Hightouch to **Medallia**.
Enter the following required fields into Hightouch:
- (Optional) **Company name**: Enter your company name.
- **Tenant name**: Enter your Medallia tenant name.
- **Client ID**: Enter your client ID.
- **Client secret**: Enter your client secret.
To obtain your client ID and client secret, go to your Medallia setup console and follow these steps:
- Navigate to **Integrations** → **Oauth**.
- Find the client you want to use and copy your **Client ID**.
- If you don't have a client secret for the selected client, generate a new client secret.
- Navigate to **Company** → **Users**
- Go to the **User Edit Roles** tab and ensure the client has sufficient permissions to edit the roles you will be syncing.
## Sync configuration
Once you've set up your Medallia destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Medallia destination you want to sync to.
### Syncing users
Sync data from any source to Medallia users.
#### Record matching
To match rows from your model to users in Medallia, you need to select the model column that contains the **Username** values.
#### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default contact fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
### Syncing to an Import API
Sync data from any source to any Medallia Import API.
In the **Which destination would you like to import to?** field, enter the **URL Unique Name** of the Medallia Web Feed you want to target. This value must exactly match the URL Unique Name configured in Medallia’s Web Feeds configuration screen. Medallia doesn’t automatically generate this value, so an admin can choose any string, but the value in Hightouch and the value in Medallia must match for imports to succeed.
This field is specific to Medallia and is separate from the Hightouch **Sync slug**, which is only used inside Hightouch for URLs and logging and is not read by Medallia.
#### Field mapping
Hightouch lets you sync import fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default contact fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
## Tips and troubleshooting
### Common errors
If your sync runs successfully in Hightouch but no data appears in Medallia, confirm that the value in **Which destination would you like to import to?** exactly matches the Web Feeds **URL Unique Name** in Medallia.
### Live debugger
### Sync alerts
---
## Meta Custom Audiences
**URL:** https://hightouch.com/docs/destinations/meta
**Description:** Sync first-party audiences to Meta Ads Manager for better targeting, lookalikes, and suppression across your campaigns.
**Section:** Destinations
## Overview
Deliver even more relevant ads by combining data from various sources within your data warehouse to build custom audiences for Meta Ads. By keeping your custom audiences updated automatically, never show an ad to someone after they purchase the item you were promoting.
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| ------------- | -------------------------------------------------------------------------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------- |
| [**Audiences**](#syncing-custom-audiences) | Create and update custom audiences in Meta | Add, Remove | [Custom Audiences docs](https://developers.facebook.com/docs/marketing-api/audiences/guides/custom-audiences) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Meta Custom Audiences
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Meta Custom Audiences** and click **Continue**. You can then authenticate Hightouch to Meta Custom Audiences either with a system user token or with OAuth.
Since the OAuth flow requires you to manually refresh this connection every 60 days, we strongly recommend using a system user token for indefinite access.
Make sure to select an **Ad Account** on the destination configuration page.
Otherwise, your sync fails with an [`'undefined' does not
exist`](#unsupported-post-request-object-with-id-undefined-does-not-exist)
error.
### Authenticate with a system user token
When authenticating with a system user token you can choose to either use an existing one or create a new one for the Hightouch integration.
To use an [existing system user token](#use-an-existing-system-user-token), you must meet the following prerequisites:
- The system user whose token you'd like to use has the `Employee` role. (`Admin` also works, but [Meta limits each business to a single Admin system user](https://developers.facebook.com/docs/business-management-apis/system-users/overview) and recommends using a regular system user to access assets, so `Employee` is the practical choice.)
- You've created an app to add as an assigned asset to the system user. For detailed instructions, see steps 1 - 5 from [creating a new system user token](#create-a-new-system-user-token).
- You've assigned the ad account that you want to create audiences for to the system user. For detailed instructions, see step 8 from [creating a new system user token](#create-a-new-system-user-token).
If you haven't met these prerequisites, follow the instructions for [creating a new system user token](#create-a-new-system-user-token).
#### Create a new system user token
To create a new token, you first need to add a new app.
You create the app in the Meta developer portal, then create the system user and generate its token in **Settings** at [business.facebook.com](https://business.facebook.com).
{/* VERIFY (use case, step 2 below): Functionality is confirmed — per Meta's Use Case Permission Mapping (https://developers.facebook.com/documentation/development/create-an-app/use-cases-permission-mapping, updated Aug 29 2025), all three Marketing API use cases require `ads_management` and offer `business_management` as optional, so any of them works for this integration. The only open item is which one the platform team wants as the recommended default. */}
Meta is replacing its older app-creation workflow with a use-case-based flow. The **legacy workflow** is still available as the **Other** option (under **Looking for something else?**, marked "going away soon"), which creates the app in Meta's old experience where you choose permissions manually.
**Existing apps** created with **Other** keep working — you don't need to recreate them.
**New apps** should follow the use-case flow below: selecting any of the three Marketing API use cases (**Create & manage ads with Marketing API**, **Measure ad performance data with Marketing API**, or **Capture & manage ad leads with Marketing API**) attaches the `ads_management` permission this integration needs.
Create the app in the [Meta developer portal](https://developers.facebook.com/apps/creation/). Meta's wizard walks you through five steps: **App details**, **Use cases**, **Business**, **Requirements**, and **Overview**:
1. Enter a name and contact email.
2. Select a Marketing API use case such as **Create & manage ads with Marketing API**. Per Meta's [use case permission mapping](https://developers.facebook.com/documentation/development/create-an-app#use-case-permission-mapping), any of the three attaches the `ads_management` this integration needs and makes `business_management` available — you select both when generating the token (step 12).
3. Connect the business portfolio that owns the ad account you want to sync to.
4. Review any listed requirements — typically none when you're accessing your own ad account.
5. Review the summary, then select **Create app**.
6. In **Settings** at [business.facebook.com](https://business.facebook.com), go to **Users** → **System users** and add a new system user with the **Employee** role. [Meta caps each business at one Admin system user and recommends a regular system user for asset access](https://developers.facebook.com/docs/business-management-apis/system-users/overview).
7. Select the system user you just created, then click **Assign assets**.
8. Assign **_both_** the app (under **Apps**) and the ad account you want to create audiences for (under **Ad accounts**), granting each **Full access** — not **Partial access** — then click **Assign assets**.
9. Back on the system user, select **Generate token**.
10. Choose the app you created to generate the token.
11. Set the expiration policy to **Never** so the token doesn't lapse and interrupt your syncs. (Meta labels **60 days** as recommended, but a system user token is meant for indefinite access.)
12. Ensure that both the `ads_management` and `business_management` permissions are selected, then click **Generate token**. Hightouch uses `ads_management` to create and update your custom audiences, and `business_management` to list ad accounts, select existing audiences, and run the test connection.
You can't change a token's permissions after it's generated. To add a permission later, you need to generate a new token.
13. Copy the generated token. Since this token won't be stored in Meta, you may want to consider storing it in a secure password vault as well.
14. Back in Hightouch, select **Use system user token** as the **Authentication method**, then paste the token you generated as the **System User Token**, select the relevant **Ads Account**, and click **Continue**.
#### Use an existing system user token
If you already have a Meta app and system user setup, in **Settings** at [business.facebook.com](https://business.facebook.com), go to **Users** → **System users**.
1. Once you've selected the existing user and assets are assigned, select **Generate token**.
2. Choose the app you created to generate the token.
3. Set the expiration policy to **Never** so the token doesn't lapse and interrupt your syncs. (Meta labels **60 days** as recommended, but a system user token is meant for indefinite access.)
4. Ensure that both the `ads_management` and `business_management` permissions are selected, then click **Generate token**. Hightouch uses `ads_management` to create and update your custom audiences, and `business_management` to list ad accounts, select existing audiences, and run the test connection. You can't change a token's permissions after it's generated — to add a permission later, generate a new token.
5. Copy the generated token. Since this token won't be stored in Meta, you may want to consider storing it in a secure password vault as well.
6. Back in Hightouch, select **Use system user token** as the **Authentication method**, then paste the token you generated as the **System User Token**, select the relevant **Ads Account**, and click **Continue**.
### Authenticate with OAuth
For the **Authentication method**, select **Log in to Meta** and log into your Meta account.
Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Meta Custom Audiences destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Meta Custom Audiences destination you want to sync to.
### Syncing custom audiences
Hightouch lets you create and maintain custom audiences via the [Meta Custom Audience API](https://developers.facebook.com/docs/marketing-api/reference/custom-audience/).
Each sync adds new users and updates existing users' identifiers you include in [record matching](#record-matching).
You can remove users that leave your model's query result by selecting this option in the sync's [delete behavior](#delete-behavior).
#### Select an existing audience or create a new one
You can create a new audience or use an existing one. When creating a new audience, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model. To use an existing audience, select the desired audience from the dropdown.
#### User identifiers
To identify which users to add or update in an audience, select model columns and the corresponding Meta Custom Audience fields.
You can match on default, custom, and Meta-specific fields.
Meta-specific fields include:
- [**External ID**](https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/external-id/): a custom identifier that you send to Meta via its APIs or tracking pixels
- **Page-scoped ID**: an ID used for Meta messenger apps and Meta pages
To increase the match rate for your records, include as many fields as possible.
#### Value-based audiences
If you're creating a new Meta Audience, you can create a [value-based audience](https://www.facebook.com/business/help/917879191754763?id=401668390442328) by mapping a numerical model column containing lifetime values to the **Lookalike Value** field.
You can then create a lookalike audience in Meta that finds the people most similar to your highest value customers. For more information refer to [Meta's docs](https://www.facebook.com/business/help/458132024732845?id=401668390442328).
#### Handling PII and hashing
By default, Hightouch automatically hashes the following fields before sending
requests to Meta:
- **Email**
- **Phone**
- **Gender**
- **First Name**
- **Last Name**
- **First Initial**
- **State**
- **City**
- **Zip Code**
- **Country**
You can disable this behavior in the sync configuration. If disabled, the data from the model should be appropriately normalized and hashed according to [Meta's hashing requirements](https://developers.facebook.com/docs/marketing-api/audiences/guides/custom-audiences/#hash).
#### Share your audience
Optionally share this audience with another Meta advertising account.
#### Delete behavior
You can choose how to handle user records in Meta when the corresponding rows are deleted in your source.
| Behavior | Description |
| -------------------- | ----------------------------------------------------- |
| **Do nothing** | Keep the contact in your Meta Custom Audience |
| **Remove from list** | Remove the contact from your Meta Custom Audience |
## Validate your setup
After you configure your sync, run a test to confirm data flows correctly:
1. Start with a small test audience or a single record.
2. Trigger a manual sync from the [Syncs overview](https://app.hightouch.com/syncs?from=docs) page.
3. After the sync completes, verify the audience in Meta Ads Manager:
- Go to [Audiences](https://adsmanager.facebook.com/adsmanager/audiences) and search for the audience name you configured in Hightouch.
- Confirm the audience size reflects the expected number of records. Meta may take up to 24 hours to fully process the audience.
4. If the sync shows errors, check the sync run log in Hightouch for specific API error messages.
---
## Tips and troubleshooting
### Matched users count
Meta returns 1k as the default when there is no data or audience is too
small
### Common errors
**Custom Audience Terms not yet accepted**
The full error message is:
```txt
An error occurred when creating the audience. Meta Custom Audiences API returned error with code 400: Custom Audience Terms not yet accepted: You'll need to agree to the Custom Audience terms before you can create or edit an audience or an ad set. See Meta, Custom Audience Terms.
```
To resolve this error, go to [Audiences](https://adsmanager.facebook.com/adsmanager/audiences) in your Meta Ads Manager account and click **Create a Custom Audience** to create a list manually. This prompts the Terms of Service to appear so you can agree to them.
You can also go to `https://business.facebook.com/ads/manage/customaudiences/tos/?{ACCOUNT_ID}` to accept the Terms of Service for each ad account that you want to use. Replace `{ACCOUNT_ID}` with the ad account ID in `act_xxxx` format.
Once you've accepted the Terms of Service, rerun your sync with the error.
**Error Code 400: Missing schema attribute in payloads**
By default, Hightouch automatically [hashes certain fields](#handling-pii-and-hashing) before sending them to Meta.
The error occurs when Hightouch attempts to hash a `null` value or `null` values are being sent to Meta.
To resolve this, [remove rows containing `null` values](/models/sql-editor#exclude-rows-with-null-values) from your model's query results or enable the [**Don't sync null values** option](/syncs/mapping-data#dont-sync-null-values) in the [advanced mapper](/syncs/mapping-data#advanced-mapper).
**Unsupported post request. Object with ID 'undefined' does not exist**
This error occurs if you don't select an **Ad Account** on the destination configuration page.
Make sure to follow the destination setup procedure outlined in the [Connect to Meta Custom Audiences](#connect-to-meta-custom-audiences) section.
**401 - Error validating access token**
As explained in the [Connect to Meta Custom Audiences](#connect-to-meta-custom-audiences) section, authenticating with OAuth requires you to reauthorize your destination **every 60 days**. Otherwise, your syncs fail with this error message.
Authenticate with a [system user
token](#authenticate-with-a-system-user-token) to avoid this reauthorization
requirement.
**400 - OAuthException code 190: Token expired**
The full error may look like: `400 - {"error":{"message":"The token has expired on...","type":"OAuthException","code":190}}`.
Meta access tokens expire periodically. Tokens also become invalid when the authorizing user changes their password or when Meta invalidates the session for security reasons. In the latter case, the error includes `error_subcode: 460`.
To resolve this error, go to your Meta Custom Audiences destination in Hightouch and click **Reconnect**. The authorizing user must have `ads_management` permission on the ad account. To avoid recurring token expiration issues, authenticate with a [system user token](#authenticate-with-a-system-user-token) instead of OAuth.
**403 - Ad account permissions error (code 200)**
The full error may look like: `(#200) Ad account owner has NOT grant ads_management or ads_read permission`.
This error occurs when the user who authorized the Meta connection doesn't have the required permissions on the ad account. To resolve it, in **Settings** at [business.facebook.com](https://business.facebook.com), go to **Users** → **System Users** and grant the user `ads_management` or `ads_read` permission on the relevant ad account. Then reconnect the destination in Hightouch.
**400 - (#100) Requires business_management permission**
The full error may look like: `400 - {"error":{"message":"(#100) Requires business_management permission to access the field.","type":"OAuthException","code":100}}`.
This error occurs when your access token is missing the `business_management` permission. Hightouch needs this permission to list ad accounts, select existing audiences, and run the test connection.
If you authenticated with a [system user token](#authenticate-with-a-system-user-token), regenerate the token in **Settings** → **Users** → **System Users** at [business.facebook.com](https://business.facebook.com) with both `ads_management` and `business_management` selected, then reconnect the destination in Hightouch.
If you authenticated with OAuth, click **Reconnect** and approve all requested permissions when prompted.
**400 - (#100) unsupported get request**
The full error may look like: `400 - {"error":("message":"(#100) Unsupported get request. Please read the Graph API documentation at https://developers.facebook.com/docs/graph-api {...}`.
This error usually happens when one or multiple Meta pages that were granted access to the [system user](#authenticate-with-a-system-user-token) have **age restrictions** or **country restrictions** enabled. To resolve the error, edit your system user to remove all pages that have these age or country restrictions. Otherwise, you can [edit the page settings](https://www.facebook.com/help/www/778445532225441) to remove the restrictions from these pages.
**Clear and fill**
Upon request, [clear and fill](/syncs/overview#clear-and-fill) can be enabled for Meta. Running the clear operation for Meta may take up to 24 hours to complete.
### Live debugger
### Sync alerts
---
## Meta Product Catalog
**URL:** https://hightouch.com/docs/destinations/meta-catalog
**Description:** Power Meta's Advantage+ and dynamic advertising campaigns with up-to-date product data synced directly from your data warehouse.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| --------- | ------------------------------------------ | -------------------- |
| Products | Sync records to products in your catalogs. | Upsert |
## Setup
If you don't have a product catalog already, navigate to Meta [Commerce Manager](https://business.facebook.com/commerce/) and click "Add Catalog" to create one.
## Connect to Meta Product Catalog
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Meta Product Catalog** and click **Continue**.
You can then authenticate Hightouch to your Meta Product Catalog either with a system user token or with OAuth.
Since the OAuth flow requires you to manually refresh this connection every 60 days, we strongly recommend using a system user token for indefinite access.
Make sure to select an **Ad Account** on the destination configuration page.
Otherwise, your sync fails with an [`'undefined' does not
exist`](#unsupported-post-request-object-with-id-undefined-does-not-exist)
error.
### Authenticate with a system user token
When authenticating with a system user token you can choose to either use an existing one or create a new one for the Hightouch integration.
To use an [existing system user token](#use-an-existing-system-user-token), you must meet the following prerequisites:
- The system user whose token you'd like to use has `Admin` access.
- You've created an app to add as an assigned asset to the system user.
- You've assigned the ad account where your catalog lives for to the system user.
When you generate the system user token, select the `ads_management`, `business_management`, and `catalog_management` permissions. Hightouch needs `business_management` to list catalogs and run the test connection. Without it, these operations fail with `(#100) Requires business_management permission to access the field`.
### Authenticate with OAuth
For the **Authentication method**, select **Log in to Meta** and log into your Meta account.
Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Meta Product Catalog destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data.
Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Meta Product Catalog destination you want to sync to.
### Syncing products
Hightouch supports syncing to the following Meta catalog products:
- `Product items`
- `Vehicles`
### Record matching
Product records can be matched from your source to your Meta catalog workspace by the product ID in the Upsert mode.
### Field mappings
You can sync columns from your source to Meta catalog's default and custom fields. Hightouch automatically detects existing custom fields from your Meta catalog account.
Review the [documentation](https://developers.facebook.com/docs/marketing-api/catalog-batch/reference#supported-fields-items-batch) on what fields are required to sync to your catalog type.
## Tips and troubleshooting
### Common errors
**400 - (#100) Requires business_management permission**
The full error may look like: `400 - {"error":{"message":"(#100) Requires business_management permission to access the field.","type":"OAuthException","code":100}}`.
This error occurs when your access token is missing the `business_management` permission. Hightouch needs this permission to list catalogs and run the test connection.
If you authenticated with a [system user token](#authenticate-with-a-system-user-token), regenerate the token in **Meta Business Settings → Users → System Users** with the `ads_management`, `business_management`, and `catalog_management` permissions selected, then reconnect the destination in Hightouch.
If you authenticated with OAuth, click **Reconnect** and approve all requested permissions when prompted.
### Live debugger
### Sync alerts
---
## Meta Conversions API
**URL:** https://hightouch.com/docs/destinations/meta-conversions
**Description:** Optimize campaigns and gain insight into ad performance by sending web, app, phone, and in-store conversion events to Meta from your data warehouse.
**Section:** Destinations
## When to use Meta Conversions API
Use this destination to improve Meta campaign performance and measurement by sending offline or server-side conversion events (for example, purchases, form fills, or calls) to Meta.
**Use Meta Conversions API when you want to:**
- Attribute conversions that happen outside the browser (for example, in your CRM or POS system)
- Improve ad delivery and targeting with more complete conversion data
- Measure performance when cookies or browser signals are limited
---
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-----------|-------------------------------------------------------------|----------------------|-----------------------------------------------------------------------------------
**Events** | Sync conversion events to Meta’s Conversions API | Insert | [Conversions API reference](https://developers.facebook.com/docs/marketing-api/conversions-api/)
See [Sync modes](/syncs/types-and-modes#sync-modes) for more information.
## Connect to Meta Conversions API
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Meta Conversions** and click **Continue**. You can then authenticate Hightouch to the Meta Conversions API by entering the following required fields into Hightouch:
1. In Hightouch, go to [**Destinations**](https://app.hightouch.com/destinations?from=docs) and click **Add destination**.
2. Select **Meta Conversions API** and click **Continue**.
3. Enter the following credentials:
- **Pixel ID** or **Database ID**
- **Access Token**
Find these in [Meta Events Manager](https://business.facebook.com/events_manager2/):
1. Open the **Data Sources** tab and select your Pixel.
2. Under **Settings**, find your **Pixel ID** in the **Details** section.
3. To create an Access Token, scroll to **Conversions API** and click **Generate access token**.
---
## Configure the sync
After connecting Meta Conversions API and selecting a [model](/getting-started/concepts#models), create a sync to start sending data.
1. Go to [**Syncs**](https://app.hightouch.com/syncs?from=docs).
2. Click **Add sync**, choose your model, and select **Meta Conversions API** as the destination.
### Required event fields
The Meta Conversions API requires the following event parameters:
- `event_name`
- `event_time`
- At least one [customer information (`user_data`)](https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/customer-information-parameters) field
The sync configuration form ensures all these are set and provides some additional options.
#### Event name
Providing an event name is required to send an event to the Meta Conversions API. You can either provide a static value or select to **use a column** from your model. Hightouch syncs the static or column value as the `event_name` parameter that the Conversions API requires.
Use Meta’s [Standard Events](https://www.facebook.com/business/help/402791146561655?id=1205376682832142) when possible (for example, `Purchase`, `Lead`, or `AddToCart`) to improve reporting accuracy.
#### Event time
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server. Hightouch syncs the timestamp as the `event_time` parameter the Conversions API requires.
If you select a column, it should be in [ISO 8601](https://www.iso.org/iso-8601-date-and-time-format.html) format. Hightouch will then automatically convert the data into the format that Meta expects.
#### Action source
The [`action_source`](https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/server-event#action-source) parameter specifies where your conversions occurred (e.g., web, app, email).
Knowing where your events took place helps Meta optimize delivery and attribution.
Certain source types require specific fields:
- **Web** events must include `event_source_url` and `client_user_agent`.
- **App** events must include `advertiser_tracking_enabled`, `application_tracking_enabled`, and `extinfo`.
If not all rows contain this field, you can set a **fallback action source** in your sync configuration.
#### Field mappings
Map your customer information and event parameters here.
The Meta Conversions API requires at least one [`user_data`](https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/customer-information-parameters) parameter for attribution.
Include as many `user_data` fields as possible to improve [Event Match Quality](https://www.facebook.com/business/help/765081237991954?id=818859032317965).
If using a **Standard Event** (like `Purchase`), ensure you include all required fields (e.g., `value`, `currency`) and any relevant optional ones.
Custom fields are sent in the `custom_data` parameter.
#### Advanced analytics
To send data through the Conversions API to the Advanced Measurement API (AMAPI), map the following fields:
- `data_processing_options`: an array of strings like `["AMO"]`
- `advanced_measurement_table`: a string
#### Test event codes
Test event codes are special codes you can include in your event payload to simulate conversion events. This allows you to test your API implementation without actually recording real conversion data.
You can optionally enter a test event code as part of your sync configuration. You can find your test event code in your Meta [Events Manager](https://business.facebook.com/events_manager2/) in the **Test events** tab.
Make sure to remove the test event code if you're running a production sync.
#### Hashing
By default, Hightouch automatically hashes the following fields:
- **email**
- **phone**
- **gender**
- **dob**
- **last name**
- **first name**
- **city**
- **state**
- **zip**
- **country**
If you want to turn hashing off, select **No** under **Would you like Hightouch to automatically detect if your PII data requires hashing?**.
---
## Event Data Quality
Hightouch provides insight into the effectiveness of your conversion API implementation by reporting on **Event Match Quality (EMQ)** and **Additional Conversions Reported**.
Data quality metrics appear in the **Overview** tab of your Meta Conversions API sync.
Your access token must include Data Quality API permissions. If metrics don’t appear, generate a new token in Meta Events Manager.
### Event Match Quality (EMQ)
The EMQ score estimates how well your conversion events match to Meta users, ad clicks, and impressions. It’s based on the “fill rate” of key `user_data` values you send with each conversion event.
A higher EMQ score generally means better attribution accuracy and ad performance, as Meta’s algorithms can use more complete data to optimize bids, budgets, and targeting.
To improve your EMQ score:
- Collect and sync more `user_data` fields with each event.
- Use [**Match Booster**](/match-booster/overview) to supplement user data with Hightouch’s third-party identity graph.
For guidance on which user attributes to prioritize, see the [Conversion API schema guide](https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/customer-information-parameters/). For more about EMQ, see [Meta’s documentation](https://www.facebook.com/business/help/765081237991954?id=818859032317965).
### Additional Conversions Reported
The Additional Conversions Reported metric helps you understand how much your business benefits from using the Conversions API alongside the Meta Pixel. More reported conversions can help you decrease your cost per result and show your ads to people who find them relevant.
Learn more in [Meta’s Additional Conversions Reported guide](https://www.facebook.com/business/help/409195343791109).
---
## Tips and troubleshooting
### Common errors
#### Meta SDK
When setting up a new datasource for CAPI, if you don't want to use the Meta SDK in your app, choose “Web” (even if the data would include app events as well). This eliminates the need to install the SDK.
In this case, use **Dataset ID** instead of **Pixel ID** when configuring the destination.
#### App data
To send app data in your CAPI events, include [app data parameters](https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/app-data#).
### Live debugger
### Sync alerts
---
## Microsoft Dynamics 365
**URL:** https://hightouch.com/docs/destinations/microsoft-dynamics
**Description:** Sync data into Dynamics 365 by writing to Dataverse tables.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
------------|-------------------------------------------------------------|------------------------|----------------------------------------------------------------------------------------------------------------------------------
**Objects** | Sync data from any source to Microsoft Dynamics 365 objects | Upsert, Update, Insert | [Objects docs](https://learn.microsoft.com/en-us/power-apps/developer/data-platform/webapi/reference/about?view=dataverse-latest)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Microsoft Dynamics 365
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Microsoft Dynamics 365** and click **Continue**. You can then authenticate Hightouch to **Microsoft Dynamics 365**.
### Authenticate with oauth
For the **Authentication method**, select **Log in to Microsoft Dynamics 365** and log into your Microsoft Dynamics 365 account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
You must enter your **Web API endpoint** in Hightouch to complete the setup. See [How to get your **Web API endpoint**](microsoft-dynamics#how-to-get-your-web-api-endpoint) for information on how to get it.
#### Access level recommendations
We recommend that you authenticate with a user that has the correct access to the environment that you will be working on.
### Authenticate with client credentials
Enter the following required fields into Hightouch:
- [**Web API endpoint**](microsoft-dynamics#how-to-get-your-web-api-endpoint)
- [**Tenant ID**](microsoft-dynamics#setting-up-your-app-registration)
- [**Client ID**](microsoft-dynamics#setting-up-your-app-registration)
- [**Client Secret**](microsoft-dynamics#setting-up-your-app-registration)
#### Setting up your app registration
1. Go to [Azure Ad portal](https://portal.azure.com/) > **App registrations**.
2. Create a single tenant app registration.
3. Once an app registration is created, copy the **Tenant ID** and **Client ID** in Hightouch.
4. To get a **Client Secret**, navigate to **Certificates & secrets** > **New client secret**.
Copy the value of **Client Secret** in Hightouch.
5. In order to use the created app registration with Microsoft Dynamics 365 API, You must grant API permission to it by going to **API Permissions** > **Add Permissions** then select **Dynamics CRM** and add the `user_impersonation` permission.
6. Finally, create a non-interactive application user with the app registration in the [Power Platform admin center](https://admin.powerplatform.microsoft.com/) with `Service Writer` role. To do this **Environments** > **[the environment you want to connect]** > **Settings** > **Application users** > **New app user**.
## How to get your Web API endpoint
In order to complete your Dynamics 365 connection setup, you will need to provide your service URL. Follow these steps to get your Web API endpoint:
1. Sign in to your [Microsoft Power Apps](https://powerapps.microsoft.com/).
2. Click the settings icon on the top right corner to open the settings sidebar.Then, click on *Developer resources*.
3. Copy the *Web API endpoint* then go back to Hightouch and paste the URL where it's required.
## Prerequisites
Before you set up a sync, we recommend you to skim through these concepts to gain an insight on how to effectively sync your data into Microsoft Dynamics 365.
### Dataverse
Dataverse is the source of truth for your Dynamics 365 applications.
Microsoft Dynamics 365 destination will sync your data into the Dataverse which will then allows you to use the data directly in your Dynamics 365 applications.
For more information about Dataverse see [Microsoft's documentation](https://docs.microsoft.com/en-us/power-apps/maker/data-platform/data-platform-intro).
#### What is a GUID?
GUID is the auto-generated primary key in your Dataverse tables. You are allowed to use GUID to match query results in Hightouch with rows in your Dataverse tables by default.
{" "}
Although GUIDs are available by default, we recommend the use of alternate keys
for record matching in your Hightouch syncs.
#### What are alternate keys?
[Alternate keys](https://docs.microsoft.com/en-us/dynamics365/customerengagement/on-premises/developer/define-alternate-keys-entity) are custom unique identifiers that can be derived from one or more columns in your Dataverse table.
it's a flexible way to set up a unique identifier in your Dataverse table to be use for record matching between your query results in Hightouch and your rows in Dataverse.
In most cases, you won't have access to Dataverse GUID in your external data warehouse for record matching so alternate keys are convenient solution.
#### Steps to setup and use an alternate key
1. Sign in to your [Microsoft Power Apps](https://powerapps.microsoft.com/).
2. In the Power Apps, navigate to tables in Dataverse.  Select the table you want to sync to. In this walk through, we will use the `Account` table.
Please make sure that you are always working on the correct environment in
Power Apps. If you haven't set up your environment yet, go to this
[link](https://docs.microsoft.com/en-us/power-platform/admin/create-environment)
for more information. Here is an
[article](https://powerapps.microsoft.com/en-us/blog/establishing-an-environment-strategy-for-microsoft-power-platform)
from Microsoft on environment strategy.
3. Once you are in the table view, select `keys` to open the alternate keys view. 
4. In the keys page:
(1) Click `New Key` then in the pop-up on the right, enter the configuration for the key you are about to create.
(2) Enter a label for this key. This label is the label that will be displayed in Hightouch's ID mapping section.
(3) Select a column that you want to use as an alternate key.
Note that alternate keys must be unique and must meet the requirements described in the [Dynamics 365 docs](https://docs.microsoft.com/en-us/dynamics365/customerengagement/on-premises/developer/define-alternate-keys-entity?view=op-9-1). If you do not have a column that meets these requirements, you can manually add a column and use it. In our example below, We're using a column that we created called `Custom External ID`.
{" "}
{" "}
If you are creating a new column to be used as external ID, you may have to
backfill the new column first before they can be used to avoid accidental duplicates.
(4) Once configured, hit save button to create the alternate key.
5. The created alternate keys will appear as an option in the identifier section of your Hightouch sync configuration. Select the name of your alternated key then map the columns for that alternate key in the **ID fields mappings** section below.
If your alternate key is composed of multiple columns, you will have to map those columns in the **ID fields mappings** section.
## Sync configuration
Once you've set up your Microsoft Dynamics 365 destination and have a [model](/getting-started/concepts#models) to pull data, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Microsoft Dynamics 365 destination you want to sync.
### Syncing objects
Sync data from any source to Microsoft Dynamics 365 objects.
#### Record matching
To match rows from your model to objects in Microsoft Dynamics 365, you need to select a model column and corresponding Microsoft Dynamics 365 field. You can match on any of the following Microsoft Dynamics 365 fields:
- [**GUID**](microsoft-dynamics#what-is-a-guid)
- [**Alternate Keys**](microsoft-dynamics#what-are-alternate-keys)
In **Insert** mode, Microsoft Dynamics 365 automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync object fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default object fields.
**Field types**
Ensure your model columns data types match the data types of the fields you want to sync to. Microsoft Dynamics 365 destination will not automatically cast your data to the data types they are expected to be in your table in Dataverse. This means that you are responsible for casting them to the valid type they need to be in. The image below shows an example of fields we pulled from Dataverse along with their types. The icons on the left indicate the type of the column. As you may notice, there are icons with `?`. This indicates that the type is either complex or unexplored by Hightouch. Reference the [Dynamics 365 docs](https://docs.microsoft.com/en-us/dynamics365/customerengagement/on-premises/customize/types-of-fields?view=op-9-1) for more details on types.
**Required fields**
You may have noticed that `First Name` and `Last Name` appear to be read-only in the destination side while `Description` is editable.
These read-only fields are fields that are marked as `Business required` in Dataverse.
This means that whenever you create a sync, you must send data for these fields.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|---------------------------------------------------------------------------
**Do nothing** | Keep the object in Microsoft Dynamics 365 with all its synced fields
**Clear** | Clear all the mapped fields, but keep the object in Microsoft Dynamics 365
**Delete** | Delete the synced objects from Microsoft Dynamics 365
## Best practices
#### Validate your configuration
To validate that your configuration is working, we highly recommend that your to run a test.
Click on `Test` button to open the test pop-up.
Click on the `Sync as added row` to make the request.
The result on the right will indicate if the request to Dataverse is successful.
Be aware that requests made through here will persist data into your
destination as if you ran an actual sync. When working on a production
environment, please make sure that you fully understand the risks.
## Tips and troubleshooting
### Need admin approval
If you see a **Need admin approval** page appear when [accessing Microsoft's OAuth login flow](#setup), your Azure Active Directory user account might not have the necessary permissions to connect applications to your organization.
Ask one of your Azure organization's admins to edit your user account permissions.
You can read more about [Azure user permissions](https://learn.microsoft.com/en-us/azure/active-directory/develop/active-directory-how-applications-are-added#who-has-permission-to-add-applications-to-my-azure-ad-instance) and [configuring user settings](https://learn.microsoft.com/en-us/azure/active-directory/manage-apps/configure-user-consent?pivots=portal#configure-user-consent-settings) in Azure's documentation.
### Common errors
### Live debugger
### Sync alerts
---
## Microsoft Excel
**URL:** https://hightouch.com/docs/destinations/microsoft-excel
**Description:** Sync warehouse data to Excel worksheets used by sales, ops, and marketing teams
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
-------------|-----------------------------------------------------------------------------|---------------------
**Mirror** | Sync the full result set of the model to the selected destination worksheet | All
**Snapshot** | Create a new worksheet on every sync with the full result set of the model | All
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Microsoft Excel
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination**
button. Select **Microsoft Excel** and click **Continue**. You can then authenticate Hightouch to Microsoft Excel by logging into your Microsoft Excel account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Microsoft Excel destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and
the Microsoft Excel destination you want to sync to.
### Sync mode
Your first configuration is the [sync mode](/syncs/types-and-modes#sync-modes). You can select either:
- **Mirror**: if you want the full result set to overwrite a particular sheet's contents
- **Snapshot**: if you want to create a new sheet on every sync; the created worksheet's name is composed of the model name and sync run timestamp, for example, `Model Name: YYYY-MM-DD T hh mm ss`.
If you select **Mirror** mode, please ensure that your sheet doesn't contain data you don't want be overwritten.
### Drive and workbook selection
You can select any drives the user you authenticated with has access to. When selecting a workbook, you can either select the workbook from a dropdown or choose to enter a workbook ID. Once you've selected the workbook, its worksheets populate the next dropdown.
### Field mapping
You can choose to **sync all columns** or select only certain model columns to sync to your Excel worksheet. When syncing all columns, the column names in Excel mirror the column names in your source. When selecting specific columns, you can enter new columns names. If you'd like to rename columns, even if you want to sync all columns, you need to map each one individually.
### Custom cell range
By default, Hightouch inserts data in a worksheet starting at cell A1. The inserted data clears and updates the entire sheet. You can optionally select a range of cells to insert the data to. This capablity lets your preserve formulas or notes in your spreadsheet's margins.
When entering a range into the **Custom Cell Range** use the format `A1:Z1000`, where `A1` is the top left cell to insert data into and `Z1000` is the bottom right cell.
Ensure you the custom cell range is large enough to contain the data you want to sync. If it isn't, the sync will fail with a [**Query results are too large for provided range**]() error.
## Tips and troubleshooting
### Need admin approval
If you see a **Need admin approval** page appear when [accessing Microsoft's OAuth login flow](#connect-to-microsoft-excel), your Azure Active Directory user account might not have the necessary permissions to connect applications to your organization.
Ask one of your Azure organization's admins to edit your user account permissions.
You can read more about [Azure user permissions](https://learn.microsoft.com/en-us/azure/active-directory/develop/active-directory-how-applications-are-added#who-has-permission-to-add-applications-to-my-azure-ad-instance) and [configuring user settings](https://learn.microsoft.com/en-us/azure/active-directory/manage-apps/configure-user-consent?pivots=portal#configure-user-consent-settings) in Azure's documentation.
### Common errors
#### My data isn't appearing in Excel
Make sure to refresh your Excel worksheet after your sync has completed to see the updated values.
#### Query results are too large for provided range
This error occurs if you've set a [custom cell range](#custom-cell-range) that isn't large enough for the dataset you're syncing. The error specifies if you need to update your row or column range.
### Live debugger
### Sync alerts
---
## Microsoft Teams
**URL:** https://hightouch.com/docs/destinations/microsoft-teams
**Description:** Teams is a business communication platform where people can work together more effectively, connect all their software tools and services, and find the information they need to do their best work — all within a secure, enterprise-grade environment.
**Section:** Destinations
## Overview
With the Microsoft Teams destination, Hightouch can serve as a notifier for various changes. This is a very flexible integration, but here are a few example use cases:
1. Notify your customer success teams when product usage for a key account suddenly drops
2. Notify an account executive when product usage surpasses a certain threshold
3. Send a daily summary of the number of sign ups, compared to the same day last month
4. Send a daily summary on the adoption of a core feature
## Supported syncing
| Sync type | Description |
| :---------------- | :--------------------------------------------------------------------------------------------------------------------------------------------- |
| **Message** | Sends individual messages for each row that was added, changed, or removed since the last sync. There will be no messages for the initial run. |
| **Batch Message** | Sends a formatted message (or multiple messages) of all the rows in the query results. |
| **Table** | Sends a formatted table of all rows in the query results as a single message. |
## Sending Teams messages \(added / changed / removed\)
Hightouch will not send any messages on the initial run to prevent a flood of
messages to your channel. Messages will start after the this initial sync.
Hightouch supports sending messages based on rows that have been added, changed, or removed since the last sync run. Each row is sent as an individual message and can be sent to different channels and templated with Liquid.
Message formatting is best when the following conditions are met:
1. You want to write a query, and have Hightouch detect the added and removed rows
2. You want a custom message sent per each added and removed rows
### Channel selection
Hightouch supports both sending messages to a specific channel that the Hightouch bot is part of, or by pulling the channel ID each row. Enable the "Use column" toggle to dynamically set the target channel.
### Added, changed, and removed messages
Hightouch allows you to template the content you would like to send in your message.
- When a row is added in your query results, Hightouch will send the row added content.
- When a row is changed in your query results, Hightouch will send the row changed content.
- When a row is deleted in your query results, Hightouch will send the row deleted content.
This content can be [templated](#liquid-templating).
## Sending Teams messages \(all results\)
The batch message mode is similar to the normal message mode, except it sends all rows in the query results rather than only added, changed, and removed rows since the last sync.
This mode also allows you to specify how many rows are sent per message, which can range from 1 to 20. With 1 row per message, it would be similar to the message mode, whereas 20 rows per message would be similar to a tabular representation.
This format is good for the following scenarios.
1. You want to write a query, and have Hightouch push a message containing **all** rows, formatted as a message, to Microsoft Teams
2. You don't want Hightouch to detect added, changed, and removed rows, but instead want to send all rows on each query run
### Channel selection
The batch message only supports sending your messages to a single channel.
### Subject and body content
For each message, Hightouch allows setting the subject and row content. Only the row content is required. The row content is repeated per row in the message and has access to the row values via templating.
This content can be [templated](#liquid-templating).
## Templating messages
### Liquid templating
For both the message and batch message modes, [Liquid templating ](https://shopify.github.io/liquid/)is supported. The most important implication of this is that columns from the added or removed row can be accessed as part of the message body. Here is an example message making use of Liquid to insert a column into the message.
```text
User with name {{ row["name"] }} and email {{ row["email"] }} has signed up
```
## Table
Table formatting sends all of your query results formatted as a text table as messages to Microsoft Teams. This mode is recommended only for small sets of data.
This format is good for the following scenarios.
1. You want to write a query, and have Hightouch push a message containing **all** rows, formatted as a table, to Microsoft Teams
2. You don't want Hightouch to detect added and removed rows, but instead want to send all rows on each query run
### Channel selection
The batch message only supports sending your messages to a single channel.
## Tips and troubleshooting
### Need admin approval
If you see a **Need admin approval** page appear when accessing Microsoft's OAuth login flow, your Azure Active Directory user account might not have the necessary permissions to connect applications to your organization.
Ask one of your Azure organization's admins to edit your user account permissions.
You can read more about [Azure user permissions](https://learn.microsoft.com/en-us/azure/active-directory/develop/active-directory-how-applications-are-added#who-has-permission-to-add-applications-to-my-azure-ad-instance) and [configuring user settings](https://learn.microsoft.com/en-us/azure/active-directory/manage-apps/configure-user-consent?pivots=portal#configure-user-consent-settings) in Azure's documentation.
### Common errors
### Live debugger
### Sync alerts
---
## MinIO
**URL:** https://hightouch.com/docs/destinations/minio
**Description:** Transfer data at scale from your warehouse to MinIO
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
| ------------ | -------------------------------------------------------------| -------------------- |
| Any data set | Sync data from a source to MinIO as CSV, JSON, or Parquet files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Prerequisites
To get started, you need:
- a [MinIO bucket](https://charts.min.io/)
- MinIO access key and secret key for use by Hightouch
## Connect to MinIO
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **MinIO** and click **Continue**. You can then authenticate Hightouch to MinIO by entering your **Bucket Name**. The Bucket Name should just be the name of the bucket, not a URL.
## Sync configuration
Once you've set up your MinIO destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the MinIO destination you want to sync to.
### Select file format
Hightouch supports syncing JSON, CSV, and Parquet files to Amazon MinIO.
### Enter filename
The filename or **object key** field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file.
Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it.
To keep different versions of the same results file, you can enable [versioning](https://docs.aws.amazon.com/AmazonMinIO/latest/userguide/Versioning.html) in your bucket, or your app can copy the data to another location.
If you are using an audience and would like to include the audience name, you will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter `-86400` (`24 hours * 60 minutes * 60 seconds`). If you want the filename's data to be one hour after the sync takes place, you would enter `3600` (`60 minutes * 60 seconds`).
### CSV options
If you're syncing to a CSV file, you have additional configuration options:
- **Delimiter**: Your options are comma (`,`), semicolon (`;`), pipe (`|`), tilde (`~`), and tab
- (Optional) Whether to **include a CSV header row** in the exported files
- (Optional) Whether to **include a [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark)** in the exported files, the BOM is ``
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `customer_id`, `email`, `first_name` and `last_name`, columns. These columns are mapped to new fields in the destination file as `id`, `email`, and `name`—a [templated](/syncs/mapping-data#template-mapping) concatenation of `first_name` and `last_name`—respectively. Hightouch exports these fields to new fields in the file and ignores all other columns from your results.
### Batch size
By default, and depending on your [sync mode](#supported-syncing), Hightouch sends one file to your MinIO bucket for each export. (In diff mode, Hightouch sends three files.)
By enabling batching, you allow Hightouch to send multiple files if your model query results are large. This can help avoid file size errors from MinIO.
### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
### Gzip compression
You can enable gzip compression to reduce the size of your exported files. Note
that Hightouch does not automatically add the `.gz` extension to your files. You
can add this or any other extension to your filename.
## Tips and troubleshooting
### Common errors
#### Access denied
The **ACCESS_DENIED** error is a permissions-related error. Ensure the crendtials in Hightouch match the credentials used during MinIO setup.
### Live debugger
### Sync alerts
---
## Mixpanel
**URL:** https://hightouch.com/docs/destinations/mixpanel
**Description:** Perform deeper analysis and better understand user behavior by bringing enriched customer data from your data warehouse into Mixpanel
**Section:** Destinations
## Overview
Ideally, your data warehouse is the single source of truth for all your customer data.
When that's the case, using your warehouse as the source for your Mixpanel data enables richer product-usage analysis.
You can sync enriched data from your warehouse to create more granular segments on Mixpanel as well as bring in data from your engagement tools.
Moreover, you can ensure that your product and growth teams work with data that's always up-to-date and in the expected format.
Lastly, if your company also uses a Business Intelligence tool such as Looker or Mode, you can ensure that the data is consistent across both systems.
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|**Objects**|Sync records to objects such as users or organizations (groups) in your destination.|Upsert|
|**Events**|Sync records as events to your destination. This is often in the form of a track call.|Insert|
|**Lookup tables**|Sync records as a CSV file to enrich event and profile properties within Mixpanel.|All|
- In **Upsert** mode, new users and groups are inserted into Mixpanel and all attributes are kept up-to-date.
- In **Insert** mode, new events are inserted as track calls.
- In **All** sync mode, the entire lookup table is replaced with each sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Getting started
To configure your Mixpanel destination you need to provide the following information:
- Project ID
- Project Token
- Service Account Username and Secret
- Group Keys, when syncing group objects
### Project ID
To find your Project ID, click the Settings icon, then Project Settings.
Then navigate to Overview > Project Details.
### Project token
To find your Project Token, click the Settings icon, then Project Settings.
Then navigate to Overview > Access Keys.
### Service account
To receive your Service Account Username and Secret you must first create a Service Account.\
Click the Settings icon, then Project Settings.
Then navigate to Service Accounts > Add Service Account and copy the **Username** and **Secret**.
Be sure to give your Service Account **Owner** or **Admin** permissions.
### Group keys
If you are syncing group profiles or assigning users to groups, you need to indicate which Group Key to use.
To find your Group Keys, click the Settings icon, then Project Settings.
Then navigate to Overview > Group Keys. Copy the appropriate **Key** value.
## Syncing data
### Objects
You can decide to sync either User Objects or Group Objects.
When syncing Group Objects, enter the [appropriate group key](#group-keys) into the **Mixpanel group key** field.
#### Record matching
You can match records from your source to your Mixpanel workspace by Mixpanel User ID, email or Group ID.
#### Field mapping
You can sync columns from your source to your default and custom fields in Mixpanel.
If you're syncing User Objects, you can see two sections: one for native Mixpanel fields and one for custom fields.
If you're syncing Group Objects, you can only see one section.
#### Users location
By default, users uploaded using Hightouch won't have any associated location in Mixpanel.
To set a location, provide an IP address \(field `$ip`\), or longitude and latitude coordinates \(fields `$longitude` and `$latitude`\).
For more information, you can visit [Mixpanel's official documentation](https://help.mixpanel.com/hc/en-us/articles/115004499343-Tracking-Geolocation-with-Server-Side-Implementation).
#### Assigning users to groups
To assign users to groups, create a sync for User Objects.
Then, create a column in your model which contains the group IDs that you want to assign the user to.
For example, to sync organizations, use a column called `organizations` containing either a group ID \(`google`\) or an array of group IDs \(`[google, apple, microsoft]`\).
Then, [find your group key](#group-keys), and use it as the destination field to sync to.
### Events
Hightouch supports sending events of a given name, for example, `Signed Up`.
Hightouch lets you choose the column containing the event timestamp.
Hightouch accepts various date time formats for the timestamp and automatically converts them to the format required by Mixpanel.
#### Record matching
You can match records from your source to your Mixpanel workspace by Mixpanel User ID.
#### Field mapping
Hightouch allows you to pass data to the event properties of a Mixpanel event.
#### Assigning events to groups
To assign events to groups, create a sync to events.
Then, create a column in your model which contains the group IDs that you want to assign the event to.
\
For example, to sync organizations, use a column called `organizations` containing either a group ID \(`google`\) or an array of group IDs \(`[google, apple, microsoft]`\).
Then, [find your group key](#group-keys), and use it as the destination field to sync to.
### Lookup tables
To sync data to your Lookup tables, choose **Enrichment**.
Then choose the specific Lookup table you want to replace.
#### Record matching
You can match records from your source to your Mixpanel workspace by the ID mapping of your choice.
The ID mapping should reference the unique identifier field of the events or profiles you are seeking to enrich.
#### Field mapping
You may also sync columns from your source to your Mixpanel Lookup table's custom fields.
#### Creating lookup tables in Mixpanel
To create Lookup tables, you can import them directly in Mixpanel.
You can do this by clicking on the Data Management icon and then on Lexicon.
Navigate to Lookup Tables > Import > Lookup Table.
#### Limitations
Mixpanel's Lookup table endpoint comes with the following limits:
- Only 100 requests are allowed in a rolling 24 hour period. Please be mindful of this when choosing your sync schedule. Mixpanel recommends updating Lookup tables at most hourly.
- A Lookup table must not exceed 100 MB uncompressed (roughly 1-2M rows).
## Tips and troubleshooting
### Common errors
#### Changing projects
If you need to change your Project ID, you also need to update the API Key that you originally provided.
Project ID and API keys are paired together in the Mixpanel platform.
### Live debugger
### Sync alerts
---
## MoEngage
**URL:** https://hightouch.com/docs/destinations/moengage
**Description:** Build more robust campaigns on MoEngage with up-to-date customer data from your data warehouse
**Section:** Destinations
## Overview
Providing relevant and timely campaigns relies on fresh and accurate data in your MoEngage account. By automatically syncing customer data from your data warehouse into MoEngage, you don't need to worry about data consistency or staleness. Instead, you can focus on building world-class customer experiences.
## Supported syncing
| Type | Description | Supported Sync Modes |
| ------------ | ------------------------------------------- | -------------------- |
| **Users** | Sync data from any source to MoEngage users | Upsert |
| **Events** | Sync events to MoEngage | Insert |
| **Segments** | Add or remove users from MoEngage cohorts | Add, Remove |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to MoEngage
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **MoEngage** and click **Continue**. You can then authenticate Hightouch to **MoEngage**.
You need to provide the following information:
- **API ID**: You can find this in your MoEngage dashboard under **Settings** > **APIs** > **General**.
- **API Secret**: You can find this in your MoEngage dashboard under **Settings** > **APIs** > **General**.
- **Business Events API Key**: This is required for syncing business events. You can find this in your MoEngage dashboard under **Settings** > **APIs** > **General**.
- **Region**: Select the region your MoEngage account is hosted in.
## Sync configuration
Once you've set up your **MoEngage** destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the MoEngage destination you want to sync to.
### Syncing users
This sync type lets you sync data to MoEngage user profiles.
#### Record matching
You can match rows from your model to users in MoEngage on their **User ID**.
#### Field mapping
Hightouch lets you sync user fields via [field mapping](/syncs/mapping-data). You can map data from any of your model columns to standard or custom user attributes in MoEngage. MoEngage expects custom properties to follow a `snake_case` format.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ---------------------------------------------------------- |
| **Do nothing** | Keep the user in MoEngage with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the user in MoEngage |
### Syncing events
This sync type lets you send events to MoEngage. Hightouch treats any records added to your model as new events to be sent to MoEngage.
When creating a sync for events, you can specify the name of the event being tracked. You can either input a static name, or use a column from your model. You can also select a column that is used as the timestamp of when the event happened. If this field is empty, Hightouch will use the time of when the event arrives at the server.
#### Record matching
You can match rows from your model to users in MoEngage on their **User ID**.
#### Field mapping
You can map data from your model columns to event properties. Any mappings from this section will be synced as attributes in the event. MoEngage expects custom properties to follow a `snake_case` format.
### Syncing segments
This sync type lets you add or remove users from MoEngage cohorts. You can either sync to an existing cohort or create a new one.
Users must already exist in MoEngage before they can be added to a cohort.
#### Record matching
You can match rows from your model to users in MoEngage on their **User ID**.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| ---------------------- | --------------------------------------------------------------- |
| **Do nothing** | Keep the user in the MoEngage cohort with all its synced fields |
| **Remove from cohort** | Remove the user from the MoEngage cohort |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Moloco
**URL:** https://hightouch.com/docs/destinations/moloco
**Description:** Moloco is the AI-powered growth engine that unlocks the value of a business's first-party data.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
------------------|---------------------------------------------------|----------------------|--------------------------------------------------------------------------
**Customer Sets** | Sync data from any source to Moloco customer sets | Add | [Customer Sets docs](https://developer.moloco.cloud/docs/getting-started)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Moloco
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Moloco** and click **Continue**. You can then authenticate Hightouch to **Moloco**.
Enter the following fields into Hightouch:
- **Ad Account ID**
- **API key**
## Sync configuration
Once you've set up your Moloco destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Moloco destination you want to sync to.
### Syncing customer sets
Sync data from any source to Moloco customer sets.
#### Creating a new customer set
You can create a new customer set via Hightouch by selecting `Create a new customer set`. If selected, Hightouch will generate a new customer set. The customer set will not be ready for 30 minutes, so the initial sync will wait 30 minutes for the customer set to be ready, and then sync your data over.
#### Record matching
To match rows from your model to customer sets in Moloco, you need to select a model column and corresponding Moloco field. You can match on any of the following Moloco fields:
- **Apple IDFA**
- **Google ADID**
#### Field mapping
Hightouch lets you sync customer set fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default customer set fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Monday
**URL:** https://hightouch.com/docs/destinations/monday
**Description:** Keep your team's projects and tickets up-to-date with updated items for your Monday boards
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
| ---------- | -----------------------------------------------------| ---------------------- | ------------------------------------------------------------------------------------------------------- |
| **Items** | Sync data from any source to a Monday board as items | Update, Insert | [Items](https://developer.monday.com/api-reference/docs/items_page)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Monday
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Monday** and click **Continue**. Authenticate Hightouch to Monday by entering your API token from your Monday account.
If you don't have an API token, follow the [instructions on **Monday** to access API tokens](https://developer.monday.com/api-reference/docs/authentication#accessing-api-tokens) depending on your role. Ensure your API token has write access to the board you are syncing to.
## Sync configuration
Once you've set up your Monday destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Monday destination you want to sync to.
Select which board you'd like to sync to. Each sync sends data to one board.
### Sync mode
Hightouch supports **Update** and **Insert** mode by using Monday's GraphQL API with version 2023-10.
**Update** mode uses the [`change_multiple_column_values`](https://developer.monday.com/api-reference/docs/columns#change-multiple-column-values) mutation.
**Insert** mode uses the [`create_item`](https://developer.monday.com/api-reference/docs/items#create-an-item) mutation.
### Field mapping
Hightouch supports the following column types:
* Checkbox
* Country
* Date
* Dependency
* Email
* Hour
* Link
* Location
* Name
* Numbers
* People
* Phone
* Rating
* Status
* Tags
* Text
* Timeline
* Week
* World Clock
You may use Hightouch's object inline mapping to construct Monday's object values from your source. Below is an example sync configuration:
Monday uses **YYYY-MM-DD** format for dates. For the most success in your sync, ensure that you use that format for your dates. You can also use Hightouch's template option to format your data.
### Record matching
In **Update mode**, Hightouch requires a unique identifier in your model with item IDs from Monday. Rows without item IDs will fail to sync.
In **Insert mode**, the source must contain a column that maps to the item name. Monday rejects any rows without an item name.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## MongoDB
**URL:** https://hightouch.com/docs/destinations/mongodb
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Overview
The MongoDB destination combines the analytical power of your data warehouse with the low-latency performance of a transactional database. It's been battle-tested with syncs up to hundreds of millions of rows.
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|Documents|Sync data to documents in a MongoDB collection |Upsert|
## Connect to MongoDB
Make sure to allowlist Hightouch IP addresses so Hightouch can reach your cluster. Refer to our
[IP address docs](/security/networking#ip-addresses) to find the relevant IP addresses for your Hightouch region. If you are using Atlas, you can add allowlist IP addresses from **Security > Network Access**.
When creating a MongoDB destination, you can either enter the **Host**, **Port**, **Database**, **User**, and **Password**, or you can provide the full URI connection string. For more information, go to the [official connection guide](https://docs.mongodb.com/drivers/node/current/fundamentals/connection/).
### SSH tunneling
Hightouch can connect directly to MongoDB over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your MongoDB instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
### Retrieve MongoDB credentials
Assign the user whose credentials you use the built-in [`readWriteAnyDatabase`](https://www.mongodb.com/docs/atlas/security-add-mongodb-users/#mongodb-atlasrole-Read-and-write-to-any-database) MongoDB role or create a [custom role](https://www.mongodb.com/docs/atlas/security-add-mongodb-roles/) and assign these actions at minimum:
| Action | Details |
| :----------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------- |
| [`insert`](https://www.mongodb.com/docs/manual/reference/privilege-actions/#mongodb-authaction-insert) | Grants permission to perform `insert` and `create` commands|
| [`remove`](https://www.mongodb.com/docs/manual/reference/privilege-actions/#mongodb-authaction-remove) | Grants permission to perform the `delete` command |
| [`update`](https://www.mongodb.com/docs/manual/reference/privilege-actions/#mongodb-authaction-update) | Grants permission to perform the `update` command |
| [`listIndexes`](https://www.mongodb.com/docs/manual/reference/privilege-actions/#mongodb-authaction-listIndexes) | Grants permission to perform the `listIndexes` command |
| [`listCollections`](https://www.mongodb.com/docs/manual/reference/privilege-actions/#mongodb-authaction-listCollections) | Grants permission to perform the `listCollections` command |
To create a user in MongoDB, use [`db.createUser()`](https://www.mongodb.com/docs/manual/reference/method/db.createUser/).
#### Authenticate with connection parameters
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your MongoDB server. For help finding this, go to the [official connection guide](https://www.mongodb.com/docs/drivers/node/current/fundamentals/connection/connect/#std-label-node-connect-to-mongodb).
- **Port**: The port number of your MongoDB server. The default port number is 27017, but yours may be different. You don't need to specify the port if you are using SRV protocol.
- **Database**: The name of the your MongoDB database.
- **Username**: This can be your personal MongoDB login or a dedicated user for Hightouch.
- **Password**: This is the password for the user specified above. If you're using [Atlas](https://www.mongodb.com/atlas/database), the user password might differ from the one you use to login to your MongoDB Atlas instance.
{/* */}
- **Is the host name a seedlist record?**: To use [SRV](https://www.mongodb.com/docs/manual/reference/connection-string/#std-label-connections-dns-seedlist) protocol (Hightouch defaults to [standard](https://www.mongodb.com/docs/manual/reference/connection-string/#std-label-connections-standard-connection-string-format)), you must toggle this setting. If you are using MongoDB Atlas and the connection string begins with protocol `mongodb+srv`, toggle this setting to connect.
{/* */}
#### Authenticate with connection string
The connection string is available in multiple locations. Follow these instructions for one way to find it.
1. From the [MongoDB UI](https://cloud.mongodb.com/v2/) click the **Connect** button next to the name of your cluster:
2. From the options displayed, select **Connect your application**:
Follow the instructions for changing the password and database name in the string. Copy the string by clicking the copy icon:
In Hightouch, in the MongoDB setup configuration, paste your saved connection string into the provided field.
## Sync configuration
### Record matching
Hightouch requires choosing one column to define how to match data in your source to documents in MongoDB. It's usually best to use the [`_id` field](https://www.mongodb.com/docs/manual/core/document/#the-_id-field) in MongoDB. Otherwise, you can also select `_id (Object ID)` if you already have [Object ID](https://www.mongodb.com/docs/manual/reference/bson-types/#objectid) values in your source data.
You can also choose to record match on a custom MongoDB field. To do this, you need to create a [unique index on a single field](https://www.mongodb.com/docs/manual/core/index-unique/#unique-index-on-a-single-field), which is a specific type of [single field index](https://www.mongodb.com/docs/upcoming/core/indexes/index-types/index-single/#single-field-indexes). Before creating this index, make sure that the field doesn't contain duplicate values, as this makes the index build fail with an `Unknown error` message.
If you record match on a custom field, MongoDB automatically generates an Object ID to insert in the `_id` field.
### Field mapping
You can map the columns you want to include when creating new MongoDB documents or updating existing ones by selecting the model column and typing the respective MongoDB field name. Otherwise, you can include every column from your model by checking the **Sync all columns** box.
### Delete behavior
You can choose how to handle documents in MongoDB when the corresponding rows are deleted in your source.
| Behavior | Description |
|-------------------------------|-----------------------------------------------|
| **Do nothing** | Keep the document in MongoDB |
| **Clear Fields** | Clear the mapped fields but keep the document |
| **Delete Destination Record** | Delete the MongoDB document completely |
## Tips and troubleshooting
### Best practices
Create a user in MongoDB specifically for Hightouch and only allow that user to access the collections you want Hightouch to access. Don't use the 'root' user.
### Compatible versions
The minimum supported MongoDB version is `4.0`. Earlier versions aren't officially supported but may work regardless.
### Common errors
{/* */}
#### MongoServerError: bad auth: Authentication failed
{/* */}
This error can occur if the destination wasn't [configured correctly](#authenticate-with-connection-parameters). Make sure to use the correct password for the user specified in the configuration. If the error is still happening, try changing your MongoDB user's password and attempting the setup again.
### Live debugger
### Sync alerts
---
## MovableInk
**URL:** https://hightouch.com/docs/destinations/movable-ink
**Description:** The Magic Behind Your Marketing Movable Ink activates data into personalized content in any customer touchpoint.
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| ------------------------- | ---------------------------------------------------------- | -------------------- | ----------------------------------------------------- |
| [**Event**](#sync-events) | Send behavioral events to Movable Ink's Customer Data API. | Insert | [Customer Data API](https://learning.movableink.com/) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Movable Ink
To connect Movable Ink to Hightouch you need an endpoint URL, an access key, and an access secret. Movable Ink's Client Experience team provisions all three. Hightouch authenticates each request with HTTP Basic auth.
1. Go to the [Destinations overview](https://app.hightouch.com/destinations) page and select **Add destination**.
2. Select **Movable Ink** from the destination catalog.
3. Enter the following credentials, then select **Continue**:
- **Endpoint URL**: the Customer Data API endpoint provided by Movable Ink. The shape is `https://collector.micpn.com/behavioral/` for US accounts or `https://collector.micpn-eu.com/behavioral/` for EU accounts, where `` is your account-specific path segment.
- **Access key**: the username portion of HTTP Basic auth.
- **Access secret**: the password portion of HTTP Basic auth.
Contact your Movable Ink Client Experience team if you don't have an endpoint
URL or credentials yet. Movable Ink's developer reference
(`learning.movableink.com`) requires a customer login.
## Sync configuration
### Sync events
Send behavioral events to Movable Ink. Hightouch inserts one event per row in the model's query results each time the sync runs.
#### Event type
Select the Movable Ink event type for this sync:
| Event type | Description |
| -------------------------------------- | -------------------------------------------------------------------------------- |
| **Page View (Product Viewed)** | A user viewed a product page. |
| **Add to Cart (Product Added)** | A user added a product to their cart. |
| **Remove from Cart (Product Removed)** | A user removed a product from their cart. |
| **Search Completed** | A user submitted a search query. |
| **Category Viewed** | A user viewed a category page. |
| **Order Completed (Conversion)** | A user completed a purchase. |
| **Identify** | An identity event, used to associate a user with timezone or other profile data. |
| **Custom** | A custom event. You provide the `event_type` name and any additional fields. |
#### Identifiers
Map at least one identifier per event:
| Destination field | Notes |
| ----------------- | --------------------------------------------------------------- |
| **User ID** | Must match the `mi_u` value in your Movable Ink Studio account. |
| **Anonymous ID** | Used when no user ID is available. |
Rows with neither a User ID nor an Anonymous ID are rejected without aborting the sync. You can review rejected rows in the [Live debugger](#live-debugger).
#### Event timestamp
Map a model column to the event timestamp. If left unmapped, Hightouch defaults to the time the sync sent the event. Hightouch validates that the timestamp is no more than 30 days in the past and no more than 1 day in the future. Events outside that window are rejected.
#### Field mapping
The mappable fields depend on the event type:
| Event type | Required fields | Optional fields |
| ---------------------------------------------- | ------------------------- | ----------------------------------------------------------- |
| **Page View / Add to Cart / Remove from Cart** | Product ID | Title, Price, URL, Image URL, Categories, Channel, Timezone |
| **Search Completed** | Query | URL, Channel, Timezone |
| **Category Viewed** | Category ID | Category URL, Channel, Timezone |
| **Order Completed** | Products | Order ID, Revenue, Discount, Coupon Code, Channel, Timezone |
| **Identify** | — | Timezone |
| **Custom** | Event name (`event_type`) | Any additional fields you map |
For **Order Completed**, the **Products** field expects an array of product objects. Use Hightouch's [inline mapper](/syncs/mapping-data#inline-mapping) or an [array column](/syncs/mapping-data#create-an-array) from your model.
For **Custom** events, Movable Ink supports up to 10 unique custom event names per account by default. Contact your Movable Ink rep if you need more.
#### Payload limits
Movable Ink rejects events that exceed its payload limits. Hightouch checks these before sending and rejects offending rows:
- Any string field longer than 256 characters.
- A total event payload larger than 32 KB.
Rejected rows are surfaced in the [Live debugger](#live-debugger).
## Tips and troubleshooting
### Validate your setup
After your first sync completes, confirm the events landed in Movable Ink:
- Ask your Movable Ink Client Experience team to confirm events are arriving on the configured endpoint. Movable Ink's developer site (`learning.movableink.com`) is customer-login-gated, so direct verification from Hightouch's side ends at a successful HTTP 2xx response from the Customer Data API.
- Check the [Live debugger](#live-debugger) for rejected rows. The most common rejections are missing identifiers, missing event names on custom events, out-of-window timestamps, oversized string fields, and oversized payloads.
### Common errors
### Live debugger
### Sync alerts
---
## mParticle
**URL:** https://hightouch.com/docs/destinations/mparticle
**Description:** Sync customer attributes and events from your data warehouse to hundreds on integrations through mParticle
**Section:** Destinations
## Overview
Through mParticle's diverse integration library, Hightouch users can sync enriched customer profiles and events to hundreds of partner integrations. Use your data warehouse as the source of truth and keep your team's tools up to date with the latest data using Hightouch and mParticle.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ----------- | -------------------------------------------------------------- | -------------------- |
| **Users** | Sync data from any source as user attributes | Upsert |
| **Events** | Sync data from any source as commerce events and custom events | Insert |
## Getting started
To give Hightouch access to mParticle, you need to enter:
- API key
- API secret
Follow the instructions in the [following sections](#retrieve-api-key-and-api-secret) to retrieve your API key and API secret.
### Retrieve API key and API secret
1. Navigate to [Inputs](https://app.mparticle.com/setup/inputs/feeds) in your mParticle account.
2. Add a feed under the **Hightouch** tab by clicking the **+** button.
3. Give your configuration a name and select **Web** under **Act as Platform**.
4. Copy your generated keys. Your **Server to Server Key** is your **API key** and your **Server to Server Secret** is your **API secret**.
## Syncing data
### Users
Sync data from any source as user attributes without an event to mParticle.
#### Record matching
You can match rows in your model with users in mParticle with the provided identifiers. Hightouch uses the identifier you select in this section and as the `external Id` for this sync.
#### Additional identifiers
In addition to the column selected for [record matching](#record-matching), you can choose to provide more user identifiers in this section. There is no limit to the number of columns you can select here.
#### Field mapping
You can sync columns from your source to mParticle as user attributes. If the customer attribute doesn't exist, Hightouch creates it.
#### Custom field mapping
You can sync columns from your source to mParticle as custom user attributes. If the customer attribute doesn't exist, Hightouch creates it.
#### Environment
In this section, select the environment you would like to sync your data to. The available choices are:
| Environment | Description |
| --------------- | -------------------------------------------------------------------------------------------------------------- |
| **Production** | All data synced is treated as production data and is forwarded to integrations for your application |
| **Development** | All data synced is treated as development data and is siloed from your production data |
mParticle recommends using **Development** mode first to ensure the data meets your expectations before moving to **Production**.
#### Delete behavior
You have the following options for how Hightouch should handle rows leaving your query results:
| Behavior | Description |
| ---------------- | ---------------------------------------------------- |
| **Do nothing** | Keep the customer in mParticle |
| **Clear Fields** | Clear the mapped fields but keep the customer record |
### Events
You can sync the following event types:
- **Custom Event**
- **Commerce Event**
#### Events and historical events
Given a valid `Date`, `DateTime`, or `timestamp` column as a `timestamp` for events, Hightouch separates events based on the age of the data.
- Hightouch sends events that are older than **28** days to the `/v2/bulkevents/historical` endpoint.
- Hightouch sends events that are newer than **28** days to the `/v2/bulkevents` endpoint.
If you choose not to provide a timestamp for your events, Hightouch uses the time the event arrives at the server. That means all events are treated as newer than 28 days.
### Custom event
Sync data from any source as a custom event.
#### Event name
Provide an event name. This is how you can reference the event in mParticle campaigns or segments.
#### Event timestamp
Provide a timestamp of when the event occurred. If this field is empty, Hightouch uses the time the event arrives at the server.
Any `DateTime` value provided from your source is converted to Unix timestamp as expected by mParticle.
#### Custom event type
Select the type of custom event you would like to sync out of the provided selections.
#### User identifiers
In this section, you can provide user identifiers for each event that occurred.
#### Event attributes field mapping
You can sync columns from your source to mParticle as event attributes. If the event attribute doesn't exist, Hightouch creates it.
#### Event attributes custom field mapping
You can sync columns from your source to mParticle as custom event attributes. If the custom event attribute doesn't exist, Hightouch creates it.
#### User attributes field mapping
You can sync columns from your source to mParticle as user attributes. If the customer attribute doesn't exist, Hightouch creates it.
#### User attributes custom field mapping
You can sync columns from your source to mParticle as custom user attributes. If the customer attribute doesn't exist, Hightouch creates it.
#### Environment
In this section, select the environment you would like to sync your data to. The available choices are:
| Environment | Description |
| --------------- | -------------------------------------------------------------------------------------------------------------- |
| **Production** | All data synced is treated as production data and is forwarded to integrations for your application |
| **Development** | All data synced is treated as development data and is siloed from your production data |
mParticle recommends using **Development** mode first to ensure the data meets your expectations before moving to **Production**.
### Commerce event
Sync data from any source as a commerce event.
#### Event timestamp
Provide a timestamp of when the event occurred. If this field is empty, Hightouch uses the time the event arrives at the server.
Any `DateTime` value provided from your source is converted to Unix timestamp as expected by mParticle.
#### User identifiers
In this section, you can provide user identifiers for each event that occurred.
#### Event attributes field mapping
You can sync columns from your source to mParticle as event attributes. If the event attribute doesn't exist, Hightouch creates it.
#### Event attributes custom field mapping
You can sync columns from your source to mParticle as custom event attributes. If the custom event attribute doesn't exist, Hightouch creates it.
#### User attributes field mapping
You can sync columns from your source to mParticle as user attributes. If the customer attribute doesn't exist, Hightouch creates it.
#### User attributes custom field mapping
You can sync columns from your source to mParticle as custom user attributes. If the customer attribute doesn't exist, Hightouch creates it.
#### Environment
In this section, select the environment you would like to sync your data to. The available choices are:
| Environment | Description |
| --------------- | -------------------------------------------------------------------------------------------------------------- |
| **Production** | All data synced is treated as production data and is forwarded to integrations for your application |
| **Development** | All data synced is treated as development data and is siloed from your production data |
mParticle recommends using **Development** mode first to ensure the data meets your expectations before moving to **Production**.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## MySQL
**URL:** https://hightouch.com/docs/destinations/mysql
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Overview
This destination combines the analytical power of your data warehouse with the low-latency performance of a transactional database.
## Supported syncing
|Sync Type|Description |Supported Sync Modes |
|-----------|----------------------------------------------|---------------------|
| **Table** |Sync data from any source to a MySQL table | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to MySQL
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **MySQL** and click **Continue**.
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your MySQL.
- **Port**: The port number of your MySQL. The default is 3306, but yours may be different.
- **Database**: The name of the database in your MySQL.
- **User**: This can be your personal MySQL login or a dedicated user for Hightouch.
- **Password**: The password for the user specified above.
You can optionally create or select a [previously configured tunnel](/security/ssh-tunneling) or upload a `.pem` CA certificate.
### SSH tunneling
Hightouch can connect directly to MySQL over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your MySQL instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
## Sync configuration
Once you've set up your MySQL destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the MySQL destination you want to sync to.
### Select table
Select the existing table in MySQL you want to sync to.
### Record matching
You can match rows from your model to rows in MySQL on any column as long as it is a [primary key or unique key](https://dev.mysql.com/doc/refman/8.0/en/partitioning-limitations-partitioning-keys-unique-keys.html).
### Field mapping
Select which model columns you want to sync to your MySQL table columns. Hightouch automatically pulls the columns from your table to make them available for you to map.
When syncing to [binary](https://dev.mysql.com/doc/refman/8.0/en/binary-varbinary.html) destination fields, Hightouch assumes the model column values to be [base64](https://en.wikipedia.org/wiki/Base64) strings. If they aren't, you can use [template mapping](/syncs/mapping-data#template-mapping) to [cast](/models/data-types-casting#casting) them if needed.
### Upsert query optimization
You can choose to use the [`LOAD DATA` statement](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_local_infile) in the upsert query during your sync. This option loads data from a text file into your database, which can be up to twenty times faster than the default `INSERT INTO` query Hightouch uses.
If you enable the `LOAD DATA` option, Hightouch replaces databases rows with the mapped model row, if it exists. This means that if any MySQL fields aren't mapped in Hightouch they receive `NULL` values.
For Hightouch to successfully sync with the `LOAD DATA` option, you need to enable the [`local_infile`](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_local_infile) system variable.
Review the [security considerations](https://dev.mysql.com/doc/refman/8.0/en/load-data-local-security.html) before enabling `local_infile`.
### Delete behavior
When rows no longer shows up from your model's query results, you can configure the sync to **Do nothing** or **Delete** those rows from the MySQL.
| Behavior | Description |
| -------------- | ----------------------------------------------- |
| **Do nothing** | Keep the synced rows in the table in MySQL |
| **Delete** | Delete the synced rows from the table in MySQL|
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## NetSuite (SOAP)
**URL:** https://hightouch.com/docs/destinations/netsuite
**Description:** Sync business-critical data from your data warehouse to Netsuite with Hightouch
**Section:** Destinations
Check out our
[playbook](https://hightouch.com/playbooks/automating-usage-based-billing) and
[blog post](https://hightouch.com/blog/automated-billing) to learn how to
automate usage-based billing to streamline your finance operations.
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| --------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| [**Entities**](#syncing-entities) | Sync data from any source to entities in NetSuite, for example, Customers, Invoices, Sales Orders, and Journal Entries | Upsert, Update, Insert | [Browse entity schemas ↗](https://system.netsuite.com/help/helpcenter/en_US/srbrowser/Browser2022_2/schema/record/account.html) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## REST vs SOAP
The Hightouch NetSuite REST destination offers a few advantages over the SOAP destination:
- The NetSuite REST API uses JSON for web service requests and responses, which is easier to read and debug in the [live debugger](/syncs/debugger) compared to XML.
- Hightouch prioritizes supporting the latest REST API features and versions.
- The [NetSuite REST](/destinations/netsuite-rest#field-mapping) destination supports the [inline mapper](/syncs/mapping-data#inline-mapping) to build objects and arrays
Unless you have a particular entity that's only supported by the SOAP API, we recommend using the REST NetSuite destination.
## Setup in NetSuite
Before you can use Hightouch to sync data to NetSuite, you need to create and and configure a NetSuite integration. Then, to grant access to Hightouch to synchronize data between your source and NetSuite, you need to create a role, a user, and an authentication token for the integration.
Depending on your NetSuite account's setup, you may need an **admin** role to
complete setup in NetSuite. Netsuite permission changes to a role or user
don't occur immediately. These changes can take a few minutes or hours to go
into effect.
### Create an integration
1. Go to **Setup → Integration → Manage Integrations** or use the global search to find and go to the **Manage Integrations** page.
This page shows a list of enabled integrations for your account.
2. Click **New** to create a new integration with the following properties:
- A **Name** that can easily be associated with Hightouch, for example, "Hightouch Integration"
- Set **State** to **Enabled**
- Check **Token-Based Authentication**
- Uncheck **TBA:Authorization flow**
- Uncheck **Authorization Code Grant**
- Leave other authentication options unchanged.
3. Scroll to the bottom of the page and click **Save**.
4. Scroll to **Client credentials** and Copy and store the **Consumer Key / Client ID** and **Consumer Secret / Client secret** in a secure location. You'll need them to connect to Hightouch.
### Create a role
1. Go to **Setup → Users/Roles → Manage Roles** and click **New** or use the global search to find and go to the **New Role** page.
2. Create a role with the following properites:
- A descriptive **Name**, for example, "Hightouch Role."
- Check **Do not restrict employee fields**
- For multi-subsidiary NetSuite users, check **All** under **Accessible subsidiaries**
- Under **Authentication**, check **Web Services only Role**.
3. Add the following permissions under **Permissions → Setup**
- Access Token Management: Full
- Custom Body Fields: Full
- Custom Column Fields: Full
- Custom Entity Fields: Full
- Custom Fields: Full
- Custom Item Fields: Full
- SOAP Web Services: Full
- User Access Tokens: Full
4. Under **Permissions → Lists** and **Permissions → Transactions**, provide **Full** permissions for the NetSuite entities you plan on syncing to. For example, if you want to update sales orders, you need to go to the **Transactions** tab and select **Sales Order** from the list. If you want to update accounts, you need to go to **Lists** and select **Accounts** from the list.
### Create a user
Now you can create a new user that uses this role.
1. Go to the **Manage Users** page. Create a new user or select the user you will be using Hightouch with.
2. For new users: fill the **Employee** form with all required fields.
3. On the **Access** tab, check **Manually assign or change password** and assign a secure password.
4. Under **Roles** Assign the role, for example, "Hightouch Role," you previously created.
5. Click **Save**.
### Create an access token
1. Go to the **Setup → Users/Roles → Access Tokens** page, or use the global search to find it.
2. Click **New Access Token**.
3. Select the integration, user, and role you previously configured.
4. When you click **Save**, NetSuite displays a **Token ID** and a **Token Secret**. Copy and store them in a secure location. You'll need them to connect to Hightouch.
### Enable SOAP web services
Finally, go to **Setup → Company → Enable Features → SuiteCloud → SuiteTalk (Web Services)** and enable **SOAP Web Services**. Once you check this box, NetSuite prompts you to agree to terms of service.
On the same page, under **Manage Authentication**, enable **Token Based Authentication**. It should look similar to the screenshot below.
## Connect to NetSuite
You're now ready to create a NetSuite destination in Hightouch. Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **NetSuite** and click **Continue**.
You can then authenticate Hightouch to NetSuite by entering the following required fields into Hightouch:
- **Account ID**: You can find this on your **Company information** page in NetSuite.
- **Consumer Key**: From your new integration
- **Consumer Secret**: From your new integration
- **Token Key**: From your new access token
- **Token Secret**: From your new access token
If your account is a Sandbox account, you need to include the suffix `_SB` as part of the **Account ID**:
- `SB` should be in uppercase.
- The Sandbox suffix should be included with an underscore (`_`).
For example, if the Account ID in your sandbox is `1234567-sb1`, then you need to set your Account ID as `1234567_SB1` in your sync configuration.
## Sync configuration
Once you've set up your NetSuite destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the NetSuite destination you want to sync to.
### Syncing entities
The Hightouch NetSuite destination allows you to sync data to supported NetSuite entities, for example, customers, invoices, vendors, etc. Hightouch also supports syncing custom records types—all custom records are part of the
**CustomRecords** namespace.
The specific entities you can sync to depend on your NetSuite implementation. You can view your available entities in the dropdown. Because of the large catalog size, it's best to search for the entities you're interested in.
#### Related records
If you like, you can choose to initialize the entity with a related record. For example, an invoice can be created from a sales order. If you enable this, you then need to select the base record.
#### Record matching
To match source rows to entities in Netsuite, you need to select a source column and corresponding Netsuite field.
Depending on the sync mode you select, you can match on the following Netsuite fields:
- **External ID**
- **Internal ID**
**Update** mode supports both, whereas **Upsert** mode only supports **External ID**.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, NetSuite automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can sync columns from your model to NetSuite default and custom fields.
Mappings mirror the NetSuite SOAP schema.
That means some of your model columns might need to be JSON objects.
NetSuite generally expects JSON objects when an entity references another entity. For example, in an [invoice](https://system.netsuite.com/help/helpcenter/en_US/srbrowser/Browser2022_2/schema/record/invoice.html), the `entity` field can reference a customer, partner, vendor, employee, contact, etc. by their ID. If you select a numerical model column with external or internal IDs, Hightouch automatically converts this to the JSON object NetSuite expects.
For other JSON objects, such as nested objects or arrays, you need to ensure your model columns match NetSuite's schema expectations.
The [NetSuite REST](/destinations/netsuite-rest#field-mapping) destination
supports the [inline mapper](/syncs/mapping-data#inline-mapping), which can
help you build objects and arrays.
NetSuite arrays are often wrapped in a JSON object.
For example, an invoice supports an `item` field that expects an array of items:
```json
{
"items": [
{ "item": { "id": "8" }, "line": 1, "amount": 1, "price": { "id": "-1" } }
]
}
```
To see the schema that NetSuite expects for different entities's fields, go to the [SOAP browser](https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/section_N3639052.html).
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ------------------------------------------------------ |
| **Do nothing** | Keep the record in NetSuite with all its synced fields |
| **Delete** | Delete the synced record from your NetSuite instance |
## Tips and troubleshooting
For Hightouch platform error codes related to destinations, see [Error
codes](/developer-tools/error-codes#other-destinations).
### Validate your setup
After your first sync completes, verify the results in NetSuite:
1. Search for the synced record in NetSuite using the identifier you mapped (External ID or Internal ID).
2. Confirm the mapped fields contain the expected values.
3. For nested fields (address books, line items), verify the JSON structure rendered correctly in NetSuite's record view.
4. If the sync shows errors, check the sync run log in Hightouch for specific SOAP or REST API error messages.
### Constructing nested JSON objects in SQL
Many NetSuite fields require nested JSON objects — for example, addresses, line items, or sublists. If your warehouse stores these values in flat columns, you can use SQL JSON functions to build the required structure in your model.
The following PostgreSQL example constructs a nested address book entry:
```sql
SELECT
external_id,
json_build_object(
'addressbookAddress',
json_build_object(
'addr1', street,
'city', city,
'state', state,
'zip', zip,
'country', country_code
),
'defaultShipping', true,
'defaultBilling', false
) AS addressbook_entry
FROM customers
```
To build an array of nested objects (for example, multiple address book entries), wrap individual objects with `json_agg`:
```sql
SELECT
external_id,
json_build_object(
'addressbook',
json_agg(
json_build_object(
'addressbookAddress',
json_build_object('addr1', street, 'city', city, 'state', state, 'zip', zip),
'defaultShipping', is_default_shipping
)
)
) AS addressbook
FROM customer_addresses
GROUP BY external_id
```
For Snowflake, use `OBJECT_CONSTRUCT` and `ARRAY_AGG` instead. For BigQuery, use `TO_JSON` with `STRUCT` and `ARRAY_AGG`.
Cast the resulting column to **Object/Array** in your model. Check the [NetSuite SOAP browser](https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/section_N3639052.html) for the exact schema each entity expects.
### Common errors
### Live debugger
### Sync alerts
---
## NetSuite (REST)
**URL:** https://hightouch.com/docs/destinations/netsuite-rest
**Description:** Sync business-critical data from your data warehouse to Netsuite with Hightouch
**Section:** Destinations
Check out our
[playbook](https://hightouch.com/playbooks/automating-usage-based-billing) and
[blog post](https://hightouch.com/blog/automated-billing) to learn how to
automate usage-based billing to streamline your finance operations.
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| -------------- | ---------------------------------------------------------------------------------------------------------------------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| **Entities** | Sync data from any source to entities in NetSuite, for example, Customers, Invoices, Sales Orders, and Journal Entries | Upsert, Update, Insert | [NetSuite schema browser](https://system.netsuite.com/help/helpcenter/en_US/srbrowser/Browser2022_2/schema/record/account.html) |
| **Transforms** | Transform records from one type into another, using data from existing records. | Insert | [NetSuite record transform](https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/section_4267258715.html) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## REST vs SOAP
The Hightouch NetSuite REST destination offers a few advantages over the SOAP destination:
- The NetSuite REST API uses JSON for web service requests and responses, which is easier to read and debug in the [live debugger](/syncs/debugger) compared to XML.
- Hightouch prioritizes supporting the latest REST API features and versions.
- The [NetSuite REST](#field-mapping) destination supports the [inline mapper](/syncs/mapping-data#inline-mapping) to build objects and arrays
Unless you have a particular entity that's only supported by the SOAP API, we recommend using the REST NetSuite destination.
## Setup in NetSuite
Before you can use Hightouch to sync data to NetSuite, you need to create and and configure a NetSuite integration.
Then, to grant access to Hightouch to synchronize data between your source and NetSuite, you need to create a role, a user, and an authentication token for the integration.
These steps are the same whether you choose to [connect Hightouch to NetSuite](#connect-to-netsuite) with basic authentication or OAuth.
The only difference is in the final step of [enabling web services](#enable-rest-web-services).
Depending on your NetSuite account's setup, you may need an **admin** role to
complete setup in NetSuite. Netsuite permission changes to a role or user
don't occur immediately. These changes can take a few minutes or hours to go
into effect.
### Create an integration
1. Go to **Setup > Integration > Manage Integrations** or use the global search to find and go to the **Manage Integrations** page.
This page shows a list of enabled integrations for your account.
2. Click **New** to create a new integration with the following properties:
- A **Name** that can easily be associated with Hightouch, for example, "Hightouch Integration"
- Set **State** to **Enabled**
- Check **Token-Based Authentication**
- Uncheck **TBA:Authorization flow**
- Uncheck **Authorization Code Grant**
- Leave other authentication options unchanged.
3. Scroll to the bottom of the page and click **Save**.
4. Scroll to **Client credentials** and Copy and store the **Consumer Key / Client ID** and **Consumer Secret / Client secret** in a secure location. You'll need them to connect to Hightouch.
### Create a role
1. Go to **Setup > Users/Roles > Manage Roles** and click **New** or use the global search to find and go to the **New Role** page.
2. Create a role with the following properites:
- A descriptive **Name**, for example, "Hightouch Role."
- Check **Do not restrict employee fields**
- For multi-subsidiary NetSuite users, check **All** under **Accessible subsidiaries**
- Under **Authentication**, check **Web Services only Role**.
3. Add the following permissions under **Permissions > Setup**
- Access Token Management: Full
- Custom Body Fields: Full
- Custom Column Fields: Full
- Custom Entity Fields: Full
- Custom Fields: Full
- Custom Item Fields: Full
- REST Web Services: Full
- User Access Tokens: Full
4. Under **Permissions > Lists** and **Permissions > Transactions**, provide **Full** permissions for the NetSuite entities you plan on syncing to. For example, if you want to update sales orders, you need to go to the **Transactions** tab and select **Sales Order** from the list. If you want to update accounts, you need to go to **Lists** and select **Accounts** from the list.
### Create a user
Now you can create a new user that uses this role.
1. Go to the **Manage Users** page. Create a new user or select the user you will be using Hightouch with.
2. For new users: fill the **Employee** form with all required fields.
3. On the **Access** tab, check **Manually assign or change password** and assign a secure password.
4. Under **Roles** Assign the role, for example, "Hightouch Role," you previously created.
5. Click **Save**.
### Create an access token
1. Go to the **Setup > Users/Roles > Access Tokens** page, or use the global search to find it.
2. Click **New Access Token**.
3. Select the integration, user, and role you previously configured.
4. When you click **Save**, NetSuite displays a **Token ID** and a **Token Secret**. Copy and store them in a secure location. You'll need them to connect to Hightouch.
### Enable REST web services
Finally, go to **Setup > Company > Enable Features > SuiteTalk (Web Services)** and enable **REST Web Services**. Once you check this box, NetSuite prompts you to agree to terms of service.
On the same page, under **Manage Authentication**, enable either :
- **Token Based Authentication**, if you plan on [connecting Hightouch](#connect-to-netsuite) to NetSuite with basic authentication
- **OAuth 2.0**, if you plan on [connecting Hightouch](#connect-to-netsuite) to NetSuite with OAuth
If you're not sure which you will use, you can check both of them.
## Connect to NetSuite
You're now ready to create a NetSuite destination in Hightouch. Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **NetSuite** and click **Continue**. You can then authenticate Hightouch to NetSuite via **Basic Auth** or **OAuth**.
OAuth only requires you to login to your NetSuite account rather than enter credentials.
### Authenticate with basic auth
Basic auth requires you to enter the following required fields into Hightouch:
- **Account ID**: You can find this on your **Company information** page in NetSuite.
- **Consumer Key**: From your new integration
- **Consumer Secret**: From your new integration
- **Token Key**: From your new access token
- **Token Secret**: From your new access token
If your account is a Sandbox account, you need to include the suffix `_SB` as part of the **Account ID**:
- `SB` should be in uppercase.
- The Sandbox suffix should be included with an underscore (`_`).
For example, if the Account ID in your sandbox is `1234567-sb1`, then you need to set your Account ID as `1234567_SB1` in your sync configuration.
### Authenticate with OAuth
For the **Authentication method**, select **OAuth** and log into your NetSuite account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your NetSuite destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the NetSuite destination you want to sync to.
### Syncing entities
The Hightouch NetSuite destination allows you to sync data to supported NetSuite entities, for example, customers, invoices, vendors, etc. Hightouch also supports syncing custom records types.
The specific entities you can sync to depend on your NetSuite implementation. You can view your available entities in the dropdown. Because of the large catalog size, it's best to search for the entities you're interested in.
#### Record matching
To match source rows to entities in Netsuite, you need to select a source column and corresponding Netsuite field.
Depending on the sync mode you select, you can match on the following Netsuite fields:
- **External ID**
- **Internal ID**
**Update** mode supports both, whereas **Upsert** mode only supports **External ID**.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, NetSuite automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can sync columns from your model to NetSuite default and custom fields.
Mappings mirror the NetSuite REST schema. That means some of your model columns might need to be JSON objects.
NetSuite generally expects JSON objects when an entity references another entity. For example, in an [invoice](https://system.netsuite.com/help/helpcenter/en_US/srbrowser/Browser2022_2/schema/record/invoice.html), the `entity` field can reference a customer, partner, vendor, employee, contact, etc. by their ID. To make that reference, you need to map a JSON object like this to the `entity` field:
```json
{ "id": 1 }
```
You can use also external IDs for references, for example:
```json
{ "externalId": 1 }
```
The [inline mapper](/syncs/mapping-data#inline-mapping) is purpose-built to help you create [JSON objects](/syncs/mapping-data#create-an-object) like this.
The inline mapper can also help you create [arrays](/syncs/mapping-data#create-an-array).
NetSuite arrays are often wrapped in a JSON object.
For example, an invoice supports an `item` field that expects an array of items:
```json
{
"items": [
{ "item": { "id": "8" }, "line": 1, "amount": 1, "price": { "id": "-1" } }
]
}
```
Refer to the [inline mapping docs](/syncs/mapping-data#create-an-array) to learn how to create an array like this and what the [required data format](/syncs/mapping-data#required-data-format) for your model is. To see the schema that NetSuite expects for different entities, go to the [REST browser](https://system.netsuite.com/help/helpcenter/en_US/APIs/REST_API_Browser/record/v1/2022.1/index.html).
#### Overwrite sublists
Updating sublist properties can result in appending the mapped items to the record's existing sublist.
This may result in unwanted duplicates. You can explicitly specify in Hightouch to overwrite your
mapped sublist items in NetSuite. Overwriting the sublist removes all existing items and updates
it with incoming data.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ------------------------------------------------------ |
| **Do nothing** | Keep the record in NetSuite with all its synced fields |
| **Delete** | Delete the synced record from your NetSuite instance |
### Transforming records
The Hightouch NetSuite destination allows you to transform records from one type to another, for example, transforming purchase orders into vendor bills.
#### Record matching
To match source rows to entities in Netsuite, you need to select a source column and corresponding Netsuite field.
The ID must be the ID of the original record that will be transformed. You may use the following IDs to match on:
- **External ID**
- **Internal ID**
#### Field mapping
You can sync columns from your model to NetSuite default and custom fields of the target record type.
Mappings mirror the NetSuite REST schema. That means some of your model columns might need to be JSON objects.
NetSuite generally expects JSON objects when an entity references another entity. For example, in an [invoice](https://system.netsuite.com/help/helpcenter/en_US/srbrowser/Browser2022_2/schema/record/invoice.html), the `entity` field can reference a customer, partner, vendor, employee, contact, etc. by their ID. To make that reference, you need to map a JSON object like this to the `entity` field:
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Nextdoor
**URL:** https://hightouch.com/docs/destinations/nextdoor
**Description:** Elevate your local marketing efforts with Nextdoor Ads and drive business growth by reaching nearby customers in their communities
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
-----------|----------------------------------------------|---------------------
**Events** | Sync data from any source to Nextdoor events | Insert
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Nextdoor
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Nextdoor** and click **Continue**. You can then authenticate Hightouch to **Nextdoor**.
Enter the following required fields into Hightouch:
- **API token**: Enter a valid API token.
## Sync configuration
Once you've set up your Nextdoor destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Nextdoor destination you want to sync to.
### Syncing events
Nextdoor's Conversions API requires the following event parameters:
- `event_name`
- `event_time`
- `action_source`
- `client_id`
- at least one `customer` parameter
#### Event name
Providing an event name is required to send an event to the Conversions API. You can either provide a static value or select to **use a column** from your model. Hightouch syncs the static or column value as the `event_name` parameter the Conversions API requires.
Supported values include:
- `conversion`
- `lead`
- `purchase`
- `sign_up`
- `custom_conversion_1`
- `custom_conversion_2`
- `custom_conversion_3`
- `custom_conversion_4`
- `custom_conversion_5`
- `custom_conversion_6`
- `custom_conversion_7`
- `custom_conversion_8`
- `custom_conversion_9`
- `custom_conversion_10`
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server. Hightouch syncs the timestamp as the `event_time` parameter the Conversions API requires.
If you select a column, it should be in [ISO 8601](https://www.iso.org/iso-8601-date-and-time-format.html) format. Hightouch automatically converts the data to the format Nextdoor expects.
#### Action source
The `action_source` is a required parameter and indicates the channel through which conversion happened, for example, over email, on your website, etc. Knowing where your events took place helps ensure your ads go to the right people.
#### Client ID
The `client_id` is a required parameter, also known as the NAM Advertiser ID. To get your `client_id`, reach out to your Nextdoor account manager.
#### Test event codes
Test event codes are special codes you can include in your event payload to simulate conversion events. This allows you to test your API implementation without actually recording real conversion data.
You can optionally enter a test event code as part of your sync configuration.
Make sure to remove the test event code if you're running a production sync.
#### Hashing
By default, Hightouch automatically hashes the following customer fields:
- **email**
- **phone_number**
- **first_name**
- **last_name**
- **date_of_birth**
- **street_address**
- **city**
- **state**
- **zip_code**
- **country**
If you want to turn hashing off, select **No** under **Would you like Hightouch to automatically detect if your PII data requires hashing?**.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Notion
**URL:** https://hightouch.com/docs/destinations/notion
**Description:** Power up your Notion databases with data from your data warehouse
**Section:** Destinations
|Sync Type |Description |Supported Sync Modes |
|--------------|----------------------------------------------------------------------------------------------|---------------------|
|Database rows | Upsert or update rows in a Notion [database](https://www.notion.so/help/intro-to-databases) |Upsert, Update |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Notion
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Notion** and click **Continue**. You can then authenticate Hightouch to Notion by logging into Notion.
Next, select which Notion pages you want to give Hightouch access to.
Be sure to give access to pages that include databases you want to sync to.
If the page containing the database within Notion has restricted access, you won't be able to configure a sync to it.
Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Notion destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Notion destination you want to sync to.
### Database selection
After selecting your desired [sync mode](#supported-syncing), select the Notion database you want to sync data to.
### Record matching
To match rows from your model to database rows in Notion, you need to select a model column and corresponding Notion field.
It's best to select a unique identifier for record matching, otherwise the intended records may not be updated properly.
Refer to the [record matching docs](/syncs/record-matching) for more information.
The Notion field you select for record matching [must be a **text** field](#column-not-appearing-for-record-matching).
### Field mapping
You can sync model columns to Notion's database properties.
Hightouch automatically detects existing database properties from your Notion databases.
#### Supported Notion properties
Hightouch supports the following Notion database properties when field mapping.
- `Checkbox`
- `Date`: Hightouch supports a single date, and not a date range
- `Email`
- `Multi Select`
- `Number`: If a number field is passed an invalid number, Hightouch attempts to convert it to a number. If the conversion result is invalid, the sync errors.
- `Phone`
- `Rich Text`
- `Select`
- `Title`
- `URL`
To learn more about Notion's available properties, please refer to [Notion's docs](https://developers.notion.com/reference/property-object).
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available on **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
| Behavior | Description |
| -------------- | ------------------------------------------------------------- |
| **Do nothing** | Keep the row in the database with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the row in the database |
| **Delete** | Delete the synced row in the database |
## Tips and troubleshooting
### Common errors
#### Database not appearing in dropdown
If the page containing the database within Notion has restricted access, the database won't appear in the dropdown menu of the sync configuration, even if Hightouch is explicitly granted access during authentication.
You may also receive a `404 - Could not find database with ID: {...}` error in the [record matching](/syncs/record-matching) section of your sync configuration, or when attempting to sync to Notion.
To fix this, go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and select your Notion destination.
Click **Log in to Notion** and add access to any required pages.
Then, return to your sync configuration or create a new one from the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs).
You may need to click the refresh button to see the new database.
#### Column not appearing for record matching
Hightouch only supports record matching on **text** fields in Notion.
Convert the field type to text in Notion, or create a new text field.
#### 400 - body failed validation \{...\} should be defined, instead was undefined
This error occurs if you try to sync `null` values to [Select](https://developers.notion.com/reference/property-object#select) or [Multi-select](https://developers.notion.com/reference/property-object#multi-select) fields in Notion.
To resolve this, check the [**Don't sync null values** option](/syncs/mapping-data#dont-sync-null-values) in the template mapper or [remove rows containing `null` values](/models/sql-editor#exclude-rows-with-null-values) from your model's query results.
You need to do this option for all field mappings to [Select](https://developers.notion.com/reference/property-object#select) or [Multi-select](https://developers.notion.com/reference/property-object#multi-select) fields in Notion, if you believe the mapped model columns might contain `null` values.
#### 400 - body failed validation \{...\} should be a string, instead was \{...\}
This error occurs if you try to sync non-text values to a [Text](https://developers.notion.com/reference/property-object#rich-text) field in Notion.
To resolve this, [cast](/models/data-types-casting#casting-from-your-model-configuration) the relevant column or columns from your model to a **String** type.
### Live debugger
### Sync alerts
---
## Ometria
**URL:** https://hightouch.com/docs/destinations/ometria
**Description:** Ometria is a customer data and marketing platform built for retail brands that want to turn fragmented data into revenue. It unifies customer data in real time, uses AI to predict behavior, and lets you trigger highly personalized campaigns across email, SMS, and onsite—so you can send the right message at the right time without duct-taping multiple tools together.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|--------------|--------------------------------------------------|----------------------|--------------------------------------------------------------------------------------------------------|
| **Contacts** | Sync data from any source to contacts in Ometria | Upsert | [Contacts docs](https://docs.ometria.com/docs/contacts-endpoint) |
| **Orders** | Sync data from any source to orders in Ometria | Upsert | [Orders docs](https://docs.ometria.com/docs/orders-endpoint) |
| **Products** | Sync data from any source to products in Ometria | Upsert | [Products docs](https://docs.ometria.com/docs/products-product-listings-and-product-variants#products) |
| **Events** | Sync data from any source to events in Ometria | Insert | [Events docs](https://support.ometria.com/hc/en-gb/articles/360011171077-Setting-up-custom-events) |
All sync types use the [push endpoint](https://docs.ometria.com/reference/post_push) to sync data to Ometria.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Ometria
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Ometria** and click **Continue**. You can then authenticate Hightouch to **Ometria**.
Enter the following fields into Hightouch:
- **API key:** To create a new API key, refer to Ometria's [API keys docs](https://support.ometria.com/hc/en-gb/articles/9882706389917-Creating-a-new-API-key).
## Sync configuration
Once you've set up your Ometria destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Ometria destination you want to sync to.
### Syncing contacts, orders, and products
Sync data from any source to contacts, orders, and products in Ometria.
#### Record matching
To match rows from your model to objects in Ometria, you need to select the model column that contains values that match the **ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom contact fields.
For orders, Ometria requires the following fields, so you must map them to complete your configuration:
- **Timestamp**
- **Grand total**
- **Subtotal**
- **Currency**
For products, Ometria requires the following fields, so you must map them to complete your configuration:
- **Title**
- **Price**
Ensure your model's columns have the same data types as the fields you want to sync to.
### Syncing events
Sync data from any source to events in Ometria.
#### Record matching
You can match rows from your model to events in Ometria on any column in your model and any field in Ometria. Ensure the data types of the model column and Ometria field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Microsoft OneDrive
**URL:** https://hightouch.com/docs/destinations/onedrive
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
| ------------ | ----------------------------------------------------------| -------------------- |
| Any data set | Sync data from a source to One Drive as JSON or CSV files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Diff mode creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Microsoft OneDrive
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Microsoft OneDrive** and click **Continue**. You can then connect to Microsoft OneDrive via OAuth.
After authorizing, you will have successfully connected Hightouch to Microsoft OneDrive.
## Sync configuration
Once you've set up your OneDrive destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the OneDrive destination you want to sync to.
### Select file format
Hightouch supports syncing JSON and CSV files to OneDrive.
### Write to an Owned folder or Shared folder
You can sync to a folder that is owned by or shared to the account connected to Hightouch.
Select either **Owned folders** or **Shared folders** to see available folder options.
### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it. To view different versions of the same file, you can right-click the file in Microsoft OneDrive and then click **Version history**.
If you are using an audience and would like to include the audience name, you will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter '-86400' (`24 hours * 60 minutes * 60 seconds`).
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `company`, `email`, `name`, and `primaryKey` fields. Hightouch exports these fields to new fields in the file and ignores all other columns from your results.
### CSV options
If you're syncing to a CSV file, you have additional configuration options:
- **Delimiter**: Your options are comma (`,`), semicolon (`;`), pipe (`|`), tilde (`~`), and tab
- (Optional) Whether to **include a CSV header row** in the exported files
- (Optional) Whether to **include a [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark)** in the exported files, the BOM is ``
### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
## Tips and troubleshooting
### Need admin approval
If you see a **Need admin approval** page appear when [accessing Microsoft's OAuth login flow](#connect-to-microsoft-onedrive), your Azure Active Directory user account might not have the necessary permissions to connect applications to your organization.
Ask one of your Azure organization's admins to edit your user account permissions.
You can read more about [Azure user permissions](https://learn.microsoft.com/en-us/azure/active-directory/develop/active-directory-how-applications-are-added#who-has-permission-to-add-applications-to-my-azure-ad-instance) and [configuring user settings](https://learn.microsoft.com/en-us/azure/active-directory/manage-apps/configure-user-consent?pivots=portal#configure-user-consent-settings) in Azure's documentation.
### Common errors
### Live debugger
### Sync alerts
---
## OneSignal
**URL:** https://hightouch.com/docs/destinations/onesignal
**Description:** Deliver mobile and web push notifications using OneSignal with fresh data from your data warehouse
**Section:** Destinations
## Overview
With the OneSignal destination, Hightouch can:
- Update tags on existing devices via the external unique ID
## Setup
To set up the OneSignal destination, navigate to Destinations and Add New. We can find the keys needed to access OneSignal at OneSignal > Settings > Keys & IDs.
## Modes
### Updating tags via external unique ID
The destination is configured by default to update tags via external unique ID. In this mode, tags are added or updated on existing devices using the external unique ID.
This mode is good for the following scenarios:
- Adding LTV based on external unique ID to a device
- Assigning segment data to a device
| Name | Description |
| :----------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------- |
| Hightouch Column Containing OneSignal External User ID | The column containing the external user ID of the OneSignal device, which is contained in the OneSignal field `external_user_id` |
| Hightouch Column to OneSignal Tag Mappings | A list of columns that will be pushed to tags in OneSignal. Each column and it's value for each row will be a key:value pair in the device's tags. |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## ChatGPT Ads (OpenAI)
**URL:** https://hightouch.com/docs/destinations/open-ai
**Description:** Advertising in ChatGPT helps advertisers reach people as they explore ideas, evaluate options, and make decisions through relevant ads designed to fit naturally into the experience.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|------------|-------------------------------------------------------|----------------------|------------------------------------------------------------------|
| **Events** | Sync data from any source to OpenAI conversion events | Insert | [Events docs](https://developers.openai.com/ads/conversions-api) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to OpenAI
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **OpenAI** and click **Continue**. You can then authenticate Hightouch to **OpenAI**.
Enter the following fields into Hightouch:
- **Pixel ID**
- **API key**
## Sync configuration
Once you've set up your OpenAI destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the OpenAI destination you want to sync to.
### Syncing events
Sync data from any source to OpenAI conversion events.
#### Record matching
You can match rows from your model to events in OpenAI on any column in your model and any field in OpenAI. Ensure the data types of the model column and OpenAI field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## OpenSearch
**URL:** https://hightouch.com/docs/destinations/opensearch
**Description:** OpenSearch is a distributed, free and open search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured.
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes|
|--|--|--|
|**Index documents**|Sync data from any source to an OpenSearch index or index alias|Upsert, Update|
## Connect to OpenSearch
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **OpenSearch** and click **Continue**. You can then authenticate Hightouch to OpenSearch by entering your:
- **Host**: The hostname or IP address of your OpenSearch server. Don't include `https://` as part of the hostname.
- **Port**: The port number of your OpenSearch server's HTTP API. The default is 3306, but yours may be different.
- **Username**: This can be your personal OpenSearch login or a dedicated user for Hightouch.
- **Password**: The password for the user specified above.
Once you've entered these fields, click **Continue**. Give your destination a descriptive name and **Finish** the destination connection setup.
## Sync configuration
Once you've set up your OpenSearch destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the OpenSearch destination you want to sync to.
### Syncing to an index
An OpenSearch [index](https://opensearch.org/docs/latest/im-plugin/index/) is the place where the data used by a search engine is stored.
You can use Hightouch to update individual [documents](https://opensearch.org/docs/latest/api-reference/document-apis/index-document/) in an index.
You can also sync your document data via [index aliases](https://opensearch.org/docs/latest/im-plugin/index-alias/).
Index aliases will only be displayed if the [`is_write_index` option](https://opensearch.org/docs/latest/im-plugin/index-alias/#index-alias-options) is configured.
#### Record matching
To match rows from your model to documents in OpenSearch, you need to select a model column and corresponding OpenSearch field. You can match on any OpenSearch field.
#### Field mapping
You can sync data to both existing fields and fields that don't yet exist in OpenSearch. If you send data to a field that doesn't exist yet, Hightouch adds the field and automatically detects its type.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ------------------------------------------------------------------ |
| **Do nothing** | Keep the document in OpenSearch |
| **Clear** | Keep the document in OpenSearch, but set mapped fields to empty |
| **Delete** | Remove the document from OpenSearch |
#### Advanced configuration
You can configure how many rows you'd like to send per batch and how many batches you'd like to send in parallel.
If your sync is rejecting all or most rows in your model with vague error messages, try decreasing the batch size.
## Tips and troubleshooting
### Common errors
#### Failed to connect to your OpenSearch client. Check your credentials.
This error can occur if your credentials are incorrect.
Ensure your hostname doesn't include `https://` and that your port is correct.
### Live debugger
### Sync alerts
---
## Optimizely
**URL:** https://hightouch.com/docs/destinations/optimizely
**Description:** Power digital experiences your customers will love with rich customer data
**Section:** Destinations
## Overview
You want to provide relevant, timely interactions to your customers, and doing so heavily relies on data in your Optimizely account to be accurate and fresh. By automatically syncing customer data and events from your data warehouse into Optimizely, you no longer need to worry about data consistency because data is no longer flowing in from multiple sources in different formats, and you can focus on building world-class customer experiences.
## Supported syncing
| Event Type | Description | Supported Sync Modes |
| --------------------- | -------------------------------------------------- | -------------------- |
| **[Log Events](https://docs.developers.optimizely.com/experimentation/v20.0.0-data/reference/post_events)** | Send any event-based data associated with a user | Upsert |
| **[Visitor Attributes](https://docs.developers.optimizely.com/experimentation/v20.0.0-data/reference/post_events)** | Add customizable data or characteristics to a user | Upsert |
| **[ODP Audiences](https://docs.developers.optimizely.com/optimizely-data-platform/reference/create-list-1)** | Maintain subscription statuses of users in ODP audiences | Add, Remove |
## Connect to Optimizely
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Optimizely** and click **Continue**. You can then authenticate Hightouch to Optimizely with a personal access token and an account ID.
### Web experimentation
#### Authenticate with a personal access token
To use the Optimizely web experimentation APIs, generate a personal access token:
1. Navigate to your [Optimizely profile](https://app.optimizely.com/v2/profile/api) and click **Generate New Token**.
2. Enter a name for the token and click **Create**.
You will also need the `Project ID`. The project ID can be found in the URL by going to your [Optimizely account](https://app.optimizely.com/v2/projects).
### Optimizely data platform
#### Authenticate with a private API key
To use the Optimizely Data Platform APIs, get your API key by following [Optimizely's guide](https://docs.developers.optimizely.com/optimizely-data-platform/reference/authentication) on ODP private keys.
## Syncing configuration
### Log events
Syncing log events lets you send user actions from any source such as offline experiment data, web-page interactions, or third-party sources.
#### Record matching
You can associate events with the visitors who made them on the `Visitor ID` field.
#### Timestamps
Hightouch allows you to send a timestamp of when an event occurred. Timestamps must be sent as milliseconds since the epoch. If an event doesn't include a timestamp, the time of the API request is used.
#### Field mapping
You can sync columns from your source to Optimizely event fields. Optimizely events require a UUID that will be used by Optimizely to de-duplicate events that are accidentally or erroneously replayed.
### Visitor attributes
Syncing visitor attributes lets you maintain up-to-date customer data and create better audiences for personalized experiences in Optimizely. Note that you cannot create net-new properties with this API: attributes must be created within Optimizely before using this endpoint. Since Optimizely audience membership is handled by whether a visitor's attributes satisfy the conditions required for the audience, sending data to this endpoint controls audience membership in Optimizely.
#### Record matching
You can associate attributes to visitors on the `Visitor ID` field.
#### Field mapping
You can sync columns from your source to Optimizely visitor attributes. Attributes must be created within Optimizely before creating mappings.
### ODP audiences
Sync data from any source to maintain subscription statuses of audiences in Optimizely.
#### Select an existing audience or create a new one
You can create a new audience segment or use an existing one. When creating a new audience, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model. To use an existing audience, select the desired audience from the dropdown.
#### User identifier
To identify which users to add or update in an audience, select a model column containing the user identifier. This identifier will be mapped to the `fs_user_id` and `web_user_id`.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Oracle DB
**URL:** https://hightouch.com/docs/destinations/oracle-db
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------ | ----------------------------------------------- | -------------------- |
| Any data set | Sync data from any source to an Oracle DB table | Upsert |
## Connect to Oracle DB
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Oracle DB** and click **Continue**. If you're not using a [tunnel](#ssh-tunneling), you can then authenticate Hightouch to Oracle DB by entering the following fields:
- **Host**: The hostname or IP address of your Oracle server.
- **Port**: The port number of your Oracle server. The default port number is 1521, but yours may be different.
- **Database**: This specifies the database to use when Hightouch executes queries in Oracle DB. This is different than the host, but your host address might contain your database name.
- **Username**: The user that has access through Hightouch to the database and tables you want to sync to. It's best to [create a new user](https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/CREATE-USER.html) specifically for Hightouch access. _Do not use the root user._
- **Password**: The password for the specified user.
To ensure your [credentials](#required-permissions) are correct, click **Test connection**. This confirms if Hightouch can connect to your database by running a basic `SELECT` query.
### Required permissions
The user you use to authenticate must have the following permissions:
- Add, update, and delete ([if applicable](#delete-behavior)) rows in your sync's table.
- Create tables in your database, which are kept only temporarily and used to perform `MERGE` operations. These tables are cleaned up at the end of each sync.
- View the following tables which are used for gathering metadata to set up the sync:
- `USER_CONSTRAINTS`
- `USER_CONS_COLUMNS`
- `USER_CATALOG`
- `USER_TAB_COLUMNS`
### SSH tunneling
Hightouch can connect directly to Oracle DB over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your Oracle DB instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
## Sync configuration
Once you've set up your Oracle DB destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Oracle DB destination you want to sync to.
Hightouch supports **Upsert** mode, with the option to [delete removed rows](#delete-behavior).
### Record matching
Hightouch requires you select a unique identifier in the table you are syncing to. The model column you select must match a `UNIQUE`, `PRIMARY KEY` column within Oracle DB.
You can see columns that fit this criteria as available options in records matching section.
If there are no fields in the dropdown, you must add a unique type column to your Oracle DB table. Then, click the refresh icon to access the newly created column.
### Column types
Hightouch works out of the box with most standard column types, including:
- `VARCHAR2`
- `NUMBER`
- `TIMESTAMP`
- `DATE`
- `BINARY_FLOAT`
- `BINARY_DOUBLE`
If you see type errors, it may be because your model is producing the wrong format. If so, use [typecasting](/models/data-types-casting) to resolve the issue.
### Field mapping
Select which model columns you want to sync to your Oracle DB table columns. Hightouch automatically pulls the columns from your table to make them available for you to map.
### Batch size
You can tune the number of rows that Hightouch upserts per query based on your needs and database threshold. The default is 1000 rows per batch.
### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | -------------------------------------- |
| **Do nothing** | Keep the row in Oracle DB |
| **Delete row** | Remove the row from Oracle DB entirely |
## Tips and troubleshooting
### Common errors
### Compatible versions
The supported Oracle DB version is `19c`. Other versions aren't officially supported but may work regardless.
### Live debugger
### Sync alerts
### Unsupported data types
Hightouch does not support `INTERVAL` types for Oracle.
---
## Orbit
**URL:** https://hightouch.com/docs/destinations/orbit
**Description:** Enrich data of your community members and automate actions
**Section:** Destinations
## Setup
To enable syncing with [Orbit](https://orbit.love) first you need to get an authentication token. Go to your Account Settings (on the menu
under your profile picture) and click API Token:
Then copy the value, and create a new Destination in Hightouch, selecting Orbit as the destination type. Lastly, paste
the API token into the field. Save the destination and it's ready to start syncing data.
## Syncing activities
When syncing activities, Hightouch treats any records added to [your source](/getting-started/concepts) as new activities and sends them to Orbit when your sync runs.
Select the _Activities_ object from the drop down menu. Then you can
define a static value for the title and the identity source for all the
activities, or define the column that will contain the information.
Lastly, choose the mappings for the columns in your model.
## Syncing members
Select the _Members_ object from the drop down menu.
### Record matching
Records can be matched from your source to your Orbit workspace by using the email field.
### Field mapping
You can sync columns from your source to Orbit Member fields.
### Rate limits
By default, Orbit limits 120 requests per IP per minute. If you experience slow syncs, please contact them to increase that limit,
and let us know.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Ortto
**URL:** https://hightouch.com/docs/destinations/ortto
**Description:** Ortto brings the customer data platform, customer journey marketing, and customer journey analytics together to grow your SaaS business.
**Section:** Destinations
## Overview
Enrich people and organization objects in Ortto and keep them up-to date with changes from your data warehouse. This destination also empowers you to customize your audiences by subscribing or unsubscribing people from them based on custom criteria.
## Supported syncing
|Sync Type |Description |Supported Sync Modes |
|------------------|-----------------------------------------------------------------------------------------------|---------------------|
|**Person** |Sync records to Ortto [people](https://help.ortto.com/user/latest/people/) | Upsert, Update |
|**Organization** |Sync records to Ortto [organizations](https://help.ortto.com/user/latest/organizations/) | Upsert, Insert |
|**Audience** |Subscribe or unsubscribe people from [audiences](https://help.ortto.com/user/latest/audiences/)| Insert |
## Getting started
Connecting Ortto to Hightouch requires an Ortto **API Key**. You can retrieve it by following these steps:
1. Navigate to **Data sources** under **More** in the Ortto UI.
2. Create a new data source by selecting **Custom API (advanced)**.
3. Enter an **API name**, for example "Hightouch."
4. Copy your **Private API key** into Hightouch.
5. Select the region of your Ortto instance from the dropdown menu.
If you need further details on creating an Ortto API key, check out the [Ortto documentation](https://help.ortto.com/user/latest/data-sources/configuring-a-new-data-source/other-integrations/custom-api.html).
## Syncing data
Hightouch supports syncing to the following to Ortto [people](#people), [organizations](#organizations), and [audiences](#audiences). You need to setup a separate sync for each object type you want to sync to.
### People
You can upsert, update, or insert [people](https://help.ortto.com/user/latest/people/) in Ortto based on records in your query results.
#### Record matching
You can match rows in your model with people on the following fields:
- Email
- External ID
#### Field mapping
You can sync columns from your query results to [people's](https://help.ortto.com/user/latest/people/) default and custom fields.
#### Delete behavior
You have the following options for how Hightouch should handle rows leaving your query results:
| Behavior | Description |Supported Sync Modes |
| ---------------------| -----------------------------------------------------------------------------|---------------------|
| **Do nothing** | Keep the person object in Ortto | Any |
| **Clear** | Clear the mapped fields but keep the person object in Ortto | Update |
| **Delete** | Delete the person object from Ortto completely | Upsert |
### Organization
You can perform upsert or insert [organizations](https://help.ortto.com/user/latest/organizations/) in Ortto based on records in your query results.
#### Record matching
You can match rows in your model with organization on the following fields:
- Name
- Website
#### Field mapping
You can sync columns from your query results to an [organization's](https://help.ortto.com/user/latest/organizations/) default and custom fields.
### Audiences
Subscribe people to [audiences](https://help.ortto.com/user/latest/audiences/) in Ortto when records for them appear in your query results.
#### Record matching
You can match rows in your model with people in Audience on email only.
#### Delete behavior
Records that no longer appear in your query results are unsubscribed from Ortto audiences.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Outreach
**URL:** https://hightouch.com/docs/destinations/outreach
**Description:** Empower your sales team to act upon customer activity by pushing enriched customer data from your warehouse into Outreach
**Section:** Destinations
## Setup
After selecting Outreach from our Destinations catalog, you will be prompted to
initiate an OAuth flow. You will then be redirected back to Hightouch.
## Syncing
Hightouch supports syncing to the following Outreach objects:
- `Accounts`
- `Prospects`
- `Sequence States` (instance of a prospect within a sequence)
### Sync modes
Here are the possible modes for how to alter Outreach objects:
- **Upsert**: pushes new objects to Outreach and updates fields that change in your warehouse.
- **Update**: updates particular fields on existing objects in Outreach. It doesn't add new objects.
`Sequence States` will only have _insert_ mode
- **Insert**: pushes new objects to Outreach. It doesn't update any objects.
### Record matching
Records can be matched from your source to your Outreach workspace by the given fields in the dropdown.
The record should be matched using a unique identifier, otherwise the intended
records may not be updated properly.
### Field mapping
You can sync columns from your source to Outreach's object properties, including custom fields, and relationships.
You will still need to refer to the custom field number, despite having a different custom display name.
### Syncing sequence states
Syncing sequence state objects is adding prospects to their corresponding sequences. To successfully start a sequence and engage
with the prospect, you must make sure your sequence has the correct settings and sending and syncing messages from mailbox are enabled (if emailing).
Creating a sequence state requires that you have information on all of these Outreach objects:
- prospects
- sequences
- mailboxes
Here is an example on how to set up the configurations:
You will select a field from your model to match to the corresponding object. You need to make sure you are using a unique identifier.
Hightouch will use that to look up each of the corresponding object's ID and combine them to create a sequence state.
## Sync performance and parallelization
Outreach has a fixed limit of 10,000 requests per hour per user. This limit resets every hour on the dot.
To accommodate this limit and avoid potentially impacting your usage of the Outreach desktop app during the sync run, Hightouch will draw out the sync by sending just one request at a time.
If the limit is hit during a sync run, Hightouch will wait until the next hour to continue the rest of the sync.
Note that the wait time can vary depending on when the rate limit is hit.
If you want to speed up your sync, you may increase the parallelization to send more requests in parallel.
Keep in mind that this will increase the risk of hitting the rate limit. We recommend that you run a test through the "Test a row"
feature in the sync configuration form page and spot check how many requests are made. You can then estimate how many requests
each sync run will make based on how many rows you expect to be added, changed, or removed. If you estimate that you will exceed 10,000 requests,
it's best to not increase the parallelization. Another safety net is to create a Hightouch specific user
for the sync so it doesn't lock out other users from using the Outreach app.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Destinations overview
**URL:** https://hightouch.com/docs/destinations/overview
**Description:** Learn how to connect destinations in Hightouch so your models can be synced into the tools where teams run campaigns, measure results, and engage customers.
**Section:** Destinations
A **destination** is any system where teams consume activated data.
Common examples include:
- **CRMs** (Salesforce, HubSpot)
- **Marketing platforms** (Braze, Iterable, Marketo)
- **Ad platforms** (Meta Ads, Google Ads, LinkedIn)
- **Support tools** (Zendesk, Intercom)
- **Analytics platforms** (Amplitude, Mixpanel)
- **Custom APIs** (via Webhook/HTTP Request destinations)
Destinations are the endpoint of Hightouch’s [**Reverse ETL**](/getting-started/concepts) pipeline. Once a [**model**](/models/creating-models) is defined, you can configure one or more [**syncs**](/syncs/overview) to deliver data into destinations.
**For marketers**: a destination is the tool where you build campaigns, measure results, or engage with customers—like an ads platform, email tool, or CRM.
---
## Add a destination
To create a new destination:
1. Go to **Integrations → [Destinations](https://app.hightouch.com/destinations?from=docs)**.
2. Click **Add destination**.
You’ll be prompted to choose from supported destination types and configure:
- **Authentication** (OAuth, API key, service account, etc.)
- **Object type** (e.g. user, event, list, custom object)
- **Sync type** supported by that destination (upsert, insert, update, event streaming, etc.)
Each destination has unique configuration options. See catalog and refer to **integration-specific documentation** for setup details and supported sync types.
---
## Security and data handling
Hightouch writes data into destinations using secure APIs:
- All outbound traffic is encrypted via TLS 1.2+
- Data is batched and delivered according to the sync configuration (mode, type, mapping)
- Sensitive fields can be masked or excluded before syncing to destinations
See [Security and compliance overview](/security/overview) for more details.
---
## Permissions
Access to individual destinations is controlled through custom roles. Admins can configure destination-specific grants in the **Destination** tab of the custom role builder, including per-destination read, write, and manage permissions. See [Roles](/workspace-management/roles#destination) for details.
---
## How destinations fit into data activation
- **Data teams**: configure destinations and map model columns to the correct destination fields.
- **Marketers**: access fresh, trusted data directly in the tools where they run campaigns or measure results.
**Example:**
- **Model**: SQL query for “users who abandoned checkout”
- **Destination**: Braze (Email + Push)
- **Sync**: upsert new/changed users nightly, send them into a “Cart Abandonment” campaign
---
## Next steps
- [Add your first destination](https://app.hightouch.com/destinations?from=docs)
- Learn about [Syncs](/syncs/overview) and how they deliver data into destinations
---
## Salesforce Pardot
**URL:** https://hightouch.com/docs/destinations/pardot
**Description:** Run better email campaigns on Salesforce Pardot with up-to-date customer data from your data warehouse
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| -------------------------------- | ------------------------------------------------ | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| **Prospect** | Sync Prospects to update list membership objects | Upsert, Update | [Prospects v5](https://developer.salesforce.com/docs/marketing/pardot/guide/prospect-v5.html), [Prospects v4](https://developer.salesforce.com/docs/marketing/pardot/guide/prospects-v4.html), [Prospects v3](https://developer.salesforce.com/docs/marketing/pardot/guide/prospects-v3.html) |
| **Lists** | Sync Prospects to update list membership objects | Update | [Lists v5](https://developer.salesforce.com/docs/marketing/pardot/guide/list-membership-v5.html), [Lists v4](https://developer.salesforce.com/docs/marketing/pardot/guide/list-memberships-v4.html), [Lists v3](https://developer.salesforce.com/docs/marketing/pardot/guide/list-memberships-v3.htm)|
| **Campaign Trigger** | Trigger email sends to Prospects | Update | [One-To-One](https://developer.salesforce.com/docs/marketing/pardot/guide/email-v5.html#one-to-one-email-create), [List](https://developer.salesforce.com/docs/marketing/pardot/guide/list-email-v5.html#list-email-create)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Salesforce Pardot
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button.
Select **Salesforce Pardot** and click **Continue**.
You can then authenticate Hightouch to **Salesforce Pardot** via OAuth.
Connect to Pardot via OAuth by logging into your Pardot account.
Then, input your **Business Unit ID**, which can be found in your Pardot App by navigating to **Setup > Pardot > Pardot Account Setup**.
Next, select the version of the Pardot API you wish to utilize. The v5 option is recommended for modern accounts.
Please note that syncs which have been configured for v3 / v4 are not currently cross-compatible with v5. For this reason, we recommend leaving your version *unchanged* once the destination is configured.
If your Pardot account type is **Pardot Developer Org**, input `pi.demo.pardot.com` in the optional **Pardot Custom Domain** field.
When left unset, this will default to `pi.pardot.com`.
## Sync configuration
Once you've set up your Salesforce Pardot destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data.
Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin.
Then, select the relevant model and the Salesforce Pardot destination you want to sync to.
### Syncing Prospects
#### Record matching
To match source rows to Prospects in Salesforce Pardot, you need to select a source column and corresponding Salesforce Pardot field.
Depending on the [sync mode](/syncs/types-and-modes#sync-modes) you select, you can match on the following Salesforce Pardot fields:
- **Prospect Email**
- **Prospect Salesforce ID**
- **Prospect ID**
In Pardot API v3 / v4, all of these IDs are valid first-order identifiers.
In large v3 / v4 syncs(\>=7500 rows per run), the `batchUpdate` and `batchUpsert` endpoints will be used, which enable us to send 50 records per API request.
Large v5 syncs automatically switch to the CSV processing endpoint, which is identifier-agnostic and uses very few API requests per 100,000 rows.
Pardot only processes one file at a time, so if you have many syncs which are sending CSV files at once, please know that you may experience higher sync times as Hightouch waits for Pardot's CSV queue to open up again.
You can examine your [Pardot CSV Console](https://pi.pardot.com/import/wizardStep1) (Admin -\> Import -\> Prospects) to view the CSV files that Hightouch is syncing to Pardot, as well as the Prospects which were created and updated as a result of them.
By contrast, small v5 syncs(\<7500 rows per run) do not utilize the CSV endpoint. These Prospect by Prospect endpoints require the Prospect Id to be
present in update requests and require the Prospect Email to be present in upsert requests.
Therefore:
- If you select Prospect Id to match records in small v5 upsert syncs, extra API calls will be made to look up the Prospect's email
- If you select Prospect Email to match records in small v5 update syncs, extra API calls will be made to look up the Prospect's Prospect Id
#### Field mapping
You can sync columns from your source to Salesforce Pardot's default and custom fields.
If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
Refer to the [v3 / v4 object docs](https://developer.salesforce.com/docs/marketing/pardot/guide/object-field-reference.html) or [v5 overview](https://developer.salesforce.com/docs/marketing/pardot/guide/version5overview.html) for more information about object fields.
#### Match list
You can select if you want to add the Prospects a **match list** (new or existing), or if you prefer not adding them to any list.
Read about [finding IDs](#finding-ids) if you can't find your match list.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results.
You have the following options:
| Behavior | Description |
| -------------------- | ------------------------------------------------------------------ |
| **Do nothing** | Keep the Prospect in Pardot |
| **Clear fields** | Keep the Prospect in Pardot but clear the synced fields |
| **Remove record** | Remove the Prospect from the specified List |
| **Delete record** | Delete the Prospect from Pardot entirely |
### Syncing Lists
#### Record Matching
To match source rows to lists in Salesforce Pardot, you need to select a source column and corresponding Salesforce Pardot field.
You can match on the following Salesforce Pardot fields:
- **Prospect Email**
- **Prospect ID**
All of the endpoints involved in List syncing prioritize Prospect Id as an identifier. List syncing is currently only supported on v5 of our integration.
Therefore, using Prospect Email in record matching will add aditional API requests to your sync runs.
#### CSV Mode
Here, we give you the ability to explicitly opt into or out of the CSV endpoint methodology.
As outlined in the Prospects section, this is a tradeoff where CSV processing can at times be slower but also enables bulk processing of data and a diminished API request count.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results.
You have the following options:
| Behavior | Description |
| -------------------- | ------------------------------------------------------------------ |
| **Do nothing** | Leave the Prospect on the List |
| **Remove record** | Remove the Prospect from the specified List |
### Sending Emails
To match source rows to Emails in Salesforce Pardot, you need to select a source column and corresponding Salesforce Pardot field.
You can match on the following Salesforce Pardot fields:
- **Prospect Email**
- **Prospect ID**
The notion of sync mode is less applicable here, because the Campaign Trigger sync type does not create an object in the destination.
Instead, it simply references a Prospect and the Email Template to send them. If that Prospect does not exist, the request will fail; this sync type does not enable the upserting of Prospects at the same time.
All of the endpoints involved in List syncing prioritize Prospect Id as an identifier.
Therefore, using Prospect Email in record matching will add aditional API requests to your sync runs.
Use the Email Template selector to preview your email content.
Read about [finding IDs](#finding-ids) if you can't find your Email Template.
Optionally, include the Folder to save this Sent Email in once it is processed by Pardot.
Read about [finding IDs](#finding-ids) if you can't find your Folder.
If you are sending Emails to Prospects which already exist in an updated state in Pardot, their template system will automatically fill out data in your template so long as it is formatted correctly.
#### List Mode
| Behavior | Description |
| -------------------- | ------------------------------------------------------------------ |
| **One-To-One** | As each row is processed, an email is immediately sent to that Prospect |
| **List** | Each batch of rows is assigned to an ephemeral List. An email is then triggered for all Prospects on that List. |
##### Advanced List Options
List mode creates new Prospect Lists that can have a single trigger send an email to the entire list.
Each List is constructed using 10,000 Prospects by default, but this number can be modified in the advanced options at the bottom of the form.
Selecting a lower number of Prospects will send emails more often, but also generate more Prospect Lists in Pardot to facilitate this.
##### Delete Mode
| Behavior | Description |
| -------------------- | ------------------------------------------------------------------ |
| **Do nothing** | Leave the Prospect List used to send the batch emails intact in Pardot |
| **Delete** | Delete the Lists associated with each batch after a Prospect List is sent an Email. |
If desired, Hightouch offers the ability to clean up Lists in Pardot after email campaigns are successfully triggered to them.
Therefore we also offer the ability to clean these Lists up after the email campaign is successfully triggered to them.
## Tips and troubleshooting
### Common errors
**"Invalid Campaign Id"**
This error happens when the **required Pardot campaign ID** is missing when inserting a **prospect record**. A Pardot campaign tracks the first interaction that a prospect has with your online marketing materials and functions similarly to a source field. This field is required when creating a Pardot prospect [(see Salesforce docs)](https://help.salesforce.com/s/articleView?id=mktg.pardot_campaigns_salesforce_campaigns.htm&type=5).
Pardot campaigns are not the same as Salesforce campaigns. In Pardot, each prospect has exactly one campaign **(1:1)**. In Salesforce, a lead or contact can belong to many campaigns **(1:many)**.
Because both systems use “Campaign Id,” it’s easy to confuse them. Be sure your models are pulling the correct ID. This integration currently **only accepts the *numeric* `campaignId`** from Pardot.
**"CRM Connector"**
If there is a CRM connector already present between your Salesforce account and Salesforce Pardot accounts, Pardot may explicitly disallow you from attaching `campaignId` to Prospect requests.
This is because the Prospects are already being automatically associated with campaigns based on the rules defined by your integration. To modify this behavior, consider reaching out to Salesforce Pardot support directly.
### Finding IDs
If you're trying to select an object from a dropdown in Hightouch but find that it is not present, you may need to find it manually.
Pardot's API is capable of enumerating roughly 3000 objects, but beyond this threshhold, they ask that you use a CSV export.
See Pardot's note about the `offset` parameter [here](https://developer.salesforce.com/docs/marketing/pardot/guide/version5overview.html#query-using-get).
Queries to enumerate Lists, Email Templates, Folders, and more are all met with this limitation.
Folders can only be sorted by their internal Pardot ID, but for all other objects we ask Pardot to enumerate them in the order they were most recently updated.
If you're trying to find an ID, an easy first step is to simply edit and update the object in Pardot, bringing it to the top of the dropdown in our form alongside anything else you've been working on recently.
If you have the object open in Pardot, however, you can often find its ID simply by looking at the URL which references it.
If this still proves ineffective, consider using Pardot's inbuilt CSV export tool which should be visible from any table of objects.
The exported CSV will give you data and metadata about the objects in question.
Once you have your ID, bring it back to the Hightouch sync configuration and use the "Edit as JSON" button to change the ID value manually.
### Live debugger
### Sync alerts
---
## Partnerize
**URL:** https://hightouch.com/docs/destinations/partnerize
**Description:** Attribute partner- and affiliate-driven sales in real time and commission your partners accurately, with server-to-server conversions sent straight to Partnerize.
**Section:** Destinations
## Overview
Partnerize is a global partnership and affiliate marketing platform that helps brands track conversions, attribute them to publishers, and manage commissions. With the Partnerize destination, you can sync conversion (sale) events from your data warehouse to Partnerize using its server-to-server (S2S) conversion tracking, enabling accurate affiliate attribution without relying on browser-side pixels.
Conversions are recorded against a specific Partnerize campaign and support single- or multi-item orders, basket-level attributes, and vertical-specific custom parameters.
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| --------- | ----------- | -------------------- | ------------- |
| **Events** | Sync conversion (sale) events from any source to Partnerize | Insert | [Partnerize API docs](https://api-docs.partnerize.com/partner/) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Partnerize
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Partnerize** and click **Continue**.
You need to provide the following credentials:
- **User Application Key**: The application key for your Partnerize user.
- **User API Key**: The API key for your Partnerize user.
Together these authenticate Hightouch with the Partnerize API (HTTP Basic authentication), which Hightouch uses to validate your connection and load the list of campaigns you can sync to.
### Obtain API credentials
In Partnerize, go to **Settings → Account Settings → API** to find your **User Application Key** and **User API Key**. The user associated with these keys must have access to the campaigns you want to track conversions against.
Test the connection before creating syncs. Hightouch validates your keys against the Partnerize API and confirms it can list your campaigns; if either check fails, double-check your User Application Key and User API Key.
## Sync configuration
Once you've set up your Partnerize destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Partnerize destination you want to sync to.
### Campaign
Select the **Partnerize campaign** that conversions should be tracked against. Hightouch loads the campaigns available to your account from the Partnerize API. Every conversion in the sync is recorded against the selected campaign.
### Field mapping
Hightouch lets you sync conversion fields via [field mapping](/syncs/mapping-data). You can map data from any of your model columns to Partnerize's conversion fields.
#### Required fields
You must map the following fields to complete your configuration:
- **Conversion reference (order ID)**: Your unique reference for the conversion — typically the order ID.
- **Click reference**: The click reference captured at the point of sale. It must match the originating click for Partnerize to attribute the conversion.
In addition, every conversion must include **at least one item** with a **category** and a **quantity** (see [Items](#items) below). These item-level fields are required by Partnerize's S2S specification, but they're enforced when the conversion is sent rather than as required mappings, because items can also be supplied as an array.
#### Items
A conversion's line items can be provided in two ways:
- **Single item** — map the top-level **Item category**, **Item quantity**, **Item value**, and **Item SKU** fields. This describes one line item.
- **Multiple items** — map a column containing an **array** of line items using the [inline mapper](/syncs/mapping-data#inline-mapping). Each entry takes the shape `{ category, sku, value, quantity }`. When mapped and non-empty, the items array takes precedence over the single-item fields.
For each item:
- `category` *(required)* — the item category.
- `quantity` *(required)* — the number of units.
- `value` *(optional)* — the item value, excluding tax and currency symbols. Only required when the campaign awards percentage-based commission.
- `sku` *(optional)* — the product SKU.
#### Optional basket-level fields
These describe the order as a whole and are omitted from the conversion when left unmapped:
- **Currency** — ISO 4217 currency code (for example, `USD`).
- **Country** — ISO 3166 country code (for example, `US`).
- **Customer type** — `new` or `existing`.
- **Voucher code** — the voucher or discount code used on the order.
- **Customer reference** — your own identifier for the customer.
- **Fulfilment date** — ISO 8601 `YYYY-MM-DD`; drives Partnerize's auto-approval window.
#### Custom parameters
Use [custom mappings](/syncs/mapping-data) to send additional, vertical-specific basket parameters (for example, `check_in_date` for travel). Each custom mapping is appended to the conversion as an extra parameter.
### Sync behavior
Partnerize conversions are recorded server-to-server through Partnerize's conversion tracking host. For each row in your model, Hightouch:
- Builds a conversion from the mapped fields and validates the required fields before sending.
- Sends one tracking request per conversion (requests within a batch are sent concurrently).
- Records the conversion against the selected campaign. A successful send returns `200 OK`, and the conversion then enters a `pending` state in Partnerize for downstream validation and approval.
Partnerize's conversion tracking host accepts a request and queues it for validation rather than rejecting incomplete data at send time. To keep your data clean, Hightouch validates the documented required fields — conversion reference, click reference, and per-item category and quantity — **before** sending. Rows that don't satisfy them are rejected and never sent.
## Tips and troubleshooting
### Value is optional
`value` is only required when a campaign awards percentage-based commission. A conversion with a category and quantity but no value is valid and will be sent; the value is simply omitted from the request.
### Required-field rejections
Rows are rejected (and not sent) when a required field is missing. The rejection message names the missing field:
- Missing conversion reference → *"A conversion reference (order ID) is required for every Partnerize conversion."*
- Missing click reference → *"A click reference is required to attribute a Partnerize conversion."*
- Item missing a category or quantity → *"Conversion item is missing required field(s): …"*
- No item fields mapped at all → *"Each conversion must include at least one item with a category and quantity."*
### Common errors
#### Authentication errors
If the connection test or campaign list fails with a 401 or 403:
- Verify your **User Application Key** and **User API Key** are correct and entered in the right fields.
- Confirm the user has access to the campaigns you want to track conversions against.
#### Conversions not appearing
If a sync succeeds but conversions don't appear as expected in Partnerize:
- Confirm the **click reference** matches a real click for the selected campaign — conversions without a matching click can't be attributed.
- Remember that received conversions enter a `pending` state and are validated by Partnerize after they're sent.
### Live debugger
### Sync alerts
---
## PartnerStack
**URL:** https://hightouch.com/docs/destinations/partnerstack
**Description:** Empower your partnership programs using Partnerstack with Hightouch
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-----------------|-----------------------------------------------------------------|----------------------|----------------------------------------------------------------------------------
**Customers** | Sync data from any source to PartnerStack customers | Upsert | [Customers docs](https://docs.partnerstack.com/reference/post_v2-customers-1)
**Transactions** | Sync data from any source to PartnerStack transactions | Insert | [Transactions docs](https://docs.partnerstack.com/reference/post_v2-transactions)
**Events** | Sync data from any source to PartnerStack S2S conversion events | Insert | [Events docs](https://docs.partnerstack.com/docs/server-to-server-s2s-tracking)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to PartnerStack
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **PartnerStack** and click **Continue**. You can then authenticate Hightouch to **PartnerStack**.
Enter the following fields into Hightouch:
If you want to use the PartnerStack public API, the following keys are required:
- **Public key**
- **Secret key**
If you want to send PartnerStack S2S conversion events, the following key is required:
- **S2S tracking token**
To find the above keys, navigate to your [PartnerStack Integration Settings](https://app.partnerstack.com/settings/integrations).
## Sync configuration
Once you've set up your PartnerStack destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the PartnerStack destination you want to sync to.
### Syncing customers
Sync data from any source to PartnerStack customers.
#### Record matching
You can match rows from your model to customers in PartnerStack on any column in your model and any field in PartnerStack. Ensure the data types of the model column and PartnerStack field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync customer fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom customer fields.
PartnerStack requires the following fields, so you must map them to complete your configuration:
- **Email**
- **Partner Key**
- **Customer Key**
If you send data for a custom field that doesn't exist, Hightouch adds the field.
Ensure your model's columns have the same data types as the fields you want to sync to.
### Syncing transactions
Sync data from any source to PartnerStack transactions.
#### Record matching
You can match rows from your model to transactions in PartnerStack on any column in your model and any field in PartnerStack. Ensure the data types of the model column and PartnerStack field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, PartnerStack automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync transaction fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default transaction fields.
PartnerStack requires the following fields, so you must map them to complete your configuration:
- **Customer Key**
- **Transaction Key**
Ensure your model's columns have the same data types as the fields you want to sync to.
### Syncing events
Sync data from any source to PartnerStack S2S conversion events.
#### Record matching
You can match rows from your model to events in PartnerStack on any column in your model and any field in PartnerStack. Ensure the data types of the model column and PartnerStack field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Pendo
**URL:** https://hightouch.com/docs/destinations/pendo
**Description:** Perform deeper analysis and better understand user behavior by bringing enriched customer data into Pendo from your data warehouse
**Section:** Destinations
## Overview
Ideally, your data warehouse should contain all your customer data. When that's the case, using your warehouse as the source for your Pendo data enables richer, more granular analysis. You can sync enriched data from your warehouse and data from other engagement tools. Moreover, you can ensure that the data in Pendo consumed by your product and growth teams is always up-to-date and in the expected format.
If your company also uses a Business Intelligence tool such as Looker or Mode alongside Pendo, you can ensure that the data is consistent across both systems since your data warehouse is the only true source of data.
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| ----------- | ------------------------------------------------------------------ | -------------------- | --------------------------------------------------------------- |
| **Events** | Sync records as events to Pendo, often in the form of a track call | Insert | [Track API reference](https://engageapi.pendo.io) |
| **Objects** | Update visitor, account, and parent account objects | Update | [API overview](https://engageapi.pendo.io) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Pendo
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Pendo** and click **Continue**. You can then authenticate Hightouch to Pendo with an **Integration Key** and **Track Event Shared Secret**.
{/* */}
### Create Pendo Integration Key
{/* */}
In your [Pendo admin](https://app.pendo.io/admin/) go to **Settings** > **Integrations** > **Integration Keys**. Then follow the instructions below:
1. Select **+ Add Integration Key**.
2. Enter a meaningful **description** for your Integration Key, for example, "Hightouch Integration."
3. Provide the key **Allow writes** access.
4. Select **Create**. Securely store this key for use in Hightouch.
For more information about Integration Keys, refer to Pendo's [Integration Key article](https://support.pendo.io/hc/en-us/articles/9491198203547-Pendo-Integration-Key#requirements-0-0).
{/* */}
### Access Track Event Shared Secret
{/* */}
To access your Track Event Shared Secret go to **Subscription Settings** > **Choose your App** > **App Details** in your [Pendo admin](https://app.pendo.io/admin/). You must be a Pendo Admin to have access to this Secret Key. You can find more information in Pendo's [events tracking documentation](https://support.pendo.io/hc/en-us/articles/360032294291-Track-Events-Configuration#segment-com-0-5).
## Sync configuration
Once you've set up your Pendo destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Pendo destination you want to sync to.
### Events
The Pendo integration lets you send events to Pendo with custom properties. You must input the following information:
- **Event name**: either a model column containing the event name or a static value.
- **Timestamp**: a model column containing the time the event occurred; if you leave this empty, Pendo uses the time the event arrives at the server.
#### Field mapping
Pendo suggests that you always supply both a Visitor ID (required) and Account ID (strongly recommended).
In addition, you can choose to map additional fields to your Pendo event payload in the custom properties.
### Objects
The Pendo integration allows users to update existing visitors, accounts, and parent accounts with metadata—both agent and custom fields.
#### Record matching
You have to choose how to match records from your model to objects in Pendo. Select a column from your model that corresponds to your Pendo profile ID. The following example screenshot shows mapping the model's **id** column to Pendo's **Visitor ID** field.
If users aren't found in Pendo, Hightouch automatically skips the row.
#### Field mapping
Pendo has two field types:
- **Agent fields** are API-created fields the agent integrated on your website or app.
- **Custom fields** are editable in the UI and through the API,
You can choose which fields you want to update on your profiles in Pendo. The following example screenshot shows mapping the model's **firstname** column to the **firstname** field in Pendo.
You cannot create agent fields in the Pendo UI. If you want to write to a new field in Pendo that doesn't exist, you can pass in the desired field's name in Hightouch. The option to create that new field will appear, and if you select it, the field will be created during the next sync.
You can create custom fields in Pendo by going to [**Settings** > **Data Mappings**](https://app.pendo.io/admin/mappings).
#### Split retries
Pendo counts all events in a batch as rejected if the request contains a single invalid event.
To pinpoint which records are getting rejected with which errors and reduce the number of valid records that get retried,
you can enable [**split retries**](/syncs/retries#split-retries).
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ---------------------------------------------------------- |
| **Do nothing** | Keep the document in Pendo |
| **Delete** | Remove the document from Pendo |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Pinterest Ads
**URL:** https://hightouch.com/docs/destinations/pinterest-ads
**Description:** Deliver better ads by syncing conversion events and customer list audiences to Pinterest Ads
**Section:** Destinations
## Overview
Manage your conversion tracking by syncing conversion events and promote your pins to a specific Pinterest audience by syncing customer lists from your data warehouse. Deliver more relevant ads, boost marketing ROI, and retarget your most loyal customers.
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| --------------------------- | --------------------------------------------------------------------------- | -------------------- | ----------------------------------------------------------------------------------------- |
| [**Conversion Events**](#syncing-conversion-events) | Sync conversion events from any source to Pinterest Ads | Insert | [Events API](https://developers.pinterest.com/docs/api/v5/events-create/) |
| [**Customer List Audiences**](#syncing-customer-list-audiences) | Sync your PII data from any source to Pinterest Ads customer list audiences | Add, Remove | [Customer Lists API](https://developers.pinterest.com/docs/api/v5/customer_lists-create/) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Pinterest Ads
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Pinterest Ads** and click **Continue**. You can then authenticate Hightouch to Pinterest Ads by selecting **Log in to Pinterest Ads** and logging into your Pinterest Ads account.
Once successful, you will be redirected back to Hightouch to enter your **advertiser account ID** and **conversion access token**.
- **Advertiser account ID**: You can locate your advertiser account ID within your Pinterest business account by navigating to
the dropdown button on the top right of the page, then **Business Access** > **Ad Accounts**. You can find your ID under your the **Asset name**.
- **Conversion access token**: If you want to send conversion events to Pinterest Ads, you also need to input a conversion access token. You can create a conversion access token within your Pinterest business account by navigating to the **Ads** dropdown then **Conversions** > **Conversion access token**. Click **Generate new token** and enter the token in Hightouch.
## Sync configuration
Once you've set up your Pinterest Ads destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Pinterest Ads destination you want to sync to.
### Syncing conversion events
Hightouch supports syncing conversion events to Pinterest Ads.
#### Record matching
You can match events from your source to Pinterest Ads using the Event ID.
#### Field mapping
You can use [field mapping](/syncs/mapping-data) to sync data to the event properties of a Pinterest Ads conversion event. Refer to the [Pinterest Ads developer docs](https://developers.pinterest.com/docs/api/v5/#operation/events/create) for descriptions of each property.
Ensure your model columns' data types match the data types of the fields you want to sync to.
For fields that expect an array , Hightouch automatically converts your row values to arrays if they aren't already. For example, you can map [string model columns](/models/creating-models#data-types-and-casting) to array type Pinterest Ads fields.
The Pinterest Ads API requires the **Action Source** field, so you must map it to complete your configuration.
**Action Source** is an enumerated field which only accepts the following `string` values:
- "app_android"
- "app_ios"
- "web"
- "offline"
#### Handling PII
Pinterest Ads requires some identity information to be normalized and hashed. We support hashing for all fields that require values to be SHA256 hashed. Disabling PII hashing requires you to do this yourself.
If you're handling hashing yourself, make sure that email addresses are in lowercase before hashing, as explained in [Pinterest's documentation](https://developers.pinterest.com/docs/api/v5/#operation/events/create).
#### Cleaning phone numbers
You can select whether Hightouch should clean your phone numbers. By default, Hightouch doesn't clean phone numbers.
Pinterest Ads requires users' phone numbers to be normalized and hashed. The phone number should only include digits with country code, area code, and number without any symbols, letters, spaces, and leading zeros.
If you enable cleaning phone numbers, Hightouch tries to convert the phone number into E.164 format and remove the leading plus (`+`) symbol.
#### Using the test events endpoint
Pinterest Ads provides a [test events endpoint](https://help.pinterest.com/en/business/article/validating-your-set-up-with-event-testing) to validate your set up for conversion events. This endpoint does not affect
production conversion data, and returns warnings and recommendations based on your set up. Syncs can be run using this endpoint and results can be seen by clicking on **Test Events** under the the **Conversions Tab**.
### Syncing customer list audiences
A **customer list audience** is a [customer list that is added to an audience](https://help.pinterest.com/en/business/article/audience-targeting).
Depending on your model query results, Hightouch adds new user records into the Pinterest Ads customer list audience or removes them.
#### User identifiers
To identify which users to add or update in an audience, select model columns and the corresponding Pinterest Ads user fields. You can match on any of the following Pinterest Ads user fields:
- Email
- IDFA
- MAID
- LR_ID
- DLX_ID
- Hashed Pinner ID
Emails must be lowercase and can be plain text or hashed using SHA1, SHA256,
or MD5. MAIDs and IDFAs must be hashed with SHA1, SHA256, or MD5.
#### New audience creation
In your sync configuration, you can tell Hightouch to create a new customer list audience for you. You can optionally assign a custom name to your audience. If no name is provided, Hightouch defaults to using your model name.
To help differentiate the name of your customer list and audience, we save the customer list name as `customer list-{customer list name}` and audience name as `audience-{audience name}`.
The following example screenshot would create a new customer audience list called "Users 1."
After creation, the customer list name will be "customer list-Users 1," and the audience name will be "audience-Users 1."
#### Using an existing audience
Instead of creating a new data customer list audience, Hightouch can also sync users to existing audiences that are already populated in your Pinterest Ads account.
The Hightouch UI shows the customer list and audience as a comma-separated string.
In the example screenshot below, the customer list audience shows as "customer list-Users 1, audience-Users 1."
#### Field mapping
You can sync columns from your model to the following identifiers in Pinterest:
- Email
- IDFA
- MAID
- LR_ID
- DLX_ID
- Hashed Pinner ID
Hightouch [Match Booster](/match-booster/overview) can help to provide additional identifiers and increase your match rates.
#### Handling PII
Pinterest Ads requires some identity information to be normalized and hashed. Hightouch supports hashing for all user types except for **Email**, since Pinterest Ads doesn't require hashing this field.
Enable PII hashing if you wish for Hightouch to normalize and hash PII fields using a SHA256 hash. Disabling PII hashing requires to do this yourself.
## Tips and troubleshooting
### Matched users count
Below only applies to the audiences sync type.
### Common errors
#### Delayed syncing
Pinterest takes around 24 hours to fully sync after Hightouch has synced data to their API. You can find more details under **customer list** on [Pinterest's audience targeting help page](https://help.pinterest.com/en/business/article/audience-targeting).
#### Smaller audience size
You may notice that the audience size may be smaller than the total number of records synced to Pinterest Ads. This is because [Pinterest filters out
records that don't have a Pinterest account](https://developers.pinterest.com/docs/api/v5/#tag/customer_lists).
#### 400 - Invalid request
You may receive a `400 - {"code":1,"message":"Invalid request: {} is not valid under any of the given schemas"}` error message because a row being sent to Pinterest contains a `null` or blank value.
Ensure that mapped fields do not contain any`null` or blank values. You can resolve this error by:
- Filtering these rows in the model's query
- Selecting the [Don't sync null values](https://hightouch.com/docs/syncs/mapping-data#dont-sync-null-values) checkbox in of the field mapping configurations.
### Live debugger
### Sync alerts
---
## Pipedrive
**URL:** https://hightouch.com/docs/destinations/pipedrive
**Description:** Enable your sales team to engage with prospects better by syncing enriched data from your warehouse to Pipedrive CRM
**Section:** Destinations
## Overview
Your leads and prospects expect you to have the necessary context before engaging with them. You can accomplish this by syncing lead and prospect data from your data warehouse to Pipedrive CRM. You no longer need to worry about executives chasing the wrong leads or engaging with warm ones without proper context.
## Supported syncing
|Sync Type|Description|Supported Sync Modes | API Reference|
|--|--|--|--|
|**Organization**|Sync data from any source to organizations|Upsert, Update| [Organization docs](https://developers.pipedrive.com/docs/api/v1/Organizations/)|
|**Person**|Sync data from any source to person objects|Upsert, Update|[Person docs](https://developers.pipedrive.com/docs/api/v1/Persons#getPersonActivities)|
|**Deal**|Sync data from any source to deals|Upsert, Update|[Deal docs](https://developers.pipedrive.com/docs/api/v1/Deals)|
|**Lead**|Sync data from any source to leads|Upsert, Update| [Lead docs](https://developers.pipedrive.com/docs/api/v1/Leads)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Pipedrive
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Pipedrive** and click **Continue**. You can then authenticate Hightouch to Pipedrive with an API key.
You can retrieve your Pipedrive API Key under **Settings > Personal preferences > API**.
For more information, see [Pipedrive's docs](https://support.pipedrive.com/en/article/how-can-i-find-my-personal-api-key).
## Sync configuration
Once you've set up your Pipedrive destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Pipedrive destination you want to sync to.
### Record matching
You can match rows in your model with objects in Pipedrive on any unique property. Pipedrive's `id` field is only available in the dropdown menu when using **Update** mode.
To match Deal objects, the [Pipedrive
API](https://developers.pipedrive.com/docs/api/v1/Deals#searchDeals) requires
a search term that is *not* the Deal ID. Available search terms
pre-populate your Deal matching field drop-down.
### Field mapping
You can sync columns from your source to Pipedrive default and custom fields.
### Delete behavior
| Behavior | Description |
| -------------- | -------------------------------------------------------------------- |
| **Do nothing** | Keep the record in Pipedrive |
| **Delete** | Remove the record from Pipedrive entirely |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## PlanetScale
**URL:** https://hightouch.com/docs/destinations/planetscale
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Supported syncing
|Sync Type|Description |Supported Sync Modes |
|-----------|--------------------------------------------------|---------------------|
| **Table** |Sync data from any source to a PlanetScale table | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to PlanetScale
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **PlanetScale** and click **Continue**.
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your PlanetScale server.
- **Port**: The port number of your PlanetScale server. The default is 3306, but yours may be different.
- **Database**: This specifies the database to use when Hightouch executes queries in Planetbase. This is different than the host, but your host address might contain your database name.
- **User**: This can be your personal PlanetScale login or a dedicated user for Hightouch.
- **Password**: The password for the user specified above.
## Sync configuration
Once you've set up your PlanetScale destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the PlanetScale destination you want to sync to.
### Select table
Select the existing table in PlanetScale you want to sync to.
### Record matching
You can match rows from your model to rows in PlanetScale on any column as long as it is a primary key or unique key.
You can see columns that fit this criteria as available options in records matching section.
If there are no fields in the dropdown, you must add a unique type column to your PlanetScale table. Then, click the refresh icon to access the newly created column.
### Field mapping
Select which model columns you want to sync to your PlanetScale table columns. Hightouch automatically pulls the columns from your table to make them available for you to map.
When syncing to [binary](https://dev.mysql.com/doc/refman/8.0/en/binary-varbinary.html) destination fields, Hightouch assumes the model column values to be [base64](https://en.wikipedia.org/wiki/Base64) strings. If they aren't, you can use [template mapping](/syncs/mapping-data#template-mapping) to [cast](/models/data-types-casting#casting) them if needed.
### Batch size
You can tune the number of rows Hightouch upserts per query based on your needs and database threshold. The default is 1000 rows per batch.
If you want to increase the sync's speed, you can increase the batch size. Keep in mind that PlanetScale fails the entire batch of rows if it detects any erroneous row. If you suspect that you will have many bad rows, don't use a high batch size. To avoid locks, ensure you account for your database's capacity when increasing the batch size.
### Delete behavior
When rows no longer shows up from your model's query results, you can configure the sync to **Do nothing** or **Delete** those rows from the PlanetScale.
| Behavior | Description |
| -------------------- | --------------------------------------------------- |
| **Do nothing** | Keep the synced rows in the table in PlanetScale |
| **Delete** | Delete the synced rows from the table in PlanetScale|
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Planhat
**URL:** https://hightouch.com/docs/destinations/planhat
**Description:** Sync real-time analytics and enrich customer data in Planhat to optimize your customer management and growth
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ------------------- | ---------------------------------------------------- | ---------------------- | ------------------------------------------------------------------- |
| **Assets** | Sync data from any source to Planhat assets | Upsert, Update, Insert | [Assets endpoint](https://docs.planhat.com/#assets) |
| **Campaigns** | Sync data from any source to Planhat campaigns | Upsert, Update, Insert | [Campaigns endpoint](https://docs.planhat.com/#campaigns) |
| **Company** | Sync data from any source to Planhat company | Upsert, Update, Insert | [Company endpoint](https://docs.planhat.com/#companies) |
| **Conversations** | Sync data from any source to Planhat conversations | Upsert, Update, Insert | [Conversations endpoint](https://docs.planhat.com/#conversations) |
| **Endusers** | Sync data from any source to Planhat endusers | Upsert, Update, Insert | [Endusers endpoint](https://docs.planhat.com/#endusers) |
| **Invoices** | Sync data from any source to Planhat invoices | Upsert, Update, Insert | [Invoices endpoint](https://docs.planhat.com/#invoices) |
| **Issues** | Sync data from any source to Planhat issues | Upsert, Update, Insert | [Issues endpoint](https://docs.planhat.com/#issues) |
| **Licenses** | Sync data from any source to Planhat licenses | Upsert, Update, Insert | [Licenses endpoint](https://docs.planhat.com/#licenses) |
| **Metrics** | Sync data from any source to Planhat metrics | Insert | [Metrics endpoint](https://docs.planhat.com/#bulkinsert_metrics) |
| **Notes** | Sync data from any source to Planhat notes | Upsert, Update, Insert | [Notes endpoint](https://docs.planhat.com/#notes) |
| **Opportunities** | Sync data from any source to Planhat opportunities | Upsert, Update, Insert | [Opportunities endpoint](https://docs.planhat.com/#opportunities) |
| **Projects** | Sync data from any source to Planhat projects | Upsert, Update, Insert | [Projects endpoint](https://docs.planhat.com/#projects) |
| **Sales** | Sync data from any source to Planhat sales | Upsert, Update, Insert | [Sales endpoint](https://docs.planhat.com/#sales) |
| **User Activities** | Sync data from any source to Planhat user activities | Insert | [User Activities endpoint](https://docs.planhat.com/#activity_bulk) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Planhat
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Planhat** and click **Continue**. You can then authenticate Hightouch to **Planhat** by entering the following required fields into Hightouch:
- **API Access Token**: You can generate an API token under the **Settings > Service Accounts** section in Planhat.
- **Tenant UUID**: You only need to provide a Tenant UUID if you plan to sync to **Metrics** or **User Activities**. You can get your Tenant UUID under in the **Developer** section in Planhat.
- **Region**: You only need to provide a region if your Planhat app subdomain isn't `app`. For example, if your app URL is `app-us1.planhat.com`, then enter `us1`. If your app URL is `app.planhat.com`, leave the region blank.
## Sync configuration
Once you've set up your Planhat destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Planhat destination you want to sync to.
### Syncing companies
Companies or accounts are your customers.
Depending on your business these might be agencies, schools, other businesses or something else.
Companies can also be your previous customers and or prospects.
#### Record matching
To match rows from your model to company in Planhat, you need to select a model column and corresponding Planhat field. You can match on any of the following Planhat fields:
- **ID**
- **Source ID**
- **External ID**
In **Insert** mode, Planhat automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync company fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom company fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available on **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
| Behavior | Description |
| -------------- | ------------------------------------------------------------ |
| **Do nothing** | Keep the company in Planhat with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the company in Planhat |
| **Delete** | Delete the synced companys from Planhat |
### Syncing endusers
An enduser represents an individual at one of your customers, typically a user of your product, a business contact or both.
#### Record matching
To match rows from your model to enduser in Planhat, you need to select a model column and corresponding Planhat field. You can match on any of the following Planhat fields:
- **ID**
- **Source ID**
- **External ID**
In **Insert** mode, Planhat automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync enduser fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom enduser fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available on **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
| Behavior | Description |
| -------------- | ------------------------------------------------------------ |
| **Do nothing** | Keep the enduser in Planhat with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the enduser in Planhat |
| **Delete** | Delete the synced endusers from Planhat |
### Syncing sales
Sales represents not recurring revenue (NRR), like an onboarding fee, or a one-off professional services project.
#### Record matching
To match rows from your model to sale in Planhat, you need to select a model column and corresponding Planhat field. You can match on any of the following Planhat fields:
- **ID**
- **Source ID**
- **External ID**
In **Insert** mode, Planhat automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync sale fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom sale fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available on **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
| Behavior | Description |
| -------------- | --------------------------------------------------------- |
| **Do nothing** | Keep the sale in Planhat with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the sale in Planhat |
| **Delete** | Delete the synced sales from Planhat |
### Syncing invoices
Planhat automatically generates invoices when a license is created or renewed.
Invoices can include multiple line items.
#### Record matching
To match rows from your model to invoice in Planhat, you need to select a model column and corresponding Planhat field. You can match on any of the following Planhat fields:
- **ID**
- **Source ID**
- **External ID**
In **Insert** mode, Planhat automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync invoice fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom invoice fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available on **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
| Behavior | Description |
| -------------- | ------------------------------------------------------------ |
| **Do nothing** | Keep the invoice in Planhat with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the invoice in Planhat |
| **Delete** | Delete the synced invoices from Planhat |
### Syncing licenses
Licenses represent your customers' subcriptions to your service and is the base for MRR or ARR calculations and most revenue reports.
#### Record matching
To match rows from your model to license in Planhat, you need to select a model column and corresponding Planhat field. You can match on any of the following Planhat fields:
- **ID**
- **Source ID**
- **External ID**
In **Insert** mode, Planhat automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync license fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom license fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available on **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
| Behavior | Description |
| -------------- | ------------------------------------------------------------ |
| **Do nothing** | Keep the license in Planhat with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the license in Planhat |
| **Delete** | Delete the synced licenses from Planhat |
### Syncing metrics
Syncing metrics lets you track dimensional data in Planhat.
While [user activity](#syncing-user-activites) says a lot about user engagement, it doesn't always reflect the value created.
Dimension data is a set of model level metrics (Company by default) to understand how well your customers are doing, and the value they get out of your service.
#### Field mapping
Hightouch lets you sync event properties via field mapping. For more information on event properties you can sync into, visit [Planhat's docs](https://docs.planhat.com/#bulkinsert_metrics).
### Syncing user activities
Syncing user activites lets you track endusers's actions in Planhat.
Keeping track of your endusers activity is an important part of the Customer Success effort.
#### Field mapping
Hightouch lets you sync event properties via field mapping. For more information on event properties you can sync into, visit [Planhat's docs](https://docs.planhat.com/#activity_bulk).
## Tips and troubleshooting
### Failed user activities
When receiving [user activities](#syncing-user-activities), Planhat's API always returns an `OK` successful response even if the user activity payload contains invalid data.
You can find these failed user events in Planhat in **Customer Intelligence > Trends & Analytics > User Activities** by clicking the **Show failed activities** icon.
### Common errors
### Live debugger
### Sync alerts
---
## Podscribe
**URL:** https://hightouch.com/docs/destinations/podscribe
**Description:** Simple, powerful podcast and audio ad analytics + tools that help everyone make better decisions.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-----------|-----------------------------------------------|----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
**Events** | Sync data from any source to Podscribe events | Insert | [Events docs](https://help.podscribe.com/sending-podscribe-conversion-events-/g4g8w3hppouCfkXZdcx6FV/sending-events-server-to-server-s2s-in-batch-or-as-an-img-tag/67UdckfSLHXAPtUAfWDyud)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Podscribe
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Podscribe** and click **Continue**. You can then authenticate Hightouch to **Podscribe**.
Enter the following fields into Hightouch:
- **Advertiser ID**
- **User ID**
## Sync configuration
Once you've set up your Podscribe destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Podscribe destination you want to sync to.
### Syncing events
Sync data from any source to Podscribe events.
#### Record matching
You can match rows from your model to events in Podscribe on any column in your model and any field in Podscribe. Ensure the data types of the model column and Podscribe field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Poplar
**URL:** https://hightouch.com/docs/destinations/poplar
**Description:** Keep your Poplar audiences and Do Not Mail list up-to-date and trigger campaigns with data from your warehouse
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
----------------------|-------------------------------------------------------|---------------------
**Segment** | Add members to existing Poplar segments | Add
**Campaign Trigger** | Trigger mail campaigns with context from your records | Add
**Suppression Lists** | Add members to Poplar's Do Not Mail list | Add
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Poplar
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination**
button. Select **Poplar** and click **Continue**. You can then authenticate Hightouch to **Poplar** by entering a Poplar **API key**.
To access your API keys, navigate to **Integrations** > **API** in Poplar. If you want to create a new one, click **Generate Access Token**.
We recommend using a **Production** mode API key. If you use a **Test** mode API key, you can only use the Campaign Trigger sync type due to Poplar API limitations. To learn more about Poplar API keys, check out [Poplar's docs](https://docs.heypoplar.com/article/246-test-production-tokens).
## Sync configuration
Once you've set up your Poplar destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and
the Poplar destination you want to sync to.
### Segment
Hightouch supports adding customer records based on rows from your model to any of your Poplar audiences.
#### Record matching
You can match rows from your model to customer records in Poplar on the fields:
- **Identifier**
- **Email**
- **Hashed Email**
- **Address**
### Campaign trigger
Hightouch lets you use records from your models to trigger mail campaigns. This sync triggers a campaign for your Poplar customers based on the recipient's information.
Once configured, your Hightouch sync triggers your campaign once rows appear in your model. Hightouch triggers your campaign for every row in your model the first time you run the sync, whenever you do a full resync, and for every new row added to your query results in subsequent syncs.
#### Creative selection
You can optionally provide a Poplar **creative ID** for the campaign you want to trigger. If you don't enter a creative ID, Poplar uses the default creative. If you haven't sent a default creative, Poplar alternates between all active creatives.
#### Field mapping
You can sync columns from your source to the following native columns in a Poplar campaign trigger:
- **Email**
- **Company**
- **First Name**
- **Last Name**
- **Address 1**
- **Address 2**
- **City**
- **State**
- **Postal Code**
- **Identifier**
- **Address**
You can also map model columns to use in [Poplar merge tags](https://docs.heypoplar.com/article/203-merge-tags).
This destination doesn't check for duplicates. Hightouch recommends that you set [suppression control](https://docs.heypoplar.com/article/204-suppression-settings) in Poplar to prevent duplicate campaign messages in case your model has duplicates.
#### Testing
You can test your campaign trigger sync by following these steps:
1. Use a **Test** mode API key within your destination configuration.
2. Set up your campaign trigger sync and run it on Hightouch.
3. Download mailing data by navigating to **Integrations** > **Mailing Data** in Poplar.
4. Verify that the trigger was set off by comparing the synced records with mailing data.
If you need to make corrections and thus want to re-run the sync, click **Resync full query** next to the **Run** button.
### Suppression lists
Hightouch supports syncing rows from your model to customer records in your Poplar **Do Not Mail** list. Members of this list are automatically suppressed across all campaigns.
#### Field mapping
You can add customer records to the Poplar' **Do Not Mail** using these fields:
- **Identifier**
- **Email**
- **Hashed Email**
- **Address**
You can select one or more of the identifiers for a single record.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## PostgreSQL
**URL:** https://hightouch.com/docs/destinations/postgres
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
The minimum supported PostgreSQL version is `9.5` since Hightouch relies on [`ON CONFLICT`](https://www.postgresql.org/docs/current/sql-insert.html) for updating rows.
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|Any data set|Sync data from any source to a PostgreSQL table|Upsert, Insert|
## Connect to PostgreSQL
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **PostgreSQL** and click **Continue**. If you're not using a [tunnel](#tunneling), you can then authenticate Hightouch to PostgreSQL by entering the following fields:
- **Host**: The hostname or IP address of your PostgreSQL server. You don't need to include the `https://`.
- **Port**: The port number of your PostgreSQL server. The default port number is 5432, but yours may be different.
- **Database**: This specifies the database to use when Hightouch executes queries in PostgreSQL. This is different than the host, but your host address might contain your database name.
- **User**: The user that has access through Hightouch to the database and tables you want to sync to. It's best to [create a new user](https://www.postgresql.org/docs/8.0/sql-createuser.html) specifically for Hightouch access. _Do not use the root user._
- **Password**: The password for the user specified above.
To ensure your [credentials](#required-permissions) are correct, click **Test connection**. This confirms if Hightouch can connect to your database by running a basic `SELECT` query.
### Required permissions
The user you use to authenticate must have the following permissions:
- Add, update, and delete ([if applicable](#delete-behavior)) rows in your sync's table.
- View the following tables which are used for gathering metadata to set up the sync:
- `INFORMATION_SCHEMA.COLUMNS`
- `INFORMATION_SCHEMA.TABLE_CONSTRAINTS`
- `INFORMATION_SCHEMA.CONSTRAINT_COLUMN_USAGE`
### SSH tunneling
Hightouch can connect directly to PostgreSQL over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your PostgreSQL instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
## Sync configuration
Once you've set up your PostgreSQL destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the PostgreSQL destination you want to sync to.
Hightouch supports **Upsert** mode, with the option to [delete removed rows](#delete-behavior), and **Insert** mode using the [`COPY FROM STDIN`](https://www.postgresql.org/docs/current/sql-copy.html)statement. This statement loads queried rows into your table faster than upsert mode. However, using this statement, PostgreSQL rejects the entire batch if any row contains a primary key or unique value that already exists.
Use insert mode if you are _only_ inserting rows and want to sync your data faster.
### Record matching
Hightouch requires you select a unique identifier in the table you are syncing to. The model column you select must match a `UNIQUE`, `PRIMARY KEY` column within PostgreSQL. If a field that satisfies these conditions still doesn't appear in the dropdown menu, ensure that you assigned the [required permissions](#required-permissions) to the Postgres user.
You can see columns that fit this criteria as available options in record matching section.
If there are no fields in the dropdown, you must add a unique type column to your PostgreSQL table. Then, click the refresh icon to access the newly created column.
### Column types
Hightouch works out of the box with all standard column types, including:
- `BIGINT`
- `INT`
- `TEXT`
- `VARCHAR`
- `TIMESTAMPTZ`
- `BOOLEAN`
- `DECIMAL`
If you see type errors, it may be because your model is producing the wrong format. If so, use [typecasting](/models/data-types-casting) to resolve the issue.
#### Array support
Hightouch supports arrays of standard JSON objects (JSON or JSONB) but doesn't support arrays of [`USER-DEFINED` type JSON objects](https://www.postgresql.org/docs/current/xtypes.html).
Because **insert** mode uses the `COPY FROM STDIN` statement, it doesn't support the array of JSONB column type (`jsonb[]`). Instead, use a JSONB column type, which natively supports arrays.
### Field mapping
You can sync columns from your model to PostgreSQL fields.
### Batch size and sleep interval
You can tune the number of rows Hightouch upserts or inserts per query based on your needs and database threshold. The default is 1000 rows per batch. You can also optionally enter a custom sleep interval in milliseconds (ms) between batches. This default is 100 ms.
If you want to increase the sync's speed, you can increase the batch size and lower the sleep interval. Keep in mind that PostgreSQL fails the entire batch of rows if it detects any erroneous row. If you suspect that you will have many bad rows, don't use a high batch size. To avoid locks, ensure you account for your database's capacity when increasing the batch size.
### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | --------------------------------------- |
| **Do nothing** | Keep the row in PostgreSQL |
| **Delete row** | Remove the row from PostgreSQL entirely |
## Tips and troubleshooting
### Common errors
#### Canceling statement due to statement timeout
This error occurs when the required execution time of your PostgreSQL query exceeds the timeout limit for the database.
To address the error, increase the timeout by executing the following query:
```json
set statement_timeout = '300 s'; -- 300 seconds, 5 minutes
```
Be sure to adjust the time to be as long as it takes for your query to execute.
### Live debugger
### Sync alerts
---
## PostHog
**URL:** https://hightouch.com/docs/destinations/posthog
**Description:** Push user behavior data from your warehouse into PostHog
**Section:** Destinations
## Setup
After selecting PostHog from our destinations catalog, you will be required to enter your PostHog [_Project API Key_](https://posthog.com/docs/api/post-only-endpoints) and _Public URL_.
- You can locate your Project API key within your PostHog account by navigating to _your PostHog project_ > _Project Settings_ > _Project API key_.
- If you are using the cloud-hosted version of PostHog, your Public URL will be either:
- For US: [https://us.i.posthog.com/batch/](https://us.i.posthog.com/batch/)
- For EMEA: [https://eu.i.posthog.com/batch/](https://eu.i.posthog.com/batch/)
- If you are using a self-hosted version of PostHog, replace [https://app.posthog.com/](https://app.posthog.com/) with the URL of your instance.
Hightouch supports syncing both `Objects` and `Events` to PostHog.
## Syncing
Hightouch supports syncing to the following PostHog objects:
- `People` (`Persons`)
### Sync modes
Hightouch supports the following sync modes:
- **Upsert** - pushes new `Person` objects to PostHog and updates properties that change in your warehouse.
### Record matching
`Person` records can be matched from your source to your PostHog workspace by Distinct ID (`distinct_id`), which is a unique identifier
in PostHog for each `Person`.
### Field mapping
You may also sync columns from your source to PostHog's `Person` properties. PostHog doesn't provide information on properties set for `People` objects,
so you will need to type the exact name of the property you would like to map to in PostHog. These property names must be spelled correctly and are case-sensitive.
## Sending events to PostHog
The PostHog integration also allows you to send events to PostHog with custom properties.
The four elements of any PostHog event are the _Event Name_ (**required**, which can be a single string used to name all events, or pulled from your warehouse per event), a _Timestamp_ (**optional**, in ISO 1806 format, or a Unix timestamp in **milliseconds**), a **Distinct ID** (**required**, the unique ID of the user corresponding to the event), and event properties.
You can also map properties that will become event `properties` in PostHog, just as you would for object properties.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## PubMatic
**URL:** https://hightouch.com/docs/destinations/pubmatic
**Description:** Drive measurable advertising performance with PubMatic's AI-powered, agentic platform — built for buyers, publishers, and commerce media across CTV and mobile.
**Section:** Destinations
PubMatic syncs upload the users in an audience segment to PubMatic over SFTP so you can target them in advertising campaigns. Each sync registers your segment with PubMatic automatically, then uploads the matching users — there's no separate taxonomy sync to schedule ahead of time.
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| ------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | -------------------- | --------------------------------------------------------------------------------------------------------- |
| [**Audience data**](#audience-data) | Upload user records mapped to a PubMatic segment as a gzipped TSV file. | Incremental | [Data file format](https://help.pubmatic.com/publisher/docs/data-partner-integration-guide-for-audiences) |
| [**Hashed email (HEM) audience**](#hashed-email-hem-audience) | Upload users keyed on hashed emails. Hightouch hashes raw emails or passes through already-hashed values. | Incremental | [Data file format](https://help.pubmatic.com/publisher/docs/data-partner-integration-guide-for-audiences) |
Both sync types send incrementally: each run uploads only the users added to or removed from your query result since the last run, as separate `addseg` and `removeseg` files. For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to PubMatic
To connect PubMatic to Hightouch you need an SFTP account and a Data Partner ID (DPID), both provided by your PubMatic Account Manager during onboarding. The onboarding email includes the SFTP hostname, username, password, DPID, and the directories to upload to.
1. Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and select **Add destination**.
2. Select **PubMatic** from the destination catalog.
3. Enter the following credentials, then select **Continue**:
- **SFTP Host**: the regional PubMatic SFTP hostname from your onboarding email. Use the region closest to your user base for fastest ingestion.
- **SFTP Port**: defaults to `22`.
- **Username** and **Password**: your PubMatic SFTP credentials.
- **Data Partner ID (DPID)**: your PubMatic-assigned data partner ID.
- **Data File Path**: the directory where audience data files should land, typically `Reports`.
- **Taxonomy File Path**: the directory where segment registration files should land, typically `taxonomy`. This must be different from the Data File Path — PubMatic separates the two file types by folder.
- **HEM Data File Path**: the directory where hashed email files should land, typically `hem`. PubMatic requires HEM data in its own directory. Fill this in only if you run Hashed email (HEM) audience syncs.
PubMatic chroots your SFTP user, so enter each path relative to your
account's home directory. Use a short name like `Reports` or `taxonomy`,
not a full server path like `/home//Reports/`.
Contact your PubMatic Account Manager before sending a segment for the first
time. PubMatic typically reviews the initial submission to confirm segment
IDs, naming, and pricing before activating the data feed.
## Sync configuration
The Audience data and Hashed email (HEM) audience sync types work the same way: Hightouch registers your segment with PubMatic, then uploads the matching users. You configure the segment on the sync itself, so each sync targets a single segment.
### Register a segment
PubMatic ingests user data only for segments that already exist in its taxonomy. Both sync types register their segment from the fields below before uploading any users, so a new segment always exists before its data arrives.
Hightouch re-registers the segment on the first sync run, on a full resync, and whenever you change any of these fields. That way edits — a rename, a new price, or a new segment ID — reach PubMatic on the next run without a separate step.
| Field | Required | Description |
| ---------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------- |
| **Segment ID** | Yes | The segment ID registered in PubMatic. Hightouch uses it to register the segment and as the `segid` value on every uploaded row. |
| **Segment name** | Yes | A human-readable name for the segment. |
| **Segment status** | Yes | `NEW` activates the segment; `EMPTY` deactivates it. Use `NEW` when registering a segment. Defaults to `NEW`. |
| **Description** | No | A description for the segment. |
| **Price (CPM in USD)** | Yes | The CPM price PubMatic requires for the segment, for example `2.0000`. |
| **Creation date** | Yes | When the segment was created, in UTC. Use the format `YYYY-MM-DD HH:MM`, for example `2026-06-17 14:30`. |
### Audience data
Audience data syncs upload the users in a registered segment, keyed on an advertising identifier you choose. Each run uploads a gzipped TSV file to your **Data File Path**.
#### Select the user identifier
Use **User ID Type** to tell PubMatic what kind of identifier is in your mapped **User ID** column. Select one of the presets — **Cookie**, **MAID (IDFA/AAID)**, **UID 1.0 by TTD**, **ID5**, **RampID by LiveRamp**, **Shared ID**, **Panorama ID by Lotame**, **Intent IQ**, or **IP Address** — or enter a custom identity partner ID assigned to you by PubMatic.
#### Map fields
Map your source columns to PubMatic fields:
| Destination field | Required | Notes |
| ----------------- | ----------- | --------------------------------------------------------------------------- |
| **User ID** | Yes | The user identifier. Must match the **User ID Type** selected above. |
| **IP Address** | Conditional | Provide either **IP Address** or **Country** (or both) for geo information. |
| **Country** | Conditional | Provide either **Country** or **IP Address** (or both) for geo information. |
Each uploaded row contains your DPID, the mapped User ID, the segment ID, IP address and country (if provided), and the selected User ID Type. Files are named with your DPID, a timestamp, and a content checksum so PubMatic can deduplicate uploads.
### Hashed email (HEM) audience
Hashed email audience syncs upload users keyed on a hashed email address instead of an advertising identifier. PubMatic resolves the hashed emails into the identifiers you target. Each run uploads a gzipped TSV file to your **HEM Data File Path**.
PubMatic supports hashed email activation in the U.S. region only.
#### Configure email hashing
Use the hashing options to tell Hightouch whether your mapped **Email** column is already hashed:
- If the column holds raw email addresses, Hightouch normalizes each address and hashes it for you.
- If the column is already hashed, Hightouch sends the values as-is.
Then set **Which algorithm are your emails hashed with?** to **SHA-256** or **MD5**. This sets the algorithm Hightouch uses when it hashes raw emails, and tells PubMatic which hash type you're sending.
#### Choose target identifiers
Use **Target UID type** to tell PubMatic which identifiers to resolve the emails into: **Cookie**, **MAID (IDFA/AAID)**, or **Cookie and MAID**. This field is optional — leave it empty to use the default target configured with your PubMatic Account Manager at onboarding.
#### Map fields
Map your source columns to PubMatic fields:
| Destination field | Required | Notes |
| ----------------- | ----------- | --------------------------------------------------------------------------- |
| **Email** | Yes | A raw email address (Hightouch hashes it) or a column that is already hashed. |
| **IP Address** | Conditional | Provide either **IP Address** or **Country** (or both) for geo information. |
| **Country** | Conditional | Provide either **Country** or **IP Address** (or both) for geo information. |
## Tips and troubleshooting
For Hightouch platform error codes related to PubMatic, see [Error codes: PubMatic](/developer-tools/error-codes#other-destinations).
### Validate your setup
After your first sync completes, verify the upload in PubMatic:
- Check that the files landed in the configured directories using your SFTP client — audience data in the Data File Path, hashed email data in the HEM Data File Path, and the segment registration file in the Taxonomy File Path. Hightouch uploads files with a `.UPLOADING` suffix and renames them once the transfer completes, so only fully renamed files should appear.
- Ask your PubMatic Account Manager to confirm that PubMatic ingested the files. Data files reference a segment by its `segid`, so any segment PubMatic hasn't finished processing is dropped on PubMatic's side until registration completes.
### Common errors
### Live debugger
### Sync alerts
---
## PAR Punchh
**URL:** https://hightouch.com/docs/destinations/punchh
**Description:** Use PAR Punchh to turn customer data into revenue—personalized offers, seamless loyalty programs, and real-time insights that keep guests coming back and spending more.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|--------------|--------------------------------------------------------|----------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Segments** | Sync data from any source to custom segments in Punchh | Add, Remove | [Segments docs](https://developers.partech.com/docs/dev-portal-platform-functions/apis/platform-functions-api/tag/custom-segments/post/api2/dashboard/custom_segments/members/bulk_add) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to PAR Punchh
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **PAR Punchh** and click **Continue**. You can then authenticate Hightouch to **PAR Punchh**.
Enter the following fields into Hightouch:
- **Server name**: The subdomain of your Punchh instance (e.g. if your URL is https://placeholder.punchh.com, enter 'placeholder').
- **API key**: Your Punchh business admin API key.
## Sync configuration
Once you've set up your PAR Punchh destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the PAR Punchh destination you want to sync to.
### Syncing segments
Sync data from any source to custom segments in Punchh.
#### Record matching
To match rows from your model to segments in PAR Punchh, you need to select a model column and corresponding PAR Punchh field. You can match on any of the following PAR Punchh fields:
- **User ID**
- **Email**
#### Field mapping
Hightouch lets you sync segment fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default segment fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Qualified
**URL:** https://hightouch.com/docs/destinations/qualified
**Description:** Convert inbound leads at scale
**Section:** Destinations
## Overview
Qualified is a conversational sales and marketing platform that helps revenue teams identify and engage with website visitors. With the Qualified destination, you can sync lead and company data from your data warehouse to Qualified, ensuring your sales team has the most up-to-date information when engaging with prospects.
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| --------- | ----------- | -------------------- | ------------- |
| **Leads** | Sync data from any source to leads in Qualified | Upsert | [Qualified API docs](https://www.qualified.com/api) |
| **Companies** | Sync data from any source to companies in Qualified | Upsert | [Qualified API docs](https://www.qualified.com/api) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Qualified
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Qualified** and click **Continue**.
You will be required to enter your Qualified API key. Generate a Qualified API key with access to the Enterprise bulk endpoints in your Qualified account. The API key must have permissions to access the bulk API endpoints for leads and companies.
The API key must have `Enterprise bulk access` to use the Qualified destination. Confirm that your API key has the necessary permissions before connecting.
Paste the API key exactly as provided; Hightouch passes it as a Bearer token in API requests.
## Sync configuration
Once you've set up your Qualified destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Qualified destination you want to sync to.
### Object selection
Choose which Qualified object to sync to:
- **Leads**: Sync lead data to Qualified
- **Companies**: Sync company data to Qualified
### Record matching
Records are matched from your model to Qualified based on the object type:
- **Leads**: Match by **Email**. The email address is the primary identifier for leads in Qualified. Qualified upserts leads by email address.
- **Companies**: Match by **Domain**. The company domain is the primary identifier for companies in Qualified. Qualified upserts companies by domain.
The record should be matched using a unique identifier. For leads, ensure the email column contains unique values. For companies, ensure the domain column contains unique values. Duplicate identifiers may cause sync errors or unexpected behavior.
Leads synced to Qualified will not appear in the Qualified interface until they have activity associated with them (such as website visits or engagement). The data is successfully synced, but Qualified requires activity before displaying leads in the platform.
### Field mapping
You can sync columns from your source to Qualified's default and custom fields. Map Hightouch columns to Qualified fields in the field mapping section.
When syncing **Leads**, you can map both:
- **Lead fields**: Fields that belong to the lead record
- **Company fields**: Fields that belong to the associated company record
When syncing **Companies**, you can map:
- **Company fields**: Fields that belong to the company record
Qualified requires object fields to be created in advance. After adding new fields in Qualified, click "Refresh fields" in the mapper to pull the latest definitions. Hightouch cannot provision fields automatically.
## Tips and troubleshooting
### Sync performance
Qualified uses bulk API endpoints to efficiently sync large volumes of data. The destination automatically:
- Groups rows into bulk jobs (up to 500 rows per job)
- Manages job polling and status checks
- Handles rate limiting with automatic retries
- Processes job results and reports rejected rows
If you experience rate limiting, contact Qualified support to understand your account's rate limits.
### Common errors
#### API key authentication errors
If you receive authentication errors, verify that:
- Your API key is correct and has not expired
- The API key has Enterprise bulk access permissions
- The API key has access to both lead and company endpoints
#### Rate limiting
If you encounter rate limit errors (429 or 503 status codes), the destination will automatically retry with exponential backoff. Contact Qualified support to understand your account's specific rate limits.
#### Field mapping errors
If you receive errors about invalid fields:
- Ensure custom fields exist in Qualified before mapping to them
- Verify field names match exactly (field names are case-sensitive)
- Check that the data types in your model match the expected types in Qualified
### Live debugger
### Sync alerts
---
## Qualtrics
**URL:** https://hightouch.com/docs/destinations/qualtrics
**Description:** Improve user experiences with rich data by syncing warehouse data to Qualtrics
**Section:** Destinations
## Overview
This integration allows you to import contacts to Qualtrics XM Directory using
endpoint for your automation. After the sync, Qualtrics will asynchronously
process the files.
## Setup
After selecting Qualtrics from our Destinations catalog, you will be asked to
provide your Qualtrics data center ID and API token.
Both of these can be found in the Qualtrics console under Account Settings >
Qualtrics IDs.
## Sending contacts
Hightouch supports syncing contacts to Qualtrics XM Directory. Rows will be
synced when they are added or changed. Rows that are removed from your model
will be ignored.
Hightouch sends your contacts to Qualtrics in batches of 10,000 rows. Each batch
is posted with a unique filename, based on the current time.
## Sync status
Because Qualtrics asynchronously processes the files on the automation's
schedule, Hightouch only shows whether the files were posted to Qualtrics
successfully. Hightouch doesn't wait for Qualtrics to process the files.
We recommend that you enable email notifications in Qualtrics so that you know
when Qualtrics is done processing the files.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## QuickBooks
**URL:** https://hightouch.com/docs/destinations/quickbooks
**Description:** Gain financial visibility and empower your accounting team by keeping your financials up to date
**Section:** Destinations
Hightouch uses minor version `65` of the **QuickBooks Online Accounting API**.
See QuickBooks's
[docs](https://developer.intuit.com/app/developer/qbo/docs/learn/explore-the-quickbooks-online-api/minor-versions)
to learn more about minor versions.
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ------------------------ | -------------------------------------------------------------------- | ---------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| **Account** | Sync data from any source to QuickBooks Accounts | Upsert, Update, Insert | [Account docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/account) |
| **Bill** | Sync data from any source to QuickBooks Bills | Upsert, Update, Insert | [Bill docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/bill) |
| **Class** | Sync data from any source to QuickBooks Classes | Upsert, Update, Insert | [Class docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/class) |
| **Customer** | Sync data from any source to Quickbook Customers | Upsert, Update, Insert | [Customer docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/customer) |
| **Deposit** | Sync data from any source to QuickBooks Deposits | Upsert, Update, Insert | [Deposit docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/deposit) |
| **Invoice** | Sync data from any source to QuickBooks Invoices | Upsert, Update, Insert | [Invoice docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/invoice) |
| **JournalEntry** | Sync data from any source to QuickBooks Journal Entries | Upsert, Update, Insert | [JournalEntry docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/journalentry) |
| **Payment** | Sync data from any source to QuickBooks Payments | Upsert, Update, Insert | [Payment docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/payment) |
| **PaymentMethod** | Sync data from any source to QuickBooks Payment Methods | Upsert, Update, Insert | [PaymentMethod docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/paymentmethod) |
| **RecurringTransaction** | Sync data from any source to QuickBooks RecurringTransaction objects | Insert | [RecurringTransaction docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/recurringtransaction) |
| **TimeActivits** | Sync data from any source to QuickBooks TimeActivity objects | Update, Insert | [TimeActivity docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/timeactivity) |
| **Vendor** | Sync data from any source to QuickBooks Vendors | Upsert, Update, Insert | [Vendor docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/vendor) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to QuickBooks
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **QuickBooks** and click **Continue**. You can then authenticate Hightouch to **QuickBooks** via OAuth.
You can choose to sync your data to a Sandbox or Production Quickbooks environment. Select your preference, click **Log in to QuickBooks**, and log into your QuickBooks account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your QuickBooks destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the QuickBooks destination you want to sync to.
### Syncing accounts
Sync data from any source to accounts in QuickBooks.
#### Record matching
You can match rows from your model to accounts in Quickbooks on any column in your model and any account field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/account) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **Name**: User recognizable name for the Account. 100 character max.
Ensure the data types of your model columns match the data types of the fields you want to sync to. Refer to QuickBooks [account object docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/account) for more information.
### Syncing bills
Sync data from any source to bills in QuickBooks.
#### Record matching
You can match rows from your model to bills in Quickbooks on any column in your model and any bill field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/customer) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **VendorRef**: Reference to the vendor for this transaction. Query the Vendor name list resource to determine the appropriate Vendor object for this reference. Use `Vendor.Id` and `Vendor.Name` from that object for `VendorRef.value` and `VendorRef.name`, respectively.
- **Line**: Individual line items of a transaction. Valid Line types include: `ItemBasedExpenseLine` and `AccountBasedExpenseLine`
Ensure the data types of your model columns match the data types of the fields you want to sync to. Refer to QuickBooks [bill object docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/bill) for more information.
### Syncing classes
Sync data from any source to classes in QuickBooks.
#### Record matching
You can match rows from your model to classes in Quickbooks on any column in your model and any class field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/class) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **Name**: User recognizable name for the Class. 100 character max.
Ensure the data types of your model columns match the data types of the fields you want to sync to. Refer to QuickBooks [class object docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/class) for more information.
### Syncing customers
Sync data from any source to customers in QuickBooks.
#### Record matching
You can match rows from your model to customers in Quickbooks on any column in your model and any customer field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can sync columns from your model to QuickBooks default and custom fields.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/customer) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
### Syncing deposits
Sync data from any source to deposits in QuickBooks.
#### Record matching
You can match rows from your model to deposits in Quickbooks on any column in your model and any deposit field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/deposit) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **DepositToAccountRef**: Identifies the account to be used for this deposit.
- **Line**: Atleast 1 line item must be included to create a deposit.
Ensure the data types of your model columns match the data types of the fields you want to sync to. Refer to QuickBooks [deposit object docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/deposit) for more information.
### Syncing invoices
Sync data from any source to invoices in QuickBooks.
#### Record matching
You can match rows from your model to invoices in Quickbooks on any column in your model and any invoice field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/invoice) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **DepositToAccountRef**: Identifies the account to be used for this deposit.
- **Line**: Atleast 1 line item must be included to create a deposit.
Ensure the data types of your model columns match the data types of the fields you want to sync to. Refer to QuickBooks [invoice object docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/invoice) for more information.
### Syncing journal entries
Sync data from any source to journal entries in QuickBooks.
#### Record matching
You can match rows from your model to jounal entries in Quickbooks on any column in your model and any journal entry field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/journalentry) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **Line**: Individual line items of a transaction. There must be at least one pair of Journal Entry Line elements, representing a debit and a credit, called distribution lines.
Ensure the data types of your model columns match the data types of the fields you want to sync to. Refer to QuickBooks [journal entry object docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/journalentry) for more information.
### Syncing payments
Sync data from any source to payments in QuickBooks.
#### Record matching
You can match rows from your model to payments in Quickbooks on any column in your model and any payment field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/payment) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **TotalAmt**: Indicates the total amount of the transaction. This includes the total of all the charges, allowances, and taxes.
- **CustomerRef**: Reference to a customer or job.
Ensure the data types of your model columns match the data types of the fields you want to sync to. Refer to QuickBooks [payment object docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/payment) for more information.
### Syncing payment methods
Sync data from any source to payment methods in QuickBooks.
#### Record matching
You can match rows from your model to payment methods in Quickbooks on any column in your model and any payment method field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/paymentmethod) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **Name**: User recognizable name for the payment method. 31 character max.
Ensure the data types of your model columns match the data types of the fields you want to sync to. Refer to QuickBooks [payment method object docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/paymentmethod) for more information.
### Syncing RecurringTransaction objects
Hightouch supports syncing RecurringTransaction objects to the **Bill** entity. This sets the [`type`](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/recurringtransaction) field on the ecurringTransaction objects.
If you're interested in syncing RecurringTransations to another entity, please .
#### Record matching
Since you can only insert RecurringTransaction objects, QuickBooks automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/customer) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the **RecurringInfo** field, so you must map it to complete your configuration. Ensure the data types of your model columns match the data types of the fields you want to sync to.
### Syncing TimeActivity objects
Sync data from any source to TimeActivity objects in QuickBooks.
#### Record matching
You can match rows from your model to TimeActivity objects in Quickbooks on any column in your model and any TimeActivity field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/customer) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **NameOf** - Value can be `Vendor` or `Employee`.
- **VendorRef** - Required if **NameOf** is set to `Vendor`.
- **EmployeeRef** - Required if **NameOf** is set to `Employee`.
Ensure the data types of your model columns match the data types of the fields you want to sync to.
### Syncing vendors
Sync data from any source to vendors in QuickBooks.
#### Record matching
You can match rows from your model to vendors in Quickbooks on any column in your model and any vendor field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can sync columns from your model to QuickBooks default and custom fields.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/customer) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## RabbitMQ
**URL:** https://hightouch.com/docs/destinations/rabbitmq
**Description:** Connect to your internal microservices via a message queue
**Section:** Destinations
## Overview
Hightouch integrates with RabbitMQ so you can send data from your warehouse as messages to a RabbitMQ broker without writing and maintaining code, letting you build a custom connector to your internal systems.
Hightouch supports all managed RabbitMQ brokers (Amazon MQ, CloudAMQP, etc.) and can also connect to self-hosted instances.
## Getting started
### Connect to your RabbitMQ server
For Hightouch to connect to your RabbitMQ server, you need to enter the connection parameters of your [RabbitMQ URI](https://www.rabbitmq.com/uri-spec.html). These parameters include the **protocol** used, server connection details (**host address** and **port**), authentication credentials (**username** and **password**) and **vhost**.
You also need to enter your [Management Portal URL](https://www.rabbitmq.com/management.html) for Hightouch to query the queues in your RabbitMQ server,.
- **Protocol**: Select the protocol used for your RabbitMQ server. This is either `amqp` or `amqps`.
- **Host**: Enter your host address to establish the underlying TCP/IP connection to the server.
- **Port**: Enter your port number to establish the underlying TCP/IP connection to the server.
- **VHost**: Enter you vhost. The default value is `/`.
- **Management Portal**: Enter your management portal URL. This can usually be found on the details page for your hosted RabbitMQ solution.
- **Username and Password**: Enter your RabbitMQ server username and password.
## Syncing data
Once you've connected your RabbitMQ server to Hightouch, you can configure a sync that send messages whenever rows are added, changed, or removed in your model.
### Configure your events trigger
Hightouch monitors your data model for added, changed, and removed rows. In this step, you specify which of these events should trigger message publishing.
### Choose your queue
In this step, you choose which queues to publish the messages to. Hightouch allows you to sync to existing queues that are already in your RabbitMQ.
Suppose you want to sync to multiple existing queues but don't want to create a new sync for every queue. As long as your model has a column associated to queue names in your RabbitMQ server, Hightouch can sync to multiple RabbitMQ queues in just one sync.
To enable this feature, toggle **USE COLUMN**, and select a column in your model containing the queue name rows.
When syncing to multiple queues, if a queue name in the selected column of
your model doesn't exist in RabbitMQ, then the sync for that message will
fail.
### Customize your message
In this step, you tell Hightouch how to build the JSON message data object using data from your model.
This destination offers three methods of composing a JSON object:
- [Using a JSON editor](#use-json-editor)
- [Using one column from a model](#use-one-column-from-model)
- [Using multiple columns from a model](#use-multiple-columns-from-model)
#### Use JSON editor
With the JSON editor, you can compose any JSON object using the [Liquid template language](https://shopify.github.io/liquid/). This is particularly useful for complex message data bodies containing nested objects and arrays, which can sometimes be difficult to model entirely in SQL.
Suppose your data model looks like this:
| full_name | age | email_address | phone_number |
| --------- | --- | ------------------ | ------------ |
| John Doe | 30 | `john@example.com` | +14158675309 |
And you want your message data like this:
```json
{
"name": "John Doe",
"age": 30,
"contact_info": [
{
"type": "email",
"value": "john@example.com"
},
{
"type": "phone",
"value": "+14158675309"
}
]
}
```
Your Liquid template should look like this:
```txt
{
"name": "{{row.full_name}}",
"age": {{row.age}},
"contact_info": [
{
"type": "email",
"value": "{{row.email_address}}"
},
{
"type": "phone",
"value": "{{row.phone_number}}"
}
]
}
```
This makes it so you can reference any column using the syntax `{{row.column_name}}`. You can also use advanced Liquid features to incorporate control flow and loops into your dynamic message data.
When injecting strings into your JSON object, be sure to surround the Liquid
tag in double quotes.
#### Use one column from model
If you're already storing JSON data in your source, or if you have the ability to construct a JSON object using SQL, you can select one column in your model that already contains the full message data.
This setting is commonly used when syncing web events that have already been collected and stored as JSON objects in your database.
#### Use multiple columns from model
For the simplest use cases, Hightouch can construct a JSON object with key/value pairs based on multiple columns in your model.
Suppose your model looks like this:
| email | first_name | last_name |
| ----------------------- | ---------- | --------- |
| `alice.doe@example.com` | Alice | Doe |
| `bob.doe@example.com` | Bob | Doe |
| `carol.doe@example.com` | Carol | Doe |
The field mapping in the screenshot above would generate the following message data for the first row:
```json
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "alice.doe@example.com"
}
```
You can use the field mapper to rename fields. For example, `first_name` can
be mapped to `customer_first_name`.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows may require performing a backfill of all rows during the initial sync. For other use cases, you might only want to send messages in response to future data changes.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Rakuten
**URL:** https://hightouch.com/docs/destinations/rakuten
**Description:** Send Ad Conversion Transactions to the Rakuten Transactions API
**Section:** Destinations
## Overview
Rakuten Advertising is a global affiliate marketing platform that connects advertisers with publishers to drive conversions and track affiliate performance. With the Rakuten destination, you can sync transaction and conversion event data from your data warehouse to Rakuten's Advertiser Transaction API, enabling accurate affiliate attribution and commission tracking.
This destination supports comprehensive transaction tracking for multiple verticals including Retail, Travel, and Financial services, with both affiliate-attributed (click-based) and Linkless transaction types.
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| --------- | ----------- | -------------------- | ------------- |
| **Events** | Sync transaction/conversion events from any source to Rakuten | Insert | [Rakuten API docs](https://api.linksynergy.com/docs) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Rakuten
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Rakuten** and click **Continue**.
You need to provide the following credentials:
- **Client ID**: Your Rakuten API client ID
- **Client Secret**: Your Rakuten API client secret
- **Advertiser ID (MID)**: Your Rakuten Merchant ID (numeric, up to 5 digits)
### Obtain API credentials
Contact your Rakuten Advertising account representative to obtain API credentials. You'll need credentials with access to the Advertiser Transaction API endpoint.
The Advertiser ID must be numeric and no more than 5 characters. Verify your credentials by testing the connection before creating syncs.
## Sync configuration
Once you've set up your Rakuten destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Rakuten destination you want to sync to.
### Event timestamp
Select the column from your model that contains the transaction timestamp. This becomes the `transaction_datetime` in Rakuten and must be in a valid datetime format. The timestamp is automatically converted to ISO 8601 UTC format.
Transaction timestamps cannot be in the future. Ensure your transaction datetime values represent when the transaction occurred.
### Field mapping
Hightouch lets you sync transaction fields via [field mapping](/syncs/mapping-data). You can map data from any of your model columns to Rakuten's transaction fields.
#### Required fields
You must map the following fields to complete your configuration:
- **Order ID**: A unique identifier for the transaction (max 40 characters)
- **Currency**: ISO 3-character currency code (e.g., `USD`, `EUR`, `GBP`)
- **Items**: An array of line item objects (see Items array section below)
#### Tracking fields
These optional fields enable affiliate attribution:
- **Click ID (ranSiteID)**: The Rakuten tracking parameter captured from affiliate redirects. This is a 34-character identifier. For Linkless transactions, this field can be omitted.
- **Landing datetime**: The UTC timestamp when the user landed on your site from the affiliate link. If not provided, this automatically defaults to 5 minutes before the transaction timestamp to ensure API compliance.
For affiliate-attributed transactions, capture the `ranSiteID` parameter from the referring URL when users arrive from Rakuten affiliate links. This enables proper commission attribution.
#### Items array
Each transaction requires at least one item. Use the [inline mapper](/syncs/mapping-data#inline-mapping) to structure the items array. Each item in the array must include:
**Required item fields:**
- `sku`: Product identifier (max 40 characters)
- `product_name`: Product name (max 512 characters)
- `amount`: Item subtotal (post-discount, excluding tax)
- `quantity`: Number of units
**Optional item fields:**
- `order_item_id`: Unique identifier for the line item
- `brand`: Product brand name
- `category`: Product category path (e.g., `Electronics>Widgets`)
- `category_id`: Category identifier
- `coupon`: Item-level coupon code applied
- `discount_amount`: Item-level discount amount
- `discount_type`: Type of discount applied
- `is_clearance`: Whether the item is on clearance
- `is_sale`: Whether the item is on sale
- `marketplace_store`: Marketplace store name
- `product_id`: Additional product identifier
Use the inline mapper to transform your source data into the required array format. If your data has one row per order, you'll need to structure the items as a JSON array.
#### Optional transaction fields
Rakuten supports extensive optional fields organized by vertical:
**General/Retail fields:**
- `alternative_order_id`, `customer_id`, `customer_status`, `customer_rank`, `customer_segment`
- `coupon`, `credit_card_type`, `commission_reason`
- `discount_amount`, `discount_type`, `order_status`
- `purchase_site`, `ship_country`, `shipped_datetime`
- `store_id`, `store_category`
**Travel fields:**
- `check_in_datetime`, `check_out_datetime`
- `reservation_start_datetime`, `reservation_end_datetime`, `reservation_length_in_days`
- `cancel_deadline_datetime`, `consumed_datetime`
**Financial fields:**
- `autopay`, `home_ownership`
- `loan_amount_requested`, `loan_apr`, `loan_approval_datetime`
- `loan_offer_apr`, `loan_offer_badge`, `loan_offer_term`
- `loan_origination_fee`, `loan_model_number`, `loan_purpose_applicant`
- `loan_term_in_months`, `monthly_housing_payment`, `publisher_lead_id`
**Offline tracking (O2O) fields:**
- `o2o_bank_partner`, `o2o_store_id`, `o2o_store_zip`
### Sync behavior
Rakuten processes transactions individually via POST requests. The destination automatically:
- Formats datetime values to Rakuten's required ISO 8601 format (without milliseconds)
- Validates the items array contains at least one item
- Handles authentication token management and refresh
- Provides detailed error messages from Rakuten's validation
## Tips and troubleshooting
### Linkless transactions
For Linkless transactions (conversions without an affiliate click), you can omit the `click_id` field. The destination automatically sets a default `landing_datetime` value (5 minutes before the transaction) to ensure API compliance.
### Date format requirements
All datetime fields must be convertible to ISO 8601 UTC format. The destination handles the conversion automatically, but ensure your source data contains valid datetime values.
The `landing_datetime` must be before the `transaction_datetime`. If you provide a landing datetime that occurs after the transaction, the Rakuten API will reject the request.
### Field length limits
Rakuten enforces strict field length limits:
- `order_id`: Maximum 40 characters
- `sku`: Maximum 40 characters
- `product_name`: Maximum 512 characters
Ensure your source data respects these limits to avoid validation errors.
### Common errors
#### Authentication errors
If you receive 401 or 403 authentication errors:
- Verify your Client ID and Client secret are correct
- Confirm the credentials have access to the Advertiser Transaction API
- Check that your Advertiser ID (MID) matches your account
#### Validation errors
If you receive 400 validation errors, check the error details for:
- Missing required fields (order_id, currency, items)
- Empty items array (at least one item is required)
- Invalid field formats (especially datetime fields)
- Field values exceeding length limits
#### Items array errors
If you receive errors about the items array:
- Ensure the items field is a valid array with at least one item
- Verify each item has all required fields: `sku`, `product_name`, `amount`, `quantity`
- Check that numeric fields (`amount`, `quantity`) contain valid numbers
### Live debugger
### Sync alerts
---
## Reddit Ads
**URL:** https://hightouch.com/docs/destinations/reddit-ads
**Description:** Run more targeted ads by syncing conversion events and audiences to Reddit
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| ---------- | -----------------------------------------------------| ---------------------- | ------------------------------------------------------------------------------------------------------- |
| [**Conversion events**](#syncing-conversion-events) | Sync data from any source to Reddit as conversion events | Insert | [Conversions](https://ads-api.reddit.com/docs/v3/operations/Post%20Conversion%20Events)|
| [**Audiences**](#syncing-custom-audiences) | Sync data from any source to Reddit as a custom audience list | Upsert | [Audiences](https://ads-api.reddit.com/docs/v3/operations/Create%20Custom%20Audience)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Reddit
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Reddit Ads** and click **Continue**.
Authorize Hightouch to access Reddit by logging into your account and agreeing to the app's usage. To successfully sync Custom Audiences, Hightouch requires the `adsedit` and `adsread` scope.
## Sync configuration
### Account ID
An Account ID is required to sync Conversion events and Audiences.
By default, Hightouch will automatically use the Reddit Ads account owned by the user authenticated. If you'd like to use a different account ID, you can navigate to the respective dashboard and grab the ID from the URL.
### Syncing conversion events
#### Prerequisites
- **Allowlisted by Reddit:** Hightouch uses Reddit's Conversions API, which is in beta.
To send conversion events, your Reddit Ads account needs to by white listed by Reddit. Reach out to your Reddit account team to request access for your account to use the Conversions API.
- **Events within 7 days:** If you are planning to set the event timestamp yourself, the event must occur within the last 7 days otherwise the conversion event will not go through.
#### Field mapping
To view the schema and the field descriptions of the conversion events endpoint, visit the [Reddit docs](https://ads-api.reddit.com/docs/v3/operations/Post%20Conversion%20Events).
Notable fields:
- **Event Source URL** (`event_source_url`): the full URL of the web page where the event occurred, including query parameters. It applies to website events only. Reddit uses the URL to detect the event's domain in Events Manager. When the Reddit click ID (`rdt_cid`) is appended as a query parameter, Reddit extracts it to improve attribution if the **Click ID** field isn't mapped.
- **Products** (`metadata[products]`): an array of product objects that passes product-level information from your catalog; Reddit requires these fields when using product ads or dynamic product ads. Each product supports `id` (the variant SKU or GTIN matching your catalog, required by Reddit), `name`, `category`, `quantity`, and `item_price`. Per-product `quantity` and `item_price` aren't replacements for the event-level **Item Count** and **Value** fields, which Reddit still recommends sending on revenue-related events.
### Syncing custom audiences
Hightouch will sync data to Reddit ads as a Custom Audience with the `CUSTOMER_LIST` type.
You can choose to create a new Custom Audience through Hightouch, or sync to an existing audience.
#### User identifiers
Reddit accepts the following user fields: `EMAIL_SHA256` and `MAID_SHA256`. At least one identifier is required to successfully sync to Reddit ads.
#### Handling PII and hashing
The Reddit Custom Audiences API will **only** accept hashed values. For audience syncs, by default, Hightouch automatically hashes first-party user data fields before sending requests to Reddit.
If the data in your model is already hashed, you can disable PII hashing to successfully send data to Reddit.
#### Delete behavior
When records leave your query result, they are removed from your audience by default. If you would not like to delete any records from your audience, you can choose to do nothing when records leave your query result.
## Validate your setup
After you configure your sync, run a test to confirm data flows correctly:
1. Start with a small test audience or a single record.
2. Trigger a manual sync from the [Syncs overview](https://app.hightouch.com/syncs?from=docs) page.
3. After the sync completes, verify the data in Reddit Ads:
- For audience syncs, go to **Audiences** in the Reddit Ads dashboard and confirm your audience appears. Reddit may take 24–48 hours to fully process audience data.
- For conversion event syncs, check the **Events Manager** to confirm events were received.
4. If the sync shows errors, check the sync run log in Hightouch for specific API error messages.
## Tips and troubleshooting
### Matched users count
Below only applies to the **Audiences** sync type.
You may observe match rates in Reddit Ads that are greater than 100%. Reddit Ads returns the count of matched users as a range. Reddit Ads does not reliably deduplicate users and each of the IDs can be counted separately, meaning the calculated audience size can exceed the actual number of unique individuals. The use of an audience size range is meant to help advertisers understand whether an audience segment is broad enough for effective ad delivery, without misrepresenting the precision or uniqueness of the reach.
### Audience shows "Invalid" in Reddit Ads
After syncing an audience to Reddit, it may appear as "Invalid" in the Reddit Ads UI even though Hightouch reports a successful sync. This typically happens because Reddit is still processing the audience data. Wait 24–48 hours for processing to complete, then trigger a full resync in Hightouch. If the audience remains invalid after a full resync, .
### Don't see your data in Reddit?
Make sure you've configured your sync to use the correct Reddit environment. You might've accidentally pointed to a different environment. If that doesn't solve your issue, . We're happy to help.
### Common errors
**event_at timestamp must be less than 168h0m0s old**
Reddit Ads rejects conversion events where the `event_at` timestamp is older than 7 days (168 hours). If you see this error or a related `RangeError: Invalid time value`, ensure your model filters events to include only those from the last 7 days. You can add a `WHERE` clause like `WHERE event_at >= DATEADD(day, -7, CURRENT_TIMESTAMP)` to your model query.
### Live debugger
### Sync alerts
---
## Redis
**URL:** https://hightouch.com/docs/destinations/redis
**Description:** Redis is an in-memory data structure store, used as a distributed, in-memory key–value database, cache and message broker, with optional durability. Redis supports different kinds of abstract data structures, such as strings, lists, maps, sets, etc.
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes | API Reference|
|--|--|--|--|
|**Redis Cache**|Sync data from any source to a Redis cache database|Upsert, Update|[Redis Cache](https://redis.io/docs/manual/client-side-caching/)|
## Connect to Redis
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Redis** and click **Continue**.
### Connect to a Redis instance
Authenticate Hightouch to Redis with your hostname and port. If your Redis instance has authentication credentials, you can also input a username and password.
#### SSH tunneling
Hightouch can connect directly to Redis over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your Redis instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
### Connect to a Redis cluster
Hightouch needs permission to access all cluster nodes of your Redis cluster. If your Redis cluster exists on a publicly available network, only the primary cluster node URL is required. Provide the cluster node URLs in the specified `redis://:` format.
#### Node address map
A node address map is a mapping between the addresses in the cluster and the addresses the client should connect to. This is useful when connecting to a cluster running on a different network than the client. For more information on node address maps specifically, refer to [`node-redis`' documentation](https://github.com/redis/node-redis/blob/master/docs/clustering.md#node-address-map).
To get the addresses of your Redis cluster nodes, run `CLUSTER NODES` in your cluster. Refer to the [Redis documentation](https://redis.io/docs/latest/commands/cluster-nodes/) for more details.
A common use case of node address mapping is connecting to AWS ElastiCache. To access ElastiCache resources through Hightouch, you will need to follow the instructions defined [in this AWS tutorial](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/accessing-elasticache.html) to create a NAT instance in the same VPC as your ElastiCache instance, but on a public subnet. This NAT instance will need to forward a port to each Redis cluster node. Ensure that you can successfully connect to your Redis cluster through the NAT instance.
When configuring your connection from Hightouch, construct the Redis cluster node URLs by using the publicly accessible IP address of your NAT instance with the corresponding ports. Then, map each cluster node address to the corresponding public address in the node address mapping section.
#### Connect over a private network
If you have a Redis cluster running on a private network with no publicly accessible endpoints, you will need to create a [SSH tunnel](/security/ssh-tunneling) to _each_ cluster node.
To get the addresses of your Redis cluster nodes, run `CLUSTER NODES` in your cluster. Refer to the [Redis documentation](https://redis.io/docs/latest/commands/cluster-nodes/) for more details.
Provide the URL for each cluster node, and map it to its corresponding SSH tunnel.
## Sync configuration
Once you've set up your Redis destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Redis destination you want to sync to.
### Syncing to a cache database
Hightouch supports syncing [**keys**](#syncing-keys) and [**members**](#syncing-members) to Redis.
### Syncing keys
Keys identify, store, and retrieve data. Each key in Redis maps to a corresponding value.
#### Supported commands
|Command |Description|Supported Sync Modes | API Reference|
|---------|-----------|---------------------|--------------|
|**`SET`**|Set key to hold the string value.|Upsert, Update|[SET Command](https://redis.io/commands/set/)|
|**`HSET`**|Set specified fields to their respective values in the hash stored at key.|Upsert, Update|[HSET Command](https://redis.io/commands/hset/)|
|**`SADD`**|Add specified members to the set stored at key.|Upsert|[SADD Command](https://redis.io/commands/sadd/)|
|**`GEOADD`**|Add specified geospatial items (longitude, latitude, name) to the specified key.|Upsert|[GEOADD Command](https://redis.io/commands/geoadd/)|
#### Record matching
To match rows from your model to records in Redis, you need to select the model column that contains values that match the **Key** field in Redis.
#### Field mapping
You can sync columns from your source to custom fields in Redis.
#### Time to live (TTL)
You can set an expiration for the keys that you are syncing to Redis by setting a [TTL value](https://redis.io/commands/ttl/). This value is the amount of time in seconds that the keys will exist in Redis before they're automatically deleted.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the key in Redis with its fields untouched |
| **Clear** | Keep the key in Redis, but set mapped fields to empty |
| **Delete** | Remove the key from Redis |
### Syncing members
Members are used for managing [sorted sets](https://redis.io/docs/data-types/sorted-sets/) in Redis.
#### Supported commands
|Command |Description|Supported Sync Modes | API Reference|
|---------|-----------|---------------------|--------------|
|**`ZADD`**|Add all specified members with the specified scores to the sorted set stored at key.|Upsert, Update|[ZADD Command](https://redis.io/commands/zadd/)|
|**`GEOADD`**|Add specified geospatial items (longitude, latitude, name) to the specified key.|Upsert|[GEOADD Command](https://redis.io/commands/geoadd/)|
#### Record matching
To match rows from your model to records in Redis, select the model column that contains the unique member values of each element of the sorted set in Redis.
#### Field mapping
The fields available for mapping depend on the [command](#supported-commands-1) that you've selected.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the member in Redis with its fields untouched |
| **Clear** | Keep the member in Redis, but set mapped fields to empty |
| **Delete** | Remove the member from the sorted set from Redis |
### Transactions
Transactions are disabled by default when connected to a Redis cluster.
You can toggle transactions for your Redis instance by configuring the options as shown.
## Tips and troubleshooting
### Common errors
#### Invalid argument type
This error can happen when syncing data types that aren't [supported by Redis](https://redis.io/docs/data-types/), such as `null` values. If your model columns contain `null` values, you can enable [Don't sync null values](/syncs/mapping-data#dont-sync-null-values) to avoid syncing them.
### Live debugger
### Sync alerts
---
## Reply.io
**URL:** https://hightouch.com/docs/destinations/replyio
**Description:** Supercharge your sales team by syncing your most up-to-date customer and lead data from your warehouse
**Section:** Destinations
## Setup
Upon selecting Reply.io as your destination you will be prompted to
input your Reply.io API key.
You can locate this API key within your Reply.io account by navigating to Settings > API Key:
## Syncing
Hightouch supports syncing the following Reply.io objects:
- `People`
### Sync modes
- **Upsert**: pushes new objects to Reply.io _and_ updates fields that change in your warehouse.
- **Update**: only updates particular fields on existing objects in Reply.io. It does _not_ add new objects.
### Record matching
Records can be matched from your source to your Reply.io workspace by the **contact's email** only.
### Field mapping
You may also sync columns from your source to Reply.io's object properties.
## Other considerations
As per their [documentation](https://apidocs.reply.io/#32e27eaa-b6b1-496e-8504-04b676aec3c1),
Reply.io's API is limited to _15,000 requests per user per month_.
This is important to keep in mind when syncing large amounts of data since **Upsert** mode
will make one API call per record synced and **Update** mode will make up to two API calls per record synced
(one to check that user exists and the other to update the user). Please plan your syncs accordingly.
[If you'd like to increase this rate limit, please contact Reply.io support.](https://support.reply.io/en/articles/978063-api-rate-limits).
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Resonate
**URL:** https://hightouch.com/docs/destinations/resonate
**Description:** Resonate tells you who your highest-value audiences are and what they’ll do next so you can act with precision, before the competition.
**Section:** Destinations
## Overview
Resonate is an audience intelligence platform that helps brands activate first-party data for targeting and match enrichment. With the Resonate destination, you can sync customer data from your data warehouse to Resonate via SFTP, enabling audience matching and activation.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------ | -------------------------------------------------------- | -------------------- |
| **Audience** | Sync customer records to Resonate for audience matching | All, Insert |
- **All**: Sends all records in your model every time the sync runs.
- **Insert**: Sends only new records added since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Resonate
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Resonate** and click **Continue**.
You will need the following credentials from Resonate:
- **Username**: Your Resonate SFTP username.
- **Private Key**: The SSH private key for SFTP authentication. Provide the full private key including the header and footer.
- **Private Key Passphrase** _(optional)_: The passphrase for your private key, if applicable.
Resonate uses SSH key-based authentication. Contact your Resonate account representative to obtain your SFTP credentials and private key.
Hightouch connects to Resonate's SFTP server at `sftp.resonate.com` on port 22. Files are automatically placed in the `data-append/inbound/` directory.
## Sync configuration
Once you've set up your Resonate destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button. Select the relevant model and the Resonate destination.
### Field mapping
Map columns from your model to Resonate's required fields. **Customer ID is required** for every sync. You must also include at least one identifier group:
| Field | Required | Notes |
| ----- | -------- | ----- |
| **Customer ID** | Yes | Your internal customer identifier |
| **Email/HEM** | Pair with HEM Type | Emails are automatically hashed (MD5 preferred). Map both HEM and HEM Type together |
| **HEM Type** | Pair with HEM | The hash type of the email (e.g. MD5, SHA-1, SHA-256) |
| **MAID** | One of the identifier groups | Mobile advertising identifier |
| **Address Line 1** | All address fields required together | Street address |
| **Address Line 2** | Optional | Secondary address (apartment, suite, etc.) |
| **City** | All address fields required together | City |
| **State** | All address fields required together | State |
| **Zip Code** | All address fields required together | 5-digit ZIP code |
| **ZIP11** | One of the identifier groups | 11-digit ZIP+4 code |
| **IP Address** | One of the identifier groups | IPv4 format |
HEM and HEM Type must be mapped together — you cannot map one without the other. Similarly, if you map any postal address field, all required address fields (Address Line 1, City, State, Zip Code) must be mapped.
Hightouch automatically hashes email values in the HEM field using MD5 (preferred), SHA-1, or SHA-256. You do not need to pre-hash emails before syncing.
### Data quality options
**Remove Duplicates**: When enabled, duplicate rows based on Customer ID are automatically removed before writing to the file.
**Skip Invalid Rows**: When enabled, rows missing required identifiers (Customer ID + at least one identifier group) are skipped rather than causing sync errors.
## File format
Resonate requires CSV files. Hightouch automatically handles file naming and placement:
- **Directory**: `data-append/inbound/`
- **File naming**: `{ModelName}_{SegmentName}_{YYYYMMDD}_{syncRunId}.csv`
## Tips and troubleshooting
### Match rate
Resonate targets a 60–80% match rate. To maximize match rates:
- Include complete postal address (Address Line 1, City, State, Zip Code) when available — this is Resonate's strongest identifier.
- Ensure email addresses are lowercase and trimmed before syncing. Hightouch handles hashing automatically.
- Remove inactive, duplicate, and low-quality records from your model before syncing.
- Suppress international records, business addresses, and P.O. boxes.
### Data requirements
Resonate enforces the following data quality requirements:
- Emails must be lowercase with no leading or trailing spaces (Hightouch hashes them automatically).
- Records missing consent should be excluded from your model.
- International records, business addresses, and P.O. boxes should be suppressed.
### Common errors
### Live debugger
### Sync alerts
---
## Responsys
**URL:** https://hightouch.com/docs/destinations/responsys
**Description:** Oracle Responsys is a cross-channel campaign management platform that delivers advanced intelligence at scale so you can create personalized messages based on customers' and prospects' individual interests and preferences.
**Section:** Destinations
## Supported syncing
|Event Type|Description|Supported Sync Modes |
|--|--|--|
|**Profile Lists**|Create base profiles with Email, phone number, or custom IDs|Upsert, Update|
|**Profile Extension Tables**|Add custom attributes and properties to the base profiles|Upsert, Update|
|**Custom / REI Events**|Add items to Events|Insert, Update|
## Getting started
To use the Responsys destination in Hightouch, you need to know your Responsys credentials and which Oracle system hosts your
Responsys account. To find your host URL refer to the [Oracle Responsys documentation](https://docs.oracle.com/en/cloud/saas/marketing/responsys-rest-api/SendRequests.html).
## Syncing data
### Profile lists
Syncing to Responsys profile lists lets you create base profiles. Base profiles primarily contain user IDs to associate
the user to records in profile extension tables.
#### Record matching
You can match rows in your model with users in Responsys on the following fields:
- Email
- Mobile number
- Customer ID
- Responsys ID (RIID)
- MD5 Hashed Email
- SHA256 Hashed Email
Hashed emails can't be used for upsert operations.
#### Field mapping
You can map additional user information such as other identifiers, country, address, or other user-created fields.
### Profile Extension Tables
Syncing to profile extension tables lets you attach additional information to base user profiles.
Profile extension tables (PETs) give you a way to store additional attributes that define behavioral, demographic or
profile preference characteristics of your contacts. Using PETs helps you organize information about your contacts
in a way that is most efficient for your queries and data imports.
Refer to the [Responsys Profile Extension Table docs](https://docs.oracle.com/en/cloud/saas/marketing/responsys-user/List_CreatePET.htm) to learn more.
#### Record matching
You can match rows in your model with users in Responsys on the following fields:
- Email
- Mobile number
- Customer ID
- Responsys ID (RIID)
- MD5 Hashed Email
- SHA256 Hashed Email
Hashed emails can't be used for upsert operations.
#### Field mapping
You can sync columns from your source to any user-created field in your Responsys PET.
### Custom / REI Events
Syncing to Responsys Custom Events (part of Responsys Event Infrastructure or REI) allows you to track and store behavioral data about your contacts.
These events capture specific actions or interactions that contacts have with your brand, such as purchases, website visits, app usage, or any custom activities you want to monitor.
API Documentation on Responsys events is available [here](https://docs.oracle.com/en/cloud/saas/marketing/responsys-rest-api/api-events.html).
#### Record matching
You can match rows in your model with users in Responsys on the following fields:
- Event Name
- Profile List
#### Field mapping
You can sync rows to Responsys events on the following fields:
- Recipient ID
- Customer ID
- Email Address
- Mobile Number
- MD5 Hashed Email
- SHA256 Hashed Email
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Retention Science
**URL:** https://hightouch.com/docs/destinations/retention-science
**Description:** AI-driven email marketing automation
**Section:** Destinations
## Overview
You want to send relevant, timely emails to your customers, and doing so heavily relies on the accuracy and freshness of the data in Retention Science. By automatically syncing customer data from your data warehouse into Retention Science, you no longer need to worry about data consistency because data is no longer flowing in from multiple sources in different formats, and your team can focus on drafting engaging emails and delivering delightful experiences.
## Supported syncing
Type | Description | Supported Sync Modes | API Reference |
----------|---------------------------------------------------------|--------------------- | ----------------- |
**Users** | Sync data from any source to Users in Retention Science | Upsert, Update | [Users docs](https://developer.retentionscience.com/?data/v3.0.0#tag/Users)|
**Lists** | Add or remove users from a list in Retention Science | Add, Remove | [Lists docs](https://developer.retentionscience.com/?data/v3.0.0#tag/Lists)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Retention Science
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination**
button. Select **Retention Science** and click **Continue**. You can then authenticate Hightouch to **Retention Science** by entering a Retention Science API key.
To get your API Key, go to your ReSci dashboard. Click on the dropdown menu in the upper right hand corner and then select **API settings**. Click **API Password** to reveal your key.
## Sync configuration
Once you've set up your Retention Science destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and
the Retention Science destination you want to sync to.
### Users
Sync data from any source to Users in Retention Science.
#### Record matching
You can match rows in your model with user in Retention Science on any of the following fields:
- **Email**
- **Record ID**: this is also called `id` or `record_id`
#### Field mapping
You can sync columns from your source to the following columns in Retention Science user:
- **Full Name**: User's full name (first and last)
- **Address Line 1**
- **Address Line 2**
- **City**
- **State**
- **Zip code**
- **Country**
- **Phone number**
- **Year of birth**
- **Gender**
- **Registartion Source**: The source of the user's registration typically used for user segmentation
- **Birthday**: User's full birthday in `YYYY-MM-DD` format
- **Send Transactional Welcome**: When `true`, ReSci sends an email to new users when there is an available template in the Welcome stage. When `false`, ReSci doesn't send welcome emails to new users. No emails are sent to existing users who are updated. The default for this setting is `true`.
- **Merge Before Upsert**: By default, a request containing a `record_id` and `email` that match two distinct users will result in an API error, with an error message stating that we cannot uniquely identify which user to update. If you set the `merge_before_upsert` field to `true`, ReSci merges these two users into one. The default for this setting is `false`.
- **Date unsubscribed**: The User's unsubscribe date. Setting this field unsubscribes a user from email. This field accepts `YYYY-MM-DD hh:mm:ss` (24-hour clock) in UTC. Note that passing a null value doesn't re-subscribe a previously unsubscribed user. If a user is unsubscribed, ReSci ignores null values.
- **Account Created On**: User's signup date in format `YYYY-MM-DD`. If this field and existing user **Account Created On** value differ, ReSci chooses the oldest date between the two.
- **Timestamps**: The timestamp is the date and time of when the data changed for a specific plugin fields. ReSci uses the newest timestamp when updating the fields. If the timestamp for the plugin data in the database is newer than the data in the call, then the data in the call will be ignored.
- **UTM**: User's UTM tags
- **Accepts Marketing**: Indicates whether the user has opted in to accepting marketing emails. For new users, the default is `true`. Existing users' marketing preferences will not be modified unless you explicitly change this field.
- **Accepts Marketing Timestamp**: The timestamp of a users subscription status change. If earlier than a user's last subscription update, the requested subscription status update will be ignored. The `date_unsubscribed` field takes precedence over this date, so if both are present this date will be ignored. If no date is passed in the current time will be used.
- **Birthday Month**: Birthday month of the user being upserted in numerical format, for example, January = 1. Note that this field is ignored if birthday is present.
- **Birthday Day of Month**: Day of the month for the users birthday. Note that this field is ignored if the **Birthday** field is present.
- **Interests**: An array of interest object representing user's interest opt-in selections.
- **Frequency Option ID**: An integer representing the ID of a user's frequency choice for subscription preferences.
The **Send Transactional Welcome** field defaults to `true`.
If a sync creates a new user record and you've enabled transactional welcome emails in Cortex, then the sync results in ReSci sending the user a welcome email.
Set **Send Transactional Welcome** to `false` if you don't want to send a welcome email.
See Retention Science's [docs](https://developer.retentionscience.com/?data/v3.0.0#tag/Users/paths/~1users/patch) for more details.
#### Delete behavior
**Delete** is only available for upsert mode.
Behavior | Description
---------------|-------------------------------------------------------------------------
**Do nothing** | Keep the customer record in Retention Science with all its synced fields
**Clear** | Unset all the mapped fields from Retention Science
**Delete** | Delete the synced record from your Retention Science user
### Lists
Add or remove users from a list in Retention Science.
#### Record matching
You can match rows in your model with list in Retention Science on any of the following fields:
- **Email**
- **Record ID**: this is also called `id` or `record_id`
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Rockerbox
**URL:** https://hightouch.com/docs/destinations/rockerbox
**Description:** Rockerbox brings clarity to the noise through accurate attribution and marketing mix modeling that helps you make the right optimizations.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes |
| ----------------------- | ---------------------------------------- | -------------------- |
| **Batch File Delivery** | Upload batch files to Rockerbox via SFTP | Insert, All, Diff |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Rockerbox
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Rockerbox** and click **Continue**. You can then authenticate Hightouch to **Rockerbox**.
Enter the following required fields into Hightouch:
- **Username**: Enter your Rockerbox batch file delivery username.
- **Password**: Enter the password for the username above.
## Sync configuration
Once you've set up your Rockerbox destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Rockerbox destination you want to sync to.
### Syncing batch file delivery
Upload batch files to Rockerbox via SFTP.
#### Field mapping
Hightouch lets you sync batch file delivery fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default batch file delivery fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Rockset
**URL:** https://hightouch.com/docs/destinations/rockset
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Setup
You will need to provide the API Key.
Once logged into your Rockset account, click the **API Keys** option under the **Manage** section in the sidebar.
You can either copy an existing key, or click the **Create API Key** button to generate one.
## Syncing
### Sync modes
Hightouch let's you select the workspace and collection to sync to.
This integration supports the Insert, Update and Upsert sync mode.
In the Insert mode, new rows will be inserted into Rockset.
In the Update mode, rows will be kept up-to-date within Rockset.
During the Upsert mode, new rows will be inserted into Rockset and existing rows will be kept up-to-date within Rockset.
### Columns to sync
For Rockset, we give you the ability to send all columns as they are represented in your model.
You can also choose to manually map fields. Only the fields that you map will be exported. All other columns from your results are ignored.
### Delete mode
The default is to do nothing, but you can also set Hightouch to delete the record when it leaves the query result.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Rokt
**URL:** https://hightouch.com/docs/destinations/rokt
**Description:** Empower your marketing team by delivering up-to-date event data to Rokt straight from your warehouse
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes | API reference |
|--|--|--|--|
|**Events**|Sync data from any source to events|Insert| [Event API docs](https://docs.rokt.com/developers/api-reference/event-api)|
|**Segments**|Create and update [audiences](https://docs.rokt.com/developers/api-reference/custom-audience-import)|Add| [Custom Audience API](https://docs.rokt.com/developers/api-reference/custom-audience-import)|
## Connect to Rokt
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Rokt** and click **Continue**. You can then authenticate Hightouch to Rokt by entering the following required fields into Hightouch:
- **App ID**
- **App Secret**
- **Account ID**: You can find this in the URL of your Rokt account. Go to your profile settings and the URL will look like this: `https://my.rokt.com/accounts//profile-settings`
You can find the AppID and the App Secret at the bottom of your Rokt profile settings.
If you haven't created an app yet, create an app and generate the two values.
For more information, refer to [Rokt's docs](https://docs.rokt.com/developers/api-reference/event-api#generating-app-id-and-app-secret).
If you plan on syncing audience segements, you also need to enter your **API Key**.
You can retrieve one by following these instructions in [Rokt's docs](https://docs.rokt.com/developers/api-reference/custom-audience-import/#authorization).
## Sync configuration
Once you've set up your Rokt destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Rokt destination you want to sync to.
### Syncing events
When syncing events, Hightouch treats any records added to [your source](/getting-started/concepts#sources) as new events and sends them to Rokt when your sync runs.
[Rokt's Event API](https://docs.rokt.com/developers/api-reference/event-api#post-events) requires the following event parameters:
- `clientEventId`
- `eventType`
- `eventTime`
The sync configuration form ensures all these are set and provides some additional options.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server. Hightouch syncs the timestamp as the `eventTime` parameter the Event API requires.
If you select a column, it should be in in UTC and RFC 3339.
The request will be rejected if it contains an event with a timestamp that is older than 18 months or more than 5 minutes into the future from the time that the Event API receives it.
Events with timestamps that are outside of this range are not processed.
#### Field mapping
You can sync model columns to Rokt's event properties, including custom fields.
You must include mappings for the **Client Event ID** and **Event Type** fields.
#### Handling PII and hashing
By default, Hightouch automatically hashes the following fields before sending
them to Rokt:
- **Email**
- **Mobile Phone**
- **First Name**
- **Last Name**
- **IP Address**
You can disable this behavior in the sync configuration. If disabled, the data from the model should be appropriately normalized and hashed according to [Rokt's hashing requirements](https://docs.rokt.com/developers/integration-guides/data-and-security/pii-data-hashing/).
### Syncing segments
You can use these lists to target or exclude specific customers in your campaigns.
You can also use these lists to determine which customers experience Rokt on your site.
The Rokt API enables you to create an audience using email information.
You select which type of email field to use in the field mapping portion of sync configuration.
#### User identifiers
To identify which users to add or update in an audience, select model columns that contain the users' email, email (MD5 Hash), or email (SHA256 Hash).
#### Select an existing audience or create a new one
You can create a new audience or use an existing one. When creating a new audience, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model. To use an existing audience, select the desired audience from the dropdown.
#### Handling PII and hashing
By default, Hightouch automatically hashes unhashed emails before sending them to Rokt.
You can disable this behavior in the sync configuration.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Roku Advertising Direct
**URL:** https://hightouch.com/docs/destinations/roku-ads
**Description:** Empower your advertising team to target and suppress first-party audiences across direct buys on Roku’s premium streaming platform inventory.
**Section:** Destinations
## Overview
Deliver more relevant ads across Roku’s premium, brand-safe inventory. Reach
addressable audiences across native, destination, video, and action ad
experiences by matching your first-party identifiers with Roku-connected
devices using Match Booster.
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| ------------- | ---------------------------------- | -------------------- | --------------------------------------------------------------------------------------- |
| [**Audiences**](#syncing-custom-audiences) | Create and update custom audiences | All | [Custom Audiences docs](https://help.ads.roku.com/en/articles/9471526-custom-audiences) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Roku Ads
This integration requires prior approval from the Roku Ads team and the Hightouch
team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your Roku rep who will verify approval. Once we have confirmed, we will unlock
the integration for you to connect.
To connect, input the `Advertiser Name` provided by your Roku rep. This must match
exactly to what is in Roku's system.
### Syncing custom audiences
You can input or select the name of the audience you plan on delivering to Roku Ads.
#### User identifiers
To identify which users to add or update in an audience, select model columns
and the corresponding Roku Ads fields. You can match on any of the following user
fields:
- Email
- Phone
- IP Address
- Name
- Postal Address
- 1P Roku IDs
If you select 1P Roku IDs (also known as `rida`), Hightouch sends these 1P Roku IDs to the Roku segment directly, along
with other matched Roku IDs based on your other mapped identifiers.
#### Delete behavior
You can choose how to handle records in Roku when the corresponding rows are deleted in your source.
| Behavior | Description |
| -------------------- | ------------------------------------------ |
| **Do nothing** | Keep the contact in your Roku audience |
| **Remove from list** | Remove the contact from your Roku audience |
## Tips and troubleshooting
### Refresh rate
Roku Ads has a max refresh rate of daily. You should not have your Roku Ads
syncs running more than daily.
### Matched device count
Match rates are not available to preview in the Hightouch platform. Contact your
Roku rep for insights into match rates and audience reach.
Keep in mind that audience counts reflect the number of Roku devices match, not
the number of users matched. Also keep in mind that matching occurs at the
household level, and that a single household may own multiple devices.
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Live debugger
### Sync alerts
---
## Rudderstack
**URL:** https://hightouch.com/docs/destinations/rudderstack
**Description:** Sync rich customer data from your data warehouse to hundreds of SaaS tools via Rudderstack's large integration library
**Section:** Destinations
## Setup
To find the write key for your RudderStack destination in Hightouch:
1. Create a new HTTP API source in RudderStack
2. Navigate to Source Settings > API Keys and copy the "Write Key"
## Syncing
### Event types
- **Track \(Generic\):** Sends events of different names when a record is added, changed, or removed in the query results
- **Identify:** Sends an identify event with related properties of the user
### User and anonymous IDs
it's required to send either the user or anonymous ID to RudderStack to identify the user associated with the event.
### Field mapping
You can sync columns from your source into properties of the user \(identify\) or properties of the event \(track\).
### Sending initial results \(track only\)
For the two "Track" event types, Hightouch allows you to choose whether to:
- Send the initial set of results as track events
- Or, only send the changes between the results in the subsequent syncs as track events
Checking the "Send events for initial result set" checkbox will result in a large amount of events being sent in the first sync.
Hightouch will always send the initial set of results as identify calls when in the "Identify" event type.
### Event names and timestamps \(generic track only\)
For the generic track mode, there are further settings that are needed to determine which event name Hightouch will send to RudderStack for each added, changed, and removed record.
In this example, Hightouch will send RudderStack:
- A user signed up event when a record is added to the query results
- A user changed subscription event when a record is changed in the query results
- A user deleted account event when a record is removed in the query results
Hightouch allows you to send a timestamp that the event occurred for generic track events. This is useful for backfilling historical events.
Hightouch also allows you to send a message ID for each record to ensure that the event is deduplicated within RudderStack in the case that the same event erroneously gets sent twice.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## S3
**URL:** https://hightouch.com/docs/destinations/s3
**Description:** Transfer data at scale from your warehouse to S3
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
| ------------ | ------------------------------------------------------------ | -------------------- |
| Any data set | Sync data from a source to S3 as CSV, JSON, NDJSON, XML, or Parquet files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Prerequisites
To get started, you need:
- an [S3 bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html)
- AWS [credentials configured in Hightouch](/security/aws) with programmatic access enabled and permission to write to the S3 path you want to use
Hightouch needs the following IAM actions to store items in your bucket:
| Action | Details |
| :------------------------------ | :---------------------------------------------------------------------------------- |
| `s3:PutObject` | Grants permission to add an object to a bucket |
| `s3:AbortMultipartUpload` | Grants permission to to abort or cancel an ongoing multipart upload in an S3 bucket |
| `s3:ListMultipartUploadParts` | Grants permission to list the parts of a multipart upload in an S3 bucket |
| `s3:ListBucketMultipartUploads` | Grants permission to list all the ongoing multipart uploads within an S3 bucket |
For small file transfers, Hightouch only uses the `s3:PutObject` action.
For large files, multi-part uploads may be necessary, hence the other required actions.
You can use the following JSON sample to create your IAM policy:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Sample",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:AbortMultipartUpload",
"s3:ListMultipartUploadParts",
"s3:ListBucketMultipartUploads"
],
"Resource": [
"arn:aws:s3:::${bucketName}/*",
"arn:aws:s3:::${bucketName}"
]
}
]
}
```
See the guide for [configuring AWS credentials](/security/aws) for more details.
## Connect to Amazon S3
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Amazon S3** and click **Continue**. You can then authenticate Hightouch to Amazon S3 by entering your **Bucket Name**. The Bucket Name should just be the name of the bucket, not a URL.
### PGP encryption
The S3 destination supports PGP encryption. Given a **PGP Public Key**, all files transferred to the destination will be encrypted.
To use PGP encryption, provide your public key in the **PGP Public Key** text area including the header and footer in the following format:
```bash
-----BEGIN PGP PUBLIC KEY BLOCK-----
-----END PGP PUBLIC KEY BLOCK-----
```
The **PGP Private Key** is required to decrypt the files synced to your
destination. You don't need to insert this value in Hightouch.
## Sync configuration
Once you've set up your S3 destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the S3 destination you want to sync to.
### Select file format
Hightouch supports syncing CSV, JSON, NDJSON, XML, and Parquet files to Amazon S3.
### Enter filename
The filename or **object key** field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file.
Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync,
Hightouch overwrites it. To keep different versions of the same results file,
you can enable
[versioning](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html)
in your bucket, or your app can copy the data to another location.
If you are using an audience and would like to include the audience name, you
will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter `-86400` (`24 hours * 60 minutes * 60 seconds`). If you want the filename's data to be one hour after the sync takes place, you would enter `3600` (`60 minutes * 60 seconds`).
### CSV options
If you're syncing to a CSV file, you have additional configuration options:
- **Delimiter**: Your options are comma (`,`), semicolon (`;`), pipe (`|`), tilde (`~`), and tab
- (Optional) Whether to **include a CSV header row** in the exported files
- (Optional) Whether to **include a [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark)** in the exported files, the BOM is ``
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `customer_id`, `email`, `first_name` and `last_name`, columns. These columns are mapped to new fields in the destination file as `id`, `email`, and `name`—a [templated](/syncs/mapping-data#template-mapping) concatenation of `first_name` and `last_name`—respectively. Hightouch exports these fields to new fields in the file and ignores all other columns from your results.
### Batch size
By default, and depending on your [sync mode](#supported-syncing), Hightouch sends one file to your S3 bucket for each export. (In diff mode, Hightouch sends three files.)
By enabling batching, you allow Hightouch to send multiple files if your model query results are large. This can help avoid file size errors from S3.
### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
### Indicator file
For diff mode syncs or syncs with batching enabled, you can include an indicator filename. By doing so, an empty file will be uploaded after all files have been uploaded to indicate successful completion of the sync.
The indicator file will not be written in the cases of a sync failure or an empty query result.
### Gzip compression
You can enable gzip compression to reduce the size of your exported files. Note
that Hightouch does not automatically add the `.gz` extension to your files. You
can add this or any other extension to your filename.
## Tips and troubleshooting
### Common errors
#### Access denied
The **ACCESS_DENIED** error is a permissions-related error. Ensure your Hightouch AWS credentials have read/write access to the S3 bucket specified in your destination. The minimal permissions required are `s3:GetObject`, `s3:PutObject`, and `s3:ListBucket`.
### Live debugger
### Sync alerts
---
## Sage Intacct
**URL:** https://hightouch.com/docs/destinations/sage-intacct
**Description:** Sync data from your warehouse to Sage Intacct to streamline your financial processes and gain comprehensive insights into your financial data.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
------------------|-------------------------------------------------------|------------------------|-------------------------------------------------------------------------------------------
**AR Adjustment** | Sync data from any source to the AR Adjustment object | Insert | [AR Adjustment docs](https://developer.intacct.com/api/accounts-receivable/ar-adjustments)
**AR Payment** | Sync data from any source to the AR Payment object | Insert | [AR Payment docs](https://developer.intacct.com/api/accounts-receivable/ar-payments)
**Contact** | Sync data from any source to the Contact object | Upsert, Update, Insert | [Contact docs](https://developer.intacct.com/api/company-console/contacts)
**Customer** | Sync data from any source to the Customer object | Upsert, Update, Insert | [Customer docs](https://developer.intacct.com/api/accounts-receivable/customers)
**Invoice** | Sync data from any source to the Invoice object | Update, Insert | [Invoice docs](https://developer.intacct.com/api/accounts-receivable/invoices)
**Journal Entry** | Sync data from any source to the Journal Entry object | Insert | [Journal Entry docs](https://developer.intacct.com/api/general-ledger/journal-entries)
**Transaction** | Sync data from any source to the Transaction object | Update, Insert | [Transaction docs](https://developer.intacct.com/api/order-entry/order-entry-transactions)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Sage Intacct
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Sage Intacct** and click **Continue**. You can then authenticate Hightouch to **Sage Intacct**.
Enter the following required fields into Hightouch:
- **Sender ID**: Enter a valid Sender ID.
- **Sender Password**: Enter a valid Sender Password.
- **Company ID**: Enter a valid Company ID.
- **User ID**: Enter a valid User ID.
- **User Password**: Enter a valid User Password.
## Sync configuration
Once you've set up your Sage Intacct destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Sage Intacct destination you want to sync to.
### Syncing AR Adjustment
Sync data from any source to the AR Adjustment object
#### Field mapping
Hightouch lets you sync AR Ajustment fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default AR Adjustment fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
### Syncing AR Payment
Sync data from any source to the AR Payment object
#### Field mapping
Hightouch lets you sync AR Payment fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default AR Payment fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
### Syncing Contact
Sync data from any source to the Contact object
#### Record matching
You can match rows from your model to Contact in Sage Intacct on any column in your model and any field in Sage Intacct. Ensure the data types of the model column and Sage Intacct field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Sage Intacct automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync Contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default Contact fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|------------------------------------------------------------------
**Do nothing** | Keep the Contact in Sage Intacct with all its synced fields
**Clear** | Clear all the mapped fields, but keep the Contact in Sage Intacct
**Delete** | Delete the synced Contacts from Sage Intacct
### Syncing Customer
Sync data from any source to the Customer object
#### Record matching
You can match rows from your model to Customer in Sage Intacct on any column in your model and any field in Sage Intacct. Ensure the data types of the model column and Sage Intacct field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Sage Intacct automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync Customer fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom Customer fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|-------------------------------------------------------------------
**Do nothing** | Keep the Customer in Sage Intacct with all its synced fields
**Clear** | Clear all the mapped fields, but keep the Customer in Sage Intacct
**Delete** | Delete the synced Customers from Sage Intacct
### Syncing Invoice
Sync data from any source to the Invoice object
#### Record matching
You can match rows from your model to Invoice in Sage Intacct on any column in your model and any field in Sage Intacct. Ensure the data types of the model column and Sage Intacct field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Sage Intacct automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync Invoice fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom Invoice fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|------------------------------------------------------------------
**Do nothing** | Keep the Invoice in Sage Intacct with all its synced fields
**Clear** | Clear all the mapped fields, but keep the Invoice in Sage Intacct
### Syncing Journal Entry
Sync data from any source to the Journal Entry object
#### Field mapping
Hightouch lets you sync Journal Entry fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default and custom Journal Entry fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
**Formatting entries**
Hightouch recommends using the [array builder inline mapper](/syncs/mapping-data#create-an-array) to map properties that the [Sage Intacct SDK](https://github.com/intacct/intacct-sdk-js/blob/acc333d883ade0e37fe1baee5118ff009c109458/src/Functions/GeneralLedger/JournalEntryLineCreate.ts#L30) expects.
The Sage Intacct SDK automatically determines the `TR_TYPE` based on the `AMOUNT` of the entry, so there's no need to map this field. To represent whether an entry is a debit or credit, use a positive or negative `AMOUNT` value, respectively. The sum of all entries in the array will be zero. The absolute value of the `AMOUNT` should not be used.
### Syncing Transaction
Sync data from any source to the Transaction object
#### Record matching
You can match rows from your model to Transaction in Sage Intacct on any column in your model and any field in Sage Intacct. Ensure the data types of the model column and Sage Intacct field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Sage Intacct automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync Transaction fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom Transaction fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|------------------------------------------------------------------
**Do nothing** | Keep the Transaction in Sage Intacct with all its synced fields
**Clear** | Clear all the mapped fields, but keep the Transaction in Sage Intacct
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Sailthru
**URL:** https://hightouch.com/docs/destinations/sailthru
**Description:** Run better marketing campaigns on Sailthru with up-to-date customer data from your data warehouse
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
------------------|-------------------------------------------------------|----------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
**Users** | Sync data from any source to Sailthru users | Upsert, Update | [Users docs](https://getstarted.meetmarigold.com/engagebysailthru/Content/developers/api/user.html?tocpath=Technical%20Documents%7CFor%20Developers%7CAPI%20Endpoints%7C_____21#POSTtocreateorupdateuser)
**Custom events** | Sync data from any source to Sailthru custom events | Insert | [Custom events docs](https://getstarted.meetmarigold.com/engagebysailthru/Content/developers/api/event.html?tocpath=Technical%20Documents%7CFor%20Developers%7CAPI%20Endpoints%7C_____5#POST)
**Purchases** | Sync data from any source to Sailthru purchase events | Insert | [Purchases docs](https://getstarted.meetmarigold.com/engagebysailthru/Content/developers/api/purchase.html?tocpath=Technical%20Documents%7CFor%20Developers%7CAPI%20Endpoints%7C_____14#POSTapurchase)
**Returns** | Sync data from any source to Sailthru return events | Insert | [Returns docs](https://getstarted.meetmarigold.com/engagebysailthru/Content/developers/api/return.html?tocpath=Technical%20Documents%7CFor%20Developers%7CAPI%20Endpoints%7C_____15#POSTareturn)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Sailthru
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Sailthru** and click **Continue**. You can then authenticate Hightouch to **Sailthru**.
Enter the following fields into Hightouch:
- **API Key**
- **API Secret**
To find your API key, navigate to the [Sailthru API Settings](https://my.sailthru.com/settings/api_postbacks).
## Sync configuration
Once you've set up your Sailthru destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Sailthru destination you want to sync to.
### Syncing users
Sync data from any source to Sailthru users.
#### Record matching
To match rows from your model to users in Sailthru, you need to select a model column and corresponding Sailthru field. The available matching fields depend on your [sync mode](/syncs/types-and-modes#sync-modes):
- **Email** works in both update and upsert modes.
- **Sailthru ID** works in update mode only.
#### List mapping
Hightouch supports syncing users to a single list. You can input the name of your list on the sync configuration page. If the list doesn't exist yet, Hightouch will create the list automatically in Sailthru.
#### Field mapping
Hightouch lets you sync user fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom user fields.
If you send data for a custom field that doesn't exist, Hightouch adds the field.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|-----------------------------------------------------------
**Do nothing** | Keep the user in Sailthru with all its synced fields
**Clear** | Clear all the mapped fields, but keep the user in Sailthru
**Delete** | Delete the synced users from Sailthru
### Syncing custom events
Sync data from any source to Sailthru custom events.
#### Record matching
You can match rows from your model to custom events in Sailthru on any column in your model and any field in Sailthru. Ensure the data types of the model column and Sailthru field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
### Syncing purchases
Sync data from any source to Sailthru purchase events.
#### Record matching
You can match rows from your model to purchases in Sailthru on any column in your model and any field in Sailthru. Ensure the data types of the model column and Sailthru field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
### Syncing returns
Sync data from any source to Sailthru return events.
#### Record matching
You can match rows from your model to returns in Sailthru on any column in your model and any field in Sailthru. Ensure the data types of the model column and Sailthru field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Salesforce
**URL:** https://hightouch.com/docs/destinations/salesforce
**Description:** Push enriched data from your warehouse into Salesforce CRM for your teams to act upon it
**Section:** Destinations
## Overview
With the Salesforce destination, Hightouch can both create and update standard or custom objects, update custom picklist field values on those objects, and create platform events to drive more automation in Salesforce.
Use the Salesforce destination to sync to:
- Salesforce Sales Cloud (CRM)
- Salesforce Financial Cloud
- Salesforce Health Cloud
- Salesforce Service Cloud
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| [**Standard and custom objects**](#objects) | Sync records to Salesforce objects such as accounts, contacts or custom objects | Insert, Update, Upsert, Archive | [Manage object records ↗](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_list.htm) |
| [**Metadata for picklist fields**](#custom-picklist-values) | Sync metadata to your Salesforce Picklist fields. | All | [Update metadata field types ↗](https://developer.salesforce.com/docs/atlas.en-us.api_meta.meta/api_meta/meta_field_types.htm) |
| [**Platform events**](#platform-events) | Publish event messages that can be received by Salesforce flows, processes, Apex triggers, and more. | Insert | [Publish platform events ↗](https://developer.salesforce.com/docs/atlas.en-us.platform_events.meta/platform_events/platform_events_intro.htm) |
- **Insert**: Insert mode pushes new objects or events to Salesforce and doesn't update them as they change in your data source. Insert is the only available mode for platform events.
- **Update**: Update mode updates fields on existing objects in Salesforce.
- **Upsert**: Upsert mode pushes new objects to Salesforce and updates fields that change in your data source.
- **Archive**: Archive mode [deletes](https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/dome_delete_record.htm) all records that appear in your model's query results.
- **All**: All mode is the only available mode for metadata for picklists. Each time you sync metadata values for a particular picklist, it overwrites all existing values.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Understand your Salesforce setup
Before you begin your Hightouch to Salesforce sync configuration, clarifying your business goals and understanding your Salesforce setup is crucial. Walk through the following considerations to ensure your team is aligned and your sync plans are clear. If you prefer, you can download this resource as a PDF to share with your team.
### Business goals
First, you need to plan what data you want to sync to Salesforce, how you want to update it, and how frequently you want to update it. Consider your business needs and consult with your team to align on the answers to these questions.
The following table shows how the responses affect your sync configuration.
|
Consideration
| Effect on sync configuration |
| ------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| What standard and custom objects do you want to sync to Salesforce? Contacts, Leads, etc.? | The response affects your [sync types](#supported-syncing)—in other words, which objects you select to sync to. You need to set up a separate sync configuration for each object you want to sync to. |
| What standard and custom data fields do these objects have that you want to sync to? | The response affects the [field mapping](#field-mapping) on each of your syncs. |
| Do you have new data to add, old data that needs updating, or a combination of both? | The response affects your [sync modes](#supported-syncing)—for example, insert mode for new data, update mode for old, and upsert mode for both. |
| How frequently do you need to update the data? | The response affects your [sync schedule](/syncs/schedule-sync-ui). This is the last step of your sync configuration. |
To learn what objects and fields you are using in your Salesforce instance, you can [inspect your Object Manager](#inspect-objects-relationships-and-fields).
### Required data specifications
Once you've confirmed the objects and fields you want to sync, you need to check that they have the correct data types and that you have permission to edit them. You can find the answers to these questions by [using the Object Manager in Salesforce](#inspect-objects-relationships-and-fields).
|
Consideration
| Effect on sync configuration |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Is the field that you would like to use for [record matching](#record-matching) an [**external ID**](https://help.salesforce.com/articleView?id=inserting_updating_or_deleting_data.htm&type=5) and [**unique**](https://help.salesforce.com/s/articleView?id=000385174&language=en_US&type=1)? | The response affects your [record matching](#record-matching); only fields that meet this criteria appear in the dropdown for record matching. |
| What are the expected data types of the fields you want to sync to? | The response affects your [field mapping](#field-mapping); if the expected type is different than your model column type, you need to [type cast your data](/models/data-types-casting) in your model configuration. |
| Are the fields you want to sync to editable? | Hightouch only displays editable fields during [field mapping](#field-mapping). |
| Does the user you are authenticating to Hightouch have edit access? | The sync will error if the user doesn't have the correct permissions. |
| Are any of the data fields lookup fields to another Salesforce object | You can set up [Lookup fields](#relationships-object-id-lookup) while field mapping. |
| Are any of the data fields formula fields? | Formula fields are non-editable and read-only, so the sync will throw an error; change the field type if you want to sync data to it. |
### Other considerations
These last considerations examine the wider context of your Salesforce instance to ensure that your sync won't conflict with other Salesforce automations or overuse your API call quota.
You can [inspect an objects' triggers and flows](#inspect-apex-triggers-and-flows) using the Object Manager. You can inspect your API usage and quota by going to **Setup → Environments → System Overview**, or by checking Salesforce's [quick reference](https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_api.htm).
Refer to your Salesforce contract in case of any custom limits.
|
Consideration
| Effect on sync configuration |
| --------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| What Salesforce workflows or APEX triggers are you running? What objects do they touch, when do they run? | The response affects your [sync schedule](/syncs/schedule-sync-ui); syncs will error if there are conflicting processes. |
| What Salesforce plan do you have and what are your API limits? | The response affects your [sync schedule](/syncs/schedule-sync-ui); syncs will error if you use up your API call quota. |
### Inspect objects, relationships, and fields
1. Login to your Salesforce instance and make sure you are in the **Sales** module.
2. Click on the gear icon in the top right and open the **Setup** page.
3. Search for **Object Manager** in the left search input and open it.
4. Search for the object you are interested in syncing to the **Quick find** input, for example **Contact**. Click on the label name to inspect the object.
5. From the left sidebar, click **Fields & Relationships**. Inspect the field names and data types to confirm which you plan to sync via Hightouch. Any fields you want to use for [record matching](#record-matching) must have the **External ID** and **Unique** types.
6. If necessary, **create a new custom field** and set it's type to Unique and an External ID.
- Click **New** in your chosen object's **Fields & Relationships**.
- Choose the field type, for example **Email** or **Text**, then click **Next**. (You'll set the Unique and an External ID types in the next step.)
- Give your new field a **Field Label** and description. Enable the **Unique** and **External ID** checkboxes. Click **Next**.
- Select the profiles to which you want to grant edit access to this field via field-level security. The field will be hidden from all profiles if you don't add it to field-level security. Click **Next**.
- Add the field to the page layouts you like and click **Save**.
### Inspect APEX triggers and flows
1. Login to your Salesforce instance and make sure you are in the **Sales** module.
2. Click on the gear icon in the top right and open the **Setup** page.
3. Search for **Object Manager** in the left search input and open it.
4. Select the object you want to inspect the APEX triggers and workflows for, for example **Contact**. Click on the object to open its detail page.
5. From the left sidebar, click **Triggers** or **Flow Triggers**, depending on which one you want to inspect first.
- The Triggers page displays a list of all the APEX triggers that are associated with the object. To inspect a specific trigger, click its name.
- The Flow Triggers page displays list of all the workflows that are associated with the object. To inspect a specific flow, click its name.
6. Once you've opened a specific workflow or trigger, you can inspect its properties, such as its name, status, and other details.
You can also edit or delete the workflow or trigger from this page, as well as view the history of changes made to it, if you have the correct permissions.
If you have any triggers or flows on objects you plan on syncing to, make sure to [schedule your Hightouch syncs](/syncs/schedule-sync-ui) so as to not conflict with them.
## Connect to Salesforce
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Salesforce** and click **Continue**. You can then authenticate Hightouch to **Salesforce** via OAuth.
Authenticate with a user that has access to update all fields, records, and objects that you need to interact with. The easiest way to do this is to authenticate with a [system administrator](https://www.salesforce.com/blog/what-is-a-salesforce-admin/) or [integration account user](https://help.salesforce.com/s/articleView?id=sf.create_sf_user.htm&language=en_US&type=5).
Click **Log in to Salesforce**, log in with a Salesforce account with the correct access level, and click **Allow**. You should then see a success notification in the Hightouch UI.
If you have Salesforce Authenticator setup as your 2FA, you may have to accept
the request from your device.
After you've authorized access to Salesforce click **Continue**. Give your destination a descriptive name, for example, "Salesforce Production," and click **Finish**.
## Sync configuration
Once you've set up your Salesforce destination, have a [model](/getting-started/concepts#models) to pull data from, and have a clear idea of the data you want to sync, you can set up your sync. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Salesforce destination you want to sync to.
### Objects
When syncing objects, you can choose to insert, update, upsert, or delete (archive) them. Consult the following table to understand which is best for your use case.
| Sync mode | Use case |
| --------- | --------------------------------------------------------------------------------------------------------------------- |
| Insert | You want to push objects into Salesforce, but don't care if the data within each object remains up to date. |
| Update | You have objects in Salesforce that you want to add information to, but you don't want to create any new objects. |
| Upsert | You want to push objects into Salesforce and want to keep the data up-to-date. |
| Archive | You have objects in Salesforce you would like to delete and your model contains those records and only those records. |
#### Record matching
Because inserts are non-idempotent when using Salesforce ID as an identifier,
Hightouch doesn't support matching on Salesforce ID when upserting. **If you'd
like to sync records based on Salesforce ID, use update mode**.
To match rows from your model to record in Salesforce, you need to select the model column and corresponding Salesforce field. Hightouch requires you use an [**external ID**](https://help.salesforce.com/articleView?id=inserting_updating_or_deleting_data.htm&type=5) type field as the field to match on. It should be an explicit external ID type in Salesforce, not just an external ID field by practice.
Using an external ID field for matching lets Hightouch use Salesforce's native [upsert API](https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/dome_upsert.htm), meaning Hightouch doesn't need to perform a search to determine if a record should be updated or inserted. As a result, sync performance improves and avoids inadvertent duplicates.
You can create a custom field and set it to the external ID type by following the [steps outlined above](#inspect-objects-relationships-and-fields). To learn more, check out the [Salesforce article on externalID field creation](https://help.salesforce.com/s/articleView?id=000383278&type=1). The information regarding audit fields in the Salesforce article doesn't apply.
Once you've created the custom field, select **Refresh** next to the destination field dropdown to access the newly created field.
An explicit external ID field will have **(recommended)** written next to its name.
#### Field mapping
You can map data from any of your model columns to native and custom fields in your Salesforce objects. Ensure the data type of your model column matches the data type of the field you want to sync to in Salesforce. For example, a [string-type](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/primitive_data_types.htm) Salesforce field like `FirstName` expects a [string-type](/models/data-types-casting#data-types) model column.
Certain Salesforce fields can be set upon record creation but can't be edited on existing records. Since Salesforce rejects updates to fields that can't be edited:
- in Update mode, the field mapping section only lists editable Salesforce fields;
- in Upsert mode, you can also map to non-editable Salesforce fields, but Hightouch omits those mappings when updating existing records, to make sure that your sync is successful.
To see if a field is editable, check its [`updateable`](https://developer.salesforce.com/docs/atlas.en-us.uiapi.meta/uiapi/ui_api_responses_field.htm) field property via the Salesforce API. You can check this property by [generating an initial access token](https://help.salesforce.com/s/articleView?id=sf.remoteaccess_oidc_initial_access_token.htm&type=5) and calling the [sObject Describe](https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/resources_sobject_describe.htm) API endpoint. The `updateable` value is included in Salesforce's API response, as shown in [this example](https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/dome_sobject_describe.htm).
When syncing to a [multi-select
picklist](https://help.salesforce.com/s/articleView?id=sf.siteforce_data_form_field_types.htm&type=5)
type field, make sure to sync `string`-type values containing
semicolon-delimited lists, as explained in [Salesforce's
documentation](https://help.salesforce.com/s/articleView?id=sf.picklist_limitations.htm&type=5).
#### Relationships (object ID lookup)
In Salesforce, you can set up different [types of relationships](https://help.salesforce.com/s/articleView?id=sf.overview_of_custom_object_relationships.htm&type=5) to associate records on a **main object** with records on a **linked object**.
Using Hightouch, you can relate your Salesforce data through **lookup relationships**, which associate records by using a foreign key.
This relationship type requires you to copy the [Salesforce ID](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/field_types.htm#i1435616) of the associated record (on the linked object) and insert it in a specific [reference field](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/field_types.htm#i1435823) in the record you're syncing (on the main object).
This reference field needs to be an [**associated field** data type](/models/data-types-casting#data-types), which corresponds to Salesforce's [**lookup** type](https://help.salesforce.com/s/articleView?id=sf.external_object_lookup_relationships.htm&type=5).
You can confirm the field is a lookup data type in Salesforce by [using the Object Manager](#inspect-objects-relationships-and-fields).
For example, you may want to associate contacts you're updating with their respective Salesforce accounts.
In this case, the **Contact** object is the main object and the **Account** object is the linked object.
Records on the **Contact** object have an [`AccountId` field](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_contact.htm) which expects the Salesforce ID of the related records on the **Account** object.
However, your data model might not contain the required Salesforce ID values.
As a workaround, Hightouch allows you to perform an **Object ID lookup** by:
1. looking up the Account object based on another unique Salesforce field,
2. retrieving the Salesforce ID of the associated record on the Account object,
3. syncing that value to the `AccountId` field on the Contact object.
Make sure to use a unique Salesforce field for the lookup.
If you already created a separate sync to the linked object, you can select the Salesforce field used for [record matching](#record-matching) in that sync.
If your source data already contains the required Salesforce ID values, you
can skip the lookup by selecting the **Salesforce ID** field in the
relationship mapping. This will also increase your sync performance.
Make sure that the sync to the linked object always runs before the one to the
main object. After creating your syncs, you can [set up a
sequence](/syncs/schedule-sync-with-sequences) to schedule them in the correct
order.
In your sync configuration, start by selecting the Salesforce reference field you want to sync to.
The left side of the mapper displays the text and inputs:
- **find [linked object]**
- **where [Salesforce field to use for lookup]**
- **equals [model column to match on]**
You then need to select the Salesforce field and model column to match on.
In this example, Hightouch looks up the corresponding record on the **Account** object by matching the value in the `unique_key` model column to the values in the `Unique_Key__c` Salesforce field.
The lookup returns the associated Salesforce ID, which Hightouch syncs to the `AccountId` field on the **Contact** object.
You can set up a lookup relationship between records on the same object, such
as relating two records on the **Account** object by mapping the [`ParentId`
lookup
field](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_account.htm).
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. Depending on the [sync mode](#objects) you're using, you can select from some or all of these options:
| Behavior | Description |
| ----------------- | -------------------------------------------- |
| **Do nothing** | Keep the record in Salesforce |
| **Clear fields** | Keep the record, but clear the mapped fields |
| **Delete record** | Delete the record in Salesforce |
In Upsert mode, you can select from all three. In Update mode, you can only select between **Do nothing** and **Clear fields**. Insert mode doesn't support delete behavior.
#### Batching and parallelization
By default, Hightouch sends batches of 1,000 records to Salesforce and sends ten batches in parallel.
You can increase the batch size up to 10,000 records per batch and parallelization up to 50 batches. Higher batch sizes and batch parallelization can increase the speed of your sync, but may eventually cause errors due to the amount of processing power they require from Salesforce.
If you run into APEX errors such as `TOO_MANY_APEX_REQUESTS` it can be helpful to lower and tune these numbers. For example, you can decrease your batch size by half and if you no longer encounter errors, you can raise the batch size limit slowly.
#### Bulk API version
Hightouch supports both legacy [Bulk API](https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asynch/api_asynch_introduction_bulk_api.htm) and [Bulk API 2.0](https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asynch/bulk_api_2_0.htm). Hightouch uses the original Bulk API by default. You can opt in to use V2.0 in the sync configuration.
The API versions have different batch and parallelization limits.
Specifically, the original Bulk API counts the number of calls and Bulk API 2.0 counts the number of records.
You can learn more about different limits and allocations between the two versions in [Salesforce's docs](https://developer.salesforce.com/docs/atlas.en-us.244.0.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_bulkapi.htm).
You should select the version that best suits your Salesforce agreement.
[**Insert** mode](#objects) doesn't support Bulk API 2.0.
#### REST API Support
Hightouch provides the option to use Salesforce's [REST API](https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/intro_rest.htm) instead of the Bulk API. This can be useful if your account runs into usage limits with the Bulk API and has a higher usage limit for the REST API. You can enable the REST API in the `Advanced` section in your sync configuration.
Hightouch currently uses `v58.0` for the REST API.
The REST API uses a default batch size of 200 records per request. If you have automations that require specific batch sizes, you can adjust this in the `Advanced` settings configuration as well.
#### Split retries
Salesforce return a `200` successful response even when part of a batch of updates failed.
In some cases, you may want to retry the failed records, like for the [`UNABLE_TO_LOCK_ROW` error](#unable_to_lock_row).
You can enable [**split retries**](/syncs/retries#split-retries) to filter out the records
that received the relevant errors and retry them again.
#### Merging objects
You can only merge **Account**, **Contact**, **Individual**, and **Lead**
objects. Only [upsert and insert](#objects) modes support merging, and you
must set [parallelization](#batching-and-parallelization) to **1**.
If your model contains child records that should be merged into a parent record, Hightouch handles merging the records into their respective parents based on two fields:
- `shared_id`
- `merge_id` or `is_child`
Hightouch treats records with the same `shared_id` as the same record and merges them together.
All child records in a group with the same `shared_id` must have a `merge_id` or `is_child` field populated.
For example, suppose you have the following records in your model's query result:
| Account | Shared ID | Is child |
| --------- | --------- | -------- |
| Account A | 1 | true |
| Account B | 1 | true |
| Account C | 1 | false |
| Account D | 2 | false |
| Account E | 3 | false |
| Account F | 3 | true |
Hightouch would merge Account A, Account B, and Account C together with Account C as the parent.
Accounts E and F will be merged together with Account E as the parent.
Select the appropriate model columns and corresponding Salesforce fields for this logic.
#### Multi-object syncs
Hightouch supports syncing to multiple objects, such as **Contact or Lead** and **Account or Lead**. When syncing to multiple objects, Hightouch checks to see if your **external ID mapping** maps to either object. If so, it updates the appropriate object.
To use multi-object syncs, make sure that your Lead's external ID field is **mapped** to the matching field on your Contact or Account. You can check by following these steps.
1. Find the **Lead** in the Salesforce **Object Manager**. Refer to the above documentation for information on [how to use and access the Object Manager](#inspect-objects-relationships-and-fields).
2. Go to **Fields & Relationships**.
3. Click **Map Lead Field**.
4. Map your Lead's external ID to the matching field on the Contact or Account.
You may also map other fields that you want to carry over from Leads when they're converted.
#### Salesforce person accounts syncs
If you need to send data to a Salesforce [person account](https://help.salesforce.com/s/articleView?id=sf.account_person.htm&type=5), follow these steps:
1. Contact Salesforce to enable person accounts if they aren't already.
1. In your Hightouch sync configuration, select **Account** as the object to sync data to. A person account is a copy of the account object.
1. Select your desired [sync mode](#objects).
1. Ensure you map at least a **LastName** field instead of **FullName**. Without this level of specificity, the sync fails.
1. Complete your configuration as usual.
### Custom picklist values
Hightouch lets you update picklist values on any object. You need to create a separate sync configuration for each picklist you want to update, even if the picklists exist on the same object.
When syncing custom picklist values, _all active picklist values_ are **replaced** with each sync. For example, if your model returns four records, all four records become values for your picklist. Hightouch doesn't check which values are already there or append the new values. Each sync completes overwrites existing picklist values.
First, select the object whose picklist you want to update, then specify the picklist whose values you wish to update:
#### Record matching
Hightouch matches rows from your model to picklist values on the values' **API name**, also referred to as `valueName`. Select the model column that includes the appropriate API names for matching. API names must be unique across both inactive and active values within a picklist.
#### Field mapping
You can optionally select a model column that custom labels. Label values must be unique amongst active values in a picklist.
#### Limitations
Custom picklists may only contain a **total of 1000 values** (active and inactive) per picklist.
### Platform events
Salesforce platform events are immutable, which means that you cannot update or delete them after they have been published. As a result, platform event syncs always use insert mode. This means that Hightouch ignores row changes and removals in your model's query results, and publishes a new platform event whenever a new row appears in your model results.
Before events appear in your Hightouch sync configuration, you first need to create platform events in your Salesforce instance. Hightouch then surfaces them within the sync configuration in a dropdown.
#### Record matching
Since platform events are **Insert** only, you don't need to match them to existing records.
#### Field mapping
You can map any column in your source to a platform event. Hightouch doesn't support creating platform event fields. When creating your event in Salesforce, ensure you're including all the relevant fields so that Hightouch can sync to them.
## Tips and troubleshooting
For Hightouch platform error codes related to Salesforce, see [Error codes:
Salesforce](/developer-tools/error-codes#salesforce).
### Sync volume and performance
Hightouch can process millions of rows to Salesforce in a single sync, given you have the available Salesforce API calls. In other words, sync volume limitations are a factor of your [Salesforce API request limits and allocations](https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_api.htm), rather than Hightouch. If you're syncing large volumes of data, you may want to finetune your [batching and parallelization configuration](#batching-and-parallelization) for optimal performance.
[Upsert mode](#objects) calls Salesforce's upsert endpoint, which can improve sync performance and lower API datapoint consumption since it only requires one single API request per batch of rows. However, there are still some additional API calls for [rejected row retries](/syncs/retries) and miscellaneous data that needs to be sent to Salesforce. This efficient Upsert mode only works when the Salesforce field used for record matching is an [explicit external ID field](#record-matching).
### Common errors
| Error message | Description |
| :---------------------------------------------- | :---------------------------------------------------------------------------------------------------------- |
| `ApiBatchItems Limit exceeded.` | Daily Bulk API limit exceeded. [More detail.](#apibatchitems-limit-exceeded) |
| `CANNOT_UPDATE_CONVERTED_LEAD` | Attempting to update a converted lead. [More detail.](#cannot_update_converted_lead) |
| `DUPLICATE_EXTERNAL_ID` | Duplicate IDs in external data. [More detail.](#duplicate_external_id) |
| `DUPLICATES_DETECTED` | Duplicate records in a sync batch. [More detail.](#duplicates_detected) |
| `INSUFFICIENT_ACCESS_ON_CROSS_REFERENCE_ENTITY` | Missing access to a cross-referenced object. [More detail.](#insufficient_access_on_cross_reference_entity) |
| `INSUFFICIENT_ACCESS_OR_READONLY` | Missing object or field permissions. [More detail.](#insufficient_access_or_readonly) |
| `INVALID_CROSS_REFERENCE_KEY` | Referenced record ID doesn't exist or is inaccessible. [More detail.](#invalid_cross_reference_key) |
| `INVALID_FIELD_FOR_INSERT_UPDATE` | Writing to a non-editable field. [More detail.](#invalid_field_for_insert_update) |
| `INVALID_OR_NULL_FOR_RESTRICTED_PICKLIST` | Invalid value for a restricted picklist. [More detail.](#invalid_or_null_for_restricted_picklist) |
| `INVALID_TYPE_ON_FIELD_IN_RECORD` | Type mismatch on a field value. [More detail.](#invalid_type_on_field_in_record) |
| `OAUTH_APP_BLOCKED` | Hightouch OAuth app blocked in Salesforce. [More detail.](#oauth_app_blocked) |
| `UNABLE_TO_LOCK_ROW` | Concurrent processes locking the same record. [More detail.](#unable_to_lock_row) |
| `ValidationError: Reference not found` | Lookup mapping references a nonexistent value. [More detail.](#validationerror-reference-not-found) |
See Salesforce's comprehensive [list of error messages and error codes](https://developer.salesforce.com/docs/atlas.en-us.api.meta/api/sforce_api_calls_concepts_core_data_objects.htm#i1421192) for more.
{/* */}
#### ApiBatchItems Limit exceeded
{/* */}
```
ApiBatchItems Limit exceeded.
```
**Cause:** Your Salesforce org has used up its daily ApiBatchItems Bulk API limit.
**Solution:** Check your [Salesforce Bulk API limits](https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_bulkapi.htm), review any other high-volume jobs running against the org, and retry the sync after the limit resets.
{/* */}
#### CANNOT_UPDATE_CONVERTED_LEAD
{/* */}
```
CANNOT_UPDATE_CONVERTED_LEAD
```
**Cause:** The sync is attempting to update a lead that has been [converted](https://help.salesforce.com/s/articleView?id=sf.leads_convert.htm&type=5) into a contact, account, or opportunity. Once converted, the lead record becomes read-only in Salesforce — it still exists in the Leads object but can no longer be updated.
**Solution:**
- Filter converted leads out of your model's query results. In Salesforce, the `IsConverted` field on the Lead object indicates whether a lead has been converted.
- If you need to update the resulting records, sync to the Contact, Account, or Opportunity object instead.
{/* */}
#### DUPLICATE_EXTERNAL_ID
{/* */}
```
DUPLICATE_EXTERNAL_ID
```
**Cause:** Records with the same IDs are coming from an external system. This can happen when a lead or contact was imported with the wrong External CRM ID, when you have null values in your primary key, or when the Salesforce field used for record matching contains duplicates.
**Solution:** Deduplicate your model's query results and ensure the field used for [record matching](#record-matching) contains unique, non-null values. In Salesforce, verify the external ID field doesn't already contain duplicate values.
{/* */}
#### DUPLICATES_DETECTED
{/* */}
```
DUPLICATES_DETECTED
```
**Cause:** The sync batch contains duplicate records. This can happen when you have null values in your primary key or when the Salesforce field used for record matching contains duplicates.
**Solution:** Ensure your model doesn't return duplicated rows. Check that the field used for [record matching](#record-matching) contains unique, non-null values.
{/* */}
#### INSUFFICIENT_ACCESS_ON_CROSS_REFERENCE_ENTITY
{/* */}
```
INSUFFICIENT_ACCESS_ON_CROSS_REFERENCE_ENTITY
```
**Cause:** The connected Salesforce user doesn't have sufficient access to a related record referenced in a lookup or relationship field.
**Solution:** Check that the connected Salesforce user has access to the object being cross-referenced in your sync. Review [field-level security](https://help.salesforce.com/s/articleView?id=sf.users_profiles_fls.htm&type=5), [sharing rules](https://help.salesforce.com/s/articleView?id=sf.security_sharing_rules.htm&type=5), and record ownership for the referenced records.
{/* */}
#### INSUFFICIENT_ACCESS_OR_READONLY
{/* */}
```
INSUFFICIENT_ACCESS_OR_READONLY
```
**Cause:** The Salesforce user connected to Hightouch lacks the required object-level or field-level permissions for the records being synced. This error can appear intermittently when some records in a batch require access that the user doesn't have.
**Solution:**
- Verify that the connected Salesforce user's profile has **read and write access** to the object being synced. Go to **Setup → Profiles** or **Setup → Permission Sets** in Salesforce and check the [object permissions](https://help.salesforce.com/s/articleView?id=sf.perm_sets_object_perms_edit.htm&type=5).
- Check [field-level security](https://help.salesforce.com/s/articleView?id=sf.users_profiles_fls.htm&type=5) for the specific fields in your sync mapping. The connected user must have write access to every mapped field, not just the object.
- If the error appears only for specific records, look for [record-level sharing rules](https://help.salesforce.com/s/articleView?id=sf.security_sharing_rules.htm&type=5), ownership restrictions, or org-wide defaults that may limit access to those records.
{/* */}
#### INVALID_CROSS_REFERENCE_KEY
{/* */}
```
INVALID_CROSS_REFERENCE_KEY
```
**Cause:** The sync is attempting to create or update a record that references a nonexistent or inaccessible record ID in a lookup or relationship field. Common causes include syncing IDs from a different Salesforce object, a different org (production vs. sandbox), or referencing records that have been deleted.
**Solution:**
- Verify that the referenced record IDs in your model exist in the target Salesforce org. IDs don't carry over between sandbox and production environments.
- Check that the connected Salesforce user has access to the referenced records. Record-level sharing rules or org-wide defaults can restrict visibility.
- Ensure lookup fields reference the correct object type — for example, an Account ID in a field that expects a Contact ID triggers this error.
{/* */}
#### INVALID_FIELD_FOR_INSERT_UPDATE
{/* */}
```
INVALID_FIELD_FOR_INSERT_UPDATE
```
**Cause:** The sync is attempting to write to a field that Salesforce doesn't allow to be inserted or updated. Salesforce has several non-editable field types that reject writes regardless of user permissions: formula fields, roll-up summary fields, auto-number fields, and system-managed fields like Created By, Created Date, Last Modified By, and Last Modified Date.
**Solution:**
- Open the [Object Manager](#inspect-objects-relationships-and-fields) in Salesforce and verify that each mapped field is editable — not a formula, auto-number, roll-up summary, or system-managed field.
- Check [field-level security](https://help.salesforce.com/s/articleView?id=sf.users_profiles_fls.htm&type=5) for the connected user's profile to confirm write access on every mapped field.
- If you're using [record types](https://help.salesforce.com/s/articleView?id=sf.user_recordtype.htm&type=5), ensure a default record type is assigned to the connected user's profile. A missing default record type can trigger this error on insert operations.
- If you need the data in a non-editable field, create a custom editable field on the same object and sync to that instead. You can then use Salesforce automation (flows or Apex triggers) to copy the value where needed.
{/* */}
#### INVALID_OR_NULL_FOR_RESTRICTED_PICKLIST
{/* */}
```
INVALID_OR_NULL_FOR_RESTRICTED_PICKLIST
```
**Cause:** The sync is attempting to write an invalid value into a restricted picklist field — for example, a value that isn't in the picklist's allowed values, a value that exceeds the field's length limit, or a data type mismatch.
**Solution:** Review the picklist field's settings in Salesforce (**Setup → Object Manager → [Object] → Fields & Relationships → [Field]**) and ensure the values in your model match the allowed picklist values exactly, including casing.
{/* */}
#### INVALID_TYPE_ON_FIELD_IN_RECORD
{/* */}
```
INVALID_TYPE_ON_FIELD_IN_RECORD
```
**Cause:** The sync is sending a value with an incompatible type — most commonly a `null` value to a Boolean field. This error can also occur when a [before-save update flow](https://help.salesforce.com/s/articleView?id=sf.flow_concepts_trigger_record.htm&type=5) sets a value that conflicts with the value sent from Hightouch.
**Solution:** Ensure your model doesn't send `null` to Boolean fields in Salesforce. If a before-save flow is conflicting, disable or modify the flow.
{/* */}
#### OAUTH_APP_BLOCKED
{/* */}
```
OAUTH_APP_BLOCKED
```
**Cause:** The Hightouch connected app is blocked in Salesforce.
**Solution:**
1. Go to **Setup → Connected Apps OAuth Usage** in Salesforce and ensure Hightouch is unblocked. The user verifying OAuth permissions must be a Salesforce admin.
2. Grant Hightouch read and write access to all the objects and fields you intend to sync.
3. Ensure the connected user's profile has the **View Setup and Configuration** permission enabled under **Administrative Permissions**.
{/* */}
#### UNABLE_TO_LOCK_ROW
{/* */}
```
UNABLE_TO_LOCK_ROW
```
**Cause:** Multiple processes — including Hightouch syncs — are trying to modify the same record at the same time. Salesforce can't work on a row while another process is touching it.
**Solution:**
- If there is an [Apex trigger or workflow](#inspect-apex-triggers-and-flows) that conflicts with your sync, pause the process or rerun the sync after the process finishes. This error is often temporary and resolves on retry.
- Lower your [parallelization](#batching-and-parallelization) to five or less to minimize the chances of two requests modifying the same record simultaneously.
- Identify long-running processes in your Salesforce environment that could be holding locks. Optimize them or break them into smaller batches.
- Ensure your [model](/getting-started/concepts#models) doesn't return duplicated rows — duplicates can also trigger this error.
See [Salesforce's Help Center](https://help.salesforce.com/s/articleView?id=000387767&type=1) for more information.
{/* */}
#### ValidationError: Reference not found
{/* */}
```
ValidationError: Reference not found
```
**Cause:** A [relationship lookup mapping](#relationships-object-id-lookup) in the sync configuration can't find the referenced value in Salesforce. The error message contains the Salesforce field name used for the lookup and the value that failed.
**Solution:** Ensure the model column used for the lookup mapping only contains values that already exist in Salesforce. Check for typos, deleted records, or ID mismatches.
### Missing or non-updating fields in sync configuration
If fields are missing from the sync configuration dropdown, or mapped fields aren't updating in Salesforce, check the following:
- **Field-level security (FLS):** Go to **Setup → Object Manager → [Object] → Fields & Relationships** in Salesforce and verify that the connected user's profile has read and write access to each field. FLS restrictions can silently prevent fields from appearing in Hightouch or prevent updates from landing.
- **Refresh fields:** If you recently added or changed fields in Salesforce, go to your sync configuration in Hightouch and select **Refresh** next to the destination field dropdown to pull the latest schema.
- Ensure fields you want to match against are set as external identifiers.
- If you authenticated with your personal user ID, make sure you have permission to view and update the fields you want.
- If you're missing the Salesforce ID field for matching on, make sure you're in the correct sync mode. Matching on Salesforce ID [isn't supported for upserting](#supported-syncing)—use update mode instead.
- If you set your sync to [Update mode](#objects), the [field mapping](#field-mapping) section only lists [`updateable`](https://developer.salesforce.com/docs/atlas.en-us.uiapi.meta/uiapi/ui_api_responses_field.htm) Salesforce fields.
- Verify that field names in your model still match Salesforce. If a field was renamed or deleted in Salesforce, the mapping breaks silently.
When syncing to multi-objects, for example, **Contact or Lead** or **Account or Lead**, ensure _both_ object types include a shared unique field marked as a [`Unique ID`](https://help.salesforce.com/s/articleView?id=000385174&type=1) or a unique `external ID`. Without meeting this requirement, you won't see any fields populate for record matching.
### Duplicate records in Salesforce
Make sure that the Salesforce field used for record matching is an **explicit external ID** field. You can learn more about this in the [record matching](#record-matching) section.
### Limitations on sandboxes
If you are building Hightouch syncs to Salesforce sandbox environments and encounter processing delays or errors, check the version of your Salesforce sandbox and upgrade to a higher version if necessary. You can find more information here: [Salesforce Sandbox Licenses and Storage Limits by Type](https://help.salesforce.com/s/articleView?id=sf.data_sandbox_environments.htm&type=5)
### Enhanced domains
If you are enabling [enhanced domains](https://help.salesforce.com/s/articleView?id=sf.domain_name_enhanced.htm&language=en_US&type=5) for your Salesforce instance, you need to [reauthorize](#connect-to-salesforce) your Salesforce destination to avoid sync disruptions.
To do so, go to your Salesforce destination configuration page, and click **Log in to Salesforce** under **Reauthorize connection**.
To learn more about enhanced domains, check out [Salesforce's enhanced domain FAQ](https://help.salesforce.com/s/articleView?id=000393816&type=1).
### Live debugger
The debugger isn't available for **All** and **Insert** sync modes.
### Sync alerts
---
## Salesforce Commerce Cloud
**URL:** https://hightouch.com/docs/destinations/salesforce-commerce-cloud
**Description:** Keep your storefront up-to-date with the latest customer info from your data warehouse
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ------------- | ---------------------------------------------------------------- | ---------------------- | ---------------------------------------------------------------------------------------------------------- |
| **Customers** | Sync data from any source to Salesforce Commerce Cloud customers | Upsert, Update, Insert | [Customers](https://developer.salesforce.com/docs/commerce/commerce-api/references/customers?meta=Summary) |
## Connect to Salesforce Commerce Cloud
If you don't have an API client already, follow these [steps](https://developer.salesforce.com/docs/commerce/commerce-api/guide/authorization-for-admin-apis.html#create-an-api-client) to create one.
Make sure you include the `sfcc.customerlists.rw` scope to your list of **Allowed Scopes**.
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Salesforce Commerce Cloud** and click **Continue**. You can then authenticate Hightouch to **Salesforce Commerce Cloud**.
Enter the following fields into Hightouch:
- **Organization ID**
- **Short code**
- **Site ID**
- **Client ID**
- **Password**
## Sync configuration
Once you've set up your Salesforce Commerce Cloud destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data.
Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Salesforce Commerce Cloud destination you want to sync to.
### Syncing customers
Sync data from any source to Salesforce Commerce Cloud customers.
#### Record matching
To match rows from your model to customers in Salesforce Commerce Cloud, you need to select a model column and corresponding Salesforce Commerce Cloud field. You can match on any of the following Salesforce Commerce Cloud fields:
- **customerNo (customer number)**
- **email**
If you decide to match on **email**, you can still map the `customerNo` field if you prefer to generate this yourself.
If `customerNo` is not mapped, Salesforce Commerce Cloud generates one for you when inserting new customers.
#### Field mapping
Hightouch lets you sync customer fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom customer fields.
Ensure the data types in your model match the data types of the fields you want to sync to.
You can refer to this API [doc](https://developer.salesforce.com/docs/commerce/commerce-api/references/customers?meta=createCustomerInCustomerList) for the customer's schema.
You can upsert customer addresses by mapping the `addresses` field in the mappings section.
Hightouch expects an array of address objects from your model. You can refer to this API [doc](https://developer.salesforce.com/docs/commerce/commerce-api/references/customers?meta=createAddressForCustomerInCustomerList)
on how to build your address object in your model or in our [inline mapper](/syncs/mapping-data#inline-mapping).
For each customer record, Hightouch gets all addresses in Salesforce Commerce
Cloud tied to the customer to match with the record's addresses. Hightouch
updates the ones that exist and inserts the ones that don't.
You can map your custom fields by entering your custom field's key, which is case sensitive (e.g. `customField`), in the custom mappings section.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ------------------------------------------------------------------------- |
| **Do nothing** | Keep the customer in Salesforce Commerce Cloud with all its synced fields |
| **Delete** | Delete the synced customers from Salesforce Commerce Cloud |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Salesloft
**URL:** https://hightouch.com/docs/destinations/salesloft
**Description:** Empower your Salesloft usage with up to date data from your data warehouse
**Section:** Destinations
## Setup
Salesloft requires an API key. To find the API key, navigate to the [Salesloft Dashboard Settings](https://accounts.salesloft.com/oauth/applications).
In the Salesloft settings, an API Key can be created on this page.
If an API key has never been created, you can create one here.
## Syncing
Hightouch supports syncing to the following Salesloft resources:
- `People`
- `Accounts`
### Sync modes
This integration supports Upsert and Update mode.
In the Upsert mode, new people or accounts will be inserted into Salesloft and all attributes will be kept up-to-date within Salesloft.
In Update mode, people and accounts will be kept up-to-date, but new entries will not be inserted.
### Record matching
Person records can be matched from your source to your Salesloft workspace by the person's email in the Upsert mode.
Account records can be matched from your source to your Salesloft workspace by your Salesloft Account Domain or CRM ID in the Upsert mode.
In Update mode, records can be matched from your source to your Salesloft workspace by ID.
### Field mapping
You can sync columns from your source to Salesloft's default and custom fields. Hightouch automatically detects existing custom fields from your Salesloft account.
You can add custom attributes for people and accounts in your Salesloft account.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Samsung Ads
**URL:** https://hightouch.com/docs/destinations/samsung-ads
**Description:** Empower your advertising team to run targeted campaigns across Samsung Ads’ premium inventory.
**Section:** Destinations
## Overview
Deliver more relevant ads across Samsung’s premium, brand-safe inventory and ad
formats that are not available through programmatic buys. Reach addressable
audiences across their ecosystem of Smart TVs, Smart Phones, and Digital Signage
by matching your first-party identifiers with Samsung IDs using Match Booster.
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| ------------- | ---------------------------------- | -------------------- | ------------------------------------------------------------------------------ |
| [**Audiences**](#syncing-custom-audiences) | Create and update custom audiences | All | [Samsung DSP API docs](https://help.dsp.samsungads.com/docs/api-documentation) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Samsung Ads
This integration requires prior approval from the Samsung Ads team and the
Hightouch team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your Samsung rep who will verify approval. Once we have confirmed,
we will unlock the integration for you to connect.
### Syncing custom audiences
You can select the name of the audience you plan on delivering to Samsung Ads.
You will need to reach out to us for every new audience you want to target. We
will set that up for you before syncing.
#### User identifiers
To identify which users to add or update in an audience, select model columns
and the corresponding Samsung Ads fields. You can match on any of the following user
fields:
- Hashed email (SHA256)
- MAID
#### Delete behavior
The only supported sync mode is `all` mode, meaning that each sync run will refresh
the entire audience. Users removed from your audience will automatically be dropped
with each refresh - only present users will be in Samsung Ads's audience.
## Tips and troubleshooting
### Refresh rate
Samsung Ads has a max refresh rate of monthly. You should not have your Samsung
Ads syncs running more than every 4 weeks.
### Matched device count
Match rates are not available to preview in the Hightouch platform. Contact your
Samsung rep for insights into match rates and audience reach.
Keep in mind that audience counts reflect the number of Samsung devices match,
not the number of users matched. Also keep in mind that matching occurs at the
household level, and that a single household may own multiple devices.
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Live debugger
### Sync alerts
---
## Sanity
**URL:** https://hightouch.com/docs/destinations/sanity
**Description:** Empower your content management through Sanity - a customizable all-code backend for content-driven websites and apps.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
--------------|-------------------------------------|------------------------|------------------------------------------------------------
**Documents** | Sync data from any source to Sanity | Upsert, Update, Insert | [Documents docs](https://www.sanity.io/docs/http-mutations)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Sanity
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Sanity** and click **Continue**. You can then authenticate Hightouch to **Sanity**.
Enter the following fields into Hightouch:
- **Project ID**: Find your Project ID on the project's main dashboard.
- **API token**: Generate or find your API token for your project under the API tab.
## Sync configuration
Once you've set up your Sanity destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Sanity destination you want to sync to.
### Syncing documents
Sync data from any source to Sanity documents. Define document schemas and `types` in your Sanity project, and sync document properties and fields to Sanity using Hightouch.
#### Record matching
To match rows from your model to documents in Sanity, you need to select the model column that contains values that match the **ID** field. You can match on any of the following Sanity fields:
- ID
- *custom matching field (recommended unique)*
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Sanity automatically generates an identifier for every new record synced.
#### Field mapping
Hightouch lets you sync document fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom document schema fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
Prior to syncing, define your document type's schema in your Sanity project. To write to Sanity documents, map your source fields
to your defined custom document schema fields.
#### Update mode and Upsert mode
Hightouch leverages Sanity's [patch mutation API ](https://www.sanity.io/docs/content-lake/http-patches) to support both Update and Upsert modes. The integration currently uses the `set`
operation, which replaces the values of the fields mapped in your configuration.
We recommend maintaining a clear separation between fields managed by Hightouch and those edited manually in Sanity. Any field mapped to
Hightouch will be overwritten during updates and upserts. To preserve human-managed content, exclude those fields from your Hightouch mappings.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## SAP HANA
**URL:** https://hightouch.com/docs/destinations/sap-hana
**Description:** Move your data into HANA
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
|--------------|-------------------------------------------|----------------------|
| Any data set | Sync data from any source to a HANA table | Upsert |
## Connect to SAP HANA
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **SAP HANA** and click **Continue**. If you're not using a [tunnel](#tunneling), you can then authenticate Hightouch to SAP HANA by entering the following fields:
- **Host**: The hostname or IP address of your SAP HANA server. You don't need to include the `https://`.
- **Port**: The port number of your SAP HANA server.
- **User**: The user that has access through Hightouch to the database and tables you want to sync to.
- **Password**: The password for the user specified above.
To ensure your [credentials](#required-permissions) are correct, click **Test connection**. This confirms if Hightouch can connect to your database by running a basic `SELECT` query.
## Record matching
Hightouch requires a unique identifier in the destination HANA table to add, remove, and update rows. The column must be either a `UNIQUE` or `PRIMARY KEY` column. At least one Hightouch field must map to a unique column.
If you map to an incompatible column, Hightouch will fail the sync with an error message.
## Column types
Hightouch works out of the box with all standard column types.
If you see type errors, it may be because your SQL query is producing the wrong format. Message us on Intercom if you need help.
## Required permissions
Your HANA credentials must be able to:
- `MERGE INTO`, `UPDATE`, `INSERT` and `DELETE` rows from the destination table.
- Create a local temporary table
- Perform select operations on system tables:
- `SCHEMAS`
- `TABLES`
- `TABLE_COLUMNS`
- `CONSTRAINTS`
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## SAP IBP
**URL:** https://hightouch.com/docs/destinations/sap-ibp
**Description:** Forecast and plan across your business by loading fresh data into SAP IBP
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes |
| ------------ | -------------------------------------------------- | -------------------- |
| **Entities** | Sync data to any of your OData entities in SAP IBP | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to SAP IBP
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **SAP IBP** and click **Continue**. You can then authenticate Hightouch to **SAP IBP**.
Enter the following required fields into Hightouch:
- **Base URL**: The full URL for the data service. For example, `https://myibp.com/sap/opu/odata/sap/IBP/MASTER_DATA_API_SRV/`.
- **Username**: The username for the communication arrangement, rather than your personal username.
- **Password**: The password for the username.
## Sync configuration
Once you've set up your SAP IBP destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the SAP IBP destination you want to sync to.
### Syncing entities
Sync data to any of your OData entities in SAP IBP.
#### Record matching
You can match rows from your model to entity in SAP IBP on any column in your model and the key property of your entity in SAP IBP. Ensure the data types of the model column and SAP IBP field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync entity fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to any available properties.
Ensure your model columns data types match the data types of the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## SAP S/4HANA
**URL:** https://hightouch.com/docs/destinations/sap-s4hana
**Description:** Explore top predictions for intelligent ERP.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
| ---------- | -----------------------------------------------------| ---------------------- | ------------------------------------------------------------------------------------------------------- |
| **Sales Orders** | Sync data from any source to SAP S/4HANA as sales orders | Upsert, Update, Insert | [Sales Orders](https://api.sap.com/api/API_SALES_ORDER_SRV/resource/Sales_Order_Header)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to SAP S/4HANA
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **SAP S/4HANA** and click **Continue**.
Choose the environment that you would like to connect to SAP S/4HANA in. This will be either **Sandbox** or **Production**.
For **Sandbox** environments, input your API key. You can find your sandbox API key in any of the API resource pages, such as [this page for sales orders](https://api.sap.com/api/API_SALES_ORDER_SRV/tryout).
Make sure that your API key has read and write access to your SAP S/4HANA instance.
For **Production** environments, you will need to set up the following in your Communication Management app:
- Communication Users
- Communication Systems
- Communication Arrangements
Use these guides from SAP S/4HANA to create communication arrangements with the necessary permissions and a user:
- [Maintaining communication users](https://help.sap.com/docs/SAP_S4HANA_CLOUD/0f69f8fb28ac4bf48d2b57b9637e81fa/eef80dda3867461c92ac1273689ed36f.html)
- [Creating communication arrangement](https://help.sap.com/docs/SAP_S4HANA_CLOUD/0f69f8fb28ac4bf48d2b57b9637e81fa/fab3fd449cf74c6384622b98831e989e.html)
- [Introductory guide to testing your API](https://help.sap.com/doc/e9d1daf327b943c99ae274c8a9871509/Cloud/en-US/Testing%20SAP%20S4HANA%20Cloud%20API%20Services.pdf)
Once you have completed setting up the above, enter the following fields to connect to SAP S/4HANA:
- **Host**: You can get this from the API environment that you can see in the communication arrangement you activated earlier. In your SAP S/4HANA URL, this is the first section of the URL after the `https://` prefix, i.e https://HOSTNAME.s4hana.ondemand.com.
- **Port**: The default value is 443. You can also find this value the communication arrangement.
- **User**: Username of the user created in Communication Management.
- **Password**: The password for the user specified above.
## Sync configuration
Once you've set up your SAP S/4HANA destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the SAP S/4HANA destination you want to sync to.
### Sync mode
Hightouch supports **Upsert**, **Update**, and **Insert** mode with the SAP ODATA API for Sales Orders.
### Field mapping
Hightouch supports syncing the Sales Order object and its nested fields.
Use Hightouch's object inline mapping to construct SAP S/4HANA's object values from your source.
### Record matching
In **Update and Upsert mode**, Hightouch requires a unique identifier in your model with item IDs from SAP S/4HANA. Rows without item IDs will fail to sync.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Segment
**URL:** https://hightouch.com/docs/destinations/segment
**Description:** Sync rich customer data from your data warehouse to hundreds of SaaS tools via Segment's large integration library
**Section:** Destinations
## Overview
Besides the ability to track first-party customer data, Segment offers robust integrations with the most popular SaaS tools. With our Segment integration, you can automatically sync enriched customer data from your data warehouse with Segment to send this data to downstream tools that Hightouch doesn't already integrate with.
## Supported syncing
|Sync Type|Description|Supported Sync Modes |API Reference|
|--|--|--|--|
|**Track \(Generic\) events**|Sends events of different names when a record is added, changed, or removed in the query results|Insert|[Track specs](https://segment.com/docs/connections/spec/track/)|
|**Track \(Segment Entered / Exited\) events**|Sends the event "Segment Entered" when a record is added and "Segment Existed" when a record is removed|Insert|[Track specs](https://segment.com/docs/connections/spec/track/)|
|**Identify events**|Sends an identify event with related properties of the user|Insert|[Identify specs](https://segment.com/docs/connections/spec/identify/)|
|**Group events**|Sends a group event to associate a user to a group|Insert|[Group specs](https://segment.com/docs/connections/spec/group/)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Segment
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Segment** and click **Continue**. You can then authenticate Hightouch to Segment using a Segment **Write Key**.
To retrieve the write key for your Segment destination in Hightouch:
1. Create a new HTTP API source in Segment using [these instructions](https://segment.com/docs/connections/sources/#create-a-source).
2. Navigate to **Source Settings > API Keys** and copy the **Write Key**.
## Sync configuration
Once you've set up your Segment destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Segment destination you want to sync to.
### User and anonymous IDs
You need to include send either the user or anonymous ID to Segment to identify the user associated with the event.
### Field mapping
You can sync columns from your source into user \(identify\) or event \(track\) properties.
### Sending initial results (Track events only)
For the two track event types, Hightouch lets you choose whether to:
- Send the initial set of results as track events as Entered / Added events
- Or, only send the changes between the results in the subsequent syncs as track events
Checking the **Send events for initial result set** checkbox results in a large amount of events being sent in the first sync.
Hightouch always send the initial results set when sending Identify and Group events.
### Event names and timestamps (Generic track events only)
For the generic track mode, there are further settings that are needed to determine which event name Hightouch sends to Segment for each added, changed, and removed record.
In the preceding screenshot, Hightouch sends Segment:
- A "User Signed Up" event when a record is added to the query results
- A "User Changed Subscription" event when a record is changed in the query results
- A "User Deleted Account" event when a record is removed in the query results
Hightouch allows you to send a timestamp that the event occurred for generic track events. This is useful for backfilling historical events.
Hightouch also lets you to send a message ID for each record to ensure that the event is deduplicated within Segment in the case that the same event erroneously gets sent twice.
## Tips and troubleshooting
### Common errors
#### Large payload issues
Segment returns a `400 Bad Request` error if your batch payload exceeds the limitations. You can find more information in [Segment's documentation](https://segment.com/docs/connections/sources/catalog/libraries/server/http-api/#errors). Adjust your batch configuration accordingly.
### Live debugger
### Sync alerts
---
## Twilio SendGrid
**URL:** https://hightouch.com/docs/destinations/sendgrid
**Description:** Run better email campaigns on SendGrid with up-to-date customer data from your data warehouse
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|-------------|-----------------------------------------------------------------------------|-------------------------------------|----------------------------------------------------------------------------------------------|
| **Contacts**| Sync data from any source to Contacts in SendGrid | Upsert | [Contacts docs](https://docs.sendgrid.com/api-reference/contacts/add-or-update-a-contact) |
| **Mail** | Trigger one-to-one transactional emails for each record in your data model | Triggered Sends (one row = one email) | [Mail Send API](https://docs.sendgrid.com/api-reference/mail-send/mail-send) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to SendGrid
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **SendGrid** and click **Continue**. You can then authenticate Hightouch to **SendGrid** by entering a SendGrid **API Key**.
To create an API Key in SendGrid, follow the steps outlined in the [SendGrid documentation](https://docs.sendgrid.com/ui/account-and-settings/api-keys#creating-an-api-key).
#### Permissions
The permissions the API key needs depend on what you plan on syncing.
| Sync Type | Required Permissions (Full Access) | Optional Permissions (Read Access) |
| ------------ | ---------------------------------- | -----------------------------------------------------------------------------------|
| **Contacts** | `Marketing` | - |
| **Mail** | `Mail Send` | `Template Engine` (for email template), `Suppressions` (for unsubscribe groups) |
## Sync configuration
Once you've set up your SendGrid destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the SendGrid destination you want to sync to.
### Syncing contacts
Upsert is the only available sync mode for the SendGrid destination. Use this sync mode to update existing contacts and insert new ones into SendGrid.
#### Record matching
To match rows from your model to contacts in SendGrid, you need to select the model column that contains your contacts' **email**.
SendGrid's `email` field is case-sensitive. Make sure the emails in your model follow a case convention, for example, all lowercase, to avoid duplicate contacts. For example, if your model has contact data for the email addresses `john@example.com` and `John@example.com`, this would create two separate contacts.
#### Field mapping
You can sync columns from your model to SendGrid default and custom fields.
## Tips and troubleshooting
### Common errors
#### Invalid list id
When you run a sync to SendGrid for the first time, Hightouch creates a new contact list. You can find this list on the **Contact Lists** page in your SendGrid workspace. If you delete this contact list, future sync runs will reject all rows with the `invalid list id` error.
To resolve this issue, create a new sync. This allows Hightouch to create a new contact list, which must exist for the sync to run successfully.
### Live debugger
### Sync alerts
---
## ServiceNow
**URL:** https://hightouch.com/docs/destinations/servicenow
**Description:** Improve your ITSM workflow by having your data in ServiceNow
**Section:** Destinations
## Overview
The ServiceNow Platform is a scalable cloud service that offers a range of applications, workflows and processes. Sharing common architecture and data, the platform incorporates a variety of ready-made suites catering for specific business areas.
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|**Standard Objects**|Sync data from any source to records in ServiceNow|Upsert, Update|
For more information about sync modes, check out the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Getting started
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select ServiceNow and then enter the name of your ServiceNow instance and credentials for an integration user.
Your integration user should have the following roles:
- `rest_service`
- `web_service_admin`
- `rest_api_explorer`
- Read access to the `sys_db_object` table
- Read access to the `sys_glide_object` table
- Read access to the `sys_dictionary` table
- Create/Write/Read access to the table being synced to
## Syncing data
Once you've set up your ServiceNow destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the ServiceNow destination you want to sync to.
### Record matching
You can match records from your source to your ServiceNow instance using any unique field in ServiceNow. This can be a custom field or an out-of-box field, as long as it's guaranteed to be unique. If the selected field isn't unique, the incorrect record may be updated.
If you're using the `sys_id` field, the value from the source must be a 32-character, lowercase, alphanumeric string.
### Field mapping
You can also sync columns from your source to any of the fields in the selected ServiceNow table.
## Tips and troubleshooting
### Maximum exection time exceeded
ServiceNow has some default request limits that can be configured in the `Transaction Quota Rules (sysrule_quota)` table.
If you're seeing `Transaction cancelled: maximum execution time exceeded` try decreasing the batch size in your sync configuration, or increasing the timeout duration of `REST Batch API request timeout` and `REST Table API request timeout` in the Transaction Quota Rules table.
### Common errors
### Live debugger
### Sync alerts
---
## Salesforce Marketing Cloud
**URL:** https://hightouch.com/docs/destinations/sfmc
**Description:** Personalize campaigns in Salesforce Marketing Cloud with rich customer data from your data warehouse
**Section:** Destinations
# Overview
Salesforce Marketing Cloud (SFMC) is a digital marketing platform that enables businesses to create personalized customer journeys across email, mobile, social, web, and more. This integration allows you to sync your data from Hightouch to SFMC to power personalized marketing campaigns, manage contact data, automate customer journeys, and trigger communications.
With the Hightouch SFMC integration, you can:
- Sync data to SFMC Data Extensions for segmentation and personalization
- Update contact information for better targeting
- Manage entry and exit from customer journeys based on your data
- Trigger email and SMS messages based on customer behaviors
This documentation will guide you through setting up the connection, configuring syncs, and troubleshooting common issues.
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| :-------------------------------------- | :----------------------------------------------------------------------------------------------------------------- | :------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [**Data Extensions**](#data-extensions) | Sync records to new or existing Data Extensions, using the SOAP API and optionally file uploads | Upsert, Mirror | [Manage Data Extensions ↗](https://help.salesforce.com/s/articleView?id=sf.mc_co_salesforce_data_extensions.htm&language=en_US&type=5) |
| [**Contacts**](#contacts) | Sync records to Marketing Cloud contacts | Upsert | [Manage SOAP API objects ↗](https://developer.salesforce.com/docs/marketing/marketing-cloud/guide/soap_web_service_objects.html) |
| [**Journeys**](#journeys) | Trigger API events to enter and exit contacts from journeys as they're added to and removed from your query result | Add, Remove | [Fire entry events ↗](https://developer.salesforce.com/docs/marketing/marketing-cloud/guide/how-to-fire-an-event.html?q=entry) and [exit contacts ↗](https://developer.salesforce.com/docs/marketing/marketing-cloud/guide/contactExitRequest.html) |
| [**Triggered send**](#triggered-send) | Send email and SMS messages to individual subscribers | Trigger | [Create triggered sends ↗](https://developer.salesforce.com/docs/marketing/marketing-cloud/guide/creating_a_triggered_send_using_the_async_api.html) |
## Salesforce Marketing Cloud setup
Before you connect Hightouch to Salesforce Marketing Cloud (SFMC), you need to create an installed package in SFMC for Hightouch to use.
If you want to sync records to Data Extensions using [either mirror mode or upsert mode with file uploads](#data-extensions), you also need to [create an FTP user](#create-ftp-user).
### Create installed package in SFMC
1. Log in to your SFMC instance. Go to **Setup**.
2. In the left sidebar, go to **Apps → Installed Packages** and click **New**.
3. Name your package, for example, "Hightouch Integration." Optionally provide a description, and click **Save**.
4. Click **Add Component** and select **API Integration** as the component type. Click **Next**.
5. Select **Server-to-Server** as the integration type and click **Next**.
#### Permissions
These are the minimum scopes for syncing to Contacts, Journeys, and Data Extensions.
Carefully select the appropriate permissions based on your specific use case.
- **Automation > Journeys**: Read, Write, Execute, Activate/Stop/Pause/Resume/Send/Schedule
- **Contacts > List and subscribers**: Read, Write
- **Data > Data Extensions**: Read, Write
- **Provisioning > Accounts**: Read
If you want to sync to [Data Extensions](#data-extensions) using mirror mode or upsert mode with
[file uploads](#automatic-file-import-via-file-uploads), grant these additional permissions:
- **Data > File Locations**: Read, Write
- **Automation > Automations**: Read, Write, Execute
If you want to trigger sends, grant these additional permissions:
- **Channels > Email**: Read
- **Channels > SMS**: Read
If you would like to grant a custom subset of scopes for just the SFMC
features you want to use, see the section on [custom scopes](#custom-scopes).
Once you save your permissions, you should see a new API Integration component in your package.
With the API Integration created, copy the **Client ID**, **Client Secret**, and subdomain to enter into Hightouch.
The subdomain is the part of the **Authentication base URI** after `https://`
and before `.auth.marketingcloudapis.com/`. For example, if your
Authentication base URI is
`https://mc6vgk-yfjd8879pq4v.auth.marketingcloudapis.com/`, your subdomain is
`mc6vgk-yfjd8879pq4v`.
### Create FTP user
You only need to create a FTP user if want to sync [Data
Extensions](#data-extensions) via file uploads. Using File Upload mode can
optimize sync performance, especially on high volume syncs (over 1 million
records). Learn more about how file upload protocol works and its tradeoffs in
the [file uploads section](#automatic-file-import-via-file-uploads). You will
need a SFMC FTP user even if you are using a blob storage file location.
If your SFMC instance has parent-child Business Units and you want to sync to
a child Business Unit using FTP, you must create the FTP user in the child
Business Unit. Creating the FTP user in the parent Business Unit will not work
for syncing to child Business Units.
1. In your SFMC instance, go to **Setup → Administration → Data Management → FTP Accounts** and click **Create User**.
2. By default, the **FTP Username** is your [Marketing Cloud Member ID (MID)](https://trailhead.salesforce.com/content/learn/modules/marketing-cloud-developer-basics/learn-administration-basics), including the current parent account or child business unit. You can keep the default value.
3. Enter an **Email Address** for the new FTP user. This can be the email account you use to log in to SFMC or another one. You won't need this address to connect to Hightouch.
4. Enter a **Password** for the user. Password complexity requirements combine Marketing Cloud password policy and server-side FTP password requirements. These policies require a minimum of 12 characters and no reuse of the five most recent passwords. Reenter the password in **Repeat Password**.
5. The next screen prompts you to choose your authentication method.
- If you want to use a password, keep this selection.
- If you want to authenticate Hightouch to SFMC using an SSH key, select **SSH Key**. Follow [SFMC's instructions](https://help.salesforce.com/s/articleView?id=sf.mc_overview_key_create_ssh_for_ftp.htm&type=5) for how to add a key. Enter the public key into the SFMC and keep the private key for when you [connect to Hightouch](#connect-to-sfmc).
6. On the next screen, select permissions for your FTP user. At a minimum, provide the user **Upload** access to the **Import** directory.
7. Allowlisting IP addresses is optional. For additional security, you can enter [Hightouch's IP addresses](/security/networking#ip-addresses). Click **Next**.
8. Select a **Home Folder**. We recommend setting the home folder to the root directory. If you choose to specify a restricted scope, ensure that this FTP user has access to the **Import** directory.
9. Make sure to copy your FTP Username and either password or SSH private key for entry into Hightouch. Click **Save**.
## Connect to SFMC
You can now connect SFMC to Hightouch. Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button.
Select **Salesforce Marketing Cloud** and click **Continue**.
### Authentication
You can authenticate Hightouch to SFMC by entering the **Client ID**, **Client Secret**, and **Subdomain** you copied during [package creation](#create-installed-package-in-sfmc).
The subdomain is the part of the **Authentication base URI** after `https://`
and before `.auth.marketingcloudapis.com/`. For example, if your
Authentication base URI is
`https://mc6vgk-y5h9sgf6c5t8vxs09pq4v.auth.marketingcloudapis.com/`, your
subdomain is `mc6vgk-y5h9sgf6c5t8vxs09pq4v`.
### FTP credentials
This section is only required if you want to [sync records to Data Extensions
using mirror mode or upsert mode with file
uploads](#automatic-file-import-via-file-uploads).
Enter your FTP user credentials:
- **FTP Username**: typically a numeric Marketing Cloud MID
- **Password** _or_ **private key**
- To use a private key, you must have [uploaded the corresponding public key in SFMC](#create-ftp-user)
- If using a private key, you must enter the complete private key, starting with `-----BEGIN...PRIVATE KEY-----`.
- If your private key requires a passphrase, enter it as well
Incorrect FTP credentials will cause file upload syncs to fail. Double-check
your credentials if you encounter connection issues.
### File location
You only need to set up a file location in SFMC if you are syncing Data
Extensions with file uploads.
Hightouch supports the following file location types:
- ExactTarget Enhanced FTP Import Directory (default)
- Relative Location under FTP Site
- Google Cloud Storage
- Amazon S3
#### ExactTarget Enhanced FTP Import Directory
For **ExactTarget Enhanced FTP**, if you have chosen a different External Key for this location, enter the External Key to ensure Hightouch is able to access the uploaded files.
#### Relative Location under FTP Site
For **Relative Location under FTP Site**, enter the Relative Folder Path and the External Key to ensure Hightouch is able to access the uploaded files.
If your FTP user has a Home Folder other than the root directory, enter the
file location relative to the FTP user's home folder in your Hightouch
destination configuration.
#### Google Cloud Storage
To use the Google Cloud Storage file location in SFMC you will need:
- Google Cloud Storage bucket
- Google Service Account with **read** access to your bucket. This will be used by SFMC to read from your bucket.
- Google Service Account with **write** access to your bucket. This will be used by Hightouch to upload files.
If you do not have a Google Service Account, you can follow the setup instructions in our [Google Cloud Platform (GCP) documentation](/security/gcp).
To set up a Google Cloud Storage file location in SFMC:
1. Go to **Setup → Data Management → File Locations** and create a new file location.
2. Configure your file location's name and external key.
3. Choose Google Cloud Storage for Location Type.
4. Enter your GCS bucket name.
5. If you would like for files to be uploaded to a specific path in your bucket, enter the path as the **Relative Location** in your file location.
6. Upload your Google Service Account credentials. Ensure that at a minimum this role has read access to your bucket.
In Hightouch, select the Cloud Credentials you'd like to use to write to your bucket. Ensure this account has write access to your bucket.
Enter your file location External Key, GCP project ID, bucket name, and relative path.
Test your connection to ensure you've successfully connected to SFMC's API, FTP, and your GCS bucket.
#### Amazon S3
To use the Amazon S3 file location in SFMC you will need:
- Amazon S3 bucket
- AWS Credentials with **read** access to your bucket. This will be used by SFMC to read from your bucket.
- AWS Credentials with **write** access to your bucket. This will be used by Hightouch to upload files.
If you do not have AWS Credentials, you can follow the setup instructions in our [AWS documentation](/security/aws).
To set up an AWS S3 file location in SFMC:
1. Go to **Setup → Data Management → File Locations** and create a new file location.
2. Configure your file location's name and external key.
3. Choose Amazon Simple Storage Service for Location Type.
4. Choose whether you'd like to authenticate with an Access Key or Access Key with IAM role and enter your AWS credentials. Ensure that at a minimum this account has read access to your bucket.
5. Enter your S3 bucket name.
6. If you would like for files to be uploaded to a specific path in your bucket, enter the path as the **AWS Relative Location** in your file location.
In Hightouch, select the Cloud Credentials you'd like to use to write to your bucket. Ensure this account has write access to your bucket.
Enter your file location External Key, AWS region, bucket name, and relative path.
Test your connection to ensure you've successfully connected to SFMC's API, FTP, and your S3 bucket.
### Custom scopes
This section is for advanced users who need specific permission scopes for
their integration.
By default, Hightouch requests permission for Journeys, Contacts, and Data
Extensions. If you want to use a subset of these features, you can grant a
custom subset of scopes in your SFMC package. After you do this, enter a
space-separated list of scopes in the **Custom Scopes (advanced)** field in the
Hightouch destination configuration.
For example, if you want to sync to Data Extensions via the SOAP API, you
can grant the following scopes in your SFMC package:
- **Data > Data Extensions**: Read, Write
- **Provisioning > Accounts**: Read
Then, enter the following scopes in the Hightouch destination configuration:
```
accounts_read data_extensions_read data_extensions_write
```
See the [SFMC docs for a full list of
scopes](https://developer.salesforce.com/docs/marketing/marketing-cloud/references/mc_rest_rest_permission_scopes/rest-permissions-and-scopes.html).
The scopes listed in Hightouch can't exceed the scopes granted in your SFMC
package. If you attempt to use scopes not granted in SFMC, your authentication
will fail.
### Allowed Business Units
This section is relevant for SFMC instances with multiple Business Units.
By default, Hightouch can access all Business Units in your SFMC instance that
have the installed package enabled. If you want to restrict access to a subset
of Business Units, you can enter a space-separated list of Business Unit IDs in
the **Allowed Business Unit IDs (advanced)** field in the Hightouch destination
configuration.
You can find the Business Unit ID, or MID, by clicking on the account dropdown at
the top of the SFMC console.
## Sync configuration
Once you've set up your Salesforce Marketing Cloud destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Salesforce Marketing Cloud destination you want to sync to.
The following sections outline the different types of syncs you can configure with SFMC:
### Data Extensions
Hightouch supports syncing objects to existing Data Extensions and creating new ones.
You can select either **Upsert** or **Mirror** mode.
- In **Upsert** mode, you can choose whether to use the SOAP API or [file uploads](#automatic-file-import-via-file-uploads). Hightouch inserts new objects into the Data Extension and keeps all mapped object attributes up-to-date.
- In **Mirror** mode, you can't select between SOAP API or file uploads. Hightouch always uses file uploads, SOAP, and an Overwrite import automation in SFMC to overwrite the entire Data Extension with your model's query results.
#### Optional configurations
You can optionally choose which **Business Unit** and which **Data Folder** you would you like to sync data to. If you leave these configuration empty, Hightouch uses the defaults configured in SFMC.
#### Automatic Data Extension creation
You can optionally choose whether Hightouch should **automatically create a Data Extension**. If you enable automatic creation, Hightouch creates a Data Extension the first time the sync runs. It doesn't create new Data Extensions on subsequent runs. If you add columns to the sync config, Hightouch automatically adds them to the Data Extension.
When creating a new extension, you can optionally specify the Data Extension's name.
If you leave the name blank, Hightouch uses the model name.
The name also becomes the Data Extension's **customer key**.
You can include timestamp variables in the Data Extension name, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the Data Extension name:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
To change the name or customer key, you need to create a new sync. Hightouch doesn't modify the Data Extension's name or customer key after the first run.
You can also optionally enter a file path when creating a Data Extension, such
as `path/to/dataExtension`. If you choose a file path that includes folders,
Hightouch will only create new folders if they do not already exist, relative
to the selected folder in the Data Folder option. This path can also include
the variable values mentioned above, such as `{YYYY}/{MM}/{DD}/{model.name}`.
#### Automatic file import via file uploads
By default, Hightouch uses the SOAP API to sync Data Extension options in upsert mode. If you select to use File Upload mode, Hightouch uploads records via file transfer to SFMC, which may lead to better sync performance. While using File Upload mode is generally faster, it provides less visibility into individual records' sync successes and failures.
Generally, blob storage file locations (such as Google Cloud Storage and Amazon S3) are more performant file servers than SFMC's default option of ExactTarget Enhanced FTP server.
To use File Upload mode, ensure:
- your SFMC package has [the appropriate scopes](#create-a-server-to-server-package)
- you've [created an FTP user](#create-ftp-user) in SFMC
- you've [authenticated the FTP user](#connect-to-sfmc) in your destination configuration
Additionally, if you are using a blob storage file location, ensure:
- your SFMC file location has read access to your bucket
- the credentials you've provided Hightouch has write access to your bucket.
When using File Upload mode, Hightouch creates and runs an [import definition](https://help.salesforce.com/s/articleView?id=sf.mc_es_import_activity.htm&type=5) to load the data into the Data Extension. By default, Hightouch includes 100,000 rows in each file. You can configure this batch size in the Hightouch UI—the maximum value is 10 million rows per file. After each batch, Hightouch processes the SFMC results file to determine row errors.
SFMC provides more detailed error responses when using the SOAP API. If you
need to debug errors and the error messages from File Upload mode aren't
helpful, we recommend trying the SOAP API.
If you enable [deletion](#delete-behavior) in upsert mode, Hightouch removes
deleted rows in the Data Extension using the [SOAP
API](https://developer.salesforce.com/docs/marketing/marketing-cloud/guide/delete.htm),
since SFMC doesn't support deletions using FTP directly. If you want to
completely overwrite the Data Extension with the results of your sync, use
**Mirror** mode.
#### Record matching
You can match objects from your source to your Data Extension on your Data Extension's [primary key](https://help.salesforce.com/s/articleView?id=000388307&type=1).
Select the model column that contains these values.
Editing the Data Extension field used for record matching is only possible
before triggering the first sync run. Otherwise, making this change results in
an error. If you want to use a different field once the sync has already run,
you need to create a new sync.
#### Field mapping
You can map any model column to any field in your Data Extension.
Make sure that you map all **non-nullable fields** when syncing new objects to the Data Extension.
A non-nullable field is a field that is required to be populated with a value.
When creating or editing a record in SFMC, if a non-nullable field is left empty, the system won't allow the record to be saved and will return an error message.
You can check which fields are non-nullable fields in SFMC by inspecting your Data Extensions:
1. Open the Email Studio in your SFMC account.
2. Select the Data Extension you want to check.
3. Inspect the **Fields** section in the Data Extension to see a list of all fields.
4. Look for the **Nullable** column in the list. This column includes a checkbox for each field.
5. If a field doesn't have a check in the **Nullable** column, it means that the field is non-nullable and must have a value in it when a record is created or updated. Make sure you map all non-nullable fields.
In the example screenshot below, since `email` is the only field without a check in the Nullable column, it is the only non-nullable field.
In **Upsert** mode, you can sync records that don't include all non-nullable fields, but only if the object already exists in the Data Extension.
SFMC requires a minimum of one field mapping when sending data using
_SOAP_-mode. This can be a placeholder field.
#### Field creation
When creating a new Data Extension with Hightouch, you can create new fields and select their types while field mapping. First enter a field name, then select its type.
Text and Decimal field types allow for additional configuration.
Hightouch adds new fields to the Data Extension on each run but doesn't remove fields that leave the model's query results.
#### Sendable and testable Data Extensions
For auto-created Data Extensions, you can select whether the Data Extension is **sendable** and **testable**.
A sendable Data Extension is a type of Data Extension that can be used to send email messages or other types of communications to subscribers. Sendable Data Extensions are designed to contain subscriber data that can be used in email marketing campaigns, triggered sends, or other types of communications.
A testable Data Extension is used to send test emails or other test communications to a small group of recipients, usually internal stakeholders or quality assurance testers. Testable Data Extensions are designed to mimic the structure of a sendable Data Extension, but with a smaller set of test data.
To be designated as sendable, a Data Extension must meet certain requirements, including:
- It must contain an email address or mobile number field that is marked as the primary sendable field.
- It must contain at least one additional field that is used as a personalization field in the email message.
- It must be mapped to a profile attribute group, which is used to store additional subscriber data that is not included in the sendable Data Extension.
To make the Data Extension you are syncing via Hightouch sendable, select the field to use as the primary sendable field and the field to map to a subscriber field. The options available for the sendable field are [the fields you mapped](#field-mapping-2).
The length of the field chosen in send relationships is [limited to 254
characters](https://help.salesforce.com/s/articleView?id=sf.mc_rn_2024_spring_cab_de_restriction.htm&type=5).
Choose the `Text - 254 characters` field type when creating a sendable field
in Hightouch to enforce this limit.
You can only set a Data Extension to be testable once you select it to be sendable.
#### Data Retention Policy
You can set a data retention policy on your Data Extension through Hightouch. Hightouch will **not** edit existing data policies unless explicitly enabled.
You can choose what data should be deleted in your policy:
| Behavior | Description |
|-----------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|
| **All records and Data Extension**| All records and the Data Extension will be deleted at the end of your data retention period |
| **All records** | All records will be deleted at the end of your data retention period, but the Data Extension will not be deleted |
| **Individual records** | Records will be deleted from your Data Extension individually as they reach the Data Extension period. The Data Extension will not be deleted.|
You can choose the data retention period in days depending on how long you'd like to store your Data Extension and records.
If you choose to reset the retention policy upon each successful sync run, the data retention period will restart whenever data is successfully imported into your Data Extension.
Data retention policies cannot be removed through Hightouch. To remove an
existing data retention policy, navigate to your Data Extension in SFMC
Contact Builder and turn off the data retention policy.
#### Delete behavior
You can choose how to handle Data Extension when the corresponding rows are deleted in your source.
| Behavior | Description |
| --------------------- | --------------------------------- |
| **Do nothing** | Keep the object in SFMC |
| **Delete the record** | Remove the SFMC record completely |
#### Concurrency
When syncing to SFMC in File Upload mode, you can configure the concurrency used in the FTP upload to SFMC in Advanced options. Enter a value between 1 and 10.
Higher concurrency values will increase upload speeds, but can cause disruptions when simultaneously running multiple Hightouch syncs to SFMC. For this reason, the default concurrency is 1.
Concurrency options are not available for blob storage file locations.
#### File Retention
By default, Hightouch will delete the CSV files used to populate a Data Extension and the results file from the Import Definition automation from your SFMC FTP server. You can choose to keep files on your SFMC FTP server in Advanced options. Files on the SFMC FTP server will automatically be deleted by SFMC after 21 days.
Files are not automatically deleted from your blob storage file locations. We recommend setting a retention policy on your bucket to avoid storing stale CSV files.
#### Notifications
If you would like to receive an email notification when an Import Definition to your Data Extension succeeds, you can enter an email address in Advanced options.
In Mirror mode, one email will be sent for each sync run. In Upsert mode, one email will be sent for each batch completion. Depending on your batch size, this could be multiple emails per sync run.
#### Triggering automations
In Mirror mode, if you would like to trigger an automation after the completion of your Data Extension sync run, you can select **Yes** or **No** and enter either an **External key** or an **Automation ID**. If **Yes**, an automation will run even if there are failed rows in the sync. If the automation itself fails, the entire sync run is marked as failed.
### Contacts
This integration supports upserting contacts into Marketing Cloud. In upsert mode, Hightouch inserts new users into SFMC and updates [mapped user attributes](#field-mapping) on existing contacts.
#### Record matching
Hightouch matches rows from your model to Marketing Cloud contacts by **Contact Key**. Select the model column that contains these values.
Contact Keys are different from Contact IDs and email addresses. Check out
[Salesforce's
article](https://trailhead.salesforce.com/content/learn/modules/marketing-cloud-contact-management/understand-contacts-and-contact-model-relationships)
to learn more.
#### Field mapping
You can sync any model columns to Marketing Cloud's contact attributes.
Ensure all the required fields in each attribute set is completed. For example, to add an email address, be sure to map both **Email Addresses.Email Address** and **Email Addresses.HTML Enabled** fields. Refer to the usage section of SFMC's [Creating Contacts](https://developer.salesforce.com/docs/atlas.en-us.noversion.mc-apis.meta/mc-apis/createContacts.htm) documentation for a list of attribute sets.
### Journeys
Hightouch can manage the entry and exit of contacts in Marketing Cloud journeys based on your model's query results.
- When a record enters your results set, Hightouch adds a contact to the journey via the contact's [Contact Key](https://trailhead.salesforce.com/content/learn/modules/marketing-cloud-contact-management/understand-contacts-and-contact-model-relationships).
- When a record leaves your results set, Hightouch removes the contact from the journey via the contact's [Contact Key](https://trailhead.salesforce.com/content/learn/modules/marketing-cloud-contact-management/understand-contacts-and-contact-model-relationships).
Hightouch does not observe duplicate primary key errors. Adding a record with
a primary key that already exists in the journey will be marked as successful.
Similarly, removing a record with a primary key that was already removed from
the journey will also be marked as successful.
#### Journey selection
Here is an example journey that you could manage with Hightouch:
Note that it's:
- A multistep journey
- Triggered by an entry event with an associated **Data Extension** and **event definition key**
Hightouch automatically looks for the event definition key if you select a journey that fits the preceding requirements.
#### User identifiers
Hightouch enters and exits contacts from the journey via **Contact Key**. Select the model column that contains these values.
Contact Keys are different from Contact IDs and email addresses. Check out
[Salesforce's
article](https://trailhead.salesforce.com/content/learn/modules/marketing-cloud-contact-management/understand-contacts-and-contact-model-relationships)
to learn more.
#### Field mapping
You can use field mapping to map any other columns in your query results to the Data Extension containing your journey's data.
The Data Extension requires the **email_address** field—be sure to map a model column to that attribute. You can do this in Hightouch by typing "email_address" as a **Field from Salesforce Marketing Cloud** in the mappings section and then selecting the appropriate model column.
### Triggered send
Hightouch can trigger emails to individual subscribers via the Marketing Cloud API. This is useful for sending personalized messages to specific users based on their behavior or attributes.
Select an existing Triggered Send definition, then map the model columns to the Triggered Send's attributes, including email addresses and custom attributes. Hightouch matches rows from your model to Marketing Cloud contacts by **Subscriber Key**.
## Tips and troubleshooting
For Hightouch platform error codes related to SFMC, see [Error codes:
SFMC](/developer-tools/error-codes#salesforce-marketing-cloud-sfmc).
### Validate your setup
After you configure your sync, run a test to confirm data flows correctly:
1. Start with a small test audience or a single record.
2. Trigger a manual sync from the [Syncs overview](https://app.hightouch.com/syncs?from=docs) page.
3. After the sync completes, verify the data in SFMC:
- For Data Extension syncs, open the target Data Extension in **Audience Builder → Contact Builder → Data Extensions** and confirm your records appear with the expected field values.
- For Contact syncs, search for the test contact in **Contact Builder** and verify attributes.
- For Triggered Send syncs, check the **Tracking** tab of the Triggered Send to confirm delivery.
4. If the sync shows errors, check the sync run log in Hightouch for specific API error messages.
### Choosing a sync method for Data Extensions
When syncing [Data Extensions](#data-extensions), three sync methods are available. Choose based on your volume and debugging needs:
- **Upsert via SOAP API** — Best for smaller datasets or when you need detailed per-row error messages. The SOAP API is limited to approximately 2,000 requests per minute, so large syncs can be slow.
- **Upsert via [file uploads](#automatic-file-import-via-file-uploads)** — Best for large datasets (over 1 million records). Hightouch uploads records via SFTP, which is significantly faster than SOAP. Error messages are less granular.
- **[Mirror](#data-extensions)** — Best when you need to fully replace the Data Extension contents each sync, such as heavy deletion use cases. Mirror always uses file uploads.
If you're unsure, start with upsert via file uploads for production syncs and switch to SOAP temporarily when debugging row-level errors.
### DateTime timezone conversions
SFMC converts all incoming datetime values to UTC and displays them in the account's configured timezone. This can cause apparent date shifts — for example, a timestamp of `2026-06-02 22:00:00 America/Los_Angeles` is stored as `2026-06-03 05:00:00 UTC`, which may display as June 3 instead of June 2.
To avoid this:
- Add a transformed column in your data source that includes explicit timezone information (for example, an ISO 8601 string with a timezone offset like `2026-06-02T22:00:00-07:00`).
- Alternatively, use a timezone conversion function in your model query. For example, in Snowflake: `CONVERT_TIMEZONE('America/Los_Angeles', 'UTC', your_timestamp_column)`.
### Common errors
| Error message | Description |
| :------------------------------------------------------------------------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Failed to create Data Extension` | Incomplete sync configuration with missing field names or types. [More detail.](#failed-to-create-data-extension) |
| `Instance validation error: ... is not a valid value for DataExtensionFieldType` | Data Extension name conflicts with an existing one. [More detail.](#instance-validation-error-data-type-value-is-not-a-valid-value-for-dataextensionfieldtype) |
| `The event data contains duplicate values for an existing primary key` | Duplicate values in the primary/subscriber key column. [More detail.](#the-event-data-contains-duplicate-values-for-an-existing-primary-key) |
| `The Custom Object field selected must have a type of Number or Long Number` | Subscriber ID field mapped to a non-numeric type. [More detail.](#the-custom-object-field-selected-must-have-a-type-of-number-or-long-number) |
| `Adding a new Data Extension definition is not allowed when doing an update-only operation` | Data Extension was deleted or name collides with an existing one. [More detail.](#adding-a-new-data-extension-definition-is-not-allowed-when-doing-an-update-only-operation) |
| `FileNotFound: Soap Error (import instance could not be validated)` | SFMC can't access the uploaded file due to FTP credential or directory issues. [More detail.](#filenotfound-soap-error-import-instance-could-not-be-validated) |
| `Mirror sync shows failure but most records are present in SFMC` | A few rows failed validation, marking the entire sync run as failed. [More detail.](#mirror-sync-shows-failure-but-most-records-are-present-in-sfmc) |
#### Failed to create Data Extension
```
Failed to create Data Extension
```
**Cause:** The sync configuration is incomplete — field names or field types are missing.
**Solution:** Open your sync configuration and ensure you've entered the field names and selected the field types when [creating new fields](#field-creation).
#### Instance validation error: 'DATA TYPE VALUE' is not a valid value for DataExtensionFieldType
```
Server was unable to read request. ---> There is an error in the XML document. ... 'DATA TYPE VALUE' is not a valid value for DataExtensionFieldType
```
**Cause:** The sync is configured to create a new Data Extension with the same name as one that already exists in SFMC.
**Solution:** Change the name of the Data Extension in your sync configuration to a unique name, and select field types for each mapped column.
#### The event data contains duplicate values for an existing primary key
```
400 - {"message":"The event data contains duplicate value for an existing primary key. Please correct the event data and try again.","errorcode":30000,"documentation":""}
```
**Cause:** The column selected as the primary or subscriber key contains duplicate values.
**Solution:** Ensure the key column doesn't contain duplicates. Update your [model definition](/models/creating-models) to deduplicate if needed. See [syncing Data Extensions](#data-extensions) for more on key configuration.
#### The Custom Object field selected must have a type of Number or Long Number
```
The Custom Object field selected must have a type of Number or Long Number
```
**Cause:** You chose to make the Data Extension sendable and set the subscriber relationship using the Subscriber ID, but the mapped field isn't a numeric type.
**Solution:** Either map a numeric field to the Subscriber ID, or map the field to the Subscriber Key instead. See SFMC's [Subscriber data view documentation](https://help.salesforce.com/s/articleView?id=sf.mc_as_data_view_subscribers.htm&type=5) for field type requirements.
#### Adding a new Data Extension definition is not allowed when doing an update-only operation
```
Could not create data extension fields: Error: Soap Error Key: ... Adding a new Data Extension definition is not allowed when doing an update-only operation
```
**Cause:** Either the Data Extension set in the sync configuration has been deleted from SFMC, or the sync is set to [automatically create a new Data Extension](#automatic-data-extension-creation) with a name that matches an existing Data Extension.
**Solution:** Ensure your sync sends data to an existing Data Extension. If the sync is creating a new Data Extension, use a unique name that doesn't already exist in your SFMC environment.
#### FileNotFound: Soap Error (import instance could not be validated)
```
FileNotFound: Soap Error Key: ... import instance could not be validated
```
**Cause:** SFMC can't access or find the file uploaded by Hightouch. This typically points to FTP credential or directory configuration issues.
**Solution:**
1. Verify the FTP credentials in your destination configuration are correct. Test them by connecting to the SFTP outside of Hightouch.
2. Ensure the FTP user has access to the correct file location. If you have multiple business units, create the FTP user in the parent business unit or the same business unit as the target Data Extension.
3. Check the FTP user's home directory in SFMC. If it's set to something other than `/`, files may be written to a nested directory instead of the `Import/` directory. Set the home directory to `/` (the default).
4. If you're using a blob-storage file location (AWS S3 or GCS), ensure the configured role has read permissions.
#### Mirror sync shows failure but most records are present in SFMC
```
Sync run failed (Mirror mode)
```
**Cause:** A small number of records failed validation on the SFMC side — for example, due to data type mismatches or missing required fields. Any rejected rows cause the entire sync run to be marked as failed.
**Solution:** Check the **Rejected rows** tab in the sync run details to identify the specific records that failed. Fix the underlying data issues in your model, then rerun the sync.
### Live debugger
### Sync alerts
---
## SFTP
**URL:** https://hightouch.com/docs/destinations/sftp
**Description:** Transfer data at scale from your warehouse to your apps through SFTP
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
|--------------|-----------------------------------------------------------------------------|----------------------|
| Any data set | Sync data from a source to SFTP as CSV, JSON, NDJSON, XML, or Parquet files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Diff mode creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to SFTP
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **SFTP** and click **Continue**. You can then authenticate Hightouch to **SFTP** by entering the following required fields into Hightouch:
- **Hostname**
- **Port**
- Credentials (see the [supported authentication methods](#supported-authentication-methods) below)
The user needs to have access to the target directory and file with **write privileges**.
A `permission denied` error message during a sync indicates the user may not have write permissions for both the directory and the file.
### Supported authentication methods
We support three authentication methods:
- Username + password
- Username + SSH private key
- Username + SSH private key + passphrase
We support two formats for the SSH private key:
- OpenSSH
- RSA
Provide the full private key for the user, including the header and footer.
For example:
```bash
-----BEGIN OPENSSH PRIVATE KEY-----
-----END OPENSSH PRIVATE KEY-----
```
### PGP encryption
The SFTP destination supports PGP encryption. Given a **PGP Public Key**, all files transferred to the destination will be encrypted.
To use PGP encryption, provide your public key in the **PGP Public Key** text area including the header and footer in the following format:
```bash
-----BEGIN PGP PUBLIC KEY BLOCK-----
-----END PGP PUBLIC KEY BLOCK-----
```
The **PGP Private Key** is required to decrypt the files synced to your destination.
You don't need to insert this value in Hightouch.
## Sync configuration
Once you've set up your SFTP destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the SFTP destination you want to sync to.
### Select file format
Hightouch supports syncing CSV, JSON, NDJSON, XML, and Parquet files via SFTP.
### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it.
If you are using an audience and would like to include the audience name, you will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter `-86400` (`24 hours * 60 minutes * 60 seconds`). If you want the filename's data to be one hour after the sync takes place, you would enter `3600` (`60 minutes * 60 seconds`).
### CSV options
If you're syncing to a CSV file, you have additional configuration options:
- **Delimiter**: Your options are comma (`,`), semicolon (`;`), pipe (`|`), tilde (`~`), and tab
- (Optional) Whether to **include a CSV header row** in the exported files
- (Optional) Whether to **include a [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark)** in the exported files, the BOM is ``
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `id`, `email` and `location`. These columns are mapped to new fields in the destination file as `user_id`, `user_email` and `user_location`. Hightouch exports these fields to the new fields in the file and ignores all other columns from your results.
`Boolean` values that appear as `True` and `False` in your model appear as `1` and blank respectively in the file created during the sync. If you'd like for these values to appear in another format in your downstream file, you can use [template mapping](/syncs/mapping-data#template-mapping) to achieve your desired format.
### Batch size
By default, and depending on your [sync mode](#supported-syncing), Hightouch sends one file for each export. (In diff mode, Hightouch sends three files.)
By enabling batching, you allow Hightouch to send multiple files if your model query results are large. This can help avoid file size errors.
### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
## Encoding
By default, Hightouch saves the file in ASCII encoding. If there are any non-ASCII characters in your results, the file will be saved with utf-8 encoding. If you've enabled [including a BOM](#csv-options), the file will be saved with utf-8 encoding with BOM.
## Tips and troubleshooting
The **Test connection** button located on the destination configuration page tries to connect to your configured server, but it can only detect if your credentials are completely invalid.
To fully test user permissions, you can create and run a one-row test sync to verify that your data is successfully synced to your SFTP server.
### Common errors
#### Renaming temporary file to target file failed
During an SFTP sync, Hightouch writes the data to a temporary file in your server and then renames it to the final filename as specified by your sync configuration.
This is an important step to avoid partial data writes and consumption by your downstream processes that can happen due to network issues.
Failure at this step indicates that the temporary file has been consumed and/or deleted by a downstream process that exists in the SFTP server before the sync has concluded.
Our suggestion is to adjust directory listeners to ignore temporary files, which are prefixed with a period `.`, or provide a different temporary file directory in your sync configuration.
Otherwise, this error can also occur if the [credentials you're using](#connect-to-sftp) don't have the necessary permissions to rename the temporary file or modify its directory.
#### Connection timed out/waiting for handshake
Usually, when this error occurs, it is likely that the IP addresses Hightouch uses are not allowlisted and therefore could be being blocked.
To help fix this, we recommend ensuring that our [IP Addresses](/security/networking#ip-addresses) are allowed within your SFTP's Firewall.
If you're unsure whether IPs are allowlisted, ask your network, IT team, or third party SFTP vendor to verify.
### Live debugger
### Sync alerts
---
## Microsoft Sharepoint
**URL:** https://hightouch.com/docs/destinations/sharepoint
**Description:** Microsoft SharePoint is a powerful file storage solution that enables seamless document management and collaboration.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
-----------------|------------------------------------------------------------|----------------------
**Any data set** | Sync data from a source to SharePoint as JSON or CSV files | Insert, All, Diff
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Microsoft Sharepoint
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Microsoft Sharepoint** and click **Continue**. You can then authenticate Hightouch to **Microsoft Sharepoint**.
For the **Authentication method**, select **Log in to Sharepoint** and log into your Microsoft Sharepoint account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Microsoft Sharepoint destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Microsoft Sharepoint destination you want to sync to.
### Select file format
Hightouch supports syncing JSON and CSV files to SharePoint.
### Select a Site and Drive
In order to sync to SharePoint, you must select a site and a drive available to your account.
You can specify a folder within the site/drive combination, or create a new folder to write the file to.
### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch will overwrite the file.
### Set filename offset
By default, Hightouch uses the UTC timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter '-86400' (`24 hours * 60 minutes * 60 seconds`).
#### Record matching
You can match rows from your model to any data set in Microsoft Sharepoint on any column in your model and any field in Microsoft Sharepoint. Ensure the data types of the model column and Microsoft Sharepoint field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync any data set fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default any data set fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync for Insert and Diff modes.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Shopify
**URL:** https://hightouch.com/docs/destinations/shopify
**Description:** Keep your storefront up-to-date with the latest customer and product info from your data warehouse
**Section:** Destinations
Shopify will be [deprecating their REST
API](https://shopify.dev/docs/apps/build/graphql/migrate) and recommends their
new [GraphQL API](https://shopify.dev/docs/api/admin-graphql). Hightouch
supports using these new GraphQL endpoints for new syncs, and existing syncs
can be updated to use the GraphQL endpoints.
## Overview
This destination lets you send customer, product, and fulfillment information from your data warehouse to Shopify so you never have to worry about stale inventory information or incorrect customer or product details.
## GraphQL vs REST API
Shopify will be deprecating their REST API in favor of their new GraphQL API. For more detailed information on this update, and how object types and inputs have changed, refer to this [Shopify resource](https://shopify.dev/docs/apps/build/graphql/migrate). Hightouch allows syncs to use either protocol for the time being, and you can change between the two protocols to see the differences between them. Most notably, the GraphQL endpoints use new object types **Product Variant** and **Product Option** that were previously coupled together for the REST API. They are now separate Object types to sync to based on existing **Products** in your shop.
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| ----------------------- | ----------------------------------------------------------------------------------------------------- | -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [**Objects**](#objects) | Sync data from any source to customer, discount, inventory item, inventory level, and product objects | Varies depending on object | [GraphQL Admin API ↗](https://shopify.dev/docs/api/admin-graphql) |
| [**Events**](#events) | Send fulfillment events to Shopify | Insert | [Create fulfillment event ↗](https://shopify.dev/docs/api/admin-rest/2024-01/resources/fulfillmentevent#post-orders-order-id-fulfillments-fulfillment-id-events) |
| [**Actions**](#actions) | Sync data from any source to various Shopify actions | Insert | [Fulfillment order actions ↗](https://shopify.dev/docs/api/admin-graphql/2024-01/mutations/fulfillmentOrderCancel) |
## Connect to Shopify
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Shopify**.
To connect your Hightouch app to Shopify, you need to create a **custom app** in your Shopify account.
Shopify requires that custom apps made after January 1, 2026 be authenticated
to using **Client credentials**. Custom apps made before January 1, 2026 can
be authenticated to using an **Admin API access token**. If you are creating a
new custom app, please use the Client credentials flow. If you have an
existing custom app, you may use the Admin API access token flow.
You then need to provide your **Store name** and either the set of **Client credentials** or an **Admin API access token** of your custom app.
### Create Client credentials for your custom app
1. In your Shopify store's home page, click **Apps** on the sidebar.
2. Click on **Develop apps** and then select **Build apps in Dev Dashboard**.
3. Click **Create app** and go through the flow using the **Start from Dev Dashboard** option.
4. Add to the **Access > Scopes** section the scopes:
```
write_customers,
read_customers,
write_discounts,
read_discounts,
write_inventory,
read_inventory,
write_price_rules,
read_price_rules,
write_products,
read_products,
read_locations,
write_assigned_fulfillment_orders,
read_assigned_fulfillment_orders,
write_merchant_managed_fulfillment_orders,
read_merchant_managed_fulfillment_orders,
write_fulfillments,
read_fulfillments,
write_orders,
read_orders
```
5. Click **Release** and under the **Settings** tab you will find your **Client ID** and **Client secret**.
### Create an Admin API access token for your custom app
1. In your Shopify store's admin, click **Apps** in the left sidebar.
2. From the dropdown which appears, select **Apps and sales channel settings**.
3. Click **Develop apps for your store**.
4. On the next screen, click **Create an App** in the top right corner. Give it a descriptive name, for example, "Hightouch integration." Then click **Create app**.
5. Under **Configuration** tab, select **Configure** to configure the Admin API Integration.
6. Select the following **Admin API access scopes**:
- `write_customers`
- `read_customers`
- `write_discounts`
- `read_discounts`
- `write_inventory`
- `read_inventory`
- `write_price_rules`
- `read_price_rules`
- `write_products`
- `read_products`
- `read_locations`
- `write_assigned_fulfillment_orders`
- `read_assigned_fulfillment_orders`
- `write_merchant_managed_fulfillment_orders`
- `read_merchant_managed_fulfillment_orders`
- `write_fulfillments`
- `read_fulfillments`
- `write_orders`
- `read_orders`
7. Click **Save** in the bottom right of the screen.
8. In **API Credentials** tab, click **Install app**, and then **Install** in the modal that appears.
9. Your **Admin API Access Token** is now available. Copy and save your Admin API access token in a secure place.
## Sync configuration
Once you've set up your Shopify destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Shopify destination you want to sync to.
### Objects
Hightouch supports syncing to these Shopify objects with the following sync modes:
| Sync Type | Supported Sync Modes |
| ----------------------------------- | -------------------------- |
| **Customers** | Upsert, Update, and Insert |
| **Discounts** | Update and Insert |
| **Inventory Item** | Update only |
| **Inventory Level** (REST API only) | Upsert, Update, and Insert |
| **Products** | Upsert, Update, and Insert |
Check out the [sync modes docs](/syncs/types-and-modes#sync-modes) to learn more.
#### Record matching
Based on the Shopify object, database records can be matched from your source to your Shopify workspace by a unique property.
| Sync Type | Supported Identifiers |
| ------------------- | --------------------- |
| **Customers** | Email, ID, or phone |
| **Discounts** | ID |
| **Inventory Item** | Inventory item ID |
| **Inventory Level** | Inventory item ID |
| **Products** | Handle |
#### Field mapping
You can sync columns from your source to Shopify's default and custom fields.
You must provide a **Price Rule ID** from an existing **PriceRule** when
syncing **Discounts**. Similarly, you must provide an **Inventory Item ID**
from an existing **Product** when syncing either **Inventory Item** or
**Inventory Level**.
#### Delete behavior
For **Products** (for the **REST API** protocol only), you can choose what Hightouch's behavior is when a record leaves the query result set.
| Behavior | Description |
| ------------------------------ | --------------------------------- |
| **Do nothing** | Keep the product in your shop |
| **Delete the Shopify product** | Remove the product from your shop |
### Events
Hightouch supports syncing to these Shopify events with the following sync modes:
| Sync Type | Supported Sync Modes |
| --------------------- | -------------------- |
| **Fulfillment event** | Insert only |
#### Field mapping
You need to include at least the **Fulfillment ID**, **Order ID**, and **Status** in your sync. Other field mappings are optional.
### Actions
Hightouch supports syncing to these Shopify events with the following sync modes:
| Sync Type | Supported Sync Modes |
| ------------------------------------------------------------------- | -------------------- |
| **Cancel fulfillment orders** | Insert only |
| **Mark fulfillment orders as incomplete** | Insert only |
| **Apply fulfillment hold on fulfillment orders with status `Open`** | Insert only |
| **Move fulfillment orders to another location** | Insert only |
| **Mark fulfillment orders as open** | Insert only |
| **Release the fulfillment hold on fulfillment orders** | Insert only |
| **Reschedule the time of scheduled fulfillment orders** | Insert only |
#### Field mapping
Each action requires you to include at least the **Fulfillment Order ID**. Some actions require additional fields:
| Action | Additional required fields |
| ------------------------------------------------------------------- | -------------------------- |
| **Apply fulfillment hold on fulfillment orders with status `Open`** | Reason |
| **Move fulfillment orders to another location** | New Location ID |
| **Reschedule the time of scheduled fulfillment orders** | New Fulfill At |
## Tips and troubleshooting
For Hightouch platform error codes related to destinations, see [Error
codes](/developer-tools/error-codes#other-destinations).
### Validate your setup
After you configure your sync, run a test to confirm data flows correctly:
1. Start with a small test audience or a single record.
2. Trigger a manual sync from the [Syncs overview](https://app.hightouch.com/syncs?from=docs) page.
3. After the sync completes, verify the data in Shopify:
- For customer syncs, go to **Customers** in Shopify Admin and search for the test customer. Confirm that the mapped fields appear with the expected values.
- For product syncs, go to **Products** and verify the synced product attributes.
4. If the sync shows errors, check the sync run log in Hightouch for specific API error messages.
### Tag sync behavior
When syncing tags to Shopify, Hightouch replaces the entire tag list on the resource with the tags in your sync payload. This means that if you sync a single tag, any existing tags not included in the payload are removed. To preserve existing tags, include all desired tags in your model's tag column — not just the new ones you want to add.
### Variants field formatting
When syncing product variants, the **Variants** field expects an array of objects. In your model, ensure the column is cast to the **Object/Array** data type — not **String**. Each object in the array represents a single variant.
For example, the column value should look like:
```json
[
{ "option1": "Red", "option2": "Small", "price": "19.99", "sku": "RED-SM" },
{ "option1": "Blue", "option2": "Large", "price": "24.99", "sku": "BLU-LG" }
]
```
If the column is a string, cast it to **Object/Array** in your [model configuration](/models/data-types-casting#casting-from-your-model-configuration), or use your warehouse's JSON functions to construct the array in your SQL model.
### Common errors
### Live debugger
### Sync alerts
---
## Sinch Campaigns
**URL:** https://hightouch.com/docs/destinations/sinch-campaigns
**Description:** Keep Sinch audiences in step with the latest customer activity, and trigger published Sinch Campaigns to any cohort with per-recipient personalization filled from your customer data.
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| ------------------------------------------ | -------------------------------------------------------------------------------------------------------------------- | -------------------- | --------------------------------------------------------------------------------------------------- |
| [**Audience**](#audience) | Replace the contents of an existing Sinch audience with the current rows in your Hightouch model. | Mirror | [Audiences API](https://developers.sinch.com/docs/campaigns/audiences/) |
| [**Campaign trigger**](#trigger-campaigns) | Trigger an existing Sinch campaign for the contacts in your model, with optional template variables and a shoot tag. | Insert | [Send to contacts](https://developers.sinch.com/docs/campaigns/api/campaigns-api/#send-to-contacts) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Sinch Campaigns
To connect Sinch Campaigns to Hightouch you need the API key from your Sinch Campaigns console. It's a pre-encoded Basic-auth token, visible under your account settings.
1. Go to the [Destinations overview](https://app.hightouch.com/destinations) page and select **Add destination**.
2. Select **Sinch Campaigns** from the destination catalog.
3. Enter the following credentials, then select **Continue**:
- **Region**: the Sinch Campaigns region your account is provisioned in. Choose **European Union** or **United States**. Sinch hosts EU and US tenants separately, so picking the wrong one fails authentication on save.
- **API key**: your Sinch Campaigns API key, copied from the console. It's already base64-encoded, so Hightouch uses it directly as the Basic auth header — there's no separate secret to enter. Hightouch uses it to mint short-lived bearer tokens for the Campaigns API and JWTs for the Audiences API.
When you select **Continue**, Hightouch authenticates against both the Sinch Campaigns API and the Sinch Audiences API. If only one check fails, the test-connection step calls out which API to investigate — usually an account that isn't provisioned for Audiences, or a key from the wrong region.
The single API key authorizes both APIs. The Sinch account Hightouch reads and
writes audiences against is derived from the key itself, so there's nothing
else to configure.
## Sync configuration
### Audience
Audience syncs keep a Sinch audience in lockstep with a Hightouch model. Each sync run uploads the model's current contents as a CSV and replaces the audience membership wholesale.
When you configure the sync, you can pick an existing Sinch audience or create
a new one by name. Hightouch populates the existing-audience dropdown by
paging through every audience in your Sinch account.
Hightouch refuses to upload a zero-row file. If the source query returns no
rows, the sync errors instead of clearing the Sinch audience. Wiping an
audience by accident is almost always a query bug, not the intent.
#### Sync mode
Audience syncs run in **Mirror** mode. Each run reuploads the full membership. Sinch's Audiences API doesn't expose an append or remove operation, so incremental updates aren't possible.
#### Configure record matching
Each row must include a phone number in [E.164 format](https://en.wikipedia.org/wiki/E.164) (for example, `+5215512345678`). Sinch identifies audience contacts by their MSISDN.
| Destination field | Required | Notes |
| ------------------------- | -------- | -------------------------------------------------------------------------------------- |
| **Phone (MSISDN, E.164)** | Yes | Mapped from a column of `phone` type. Becomes the `msisdn` column in the uploaded CSV. |
#### Map fields
Add a mapping for each extra attribute you want to attach to contacts in the audience. Each mapping becomes a column in the uploaded CSV, and the mapping name is the column header Sinch sees.
Mapping names must use lowercase letters, digits, and underscores, and must start with a letter (for example, `first_name`). Date values are converted to ISO 8601 before being written to the CSV.
### Trigger campaigns
Campaign trigger syncs fire an existing Sinch campaign for the rows in your model. Each row becomes one recipient, and positional template variables in the campaign template are filled from the row's mappings.
Only campaigns marked **ready_to_shoot** in Sinch appear in the campaign
dropdown. Draft, paused, or unpublished campaigns must be published in the
Sinch console first.
#### Sync mode
Campaign trigger syncs run in **Insert** mode. Each batch of rows is submitted to Sinch's `send-to-contacts` endpoint, which returns a task identifier. Hightouch then polls `/tasks/{task_id}` every 5 seconds until Sinch reports the task as finished.
Hightouch sends recipients in batches of 500 with a parallelization of 1. Sinch's API rejects concurrent sends to the same campaign, so a sync to a single campaign always runs sequentially. Syncs that span multiple campaigns are still grouped per-campaign before submission.
#### Map fields
| Destination field | Required | Notes |
| ----------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Campaign** | Yes | The Sinch campaign to send. Pick from the dropdown (column or constant). Only campaigns marked `ready_to_shoot` are available. |
| **Phone** | Yes | Source column carrying the recipient's phone number in E.164 format. Sinch silently drops rows that don't parse. |
| **Shoot tag** | No | Optional column or constant. Tags every send in this run so the results roll up together under the Sinch **Shoot Stats** endpoint. If omitted, Sinch generates one. |
#### Map template variables
Sinch substitutes mapped values into the campaign template **positionally**. The first mapping becomes `[[A]]`, the second becomes `[[B]]`, and so on. Add one mapping per template placeholder, in the same order they appear in the template. The mapping name is for your reference and is not sent to Sinch.
Date values are converted to ISO 8601 before being substituted, since Sinch's templates render the raw text as-is.
## Tips and troubleshooting
### Validate your setup
After your first sync completes, verify the upload in Sinch:
- **Audience syncs:** open the audience in the Sinch console and confirm the contact count matches the row count from your model.
- **Campaign trigger syncs:** check the campaign's stats by shoot tag. If you configured a shoot tag, Sinch rolls every send from this run under it; otherwise, use the run's timestamp.
### Per-row errors on campaign triggers
When Sinch reports that a `send-to-contacts` task failed, Hightouch can't always map the failure to specific rows — Sinch's error payload isn't consistently keyed across error types. Hightouch surfaces Sinch's full error message against every recipient in the failing batch, so the [Live debugger](#live-debugger) is the most reliable place to see what went wrong.
### Common errors
### Live debugger
### Sync alerts
---
## Sinch MessageMedia
**URL:** https://hightouch.com/docs/destinations/sinch-messagemedia
**Description:** Activate your full customer dataset for SMS and WhatsApp campaigns. Keep contacts and audience lists in Sinch MessageMedia in sync with the data your marketing team relies on, so every send is targeted at the right people with the right context.
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| ---------------------------------------- | ------------------------------------------------------------------------------------- | -------------------- | --------------------------------------------------------- |
| [**Contacts**](#sync-contacts) | Upsert contact records (phone numbers and attributes) into Sinch's contact directory. | Upsert | [Contacts API](https://developers.messagemedia.com/) |
| [**Contact lists**](#sync-contact-lists) | Sync Hightouch audiences into long-lived Sinch contact lists. | Add, Remove | [Contact lists API](https://developers.messagemedia.com/) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Sinch MessageMedia
To connect Sinch MessageMedia to Hightouch you need an API key and API secret from your Sinch HUB account. Hightouch authenticates each request with HTTP Basic auth.
1. Go to the [Destinations overview](https://app.hightouch.com/destinations) page and select **Add destination**.
2. Select **Sinch MessageMedia** from the destination catalog.
3. Enter the following credentials, then select **Continue**:
- **API key**: the API key from your Sinch HUB account.
- **API secret**: the API secret from your Sinch HUB account.
The destination targets the Sinch MessageMedia HUB Contacts API
(`api.messagemedia.com`). It does not currently authenticate against the Sinch
CEP Campaigns API hosts.
## Sync configuration
### Sync contacts
Upsert contact records into your Sinch contact directory. Each row in your model becomes a contact identified by phone number.
#### Sync mode
Contacts syncs run in **Upsert** mode. Hightouch creates new contacts that aren't already in Sinch and updates existing contacts when fields change.
Sinch's HUB API returns a 409 conflict if a contact's phone number or alias is
already in use by a different record. Hightouch surfaces these conflicts as
rejected rows in the [Live debugger](#live-debugger) rather than aborting the
sync.
#### Record matching
Map a model column to **Phone number (msisdn)**. Sinch matches contacts by phone number, in international format (`+E.164`).
#### Channel configuration
Sinch contacts each have one or more channels. Hightouch sends the mapped phone number on every row's primary channel using the type and subscription state configured here:
| Field | Description |
| ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| **Channel type** | The channel to associate with the phone number. Choose **SMS** (default) or **WHATSAPP**. Sinch's account-level validator rejects other values. |
| **Default subscription state** | The subscription state applied when a new contact is created. Choose **SUBSCRIBED** (default) or **UNSUBSCRIBED**. |
#### Field mapping
Map source columns to Sinch's standard contact fields:
| Destination field | Type | Notes |
| ----------------- | ------ | ----------------------------------------------------------------------- |
| **First name** | string | |
| **Last name** | string | |
| **Alias** | string | Must be unique across your Sinch account. Duplicate aliases return 409. |
| **Date of birth** | date | |
| **Country** | string | |
| **State** | string | |
| **Location** | string | |
| **Note** | string | |
To send fields that aren't in this list, configure them as **Custom fields** below.
#### Custom fields
Map model columns to custom fields configured in your Sinch HUB account. Hightouch fetches your account's custom field definitions and shows them in the dropdown — you map a source column to a specific Sinch custom field UUID.
Create the custom fields in Sinch HUB before configuring the mapping.
Hightouch only lists custom fields that already exist in your account.
### Sync contact lists
Sync a Hightouch audience into a long-lived Sinch contact list. Best for reusable, consented populations that drive recurring campaigns.
Contact list syncs require contacts to **already exist in Sinch** and to be
referenced by their Sinch UUID. Run the [Contacts](#sync-contacts) sync first,
write the returned UUIDs back to your warehouse, and then configure the
contact list sync against the UUID column. Mapping by phone number is not
supported because Sinch's HUB API does not currently let Hightouch look up a
UUID from a phone number.
#### Choose a contact list
Select an existing Sinch contact list, or create a new one from the sync configuration form. Hightouch fetches your account's contact lists and lets you pick one.
#### Record matching
Map a model column containing Sinch contact UUIDs to **Contact UUID (contact_id)**. Each row must contain a UUID that Sinch already knows about — populate this column from the IDs returned by the Contacts sync.
#### Delete behavior
Choose how Hightouch treats rows that leave your audience query results:
| Behavior | Description |
| ------------------------ | ----------------------------------------------------------------------------------------------------------------- |
| **Do nothing** (default) | Leave the contact's list membership unchanged. |
| **Remove from list** | Remove the contact from the Sinch contact list. The contact itself is not deleted from Sinch's contact directory. |
## Tips and troubleshooting
### Validate your setup
After your first sync completes:
- Open your Sinch HUB account and search the contact directory for one of the phone numbers you synced. Confirm that the contact appears with the expected channel type, subscription state, and mapped fields.
- For contact list syncs, open the target list in Sinch HUB and confirm the expected contacts are members.
- Check the [Live debugger](#live-debugger) for rejected rows. The most common rejections on contact syncs are 409 conflicts caused by duplicate aliases or phone numbers already attached to a different contact.
### Recommended workflow
1. Build a Contacts sync that upserts phone numbers and attributes into Sinch. Capture the returned Sinch UUIDs back to your warehouse.
2. Build a Contact lists sync that uses those UUIDs as the record-matching identifier to drive list membership from your audiences.
### Common errors
### Live debugger
### Sync alerts
---
## Singular
**URL:** https://hightouch.com/docs/destinations/singular
**Description:** Enrich your analytics in Singular with event data from your warehouse.
**Section:** Destinations
## Overview
This integration lets you send server-to-server events to Singular without writing a custom pipeline. Whether you're calculating cross-device attribution, analyzing ROI, or using Singular for fraud prevention, including event data helps you have a more complete analytics view.
## Supported syncing
| Type | Description | Supported Sync Modes |
| ---------- | ---------------------------------------------------- | -------------------- |
| **Events** | Hightouch supports syncing in-app events to Singular | Insert only |
## Getting started
Start by creating an instance of Singular destination in Hightouch. Upon creating your instance, you must enter a Singular API key for authentication.
### Authenticating with an API key
To get your API key, navigate to `Developer Tools` > `API Keys` and copy your API key into Hightouch.
## Syncing data
### Events
Any records added to [your source](/getting-started/concepts) are treated as new events and are sent to Singular when your sync runs.
`Platform`,`Package Name`, and `OS Version` fields are the minimum required fields to make the API call to Singular. For a richer analytics, we recommend that you provide all relevant fields in your use case.
The supported platforms are `iOS` and `Android`, case sensitive. If the
platform is `Android`, `andi` field is also required.
If you don't map the `ip` field, it will be defaulted with one from
Hightouch's servers.
You can use custom mappings to attach custom data relevant to your application.
Custom mappings are included as the [`e`
parameter](https://support.singular.net/hc/en-us/articles/360048588672-Server-to-Server-S2S-API-Endpoint-Reference#Optional_Parameters)
of the API request payload.
For more information about required fields, refer to [Singular's documentation](https://support.singular.net/hc/en-us/articles/360048588672#Event_Notification_Endpoint).
## Tips and troubleshooting
### Common errors
If you receive errors, ensure that your including all of Singular's required fields: `Platform`, `Package Name`, `OS Version` and `andi` if your `Platform` is `Android`.
### Live debugger
### Sync alerts
---
## Slack
**URL:** https://hightouch.com/docs/destinations/slack
**Description:** Sync your warehouse with Slack and push business metrics and user activity data to Slack in real-time
**Section:** Destinations
## Overview
With the Slack destination, Hightouch can serve as a notifier for various data-related events or changes.
This is a very flexible integration that you can use for many different use cases, such as:
- notifying your Customer Success teams when product usage for a key account suddenly drops
- notifying an Account Executive when product usage surpasses a certain threshold
- sending a daily summary of the number of sign-ups compared to the same day in the previous month
- sending a daily summary on the adoption of a core feature
## Supported syncing
| Sync type | Description |
| :---------------- | :--------------------------------------------------------------------------------------------------------------------------------------------- |
| **Message** | Sends individual messages for each row that was added, changed, or removed since the last sync. There will be no messages for the initial run. |
| **Batch Message** | Sends a formatted message (or multiple messages) of all the rows in the query results. |
| **Table** | Sends a formatted table of all rows in the query results as a single message. |
| **CSV** | Sends a CSV file of all rows in the query results attached to a single message. |
### Slack error alerting
To learn more about setting up Slack as your error alerting tool, see the [Alerting](/syncs/alerting) page.
## Connect to Slack
### Authenticating with OAuth
Use OAuth to connect Slack by navigating to **Destinations** > **Create New** > **Slack**.
Click **Connect** and log in with your desired Slack account.
Once you have selected your desired workspace, click **Allow** and you will see an **Authorization successful** screen in Hightouch.
Finish your setup by clicking the continue button and naming your destination.
### Adding the Hightouch bot to a channel
In order for Hightouch to message your channel, you need to ensure the Hightouch bot is added as an integration.
Follow the steps below to add the bot to your channel.
You can add the Hightouch bot both to public and private Slack channels.
Adding the Hightouch bot can be done by typing `/invite @hightouch` in the Slack composer. If that doesn't work you can also add the Hightouch bot by following these steps:
1. Go to your channel and click the dropdown button (**∨**) next to the channel name.
2. Click **Integrations** > **Add an App**.
3. Search for **Hightouch** and click **Add** next to the Hightouch app name.
4. The Hightouch bot should now appear in the channel settings.
## Sync configuration
Once you've set up your Slack destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data.
Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin.
Then, select the relevant model and the Slack destination you want to sync to.
### Syncing messages
Hightouch supports sending messages based on rows that have been [added, changed, or removed](/getting-started/concepts#change-data-capture-cdc) since the last sync run.
{" "}
This is only applicable when using the **Message** sync type
Each row is sent as an individual message and can be [templated with Liquid](#templating-messages).
You can also send messages to [different Slack channels](#channel-selection).
Hightouch doesn't send any messages on the initial sync run to prevent a flood of messages to your channel. Messages start appearing **after** the initial sync run. The same behavior applies to [full resyncs](/syncs/overview#resync-full-query) as well.
If you want avoid this behavior, modify your model query to return only one "test" row. Hightouch processes this row during the first sync run and doesn't send it as a message to Slack. Afterward, update your model query to include all rows from your source. Hightouch syncs all subsequent rows to Slack during the second sync run.
#### Channel selection
After adding the Hightouch bot to a Slack channel, you can select it from this dropdown menu or enter a channel ID manually.
You can also enable the **Use column** toggle to select a model column instead of a Slack channel name.
The model column selected in the dropdown menu must contain the correct Slack channel ID for every row in your model.
This also allows you to dynamically set the target channel, by inserting different Slack channel IDs for different rows in your model.
#### Private channels
Private channels are hidden from the dropdown. To send to a private channel you can manually enter the channel ID.
Check out the [Common
errors](#slack-channel-not-appearing-in-sync-configuration) section if you
don't see your Slack channel in this menu.
#### Added, changed, and removed messages
Hightouch allows you to send different messages based on the row type:
- when a row is **added** to your query results, Hightouch will send the _row added_ content.
- when a row is **changed** in your query results, Hightouch will send the _row changed_ content.
- when a row is **deleted** from your query results, Hightouch will send the _row deleted_ content.
This content can be [templated](#liquid-templating) and also supports the [Slack Block Kit](#slack-block-kit).
Check out the [Common errors](#parseerror-unexpected-token) section if your
column name contains spaces.
### Syncing batch messages
The batch message sync mode is similar to the normal [message](#syncing-messages) mode, except it sends all rows in the query result rather than only [added, changed, or removed rows](/getting-started/concepts#change-data-capture-cdc).
This sync mode is ideal if you want to write a template and have Hightouch push a message containing **all** rows to Slack, formatted as a single message.
This mode also allows you to specify how many rows are sent per message, which can range from 1 to 50.
With 1 row per message, it behaves similar to the [message](#syncing-messages) sync mode since it creates multiple messages in Slack.
Whereas 50 rows per message is similar to the [table](#syncing-tables) sync mode.
#### Channel selection
The batch message sync mode only supports sending your messages to a single channel, unlike the [message](#channel-selection) mode.
#### Private channels
Private channels are hidden from the dropdown. To send to a private channel you can manually enter the channel ID.
#### Header, body, and footer content
For each message, Hightouch allows setting the header, body, and footer content.
The header and footer are optional, whereas the message body is required.
The body is repeated for every row in your model.
This content can be [templated](#liquid-templating) and also supports the [Slack Block Kit](#slack-block-kit).
Check out the [Common errors](#parseerror-unexpected-token) section if your
column name contains spaces.
### Syncing tables
The table sync mode is similar to the [batch message](#syncing-batch-messages) mode.
It sends all of your query results as a table-formatted message to Slack.
This mode is only recommended for small datasets.
### Syncing CSV files
The CSV sync mode is similar to the [batch message](#syncing-batch-messages) and [table](#syncing-tables) modes.
It sends all of your query results as a CSV file attached to a Slack message.
### Templating messages
[Liquid templating](/syncs/mapping-data#liquid-reference) is supported for the [message](#syncing-messages) and [batch message](#syncing-batch-messages) sync modes.
This allows you to include model columns as part of the message body to insert dynamic values in your Slack messages.
Here is an example message making use of Liquid to include model columns in a message:
```text
User with name {{ row['first name'] }} and email {{ email }} has signed up
```
Check out our [Extended Liquid use cases](/syncs/mapping-data#extended-liquid-use-cases) section to learn how to format [dates](/syncs/mapping-data#date-function), [strings](/syncs/mapping-data#string-functions) or [currency amounts](/syncs/mapping-data#currency-amounts).
#### Tagging Slack users \(@...\)
Hightouch supports tagging users by either their **Slack User ID** or by email:
- **Tagging by User ID:** tag by user ID by writing `<@{{ userId }}>` where `userId` is the column that contains the user's Slack ID. If the `userId` is a static value, you can write something like `<@UL53466SY>`.
- **Tagging by Email:** tag by email by writing `<@{{ email | slack_user_lookup }}>` where email is the column that contains the user's email used in Slack. If the email is a static value, you can write something like `<@{{ "example@gmail.com" | slack_user_lookup }}>`.
#### Inserting URLs
You can insert URLs by highlighting a section of your template and pasting the URL.
Otherwise, you can build URLs directly by following this format: ``.
For example, a URL that points to a Salesforce organization's page could look like this: ``
We recommend using the second method when building dynamic URLs with [liquid templating](#templating-messages).
#### Unfurling links and blocking unfurl
By default, Slack automatically unfurls (displays a preview of) URL links in your messages.
To prevent automatic unfurling, surround the URL with backticks (`` ` ``), for example:
`` `https://www.hightouch.com/` ``
The [backtick](https://en.wikipedia.org/wiki/Backtick) is usually located
above the tab key on the left-hand side of the keyboard.
Blocking URL unfurling isn't possible when using the [Slack Block
Kit](#slack-block-kit). Please{" "}
{" "}
if you would like to request this feature.
#### Formatting text for app for Slack surfaces
Slack provides an [extensive guide](https://api.slack.com/reference/surfaces/formatting) to formatting surfaces in the app for Slack with instructions on how to:
- format text
- insert emojis
- mention users and channels
- format dates, quotes, and lists
{/* */}
### Slack Block Kit
{/* */}
To enable the Slack Block Kit, select **Slack Block Kit** for the content type.
Make sure to use [Slack's Block Kit Builder](https://app.slack.com/block-kit-builder) to draft your message content.
Each message content must be an array of Slack blocks, here is an example of a valid configuration.
The **Test Block Kit** button under the message content allows you to view your templates as a Slack message ([view an example](https://app.slack.com/block-kit-builder/T0215BZEKB2#%7B%22blocks%22:%5B%7B%22type%22:%22section%22,%22text%22:%7B%22type%22:%22mrkdwn%22,%22text%22:%22hello!%22%7D%7D,%7B%22type%22:%22section%22,%22text%22:%7B%22type%22:%22mrkdwn%22,%22text%22:%22email:%20%7B%7B%20email%20%7D%7D%22%7D%7D,%7B%22type%22:%22section%22,%22text%22:%7B%22type%22:%22mrkdwn%22,%22text%22:%22goodbye!%22%7D%7D%5D%7D)).
### Customizing bot appearance
Hightouch also allows you to customize how the Hightouch bot is displayed when sending messages to your Slack channel.
You can find these fields in your sync configuration:
| Name | Description |
| :----------- | :------------------------------------------------------------------------- |
| Display name | Display name of the Hightouch bot |
| Icon URL | URL of a 48x48 image that you wish to use as the bot icon |
| Emoji | Slack emoji to use instead of an icon image (for example, `:sweat_smile:`) |
Keep in mind that users in the channel will be able to see the real bot name and image by clicking on it.
## Tips and troubleshooting
For Hightouch platform error codes related to Slack, see [Error codes: Slack](/developer-tools/error-codes#slack).
### Common errors
#### Slack channel not appearing in sync configuration
If your Slack channel isn't appearing in the [channel selection menu](#channel-selection), make sure that you [added the Hightouch bot](#adding-the-hightouch-bot-to-a-channel) to the channel.
If this still doesn't work, try [reauthorizing your Slack destination](#authenticating-with-oauth) in Hightouch.
As a next troubleshooting step, remove the Hightouch app for Slack from your Slack workspace by visiting the [Hightouch app page](https://hightouchexternal.slack.com/apps/A010ZEEB44R-hightouch?tab=settings&next_id=0), opening the **Configuration** tab, and clicking **Remove App**.
Then [set up a new Slack destination](#authenticating-with-oauth) and [add the Hightouch bot](#adding-the-hightouch-bot-to-a-channel) to your Slack channel once more.
#### Invalid_blocks
Using `"plain_text"` instead of `"mrkdwn"` in your Slack Block Kit syntax is the most common cause of this error.
You can find examples of the correct syntax to use in the [Slack Block Kit section](#slack-block-kit) of this page.
#### Cannot read properties of undefined (reading 'length')
You may receive the `Cannot read properties of undefined (reading 'length')` error when syncing [batch messages](#syncing-batch-messages).
If you're using the [Slack Block Kit](#slack-block-kit) as the content type, your sync might return this error instead:
`Parsing failed with error SyntaxError: Unexpected end of JSON input`.
In either case, make sure to insert **body** content in the sync configuration.
Header and footer content is optional.
{/* */}
#### ParseError: unexpected token
{/* */}
The full error message may look like, `unexpected token at "name", line:1, col:13`. You may receive this error because a column header that is being referenced in your message contains a space. For example:
```text
First Name: {{ first name }}
```
To resolve this error you can try the following options:
- Rename your column so that the name does not contain spaces, for example, `firstName` or `first_name`.
An example will look like:
```text
First Name: {{ first_name }}
```
- If you cannot change the column's name then you can use this language of [liquid templating](/syncs/mapping-data#liquid-reference). An example will look like:
```text
First Name: {{ row['first name'] }}
```
### Live debugger
### Sync alerts
{" "}
---
## Smartsheet
**URL:** https://hightouch.com/docs/destinations/smartsheet
**Description:** Sync your warehouse data to your Smartsheet sheets
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-----------|-----------------------------------|------------------------|-----------------------------------------
**Sheets** | Sync data to sheets in Smartsheet | Upsert, Update, Insert | [Smartsheet API Reference](https://smartsheet.redoc.ly)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Smartsheet
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Smartsheet** and click **Continue**. You can then authenticate Hightouch to **Smartsheet** with either a Smartsheet API key or with OAuth.
### Authenticate with API key
To retrieve an API key, go to your Smarsheet dashboard.
Select **Account** > **Apps & Integrations** in the side nav.
In the **Personal Settings** form that appears, select **API Access**.
On the **API Access** tab, you can either generate a new token by clicking **Generate new access token** or use an existing one.
Enter the access token into **API Key** field in Hightouch and then click **Continue**.
Enter a descriptive name for your destination, and complete setup.
### Authenticate with OAuth
For the **Authentication method**, select **Log in to Smartsheet** and log into your Smartsheet account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Smartsheet destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Smartsheet destination you want to sync to.
### Syncing rows
Hightouch supports inserting, upserting, and updating data as rows into Smartsheet sheets.
Read through the [section below](#duplicate-rows-in-sheet) if you're using Upsert mode and duplicates are being created in your sheet.
#### Record matching
You can match rows from your model to rows in Smartsheet on any column in your model and any field in Smartsheet. Ensure the data types of the model column and Smartsheet field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
We recommend selecting your sheet's [primary column](https://help.smartsheet.com/articles/504645-primary-column) for record matching.
Read through the [section below](#duplicate-rows-in-sheet) if the values you're using for record matching start with a leading zero as that can cause duplicates.
In **Insert** mode, Smartsheet automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
You can sync columns from your model to columns within your selected Smartsheet sheet.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|------------------------------------------------------------
**Do nothing** | Keep the row in Smartsheet with all its synced fields
**Clear** | Clear all the mapped fields, but keep the row in Smartsheet
**Delete** | Delete the synced record from your Smartsheet sheet
## Tips and troubleshooting
### Common errors
#### Duplicate rows in sheet
As explained in the [Values starting with zero](https://help.smartsheet.com/learning-track/level-1-get-started/columns#:~:text=Values%20starting%20with%20zero) section, Smartsheet needs to add a leading apostrophe to any values that start with a leading zero. This process can cause duplicates when updating a sheet via API, so we recommend avoiding any leading zeros in your [record matching](#record-matching) values.
Furthermore, [Upsert mode](#supported-syncing) might create duplicates in your sheet if you run your sync too frequently (such as every 24 hours or more often). This is because it calls Smartsheet's [Search sheet](https://smartsheet.redoc.ly/tag/search/#operation/list-search-sheet) API endpoint, which might not return rows that were added or edited during the past 24 hours.
To avoid this, you can schedule your sync so it doesn't run as frequently or you can replace your Upsert mode sync with a two-part setup:
- for Update mode: one model, sync (set to Update mode), and Smartsheet sheet
- for Insert mode: another model, sync (set to Insert mode), and Smartsheet sheet
The data from the "Insert mode" sheet can then be synced to the "Update mode" sheet by a [copy rows](https://help.smartsheet.com/articles/2479626-automatically-copy-rows-between-sheets) Smartsheet workflow. If the workflow is set up correctly, this eliminates the risk of duplicate rows being created in your "Update mode" sheet.
{/* */}
#### errorCode: 1051
{/* */}
This error can happen if you try syncing data to [locked columns or rows](https://help.smartsheet.com/articles/522077-locking-unlocking-columns-and-rows).
### Live debugger
### Sync alerts
---
## Snapchat
**URL:** https://hightouch.com/docs/destinations/snapchat
**Description:** Empower your marketing team to run highly granular campaigns on Snapchat
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| --------------------- | --------------------------------------------------------- | -------------------- | ---------------------------------------------------------------------------------------------- |
| [**Segments**](#syncing-segments) | Create and keeps audience segments up-to-date in Snapchat | Add, Remove, Update | [Audience segment docs](https://marketingapi.snapchat.com/docs/#create-an-audience-segment) |
| [**Conversion Events**](#syncing-conversion-events) | Create and send conversion events to Snapchat | Insert | [Conversions API docs](https://marketingapi.snapchat.com/docs/conversion.html#conversions-api) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Snapchat
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Snapchat** and click **Continue**. You can then authenticate Hightouch to **Snapchat**.
For the **Authentication method**, select **Log in to Snapchat** and log into your Snapchat account.
Then, authorize Hightouch to access your Snapchat account:
Once successful, you will be redirected back to Hightouch.
There, select the Snapchat business organization you want to sync data to.
Finally enter a descriptive name for your destination to complete setup.
## Sync configuration
Once you've set up your Snapchat destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Snapchat destination you want to sync to.
### Syncing segments
You can use Hightouch to create and keep audience segments in Snapchat up-to-date using [Snapchat's Ads API](https://marketingapi.snapchat.com/docs/).
Begin by selecting the Snapchat Ad Account your audience segments should belong to.
#### Select an existing audience or create a new one
You can create a new audience segment or use an existing one. When creating a new audience, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model. To use an existing audience, select the desired audience from the dropdown.
#### Audience member retention
You can specify a custom retention period that an audience segment should retain members.
Snapchat's default is 9,999 days.
You can only set custom values when first creating an audience segment.
If you want to set a new retention period, you need to create a new sync.
#### User identifiers
To identify which users to add or update in an audience, select model columns and the corresponding Snapchat fields. You can match on any of the following Snapchat fields:
- Email
- Phone
- Mobile ID
### Syncing conversion events
Version 2 will be deprecated in 2025. Hightouch will automatically migrate
syncs to Version 3 before then.
Snapchat's [Conversions API](https://marketingapi.snapchat.com/docs/conversion.html) requires the following event parameters:
- event name (named `event_type` in Snapchat's Conversions API)
- event conversion type (`event_conversion_type`)
- event tag (`event_tag`)
- timestamp
The sync configuration form ensures all these are set and provides some additional options.
For more information about Snapchat's Conversions API and it's prerequisites, check out [Snapchat's API help article](https://businesshelp.snapchat.com/s/article/api-apply?language=en_US).
#### Event name and conversion type
Providing an event name is required to send an event to the Conversions API. You can either provide a static value or select to **use a column** from your model.
You can select your event conversion type in the same way.
Snapchat's Conversions API accepts the following values:
{/* */}
| Name | Values |
| ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `event_type` | "PURCHASE," "SAVE," "START_CHECKOUT," "ADD_CART," "VIEW_CONTENT," "ADD_BILLING," "SIGN_UP," "SEARCH," "PAGE_VIEW," "SUBSCRIBE," "AD_CLICK," "AD_VIEW," "COMPLETE_TUTORIAL," "INVITE," "LOGIN," "SHARE," "RESERVE," "ACHIEVEMENT_UNLOCKED," "ADD_TO_WISHLIST," "SPENT_CREDITS," "RATE," "START_TRIAL," "LIST_VIEW," "APP_INSTALL," "APP_OPEN," "CUSTOM_EVENT_1," "CUSTOM_EVENT_2," "CUSTOM_EVENT_3," "CUSTOM_EVENT_4," "CUSTOM_EVENT_5" |
| `event_conversion_type` | "OFFLINE," "WEB," "MOBILE_APP" |
{/* */}
See the [Snapchat Marketing API documentation](https://marketingapi.snapchat.com/docs/conversion.html#parameters-for-event-type-platform) for more information about event parameters.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server.
If you select a column, it should be in UNIX timestamp, calculated in seconds or milliseconds.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
You must include the **event tag** field that Snapchat's Conversions API requires for each event.
Event tags are custom values you create, for example, "in-store," "weekend sales," "back-to-school campaign," etc.
Check out Snapchat's [Conversion parameters documentation](https://marketingapi.snapchat.com/docs/conversion.html#conversion-parameters) for more information about the parameters you can map.
**Version 3**
The new version of the conversion API takes many of the same parameters as the previous version but they are split into multiple mapping groups.
Checkout Snapchat's version 3 [Conversion parameters documentation](https://docs.snap.com/api/marketing-api/Conversions-API/Parameters) for more information about the parameters you can map.
### PII hashing
Snapchat requires identity information to be normalized and hashed using a SHA256 hash.
Enable PII hashing if you wish for Hightouch to normalize and hash these values for you.
If you turn off PII hashing, you must ensure your model data is hashed before sending it to Snapchat.
## Tips and troubleshooting
### Matched users count
Below only applies to the segment sync type.
### Common errors
### Live debugger
### Sync alerts
---
## Snowflake Iceberg
**URL:** https://hightouch.com/docs/destinations/snowflake-iceberg
**Description:** Hundreds of industry leaders use Hightouch to turn Snowflake into a marketing, sales, success and operational engine
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes |
|--------------|--------------------------------------------------------|------------------------|
| Any data set | Sync data from any source to a Snowflake Iceberg table | Upsert, Insert, Mirror |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
This destination is for Snowflake-managed Iceberg tables. If you want to sync to standard Snowflake tables, use the [Snowflake destination](/destinations/snowflake) instead.
## Prerequisites
Before setting up the Snowflake Iceberg destination, you must have an [external volume](https://docs.snowflake.com/en/user-guide/tables-iceberg-configure-external-volume) configured in Snowflake. Iceberg tables require an external volume that specifies where the table data and metadata are stored.
## Snowflake credential setup
To allow Hightouch access to Snowflake for Iceberg tables, it's best to create a user specifically provisioned with access to the required tables, schemas, and external volumes. You can also use a personal Snowflake login for your credentials, as long as it has the correct permissions, specifically:
- Add, update, and delete ([if applicable](#delete-behavior)) rows in your sync's Iceberg table
- View the `INFORMATION_SCHEMA.COLUMNS` table which is used for gathering metadata to set up the sync
- `USAGE` privilege on the external volume used for Iceberg tables
### Service account setup
You can use the following SQL template to create a service account with the necessary roles and permissions:
```sql
-- Edit the following variables
set ht_username='HIGHTOUCH_USER';
set ht_default_warehouse='';
set ht_database='';
set ht_default_namespace='';
set ht_default_role='HIGHTOUCH_ICEBERG_ROLE';
set ht_external_volume='';
set ht_comment='Used for Hightouch Iceberg integrations';
-- Set role for grants
USE ROLE ACCOUNTADMIN;
-- Create a role for Hightouch
CREATE ROLE IF NOT EXISTS identifier($ht_default_role)
COMMENT = $ht_comment;
-- Create Hightouch's user
CREATE USER IF NOT EXISTS identifier($ht_username)
TYPE=service
rsa_public_key=''
DEFAULT_WAREHOUSE=$ht_default_warehouse
DEFAULT_NAMESPACE=$ht_default_namespace
DEFAULT_ROLE=$ht_default_role
COMMENT=$ht_comment;
-- Grant permissions to the role
GRANT ROLE identifier($ht_default_role) TO ROLE SYSADMIN;
GRANT USAGE ON WAREHOUSE identifier($ht_default_warehouse) TO ROLE identifier($ht_default_role);
GRANT ROLE identifier($ht_default_role) TO USER identifier($ht_username);
GRANT USAGE ON DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT USAGE ON ALL SCHEMAS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON ALL TABLES IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON FUTURE TABLES IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON ALL VIEWS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON FUTURE VIEWS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
-- Iceberg-specific permissions
GRANT USAGE ON EXTERNAL VOLUME identifier($ht_external_volume) TO ROLE identifier($ht_default_role);
GRANT CREATE ICEBERG TABLE ON SCHEMA TO ROLE identifier($ht_default_role);
```
Once you've created a Snowflake service account, you're ready to set up the connection in Hightouch.
## Connect to Snowflake Iceberg
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Snowflake Iceberg** and click **Continue**. If you're not using a [tunnel](#ssh-tunneling), you can then authenticate Hightouch to Snowflake by entering the following fields:
- **Account identifier** - You can find this identifier at the beginning of your Snowflake URL, for example, (`https://ACCOUNT_IDENTIFIER.snowflakecomputing.com`).
- **Warehouse** - This specifies the warehouse to use when Hightouch executes queries in Snowflake.
- **Database** - This specifies the database to use when Hightouch executes queries in Snowflake.
**Account identifier** format may differ based on Snowflake account age. For example, older Snowflake accounts often have identifiers that look like `ACCOUNT_LOCATOR.CLOUD_REGION_ID.CLOUD`, whereas newer Snowflake accounts have identifiers that look like `ORGNAME-ACCOUNT_NAME`.
For more details, visit [Snowflake's account identifier docs](https://docs.snowflake.com/en/user-guide/admin-account-identifier.html).
### Iceberg configuration
- **External volume** - The Snowflake external volume that provides storage for Iceberg tables. This must be configured in your Snowflake account before setting up the destination.
- **Base location** (optional) - A prefix path within the external volume where Iceberg tables will be stored.
### Credentials
- **Username** - This can be your personal Snowflake login or a dedicated user for Hightouch. **Note**: Use the "Login name", which may [differ from the Snowflake username](https://docs.snowflake.com/en/user-guide/admin-user-management#using-classic-console).
- **Role** (optional) - Use this field to specify the role Hightouch should use when executing queries in Snowflake. If left blank, Hightouch uses the user's default role.
You have two options for finalizing your credentials:
- RSA key pair (recommended)
- Password
Snowflake is [deprecating password authentication](https://www.snowflake.com/en/blog/blocking-single-factor-password-authentification/) in favor of RSA key pair authentication or multi-factor authentication. Hightouch will continue to support password authentication for existing customers until they have migrated.
To ensure your credentials are correct, click **Test connection**. This confirms if Hightouch can connect to your database and verify access to the external volume.
### RSA authentication
Generate a [private key](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#step-1-generate-the-private-key) and [public key](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#step-2-generate-a-public-key) by running the following commands in your terminal:
```bash
$ openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8
$ openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub
```
**Save** the password you used for the key, you will need to enter it when creating the Snowflake Iceberg destination in Hightouch.
Then in Snowflake, execute an `ALTER USER` command to assign the public key to your Hightouch Snowflake user:
```bash
alter user ht_user set rsa_public_key='MIIBIjANBgkqh...';
```
Finally, in Hightouch under the **Authentication Method** section, drag and drop or upload the private key you just generated into the **Private key file** field. Enter the password for your private key in the box below here, unless you used an unencrypted key.
### SSH tunneling
Hightouch can connect directly to Snowflake over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your Snowflake instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
## Sync configuration
Once you've set up your Snowflake Iceberg destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Snowflake Iceberg destination you want to sync to.
### Table selection
Select the schema and Iceberg table you want to sync to. You can either select an existing Iceberg table or enable **Automatically create and update Iceberg table** to have Hightouch create a new Iceberg table using your configured external volume. If auto-create is enabled and the table already exists, Hightouch alters its definition to add new columns as needed.
### Sync mode
Hightouch supports **Upsert** mode using the [`MERGE INTO`](https://docs.snowflake.com/en/sql-reference/sql/merge) statement, with the option to [delete removed rows](#delete-behavior), **Insert** mode using the [`COPY INTO`](https://docs.snowflake.com/en/sql-reference/sql/copy-into-table) statement, and **Mirror** mode which will truncate the table and then insert all rows from the model.
**Mirror** mode will remove any existing data in the table before inserting all rows from the model
### Record matching
To match rows from your model to rows in the Iceberg table, Hightouch requires you to select a unique identifier in the table you are syncing to.
The Snowflake column you select must be set as a `UNIQUE`, `PRIMARY KEY` column within Snowflake.
You can see columns that fit this criteria as available options in records matching section.
If there are no fields in the dropdown, you must add a unique type column to your Iceberg table. Then, click the refresh icon to access the newly created column.
In **Insert** mode, Snowflake automatically generates an identifier for every new record synced, so there is no need to match an existing record.
### Column types
Iceberg tables have some data type limitations compared to standard Snowflake tables:
| Supported Types | Notes |
|------------------|----------------------------------------------------------|
| Text (VARCHAR) | Standard text/string data |
| Number | INTEGER, FLOAT, NUMBER |
| Boolean | TRUE/FALSE values |
| Date | DATE type |
| Timestamp (NTZ) | Timestamp without timezone |
| Timestamp (LTZ) | Timestamp with local timezone (recommended for Iceberg) |
| Time | TIME type |
**Semi-structured data types are not supported** by Iceberg tables. This includes `VARIANT`, `OBJECT`, and `ARRAY` types. If your source data contains these types, you must transform them before syncing.
`TIMESTAMP_LTZ` is the recommended timestamp type for Iceberg tables. It internally stores UTC time with a specified precision. All operations are performed in the current session's time zone, controlled by the [TIMEZONE](https://docs.snowflake.com/en/sql-reference/parameters.html#label-timezone) session parameter. Note that `TIMESTAMP_TZ` is not supported by Iceberg tables.
If you see type errors, it may be because your model is producing the wrong format. If so, use [typecasting](/models/data-types-casting) or [liquid templating](https://hightouch.com/docs/syncs/mapping-data#liquid-template-format) to resolve the issue.
### Field mapping
You can sync columns from your model to the columns in your Iceberg table. When auto-create is enabled, you can create new columns by mapping to column names that don't exist yet.
### Batch size
You can tune the number of rows Hightouch upserts or inserts per query based on your needs and database threshold. The default is 50,000 rows per batch.
If you want to increase the sync's speed, you can increase the batch size. Keep in mind that in upsert mode, Snowflake fails the entire batch of rows if it detects any erroneous row. If you suspect that you will have many bad rows, don't use a high batch size. To avoid locks, ensure you account for your database's capacity when increasing the batch size.
### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
|----------------|------------------------------------------------|
| **Do nothing** | Keep the row in the Iceberg table |
| **Delete row** | Remove the row from the Iceberg table entirely |
## Tips and troubleshooting
### External volume errors
If you see errors related to the external volume during connection testing, verify:
- The external volume exists in your Snowflake account
- Your user has `USAGE` privilege on the external volume
- The external volume name is spelled correctly in the destination configuration
### Common errors
### Live debugger
### Sync alerts
---
## Split
**URL:** https://hightouch.com/docs/destinations/split
**Description:** Sync data from your warehouse to Split.io for data-driven feature release and management, enhancing user experience through targeted deployment.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-------------|-------------------------------------------------------|----------------------|-------------------------------------------------------------------
**Users** | Sync data from any source to Split users | Update, Insert | [Users docs](https://docs.split.io/reference/users-overview)
**Segments** | Sync data from any source to new or existing segments | Add, Remove | [Segments docs](https://docs.split.io/reference/segments-overview)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
{/* */}
## Connect to Split
{/* */}
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Split** and click **Continue**. You can then authenticate Hightouch to **Split** by entering your Split **API key**.
To retrieve an API key in Split, click your user icon located at the top left corner of the screen. From the drop-down menu, select **Admin setting**. In the **Workspace settings** section, select **API keys**. You have the option to use an existing key or generate a new one by clicking on the **Actions** button at the top right corner and selecting **Create Admin API key**.
For more information, refer to [Split's API keys article](https://help.split.io/hc/en-us/articles/360019916211-API-keys).
## Sync configuration
Once you've set up your Split destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Split destination you want to sync to.
### Syncing users
When syncing users, you can choose to update or insert them. When inserting new users, they are created with a Pending status.
#### Record matching
You can match rows from your model to users in Split on any column in your model and any user field in Split. Ensure the data types of the model column and Split field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
You can map data from any of your model columns to fields in Split. If you are inserting users, you have the option to include email and groups. Additionally, you can update display name, email, disable 2FA, activate, and deactivate a user. Ensure the data types of your model columns match the data types of the fields you want to sync to.
### Syncing segments
Sync data from any source to new or existing segments. You need to specify the [workspace](https://docs.split.io/reference/workspaces-overview), [environment](https://docs.split.io/reference/environments-overview), and [traffic type](https://docs.split.io/reference/traffic-types-overview) to sync your segment. Segments are limited to 100,000 members and each update can only include 10,000 new members at a time.
#### Record matching
To match rows from your model to records in Split, you need to select the model column that contains the **identifier** field.
#### Create a segment
To create a new segment list, select **Create a new audience**. You can provide a name for this segment, or Hightouch defaults to the name of the model used to create the sync.
When you create a segment, it will be added to the chosen environment. If you enter a name that already exists, segment creation fails.
#### Use an existing segment
To use an existing segment list, select **Use existing segment**. You can select any of your existing Split segments from the drop-down select menu.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Spotify Ads Manager
**URL:** https://hightouch.com/docs/destinations/spotify
**Description:** Sync first-party audiences and conversion events to Spotify Ads Manager for better targeting, suppression, and measurement.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
----------------|--------------------------------------------------------|----------------------|----------------------------------------------------------------------------------------------------
**Audiences** | Sync data from any source to Spotify audiences | All | [Audiences docs](https://developer.spotify.com/documentation/ads-api/reference/v3.0/createAudience)
**Conversions** | Sync data from any source to Spotify conversion events | Insert | [Conversions docs](https://adshelp.spotify.com/s/article/Spotify-Conversions-API-US?language=en_US)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Spotify
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Spotify** and click **Continue**. You can then authenticate Hightouch to **Spotify**.
For the **Authentication method**, select **Log in to Spotify** and log into your Spotify account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Spotify destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Spotify destination you want to sync to.
### Syncing audiences
Sync data from any source to Spotify audiences.
#### Record matching
You can match rows from your model to audience members in Spotify using any of the supported identifiers. Ensure the data types of the model column and Spotify field you select match.
The supported identifiers are:
- Hashed email (`EMAIL_ID_SHA256`)
- Hashed device ID (`DEVICE_ID_SHA256`)
- Hashed phone number (`PHONE_NUMBER_SHA256`)
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync audience fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default audience fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
### Syncing conversions
Sync data from any source to Spotify conversion events.
#### Record matching
You can match conversion events to users in Spotify using any of the supported identifiers. Ensure the data types of the model column and Spotify field you select match.
The supported identifiers are:
- Hashed email (`hashed_emails`)
- Device ID (`device_id`)
- Hashed phone number (`hashed_phone_number`)
- IP address (`ip_address`)
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Sprig
**URL:** https://hightouch.com/docs/destinations/sprig
**Description:** Empower your team to run surveys with AI feedback.
**Section:** Destinations
## Overview
Sprig is a product experience platform that helps teams collect in-context user feedback through surveys, session replays, heatmaps, and AI-powered analysis. Use Hightouch to sync user attributes from your data warehouse to Sprig so you can target studies to the right audiences, filter responses by user properties, and keep your Sprig user profiles enriched with up-to-date data.
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| --------- | ----------- | -------------------- | ------------- |
| [**Users**](#syncing-users) | Sync user profiles and attributes to Sprig | Upsert | [V2 Users API](https://docs.sprig.com/reference/post-user-v2-1) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
---
## Connect to Sprig
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Sprig** and click **Continue**.
### API key
To authenticate, you need a Sprig API key. Sprig uses the [Data Import API](https://docs.sprig.com/docs/apis) key for server-side integrations.
To find your API key in Sprig:
1. In the Sprig navigation pane, click **Integrations**.
2. Under **Sources**, click **Enrichment**.
3. Click **Data Import API**.
4. Copy the API key for the environment you want to sync to (Production or Development).
Paste your API key into the **API Key** field in Hightouch and click **Test connection** to verify the credentials.
Sprig provides separate API keys for Production and Development environments.
Make sure you use the key that matches the environment you want to sync data to.
Refer to the [Sprig API documentation](https://docs.sprig.com/docs/apis#find-your-api-keys) for details.
Configure any default sync alerts for this destination, then click **Continue** to finalize.
---
## Sync configuration
Once you've set up your Sprig destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Sprig destination you want to sync to.
### Syncing users
Sync user profiles and attributes to Sprig. Hightouch uses the [Sprig V2 Users API](https://docs.sprig.com/reference/post-user-v2-1) to upsert user records. When a matching user exists in Sprig, Hightouch updates their attributes. When no match is found, Hightouch creates a new user profile.
#### Record matching
To match rows from your model to users in Sprig, select a model column to map to the Sprig **User ID** field. This is the unique identifier Sprig uses to track individual users across your product.
The User ID should be a stable, unique identifier for each user, such as an internal user ID or database primary key. This must match the User ID you use in your [Sprig SDK installation](https://docs.sprig.com/docs/web-javascript) so that synced attributes are associated with the correct user profiles.
#### Field mapping
Hightouch lets you sync user attributes via [field mapping](/syncs/mapping-data). You can map data from any of your model columns to Sprig user attributes.
Sprig supports attributes with string, numeric, and boolean values. Attributes synced through Hightouch appear in Sprig and can be used to:
- **Filter study audiences**: Target surveys and feedback prompts to users with specific attribute values.
- **Filter study responses**: Drill into collected responses based on user attributes.
Any attribute key you sync that doesn't already exist in Sprig is created automatically. Refer to the [Sprig Attributes documentation](https://docs.sprig.com/docs/attributes) for more details on how attributes work.
Sprig does not support deleting individual attributes from a user profile.
If you sync an attribute in error, you can overwrite its value but you cannot remove the key entirely unless you delete the user's data.
---
## Tips and troubleshooting
### Rate limits
Enterprise Sprig customers have an API rate limit of 1,000 requests per second. If you're on a Starter or Free plan, API access may be limited. Refer to the [Sprig API documentation](https://docs.sprig.com/docs/apis#rate-limiting) for current rate limit details.
### Environment mismatch
Make sure the API key you use in Hightouch corresponds to the correct Sprig environment (Production or Development). Using a Development key will sync data to your Development environment, which won't affect your live studies or production user profiles.
### Common errors
| Error | Cause | Fix |
| ----- | ----- | --- |
| `401 Unauthorized` | The API key is invalid or has been revoked. | Generate a new API key in Sprig under **Integrations > Enrichment > Data Import API** and update the destination in Hightouch. |
| `429 Too Many Requests` | You've exceeded the Sprig API rate limit. | Reduce sync frequency or contact Sprig to increase your rate limit. |
### Live debugger
### Sync alerts
---
## SQL Server
**URL:** https://hightouch.com/docs/destinations/sqlserver
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Connect to SQL Server
### SSH tunneling
Hightouch can connect directly to SQL Server over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your SQL Server instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
The [connection string](#connection-string) method doesn't currently support SSH tunneling. Please {" "} if you're interested in this feature.
### Azure firewall setup
By default, Azure protects your SQL Server from any external IP address from connecting to your SQL Server. Follow these steps to add Hightouch's IP addresses to the firewall:
1. Log into your [Azure Console Dashboard](https://portal.azure.com/#home).
2. Click on **SQL databases** under **Services**.
3. Select the SQL database you want to update.
4. Select **Set server firewall** .
5. Under **Public network access**, select **Selected networks**.
6. Scroll to **Firewall rules** and click **Add a firewall rule**.
7. Allowlist the [Hightouch IP addresses](https://hightouch.com/docs/security/networking#ip-addresses) for your region.
8. Click the **Save** button on the bottom left.
#### Required permissions
Both setup options require:
- Server name \(IP address or Azure hostname\)
- Port number (1433 by default)
- Database name
- SQL Server Username
- SQL Server Password
Your credentials must be able to:
- Add/update/delete rows in your sync's table.
- View these tables:
(\*These are used for gathering metadata to set up the sync.)
- `INFORMATION_SCHEMA.COLUMNS`
- `INFORMATION_SCHEMA.TABLE_CONSTRAINTS`
- `INFORMATION_SCHEMA.CONSTRAINT_COLUMN_USAGE`
- `SYS.COLUMNS`
To ensure your credentials are correct, click **Test connection**. This will confirm
if Hightouch is able to properly connect to your database by running a simple `SELECT` query.
#### Password form
#### Connection string
Connection string must be in this format:
`Server=server;Database=database;User Id=username;Password=password;Encrypt=boolean`
## Sync configuration
Hightouch supports **Upsert** mode with the option to delete removed rows.
Select the schema and table you want to sync to. A common schema to use is the database's default, `dbo`.
### Record matching
Records can be matched from your source to your database table using any primary key or unique column. Hightouch only shows columns that are of the `PRIMARY KEY` or `UNIQUE` type in the dropdown.
**Note**: We do not display `IDENTITY` columns as those can't be edited or set
when inserting.
If there are no fields in the dropdown, you must add a primary key or unique constraint to your table in SQL Server. Then, click the refresh icon to access the newly created column.
### Field mapping
Select the fields from your source that you want to sync to SQL Server.
Make sure you account for column types and columns that are **non-nullable**.
If you upsert any row with the wrong column type or set a null value for a
non-nullable column, the entire batch with that row (up to 1k rows) will fail.
## Tips and troubleshooting
### Common errors
{/* */}
#### Cannot open server 'hightouch-test' request by login
{/* */}
If you see the following error message, you need to [update your firewall settings](#azure-firewall-setup) in your Azure Console.
```json
Cannot open server 'hightouch-test' request by login.
Client with IP address '54.196.30.169' is not allowed to access the server.
To enable access, use the Windows Azure Management Portal or run sp_set_firewall_rule
on the master database to create a firewall rule for this IP address or address range.
It may up take up to five minutes for this change to take effect.
```
#### "Invalid time" error
This error occurs when incorrectly formatted data is sent to a [SQL Server `time` field](https://learn.microsoft.com/en-us/sql/t-sql/data-types/time-transact-sql?view=sql-server-ver16).
To resolve this issue, you need to reformat your model column data. For example, if your model column contains values formatted like `hh:mm:ss`, you need to use the [template mapper](/syncs/mapping-data#template-mapping) to change the data's format to:
```
1970-01-01T{{ model.row['column_name'] }}Z
```
Be sure to replace `column_name` with the name of your time column.
SQL Server accepts these types of values and converts them to the `hh:mm:ss.nnnnnnn` format SQL Server expects.
### Live debugger
### Sync alerts
---
## StackAdapt
**URL:** https://hightouch.com/docs/destinations/stackadapt
**Description:** Syncing data from your warehouse to StackAdapt to enable data-driven advertising decisions and optimized targeting by leveraging comprehensive customer insights.
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| ------------------- | ------------------------------------------------------------------------------- | -------------------- | --------------------------------------------------------------------------------------------- |
| [**CRM Segments**](#syncing-segments) | Sync data from any source to StackAdapt CRM segments | Add | [Segments docs](https://docs.stackadapt.com/graphql/scenarios/crm-audience-creation-with-pii) |
| [**Device Audience**](#syncing-device-audiences) | Sync mobile advertising IDs (MAID / IDFA / GAID) to StackAdapt device audiences | Add | [Segments docs](https://docs.stackadapt.com/graphql/scenarios/crm-audience-creation-with-pii) |
| [**Pixel API**](#syncing-pixel-events) | Sync data from any source to Pixel events | Add | [Pixel API docs](https://docs.stackadapt.com/pixel#http-method) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to StackAdapt
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **StackAdapt** and click **Continue**. You can then authenticate Hightouch to **StackAdapt** by entering the following StackAdapt tokens:
- **Pixel ID**: (Only necessary for using Pixel API). If you don't have a Pixel ID, create one in StackAdapt.
- **GraphQL API token**: (Required for CRM segment and device audience syncs). If you don't have an API token, please reach out to your account manager.
## Sync configuration
Once you've set up your StackAdapt destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the StackAdapt destination you want to sync to.
### Syncing segments
Sync data from any source to StackAdapt CRM Segments
#### Record matching
You can match rows from your model to segment in StackAdapt on any column in your model and any field in StackAdapt. Ensure the data types of the model column and StackAdapt field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
StackAdapt accepts the following fields: address, city, email, first name, last name, phone number, state, and zip code. For hashed PII data, you must select one of the following valid combinations of these identifiers:
- email
- first name, last name, email
- first name, last name, phone number
- first name, last name, zip code
If you do not provide at least one of the possible combinations, StackAdapt will be unable to find matches. You can learn more here: [StackAdapt CRM Audience Creation](https://support.stackadapt.com/hc/en-us/articles/360048698853-CRM-Audience-Creation-with-LiveRamp).
### Syncing device audiences
Sync raw mobile advertising IDs (MAID / IDFA / GAID) to a StackAdapt device audience. StackAdapt does not hash device IDs, so the column you map must contain unhashed identifiers.
#### Sync mode
Choose between sending records in batches via the GraphQL API or uploading a file via a pre-signed URL:
- **Inline API (Default)**: sends added rows from your query results in each run via the GraphQL API.
- **File Upload API**: uploads the device IDs as a file via a pre-signed URL. Recommended for very large audiences.
#### Device ID column
Map a single model column to **Device ID (MAID / IDFA / GAID)**. Rows with missing, empty, or non-string values in this column are skipped and surfaced in the [Live debugger](#live-debugger) as rejected rows.
#### Cross-device targeting
Optionally enable **Cross-device targeting** so StackAdapt extends the audience across related devices for the same user.
#### Advanced
Under **Advanced**, you can override the default batch size. The default is 1,000 rows per batch and the maximum is 10,000. Decreasing the batch size can help if you hit request-size errors.
## Handling PII and hashing
You can sync data to StackAdapt in one of two formats: either as a list of strings containing hashed PII data or as a list of objects containing users' information.
Currently, StackAdapt only supports SHA_1 as the hashing algorithm. For more information, refer to the [StackAdapt documentation on hashed PII data](https://docs.stackadapt.com/graphql/scenarios/crm-audience-creation-with-pii#option-2-use-hashed-pii-data-and-specify-the-hashing-algorithm).
### Syncing pixel events
Sync data from any source to StackAdapt pixel events
#### Field mapping
Hightouch lets you map model columns to event properties. The required event properties are:
- IP address
- Page title
- User agent
- URL of target website
#### Event args
You can also map additional event data that Hightouch sends to StackAdapt via the args property which can be used for pixel rule matching.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Statsig
**URL:** https://hightouch.com/docs/destinations/statsig
**Description:** Sync to Statsig segments and events from your warehouse to analyze product performance and run experiments.
**Section:** Destinations
## Overview
Sync to segments and events from your warehouse to Statsig to analyze product performance and run experiments.
## Supported syncing
| Type | Description | Supported Sync Modes |
| ------------ | -------------------------------------------------- | -------------------- |
| **Segments** | Sync data from any source to segments in Statsig | Add, Remove |
| **Events** | Sync data from any source to log events in Statsig | Insert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Statsig
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination**
button. Select **Statsig** and click **Continue**. You can then authenticate Hightouch to **Statsig** by entering your credentials.
Enter the following fields into Hightouch based on the types of syncs you plan on creating:
- **Console API key**: required for syncing segments
- **Client-SDK API key**: required for syncing events
You can find these API keys in the **Keys & Environments** section on your Statsig settings page. Check out Statsig's [Console API docs](https://docs.statsig.com/console-api/introduction) and [HTTP API docs](https://docs.statsig.com/http-api) for more information about how to create and access these keys.
## Sync configuration
Once you've set up your Statsig destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and
the Statsig destination you want to sync to.
### Segments
Sync data from any source to Statsig [segments](https://docs.statsig.com/console-api/segments). Hightouch only supports syncing to the ID list segment type.
#### Record matching
Hightouch matches rows from your model to Statsig segments by segment **id**. Select the model column that contains these values.
#### Create a segment
To create a new segment list, select **Create a new audience**. You can provide a name for this segment, or Hightouch defaults to the name of the model used to create the sync.
Then, select the **ID type** of the segment list you want Hightouch to create. Hightouch supports user ID and stable ID types.
You can change the ID type to a [custom ID](https://docs.statsig.com/guides/experiment-on-custom-id-types) on the segment's page in Statsig.
#### Use an existing segment
To use an existing segment list, select **Use existing segment**. You can select any of your existing Statsig segments from the drop-down select menu. All available options are of the **ID list** type.
### Events
Sync data from any source to [events](https://docs.statsig.com/guides/logging-events) in Statsig.
Statsig's [event explorer](https://docs.statsig.com/events-explorer) requires you to be on a [Pro Plan](https://www.statsig.com/pricing). You won't be able to view events you send via Hightoucht in the Statsig app without it.
#### Event name
Providing an event name is required to log an event in Statsig. You can either provide a static value or select to **use a column** from your model.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server.
Hightouch converts any `DateTime` values from your source to Unix timestamp as expected by Statsig.
#### Field mapping
You can sync columns from your source to different event and user attributes in the mappings section.
## Tips and troubleshooting
### Rate limits
Statsig has a default rate limit of 10 requests/second and 4500 request/15 minutes on Console API. If you're syncing segments and run into a rate limit error, [schedule](/syncs/schedule-sync-ui) your sync to run less frequently or .
### Common errors
### Live debugger
Hightouch provides complete visibility into the API calls made during each of your sync runs. We recommend reading our
[article on debugging tips and tricks](/syncs/debugger) to learn more.
### Live debugger
### Sync alerts
---
## Stripe
**URL:** https://hightouch.com/docs/destinations/stripe
**Description:** Enrich Stripe with data from your warehouse to create, manage, and drive revenue analysis from your customers, products, prices, subscriptions, and invoices resources.
**Section:** Destinations
## Overview
By combining customer data with subscription data from Stripe, you can derive insights into the most common user actions that turn free accounts into paid ones. Additionally, you can figure out which accounts are at risk of churning when you see little to no activity in paid accounts, allowing you to proactively reach out to at-risk customers before the churn.
Check out our [playbook](https://hightouch.com/playbooks/automating-usage-based-billing) and [blog post](https://hightouch.com/blog/automated-billing) to learn how to automate usage-based billing to streamline your finance operations.
## Supported syncing
|Object Type|Description|Supported Sync Modes| API Reference|
|--|--|--|--|
|**Customers**|This object represents a customer of your business. It lets you create recurring charges and track payments that belong to the same customer. |Upsert, Update, Insert|[Customer endpoints](https://stripe.com/docs/api/customers)|
|**Products**|Products describe the specific goods or services you offer to your customers.|Upsert, Update, Insert|[Product endpoints](https://stripe.com/docs/api/products)|
|**Prices**|Prices define the unit cost, currency, and (optional) billing cycle for both recurring and one-time purchases of products. |Upsert, Update, Insert|[Price endpoints](https://stripe.com/docs/api/prices)|
|**Subscriptions**|Subscriptions allow you to charge a customer on a recurring basis.|Upsert, Update, Insert|[Subscription endpoints](https://stripe.com/docs/api/subscriptions)|
|**Invoices**|Invoices are statements of amounts owed by a customer, and are either generated one-off, or generated periodically from a subscription.|Upsert, Update, Insert|[Invoice endpoints](https://stripe.com/docs/api/invoices)|
|**Quotes**|A Quote is a way to model prices that you'd like to provide to a customer. Once accepted, it will automatically create an invoice, subscription or subscription schedule.|Upsert, Update, Insert|[Quote endpoints](https://stripe.com/docs/api/quotes)|
|**Invoice items**|Invoice Items represent the component lines of an invoice.|Upsert, Update, Insert|[Invoice item endpoints](https://stripe.com/docs/api/invoiceitems)|
|**Meter events**|Meter events report customer usage of a [billing meter](https://docs.stripe.com/billing/subscriptions/usage-based) for usage-based billing. They replace the deprecated usage records endpoint.|Insert|[Meter event endpoints](https://docs.stripe.com/api/billing/meter-event)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Stripe
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Stripe** and click **Continue**. You can then authenticate Hightouch to Stripe with an API key.
You can retrieve your an API key by following these steps:
1. Log in to [Stripe](https://dashboard.stripe.com/login).
2. Visit your [Account Dashboard Homepage](https://dashboard.stripe.com/dashboard).
3. Copy your **Secret key**.
If you don't see your Secret key on your dashboard homepage, you can also access it under **Developers**.
If you haven't created an API key yet, Stripe requires you to enter your business details on the [Stripe onboarding page](https://dashboard.stripe.com/account/onboarding).
## Sync configuration
Once you've set up your Stripe destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Stripe destination you want to sync to.
Your sync configuration options depend on the Stripe object you are syncing to:
- [Customers](#customers)
- [Products](#products)
- [Prices](#prices)
- [Subscriptions](#subscriptions)
- [Invoices](#invoices)
- [Invoice items](#invoice-items)
- [Quotes](#quotes)
- [Meter events](#meter-events)
### Customers
#### Record matching
In **Upsert** mode, you can match customer records from your source to your Stripe workspace on the **Customer Email** field.
In **Update** mode, you can match customer records from your source to your Stripe workspace on either **Customer ID** or **Customer Email**.
#### Field mapping
You can sync columns from your source to Stripe default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
Refer to the [inline mapping docs](/syncs/mapping-data#inline-mapping) to see an example of how to map nested objects like a customer's shipping address.
#### Delete behavior
The delete behavior defines how the sync behaves when the Hightouch sync engine recognizes that a record has been removed from your source. The Stripe integration only supports deletions in **Upsert** mode.
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the customer in Stripe |
| **Delete the Stripe record** | Remove the customer from Stripe |
### Products
#### Record matching
You can match rows in your model with products in Stripe on the following fields on the **Product ID** field.
#### Field mapping
You can sync columns from your source to Stripe default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
Refer to the [inline mapping docs](/syncs/mapping-data#inline-mapping) to see an example of how to map nested objects like a products's metadata.
#### Delete behavior
The delete behavior defines how the sync behaves when the Hightouch sync engine recognizes that a record has been removed from your source. The Stripe integration only supports deletions in **Upsert** mode.
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the product in Stripe |
| **Delete the Stripe record** | Remove the product from Stripe |
### Prices
#### Record matching
You can match rows in your model with prices in Stripe on the following fields on the **Price ID** field.
#### Field mapping
You can sync columns from your source to Stripe default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
Refer to the [inline mapping docs](/syncs/mapping-data#inline-mapping) to see an example of how to map nested objects like a price's `recurring` object.
#### Delete behavior
The Stripe integration doesn't suppport deleting prices.
### Subscriptions
#### Record matching
You can match rows in your model with subscriptions in Stripe on the following fields on the **Subscription ID** field.
#### Field mapping
You can sync columns from your source to Stripe default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
Refer to the [inline mapping docs](/syncs/mapping-data#inline-mapping) to see an example of how to map nested arrays like a subscription's items.
#### Associations
Hightouch can perform a reference look-up to associate a subscription to a customer. After selecting the **Customer ID** Stripe field to map, a reference input appears. Similar to the [**Customer** sync type](#customers), Hightouch can conduct a look-up using a **Customer Email**. Hightouch then sends the respective **Customer ID** in the request to Stripe.
If you select **Customer ID**, a look-up is not required.
#### Delete behavior
The delete behavior defines how the sync behaves when the Hightouch sync engine recognizes that a record has been removed from your source. The Stripe integration only supports deletions in **Upsert** mode.
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the price in Stripe |
| **Delete the Stripe record** | Remove the price from Stripe |
### Invoices
#### Record matching
You can match rows in your model with invoices in Stripe on the following fields on the **Invoice ID** field.
#### Field mapping
You can sync columns from your source to Stripe default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
Refer to the [inline mapping docs](/syncs/mapping-data#inline-mapping) to see an example of how to map nested objects like an invoice's `customer address` object.
#### Associations
Hightouch can perform a reference look-up to associate an invoice to a customer. After selecting the **Customer ID** Stripe field to map, a reference input appears. Similar to the [**Customer** sync type](#customers), Hightouch can conduct a look-up using a **Customer Email**. Hightouch then sends the respective **Customer ID** in the request to Stripe.
If you select **Customer ID**, a look-up is not required.
#### Delete behavior
The delete behavior defines how the sync behaves when the Hightouch sync engine recognizes that a record has been removed from your source. The Stripe integration only supports deletions in **Upsert** mode.
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the invoice in Stripe |
| **Delete the Stripe record** | Remove the invoice from Stripe |
### Invoice items
#### Record matching
You can match rows in your model with invoice items in Stripe on the following fields on the **Invoice Item ID** field.
#### Field mapping
You can sync columns from your source to Stripe default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
Refer to the [inline mapping docs](/syncs/mapping-data#inline-mapping) to see an example of how to map nested arrays like invoice items.
#### Associations
Hightouch can perform a reference look-up to associate an invoice item to a customer. After selecting the **Customer ID** Stripe field to map, a reference input appears. Similar to the [**Customer** sync type](#customers), Hightouch can conduct a look-up using a **Customer Email**. Hightouch then sends the respective **Customer ID** in the request to Stripe.
If you select **Customer ID**, a look-up is not required.
#### Delete behavior
The delete behavior defines how the sync behaves when the Hightouch sync engine recognizes that a record has been removed from your source. The Stripe integration only supports deletions in **Upsert** mode.
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the invoice item in Stripe |
| **Delete the Stripe record** | Remove the invoice item from Stripe.* |
*_Deletion is only supported if the associated invoice item is not attached to an invoice or if it's attached to a draft invoice, otherwise, Hightouch will do nothing to the existing invoice item in Stripe._
### Quotes
#### Record matching
You can match rows in your model with quotes in Stripe on the following fields on the **Quote ID** field.
#### Field mapping
You can sync columns from your source to Stripe default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
Refer to the [inline mapping docs](/syncs/mapping-data#inline-mapping) to see an example of how to map nested objects like a quote's `computed` object.
#### Delete behavior
The delete behavior defines how the sync behaves when the Hightouch sync engine recognizes that a record has been removed from your source. The Stripe integration only supports deletions in **Upsert** mode.
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the quote in Stripe |
| **Delete the Stripe record** | Remove the quote from Stripe |
### Meter events
Meter events report customer usage against a [billing meter](https://docs.stripe.com/billing/subscriptions/usage-based) and are the recommended way to send usage-based billing data now that Stripe has deprecated usage records. Meter events only support **Insert** mode.
#### Field mapping
| Field | Description |
| -- | -- |
| **event_name** | Required. The event name of the meter this usage is reported against. |
| **stripe_customer_id** | Required. The Stripe customer ID the usage belongs to. This is the default key Stripe reads for customer mapping. |
| **value** | Required. The usage quantity to record for the event. |
| **identifier** | Optional. A unique identifier for the event. Stripe enforces uniqueness over a rolling 24-hour period, so use it to de-duplicate retried rows. If omitted, Stripe generates one. |
If your meter is configured with custom payload keys, map those columns under **custom fields** — they are added to the event payload alongside the customer and value.
#### Event timestamp
You can optionally map a column containing the event timestamp. Provide it in ISO 8601 format or as Unix time in milliseconds; Hightouch converts it to the Unix seconds Stripe expects. Stripe requires the timestamp to fall within the past 35 days and no more than 5 minutes in the future. If left blank, Stripe records the event at the time it arrives.
## Tips and troubleshooting
### Common errors
#### Mismatched or missing data
Some Stripe object types have conditionally required fields based on other field values. You may receive Stripe API errors if the data you are syncing is incorrectly structured or if expected data is missing. We strongly recommend using [Stripe's API documentation](#supported-syncing) as reference when creating your Hightouch models and sync configurations.
If you run into any issues, . We're here to help.
### Live debugger
### Sync alerts
---
## PAR Retail Stuzo Open Commerce
**URL:** https://hightouch.com/docs/destinations/stuzo
**Description:** Stuzo helps Convenience & Fuel Retailers Know and Activate more customers and data in real-time, leading to more visits, more gallons, and bigger baskets.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
-------------|-----------------------------------------------------------------|---------------------
**Segments** | Sync data from any source to PAR Retail segments | Add, Remove
**Events** | Sync data from any source to track loyalty points in PAR Retail | Insert
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to PAR Retail Open Commerce
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **PAR Retail Open Commerce** and click **Continue**. You can then authenticate Hightouch to **PAR Retail Open Commerce**.
Enter the following fields into Hightouch:
- **API key**: Assigned by Open Commerce and used to identify the communicating application
- **Client ID**: Shared by Open Commerce and used to generate an auth token
- **Client secret**: Shared by Open Commerce and used to generate an auth token
## Sync configuration
Once you've set up your PAR Retail Open Commerce destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the PAR Retail Open Commerce destination you want to sync to.
### Syncing segments
Sync data from any source to PAR Retail Open Commerce segments
#### Record matching
To match rows from your model to segments in PAR Retail Open Commerce, you need to select the model column that contains values that match the **Membership UUID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync segment fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default segment fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Split retries
PAR Retail Open Commerce counts all segments in a batch as rejected if the request contains a single invalid segment. To pinpoint which segments are getting rejected with which errors and reduce the number of valid segments that get retried, you can enable [split retries](/syncs/retries#split-retries).
### Syncing events
Sync data from any source to track loyalty points in PAR Retail Open Commerce
#### Record matching
You can match rows from your model to events in PAR Retail Open Commerce on any column in your model and any field in PAR Retail Open Commerce. Ensure the data types of the model column and PAR Retail Open Commerce field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## SugarCRM
**URL:** https://hightouch.com/docs/destinations/sugar-crm
**Description:** Feed the funnel and build the right pipeline when you choose the marketing automation platform built for marketing mavericks. Create campaigns, engage one-to-one at scale, predict needs, and understand what is working. With Market, you can let the platform do the work.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|-------------|-----------------------------------------------|----------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Objects** | Sync data from any source to SugarCRM objects | Upsert, Update | [Objects docs](https://support.sugarcrm.com/documentation/sugar_developer/sugar_developer_guide_14.1/integration/web_services/rest_api/endpoints/module_post/) [Bulk docs](https://support.sugarcrm.com/documentation/sugar_developer/sugar_developer_guide_14.1/integration/web_services/rest_api/endpoints/bulk_post/) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to SugarCRM
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **SugarCRM** and click **Continue**. You can then authenticate Hightouch to **SugarCRM**.
Enter the following fields into Hightouch:
- **URL** (Required): The base URL of your SugarCRM account
- **Username** (Required): The username of the user authenticating to the system.
- **Password** (Required): The plaintext password the user authenticating to the system.
## Sync configuration
Once you've set up your SugarCRM destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the SugarCRM destination you want to sync to.
### Syncing objects
Sync data from any source to SugarCRM objects. Currently supported objects are:
- **Contacts**
- **Accounts**
- **Opportunities**
#### Record matching
You can match rows from your model to objects in SugarCRM on any column in your model and any field in SugarCRM. Ensure the data types of the model column and SugarCRM field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default contact fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
|----------------|---------------------------------------------------------------|
| **Do nothing** | Keep the contact in SugarCRM with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the contact in SugarCRM |
| **Delete** | Delete the synced object from SugarCRM |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Taboola
**URL:** https://hightouch.com/docs/destinations/taboola
**Description:** Reach your audiences in their moment of next, when they finish consuming content and are most likely to convert.
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| --------------- | --------------------------------------------------------- | -------------------- | -------------------------------------------------------------------------------------------- |
| [**Audiences**](#syncing-audiences) | Sync data from any model to Taboola first-party audiences | Add, Remove | [Audiences docs](https://developers.taboola.com/backstage-api/reference/add-or-remove-users) |
| [**Conversions**](#syncing-conversions) | Sync data from any model to Taboola conversions | Add | [Conversion docs](https://help.taboola.com/hc/en-us/article_attachments/7080790629271) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Taboola
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Taboola** and click **Continue**. You can then authenticate Hightouch to **Taboola** by entering the following required fields into Hightouch:
- **Account ID**: Taboola advertiser ID
- **Numeric Account ID**: Unique identifier for your Taboola account
- **Client ID**: Taboola Backstage API Client ID
- **Client Secret**: Taboola Backstage API Client Secret
Taboola provides the account ID, the client ID, and client secret to you during the [API on-boarding](https://developers.taboola.com/backstage-api/reference/welcome) process.
To get access to the Taboola Backstage API, reach out to Taboola's customer success team.
You can find your Taboola numeric account ID in the URL when signed in to Taboola. Example: `https://ads.taboola.com/campaigns?locale=en&accountId=`
## Sync configuration
Once you've set up your Taboola destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Taboola destination you want to sync to.
### Syncing audiences
Sync data from any model to Taboola [first-party audiences](https://developers.taboola.com/backstage-api/reference/onboarding-overview).
#### User identifiers
To identify which users to add or update in an audience, select model columns and the corresponding Taboola fields.
You can match on any of the following Taboola fields:
- **Email**
- **Device ID**
- **Taboola ID**
#### Handling PII and hashing
By default, Hightouch automatically hashes the **Email** fields before sending requests to Taboola.
You can disable this behavior in the sync configuration. If disabled, the data from the model should be appropriately normalized and hashed according to [Taboola's hashing requirements](https://developers.taboola.com/backstage-api/reference/user-id-object).
### Syncing conversions
Sync data from any model to Taboola [server-to-server conversions](https://help.taboola.com/hc/en-us/articles/115006850567-How-to-Track-Conversions-Using-Server-to-Server-Integration-S2S-).
#### Field mapping
Hightouch lets you sync conversion event properties via field mapping.
## Tips and troubleshooting
For Hightouch platform error codes related to Taboola, see [Error codes: Taboola](/developer-tools/error-codes#other-destinations).
### Matched users count
Below only applies to the audiences sync type.
### Common errors
### Live debugger
### Sync alerts
---
## Talon.One
**URL:** https://hightouch.com/docs/destinations/talon-one
**Description:** Sync customer data to Talon.One audiences
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
|----------------------|------------------------------------------------------------------------|----------------------|------------------------------------------------------------------------------------|
| **Customer Profile** | Sync data from any source to a Talon.One audience as customer profiles | Upsert | [Customer Profile Audiences](https://docs.talon.one/integration-api#tag/Audiences) |
| **Events** | Sync data from any source to a Talon.One events | Insert | [Events](https://docs.talon.one/integration-api#tag/Events/operation/trackEventV2) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Talon.One
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Talon.One** and click **Continue**.
Enter your Talon.One base URL. To find your base URL, simply navigate to your Talon.One Campaign Manager home page and enter the URL.
This should be in the format `https://.talon.one/`.
You will need an Integration API key and a Management API key to successfully sync data from Hightouch.
If you do not have an Integration API key, follow the instructions [here](https://docs.talon.one/integration-api#section/Authentication/api_key_v1) to create one. If you have multiple Integration API keys, choose one to serve as your default.
If you do not have a Management API key, follow the instructions [here](https://docs.talon.one/management-api#section/Authentication/management_key) to create one. Enable access to the `v1/audiences`, `/v1/customer_search`, and `/v1/attributes`endpoint when creating your Management API key for the optimal experience. If you are connecting multiple applications, ensure this key has access to all of the applications.
Enter both of these API keys into Hightouch, and test that your connection is successful for both APIs.
You can connect multiple Talon.One applications in one Hightouch destination by entering multiple Integration API keys in the **Multiple application configuration** section. Enter an application name and corresponding Integration API key. When configuring your sync configuration, you can choose which application you'd like to sync to.
## Sync configuration
Once you've set up your Talon.One destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Talon.One destination you want to sync to.
### Syncing audiences
#### Audience selection
Select whether you'd like to create a new audience or sync to an existing audience. If you would like to create a new audience, enter the name of the audience in the sync configuration form.
#### Record matching
Talon.One uses `Profile Integration ID` to identify each customer profile. Choose which column in your model contains the Profile Integration ID.
#### Rule Engine
By default, adding customer profiles to Talon.One audiences does not trigger the Rule Engine. If you have defined [rules](https://docs.talon.one/docs/product/rules/overview) in Talon.One that dynamically apply effects to customers, enable this option.
This option is **less performant** than the default option, so you may observe slower sync speeds.
#### Custom attributes mapping
If you have created customer profile custom attributes, you can map to those attribute values in Hightouch. Choose which column in your model should be mapped to each attribute.
Customer profile custom attributes cannot be created in Hightouch. To create new custom attributes, follow the [instructions on Talon.One](https://docs.talon.one/docs/product/account/dev-tools/managing-attributes#creating-a-custom-attribute).
#### Delete behavior
When records leave your query result, they are removed from your audience by default. If you would not like to delete any records from your audience, you can choose to do nothing when records leave your query result.
### Syncing events
Sync data from any source to events in Talon.One.
#### Record matching
You can match rows from your model to events in Talon.One on any column in your model and the **Profile ID** field in Talone.One.
The Talon.One events API requires the **Profile ID** field, so you must map it to complete your configuration.
#### Attributes mapping
Hightouch lets you sync Talon.One event attributes that have already been created in your Talon.One account via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## TikTok
**URL:** https://hightouch.com/docs/destinations/tiktok
**Description:** Empower your marketing team by utilizing web events in TikTok
**Section:** Destinations
## Overview
With Hightouch and the TikTok marketing API, you can send data from your warehouse to events and custom audiences in TikTok.
Deliver relevant ads by utilizing data from various sources within your data warehouse.
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| -------------------- | --------------------------------------------------------- | ---------------------- | ------------------------------------------------------------------------------------- |
| [**Web events**](#syncing-web-events) | Sync data from any source to TikTok's pixel events. | Insert | [Events API](https://business-api.tiktok.com/portal/docs?id=1739289524101122) |
| [**Offline events**](#syncing-offline-events) | Sync data from any source to TikTok's offline events. | Insert | [Offline Events API](https://ads.tiktok.com/marketing_api/docs?id=1758427539356673) |
| [**App events**](#syncing-app-events) | Sync data from any source to TikTok's app events. | Insert | [App Events API](https://business-api.tiktok.com/portal/docs?id=1740857868662786) |
| [**Custom audiences**](#syncing-custom-audiences) | Create audience lists and keep them up-to-date in TikTok. | Add, All (file upload) | [Custom Audiences API](https://ads.tiktok.com/marketing_api/docs?id=1739940504185857) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to TikTok
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **TikTok** and click **Continue**. You can then authenticate Hightouch to TikTok via OAuth.
Click **Log in to TikTok**, log in to your TikTok business account, and then click **Authorize**.
Once you've finished, Hightouch displays an **Authorization successful** message. To finish connecting, give your TikTok destination a descriptive name.
## Sync configuration
Once you've connected your TikTok destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the TikTok destination you want to sync to.
### Syncing web events
Syncing to TikTok's [web events](https://ads.tiktok.com/help/article?aid=10003669) lets you report different events from your source model.
#### Pixel code
You need to provide your TikTok pixel code to sync web events. To find the pixel code, navigate to your [TikTok Marketing Dashboard](https://ads.tiktok.com)
and go to **Assets menu** > **Manage Web Events**. Select the pixel you want to sync to and copy the ID.
Check out our [blog post](https://hightouch.com/blog/what-is-a-tiktok-pixel)
to learn how to implement a TikTok Pixel to help track conversions.
#### Event name
You can specify the event name or type, by a static value or based on a column in your model. The name must be one of these accepted values:
AddPaymentInfo, AddToCart, AddToWishlist, ClickButton, CompletePayment, CompleteRegistration, Contact, Download, InitiateCheckout,
PlaceAnOrder, Search, SubmitForm, Subscribe, or ViewContent.
#### Field mapping
You can send additional details on your web events such as the timestamp, event (object) properties, and context (customer) properties.
TikTok recommends that you provide as much context information as possible to increase probability of matching
web events with TikTok ads.
If mapping the `Click Id` field, TikTok recommends that you also send the
`Page url` field for best results.
#### Batch size
You can increase the TikTok events API batch size up to 1,000 per call. This can improve your
sync's speed performance. The default is one event per API call. Keep in mind that
TikTok fails an entire batch if it detects an erroneous row. If you suspect
that you could have erroring rows, don't use a high batch size.
#### Test event codes
Test event codes are special codes you can include in your event payload to simulate events. This allows you to test your API implementation without actually recording real event data.
You can optionally enter a test event code as part of your sync configuration. Navigate to **Events Manager** by clicking the **Assets** tab, **Event** and then **Manage Web Events**. You can find your test event code in the **Test Events** tab.
Make sure to remove the test event code if you're running a production sync.
#### PII hashing
TikTok requires customer identity information (email, external_id, and phone_number) to be hashed using a SHA256 hash.
By default, Hightouch automatically hashes these properties for you but you can choose to opt out of this and
hash the identity information yourself.
### Syncing offline events
Syncing to TikTok's [offline events](https://ads.tiktok.com/marketing_api/docs?id=1758427539356673) lets you
track and measure offline activity from people that see or interact with your ads.
Syncing offline events is an allowlist-only feature in TikTok. If you want
access, please contact your TikTok representative. If you initially configured
your TikTok destination before 06/06/23, you need to
[re-authenticate](#connect-to-tiktok) your connection to TikTok.
#### Event set ID
You need to select an event set ID to sync your offline events. You can create a new event set in your TikTok offline events manager dashboard.
#### Event name
You can specify the event name or type, by a static value or based on a column in your model. The name must be one of these accepted values:
AddPaymentInfo, AddToCart, AddToWishlist, ClickButton, CompletePayment, CompleteRegistration, Contact, Download, InitiateCheckout,
PlaceAnOrder, Search, SubmitForm, Subscribe, or ViewContent.
#### Field mapping
You need, at minimum, the event name and at least one user property, emails or phones. The emails and phones array values must include at least one value,
but you can include multiple elements. If you have single email or phone number as strings in a model column, Hightouch converts them into arrays with one element.
#### Batch size
You can increase the TikTok events API batch size up to 100 per call. This can improve your
sync's speed performance. The default is 100 events per API call. Keep in mind that
TikTok fails an entire batch if it detects an erroneous row. If you suspect
that you could have erroring rows, don't use a high batch size.
#### PII hashing
TikTok requires user identity information (emails and phone_numbers) to be hashed using a SHA256 hash.
By default, Hightouch automatically hashes these properties for you but you can choose to opt out of this and
hash the identity information yourself.
### Syncing app events
Syncing to TikTok's [app events](https://business-api.tiktok.com/portal/docs?id=1740857868662786) lets you
track mobile app activity like launching the app or in-app purchases.
Syncing app events is an allowlist-only feature in TikTok. If you want access,
please contact your TikTok representative. If you initially configured your
TikTok destination before 06/06/23, you need to
[re-authenticate](#connect-to-tiktok) your connection to TikTok.
#### TikTok app ID
You need to select an app ID to sync your app events. You can create a new app set in your TikTok ads manager dashboard.
To learn about how to generate a TikTok App ID, see [TikTok App ID](https://ads.tiktok.com/help/article/how-to-generate-a-tiktok-app-id).
#### Field mapping
You can send additional details on your app events such as the timestamp, event properties, and context properties.
TikTok recommends that you provide as much context information as possible to increase the probability of matching
app events with TikTok ads.
#### Batch size
You can increase the TikTok events API batch size up to 100 per call. This can improve your
sync's speed performance. The default is 100 events per API call. Keep in mind that
TikTok fails an entire batch if it detects an erroneous row. If you suspect
that you could have erroring rows, don't use a high batch size.
### Syncing custom audiences
Hightouch lets you create and maintain custom audiences via the [File upload API](https://ads.tiktok.com/marketing_api/docs?id=1739940501023746) or [Segment API](https://ads.tiktok.com/marketing_api/docs?id=1739940504185857).
You can read more on the different APIs and decide which is best for your use case in [TikTok's docs](https://ads.tiktok.com/marketing_api/docs?id=1739566528222210).
{/* */}
#### Segment API mode (Recommended)
{/* */}
TikTok lets you make real-time batch updates through their [Segment API](https://ads.tiktok.com/marketing_api/docs?id=1739940504185857). Hightouch sends data to the selected audience list
as data gets added in your source model. We recommend this integration type because the API is newer and the integration is maintained by our team.
#### User identifiers
To identify which users to add or update in an audience, select model columns and the corresponding Tiktok fields. You can match on any of the following hashed user fields:
- IDFA_SHA256
- IDFA_MD5
- GAID_SHA256
- GAID_MD5
- EMAIL_SHA256
- PHONE_SHA256
Hightouch automatically hashes the selected properties for you unless you specify otherwise.
{/* */}
#### File upload API mode (Deprecated)
{/* */}
While TikTok still supports this API, Hightouch has deprecated the
integration. We no longer add enhancements to the File upload API, and by end
of 2023, we will programmatically migrate existing syncs to the Segment API
mode and fully sunset this functionality.
TikTok lets you upload an entire audience through their [file upload API](https://ads.tiktok.com/marketing_api/docs?id=1739940567842818). Hightouch uploads the entire model query results for each sync run.
These results effectively replaces the previous file upload.
#### Record matching
The File upload mode lets you match on any one of these hashed user properties:
- IDFA_SHA256
- IDFA_MD5
- GAID_SHA256
- GAID_MD5
- EMAIL_SHA256
- or PHONE_SHA256
The property contains the name of what type of hash TikTok expects. Hightouch
automatically hashes the selected property for you unless you specify otherwise. Hightouch also creates the audience list,
if it doesn't exist yet, with the selected property's hash type.
#### Advertiser account
For both modes, you need to select the advertiser account you want to sync to.
Editing the advertiser account in the sync configuration is only possible
before triggering the first sync run. Otherwise, making this change results in
an error. If you want to use a different advertiser account once the sync has
already run, you need to create a new sync.
#### Custom audience list
Hightouch lets you use an existing audience in TikTok or create a new one for both modes. Note that if you're using Segment mode, you can't
change your audience list name once your sync has run, instead, you will have to create a new sync.
## Tips and troubleshooting
For Hightouch platform error codes related to TikTok, see [Error codes: TikTok](/developer-tools/error-codes#tiktok).
### Matched users count
Below only applies to the custom audiences sync type.
### Common errors
#### Custom audiences
If you want to switch your custom audience sync mode from [File upload API](#file-upload-api-mode-deprecated) to [Segment API](#segment-api-mode-recommended), you need to run a [full resync](/syncs/overview#resync-full-query) after saving the new configuration.
#### Error 40001: No permission to operate advertiser
This error can happen if the TikTok user credentials used to [connect to TikTok](#connect-to-tiktok) don't have the necessary permissions to send data to the [advertiser account](#advertiser-account) selected in the sync configuration. When this error happens, you might also see that the advertiser account dropdown menu in the sync configuration is empty.
To solve this, make sure that the credentials used to connect TikTok to Hightouch have the necessary permissions to send data to your advertiser accounts.
#### File upload API mode
Audiences with fewer than 1000 users are invalid and won't be available in TikTok. For more information, please refer to [TikTok's documentation](https://ads.tiktok.com/help/article?aid=9611).
TikTok allows one file-based audience upload per day per audience. See TikTok's [docs](https://ads.tiktok.com/marketing_api/docs?id=1708580518247426) for more information on their limits.
#### Web events
Most errors that occur with web event syncs are validation errors, like an invalid currency code or phone number. TikTok rejects any requests with invalid data, so Hightouch preemptively rejects these rows to make syncs run more efficiently. Please refer to TikTok's [documentation](https://ads.tiktok.com/marketing_api/docs?id=1740858565852225) to learn more about the expected phone number format and currency codes.
TikTok may update their currency code list.
### Live debugger
Hightouch provides complete visibility into the API calls made during each of your TikTok sync runs, _except for syncs in custom audience file upload mode_. In this mode, Hightouch syncs the full query results set, and therefore doesn't make a [change data capture computation](/getting-started/concepts#change-data-capture-cdc). As a result, you won't see any payload details in the [Live debugger](/syncs/debugger#inspecting-payloads).
For the web event and custom audience segment mode syncs, we recommend reading our [article on debugging tips and tricks](/syncs/debugger) to learn more.
### Live debugger
### Sync alerts
---
## Topsort
**URL:** https://hightouch.com/docs/destinations/topsort
**Description:** Use Topsort to quickly turn your marketplace into a high-performing ad revenue engine with AI-driven, privacy-first infrastructure.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|--------------|-----------------------------------------------|----------------------|-----------------------------------------------------------------------------|
| **Segments** | Sync data from any source to Topsort segments | Add, Remove | [Segments docs](https://docs.topsort.com/en/api-reference/segments-service) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Topsort
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Topsort** and click **Continue**. You can then authenticate Hightouch to **Topsort**.
Enter the following fields into Hightouch:
- **API key**: To obtain your API key, refer to the Topsort [API documentation](https://docs.topsort.com/en/api-reference/authentication).
## Sync configuration
Once you've set up your Topsort destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Topsort destination you want to sync to.
### Syncing segments
Sync data from any source to Topsort segments.
#### Record matching
To match rows from your model to segments in Topsort, you need to select the model column that contains values that match the **User ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Totango
**URL:** https://hightouch.com/docs/destinations/totango
**Description:** Enrich your customer success platform with rich customer data from your warehouse to improve product adoption and reduce churn
**Section:** Destinations
## Setup
To configure your Totango destination you will need to provide the following information: Service ID and API Key.
### Service ID
To find your Service ID, you'll need to go to the upper right hand corner of the Totango dashboard, which is your profile picture. It will be listed at the bottom of the small context window that pops up.
Your service ID is listed as a string which we only need one part. In the screenshot above, the full service ID is 'SP-229775-0'. We will only need the middle part of the ID to authenticate with Totango. Using the example provided, you would only need to use '229775' when filling out the destination details on the Hightouch destination setup page.
### API key
To find your API Key, you'll need to go back to your profile picture.
When you click it, you will want to go to 'Edit profile'.
Once in your profile you'll want to go to the 'Integrations' page.
On the Integrations you will find your API Token Key. You will use this key when filling out the destination details on the Hightouch destination setup page.
## Supported modes
The Totango destination supports the upsert mode:
- **Upsert**: Upsert mode pushes new accounts or account users to Totango and updates fields that change in your warehouse.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## The Trade Desk
**URL:** https://hightouch.com/docs/destinations/tradedesk
**Description:** Sync audiences and conversion events to The Trade Desk using data from your warehouse for first-party optimization, third-party monetization, and retail media use-cases.
**Section:** Destinations
## Overview
Power your The Trade Desk platform with up-to-date data straight from your data warehouse. Sync first-party data or use highly customizable postback URLs to handle data your way. Keeping your data updated equips your team with the necessary tools they need to deliver a top of the line advertising experience.
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| ----------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | -------------------- | -------------------------------------------------------------------------------------------------------------- |
| [**Postback events**](#syncing-postback-events) | Sync data from any source to The Trade Desk via a postback URL | Insert | [Conversion Events API](https://partner.thetradedesk.com/v3/portal/data/doc/DataConversionEventsApi) |
| [**Real-time conversion events**](#syncing-real-time-conversion-events) | Sync data from any source to The Trade Desk as real-time conversion events | Insert | [Real-time Conversions API](https://partner.thetradedesk.com/v3/portal/data/doc/post-track-realtimeconversion) |
| [**Offline conversion events**](#syncing-offline-conversion-events) | Sync data from any source to The Trade Desk as offline conversion events | Insert | [Offline Conversions API](https://partner.thetradedesk.com/v3/portal/data/doc/DataOfflineMeasurement) |
| [**CRM data segments**](#syncing-crm-data-segments) | Sync your Customer relationship management (CRM) data from any source to The Trade Desk data segments | Add, Remove | [CRM Data API](https://partner.thetradedesk.com/v3/portal/data/doc/DataIntegrateCRMData) |
| [**CRM data conversions**](#syncing-crm-data-conversions) | Sync query results as conversions to CRM Data segments | Insert | [CRM Data API](https://partner.thetradedesk.com/v3/portal/api/area/Crm%20Data) |
| [**First-party data segments**](#first-and-third-party-data-segments) | Sync your first-party data from any source to The Trade Desk first-party data segments | Add | [First-party Data API](https://partner.thetradedesk.com/v3/portal/data/doc/post-data-advertiser-firstparty) |
| [**Third-party data segments**](#first-and-third-party-data-segments) | Sync your third-party data from any source to The Trade Desk third-party data segments | Add | [Third-party Data API](https://partner.thetradedesk.com/v3/portal/data/doc/post-data-thirdparty) |
| [**Event opt-out or deletion**](#syncing-event-opt-out-or-deletion) | Opt-out or fully delete customer data from The Trade Desk | Insert | [Deletion/Opt-out API](https://partner.thetradedesk.com/v3/portal/data/doc/DataApiReferenceDeletionOptout) |
## Getting started
You need to enter different credentials depending on whether you're syncing events, CRM data segments, or first-party data segments. If you plan to use multiple sync types, enter the necessary credentials for all of them. Entering all the required credentials lets you use the same destination setup for every Trade Desk sync.
Once you have entered your inputs, press the **Test Connection** button to check you have inputted valid credentials.
### Postback events credentials
For Hightouch to sync postback events, you need to enter a root postback URL.
### Conversion events credentials
For Hightouch to sync real-time conversion events, you need the following:
- Advertiser ID
### Offline conversion events credentials
For Hightouch to sync offline conversion events, you need the following:
- Advertiser ID
- Offline data provider ID
- Offline data provider secret key
If you would like to create offline tracking tags through Hightouch, an **API token** is also required.
### CRM data segment credentials (not recommended)
For Hightouch to sync data to your CRM data segments, you need the following:
- Advertiser ID
- Environment - select either your `Production` or `Sandbox` environment
- API token
You can create an API token by logging onto The Trade Desk platform using your API credentials. These API credentials can be provided by your account manager. Navigate to the **Manage API Tokens** option under the user profile menu at the top-right of The Trade Desk UI. Then either create a Long- or Short-Lived API token. Once you have copied the API token, navigate back to Hightouch and paste the API token.
Hightouch recommends [first-party data
segments](#first-and-third-party-data-segments) for audience syncing and
[offline conversion events](#syncing-offline-conversion-events) for conversion
tracking instead of CRM data segments. If you're sending emails or phone
numbers, use [ID Express](#id-express) to convert them to UID2 identifiers.
CRM data segments only support one identifier type per sync and require
unique, correctly hashed IDs, which makes them more prone to sync failures
than these other options.
### First-party data segment credentials
For Hightouch to sync data to your first-party data segments, you need the following:
- Advertiser ID
- Environment - select a server location of your choice
- Advertiser secret key
You can obtain your decoded advertiser secret key for uploading first-party data from your advertiser preferences page in The Trade Desk UI. If you can't find your secret key, contact your Account Manager. When you send data, the included secret key is validated against the secret key the platform has on file for the Advertiser ID.
### Third-party data segment credentials
For Hightouch to sync data to your third-party data segments, you need the following:
- Data provider ID
- Environment - select a server location of your choice
- Data provider secret key
You can obtain your decoded data provider secret key for uploading third-party data from your advertiser preferences page in The Trade Desk UI. If you can't find your secret key, contact your Account Manager. When you send data, the included secret key is validated against the secret key the platform has on file for the data provider ID.
### Event opt-out or deletion credentials
For Hightouch to sync opt-out or deletion requests, you need different credentials depending on the data type:
- **First-party data (Advertiser)**: Requires the Advertiser ID and Advertiser secret key from your first-party data credentials
- **Third-party data**: Requires the Data provider ID and Data provider secret key from your third-party data credentials
- **Merchant data**: Requires the Data provider ID and Data provider secret key from your third-party data credentials
When setting up for first-party data segments, upload your data to the
hostname closest to the location of the users targeted by your campaign. Using
the closest hostname reduces latency. After a short delay, Hightouch
replicates the data to other data centers globally. Please don't upload the
same data to more than one data center. Certain data types have exceptions
including how they replicate to countries with specific privacy regulations.
Refer to [The Trade Desk
docs](https://api.thetradedesk.com/v3/portal/data/doc/DataEnvironments) for
more details.
## Sync configuration
### Syncing postback events
Hightouch supports inserting impression data as events to The Trade Desk.
#### Field mapping
You can sync columns from your source to The Trade Desk event custom fields.
### Syncing real-time conversion events
Sync data from any source to The Trade Desk Ads as real-time conversion events.
#### Field mapping
Hightouch lets you sync event properties via field mappings.
Hightouch provides the ability to convert hashed emails and phone identifiers
to UID2 identifiers during sync execution. UID2 values will be sent as the
`adid` value on your event. See [ID
Express](/destinations/tradedesk#id-express) for details.
#### Custom mappings
You can also map additional custom data using The Trade Desk's ten sequentially numbered custom dynamic properties to provide additional conversion metadata (`t1`-`t10`).
### ID Express
ID Express is available to all Customer Studio and Match Booster customers. If
you do not have access to this feature, contact your rep to inquire about
eligibility.
ID Express is a one-click solution that enables Hightouch customers to convert hashed emails and phone numbers into Unified ID 2.0 (UID2) identifiers for privacy-conscious audience targeting and conversion tracking through The Trade Desk.
#### Using ID Express
To use ID Express, map the appropriate email or phone number fields to the ID Express mappings in the configuration interface:
- `Hashed email → UID2`
- `Hashed phone → UID2`
Hightouch uses hashed identifiers when translating to UID2. If your email or phone number attributes are not hashed already, Hightouch automatically detects unhashed data and hashes them for you.
You can choose to map one or both fields to maximize match rates. You can also map other non-PII identifiers alongside these fields for additional reach, including Mobile Ad IDs (MAIDs) and The Trade Desk IDs (TTIDs).
#### How ID Express works
At sync time, Hightouch translates the hashed email and phone number values that you provide into UID2s before sending them to The Trade Desk. Hightouch then shares only the UID2 identifiers downstream with The Trade Desk, excluding the email and phone data.
If non-PII identifiers like MAIDs and TDIDs are also mapped, Hightouch recombines those identifiers with the UID2s and sends all of them to The Trade Desk to maximize match rates.
The Trade Desk then processes the UID2s and other IDs and makes them available as audiences and conversion events. Audiences can then be used to target and suppress devices across the open internet, and conversion events can be used to measure campaign results and optimize ad performance by providing better data to Kokai’s AI systems.
### Syncing offline conversion events
Sync data from any source to The Trade Desk as offline conversion events.
#### Offline conversion event types
Select which offline conversion event type you'd like to send to The Trade Desk:
- First party: these events are used in retargeting, campaign optimization, and measurement.
- Third party: send events to an advertising partner’s seat in The Trade Desk. You can set your conversion feed, CPM rate, and advertiser access for self-service monetization as well.
- Merchant: these events are used for retail use-cases, including SKU-level reporting for retailers and monetization by sharing conversion-based audience and measurement feeds
For **merchant** events, `Order ID` is a required field for each event. You also have the ability to include **line items** with each event.
For **third party** events, provide the advertiser ID you'd like to share these events with in the sync configuration. You can also set the CPM rate and media cost for these events in the sync configuration.
#### Offline tracking tags
An [offline tracking tag](https://partner.thetradedesk.com/v3/portal/data/doc/DataOfflineMeasurement#step1) is required for each offline conversion event in The Trade Desk. This offline tracking tag must be associated with the same offline data provider ID that is in the destination configuration.
Select whether you'd like to use the same tracking tag for all rows in your sync, or map the tracking tag ID from a column in your source.
You can optionally create a new offline tracking tag with the former option. Provide the name for your new offline tracking tag, and ensure that you have provided an API token in your destination configuration.
If you choose to create a new tracking tag or data rate through Hightouch, the
first run of the sync will **fail**. This is intentional. The Trade Desk will
not attribute your data to your segment until you assign a campaign to the
created offline tracking tag. After the first failed sync run, navigate to The
Trade Desk UI to assign a campaign to the created tracking tag before
attempting to run the sync again. The data in your model will be synced to the
segment on the next sync run. The Trade Desk processes new data rates nightly.
For best performance, we recommend waiting until the data rate processes
before running the sync again.
#### Field mapping
At least one ID and the timestamp is required for each offline event. If you provide an `ImpressionId`, no other IDs are required.
Hightouch supports the following ID types for offline conversion events:
| ID Type | Data Type | Description |
| ----------------------- | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **TDID** | GUID | The Trade Desk 36-character GUID (including dashes) for this user. |
| **DAID** | GUID | The raw device ID for this user, sent in 36-character GUID format (including dashes). Use iOS IDFA or Android’s AAID. |
| **UID2** | String | The raw UID2 value, also known as UID2. This value is case-sensitive. Raw UID2s are generated and managed using UID2 APIs. |
| **UID2Token** | String | The encrypted UID2 token, also known as an advertising token. This token is case-sensitive. |
| **EUID** | String | The raw European Unified ID value (EUID), which offers user transparency and privacy controls designed for European regions. |
| **EUIDToken** | String | The encrypted EUID token, also known as an advertising token. |
| **RampID** | String | The 49-character or 70-character RampID (previously known as IdentityLink). Must be a RampID from LiveRamp mapped specifically for The Trade Desk. |
| **Hashed email → UID2** | String | A SHA-256–hashed email address that Hightouch automatically converts to a UID2 identifier during sync execution (via [ID Express](/destinations/tradedesk#id-express)). |
| **Hashed phone → UID2** | String | A SHA-256–hashed phone number that Hightouch automatically converts to a UID2 identifier during sync execution (via [ID Express)](/destinations/tradedesk#id-express). |
Hightouch provides the ability to convert hashed emails and phone identifiers
to UID2 identifiers during sync execution. UID2 values will be sent as the
`adid` value on your event. See [ID
Express](/destinations/tradedesk#id-express) for details.
#### Custom mappings
You can also map additional custom data using The Trade Desk's ten sequentially numbered custom dynamic properties to provide additional conversion metadata (`t1`-`t10`).
### Syncing CRM data segments
Hightouch supports inserting and removing your data to your CRM data segments in The Trade Desk. CRM data is first-party data that includes all client information collected and shared across your organization.
#### User identifiers
To identify which users to add or update in an CRM data segment, select model columns and the corresponding Trade Desk fields. You can match on any of the following Trade Desk fields:
- Email
- Email (SHA256)
- Phone
- Phone (SHA256)
#### Creating a new segment
In your sync configuration, you can tell Hightouch to create a new data segment for you. You can optionally assign a custom name to your segment. If no name is provided, Hightouch defaults to using your model name.
You also need to select which region you would like to create your CRM data segment in. This is based on the origin of the PII.
#### Using an existing segment
Additionally, instead of creating a new data segment Hightouch can also sync users to existing segments that are already populated in your The Trade Desk account.
#### PII hashing
You can enable PII hashing if you wish for Hightouch to detect whether your data requires normalization and hashing based on the chosen record mapping. For example, if you select `Email (SHA256)` and the model column selected with the email data is unhashed, Hightouch hashes the value for you.
#### TTL
Data synced into The Trade Desk has a Time-To-Live (TTL) value. This is the amount of time the data remains active in the segment. The default recommended value by The Trade Desk is 90 days. If you wish to customize this interval, you can choose a column or enter the number of days (between 0 and 180) you want your data to live in the segment for.
Setting this value to 0 will mark synced rows for deletion. If all rows in
your segment have a TTL of 0, your segment will be deleted, and subsequent
sync runs may error.
#### Full Resync interval
This is the interval Hightouch will use to schedule a full resync of your data to The Trade Desk. The Trade Desk recommends a full data refresh every 30 days, however you can choose to retain hashed data for up to 90 days or process it once.
### Syncing CRM data conversions
Hightouch supports syncing up-to-date query results as conversions to CRM Data segments in The Trade Desk. This sync type is designed specifically for conversion tracking and differs from CRM data segments by requiring a timestamp for each conversion event and only supporting insert operations.
#### User identifiers
To identify which users to send conversion events for, select model columns and the corresponding Trade Desk fields. You can match on any of the following Trade Desk fields:
- Email
- Email (SHA256)
- Phone
- Phone (SHA256)
#### Creating a new conversion segment
In your sync configuration, you can tell Hightouch to create a new conversion segment for you. You can optionally assign a custom name to your segment. If no name is provided, Hightouch defaults to using your model name.
You also need to select which region you would like to create your CRM data segment in. This is based on the origin of the PII.
If you choose to create a new CRM conversions segment through Hightouch, the
first run of the sync will **fail**. This is intentional. The Trade Desk will
not attribute your data to your segment until you assign a campaign to the
created tracking tag. After the first failed sync run, navigate to The Trade
Desk UI to assign a campaign to the created tracking tag before attempting to
run the sync again. The data in your model will be synced to the segment on
the next sync run.
#### Using an existing conversion segment
Instead of creating a new conversion segment, Hightouch can also sync conversion events to existing conversion segments already populated in your The Trade Desk account.
#### PII hashing
You can enable PII hashing if you wish for Hightouch to detect whether your data requires normalization and hashing based on the chosen record mapping. For example, if you select `Email (SHA256)` and the model column selected with the email data is unhashed, Hightouch hashes the value for you.
#### Field mappings
CRM data conversions require a timestamp for each conversion event. You can also include additional conversion parameters:
| Field | Data Type | Description |
| ------------------ | --------- | ------------------------------------------------------------ |
| **Timestamp** | Timestamp | Required. The timestamp of the conversion event. |
| **Value** | Number | The monetary value of the conversion. |
| **Currency** | String | The currency code for the conversion value (e.g., USD, EUR). |
| **Country** | String | The country where the conversion occurred. |
| **Region** | String | The region/state where the conversion occurred. |
| **Metro** | Number | The metro code where the conversion occurred. |
| **City** | String | The city where the conversion occurred. |
| **Order ID** | String | A unique identifier for the order/conversion. |
| **Privacy type** | Enum | The privacy regulation type: GDPR or GPP. |
| **Is applicable** | Boolean | Whether the privacy settings apply to this conversion. |
| **Consent string** | String | The consent string for privacy compliance. |
#### Custom mappings
You can also map additional custom data using The Trade Desk's ten sequentially numbered custom dynamic properties to provide additional conversion metadata (`TD1`-`TD10`).
### First and third party data segments
Hightouch supports inserting your data to first and third party data segments in The Trade Desk.
#### Field mappings
You can sync columns from your model to The Trade Desk on the following fields:
- Trade Desk GUID: The Trade Desk 36-character GUID (including dashes) for a user.
- Raw Device ID: the raw device ID for this user, sent in 36-character GUID format, including dashes. Use iOS IDFA or Android's AAID.
- UID2: The 256-bit UID2 as a base64-encoded string.
- UID2Token: The encrypted UID2 token.
- EUID: The user's European Unified ID as a 44-character base64-encoded SHA-256 string. This ID is case-sensitive.
- EUIDToken String The encrypted EUID token.
- RampID: The 49-character or 70-character RampID, previously known as IdentityLink. This must be a RampID from LiveRamp that is mapped specifically for The Trade Desk. For details about mapping a RampID, see [LiveRamp documentation](https://sidecar.readme.io/docs/getting-started).
- ID5: Data providers holding these IDs should engage with ID5 directly to obtain the necessary decryption keys. For details see the [ID5 site](https://id5.io/). **NOTE: The ID5 ID must be in decrypted form.**
- netID: A 70-character base64-encoded string. For details see the [netID Developer Portal](https://developerzone.netid.dev/latest/devportal/).
- FirstId: A first-party cookie typically set by publishers in France. For details see the [First-id site](https://www.first-id.fr/).
- Hashed email → UID2: A SHA-256–hashed email address that Hightouch automatically converts into a Unified ID 2.0 (UID2) identifier during sync execution. If your email attribute is unhashed, Hightouch hashes it before translation.
- Hashed phone → UID2: A SHA-256–hashed phone number that Hightouch automatically converts into a Unified ID 2.0 (UID2) identifier during sync execution. If your phone attribute is unhashed, Hightouch hashes it before translation.
For more information on how Hightouch converts hashed identifiers to UID2, see
[ID Express](/destinations/tradedesk#id-express).
#### Creating a new segment
In your sync configuration, you can tell Hightouch to create a new data segment for you. You can optionally assign a custom name to your segment. If no name is provided, Hightouch defaults to using your model name.
#### Using an existing segment
Instead of creating a new data segment Hightouch can also sync users to existing segments already populated in your The Trade Desk account.
#### Full resync interval
Data synced into The Trade Desk has a Time-To-Live (TTL) value. This is the amount of time the data remains active in the segment. The default value recommended value by The Trade Desk is 30 days. If you wish to customize this interval, enter the number of days you want your data to live in the segment for. After it expires, Hightouch triggers a full resync on your next sync run to keep your data refreshed in The Trade Desk.
#### Data taxonomy
When creating a new third-party segment, you can create a corresponding element in your third-party data taxonomy in The Trade Desk. See [The Trade Desk's data taxonomy design documentation](https://partner.thetradedesk.com/v3/portal/data/doc/DataTaxonomyDesign) for more details.
To create a new data element, you need to update your **destination** configuration with a long-lived API token.
Provide the following details when creating a new data element:
- Provider Element ID: the identifier for your segment. Hightouch defaults to the **sync ID**.
- Parent Element ID: the identifier of the folder where this segment should live. Defaults to Root.
- Display name
- Description
#### Configure third-party data access
You can make this segment available to partners and advertisers by setting a data rate for it. See [The Trade Desk's data rates documentation](https://partner.thetradedesk.com/v3/portal/data/doc/DataRatesOverview) for more on third-party audience availability and data rate levels. In your sync configuration, provide:
- **Advertiser ID**: the advertiser you want to share this segment with.
- **CPM Rate**: the rate, in USD, you want to charge for this segment.
- **Percent of Media Cost Rate**: the percentage of media cost to use alongside the CPM rate cap, entered as a decimal (for example, `0.12` for 12%).
Contact your rep at The Trade Desk if you want to use this feature for your segment.
Hightouch confirms that the data segment and data rate batch are approved, but doesn't wait for processing to complete. Check the status of your segment and data rate batch in The Trade Desk UI to confirm completion.
### Syncing event opt-out or deletion
Hightouch supports opt-out or full deletion of customer data from The Trade Desk. This sync type allows you to manage customer privacy requests by either opting out customer IDs from being used in campaigns or completely deleting their data from your seat in The Trade Desk.
#### Data types
Select which type of data you'd like to send opt-out or deletion requests for:
- **First-party data (Advertiser)**: Opt-out or delete customer data from first-party advertiser campaigns
- **Third-party data**: Opt-out or delete customer data from third-party data segments
- **Merchant data**: Opt-out or delete customer data from merchant campaigns
#### Required configuration
Depending on the data type you select, you'll need to provide additional configuration:
- **Third-party data**: `Brand ID` is required
- **Merchant data**: `Merchant ID` is required
- **First-party data**: Uses the `Advertiser ID` from your destination configuration
#### User identifiers
Configure how the columns in your query results should be mapped to ID fields for opt-out or deletion requests. You can map to the following Trade Desk ID fields:
- `Trade Desk GUID`
- `Raw Device ID`
- `UID2`
- `UID2Token`
- `EUID`
- `EUIDToken`
- `RampID`
- `ID5`
- `netID`
- `FirstId`
#### Opt-out vs. full deletion
By default, this sync type performs an opt-out operation, which prevents the customer's data from being used in campaigns while retaining the data in your datasets.
If you enable the **full delete** option, Hightouch will delete the customer's ID from your datasets entirely, including:
- Data segments
- REDS feeds
- All other associated data in The Trade Desk
Full deletion is irreversible. Once customer data is deleted, it cannot be
recovered. Use this option carefully when processing deletion requests to
comply with privacy regulations like GDPR or CCPA.
#### Sync mode
This sync type only supports insert mode. Each sync run will process the rows in your model as new opt-out or deletion requests.
## Tips and troubleshooting
### Matched users count
Below only applies to the CRM data segment sync type.
Hightouch retrieves the audience metadata from The Trade Desk in real-time. The matched user
count reflects the active IDs based on a **7-day** look back window where the ID came up for bidding.
### Common errors
### Live debugger
### Sync alerts
---
## Twilio
**URL:** https://hightouch.com/docs/destinations/twilio
**Description:** Send SMS updates to your customers in real time.
**Section:** Destinations
## Overview
With the Twilio SMS destination, Hightouch can serve as a notifier for various changes. This is a very flexible integration, but here are a few example use cases:
1. Send 2FA codes
2. Send usage notifications and alerts
3. Sending password reset links
## Sending messages \(added / changed / removed\)
Hightouch will not send any SMS messages on the initial run to prevent a flood
of messages to your channel. Messages will start after this initial sync.
Hightouch supports sending messages based on rows that have been added, changed, or removed since the last sync run. Each row is sent as an individual SMS message and templated with Liquid.
### Recipient and Sender selection
Hightouch supports both sending messages to specific static recipients, or a phone number in each row. Use the mapper to map rows or static numbers to the `To` and `From` fields.
#### Messaging Service SID
You can also enter or select a Twilio Messaging Service SID in the mapping field.
The Messaging Service is setup inside [Twilio](https://www.twilio.com/docs/messaging/services#send-a-message-with-a-messaging-service). It allows you to send messages from a phone number pool which is useful when you need larger message quotas.
### Phone number cleaning
Hightouch can attempt to clean phone numbers in your data to ensure they are formatted correctly in [E.164](http://en.wikipedia.org/wiki/E.164) format.
When cleaning phone numbers, you will need to provide a 2 letter [ISO code](https://en.wikipedia.org/wiki/ISO_3166-2) for the country the numbers belong to.
### Message content
#### Added, changed, and removed phone numbers
For each sms, Hightouch allows you template the content you would like to send in your message (required). The row content is repeated per row in the message and has access to the row values via templating.
- When a row is added in your query results, Hightouch will send the row added content.
- When a row is changed in your query results, Hightouch will send the row changed content.
- When a row is deleted in your query results, Hightouch will send the row deleted content.
This content can be [templated](#liquid-templating).
## Templating messages
### Liquid templating
For both the message modes, [Liquid templating ](https://shopify.github.io/liquid/)is supported. The most important implication of this is that columns from the added or removed row can be accessed as part of the message body. Here is an example message making use of Liquid to insert a column into the message.
```text
Welcome to the platform {{ row["name"] }}, here is your verification code {{ row["2fa_code"] }}.
```
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Unify
**URL:** https://hightouch.com/docs/destinations/unify
**Description:** Capture intent data, run AI agents, and engage every prospect with personalized outbound using Unify.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
------------|--------------------------------------------|----------------------|------------------------------------------------------------------------------
**Records** | Sync data from any source to Unify records | Upsert, Insert | [Records docs](https://unify-19-preview.mintlify.app/developers/introduction)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Unify
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Unify** and click **Continue**. You can then authenticate Hightouch to **Unify**.
Enter the following fields into Hightouch:
- **API key**: Generate an API key in Unify by navigating to Settings -> API Keys.
## Sync configuration
Once you've set up your Unify destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Unify destination you want to sync to.
### Syncing records
Sync data from any source to Unify records. Select an object type to sync those types of records.
#### Record matching
You can match rows from your model to object records in Unify on any column in your model to the object type's unique field in Unify. Ensure the data types of the model column and Unify field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Unify automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync object record attributes via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the object's record attributes.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|----------------------------------------------------
**Do nothing** | Keep the record in Unify with all its synced fields
**Delete** | Delete the synced records from Unify
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Upland Waterfall
**URL:** https://hightouch.com/docs/destinations/upland-waterfall
**Description:** Elevate your customer experience and drive higher levels of engagement using personalised, targeted SMS/MMS mobile messaging with Waterfall.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
| ---------- | -----------------------------------------------------| ---------------------- | ------------------------------------------------------------------------------------------------------- |
| **Subscribers** | Sync data from any source to an Upland Waterfall list as subscribers | Upsert | [Subscriber Lists](https://apidocs.waterfall.com/#introduction)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Upland Waterfall
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Upland Waterfall** and click **Continue**.
Enter your API key from your Upland Waterfall account.
If you do not have an Upland Waterfall API key, contact your Upland Waterfall account manager.
You can optionally provide a default list ID. This will be the default list to add subscribers to for all syncs created with this destination.
## Sync configuration
Once you've set up your Upland Waterfall destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Upland Waterfall destination you want to sync to.
### Sync mode
- **Upsert**: pushes new objects to Upland Waterfall and updates mapped fields that change in your source.
### List selection
Select which list in your Upland Waterfall account you'd like to upsert subscribers to. If you configured a default list ID in your destination, selecting a list is not required in the sync configuration.
### Field mapping
Hightouch will fetch all metadata fields from Upland Waterfall as mapping options. You can map any field from your source to a metadata field with the inline mapper.
### Record matching
Upland Waterfall requires all subscribers to have a `msisdn`. You must select a column in your model to map to this field.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## User Interviews
**URL:** https://hightouch.com/docs/destinations/user-interviews
**Description:** Gain deeper understanding of your custom participants with User Interviews
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-----------------|-----------------------------------------------------------|------------------------|--------------------------------------------------------------------------------------------------
**Participants** | Sync data from any source to User Interviews participants | Upsert, Update, Insert | [Participants endpoint](https://api-docs.userinterviews.com/reference/patch_api-participants-batches)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to User Interviews
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **User Interviews** and click **Continue**. You can then authenticate Hightouch to **User Interviews** by entering the following required fields into Hightouch:
- **API key**: The User Interviews API key to authenticate requests. See [obtaining an API key](https://api-docs.userinterviews.com/reference/authentication#obtaining-an-api-key) for information on how to get your API key.
- **Environment**: **Sandbox** or **Production**.
## Sync configuration
Once you've set up your User Interviews destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the User Interviews destination you want to sync to.
### Syncing participants
Update existing participants' fields and characeristics, insert new participants, or do both with upsert mode.
#### Record matching
To match rows from your model to participant in User Interviews, you need to select the model column that contains values that match the particpants' **email** field. Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, User Interviews automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync participant fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default participant fields and characteristics.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available for **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
Behavior | Description
---------------|-------------------------------------------------------------------------
**Do nothing** | Keep the participant in User Interviews with all its synced fields
**Clear** | Clear all the mapped fields, but keep the participant in User Interviews
**Delete** | Delete the synced participants from User Interviews
#### Split retries
User Interviews counts all participants in a batch as rejected if the request contains a single invalid participant. To pinpoint which participants are getting rejected with which errors and reduce the number of valid participants that get retried, you can enable [split retries](/syncs/retries#split-retries).
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Userflow
**URL:** https://hightouch.com/docs/destinations/userflow
**Description:** Sync your freshest data to Userflow to drastically improve your user onboarding
**Section:** Destinations
## Setup
Upon selecting Userflow as your destination you will be prompted to
input your Userflow API key.
You can locate this API key within your Userflow account by navigating to _Settings > API_:
## Syncing
Hightouch supports syncing the following Userflow objects:
- `Users`
- `Groups`
### Sync modes
- **Upsert**: pushes new objects to Userflow _and_ updates fields that change in your warehouse.
- **Update**: only updates particular fields on existing objects in Userflow. It does _not_ add new objects.
### Record matching
Records can be matched from your source to your Userflow workspace by the **user ID** or **group ID**, depending on the object you are syncing.
### Field mapping
You may also sync columns from your source to Userflow's object properties. You have the option
of mapping to attributes that already exist on the Userflow object, or to create new attributes for current
and future use.
_\*NOTE: Aside from `groups` and `memberships` on `User` objects, all other fields will be associated with the `Group` or `User` object's `attributes` field within Userflow._
### A note on "groups" and "memberships"
The Userflow API allows you to create/update both groups and memberships through creating/updating
a user by including the appropriate `groups`/`memberships` as fields on the user object in the
request payload.
However, please be aware that their inclusion is not required and at most only one of either `groups`
or `memberships` can be set on a user per request. If your intention is only to sync group information,
it may be best to sync `Group` objects instead.
For more information on `groups` and `memberships`, including how they are properly formatted on a `User` object,
please consult the [Userflow API documentation](https://userflow.com/docs/api#create-or-update-a-user):
- Formatting `groups` for a `User`:
- Formatting `memberships` for a `User`:
### Pruning memberships
Hightouch also supports the ability to prune memberships of a user upon sync through the `prune_memberships`
property in the request payload. As per the Userflow API documentation:
> _"By default, the API will only update the memberships/groups that's included in the request. Existing memberships that aren't included will not be removed, unless you set `prune_memberships` to `true`. Only set `prune_memberships` to `true`, if the groups or memberships list is set and contains *all* the groups the user belongs to. When a membership is deleted, the group itself is left intact."_
When either `groups` or `memberships` are chosen to be synced to the destination, Hightouch will give you
the option to toggle membership pruning on or off.
_\*NOTE: By default, membership pruning is disabled._
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Vero Cloud
**URL:** https://hightouch.com/docs/destinations/vero
**Description:** Enable your sales team to notify users based on events in your warehouse with Vero Cloud.
**Section:** Destinations
## Setup
Grab your Vero Auth Token under [Settings > Project Settings > API Credentials](https://app.getvero.com/settings/project).
## Syncing
Hightouch supports syncing to the following Vero objects:
- Users
- Events
### Sync modes
Hightouch supports Update and Insert sync modes.
During Upsert mode, new objects will be inserted into Vero and all attributes will be kept up-to-date within Vero.
During Insert mode, existing objects will be ignored but new objects will be inserted into Vero.
### Upserting users
it's required to send a User ID to Vero to identify the user.
You may send any number of additional fields with the user.
### Inserting events
it's required to send a User ID and email to Vero to identify the user for the event.
You must also configure an Event Name to be used for the Sync.
You may send any number of additional fields with the event.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Viant
**URL:** https://hightouch.com/docs/destinations/viant
**Description:** Viant is a pioneer in people-based digital advertising. Use AI to help advertisers plan, execute, and measure their digital advertising campaigns.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
--------------|----------------------------------------------|----------------------|-----------------------------------------------------------------------------
**Audiences** | Sync data from any source to Viant audiences | Add | [Audiences docs](https://docs.api.viantinc.com/reference/how-to-get-started)
**Conversion events** | Send online or offline conversion event data to Viant | Insert | [Conversions docs](https://docs.api.viantinc.com/reference/onboard-onlineoffline-conversion-data)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Viant
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Viant** and click **Continue**. You can then authenticate Hightouch to **Viant**.
For the **Authentication method**, select **Log in to Viant** and log into your Viant account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Viant destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Viant destination you want to sync to.
### Syncing audiences
Sync data from any source to Viant audiences.
#### Record matching
You can match rows from your model to audiences in Viant on any column in your model and any field in Viant. Ensure the data types of the model column and Viant field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Accepted ID formats
Each Viant Audience Segment has a specified `dataType` that gets set when creating a new segment for the first time. If using an exsiting segment, the label will show what `dataType` the segment has. Viant supports the following `dataTypes`:
- `Email (sha_256)`
- `Address (sha_256)`
- `IP address`
- `Cookie`
- `Mobile ID`
If using `Email (sha_256)` or `Address (sha_256)`, Hightouch can auto apply hashing if selected. To use `Address (sha_256)`, please refer to Viant's docs on [preparing your user data](https://docs.api.viantinc.com/reference/prepaing-your-user-data) for formatting and standardizing your physical addresses before hashing.
#### Field mapping
Hightouch lets you sync audience fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default audience fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Split retries
Viant counts all audiences in a batch as rejected if the request contains a single invalid audience. To pinpoint which audiences are getting rejected with which errors and reduce the number of valid audiences that get retried, you can enable [split retries](/syncs/retries#split-retries).
### Syncing conversion events
Send online and offline conversion event data to Viant's DSP (Adelphic) platform.
#### Account configuration
| Field | Description |
|-------|-------------|
| **Account ID** (required) | Your Adelphic account ID |
| **Advertiser IDs** (optional) | List of advertiser IDs for conversion attribution |
#### User identifiers
At least one user identifier is required per conversion event. Viant supports the following identifier types:
- `Email (SHA256)` — `hashed_email`
- `Address (SHA256)` — `hashed_physical_address`
- `IP address`
- `Cookie`
- `Mobile ID` (IDFA/GAID)
If using `Email (SHA256)` or `Address (SHA256)`, Hightouch can auto-apply SHA256 hashing if selected. To use `Address (SHA256)`, refer to Viant's docs on [preparing your user data](https://docs.api.viantinc.com/reference/prepaing-your-user-data) for formatting and standardizing your physical addresses before hashing.
#### Field mapping
**Required fields:**
| Field | Description |
|-------|-------------|
| Timestamp | The timestamp of the conversion event |
| Transaction ID | Unique identifier for the transaction |
| Conversion event type | The type of conversion event (see supported types below) |
**Supported conversion event types:** `PageView`, `LandingPage`, `ItemView`, `AddToCart`, `InitiateCheckout`, `AddPaymentInfo`, `Purchase`, `Lead`
If you have any non-standard event you cannot map to the above list, you can pass a custom event name instead of above standard list as part of same conversion event type field.
**Optional fields:**
| Field | Description |
|-------|-------------|
| Amount | Monetary value of the conversion |
| Currency | Currency code (e.g., USD) |
| Store ID | Store or location identifier |
| Channel | `online` or `offline` |
| Purchased items | Array of item details (`itemId`, `productName`, `productCategory`, `price`, `quantity`) |
| Conversion location | Country, state, and/or city of the conversion |
#### Custom fields
You can include up to 10 custom key-value pairs for additional metadata. All custom field values are sent as strings.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Vistar Media
**URL:** https://hightouch.com/docs/destinations/vistar-media
**Description:** Empower your advertising team to run digital out-of-home (DOOH) campaigns with data-driven programmatic precision.
**Section:** Destinations
## Overview
Captivate your audience with digital out-of-home (DOOH) ads that are tailored to
the location, weather, and time of day based on the foot patterns of your
first-party audiences.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------- | ---------------------------------- | -------------------- |
| **Audiences** | Create and update custom audiences | All |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Vistar Media
This integration requires prior approval from the Vistar Media team and the
Hightouch team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your Vistar Media rep who will verify approval. Once we have confirmed,
we will unlock the integration for you to connect.
To connect, input the `Client Name` provided by your Vistar rep.
### Syncing custom audiences
You can input the ID of the audience you plan on delivering to Vistar along with
other properties.
#### User identifiers
To identify which users to add or update in an audience, select model columns
and the corresponding Vistar fields. You can match on any of the following user
fields:
- MAID
#### Delete behavior
The only supported sync mode is `all` mode, meaning that each sync run will refresh
the entire audience. Users removed from your audience will automatically be dropped
with each refresh - only present users will be in Vistar's audience.
## Tips and troubleshooting
### Refresh rate
Vistar Media has a max refresh rate of monthly. Though they recommend a one-and-done
sync run for each campaign. You should not have your Vistar syncs running more than
every 4 weeks (we recommend a manual schedule configuration).
### Matched device count
Match rates are not available to preview in the Hightouch platform. Contact your
Vistar rep for insights into match rates and audience reach.
Keep in mind that audience counts reflect the number of devices match, not
the number of users matched. Also keep in mind that matching occurs at the
household level, and that a single household may own multiple devices.
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Live debugger
### Sync alerts
---
## Vitally
**URL:** https://hightouch.com/docs/destinations/vitally
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API reference |
|-------------------|-----------------------------------------------------------------------------------------------------|------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|
| REST Objects | Sync records to objects such as Accounts, Organizations or Users in Vitally using the REST API | Upsert, Update, Insert | [REST Object endpoints](https://docs.vitally.io/pushing-data-to-vitally/rest-api) |
| Analytics Objects | Sync records to objects such as Accounts, Organizations or Users in Vitally using the analytics API | Upsert | [Analytics object endpoints](https://docs.vitally.io/pushing-data-to-vitally/using-the-vitally-api/https-analytics-api/analytics-api-batch) |
| Events | Sync records as events to Vitally. | Upsert | [Events docs](https://docs.vitally.io/pushing-data-to-vitally/integrations/data-warehouses/configuring-sync/syncing-events) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Vitally
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select Vitally and click **Continue**. You can then authenticate Hightouch to by entering the following fields into Hightouch:
- **Region**: Either **EU** or **US**.
- **Analytics API Token:** This is required for syncing analytics objects and events.
- **REST API Token:** This is required for the REST endpoints.
Follow these instructions to generate tokens.
1. Log into your Vitally account, click the wheel icon on the bottom sidebar.
2. Under **Data Management**, select the **Integrations** option.
3. For the Analytics API Token, select the **Vitally Analytics API** card. Otherwise, skip to step 6 for the REST API Token.
4. Toggle on the integration.
5. Click the **Integrate via HTTPS API** tab and copy the **API KEY**. This is the Analytics **API Token** you need to enter in Hightouch.
6. For the REST API Token, select the **Vitally REST API** card.
7. Toggle on the integration and click to generate the API Key.
8. Copy the **Basic Auth Header** once it's generated to use in Hightouch. This is the **REST API Token** you need to enter in Hightouch.
Once you've entered the relevant token or tokens, click **Continue** and enter a descriptive name for your destination to complete setup.
## Sync configuration
Once you've set up your Vitally destination, have a [model](/getting-started/concepts#models) to pull data from, and have a clear idea of the data you want to sync, you can set up your sync. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Vitally destination you want to sync to.
### Syncing REST objects
Hightouch supports syncing to account, organization, and user objects in Vitally.
#### Record matching
To match rows from your model to objects in Vitally, you need to select the model column that contains values that match the [**External ID** field](https://docs.vitally.io/pushing-data-to-vitally/the-significance-of-external-id) in Vitally. When syncing users, you can also match on **Email**.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync object fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to any of an object's default fields.
Ensure the data type of your model column matches the data type of field you want to sync to in Vitally.
For example, a Vitally `text` field expects a model column with the `string` [data type](/models/data-types-casting#data-types).
When using **Upsert** or **Update** mode to sync users, if you [match on **Email**](#record-matching),
you need to map both the **External ID** **and** either **Account IDs** or **Organization IDs**.
Otherwise, Hightouch displays a `Required field` error and a `Must specify at least one of: accountIds or organizationIDs` error.
If you match users on **External ID**, you only need to provide either **Account IDs** or **Organization IDs**.
Otherwise, Hightouch displays a `Must specify at least one of: accountIds or organizationIDs` error.
Vitally accounts, organizations, and users objects contain an editable [**traits** attribute](https://docs.vitally.io/account-health-scores-and-metrics/traits) to store custom key/value pairs.
This is useful for storing information like a customer's plan, number of licenses, name, etc.
To properly sync traits to Vitally, you need to format the traits field as an `object` data type rather than a `string`.
You can create an `object` of all the fields you want to update, and then map them to traits.
An example traits attribute for an organization object could look like this:
```
{
"organizationId": "Id_value",
"organizationName": "Name_value"
}
```
You can use [template mapping](/syncs/mapping-data#template-mapping) to create objects like this in your sync configuration.
In the following example screenshot, the template uses this [Liquid](https://shopify.github.io/liquid/basics/introduction/) function
```
{{ | json_construct : 'organizationName', row['organizationName'] }}
```
to create an object where the key's name is `organizationName` and the value is the row's `organizationName` column.
For more information, refer to the [Vitally's API documentation](https://docs.vitally.io/pushing-data-to-vitally/rest-api/organizations).
You can sync columns from your model to existing traits in Vitally and/or create new ones by adding them directly to the traits object in your Hightouch sync.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| --- | --- |
| Do nothing | Keep the object in Vitally with all its mapped properties |
| Delete | Delete the object in Vitally and all its mapped properties |
### Syncing analytics objects
Hightouch supports syncing to account, organization, and user objects in Vitally with the analytics API.
#### Record matching
To match rows from your model to objects in Vitally, you need to select a model column that contains the corresponding ID for each supported object.
- **Account ID** for account objects
- **Organization ID** for organization objects
- **User ID** for user objects
#### Field mapping
Hightouch lets you sync analytics object fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default object fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Traits mapping
Hightouch allows you to map traits for the analytics objects. Any mappings from this section will be synced as additional attributes in the traits parameter.
### Syncing events
Hightouch supports syncing product events in Vitally.
[Vitally's API](https://docs.vitally.io/pushing-data-to-vitally/integrations/data-warehouses/configuring-sync/syncing-events) requires the following event parameters:
- `event`: the name or type of event
- `timestamp`: the time the event was generated
- `message_id`: a unique ID for the event
And at least one of:
- `user_id`: the ID of the user that 'owns' the event
- `account_id`: the ID of the account that 'owns' the event
The sync configuration form ensures all these are set and provides some additional options.
#### Event name
Providing an event name is required to send an event to the Vitally's API. You can either provide a static value or select to **use a column** from your model. Hightouch syncs the static or column value as the `event` parameter the API requires.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server. Hightouch syncs the timestamp as the `timestamp` parameter the API requires.
#### Field mapping
Field mapping is where you select which event parameters you want to send to the Vitally's API.
The API requires you to map at least one of the following fields in order to save your sync.
- Account ID
- Organization ID
- User ID
You can also include data to sync as custom event fields in Vitally.
## Tips and troubleshooting
### Common errors
#### Must specify at least one of: accountIds, organizationIDs, user_id
You may receive an error like this in the Hightouch UI if you don't include all necessary values in your sync.
Consult the [field mapping](#field-mapping) section of your relevant sync type to learn more.
### Live debugger
### Sync alerts
---
## Vizio Ads
**URL:** https://hightouch.com/docs/destinations/vizio-ads
**Description:** Empower your advertising team to run targeted campaigns across Vizio Ad’s premium Smart TV inventory.
**Section:** Destinations
## Overview
Deliver more relevant ads across Vizio’s premium, brand-safe Smart TV ad inventory.
Reach addressable audiences by matching your first-party identifiers to Vizio IDs
using Match Booster.
## Supported syncing
| Sync type | Description | Supported sync modes |
| ------------------------------------------ | ---------------------------------- | -------------------- |
| [**Audiences**](#syncing-custom-audiences) | Create and update custom audiences | All |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Vizio Ads
This integration requires prior approval from the Vizio Ads team and the Hightouch
team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your Vizio rep who will verify approval. Once we have confirmed, we will
unlock the integration for you to connect.
To connect, input the following provided by your Vizio rep:
- Access key
- Secret key
- Customer directory
### Syncing custom audiences
You can input the ID of the audience you plan on delivering to Vizio Ads.
#### User identifiers
To identify which users to add or update in an audience, select model columns
and the corresponding Vizio Ads fields. You can match on any of the following user
fields:
- IP address
#### Delete behavior
The only supported sync mode is `all` mode, meaning that each sync run will refresh
the entire audience. Users removed from your audience will automatically be dropped
with each refresh - only present users will be in Vizio Ads's audience.
## Tips and troubleshooting
### Refresh rate
Vizio Ads has a max refresh rate of daily. You should not have your Vizio Ads syncs
running more than daily.
### Matched device count
Match rates are not available to preview in the Hightouch platform. Contact your
Vizio rep for insights into match rates and audience reach.
Keep in mind that audience counts reflect the number of Vizio devices match, not
the number of users matched. Also keep in mind that matching occurs at the
household level, and that a single household may own multiple devices.
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Live debugger
### Sync alerts
---
## Voucherify
**URL:** https://hightouch.com/docs/destinations/voucherify
**Description:** Voucherify helps businesses quickly launch and manage promotions, loyalty programs, referrals, and gift cards through a flexible API-first platform, reducing development effort while enabling personalized customer incentives.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|---------------|------------------------------------------------------------|----------------------|--------------------------------------------------------------------------------------|
| **Customers** | Sync data from any source to Contacts in Voucherify | Upsert | [Customers docs](https://docs.voucherify.io/api-reference/customers/customer-object) |
| **Events** | Sync data from any source to custom events in Voucherify | Insert | [Events docs](https://docs.voucherify.io/api-reference/events/track-custom-event) |
| **Segments** | Sync data from any source to active segments in Voucherify | Add, Remove | [Segments docs](https://docs.voucherify.io/api-reference/segments/create-segment) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Voucherify
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Voucherify** and click **Continue**. You can then authenticate Hightouch to **Voucherify**.
Enter the following fields into Hightouch:
- **Region**
- **Application ID**
- **App token**
## Sync configuration
Once you've set up your Voucherify destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Voucherify destination you want to sync to.
### Syncing customers
Sync data from any source to Contacts in Voucherify.
#### Record matching
To match rows from your model to customers in Voucherify, you need to select the model column that contains values that match the **Source ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync customer fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom customer fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
|----------------|------------------------------------------------------------|
| **Do nothing** | Keep the customer in Voucherify with all its synced fields |
| **Delete** | Delete the synced customers from Voucherify |
### Syncing events
Sync data from any source to custom events in Voucherify.
#### Record matching
You can match rows from your model to events in Voucherify on any column in your model and any field in Voucherify. Ensure the data types of the model column and Voucherify field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
### Syncing segments
Sync data from any source to active segments in Voucherify.
#### Record matching
To match rows from your model to segments in Voucherify, you need to select the model column that contains values that match the **Source ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync segment fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default segment fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## VWO Attribute Lists
**URL:** https://hightouch.com/docs/destinations/vwo
**Description:** Boost conversions with precise targeting and personalized experiences using VWO's powerful optimization tools.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| ------------------- | ------------------------------------------------ | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Attribute Lists** | Sync data from any source to VWO Attribute Lists | Insert, Update, Upsert | [Attributes](https://help.vwo.com/hc/en-us/articles/8681465703705-Working-With-Attributes-in-VWO) and [Attribute Lists](https://help.vwo.com/hc/en-us/articles/900003106343-Targeting-Visitors-Using-Attributes-List-in-VWO) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to VWO
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **VWO** and click **Continue**. You can then authenticate Hightouch to **VWO**.
Enter your **Authorization Token** into Hightouch.
If you don't already have a token, you can generate one [here](https://app.vwo.com/#/developers/tokens).
## Sync configuration
Once you've set up your VWO destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the VWO destination you want to sync to.
### Syncing Attribute Lists
Sync data from any source to VWO Attribute Lists.
Currently, we do not enable the creation of non-existent attribute lists, or the ability to browse attribute lists.
Syncing to VWO Attribute Lists requires a knowledge of the Attribute List ID, which you should be able to find in the VWO console.
#### Column selection
Configuring your VWO sync is as simple as selecting the column in your source which you are associating with the selected Attribute List.
When run, Hightouch will sync this column from your source to your Attribute List, diffing changes and keeping your Attribute List up to date.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## WebDAV
**URL:** https://hightouch.com/docs/destinations/webdav
**Description:** Sync data from your warehouse to a WebDAV server as CSV, JSON, NDJSON, XML, or Parquet files.
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
| ------------ | ------------------------------------------------------------------------------- | -------------------- |
| Any data set | Sync data from a source to a WebDAV server as CSV, JSON, NDJSON, XML, or Parquet files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Diff mode creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to WebDAV
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **WebDAV** and click **Continue**. You can then authenticate Hightouch to your WebDAV server by entering the following fields:
- **WebDAV Server URL**: The base URL of your WebDAV server, for example `https://dav.example.com/files/`. Hightouch uploads files relative to this URL.
- **Username**: The username Hightouch uses to authenticate.
- **Password**: The password for the user.
The user you configure must have write access to the target directory on your WebDAV server. A `403 Forbidden` response during a sync indicates insufficient permissions.
Hightouch uses HTTP Basic Authentication and sends an `OPTIONS` request to verify that the server is reachable and the credentials are valid when you test the connection.
Hightouch buffers each file to disk before uploading so it can set an accurate `Content-Length` header. Some WebDAV servers reject chunked transfer encoding, so this ensures compatibility.
## Sync configuration
Once you've set up your WebDAV destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the WebDAV destination you want to sync to.
### Select file format
Hightouch supports syncing CSV, JSON, NDJSON, XML, and Parquet files via WebDAV.
### Enter filename
The filename field lets you specify the path and name of the file relative to your WebDAV server URL.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would place the file at `https://dav.example.com/files/upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json`, so the `upload` directory must already exist on your server.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it.
If you are using an audience and would like to include the audience name, you will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter `-86400` (`24 hours * 60 minutes * 60 seconds`). If you want the filename's date to be one hour after the sync takes place, enter `3600` (`60 minutes * 60 seconds`).
### CSV options
If you're syncing to a CSV file, you have additional configuration options:
- **Delimiter**: Your options are comma (`,`), semicolon (`;`), pipe (`|`), tilde (`~`), and tab
- (Optional) Whether to **include a CSV header row** in the exported files
- (Optional) Whether to **include a [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark)** in the exported files, the BOM is ``
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
### Batch size
By default, and depending on your [sync mode](#supported-syncing), Hightouch sends one file for each export. (In diff mode, Hightouch sends three files.)
By enabling batching, you allow Hightouch to send multiple files if your model query results are large. This can help avoid file size errors.
### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, Hightouch won't upload any files if your model's query results haven't changed since your last sync.
## Tips and troubleshooting
The **Test connection** button on the destination configuration page sends an `OPTIONS` request to verify that the server is reachable and the credentials are valid. To fully test write permissions, create and run a one-row test sync to verify that data reaches your WebDAV server.
### Common errors
#### 401 Unauthorized
The username or password is incorrect, or the server requires a different authentication method. Verify your credentials and confirm that the server accepts HTTP Basic Authentication.
#### 403 Forbidden
The user doesn't have write permission for the target path. Grant write access to the configured user in your WebDAV server, or update the destination to use a path the user can write to.
#### 405 Method Not Allowed
The server doesn't support `PUT` requests at the target path. Confirm that your WebDAV server has write operations enabled and that the target path allows writes.
### Live debugger
### Sync alerts
---
## Webflow
**URL:** https://hightouch.com/docs/destinations/webflow
**Description:** Bring in the power of personalization and dynamic content to your web projects on Webflow using rich customer data from your data warehouse
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
| -------------- | --------------------------------------------------------- | -------------------- | ---------------------------------------------------------------------------- |
| **Collections**| Sync data from any source to Webflow CMS collection items | Upsert, Update |[Create Collection Item](https://developers.webflow.com/reference/create-item)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Webflow
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Webflow** and click **Continue**. You can then authenticate Hightouch to Webflow by entering your **API Token** from your Webflow project.
To generate an API token in Webflow:
1. Go to **Apps & Integrations** and scroll down to the **API Access** section.
2. Click **Generate API Token**.
3. Copy the generated token to your clipboard.
> For the most accurate instructions, see [Webflow’s API Token Documentation](https://developers.webflow.com/data/v2.0.0/reference/authentication).
## Sync configuration
Once you've set up your Webflow destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Webflow destination you want to sync to.
### Syncing collections
#### Record matching
To match source rows to Webflow collection items, you need to select a source column and corresponding Webflow field. You can match on any Webflow field that is set to [plain text](https://webflow.com/glossary/plain-text-field) type. Using the `Slug` field is recommended.
#### Field mapping
You can sync columns from your source to Webflow's collection properties.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| ----------------- | -------------------------------------------- |
| **Do nothing** | Keep the record in Webflow |
| **Clear fields** | Keep the record, but clear the mapped fields |
| **Delete record** | Delete the record in Webflow |
**Clear fields** is only available in Update mode, whereas **Delete record** is only available in Upsert mode.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## WhatsApp
**URL:** https://hightouch.com/docs/destinations/whatsapp
**Description:** Message your customers when and where it matters most.
**Section:** Destinations
## Overview
Deliver critical customer journey communications in real-time to where it matters: your customers' pockets. Combine data from multiple sources to send promotional messages, special offers or order updates to customers who have actively opted into your company's communications via WhatsApp, initiating the first of many customer communication experiences.
## Supported syncing
|Message Type|Description|Supported Sync Modes | API Reference |
|--|--|--|--|
|**Message Templates**|Send a business-initiated message using an approved message template|Insert| [Message template reference docs](https://developers.facebook.com/docs/whatsapp/cloud-api/reference/messages#template-messages) |
## Connect to WhatsApp
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **WhatsApp** and click **Continue**. You can then authenticate Hightouch by entering the following fields:
- Your WhatsApp phone number ID
- Facebook business manager ID
- System User token.
The System User's token must be generated with the following scopes:
- `whatsapp_business_management`
- `whatsapp_business_messaging`
Before finalizing the destination creation, you need to agree to [WhatsApp's opt-in policy](https://developers.facebook.com/docs/whatsapp/overview/getting-opt-in).
## Sync configuration
Once you've set up your WhatsApp destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the WhatsApp destination you want to sync to.
### Message templates
WhatsApp message templates are specific message formats that businesses use to send out notifications or customer care messages to people that have opted in to notifications. Business-initiated messages-the types of messages Hightouch enables your team to send-must use a template approved by WhatsApp. If you don't see the template you would like to use in the sync configuration form, check the template's status in your [WhatsApp Template Manager](https://business.facebook.com/wa/manage/message-templates).
### Phone numbers
A column from your model must contain the user phone numbers to send a message to.
### Field mapping
If the selected template contains variables, each variable must be mapped to create a complete message.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Workday
**URL:** https://hightouch.com/docs/destinations/workday
**Description:** Sync enriched data from your warehouse to your Workday instance
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ----------------- | -------------------------------------------------- | -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Organizations** | Sync data from any source to Workday organizations | Upsert, Update | [Organization docs](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Add_Update_Organization.html) |
| **Workers** | Sync data from any source to Workday workers | Update | [Home contact information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Home_Contact_Information.html), [work contact information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Work_Contact_Information.html), and [personal information docs](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Personal_Information.html) |
| **Positions** | Sync data from any source to Workday positions | Update | [Position docs](https://community.workday.com/sites/default/files/file-hosting/productionapi/Staffing/v40.1/Edit_Position.html) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Workday
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Workday** and click **Continue**. You can then authenticate Hightouch to **Workday** by entering the following required fields into Hightouch:
- **Hostname**
- **Tenant ID**
- **Username**
- **Password**
The **hostname** is the hostname of the Workday SOAP endpoints you want to use.
You only need to enter the hostname, without `https://` nor particular endpoint URL.
For example, if you want to use this SOAP endpoint: `https://wd2-impl-services1.workday.com/ccx/service/your_company/Human_Resources/v40.1`,
enter `wd2-impl-services1.workday.com`.
If you don't have a hostname, enter `wd2-impl-services1.workday.com`.
You can find your **tenant ID** from your URL when you are logged into Workday.
For example, if the URL is `https://impl.workday.com/your_company/d/home.html`, then your tenant ID is `your_company`.
Workday recommends that you use Integrated System User (ISU) to when connecting to Workday via third-party services like Hightouch.
See [Workday's docs](https://doc.workday.com/reader/Z9lz_01hqDMDg6NSf7wCBQ/esBDCh5D66sgBhIxmQ5U5g) for more information on setting up an ISU.
## Sync configuration
Once you've set up your Workday destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Workday destination you want to sync to.
### Syncing organizations
This sync type adds new organizations and/or updates existing organizations with the supplied information.
A new effective-dated organization name is automatically created if any attributes used for organization name are different as of specified effective date.
To use this sync type, you need to have **Put** access for the **Manage: Organization Update Integration** domain.
#### Record matching
To match rows from your model to organization in Workday, you need to select a model column and corresponding Workday field.
- In **upsert** mode, you can match on either **Organization Reference ID** or **Integration ID Data**.
- In **update** mode, you must match on **Integration ID Data**.
If you are matching with **Integration ID Data**, you must also provide the **System ID** in the attributes mapping.
#### Field mapping
Hightouch lets you sync organization fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default organization fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
### Syncing workers
This sync type updates existing workers in Workday.
#### Record matching
To match rows from your model to worker in Workday, you need to select a model column and corresponding Workday field. You can match on any of the following Workday fields:
- **Universal_ID_Reference**
- **Person_Reference**
When matching with **Universal_ID_Reference**, you must also specify its **Reference Type** int the attributes mapping. Valid values are:
- `WID`
- `Universal_Identifier_ID`
When matching on **Person_Reference**, you must also specify its **Reference Type** in the attributes mapping.
See the [RoleObject](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Home_Contact_Information.html#RoleObjectIDType)'s `@type` for the full list of valid values.
#### Field mapping
Hightouch lets you use [field mapping](/syncs/mapping-data) to update the following kinds of worker information:
- [home contact information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Home_Contact_Information.html#Person_Contact_Information_DataType)
- [work contact information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Work_Contact_Information.html#Person_Contact_Information_DataType)
- [personal information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Personal_Information.html#Change_Personal_Information_DataType)
Field names for [home contact information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Home_Contact_Information.html#Person_Contact_Information_DataType) begin with _Home_,
field names for [work contact information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Work_Contact_Information.html#Person_Contact_Information_DataType) begin with _Work_, and field names for [personal information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Personal_Information.html#Change_Personal_Information_DataType) begin with _Personal Information_.
You can map data from any of your model columns to these fields. Ensure your model columns data types match the data types of the fields you want to sync to.
### Syncing positions
This sync type updates existing positions in Workday.
#### Record matching
To match rows from your model to position in Workday, you need to select a model column and corresponding Workday field. You can match on any of the following Workday fields:
- `WID`
- `Position_ID`
#### Field mapping
Hightouch lets you sync position fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default position fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
## Tips and troubleshooting
### Common errors
#### The task submitted is not authorized
This error means your Workday user doesn't have enough permission to perform the API operation configured in the Hightouch sync.
You can resolve the error by granting the user you [authenticated with](#connect-to-workday) with the permission required for the endpoint.
For example, if you get this error when you're trying to make changes with the [Change_Home_Contact_Information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Home_Contact_Information.html) endpoint, you should ensure your user has permission to use this API.
### Live debugger
### Sync alerts
---
## Workday Adaptive Planning
**URL:** https://hightouch.com/docs/destinations/workday-adaptive-planning
**Description:** Sync data from your warehouse to your Workday Adaptive Planning modeled sheets.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ----------------- | ---------------------------------------- | -------------------- | -------------------------------------------------------------------------------------------------------------- |
| **Modeled sheets** | Import data to a selected modeled sheet | Insert, Mirror | [Modeled sheet docs](https://adaptiveplanning.doc.workday.com/r/DG7oXjCPB6kIw6Th6awNUg/~SS8TWR04JTiriCxM4rMBA) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Workday Adaptive Planning
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Workday Adaptive Planning** and click **Continue**. You can then authenticate Hightouch to **Workday Adaptive Planning** by entering the following required fields into Hightouch:
- Workday **Username**
- Workday **Password**
Once you've entered these fields, click **Continue**. Give your destination a descriptive name and **Finish** the destination connection setup.
## Sync configuration
Once you've set up your Workday Adaptive Planning destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Workday Adaptive Planning destination you want to sync to.
The Workday Adaptive Planning destination allows you to **Mirror** your data to a spreadsheet or insert new rows with **Insert** mode.
**Mirror** mode replaces the entire selected sheet's contents with the rows from your model.
Ensure that your sheet doesn't contain data that shouldn't be overwritten if you select this mode.
### Syncing modeled sheets
You can insert rows into or completely replace the selected modeled sheet with data from your model.
To begin, select the sheet you want to sync data to and its version.
#### Record matching
In **Insert** and **Mirror** mode, Workday Adaptive Planning automatically generates an
identifier for every new record synced, so there is no need to match an
existing record.
#### Field mapping
Hightouch lets you sync sheet fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to your modeled sheet fields.
Workday Adaptive Planning requires the following fields, so you must map them to complete your configuration:
- **Approval**
- **Asset Class**
- **Level**
- **Priority**
Ensure your model columns data types match the data types of the fields you want to sync to.
Workday Adaptive Planning uses numbers to represent boolean values. Use `0` for
`false` and `1` for `true`.
## Tips and troubleshooting
### Common errors
#### Data containing pipe symbols
Workday Adaptive Planning uses the pipe symbol (`|`) to separate the sheet's columns.
Make sure your data does not contain any pipe symbols since it will cause a mismatch
in the header and row data.
### Live debugger
### Sync alerts
---
## X Ads (formerly Twitter Ads)
**URL:** https://hightouch.com/docs/destinations/x
**Description:** Send conversion web events to X and run campaigns with custom audiences, retargeting, and lookalikes directly from your warehouse
**Section:** Destinations
## Overview
Build more robust campaigns on X (formerly Twitter) with up-to-date customer data from your data warehouse.
## Supported syncing
| Sync type | Description | Supported sync modes |
| --------------------------------------------------- | --------------------------------------------------- | -------------------- |
| [**Custom audiences**](#syncing-custom-audiences) | Create audience lists and keep them up-to-date in X | Add |
| [**Conversion events**](#syncing-conversion-events) | Sync web conversion events to X | Insert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to X
1. Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button.
2. Select **X** in the destination catalog and click **Continue**.
3. Proceed with the OAuth steps until you are redirected to the Hightouch website.
4. Give your destination a name and unique slug (for example, “X Production” and `x-production`).
5. Click **Finish** to create your X destination.
## Sync configuration
### Syncing custom audiences
This integration supports _Segment Mode_. New users in the model will be inserted into the X
Custom Audience. Users that are removed from the model will be removed from the
Custom Audience.
#### X account selection
In this section, select the X account you would like to sync your data to.
#### Audience selection
This integration supports syncing to an **existing audience** or **creating a new audience**.
To use an **existing segment**:
- Select `Use existing audience` from the radio buttons.
- Select the name of the audience from the dropdown menu.
To **create a new audience**:
- Select `Create a new audience` from the radio buttons.
- Specify a custom name for this audience in the input field. Otherwise Hightouch will use the name of the model as the audience's name.
#### User identifers
Hightouch allows you to sync columns from your source to the supported X fields.
X allows mapping to various user identifiers, such as Email, Device ID, X Handle, etc. X requires _all fields_ for users
to be both **normalized** and **hashed** using SHA256 (with the exception of Partner User ID).
You can achieve this in your model with a function
by using the `encode(digest(value, 'sha256'),'hex')` function in Postgres, or refer to the [Template Mappings](/destinations/X#template-mappings) Section in this documentation to learn about how Hightouch can handle hashing these records for you.
### Syncing conversion events
This integration supports syncing _Web Conversion Events_.
#### X account selection
In this section, select the X account you would like to sync your data to.
#### Pixel ID (Universal Website Tag) account section
If the **Pixel ID** exist in your model, toggle **Use Column**, and select the appropriate column from the dropdown menu.
To retrieve your Pixel ID:
- Navigate to your [Events Manager](https://business.x.com/en/help/campaign-measurement-and-analytics/conversion-tracking-for-websites) page.
- Copy the ID and paste it in the Hightouch sync form.
#### Event selection
Given the correct **Pixel ID**, Hightouch will filter for the eligible events that allow conversion events to be synced to.
Select the event you'd like to sync to from the dropdown menu. If the **ID** for these events exist in your model, toggle **Use Column**, and select the appropriate column from the dropdown menu.
#### Identifier section
When syncing conversion events, X expects one identifier at minimum, or you can provide both.
Select your choice for the sync from the radio buttons. This will render the fields in the mappings section, which you can map to from your model.
#### Field mapping
Hightouch allows you to sync columns from your source to the supported X fields.
If you wish to use the **Email Address** as an identifier, X expects it to be hashed using SHA256.
You can achieve this in your model with a function by using the encode(digest(value, 'sha256'),'hex') function in Postgres, or refer to the [Template Mappings](/destinations/X#template-mappings) section in this documentation to learn about how Hightouch can handle hashing these records for you.
### Template mappings
You can leverage Hightouch's Template Mappings feature to allow Hightouch to handle hashing certain fields for you automatically.
To use this feature:
- Select **Template** in the mappings window.
- Select the column you would like to hash from the **Variables** section.
- Select **sha256** from the **Functions** section.
- Click **Apply** to save.
To find out more about Template Mapping and other Mapping features, visit our [Mapping Data](/syncs/mapping-data) section.
## Tips and troubleshooting
Below only applies to the audiences sync type.
### Common errors
#### 400: INVALID_PARAMETER
The full error message looks something like this:
```json
{
"request": [
{
"params": {
"account_id": "01ab234cdef",
"custom_audience_id": "0a1b2"
},
"operation_type": "Update"
}
],
"operation_errors": [
[
{
"code": "INVALID_PARAMETER",
"message": "Tailored audience attribute value can not be empty.",
"parameter": ""
}
]
]
}
```
This error occurs if any of your [field mappings](#field-mapping) include `null` values.
X doesn't accepted hashed `null` values.
To resolve this, [remove rows containing `null` values](/models/sql-editor#exclude-rows-with-null-values) from your model's query results or enable the [**Don't sync null values** option](/syncs/mapping-data#dont-sync-null-values) in the [advanced mapper](/syncs/mapping-data#advanced-mapper).
### Live debugger
### Sync alerts
---
## Xandr
**URL:** https://hightouch.com/docs/destinations/xandr
**Description:** Enable your marketing team to target granular audiences and measure real-world outcomes
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
|--------------|---------------------------------------------|----------------------|-----------------------------------------------------------------------------------------------------|
| [**Segments**](#syncing-segments) | Sync data from any source to Xandr segments | Add, Remove | [Segments docs](https://docs.xandr.com/bundle/xandr-api/page/uploading-segment-data-using-bss.html) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Xandr
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Xandr** and click **Continue**. You can then authenticate Hightouch to **Xandr**.
To get started with the Xandr API, you need to request access by contacting the [Xandr support team](https://help.xandr.com/s/login/). For more information, read the [Xandr API docs](https://docs.xandr.com/bundle/xandr-api/page/api-onboarding-process.html).
Before configuring your Xandr destination, you will need to follow [these steps](https://docs.xandr.com/bundle/xandr-api/page/token-based-api-authentication.html) to generate private and public keys. After generating your keys, register your public key with your Xandr account.
Enter the following required fields into Hightouch:
- **Username**: Enter your username.
- **Member ID**: Enter your member ID.
- **Public Key Name**: Enter the name of the public key.
- **Private Key**: Enter your private key
## Sync configuration
Once you've set up your Xandr destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Xandr destination you want to sync to.
### Syncing segments
Sync data from any source to Xandr segments.
#### Record matching
To match rows from your model to segment in Xandr, you need to select a model column and corresponding Xandr field. You can match on any of the following Xandr fields:
- **Xandr ID**
- **Extended ID**
- **External ID**
- **Device ID**
- **IFA**
#### Field mapping
Hightouch lets you sync segment fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default segment fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Xero
**URL:** https://hightouch.com/docs/destinations/xero
**Description:** Gain financial visibility and empower your accounting team by keeping your financials up to date
**Section:** Destinations
## Overview
Give yourself confidence in the financial health of your business by integrating your Xero usage with Hightouch.
We automate your financial workflows and eliminate inconsistent data issues and manual data entry.
With this integration, you can deliver up-to-date data directly from your source of truth to allow your team the luxury to focus
on making financial decisions that benefit your business instead of wasting time finding, cleaning, and moving relevant data.
## Getting started
### Connect to Xero
**Authorize Xero with OAuth and create your destination.**
#### Add Xero as a destination in Hightouch
1. Log in to Hightouch or create a free Hightouch account
2. Click on **Destinations** in the left sidebar
3. Click on **Add destination** in the top right corner
4. Select **Xero** in the destination catalog and click **Continue** to proceed
5. Complete the authorization steps until you are redirected back to the Hightouch application
6. Give your destination a name and unique slug (for example, `Xero Production` and `Xero-production`)
7. Click **Finish** to create your Xero destination.
### Connect to your data source
Hightouch sits on top of your data warehouse and can read from Snowflake, Redshift, BigQuery, Databricks, Postgres, MySQL, and many other data sources. You can even fetch data from Google Sheets and Airtable. Hightouch can help activate your data wherever it resides.
We've written dedicated guides for each supported data source. Take your pick:
- Connect to [Airtable](/sources/airtable)
- Connect to [Amazon Athena](/sources/amazon-athena)
- Connect to [Amazon Redshift](/sources/amazon-redshift)
- Connect to [Amazon S3](/sources/s3)
- Connect to [ClickHouse](/sources/clickhouse)
- Connect to [Databricks](/sources/databricks)
- Connect to [Elasticsearch](/sources/elasticsearch)
- Connect to [Firebolt](/sources/firebolt)
- Connect to [Google BigQuery](/sources/google-bigquery)
- Connect to [Google Sheets](/sources/google-sheets)
- Connect to [Metabase](/sources/metabase)
- Connect to [MongoDB](/sources/mongodb)
- Connect to [MySQL](/sources/mysql)
- Connect to [Palantir Foundry](/sources/foundry)
- Connect to [PostgreSQL](/sources/postgresql)
- Connect to [Rockset](/sources/rockset)
- Connect to [SFTP](/sources/sftp)
- Connect to [Snowflake](/sources/snowflake)
- Connect to [SQL Server](/sources/sqlserver)
- Connect to [Trino](/sources/trino)
If you're missing the credentials necessary to connect to your data source,
try inviting your teammates to Hightouch. You can have unlimited collaborators
in your workspace.
#### Connect to a sample database instead
Waiting for credentials to your data warehouse? No problem. You can connect to one of our [sample datasets](/sources/sample-data) to get started right away. For the rest of this tutorial, we'll use the **B2B SaaS** sample source, which contains a `users` table. This sample source is available when you log in for the first time—no setup required.
Next, we'll write a SQL query to define which users should be synced to our Xero accounts resource. You can follow the same pattern to sync to other Xero resources.
### Create a model
In Hightouch, a model represents a query that filters or transforms the records in your data source. Models are used to determine exactly which records to sync between sources and destinations.
When you connect to a data warehouse, your models will probably be written in SQL. You can also import models from tools like dbt and Looker.
In this example, we'll keep it simple. Suppose you want to sync all your customers to Xero. To accomplish this, you'll want to create a model that looks like this:
```sql
SELECT * FROM users;
```
1. Click on **Models** in the left sidebar.
2. Click on **Add model** in the top right corner.
3. Select **Demo Database** as your source and click **Continue** to proceed.
4. Select **SQL Editor** as your modeling method.
5. Paste the SQL query from above into the editor on the left side of the page.
6. Click **Preview** to execute the query.
7. Click **Continue** to proceed.
8. Give your model a name and unique slug (for example, "All Users" and `all-users`).
9. Select **ID** as the primary key for your model.
10. Click **Finish** to create your model.
We just created a model using a SQL query that retrieves all our current users. Next, we'll configure our sync to add these users to Xero.
### Create a sync
1. Click on **Syncs** in the left sidebar.
2. Click on **Add sync** in the top right corner.
3. Select **All Users** as your model and click **Continue** to proceed.
4. Select **Xero** as your destination and click **Continue** to proceed.
## Sync configuration
### Object selection
Hightouch supports syncing to the following Xero resources:
- **Accounts**.
- **Contacts**.
- **Payments**.
- **Invoices**.
- **Items**.
### Accounts
#### Account type selection
- Select the type of accounts you would like to sync to your destination. To sync dynamic types, toggle 'use column' and select a column from your model that contains supported enum values by Xero.
Selecting an account type from the drop-down menu allows Hightouch to provide
you with the appropriate validations for each account type. These validations
will not exist if you choose to sync dynamic types from your model and will
require you to be familiar with what is required by Xero.
#### Sync modes
For **Bank** account type, our integration supports:
- **Insert** mode.
For all other account types, our integration supports:
- **Upsert**: Records that aren't found in Xero given the provided identifier will be inserted. Existing records will be updated.
- **Update**: Records that are found in Xero given the provided identifier will be updated.
- **Insert**: Records will be pushed to your destination.
#### Record matching
Records can be matched from your source to your Xero workspace by:
- **Account ID**
- **Name**
#### Field mapping
Hightouch allows you to sync columns from your source to the supported Xero fields.
#### Delete records
When the Hightouch sync engine recognizes that a record has been removed from your source, you can either:
In **Upsert** mode:
- Do nothing. An existing record in the destination would remain untouched.
- Delete the record in the destination. Only non-system accounts and accounts not used on transactions can be deleted.
### Contacts
#### Sync modes
This integration supports:
- **Upsert** mode: Records that aren't found in Xero given the provided identifier will be inserted. Existing records will be updated.
#### Record matching
Records can be matched from your source to your Xero workspace by:
- **Name**.
#### Field mapping
Hightouch allows you to sync columns from your source to the supported Xero fields.
### Payments
#### Sync modes
This integration supports:
- **Insert** mode: Records will be pushed to your destination.
#### Invoice record matching
Invoice records can be matched from your source to your Xero workspace by:
- **Invoice ID**
- **Invoice Number**
- **Credit Note ID**
- **Credit Note Number**
#### Account record matching
Account records can be matched from your source to your Xero workspace by:
**Account ID**
**Code**
#### Field mapping
Hightouch allows you to sync columns from your source to the supported Xero fields.
### Invoices
#### Invoice type selection
Select the type of invoices you would like to sync to your destination. To sync dynamic types, toggle 'use column' and select a column from your model that contains supported enum values by Xero.
#### Sync modes
This integration supports:
- **Upsert** mode: Records that aren't found in Xero given the provided identifier will be inserted. Existing records will be updated.
- **Insert** mode: Records that aren't found in Xero given the provided identifier will be inserted.
#### Record matching
In **Upsert** mode, records can be matched from your source to your Xero workspace by:
- **Invoice Number**
#### Field mapping
Hightouch allows you to sync columns from your source to the supported Xero fields.
#### Associations
Hightouch can perform a reference look-up for reference fields. Select an identifier to use to perform the look-up and use the respective contact to create the invoice object.
You can perform a look-up using:
- **Contact Number**
- **Email Address**
- **Name**
If the `Contact ID` exists in your model, you can skip the lookup step by selecting `Contact ID` from the drop-down menu.
### Items
#### Invoice type selection
Select the type of invoices you would like to sync to your destination. To sync dynamic types, toggle 'use column' and select a column from your model that contains supported enum values by Xero.
#### Sync modes
This integration supports:
- **Upsert** mode: Records that aren't found in Xero given the provided identifier will be inserted. Existing records will be updated.
- **Insert** mode: Records will be pushed to your destination.
#### Record matching
In **Upsert** mode, records can be matched from your source to your Xero workspace by:
- **Item Code**
#### Field mapping
Hightouch allows you to sync columns from your source to the supported Xero fields.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Xtremepush
**URL:** https://hightouch.com/docs/destinations/xtremepush
**Description:** Sync data to Xtremepush as user profiles, to trigger campaigns, or to maintain user list membership.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
----------------------|------------------------------------------------------------------------|----------------------|---------------------------------------------------------------------------------
**User profiles** | Sync data from any source to Xtremepush user profiles | Upsert | [User profiles docs](https://docs.xtremepush.com/reference/user-profile-import)
**User lists** | Sync data from any source to maintain Xtremepush user list memberships | Add, Remove | [User lists docs](https://docs.xtremepush.com/reference/list-import)
**Campaign triggers** | Trigger campaigns using source data | Insert | [Campaign triggers docs](https://docs.xtremepush.com/reference/campaign-execute)
**Events** | Sync data from any source to Xtremepush events | Insert | [Events docs](https://docs.xtremepush.com/reference/event-batch-hit)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Xtremepush
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Xtremepush** and click **Continue**. You can then authenticate Hightouch to **Xtremepush**.
Enter the following fields into Hightouch:
- **Base URL**
- **App token**
## Sync configuration
Once you've set up your Xtremepush destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Xtremepush destination you want to sync to.
### Syncing user profiles
Sync data from any source to Xtremepush user profiles.
#### Record matching
To match rows from your model to user profiles in Xtremepush, you need to select the model column that contains values that match the **User ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync user profile fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom user profile fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|---------------------------------------------------------------------
**Do nothing** | Keep the user profile in Xtremepush with all its synced fields
**Clear** | Clear all the mapped fields, but keep the user profile in Xtremepush
**Delete** | Delete the synced user profiles from Xtremepush
### Syncing user lists
Sync data from any source to maintain Xtremepush user list memberships.
#### Record matching
You can match rows from your model to user lists in Xtremepush on any column in your model and any field in Xtremepush. Ensure the data types of the model column and Xtremepush field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync user list fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default user list fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
### Syncing campaign triggers
Trigger campaigns using source data.
#### Record matching
You can match rows from your model to campaign triggers in Xtremepush on any column in your model and any field in Xtremepush. Ensure the data types of the model column and Xtremepush field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
### Syncing events
Sync data from any source to Xtremepush events.
#### Record matching
You can match rows from your model to events in Xtremepush on any column in your model and any field in Xtremepush. Ensure the data types of the model column and Xtremepush field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Yahoo
**URL:** https://hightouch.com/docs/destinations/yahoo
**Description:** Ensure that your campaigns are efficiently reaching the right audiences for the best results with Yahoo DSP.
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| ---------------------------------------- | --------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------- |
| [**Conversion events**](#syncing-events) | Sync data from any source to any events | Insert | [Conversions API](https://help.yahooinc.com/dsp-api/docs/standard-yahoo-conversion-api) |
| [**Segments**](#syncing-segments) | Create and update audiences | Add | [Audiences API](https://developer.yahooinc.com/dsp/api/docs/traffic/audience/about-audience.html) |
## Connect to Yahoo
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Yahoo** and click **Continue**.
The Yahoo destination has two independent credential sections: [**Segment integration**](#configure-segment-credentials) and [**Conversion events**](#request-and-configure-conversion-event-credentials). Configure whichever section matches the sync type you plan to use.
You don't have to complete both sections during setup. For example, you can
configure **Segment integration**, save the destination, and come back later
to add **Conversion events** credentials from the destination's
**Configuration** tab once Yahoo has replied to your request.
### Configure Segment credentials
For Hightouch to sync data to your segments, you need to enter:
- **Client ID**
- **Client Secret**
To retrieve these, open your [Yahoo DSP Dashboard](https://sso.admanagerplus.yahoo.com/app/dashboard/list). In the upper-right corner, select your name, then **My Account**.
Activate the API and agree to the terms.
A success message displays your credentials.
Copy the client ID and secret, and keep it in a safe place.
For more details, visit the [Yahoo DSP Authentication docs](https://developer.yahooinc.com/dsp/api/docs/authentication/vmdn-auth-overview.html?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAABxrNZzuVEB2pL8Fdf9cjsTku2w7ooTz2Mz7BCA_InEazpCygsScZNhzJVJ4SJypKZzOTik4_avcNoAAspLZaQ6nEuoKvVg8OUVL6VMTkOdeVAup900GJ8S_ql543N_0WRsyyQgpAr41kcwJpHGDYLIQjtfViMeWaVYod_mxkmhR).
### Request and configure Conversion event credentials
The Yahoo Conversion API requires approval from both Yahoo and Hightouch before you can configure it.
#### Step 1: Gather your Yahoo DSP identifiers
You need this information from your Yahoo DSP account before you contact Yahoo:
- **Client name**: Your advertiser name, found on your advertiser profile page in Yahoo DSP.
- **Pixel ID**: The ID of the pixel you'll use for the Conversion API. To find or create one, go to your advertiser in Yahoo DSP, open the **Tracking** tab, and select **Get Pixel Code**. This lists all pixels associated with your account. If you don't have one yet, select **Create New Pixel**, name it, and save it. The pixel ID appears in the list and is also embedded in the full pixel tag.
- **MDM ID**: Your advertiser's MDM ID, if you know it. This is optional.
Use a pixel dedicated to the Conversion API only. Don't share it with other
tracking methods.
#### Step 2: Request credentials from Yahoo
In the destination's **Conversion events** section, download the **Hightouch public key**. Then select **Email Yahoo team** to open a pre-filled email addressed to `technical.solutions@yahooinc.com` with the public key attached:
```text
Subject: Yahoo CAPI client credentials request
Hi Yahoo Team,
I'm interested in using the Yahoo Conversions API (CAPI) integration through Hightouch and would like to request that client credentials be generated for our account.
Client Name:
Pixel ID:
MDM ID:
For your convenience, the Hightouch public key is attached.
Please let me know if you need any additional information to complete the setup.
Thank you,
```
You can also send this request yourself instead of using the pre-filled draft. Yahoo encrypts your credentials with the public key and sends back an encrypted credentials file.
#### Step 3: Upload your credentials to Hightouch
Once you receive the encrypted credentials file from Yahoo, upload it in the **Conversion events** section –– either in the setup flow or in the destination's **Configuration** tab.
## Sync configuration
Once you've set up your Yahoo destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Yahoo destination you want to sync to.
### Syncing events
When syncing events, Hightouch treats any records added to [your source](/getting-started/concepts#sources) as new events and sends them to Yahoo DSP when your sync runs.
#### Pixel ID
You must set a Pixel ID when syncing conversion events to Yahoo DSP. [Create a new Pixel ID](https://help.yahooinc.com/dsp-api/docs/standard-yahoo-conversion-api#create-a-pixel-id-yp-id-in-the-yahoo-dsp) in Yahoo DSP under the tracking tab for your advertiser in Yahoo DSP.
You can also map a model column to the Pixel ID field instead of entering a static value. This lets a single sync route events to different pixels, based on a value in each row.
The Pixel ID used for CAPI should be unique and **only** used for CAPI.
#### Choose a delivery method
Yahoo CAPI accepts conversion events through both a streaming and a batch endpoint.
- **Batch** (recommended): Yahoo processes batch events once a day. Use batch delivery when you're sending more than 100,000 events. The batch endpoint doesn't return an indication of rejected rows.
- **Streaming**: Yahoo processes streamed events multiple times a day. Streaming works best for smaller event loads.
#### Event timestamp
You can include a timestamp of when the event occurred. If this field is empty, Hightouch uses the time the event arrives at the server.
Hightouch converts any `DateTime` values to Epoch UTC milliseconds per Yahoo DSP's expectations.
#### Field mapping
Ensure that at least one of the `userData` fields is included in your mapped fields, otherwise Yahoo DSP will reject the event.
For more information about required and supported parameters, refer to [Yahoo's Conversion API documentation](https://help.yahooinc.com/dsp-api/docs/standard-yahoo-conversion-api#capi-payload).
#### Automatically hash PII
By default, Hightouch detects and hashes values for fields that Yahoo requires to be hashed, including `userData.email` and `userData.phone`. Turn this off if you're already sending hashed values.
#### Rate limits
Yahoo CAPI's default rate limits are 200 requests per second and 10 MB of data per second on the batch endpoint. If Yahoo has enabled a higher rate limit for your pixel, expand **Advanced** in the sync configuration and enter it in the rate limit field. Hightouch adjusts its send behavior accordingly.
#### Configure conversions in Yahoo DSP
After Hightouch begins syncing conversion events to your pixel, configure how Yahoo DSP interprets that data. Create **pixel rules** in the Yahoo DSP UI to segment the incoming event stream into named conversions you can attach to campaigns.
##### Create a pixel rule
1. In Yahoo DSP, go to your advertiser and open the **Tracking** tab.
2. Select the pixel you're using for CAPI, then select **Create Rule**.
3. Name the rule and associate it with your pixel.
4. Set the **goal type** — for example, `Purchase`.
5. Enter a placeholder value, such as `1`, in the value field. If you pass a dynamic value in your Hightouch sync, it overrides this placeholder at the event level.
##### Define rule conditions
Pixel rules filter incoming events by matching on event data fields:
| Yahoo DSP field | Maps from |
| --------------- | ------------- |
| Category | `category` |
| Label | `subcategory` |
| Action | `eventName` |
You can create conditions using these operators:
- **Equals**: works with strings and numbers.
- **Contains**: works with strings.
- **Less than** / **Greater than**: works with numbers only.
- **Regex**: matches one of several values. Pipe-delimit values inside parentheses, for example `(purchase|checkout|order_complete)`.
##### Set audience membership recency
Each rule automatically creates an audience segment. You can configure how recently a user must meet the rule's conditions to be included — for example, at least once in the last 30 days, or at least three times in the last 15 days. The time unit can be hours or days. This recency window controls audience membership, not conversion attribution look-back.
##### Attach the rule to a campaign line
1. Open the relevant campaign in Yahoo DSP and go to the line item.
2. In **Line Properties**, set a **Goal Type** and select your pixel rule.
3. Set your click and view look-back windows. Both default to 7 days, with a range of 1 hour to 30 days.
### Syncing segments
The Yahoo DSP enables you to build audiences that are most responsive to your campaigns, are most likely to convert, or are otherwise advantageous.
Begin by selecting the Yahoo account you would like to sync to.
Then, select whether to create a new audience or use an existing one.
When creating a new audience, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model. To use an existing audience, select the desired audience from the dropdown.
### Audience type selection
Yahoo DSP API allows you to create different types of audiences.
Hightouch supports email address, device ID, and IP address types.
You can only select one audience type per sync.
You can later combine the different audience types when building your campaigns in Yahoo.
#### User identifiers
To identify which users to add or update in an audience, select model columns and the corresponding Yahoo fields.
The available fields vary depending on your selected audience type.
Yahoo DSP's API doesn't support removing identifiers or audience members.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Yahoo DataX
**URL:** https://hightouch.com/docs/destinations/yahoo-datax
**Description:** Yahoo DataX enables partners to seamlessly and securely activate their data and enrich it with Yahoo's robust data assets for more powerful advertising.
**Section:** Destinations
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| --- | --- | --- | --- |
| [**Audiences**](#syncing-audiences) | Sync data from any source to Yahoo DataX audiences | Add | [Audiences docs](https://help.yahooinc.com/datax/docs/user-audience-data) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Using the Yahoo DataX Hightouch integration
To use the Yahoo DataX Hightouch integration, contact your Yahoo representative. Once approved, your Hightouch account will be allowed to use the Yahoo DataX integration.
## Taxonomy
Hightouch manages your Yahoo audience structure (also known as taxonomy). Each workspace you connect to the Yahoo DataX destination creates it's own taxonomy that you can eventually see in Yahoo DSP. Yahoo DataX supports `Syndicated` and `Custom` audiences based on the type of audiences you want to sync.
`Syndicated` audiences can be accessed by all advertisers that you have whitelisted in the destination configuration. `Custom` audience access is granted at the audience level, providing more granular access control for privately listed audiences. Typically `Custom` audiences are only shared with one advertiser. However, you can share with multiple advertiser MDM IDs.
Learn more about Yahoo DataX's taxonomy [here](https://help.yahooinc.com/datax/docs/taxonomy-1).
## Segment Fees
Hightouch manages your Yahoo audience's segment fees. Each audience can have any of the four segment fee types that Yahoo DataX supports:
- `Display`
- `Video`
- `Connected TV`
- `Audio`
CPM values can range between $0 and $50 (both inclusive). If you do not set segment fees, they will by default be 0 in Yahoo DSP.
**Note:** Yahoo only updates audience fees once per day, at the end of each day. If you have updated audience fees but do not see the change in Yahoo DSP, you may need to wait a day.
Learn more about audience fees [here](https://help.yahooinc.com/datax/docs/fee).
## Connect to Yahoo DataX
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Yahoo DataX** and click **Continue**. You can then authenticate Hightouch to **Yahoo DataX**.
Enter the following fields into Hightouch:
- **Brand name**: This will be the name of your brand's top-level audience folder shown in Yahoo DSP. It will appear under `Hightouch > BRANDED > `
- (Optional) **Your MDM ID for internal access**: Enter your company’s own Master Data Management identifier (MDM ID) to grant your team access to view and use all of your `Syndicated` and `Custom` audiences within the Yahoo DSP. Contact your Yahoo representative for your MDM ID.
- (Optional) **Syndicated audience whitelist MDM IDs**: `Syndicated` audiences are accessible to all Yahoo DSP advertisers who have been whitelisted here. To whitelist advertisers and grant them access to your `Syndicated` audiences, enter their MDM IDs here. Contact your Yahoo representative for valid MDM ID values.
## Sync configuration
Once you've set up your Yahoo DataX destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Yahoo DataX destination you want to sync to.
### Syncing audiences
Sync data from any source to Yahoo DataX audiences.
#### Record matching
You can match rows from your model to audiences in Yahoo DataX on any column in your model and any field in Yahoo DataX. Ensure the data types of the model column and Yahoo DataX field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync audience fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default audience fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
Yahoo DataX allows the following identifier`urnType`s for your audiences. You may select any number of identifier types, and each identifier will be sent in the sync.
- `Email (sha_256)`
- `Phone (sha_256)` (referred to as `HASHEDID` by Yahoo)
- `ZIP4`
- `IDFA`
- `GPAVID`
After Hightouch finishes syncing, audience uploads typically takes 24-48 hours to reflect in Yahoo DSP.
## Tips and troubleshooting
### Rate limit recommendation
It is highly recommended that you stagger your Yahoo DataX audience syncs. Yahoo DataX has a rate limit of `100 req/hour`, and to avoid hitting rate limits, please stagger your sync schedules.
### Common errors
### Live debugger
### Sync alerts
---
## Zapier
**URL:** https://hightouch.com/docs/destinations/zapier
**Description:** Use Zapier as a destination if you are trying to quickly migrate system or if you need to connect a service we do not yet support.
**Section:** Destinations
## Overview
Zapier is best used for when there's a destination we do not yet support or if you want make a quick migration to using Hightouch from Zapier. Regardless, it's highly recommend you follow the steps to ensure a smooth setup as Zapier is quite unique compared to our other destinations
## Using Zapier
Due to limitations, we use Zapier's webhook trigger to send data as it's they easiest way set up for users.
#### Rate limits
Something to keep in mind, as you process all your rows of data is that Zapier limits the amount of times Hightouch can send data to the webhook. For example, Zapier **only accept 10,000 requests every 5 minutes**. It's important to keep this in mind when selecting which triggers you want to activate for sending your records, as each record triggered will count as 1 request sent.
[More Information On Zapier's Webhook Limit](https://zapier.com/help/troubleshoot/behavior/rate-limits-and-throttling-in-zapier#step-4)
## Setup
### Connect to Zapier
#### Creating Webhook URL in Zapier
1. Log in to Zapier
2. Click on **Create Zap** (skippable if editing zaps)
3. Click on **Trigger** tab and select the **Webhooks by Zapier** trigger
4. Click on **Trigger Event**, select **Catch Hook**, and then click **Continue**
5. Copy your custom webhook URL
6. Click **Continue**
it's important to not test the trigger until you are in the sync creation
step
#### Adding destination in HT
1. Log in to Hightouch or create a free Hightouch account
2. Click on **Destinations** in the left sidebar
3. Click on **Add destination** in the top right corner
4. Select **Zapier** in the destination catalog and click **Continue** to proceed
5. Paste your webhook URL from Zapier into the **Zapier Webhook Trigger URL ** field and click **Continue** to proceed
6. Give your destination a name and unique slug (for example, "Zapier (Google Sheet Zap)" and `zapier-googlesheet-zap`)
7. Click **Finish** to create your Zapier destination.
### Connect to your data source
Hightouch sits on top of your data warehouse and can read from Snowflake, Redshift, BigQuery, Databricks, Postgres, MySQL, and many other data sources. You can even fetch data from Google Sheets and Airtable. Hightouch can help activate your data wherever it resides.
We've written dedicated guides for each supported data source. Take your pick:
- Connect to [Airtable](/sources/airtable)
- Connect to [Amazon Athena](/sources/amazon-athena)
- Connect to [Amazon Redshift](/sources/amazon-redshift)
- Connect to [ClickHouse](/sources/clickhouse)
- Connect to [Databricks](/sources/databricks)
- Connect to [Elasticsearch](/sources/elasticsearch)
- Connect to [Firebolt](/sources/firebolt)
- Connect to [Google BigQuery](/sources/google-bigquery)
- Connect to [Google Sheets](/sources/google-sheets)
- Connect to [Looker](/sources/looker)
- Connect to [Metabase](/sources/metabase)
- Connect to [MySQL](/sources/mysql)
- Connect to [Palantir Foundry](/sources/foundry)
- Connect to [PostgreSQL](/sources/postgresql)
- Connect to [Rockset](/sources/rockset)
- Connect to [SFTP](/sources/sftp)
- Connect to [Snowflake](/sources/snowflake)
- Connect to [SQL Server](/sources/sqlserver)
- Connect to [Trino](/sources/trino)
If you're missing the credentials necessary to connect to your data source,
try inviting your teammates to Hightouch. You can have unlimited collaborators
in your workspace.
#### Connect to a sample database instead
Waiting for credentials to your data warehouse? No problem. You can connect to one of our [sample datasets](/sources/sample-data) to get started right away. For the rest of this tutorial, we'll use the **B2B SaaS** sample source, which contains a `users` table with names, emails, locations, and other user attributes. This sample source is available when you log in for the first time—no setup required.
### Create a model
In Hightouch, a model represents a query that filters or transforms the records in your data source. Models are used to determine exactly which records to sync between sources and destinations.
In this example, we'll keep it simple. Suppose you want to use Zapier to retrieve information for your customers located in the city of Nashville. To accomplish this, you'll want to create a model that looks like this:
1. Click on **Models** in the left sidebar
2. Click on **Add model** in the top right corner
3. Select **Demo Database** as your source and click **Continue** to proceed
4. Select **SQL Editor** as your modeling method
5. Paste following the SQL query into the editor on the left side of the page
```sql
SELECT * FROM users WHERE location = 'Nashville';
```
6. Click **Preview** to execute the query
7. Click **Continue** to proceed
8. Give your model a name and unique slug (for example, "Users in Nashville" and `users-in-nashville`)
9. Select **ID** as the primary key for your model
10. Click **Finish** to create your model.
### Create a sync
1. Click on **Syncs** in the left sidebar
2. Click on **Add sync** in the top right corner
3. Select **Users in Nashville** as your model and click **Continue** to proceed
4. Select **Zapier** as your destination and click **Continue** to proceed
5. Select whether you want to map your data or keep as is
6. Click on **Acknowledgement of testing** checkbox to en
7. Select **Create a new segment**
8. Give a name to your segment (for example, "My beta test")
9. Click **Continue** to proceed to the last step
10. Go back to the Zap you created, and click **Test Trigger**
11. Verify that a request was found in Zapier
12. Go back to Hightouch, and click **Continue** to proceed to the last step
13. Select **Interval** to tell your sync to run on a set interval
14. Configure your sync schedule to run every **5 minutes**
15. Click **Finish** to create your sync.
## Tips and troubleshooting
### Common errors
#### Couldn't find request
There are a few reasons you may encounter this error:
1. You tried to test the webhook trigger in Zapier, before you tested it in Hightouch.
2. You tested in Hightouch, however you may have tested it for a trigger you did not enable. For example, if you selected to trigger on **Record Changed**, you need to make sure run test as **Sync As Row Changed**
### Live debugger
### Sync alerts
---
## Zendesk
**URL:** https://hightouch.com/docs/destinations/zendesk
**Description:** Deliver better support by making user events and attributes available inside Zendesk
**Section:** Destinations
## Setup
Navigate to Admin Settings > Apps and integrations > Zendesk API and add a new API token for Hightouch. Make sure you enable _Token access_
Copy the API token and fill in the fields in Hightouch destination setup:
- **API token:** token from Zendesk
- **Email:** email used to login to Zendesk
- **Support URL:** URL for your Zendesk support site. make sure to include the `https://` (ex: `https://hightouch.zendesk.com`)
## Syncing
### Supported objects
This integration support syncing the following objects:
- `Users`
- `Organizations`
- `Tickets`
- `Custom Objects`
- `Legacy Custom Objects`
- `Organization Memberships`
Zendesk has [custom objects](https://support.zendesk.com/hc/en-us/articles/5914453843994-Understanding-custom-objects) and [legacy custom objects](https://support.zendesk.com/hc/en-us/articles/4408834725402). If your Zendesk account has any custom objects, they show up in the objects dropdown.
Custom objects have a set prefix: **(Custom)**. Legacy custom objects have a set prefix: **(Legacy custom)**
### Sync modes
- **Upsert**: pushes new objects to Zendesk and updates fields that change in your warehouse.
- **Update**: updates particular fields on existing objects in Zendesk. It doesn't add new objects.
- **Insert**: pushes new objects to Zendesk. It doesn't update any objects. (Only available for `Organization Memberships`)
### Record matching
Records can be matched from your source to your Zendesk objects by selecting a column in your results that matches the external ID, Zendesk ID, email \(_contact only_\), or custom fields of the Zendesk object.
#### Organization memberships
Organization memberships will behave differently. These objects will be matched by both the `user_id` _and_ `organization_id`.
This field should be a combination of your user and organization columns that you will use in the mappings section. For example,
if you are using the user's `email` and the organization's `external_id`, this field should be: `user_email:org_externalId`. Do not edit/change
either the user or organization field. Instead, you will need to remove the old row and create a new one to represent a new organization membership.
Your model should look similar to below:
### Field mapping
You can sync columns from your source to Zendesk's default fields, as well as custom fields for your users, organizations and tickets objects. For more info on how to add custom fields to users, refer to the [Zendesk docs](https://support.zendesk.com/hc/en-us/articles/203662066).
For custom objects, you will be able to sync to any properties from your defined schema.
### Relationship records
You can create associations between existing object records using a predetermined relationship type. The associations can be found in the field mappings. For more details, you can refer to the [Zendesk docs](https://developer.zendesk.com/api-reference/custom-data/custom-objects-api/relationships).
### Search batch size
Zendesk's search API can return up to 1000 results, but by default Hightouch only searches for 100 records
at a time. You can increase this amount up to 500 for a faster sync.
However, increasing the batch size
can increase the risk of receiving a ['414 - Long URI`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/414) error. Conversely, if you are experiencing these errors
with the default batch size setting, you can lower the batch size to avoid the errors.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Zoho CRM
**URL:** https://hightouch.com/docs/destinations/zohocrm
**Description:** Push enriched data from your warehouse into Zoho CRM for your teams to act upon it
**Section:** Destinations
## Setup
After selecting Zoho CRM from our Destinations catalog, you will be prompted to
initiate an OAuth flow. Connect Zoho CRM via OAuth by granting permission to the Hightouch app.
## Syncing
Hightouch supports syncing to the following Zoho CRM Modules:
- `Leads`
- `Accounts`
- `Contacts`
- `Deals`
- `Campaigns`
- `Cases`
- `Solutions`
- `Products`
- `Vendors`
- `Price Books`
- `Quotes`
- `Sales Orders`
- `Purchase Orders`
- `Invoices`
- `Custom Modules`
### Sync modes
- **Upsert** - pushes new records to a module and all fields will be kept up-to-date within Zoho CRM.
### Record matching
Records can be matched from your source to your Zoho CRM workspace by using Zoho's _System-defined
Duplicate Check Fields_ or an _External ID_ you set for a module in your workspace settings.
To know more about each modules's _Sytem-defined Duplicate Check Fields_, refer to the _Sytem-defined Duplicate Check Fields_
section in the [Upsert Records API](https://www.zoho.com/crm/developer/docs/api/v2/upsert-records.html). Additionally, to learn
about creating an _External_ID_, refer to [this Zoho docs article](https://www.zoho.com/crm/developer/docs/api/v2/records-api-ext-id-overview.html).
### Field mapping
You can sync columns from your model to Zoho's modules. Ensure the module's _mandatory field_
is included in the sync and the model columns representing the module fields are of the correct data type.
To know more about each module's _mandatory fields_, refer to the _System-defined mandatory fields for
each module_ section in [this Zoho docs page](https://www.zoho.com/crm/developer/docs/api/v2/insert-records.html). Additionally,
to know more about module field types, refer to the _Sample Attributes_ section.
### Delete mode
You can choose what Hightouch's behavior is when records leave the query result set. The default is doing nothing, but you can also set Hightouch to delete the Zoho CRM record.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Zoomd
**URL:** https://hightouch.com/docs/destinations/zoomd
**Description:** Target and suppress customers with mobile ads across 300M mobile devices using AI-driven insights, ensuring more effective and personalized campaigns.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
--------------|-----------------------------------------------|----------------------|-----------------------------------------------------------------------------------
**Audiences** | Sync data from any source to Zoomd audiences | Add, Remove | [Audiences docs](https://developers.kayzen.io/reference/audience-creation-via-api)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Zoomd
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations?from=docs) and click the **Add destination** button. Select **Zoomd** and click **Continue**. You can then authenticate Hightouch to **Zoomd**.
Enter the following fields into Hightouch:
- **API key**
## Sync configuration
Once you've set up your Zoomd destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs?from=docs) and click the **Add sync** button to begin. Then, select the relevant model and the Zoomd destination you want to sync to.
### Syncing audiences
Sync data from any source to Zoomd audiences.
#### Record matching
To match rows from your model to audiences in Zoomd, you need to select a model column and corresponding Zoomd field. You can match on any of the following Zoomd fields:
- **IDFA**
- **GAID**
#### Field mapping
Hightouch lets you sync audience fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default audience fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Zuora
**URL:** https://hightouch.com/docs/destinations/zuora
**Description:** Sync business-critical data from your data warehouse to Zuora with Hightouch
**Section:** Destinations
## Setup
After selecting Zuora from our Destinations catalog, you will be prompted to enter your Zuora Base URL, Client ID, and Client Secret.
Use [this table](https://www.zuora.com/developer/api-reference/#section/Introduction/Access-to-the-API) from Zuora to select the correct Zuora URL, and follow
[these steps](https://knowledgecenter.zuora.com/Billing/Tenant_Management/A_Administrator_Settings/Manage_Users?_gl=1%2Ap9mz1o%2A_ga%2AMzI5ODc0MjYxLjE2NDY3NjQxMDI.%2A_ga_MY8CQ650DH%2AMTY1NDA0MTU1MC40My4xLjE2NTQwNDE2MTIuMA..&_ga=2.23071486.552820277.1653943857-329874261.1646764102#Create_an_OAuth_Client_for_a_User)
to create an OAuth Client in Zuora that will provide you with the required Client ID and Secret.
## Syncing
Hightouch supports syncing to the following Zuora objects:
- `Accounts`
- `Subscriptions`
- `Usage`
### Sync modes
Here are the possible modes for how to alter Outreach objects:
- **Upsert**: pushes new objects to Zuora and updates fields that change in your warehouse.
- Supported for `Accounts` and `Subscriptions`.
- **Update**: updates particular fields on existing objects in Zuora. It doesn't add new objects.
- Supported for `Accounts`, `Subscriptions`, and `Usage`.
- **Insert**: adds new objects in Zuora.
- Supported for `Usage`.
### Record matching
Records can be matched from your source to your Zuora workspace by the given fields in the dropdown.
The following are supported record ID's for each object:
- **`Accounts`**: `Account Number`, `Account ID`
- **`Subscription`**: `Subscription Number`, `Subscription ID`
- **`Usage`**: `Usage Record ID`
A record ID is required when performing `Upsert` or `Update` actions, but not
`Insert`.
### Field mapping
You can sync columns from your source to Zuora's object properties, including
custom fields.
#### Required fields
Each object / action pair will have their own required properties to be mapped. These required properties will be in their own mapping section during sync configuration. Be sure to scroll down to our [Zuora Object Notes](#object-notes) to see which fields are required, and their respective data types.
#### Optional and custom fields
Any optional built-in fields or custom fields sent to Zuora will be mapped in the custom mappings section of sync configuration. Scroll down to our [Zuora Object Notes](#object-notes) to see Zuora's optional built-in fields, and their respective data types.
#### Helpful tip:
Use our advanced mapper to fill in static values for numbers or string values that may be consistent across your sync, that may not have a column in your data warehouse. See the example below.
### Zuora object notes
#### Accounts
The following fields can be used for `Accounts` for each respective action.
{/* */}
`Upsert` *(click to view table)*
| Field Name | Notes | Required |
| ----------------------- | ------- | -------- |
| accountNumber | string |
| applyCredit | boolean |
| applyCreditBalance | boolean |
| autoPay | boolean |
| batch | string |
| billCycleDay | integer | yes |
| billToContact.firstName | string | yes |
| billToContact.lastName | string | yes |
| collect | boolean |
| communicationProfileId | string |
| creditMemoReasonCode | string |
| creditMemoTemplateId | string |
| crmId | string |
| currency | string | yes |
| debitMemoTemplateId | string |
| documentDate | string |
| invoice | boolean |
| invoiceCollect | boolean |
| invoiceTargetDate | string |
| invoiceTemplateId | string |
| name | string | yes |
| notes | string |
| parentId | string |
| paymentGateway | string |
| paymentTerm | string |
| runBilling | boolean |
| salesRep | string |
| sequenceSetId | string |
| soldToSameAsBillTo | boolean |
| tagging | string |
| targetDate | string |
| Class\_\_NS | string |
| CustomerType\_\_NS | string |
| Department\_\_NS | string |
| IntegrationId\_\_NS | string |
| IntegrationStatus\_\_NS | string |
| Location\_\_N | string |
| Subsidiary\_\_NS | string |
| SyncDate\_\_NS | string |
| SynctoNetSuite\_\_NS | string |
`Update` *(click to view table)*
| Field Name | Notes |
| ------------------------- | ------- |
| autoPay | boolean |
| batch | string |
| billCycleDay | integer |
| billToContact | Contact |
| communicationProfileId | string |
| creditMemoTemplateId | string |
| crmId | string |
| debitMemoTemplateId | string |
| defaultPaymentMethodId | string |
| invoiceDeliveryPrefsEmail | boolean |
| invoiceDeliveryPrefsPrint | boolean |
| invoiceTemplateId | string |
| name | string |
| notes | string |
| parentId | string |
| paymentGateway | string |
| paymentTerm | string |
| salesRep | string |
| sequenceSetId | string |
| soldToContact | Contact |
| tagging | object |
| Class\_\_NS | string |
| CustomerType\_\_NS | string |
| Department\_\_NS | string |
| IntegrationId\_\_NS | string |
| IntegrationStatus\_\_NS | string |
| Location\_\_NS | string |
| Subsidiary\_\_NS | string |
| SyncDate\_\_NS | string |
| SynctoNetSuite\_\_NS | string |
Notes:
- By default, `autoPay` is set to `false` for inserted `Accounts`, unless it's defined in the mappings for your sync.
- If you would like to set `autoPay` to true, you must include either a `paymentMethod`, `creditCard`, or `hmpCreditCardPaymentMethodId` in your mappings.
- If either the `workEmail` or `personalEmail` fields are included in mappings, the account's email delivery preference is automatically set to `true`.
- To see more information on any respective field, view the Zuora API docs for [Creating an account](https://www.zuora.com/developer/api-reference/#operation/POST_Account) or [Updating an account](https://www.zuora.com/developer/api-reference/#operation/PUT_Account) via the API.
#### Subscriptions
The following fields can be used for `Subscriptions` for each respective action.
`Upsert` *(click to view table)*
| Field Name | Notes | Required |
| -------------------------------------- | ------- | -------- |
| accountKey | string | yes |
| applyCredit | boolean |
| applyCreditBalance | boolean |
| autoRenew | boolean |
| collect | boolean |
| contractEffectiveDate | string | yes |
| creditMemoReasonCode | string |
| customerAcceptanceDate | string |
| documentDate | string |
| externallyManagedBy | string |
| gatewayId | string |
| initialTerm | integer |
| initialTermPeriodType | string |
| invoice | boolean |
| invoiceCollect | boolean |
| invoiceOwnerAccountKey | string |
| invoiceSeparately | boolean |
| invoiceTargetDate | string |
| notes | string |
| paymentMethodId | string |
| renewalSetting | string |
| renewalTerm | integer | yes |
| renewalTermPeriodType | string |
| runBilling | boolean |
| serviceActivationDate | string |
| subscriptionNumber | string |
| subscribetoRatePlans.productRatePlanId | string | yes |
| targetDate | string |
| termStartDate | string |
| termType | string | yes |
| CpqBundleJsonId\*\*QT | string |
| OpportunityCloseDate\*\*QT | string |
| OpportunityName\*\*QT | string |
| QuoteBusinessType\*\*QT | string |
| QuoteNumber\*\*QT | string |
| QuoteType\*\*QT | string |
| IntegrationId\*\*NS | string |
| IntegrationStatus\*\*NS | string |
| Project\*\*NS | string |
| SalesOrder\*\*NS | string |
| SyncDate\_\_NS | string |
`Update` *(click to view table)*
| Field Name | Notes |
| -------------------------------- | ---------------- |
| add | Array of objects |
| applicationOrder | Array of strings |
| applyCredit | boolean |
| applyCreditBalance | boolean |
| autoRenew | boolean |
| clearingExistingBillToContactId | boolean |
| clearingExistingPaymentTerm | boolean |
| collect | boolean |
| creditMemoReasonCode | string |
| currentTerm | integer |
| currentTermPeriodType | string |
| documentDate | string |
| externallyManagedBy | string |
| includeExistingDraftDocItems | boolean |
| includeExistingDraftInvoiceItems | boolean |
| invoice | boolean |
| invoiceCollect | boolean |
| invoiceSeparately | boolean |
| invoiceTargetDate | string |
| notes | string |
| preview | boolean |
| previewType | string |
| renewalSetting | string |
| renewalTerm | integer |
| renewalTermPeriodType | string |
| runBilling | boolean |
| targetDate | string |
| termStartDate | string |
| termType | string |
| CpqBundleJsonId\_\_QT | string |
| OpportunityCloseDate\_\_QT | string |
| OpportunityName\_\_QT | string |
| QuoteBusinessType\_\_QT | string |
| QuoteNumber\_\_QT | string |
| QuoteType\_\_QT | string |
| IntegrationId\_\_NS | string |
| IntegrationStatus\_\_NS | string |
| Project\_\_NS | string |
| SalesOrder\_\_NS | string |
| SyncDate\_\_NS | string |
{/* */}
Notes:
- Only the fields to be changed should be specified. If an empty field is submitted with this operation, the corresponding field in the account is emptied.
- To see more information on any respective field, view the Zuora API docs for [Creating a subscription](https://www.zuora.com/developer/api-reference/#operation/POST_Subscription) or [Updating a subscription](https://www.zuora.com/developer/api-reference/#operation/PUT_Subscription) via the API.
#### Usage
The following fields can be used for `Usage` for each respective action.
`Insert` *(click to view table)*
| Field Name | Type | Required |
| ------------------ | -------------------- | -------- |
| AccountId | string |
| AccountNumber | string |
| ChargeId | string |
| ChargeNumber | string |
| Description | string |
| EndDateTime | string \ |
| Quantity | double | yes |
| StartDateTime | string \ | yes |
| SubmissionDateTime | string \ |
| SubscriptionId | string |
| SubscriptionNumber | string |
| UOM | string | yes |
`Update` *(click to view table)*
| Field Name | Notes |
| ------------------ | -------------------- |
| EndDateTime | string \ |
| Quantity | double |
| RbeStatus | string |
| StartDateTime | string \ |
| SubmissionDateTime | string \ |
| UOM | string |
Notes
- Any field with a type of `string ` must be in the format `"YYYY-MM-DDTHH:mm:ss"`.
- To see more information on any respective field, view the Zuora API docs for [Creating a usage record](https://www.zuora.com/developer/api-reference/#operation/Object_POSTUsage) or [Updating a usage record](https://www.zuora.com/developer/api-reference/#operation/Object_PUTUsage) via the API.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Build audiences with AI
**URL:** https://hightouch.com/docs/customer-studio/agents
**Description:** Create, refine, and understand audiences using natural language in Customer Studio.
**Section:** Customer Studio
The Agent is available in [Customer Studio](/customer-studio/overview) when
you build or edit an audience.
| | |
| ----------------- | ----------------------------------------------------- |
| **Audience** | Marketers, growth teams, and campaign managers |
| **Prerequisites** | [Customer Studio setup](/customer-studio/quick-start) |
Use the Agent in Customer Studio to build, refine, and understand audiences with natural language. Describe the audience you want, ask questions about an existing audience, or request changes. The Agent translates your input into audience filters, summaries, or analysis that you review before saving.
This page covers the Agent in the audience builder. Hightouch also offers a standalone [Agents](/agents/overview) workspace for open-ended analysis, reporting, and recommendations. See [Which agent should I use?](/agents/compare-agents) to understand the differences.
---
## What you'll learn
In this article, you'll learn how to:
- [Build audiences using natural language prompts](#build-audiences-with-natural-language)
- [Explore available fields, traits, and events in your schema](#explore-your-schema)
- [Summarize, analyze, and explore existing audiences](#understand-and-analyze-audiences)
- [Refine audience definitions through follow-up prompts](#refine-and-improve-audiences)
- [Debug audience size changes and unexpected results](#troubleshoot-audience-issues)
- [Improve results with more specific prompts and schema metadata](#tips-for-better-results)
- [Understand how Hightouch handles data when you use the Agent](#ai-and-data-privacy)
---
## Overview
The Agent is available when you build or edit an audience in Customer Studio. Select **Agent** to open a chat panel alongside the visual builder.
From the Agent panel, you can:
- Build audience definitions from natural language prompts
- Ask questions about an existing audience
- Get summaries and analysis of audience composition
- Refine audience filters through follow-up prompts
- Explore available events, traits, columns, and properties
The Agent works best when your schema has clear, descriptive metadata. If the
Agent can't find the right field, the most common cause is missing or unclear
column descriptions. See [Enrich your schema
metadata](#enrich-your-schema-metadata).
---
## Get started
### Open the Agent
1. Go to [**Customer Studio → Audiences**](https://app.hightouch.com/customer-studio/audiences?from=docs).
2. Create a new audience from [scratch](/customer-studio/usage) or from a [template](/customer-studio/templates), or open an existing audience.
3. Open the **Definition** tab.
4. Select **Agent**.
### Quick actions
The Agent shows different quick actions depending on whether you're working with a new or existing audience. Select a quick action to start, or type your own prompt.
| Context | Sample quick actions |
| --------------------- | ----------------------------------------------------------------------- |
| **New audience** | **Create audience**, plus template or other starter actions depending on workspace setup |
| **Existing audience** | **Generate insights**, **Suggest improvements**, **Summarize audience** |
---
## Build audiences with natural language
Describe the audience you want to build, and the Agent drafts an audience definition with matching filters and conditions.
1. Open the Agent on a new or existing audience.
2. Enter a prompt that describes the audience you want to create. You can also attach images or files for additional context.
For example:
`Create an audience of customers who added items to their cart in the last 30 days but did not complete an order.`
3. Review the audience definition in the visual builder.
4. Check that the filters and conditions match your intent.
5. If the definition needs changes, send a follow-up prompt. The Agent keeps context from the conversation, so you can refine without starting over.
### Prompt patterns for common tasks
These patterns show how to phrase requests for common audience-building workflows. Include specific field names, time windows, and thresholds when you know them.
#### Create an audience from criteria
| What you want | Example prompt |
| ------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| Target a behavioral segment | `Build an audience of customers who purchased at least twice in the last 90 days.` |
| Target by attribute | `Create an audience of users whose account type is enterprise and whose country is United States.` |
| Combine behavior and attributes | `Build an audience of customers who completed an order in the last 30 days and have a lifetime value over $500.` |
| Use event properties | `Create an audience of users who viewed a product in the Electronics category in the last 7 days.` |
#### Refine iteratively
The Agent keeps context from your conversation, so you can refine step by step instead of writing one complex prompt.
| Step | Example prompt |
| ----------------- | ---------------------------------------------------------------------------- |
| Start broad | `Build an audience of active users.` |
| Add specificity | `Narrow it to users who logged in at least three times in the last 30 days.` |
| Add a suppression | `Exclude anyone who already converted.` |
| Check the result | `How many users match this definition?` |
| Adjust | `That's too narrow. Change the login threshold to at least once.` |
If you don't want the Agent to change an existing filter, say so explicitly:
`Add a new filter for email engagement, but don't change the purchase filter.`
---
## Explore your schema
If you're not sure which fields, traits, events, or properties are available for filtering, ask the Agent to explore your schema. This is especially useful when working with a data model you didn't set up yourself.
### Find the right field
| What you want to find | Example prompt |
| ------------------------------ | ------------------------------------------------------------- |
| Columns on the parent model | `What columns are available on the parent model?` |
| Columns related to a concept | `What columns can I use to filter by age or tenure?` |
| Available events | `What events are available for audience filtering?` |
| Properties on a specific event | `What properties are available on the Order Completed event?` |
| Available traits | `What traits can I use in this audience?` |
| Related models | `What related models are connected to this parent model?` |
| Values in a column | `What are the distinct values in the account_type column?` |
### Narrow your search
If the Agent returns too many fields or you can't find what you need, narrow your request:
- **By topic:** `What fields relate to email engagement?`
- **By data type:** `Which columns contain dates or timestamps?`
- **By event:** `What properties does the Page Viewed event have?`
### When the Agent can't find a field
If the Agent says a field doesn't exist, check the following:
- The field might exist under a different name. Ask the Agent: `List all columns that contain the word "email".`
- The field might live on a related model rather than the parent model. Ask: `What related models are available, and what columns do they have?`
- The field might not be exposed in your schema. Column descriptions and event labels are set during [schema configuration](/customer-studio/schema). If a field is missing, ask your data team to check whether it's included in the parent model or related models.
The Agent reads from your schema metadata — model names, column descriptions,
event labels, and related model definitions. Fields without descriptions are
harder for the Agent to find and use correctly.
---
## Understand and analyze audiences
Use the Agent to get plain-language explanations and analysis of an audience without writing queries or leaving the builder.
Select a quick action like **Summarize audience** or **Generate insights**, or enter your own prompt:
`Summarize this audience in 30 words or less for a business user.`
`Which conditions are excluding the most members from this audience?`
`Break down this audience by most recent order value.`
### Analysis prompts
| What you want | Example prompt |
| ---------------------- | ---------------------------------------------------------------------------------- |
| Plain-language summary | `Explain what this audience targets in one sentence.` |
| Size breakdown | `How many users match each condition in this audience?` |
| Exclusion analysis | `Which filter is excluding the most members?` |
| Segment comparison | `How does this audience compare to all customers in terms of average order value?` |
| Condition inspection | `What does the third filter in this audience do?` |
Agent responses can be long. For shorter output, include a word limit or
format in your prompt: `Summarize this in under 30 words.`
---
## Refine and improve audiences
Use the Agent to update an existing audience definition through follow-up prompts. Describe the change you want, then review the updated definition in the visual builder.
You can also send a specific filter to the Agent from the visual builder. Select the **more options** menu on any filter and choose **Add filter to chat**.
### Modify an existing audience
| What you want | Example prompt |
| ---------------------- | ----------------------------------------------------------------------------- |
| Add a condition | `Add a filter to include only users who opened an email in the last 14 days.` |
| Remove a condition | `Remove the filter for country.` |
| Add a suppression | `Exclude anyone who unsubscribed or bounced in the past 60 days.` |
| Narrow by time window | `Change the purchase window from 90 days to 30 days.` |
| Duplicate with changes | `Create the same audience but only for users in the UK.` |
### Apply or undo changes
When the Agent suggests changes to your audience definition:
1. Select **Apply these changes** in the Agent panel to update the visual builder.
2. Review the updated filters and conditions in the visual builder.
3. Select **Save audience** to keep the changes, or **Discard changes** to revert to the original definition.
If you've already applied changes, select **Undo** in the banner at the top of the builder to revert before saving.
---
## Troubleshoot audience issues
Use these patterns when an audience isn't behaving the way you expect. The Agent can help you diagnose size changes, understand exclusions, and identify configuration issues.
### Audience is smaller than expected
Ask the Agent to break down which conditions are filtering out the most users:
`Which filter is removing the most members from this audience?`
`How many users match each individual condition?`
Common causes of unexpectedly small audiences:
- **Overlapping conditions** — Two filters that individually match many users can combine to match very few. Ask the Agent: `How many users match both the purchase filter and the email filter together?`
- **Time windows too narrow** — A 7-day window captures fewer users than a 30-day window. Ask: `What if I change the time window to 30 days?`
- **Missing data** — If a column contains null values, users with null values are excluded. Ask: `How many users have a null value for this field?`
### Audience size changed unexpectedly
If an audience grew or shrank and you're not sure why:
`When did this audience last change in size, and by how much?`
`What could explain a drop in this audience over the last week?`
Common causes:
- **Upstream data changes** — The source data in your warehouse changed. Check whether recent data loads added, removed, or updated rows.
- **Schema or relationship changes** — A related model or trait definition changed, which affected filter evaluation.
- **Time-based filters** — Rolling time windows like "last 30 days" naturally shift which users qualify.
### Agent gives an error or can't complete the request
If the Agent returns an error or says it can't complete your request:
- **Unsupported data type** — Some column types can't be used in filters directly. Ask: `What data type is the [column name] column?` If the column type is unsupported, you may need to create a [computed trait](/customer-studio/traits) or ask your data team to transform the column upstream.
- **Cross-model limitations** — The Agent works within the context of the current audience's parent model and its related models. It can't query across separate parent models or access data from other audiences.
- **Query timeouts** — If analysis queries take too long, try narrowing the scope: `Analyze the first three conditions only.`
### When to use the Agent vs. the standalone Agents chat
The audience Agent works within the context of a single audience: building, editing, analyzing, or debugging one audience definition using the schema already loaded.
For broader tasks — campaign strategy, cross-audience comparisons, reporting, or questions that go beyond the current audience's schema — use [Agents](/agents/overview).
---
## Tips for better results
### Enrich your schema metadata
The Agent uses your schema metadata — model names, column descriptions, and event labels — to understand your data. Clear, descriptive metadata improves the accuracy of Agent responses more than any other factor.
To review and update your schema metadata, go to [**Customer Studio → Schema**](/customer-studio/schema). Add or update:
- **Column descriptions** — Describe what each column contains in plain language.
- **Event labels** — Give events descriptive names that match how your team talks about them.
- **Related model names** — Name related models after the real-world objects they represent.
### Be specific
Instead of `Find my best customers`, try:
`Find customers who completed at least 3 orders in the last 90 days with an average order value over $100.`
Include time windows, thresholds, and the specific events or filters you want. If you don't want the Agent to modify an existing filter, say so explicitly.
### Ask about what's available before building
If you're new to a data model, start by exploring what's available before building the audience:
`What events and properties are available for audience filtering?`
`What traits can I use?`
This helps you write more precise prompts and avoids building on fields that don't exist.
### Review before saving
Always review the audience definition in the visual builder before saving. The Agent can make mistakes, especially with complex logic, vague prompts, or schema fields with unclear names.
---
## AI and data privacy
### How data is processed
When you use the Agent, Hightouch sends relevant audience context to the AI model so it can generate a response. This context can include attributes, behavior signals, and schema metadata.
This data is:
- Not retained after the response is generated
- Not used to train or fine-tune models made available to other customers
- Governed by Hightouch's [AI Terms of Service Addendum](https://hightouch.com/ai-addendum)
### Chat privacy
Your conversations with the Agent are private to you. Other team members can't see your audience-specific chats.
---
## Audience overviews
**URL:** https://hightouch.com/docs/customer-studio/audience-overviews
**Description:** Audience overviews enable visual confirmation of audience construction and health.
**Section:** Customer Studio
| | |
| ----------------- | --------------------------------------------------- |
| **Audience** | Marketers |
| **Prerequisites** | A saved audience. (Optional) At least one dashboard |
_Use the Audience Overview as your audience’s homepage to track size, activity, and sync health, or attach a custom dashboard to surface the metrics most relevant to your goals._
The Audience Overview page is the landing page for all of your audiences. You’ll see a dashboard of charts that help you validate how an audience is built and monitor its health over time.
## What you’ll see
When you open an audience, the **Overview** tab shows a default dashboard with key charts:
- **Audience size**: Current total number of members
- **Audience size over time**: A line chart showing membership growth and change trends
- **Audience events**: Top events by members of the audience
- **Syncs**: Completed syncs over past 7 days
You will need to refresh to view chart information.
### Saved metrics
If your workspace has defined [metrics](/campaign-intelligence/metrics) for your parent model (for example, purchases or revenue), these appear as key KPIs in the Audience Overview.
These appear at the end of the default dashboard. If you use a custom dashboard, these will not appear.
## Custom Dashboards
You can replace the default [dashboard](/campaign-intelligence/dashboards) with one of your own:
1. Go to **Audience settings** (during creation or later by editing)
2. Under **Overview dashboard**, select an existing dashboard to attach
The attached dashboard automatically filters to your audience.
**Key points:**
- The same dashboard can be reused across multiple audiences
- Global dashboard filters are hidden automatically to avoid conflicts when rendered in the audience overview. The original dashboard’s filters remain intact.
## Features and behaviors
- **Time filters**: You can switch between 7, 30, 60, 90-day, or custom windows.
- **Caching**: Charts respect the same caching rules as dashboards, so repeated queries are optimized.
- **Sampling**: If an audience uses sampling, the Overview reflects those settings.
- **Permissions**: Anyone who can view details about an audience's members can also see its Overview. Only editors can change or attach dashboards.
- **AI description**: You can automatically generate a semantic summary of your audience.
---
## Snapshots
**URL:** https://hightouch.com/docs/customer-studio/audience-snapshots
**Description:** Audience snapshots give data marketers a record of membership history over time.
**Section:** Customer Studio
| | |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | **Platform admins** who want a historical record of audience membership to analyze when users entered or exited campaigns. |
| **Prerequisites** |
A source with [Lightning sync engine](/syncs/lightning-sync-engine) enabled
Warehouse [source](/getting-started/concepts#sources) with write permissions
|
_Audience snapshots give marketers a historical record of who was in an audience and when. This is useful for evaluating campaign reach, tracking changes in membership over time, and enabling downstream analysis or reporting._
---
## What you'll learn
After reading this article, you'll be able to:
- Understand how audience snapshots work in Customer Studio
- Enable snapshotting on your source
- Access and interpret snapshot data in your warehouse
---
### Overview
After each sync of an audience, Customer Studio saves a **snapshot** of that audience's membership. These snapshots are written to your data warehouse and include:
- _Only_ **audience member IDs** and no other customer data
- A **timestamp** for when each member entered or exited the audience
- The event type (`enter` or `exit`)
- An optional **split group ID**, if audience [experiments](/customer-studio/experiments) are enabled
{" "}
Audience snapshotting requires `WRITE` permissions to your warehouse. Snapshots
are only available for sources using the [Lightning sync engine](/syncs/lightning-sync-engine).{" "}
## Snapshot table schema
Snapshots are stored in a single shared table in the [**Hightouch Planner schema**](/syncs/lightning-sync-engine#warehouse-schemas) in your warehouse: `hightouch_planner.audience_membership`
| Column | Type | Description |
| ---------------- | ----------- | --------------------------------------------------------------------------------------- |
| `ht_row_id` | `string` | The primary key of the user. Stringified if the original value is not a string. |
| `ht_audience_id` | `string` | The ID of the audience. |
| `ht_timestamp` | `timestamp` | When the user entered or exited the audience. |
| `ht_event_type` | `string` | Either `enter` or `exit`, depending on the change. |
| `ht_split_group` | `string` | The split group ID the user belongs to (if applicable). `null` if no split was applied. |
Each table row contains the audience member's ID, the timestamp of when the snapshot occurred, and either an `enter` or `exit` event type.
An audience's first snapshot records current membership in the form of `enter` events for every member.
Subsequent snapshots only contain changes to membership—either `enter` events for new members or `exit` events for members who aren't present in that snapshot but were in the previous one.
## Enable snapshots
To enable snapshots for a given source:
1. Navigate to your [**Source**](/getting-started/concepts#sources)
2. Go to the `Sync log` tab.
3. Enable **Audience snapshotting** using the toggle.
Once enabled, snapshots will be automatically taken after each successful sync.
---
## Plan your campaign workflow
**URL:** https://hightouch.com/docs/customer-studio/campaign-workflow
**Description:** Plan which Customer Studio features will handle audience qualification, timing, overlap, consent, and measurement before you build a campaign.
**Section:** Customer Studio
| | |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Audience** | Marketers, marketing operations teams, and platform admins planning campaigns that use multiple Customer Studio features together. |
| **Prerequisites** |
A configured [schema](/customer-studio/schema) with a parent model
Familiarity with [audiences](/customer-studio/usage) and [syncs](/customer-studio/syncs)
|
_Before you build a campaign, decide which Customer Studio feature should handle each job: audience qualification, timing, overlap, consent, and measurement. Use this page to plan campaigns that combine audiences, journeys, syncs, experiments, and governance features._
---
## What you'll learn
- [How Customer Studio features for campaigns work together](#how-customer-studio-features-for-campaigns-work-together)
- [Which feature should handle each part of the campaign](#choose-which-feature-to-use)
- [When to use journeys instead of an audience sync](#when-to-use-journeys)
- [How to manage overlap across campaigns](#manage-overlap-across-campaigns)
- [How consent and access rules apply at sync time](#apply-consent-and-access-rules-at-sync-time)
- [How to validate each layer before launch](#validate-each-layer-before-launch)
- [Common campaign patterns](#campaign-patterns)
---
## How Customer Studio features for campaigns work together
The core Customer Studio flow is:
1. Define a schema
2. Build audiences and traits from that schema
3. Sync those audiences to destinations
Campaign planning features build on that foundation.
**Schema** defines which records, events, and fields are available for campaign logic. Audiences, traits, journeys, and syncs all depend on the schema. If a required field changes upstream, downstream campaign assets can break.
**Audiences** define who qualifies for a campaign. They are the starting point for most campaign workflows and feed into syncs, journeys, experiments, and priority lists.
**Syncs** deliver campaign data to destinations. Messages are not sent from Customer Studio directly. A campaign action only happens when a sync sends data to a downstream tool.
**Campaign planning tools** control timing, sequencing, overlap, and reuse:
- **Journeys** handle step order, delays, branching, and exits, but they do not run in real time. Journeys evaluate on a fixed schedule, so entry, movement, exits, and sends only happen when the journey runs.
- **Priority lists** decide which audience wins when a user qualifies for multiple competing audiences.
- **Experiments** split eligible users into treatment and holdout groups so you can measure lift.
- **Templates** let teams reuse approved sync and journey configurations.
**Governance features** control what can actually be delivered:
- **Destination rules** apply destination-specific eligibility or consent rules at sync time.
- **Row-level access** limits which records a team can see and activate.
- **OneTrust integration** enforces consent before data is sent downstream.
- **Data masking** redacts or hashes sensitive fields before delivery.
---
## Choose which feature to use
Start with the campaign job you need to do, not with the feature list.
- If you only need to keep a segment updated in a destination, an audience sync may be enough.
- If you need timing, sequencing, or branching, use a journey.
- If you need to resolve conflicts between campaigns, use a priority list or journey exclusions.
- If you need channel-specific eligibility or consent enforcement, use destination rules.
- If you need lift measurement, use an experiment.
Start with one or two campaign patterns you care about most instead of designing a universal framework upfront.
| If you need to... | Use this feature | Why |
| ----------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Run a timed, multi-step campaign with delays, branches, exits, and sends | [Journey](/customer-studio/journeys) | Journeys manage campaign timing and progression across multiple steps. |
| Decide which campaign should win when a user qualifies for multiple competing audiences | [Priority list](/customer-studio/priority-lists) or [journey exclusions](/customer-studio/journeys#step-2-configure-the-start-tile) | Priority lists rank competing audiences at activation time. Journey exclusions prevent simultaneous participation across journeys. |
| Enforce consent, privacy, or destination-specific eligibility rules | [Destination rules](/customer-studio/destination-rules) | Destination rules filter records at sync time without changing campaign eligibility or audience membership. |
| Measure whether a campaign treatment produced incremental lift | [Experiment](/campaign-intelligence/experiments) | Experiments split eligible users into holdout and treatment groups and compare outcomes. |
| Reuse a standard destination setup across campaigns | [Sync template](/customer-studio/templates#sync-templates) | Sync templates store reusable destination mappings and settings. |
| Reuse a proven campaign flow across teams | [Journey template](/customer-studio/templates#journey-templates) | Journey templates store reusable workflow structure and configuration. |
| Decide whether campaign logic belongs upstream or in Customer Studio | [Plan your data model](/customer-studio/data-model) | Put reusable business logic and core metrics upstream. Use Customer Studio traits for activation-specific logic that marketers need in audiences. |
---
## When to use journeys
Not every campaign needs a journey. Many campaign use cases can be handled with a single audience synced on a schedule.
**Use a journey when:**
- The campaign has multiple steps with timing between them
- Users should follow different paths based on behavior or attributes
- You need in-flow experimentation with A/B Split tiles
- Users should exit early when they complete a goal or become ineligible
- You need to persist state or variables through the flow
**Use an audience sync alone when:**
- The campaign is a single activation step
- There is no sequencing, branching, or delay logic
- Audience membership already expresses all required logic
- You want the simplest possible campaign setup
**Rule of thumb:** if the campaign has one action and no timing logic, start with an audience sync. If it has multiple steps or decision points, start with a journey.
---
## Manage overlap across campaigns
When you run multiple campaigns at once, the same user can qualify for more than one. Without explicit overlap controls, users can enter competing campaigns or receive conflicting downstream messages.
### Priority lists vs. journey exclusions
| Mechanism | How it works | Best for |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Priority list** | Ranks overlapping audiences at sync time. Each user is assigned to the highest-priority audience they qualify for. | Choosing which campaign should activate the user when multiple campaign audiences qualify. |
| **Journey exclusions** | Prevents a user from entering a journey if they are already in another specified journey. | Preventing concurrent journey participation across campaigns. |
Use **priority lists** to rank competing campaign audiences. Use **journey exclusions** to prevent simultaneous journey enrollment. In more complex setups, use both.
### Stagger campaign schedules
If two campaigns evaluate on similar cadences, offset their run times. That reduces the chance that both process the same user before exclusion logic takes effect.
### Use a shared campaign-state trait
For more advanced coordination, maintain a trait such as `current_lifecycle_stage` upstream. Use it across audiences, journeys, and syncs so every campaign checks the same state signal.
---
## Apply consent and access rules at sync time
Destination rules, row-level access, and consent enforcement all apply **after** audience qualification and journey evaluation. That means a user can qualify for a campaign and appear in journey counts, but still be filtered out before delivery.
### Destination rules
[Destination rules](/customer-studio/destination-rules) add sync-time filtering for a specific destination. They do not change audience membership.
Use destination rules when:
- Consent differs by channel
- Destination-specific business rules apply
- The campaign audience is correct, but not every eligible member should be sent to that destination
Destination rules do not affect journey logic. Journey counts can look correct while destination delivery counts are lower because sync-time rules filtered users out.
### Row-level access and consent
[Row-level access](/workspace-management/row-level-access) controls which records a team can view and activate. [OneTrust integration](/customer-studio/onetrust) enforces consent before data leaves the warehouse.
When counts look lower than expected, check these layers in order:
1. **Audience definition**
2. **Priority list behavior**
3. **Destination rules**
4. **Row-level access**
5. **Consent enforcement**
---
## Validate each layer before launch
If a campaign uses multiple Customer Studio features together, validate each layer independently instead of assuming the full workflow is correct.
### Pre-launch checklist
1. **Audience:** Preview membership and confirm the right users qualify.
2. **Priority lists:** Confirm ranking if the campaign audience belongs to a priority list.
3. **Journey logic:** Run a [simulation](/customer-studio/journeys#run-a-simulation) to validate branches and conditions.
4. **Sync configuration:** Run a [test sync](/customer-studio/journeys#run-a-test-sync) to validate mappings and payload structure.
5. **Destination rules:** Review any sync-time filters on the destination.
6. **Live validation:** Start with a small test audience and monitor the first runs end to end.
### Debugging count mismatches
When journey counts and destination counts do not match, the drop usually happens in a layer between campaign qualification and delivery.
| Layer | What can reduce counts | How to check |
| -------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------- |
| Priority list | A higher-priority audience claims the user | Review audience ranking and overlapping audience membership |
| Destination rules | Sync-time filtering removes the user | Review destination-specific rules |
| Row-level access | The team cannot access the record | Review team access configuration |
| Consent | The user does not have valid consent | Review consent status in the warehouse or OneTrust |
| Destination platform | The destination applies its own suppression or deduplication | Review destination delivery logs |
---
## Campaign patterns
These examples show how different Customer Studio features can work together in campaign workflows.
### Staged lifecycle campaigns with overlap control
**Scenario:** Run welcome, active-user, re-engagement, and winback campaigns without conflicts.
**How to build it:**
- **Priority list** ranks campaign audiences
- **Separate journeys** manage timing and messaging for each campaign
- **Journey exclusions** prevent concurrent participation
- **Shared state trait** coordinates campaign stage across workflows
### Channel routing with consent enforcement
**Scenario:** Deliver a campaign through email, SMS, or push based on channel eligibility and consent.
**How to build it:**
- **Journey** controls campaign timing and branching
- **Audience** defines campaign eligibility
- **Destination rules** enforce per-channel consent and delivery rules
- **Experiment** measures whether channel routing improves outcomes
This separates campaign eligibility, campaign timing, delivery rules, and measurement into the right layers.
### Campaign measurement with holdout
**Scenario:** Measure the incremental lift of a re-engagement campaign.
**How to build it:**
- **Audience** defines the eligible campaign population
- **Experiment** creates treatment and holdout groups
- **Journey** runs the treatment flow
- **Intelligence** measures outcome differences between groups
---
## Avoid common setup mistakes
- **Putting consent logic inside journey branches.** Consent should usually be enforced at sync time through destination rules.
- **Using journey exclusions as the only overlap mechanism.** Exclusions prevent simultaneous journey entry, but they do not rank competing audiences.
- **Duplicating audience logic in destination rules.** Put shared campaign eligibility in the audience. Reserve destination rules for destination-specific logic.
- **Editing live journeys instead of replacing them.** Large structural changes can affect users already in progress.
- **Skipping priority-list validation.** A higher-priority audience can claim users before a campaign starts.
- **Testing only the journey.** Simulation does not model destination rules, consent filtering, or priority-list behavior.
---
## What's next
- [**Journeys**](/customer-studio/journeys) — Build and run multi-step campaign flows.
- [**Priority lists**](/customer-studio/priority-lists) — Control which campaign audience activates each user.
- [**Destination rules**](/customer-studio/destination-rules) — Apply sync-time delivery rules by destination.
- [**Experiments**](/campaign-intelligence/experiments) — Measure campaign lift with treatment and holdout groups.
- [**Plan your data model**](/customer-studio/data-model) — Decide which campaign logic belongs upstream versus in Customer Studio.
---
## Catalog recommendations
**URL:** https://hightouch.com/docs/customer-studio/catalog-recommendations
**Description:** Generate personalized product recommendations for every user from any item catalog in your data warehouse.
**Section:** Customer Studio
| | |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers and data teams |
| **Prerequisites** |
A defined [schema](/customer-studio/schema) with at least one parent model (for example, `Users`).
A catalog of items (products, offers, digital content, etc.) represented as a table in your warehouse.
Event data linking users to catalog items (for example, purchases, add-to-cart events, email clicks).
|
_Catalog recommendations use machine learning to generate the top recommended items from any catalog for each user, so you can personalize syncs and audience targeting at scale._
---
## What you'll learn
- [Understand how catalog recommendations work](#overview)
- [Set up a catalog in your schema](#set-up-a-catalog)
- [Create a catalog recommendation trait](#create-a-catalog-recommendation)
- [Evaluate recommendation quality](#evaluate-recommendation-quality)
- [Use recommendations in syncs and audiences](#use-catalog-recommendations)
---
## Overview
Standard traits like counts and aggregations tell you what users have done. Catalog recommendations go further by predicting which items each user is most likely to engage with next.
Given a catalog of any size — hundreds to hundreds of thousands of items — Hightouch trains a model on your historical interaction data and generates a ranked list of the top 20 recommended items per user. You can then sync those recommendations to any destination or use them to build highly targeted audiences.
Catalog recommendations are part of the [predictive traits](/customer-studio/predictive-traits) suite and share the same ML infrastructure.
### How it works
Hightouch uses three inputs to train the recommendation model:
- **Users** — the parent model in your schema (for example, `Users`).
- **Catalog** — a table of items in your warehouse (for example, `Products`, `Offers`, `Listings`).
- **Events** — interactions between users and catalog items (for example, purchases, add-to-cart events, page views, email clicks).
Each time the model runs, Hightouch pulls training data from your warehouse, trains the model, and writes recommendations back. The model generates a ranked list of the top 20 items for each user based on their predicted propensity toward each item.
### Use cases
- **Personalized product syncs** — Sync each user's top recommended products to an ESP or ad platform by mapping the catalog recommendation trait as a sync field, then use the ranked product array to populate personalized email templates or dynamic ad creative.
- **Targeted audience building** — Create an audience in Customer Studio filtered by catalog recommendation trait (for example, users whose top five include a specific product), then sync that audience to an ads platform to run a targeted launch campaign for that product line.
## Set up a catalog
Before you create a catalog recommendation trait, you need a catalog configured for recommendations in your schema.
1. Go to **Customer Studio → Schema**.
2. Add or select the table that represents your catalog (for example, `Products`). The catalog must be connected to a parent model through event or relationship tables.
3. On the catalog model's detail page, go to the **Recommendations** tab and enable **catalog recommendations**.
4. Select the **parent model** that represents your users.
5. Define the **key events** that capture how users interact with catalog items (for example, `AddToCart`, `Purchase`, `EmailClick`). These events are what the model uses to learn user preferences.
You can use multiple event types to train a single recommendation model. For
example, combining purchase events with add-to-cart events gives the model
more signal about user preferences.
## Create a catalog recommendation
To create a catalog recommendation:
1. Go to **Customer Studio → Traits**.
2. Click **Create → New trait**.
3. In the **Method** step, select a **parent model** (for example, `Users`) that has a relationship with the catalog you want to generate recommendations for.
4. Under **Calculation method**, select **Catalog Recommendation**.
5. Click **Continue**.
6. In the **Calculation** step, select the **catalog** you want to generate recommendations from. The remaining settings on this page are optional.
### Filter the catalog
You can optionally add filters to specify which items to include from your catalog. Filters use columns from the catalog table.
For example, if your catalog contains all products but you only want to recommend shoes, you can add a filter where the product name contains "shoes" or the category equals "footwear."
### Filter eligible users
Under **Who should this prediction apply to?**, you can filter which users receive recommendations. By default, every user in your parent model is scored. To narrow the pool, add filters based on something users have already done — for example, signing up, viewing a page, or starting checkout.
### Set a suppression filter
Optionally, enable the **suppression filter** to suppress recommendations for items that a user has already interacted with. When enabled, you select the event that represents the interaction (for example, a purchase event), and the model excludes those items from that user's recommendations.
### Set the update schedule
Under **How often should scores be updated?**, choose how often the model should retrain and refresh recommendations. Each run uses warehouse compute to pull data and write results, so the schedule affects warehouse usage. The model generates 20 recommended items per user on each run.
More frequent updates keep recommendations fresh but increase warehouse usage.
Less frequent updates are appropriate when your catalog and user behavior
change slowly.
After configuring the calculation, click **Continue to finalize**. In the **Finalize** step, name your trait, optionally add a description, review the calculation summary, and click **Create trait**.
The model begins training immediately. Training time varies based on the size of your data.
## Evaluate recommendation quality
After the model completes training, the **Prediction analysis** tab on the trait detail page displays performance metrics. The page shows the total number of users and items evaluated, along with the last run date.
### Hit Rate@20
Hit Rate@20 measures the fraction of users where at least one relevant item appeared in the top 20 recommendations. A higher hit rate is better. The analysis page also shows a **Model vs Random Baseline** comparison that visualizes the model's hit rate against what you'd expect from randomly assigning items — this lift metric helps you gauge how much value the model adds.
### NDCG@20
NDCG (Normalized Discounted Cumulative Gain) at 20 measures recommendation ranking quality, accounting for position. Values closer to 1 indicate better ranking — meaning items the user is most likely to engage with are placed closer to the top of the list.
### Average position of first hit
Among users who received a relevant recommendation, this metric shows the average rank position where the first relevant item appears. A lower number means the model tends to place relevant items near the top of the list.
## Use catalog recommendations
Once a catalog recommendation trait is created and trained, you can use it in two ways.
### Sync recommendations to a destination
When you set up a sync, you can select the catalog recommendation trait as a field to include. Recommendations are synced as an array of objects, where each object represents a catalog item. This lets you power personalized content in any downstream tool — email templates, push notifications, ad creative, and more.
1. Go to **Syncs** and create or edit a sync.
2. In the field mapping step, select your catalog recommendation trait.
3. The recommendation data syncs as a structured array to your destination.
### Build audiences from recommendations
You can use catalog recommendations to define audience membership. For example, to target users who are predicted to be interested in a specific product:
1. Go to **Customer Studio → Audiences** and create a new audience.
2. Add a filter using your catalog recommendation trait.
3. Set a percentile range or filter by specific items to define which users qualify for the audience.
This lets you build launch audiences, cross-sell segments, or personalized campaign groups based on predicted product affinity rather than past behavior alone.
## Related articles
- [Predictive traits](/customer-studio/predictive-traits)
- [Lookalike audiences](/customer-studio/lookalike-audiences)
- [Traits](/customer-studio/traits)
- [Audiences](/customer-studio/usage)
- [AI Decisioning](/ai-decisioning/overview)
---
## Coupon assignment
**URL:** https://hightouch.com/docs/customer-studio/coupons
**Description:** Use coupon assignment to attach unique coupons (and related data) to users.
**Section:** Customer Studio
## Overview
Coupon assignment lets you attach per-user values (such as coupon codes or offer metadata) after defining an audience and before sending data to your destinations.
This is useful when:
- you need to assign **unique values per user** (e.g. single-use coupons)
- values come from a **shared dataset** (e.g. a coupon table)
- you shouldn't assign a coupon to a user that **isn't in your campaign**
---
## How it works
Coupon assignment happens inside an **audience-based Journey**. For each user, Hightouch assigns a value from a dataset and makes it available for downstream syncs.
---
## Example
- Parent model: **10,000 users**
- Campaign A audience: **500 users**
- Coupon pool: **500+ unique codes**
---
## Step 1: Create a coupon pool
Create a table in your warehouse that contains your coupon data.
Include:
- a coupon value column (e.g. `coupon_code`)
- optional metadata (e.g. `expiration_date`)
- a column filter on if you have cross-campaign coupons consolidated in one model (e.g. `campaign_id`)
You can set it up in your Customer Studio schema with a `coupons` tag.
---
## Step 2: Create an audience and Journey
Build an audience for your campaign (e.g. **Campaign A**) and create a Journey with:
> User is in Audience: Campaign A
Only users in this audience will receive a coupon.
Event based journeys are not supported.
---
## Step 3: Add a coupon assignment node
In the **Assign coupon** node:
- select your coupon dataset
- apply filters (e.g. `campaign_id = 10_dollars_off`)
- choose an "assignment" columns (e.g. `coupon_code`). This is the column that will be unique (1:1) to a user.
- choose any additional columns to assign (e.g. `expiration_date`, `coupon_threshold`). These do not need to be unique to the user.
- define variable names
For each user:
1. **Reuse existing assignment** (if present)
2. **Assign a new coupon** (if not)
3. **Store the result** in:
- Journey variables (for syncs)
- Journey logs
Assignments are deterministic per coupon journey node: the same user will receive the same values when entering the same journey node.
---
## Step 4: Sync assigned values
Use the assigned variables in a Sync node (e.g. to Braze or Salesforce). If you do not see the assigned variables in the Sync node, make sure to save the Journey first.
---
## Tracking
Assignments are recorded in Journey logs and can be used to:
- map coupons to users
- join with downstream usage or redemption data
---
## Behavior and constraints
- **Inventory required:** If coupons < audience size, the Journey fails before sending.
- **Consistent assignment**: Even if the underlying coupon pool changes, what coupon and coupon metadata a user is assigned stays the same if they visit the same coupon assignment node again. If you would like the user to be assigned a different value, use a different coupon assignment node.
---
## Data masking
**URL:** https://hightouch.com/docs/customer-studio/data-masking
**Description:** Data masking capabilities to help protect sensitive information, including personally identifiable information.
**Section:** Customer Studio
| | |
| ----------------- | -------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Platform admins |
| **Prerequisites** | [Set up for platform admins and data teams →](/customer-studio/quick-start#setup-for-platform-admins-and-data-teams) |
_Protect personally identifiable information by automatically detecting sensitive data, and manually reviewing and controlling access as needed._
## Overview
Data masking automatically detects and allows you to take action towards protecting sensitive data in your datasets. Hightouch’s data masking feature helps you classify columns, enforce privacy controls, and maintain compliance while preserving data utility.
## Privacy levels
Data masking uses a multi-level privacy system:
**Approved**: Data cleared for unrestricted use.
**Blocked**: Data completely restricted from access and syncs.
**Redacted**: Data hidden in profile details, column suggestions, and analytics. Remains accessible for syncs and audience filters.
**Sync-only**: Data can sync to destinations but is not visible in the Hightouch app outside of syncs.
**Sync-blocked**: Data can be used in the Hightouch app but cannot be synced.
**Unreviewed**: Data not yet reviewed.
Unreviewed data continues to function normally until you assign a privacy level.
## Automatic PII detection
The system automatically flags potential personally identifiable information for review. Auto-flagging only applies to non-reviewed columns and will not override your existing classifications.
## Managing columns
### View and search columns
Access the data masking interface from the [**Governance**](https://app.hightouch.com/governance/?from=docs) tab. The table view shows all columns sorted by most recently updated.
Use the search bar to filter columns by:
- Model
- Source
- Privacy level
- Flagged status
### Check auto-detected columns
Columns flagged as potential PII display a **warning badge**. You can filter for flagged columns, review them, and apply the correct privacy level.
### Set privacy level
You can set privacy levels individually or in bulk:
1. Select one or more columns.
2. Choose a privacy level from the dropdown:
1. **Blocked**: For highly sensitive data that must never sync downstream.
2. **Redacted**: For columns where data can be synced but values must stay hidden in the Hightouch app.
3. **Sync-only**: For columns where data should be synced to destinations but is not visible in the Hightouch app outside of syncs.
4. **Sync-blocked**: For columns where data can be used in the Hightouch app but cannot be synced.
5. **Approved**: For data cleared through your review process.
3. Apply the privacy level.
Once applied, the system removes auto-detection flags for those columns.
### Categorize columns
To improve management and filtering, assign categories to your columns. For example:
- Email
- Name
- Birthdate
- Address
- Phone
- IP address
## Permissions
Access to data masking is controlled by **schema permissions**. To edit privacy levels and PII categories, users must have `Configure schema` access on the source.
See [Source-level permissions](/workspace-management/roles#source) for more details.
---
## Plan your data model
**URL:** https://hightouch.com/docs/customer-studio/data-model
**Description:** Choose the right grain, model topology, naming conventions, and data types before configuring your Customer Studio schema.
**Section:** Customer Studio
| | |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Audience** | Data or analytics engineers, platform admins |
| **Prerequisites** |
A connected data source (e.g., Snowflake, BigQuery, Databricks)
Technical understanding of your data and its relationships
|
_Before configuring your schema in Customer Studio, plan which data to expose and how to structure it. The decisions on this page determine what marketers can target, filter on, and sync to destinations._
---
## What you'll learn
- [Choose the right grain](#choose-the-right-grain) for your parent model
- [Decide where data belongs](#choose-the-right-model-type): parent model, related model, event model, trait, or merge column
- [Decide when a value should be a trait](#when-to-model-something-as-a-trait) vs. upstream logic
- [Structure a starter schema](#start-with-a-recommended-shape) and expand it over time
- [Name models and columns](#name-models-and-columns-for-marketers) so marketers can self-serve
- [Avoid common anti-patterns](#avoid-common-anti-patterns) that lead to confusing audience behavior
---
## Overview
The schema is the contract between data teams and marketers. It determines what data marketers can access in the audience builder, what they can filter on, and what they can sync to destinations. A clean, well-named schema makes Customer Studio self-serve. A messy one turns every audience into a data-team request.
In Customer Studio, you'll define three types of models:
- **Parent model**: The base entity audiences are built from (for example, `Users`)
- **Related models**: Additional context joined to the parent (for example, `Households`)
- **Event models**: Timestamped behaviors (for example, `Product Viewed`)
Together, these models and their relationships form your data schema — a map of what marketers can filter on when building audiences.
---
## Coordinate with business teams
Before building your schema, align with marketers or campaign owners on what they need to target and filter on. Start with one or two high-priority use cases rather than designing for everything at once. Good first use cases include:
- Suppression audiences (for example, exclude recent purchasers from ad spend)
- Re-engagement windows (for example, users inactive for 30+ days)
- Lifecycle stage audiences (for example, trial users who haven't activated)
Use these use cases to decide which tables to expose, which relationships to define, and how to name models and columns so marketers can find what they need without asking for help.
---
## Choose the right grain
Your parent model's **grain** — what one row represents — is your targeting grain. One row in the parent model equals one entity that can appear in an audience, be deduplicated, and be synced to a destination.
Before choosing a table, decide what one row should represent. Common choices include individual users, accounts, households, or devices. This choice shapes everything downstream:
- Audience membership is evaluated per parent row
- Sync output contains one record per qualifying parent row
- Traits and filters resolve back to the parent grain
If you set the parent model at the account grain, you can target accounts but not individual users within them. If you need both, you'll need separate parent models or a user-level grain with account data joined as a related model.
### When to use multiple parent models
Some businesses need to target different entities for different use cases. A B2B company might target individual users for product adoption campaigns and accounts for renewal outreach. A marketplace might target buyers and sellers separately.
The rule of thumb: the parent model represents who you're communicating with. If that varies by use case, create separate parent models — each with its own related models, events, and traits. Each parent model powers its own independent set of audiences.
The grain doesn't always map to a person. For example, an airline that upsells passengers for each upcoming flight might set its parent model to one row per traveler per flight — not one row per traveler. That way, each flight booking gets its own campaign, and marketers can target the December trip and the February trip independently.
Common scenarios that call for multiple parent models:
- B2B companies targeting both users and accounts
- Marketplaces with distinct buyer and seller audiences
- Products with both end users and admin contacts
Start with one parent model for your highest-priority use case. Add a second only when you have a concrete campaign that requires a different targeting entity. Avoid creating extra parent models just to expose more fields — in most cases, the better solution is a related model, a through relationship, or a trait. More parent models create more audience entry points, more duplicated configuration, and more opportunities for marketers to choose the wrong one.
If a related model or event table connects to multiple parent models, share it via relationships instead of duplicating it.
---
## Choose the right model type
Not all data belongs in the same place. Use this table to decide where a piece of data should live in your schema:
| If you need to... | Use a... | Example |
| :---------------------------------------------------- | :----------------------------------------------------------------------- | :--------------------------------------------------------- |
| Target an entity directly | [Parent model](/customer-studio/schema#define-the-parent-model) column | `region`, `signup_date`, `plan_tier` |
| Filter on repeated records linked to the parent | [Related model](/customer-studio/schema#add-related-models) | Purchases, subscriptions, support tickets |
| Filter on timestamped behaviors | [Event model](/customer-studio/schema#add-event-models) | Page views, logins, cart additions |
| Collapse many rows into one value per parent row | [Trait](/customer-studio/traits) | Total spend, days since last login, most frequent category |
| Access a parent field inside a related or event model | [Merge column](/customer-studio/schema#merge-columns-across-models) | Show `region` on purchase records |
Sync field mappings draw from parent model columns, merge columns, and traits. Data on related and event models is available for audience filtering but does not appear directly in sync output. To include a value from a related model in a sync, create a [trait](/customer-studio/traits) that rolls it up to the parent or add it as a [merge column](/customer-studio/schema#merge-columns-across-models).
If a marketer asks "why can't I use this field in my sync mapping?" the answer
is almost always about model topology. Fields on related or event models are
available for audience filtering, but sync field mappings draw from parent
model columns, merge columns, and traits. To make a related-model field
syncable, either add it as a merge column, create a trait that rolls it up to
the parent, or restructure upstream so the value lives on the parent model
directly.
---
## When to model something as a trait
The decision table above includes traits as an option for collapsing many rows into one value per parent row. Use a [trait](/customer-studio/traits) when the value is a reusable derived attribute that marketers will filter on repeatedly — lifetime value, days since last purchase, total number of orders, or most frequent product category.
Traits are the right choice when:
- The value doesn't exist as a column on the parent model and needs to be computed from related or event data
- Multiple audiences will use the same calculation, so you want to define it once
- Marketers need the value in sync field mappings (traits on the parent model are syncable; related/event fields are not)
Keep the logic upstream instead when:
- The calculation defines a core business metric your team governs centrally (for example, official LTV or churn score)
- The computation is expensive and benefits from materialization in the warehouse
- Multiple teams outside Customer Studio need the same value
For details on creating and managing traits, see [Traits](/customer-studio/traits).
---
## Start with a recommended shape
You don't need to model your entire warehouse on day one. A good starting point:
- 1 parent model (for example, `Users`) with common attributes flattened onto it (email, region, plan, signup date)
- 2–3 high-value related models (for example, Purchases, Subscriptions)
- 1–2 event tables (for example, Page Views, Logins)
Expand the schema as use cases grow. It's easier to add models than to clean up a schema that exposes too much too early.
---
## Name models and columns for marketers
Marketers interact with the schema through the audience builder. Names should be immediately understandable without warehouse context:
- Use plain names: `Signup date` not `created_at`, `Region` not `geo_region`
- Name models after what they represent, not where they come from: `Users` not `braze_user_attributes`, `Purchases` not `fact_order_items`
- Use [column aliases](/customer-studio/schema#rename-columns-using-aliases) to rename technical column names without changing your warehouse
- Disable columns that add noise (internal IDs, ETL metadata, debugging fields)
---
## Keep data types consistent
Mismatched data types between models cause relationship errors and confusing audience results:
- Keep primary key types consistent across models — an `integer` primary key on the parent and a `bigint` foreign key on a related model can cause join failures
- Store timestamps as UTC timestamp or datetime types, not strings
- Cast nested or JSON fields upstream where practical — the audience builder works best with flat, typed columns
---
## Decide what belongs upstream vs. in Hightouch
If logic is business-critical, reusable, or defines a core metric, put it upstream in your warehouse (as a materialized view, dbt model, or table). Hightouch traits and computed columns work well for activation-specific calculations, but they are not a substitute for core data modeling.
### Build a presentation layer first
The most successful Customer Studio implementations start with a stable, marketer-friendly schema layer in the warehouse — a set of pre-joined, pre-aggregated models (for example, dbt marts) that are purpose-built for activation. This layer should flatten heavy joins, materialize expensive logic, and present clean, well-named columns so Customer Studio doesn't have to recompute them at query time.
As a rule of thumb: flatten and materialize heavy joins upstream. Large event tables should be partitioned or clustered by timestamp. The fewer expensive joins Customer Studio has to run at query time, the faster audience previews and syncs will be.
### Keep parent models lean
Expose only the columns marketers need for filtering and sync field mappings. Move less-frequently used attributes into related models. A lean parent model makes the audience builder faster and easier to navigate.
### Handle JSON and nested fields upstream
JSON and nested fields are supported in Customer Studio, but they work best when cast to flat, typed columns in your warehouse presentation layer. If you need to expose nested data, be explicit about the expected shape and document the structure for marketers.
---
## Avoid common anti-patterns
- **Wrong grain on the parent model.** Setting the parent at household or account grain when you need to target individuals leads to audience count confusion and inability to personalize per user.
- **Exposing too much of the warehouse.** Only expose the tables and columns marketers actually need. A schema with 50 models and hundreds of columns makes the audience builder harder to use, not easier.
- **Naming models after their sync destination.** A parent model named `Braze user attributes` makes sense for one sync but confuses everyone when you add more destinations. Name models after the entity they represent.
- **Skipping join key validation.** Test your relationships with a small audience before building production campaigns. A misconfigured join key silently produces wrong audience counts.
---
## What's next?
Once you've planned your data model, configure it in Customer Studio:
- [**Define your schema**](/customer-studio/schema)
Create parent, related, and event models, configure columns, and set up relationships.
- [**Create traits**](/customer-studio/traits)
Build reusable calculated values marketers can use in audience filters and sync field mappings.
---
## Destination rules
**URL:** https://hightouch.com/docs/customer-studio/destination-rules
**Description:** Destination rules allow organizations to filter out users from destinations they have not consented to during during syncing.
**Section:** Customer Studio
| | |
| ----------------- | -------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Platform admins |
| **Prerequisites** | [Set up for platform admins and data teams →](/customer-studio/quick-start#setup-for-platform-admins-and-data-teams) |
_Destination rules allow organizations to enforce row-level governance by filtering out users from being synced to destinations they haven’t consented to. This ensures that only eligible data reaches specific destinations based on compliance, privacy, or business rules._
---
## What you'll learn
- After reading this article, you’ll be able to:
- Understand what destination rules are and when to use them
- Create and configure destination rules in the UI
- Preview the impact of a rule on an audience
- Apply rules automatically to syncs without modifying audience definitions
---
## Overview
Destination rules control which records can be synced to specific destinations. They apply additional filtering on top of the audience definition at **sync time**, without affecting the actual audience membership.
A common use case is enforcing **user consent**: for example, excluding users who haven’t opted in to email or advertising platforms.
{" "}
Destination rules do not change your audience composition. They only affect **who
gets synced** to a destination, not who qualifies for the audience.{" "}
## Set up destination rules
1. Go to **Customer Studio > Governance**, then click the [Destination rules tab](https://app.hightouch.com/governance/destination-rules?from=docs)
2. Select a **[parent model](/customer-studio/schema)**
3. Click **Add destination rule**
3. Select a **destination** from the dropdown
4. Click **Add filter** to define which records are eligible to sync
The filter builder supports the same condition logic used in the [audience builder](/customer-studio/usage), including nested `AND/OR` logic.
## Preview destination rules
You can preview the effect of a destination rule in two places:
- During **audience creation**
- From the audience’s **definition tab** (when editing an existing audience)
In both cases:
1. Use the `Preview a destination` dropdown to select a destination with an existing rule.
2. Hightouch applies the rule to your audience definition and updates the preview.
3. You’ll see how the rule impacts calculated size and membership before syncing.
This preview does **not** change the audience itself—it only simulates how the rule would affect syncing to the selected destination.
## Using destination rules in syncs
Create a sync with a parent model (proxied by the audience) and destination that has an existing rule.
Rows that don't meet a destination rule will be excluded at sync time. All syncs going forward will automatically have the rule applied.
Note that these excluded rows are still included in the audience definition, so they remain unaffected when using Customer Studio features that don't involve syncing.
---
## Experiments
**URL:** https://hightouch.com/docs/customer-studio/experiments
**Description:** Audience Experiments unlock seamless management of A/B and multivariate testing efforts by allowing users to create randomized test groups within audiences in Hightouch.
**Section:** Customer Studio
**Experiments** were previously called **Splits**.
| | |
| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers and analysts who want to compare the effectiveness of different campaign strategies using real audience data. |
| **Prerequisites** |
A saved [audience](/customer-studio/usage)
At least one [sync](/customer-studio/syncs) or campaign ready to launch
|
_Run controlled experiments directly in Customer Studio by creating randomized groups to test and measure the true impact of your marketing campaigns._
---
## What you'll learn
- [Understand when and why to use audience experiments](#when-to-use-experiments)
- [Create A/B or multivariate experiment groups within a saved audience](#1-create-experiment-groups)
- [Sync each group to separate treatments, campaigns, or destinations](#1-create-experiment-groups)
- [Measure campaign effectiveness using Experiments in Campaign Intelligence](#2-measure-results)
- [Use stratified sampling to ensure balanced test and control groups](#stratified-sampling)
- [Access holdout group logs for detailed post-campaign reporting in your warehouse](#holdout-group-logs)
---
## Overview
Audience **Experiments** enable you to create randomized audience groups for accurate, controlled testing of campaign strategies. By assigning audience members to distinct test groups, you can compare outcomes between treatments, channels, or creatives and understand what’s truly driving performance.
You can also configure experiments directly on **A/B split nodes** within
[Journeys](/customer-studio/journeys#ab-split).
---
### What you can measure
- **Incrementality**: Did your campaign actually cause the desired outcome?
- **A/B/n testing**: Compare creatives, messages, or channels.
- **Personalization impact**: Determine whether tailored experiences outperform generic ones.
- **Channel effectiveness**: Identify which channels drive more downstream conversions.
---
### When to use experiments
Use audience experiments when you want to move beyond assumptions like:
- **Correlation**: “Purchases increased—but was it because of the campaign?”
- **Attribution**: “A user converted after seeing my ad—but would they have converted anyway?”
Running a controlled experiment isolates the impact of your campaign by comparing outcomes between randomized groups.
---
## How audience experiments work
| Stage | What it means | Hightouch feature |
| --------------------------- | ------------------------------------------ | ------------------------ |
| 1. Define your audience | Choose your target group | Build audiences |
| 2. Create experiment groups | Randomly assign audience members | Experiments |
| 3. Apply a treatment | Run a campaign for one or more groups | Syncs |
| 4. Measure results | Compare outcomes between groups | Experiments |
| 5. Interpret the impact | Evaluate lift and statistical significance | Experiment results chart |
---
## Example use cases
#### Email campaign holdout
- 50/50 experiment
- Group A receives promotional email
- Group B is held out
- Compare conversions or revenue per user
#### Ad creative A/B test
- Groups A and B synced to separate ad sets
- Compare CTR, CPC, or downstream conversion
#### Personalization test
- Group A receives personalized product recommendations
- Group B receives a generic message
- Compare engagement and purchases
#### Onboarding lifecycle experiment
- Test different onboarding flows
- Measure feature adoption or retention after 14 days
#### Channel comparison
- Group A receives email
- Group B receives paid ads
- Compare downstream LTV or sign-up quality
---
## Setup steps
### 1. Create experiment groups
1. Open your audience in [**Customer Studio**](https://app.hightouch.com/audiences?from=docs).
2. Go to the **`Experiments`** tab.
3. Toggle **`Enable experiment groups`**.
4. Under **`Experiment groups`**, configure:
- Number of groups (e.g., 2 for A/B tests, 3+ for multivariate)
- Percentage distribution (e.g., 50/50)
- Group names (e.g., `Holdout`, `Treatment A`)
5. Optionally add more groups for A/B/n testing.
6. For each group, configure syncs to destinations. If you sync a single group to multiple destinations, all members of that group are sent to every destination — the group is not subdivided or split between them.
7. Click **`View results`** to begin analysis.
---
### 2. Measure results
Once your audience has at least one sync involving experiment groups, Hightouch automatically generates a corresponding **Experiment**.
Clicking **`View results`** opens the Experiment overview.
You can also access Experiments anytime under:
**`Intelligence` → [`Experiments`](https://app.hightouch.com/experiments?from=docs)`**
Use Experiments to:
- Compare lift across groups
- View statistical significance
- Analyze results using primary or secondary metrics
- Normalize results per member or against a baseline
Older *Splits measurement charts* have been deprecated. All measurement now
occurs in **Experiments**.
[**Learn how to configure and interpret Experiments**](/campaign-intelligence/experiments)
---
## Stratified Sampling
Simple randomization may not provide balanced groups when your audience contains diverse characteristics (e.g., geography, loyalty tier). **Stratified sampling** ensures proportional representation across experiment groups based on selected variables.
Stratified sampling is only available for one-time syncs.
### How to enable stratified sampling
1. Open your audience and go to the **`Experiments`** tab.
2. Under **`Advanced configuration`**, toggle **`Stratified sampling`**.
3. Select one or more columns to stratify by (e.g., `loyalty_tier`).
4. Save your changes.
---
## Holdout group logs
Holdout group logs are used primarily by **data teams** to perform row-level
warehouse analysis. Marketers should rely on Experiment results in the
Customer Studio UI.
Holdout group logs track which rows were **excluded** from a sync as part of a holdout (e.g., a 20% control group). This enables advanced SQL or BI analysis.
### How to enable holdout group logs
1. to enable the *Holdout group logs* feature flag. Requirements: - [Lightning
sync engine](/syncs/lightning-sync-engine) - A destination with **write access**
to your warehouse{" "}
2. In your workspace:
- Go to **`Integrations → Sources`**
- Select a source and open the **`Sync Logs`** tab
- Enable **`Audience holdout group logs`**
When enabled, Hightouch logs excluded rows in the `hightouch_audit.audience_holdout` table.
### Table definition
| COLUMN | DESCRIPTION |
| ------------- | -------------------------------- |
| `sync_id` | Sync ID |
| `sync_run_id` | Sync run ID |
| `model_id` | Model/audience ID |
| `timestamp` | Timestamp of sync |
| `row_id` | Primary key value from the model |
| `fields` | JSON snapshot of model data |
| `split_group` | Name of the experiment group |
---
## General models
**URL:** https://hightouch.com/docs/customer-studio/general-models
**Description:** General models make warehouse tables available for agent queries without adding them to your Customer Studio schema.
**Section:** Customer Studio
| | |
| ----------------- | ------------------------------------------------------------------------------------------- |
| **Audience** | Data teams, platform admins |
| **Prerequisites** | A connected data [source](/sources/overview) (for example, Snowflake, BigQuery, Databricks) |
_General models let you make warehouse tables available for [agent](/agents/overview) queries without adding them to your Customer Studio schema. Use general models for data that agents should be able to query — such as sales figures, competitive datasets, or web analytics — but that doesn't belong in audience building or sync workflows._
---
## What you'll learn
- [Understand when to use general models vs. parent models](#when-to-use-general-models)
- [Add general models to your schema](#add-general-models)
- [Configure permissions and column access](#permissions-and-column-access)
- [Understand how agents query general models](#how-agents-use-general-models)
---
## Overview
As teams expand their use of agents beyond Customer Studio data, they often need agents to query warehouse tables that aren't suited to parent models. Examples include third-party sales data, product catalogs, web session logs, and competitive benchmarks.
General models solve this by providing a model type that shares governance and configuration features with parent models, but is scoped exclusively to agent queries. General models do not appear in the audience builder or sync flows.
### When to use general models
| Scenario | Model type |
| -------------------------------------------------------------------------- | ----------------- |
| Data should be available in the audience builder and sync flows | Parent model |
| Data provides context joined to a parent model (for example, `Households`) | Related model |
| Data represents timestamped user behaviors (for example, `Product Viewed`) | Event model |
| Data should be queryable by agents but not used in audiences or syncs | **General model** |
### How general models differ from parent models
General models share several characteristics with parent models: columns, configuration tabs, column access controls, and descriptions. However, there are key differences:
- **Primary key is optional.** Unlike parent models, general models do not require a primary key.
- **Bulk add from warehouse.** You can select multiple warehouse tables at once when creating general models.
- **Not available for audience building or sync configuration.** General models can be managed in schema and governance surfaces, but they do not appear as selectable models in the audience builder or sync flows.
- **Agent-focused model type.** General models are intended for agent queries rather than Customer Studio activation workflows.
---
## Add general models
### 1. Open the schema
Go to **Customer Studio → Schema**. Open the **General models** drawer.
### 2. Select tables
Click **Add general model**. In step 1 of the wizard, browse or search your warehouse tables and select one or more to add.
### 3. Add details
In step 2, name your tables and add descriptions to help the agent understand your data. Names and descriptions are optional but improve agent accuracy.
Click **Create models** to finish.
### 4. Configure each model
After adding, your general models appear in the drawer list.
Click into a general model to configure:
- **Columns** — review columns, add descriptions, and toggle column visibility for agents
- **Description** — add a plain-language description so agents can find and use the model effectively
Clear, descriptive model and column names help agents select the right data
when answering user questions. Adding descriptions is optional but improves
agent accuracy.
---
## Permissions and column access
### Model permissions
Adding, editing, and deleting general models requires the **Configure schema** grant on the source. General models reuse the existing model permissions managed through [user groups and roles](/workspace-management/roles). The relevant per-model permission is **View model data**. Per-model permissions for general models appear in the **Models** tab in the role configuration panel, alongside Customer Studio schema model permissions.
Agents can only query general models that the end user has permission to access. If a user's group does not have the **View model data** grant for a particular general model, agents cannot query it on that user's behalf.
### Column access
Each column has a privacy level that controls how agents can use it. You can block a column from agent queries entirely, redact values so agents can filter without seeing raw data, or fully approve the column for agent use. General models use the same underlying privacy system as other model types.
You configure privacy levels per column in the column detail view.
---
## How agents use general models
Agents can query any general model the user has permission to access. There is no explicit model selection required when starting a chat or creating a report — the agent automatically identifies which models are relevant to the user's question.
---
## Related articles
- [Define data schema](/customer-studio/schema)
- [Agents overview](/agents/overview)
- [Roles](/workspace-management/roles)
---
## Audience insights
**URL:** https://hightouch.com/docs/customer-studio/insights
**Description:** Audience insights give marketers deeper understanding of the composition of their audiences.
**Section:** Customer Studio
| | |
| ----------------- | -------------------------------------------- |
| **Audience** | Marketers and analytics/data engineers |
| **Prerequisites** | [Create audiences →](/customer-studio/usage) |
_Understand how to use Insights in Customer Studio to explore metrics, analyze trends, and power downstream targeting and enrichment workflows._
---
## What you'll learn
After reading this article, you will be able to:
- Calculate and interpret the size of an audience.
- Preview individual audience members before syncing.
- Explore audience overlaps to ensure clean campaign segmentation.
- Use audience breakdowns to understand the distribution of traits and metrics.
- Navigate and use the **Show insights** panel during audience definition.
- Explore members of your parent model outside of a specific audience.
---
## Overview
Understand and validate the size, composition, and characteristics of your audience using **Insights**. While defining or editing an audience in Customer Studio, you can view summary metrics, compare overlaps, and break down key traits inside the visual builder.
In addition to audience-level insights, Customer Studio includes [Campaign Intelligence](/campaign-intelligence/overview)—a suite of tools for deeper analysis across campaigns, including [insights charts](/campaign-intelligence/insights), [metrics](/campaign-intelligence/metrics), [funnels](/campaign-intelligence/funnels), and [experiments](/campaign-intelligence/experiments).
## Access audience insights
You can access insights while creating or editing an audience in two places:
- In the **audience creation flow**, during the `Define audience` step
- In the **audience editor**, under the `Definition` tab
Click the `Show insights` button in the top-right of the [filter builder](/customer-studio/usage#step-2-add-filters) toolbar.
## Calculate audience size
To see how many users match your audience filters, click `Calculate size`. The audience size is shown at the top of the builder and is updated when filters change or a sync occurs.
You can also calculate the size of specific **filter groups** by clicking `Calculate size` within any top-level `AND` block. This helps you understand how each filter group contributes to—or reduces—overall audience membership.
When [schema labeling](/customer-studio/schema#label-your-models) is enabled for a People ↔ Household or People ↔ Account relationship, you can view the size of both models.
For example, if your audience is based on a People model, you can also see how many Households or Accounts contain at least one person matching your filters.
If sampling is enabled on your data models, preview results in the audience
builder may be returned using sampled data for faster performance. [Learn more
about sampling.](/customer-studio/sampling)
## Member preview
Click the size preview count to more closely inspect the list of members who currently meet your audience criteria and ensure you're building the correct audience. If you've chosen to [redact any columns](/customer-studio/schema#set-privacy-levels-for-columns) in your model configuration, they are redacted here.
Clicking on a particular member will open the member details panel to give you an even deeper look at your audience members.
When [schema labeling](/customer-studio/schema#label-your-models) is enabled for a People ↔ Household or People ↔ Account relationship, you can view details for both models and see the membership between them.
For example, if your audience is based on a People model, you can view the Households or Accounts that contain at least one person matching your filters.
Within each person, you can see which households or accounts they belong to, and vice versa.
## Audience overlaps
Audience Overlaps can only be set up between audiences that use the same
[parent model](/customer-studio/schema#define-the-parent-model).
With many audiences syncing simultaneously, it can be difficult to ensure that end users aren't being targeted by multiple campaigns.
**Audience overlaps** help you validate the composition of your audiences by letting you compare one audience against another. You can find audience overlaps within the `Show insights` panel when defining your audience.
Select a previously created audience from the dropdown to compare the audiences.
For more information, check out the [**Audience Overlaps** blog post](https://hightouch.com/blog/feature-release-audience-overlaps).
## Audience breakdowns
**Breakdowns** help quickly check your audience segmentation and ensure the audience meets your needs by enabling you to better understand the distribution of specific characteristics within audiences.
Breakdowns help you explore how an audience is distributed across key traits or metrics—for example, how many users fall into each region, loyalty tier, or spending bucket.
To create a breakdown:
1. Open the `Show insights` panel.
2. Scroll to the **Breakdowns** tab.
3. Choose a metric or property, such as:
- **Saved metrics** (custom metrics from your workspace)
- **Events** (e.g., Bookings)
- **Relations** (e.g., Preferred Hotel)
- **Properties** (e.g., Region, Language)
- **Traits** (custom trait columns)
You can also **group by** a user property (e.g., group `Total Bookings` by `Region`) and choose an **aggregation type** (e.g., count, average).
Breakdowns are shown as charts within the panel, helping you visually validate whether your audience reflects your desired segmentation.
### Numerical metrics
When you select a numerical property or a model-based metric (e.g. events), you can optionally select a non-numerical user property to group the metric by.
|  |  |
| ------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------- |
Except in the case of `Saved metrics`, if no `Group by` option is selected, the chart will group by the metric itself.
|  |  |
| ---------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
Depending on the type of numerical metric you select, you are able to choose additional options on what to view in your breakdown.
|  |  |
| --------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
### Further exploration
To explore a breakdown in more depth, you can open it in [Charts](/campaign-intelligence/overview) to apply additional filters and groupings, explore different visualizations, and save the chart for future reference.
For more information, check out the [**Audience Breakdowns** blog post](https://hightouch.com/blog/audience-breakdowns).
## Profiles
In cases where you haven't built a specific audience, but need to explore members of your parent model to understand available attributes or lookup specific members to troubleshoot or understand behavior, use the Profiles tab.
After selecting a parent model, you can search members by Primary Key, or use the Filter option to locate users using any of the conditions available throughout Customer Studio.
Select a specific member to inspect their profile and dive into their attributes, recent events, key metrics, audience memberships and more.
## FAQ
#### Is our insight data saved by Hightouch?
**No**, Hightouch never saves your data.
#### Do Breakdowns and Overlaps reflect if an audience belongs to a priority list?
Yes-the insight tools take [audience priority](/customer-studio/priority-lists) into account.
---
## Journeys
**URL:** https://hightouch.com/docs/customer-studio/journeys
**Description:** Build multi-step campaign flows on a visual canvas. Journeys evaluate on a fixed schedule, so entry, delays, exits, and sends happen on runs, not in real time.
**Section:** Customer Studio
| | |
| ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers and operations teams building scheduled, multi-step campaign flows such as welcome series, winback programs, nurture tracks, and cross-channel lifecycle campaigns. |
| **Prerequisites** |
A defined [schema](/customer-studio/schema) with a parent model
At least one [destination](/getting-started/concepts#destinations) connected (you can create syncs directly from journey tiles or configure them in advance)
|
_Journeys let you build multi-step campaign flows on a visual canvas with delays, branches, experiments, and sends to destinations. Journeys evaluate on a fixed schedule, so they are not a real-time event processor. Entry, movement, exits, and sends only happen when the journey runs._
Before building your first journey, read [Plan your campaign
workflow](/customer-studio/campaign-workflow) to understand how journeys,
priority lists, destination rules, and experiments work together — and where
each piece of campaign logic belongs.
---
## What you'll learn
- [How journey evaluation works and why it matters for timing](#how-journeys-evaluate)
- [How to plan entry method, schedule, and sync mode before building](#plan-your-journey-before-you-build)
- [How to build and configure a journey on the canvas](#create-a-journey)
- [How to validate logic, payloads, and end-to-end behavior before launch](#test-and-validate-before-launch)
- [How to monitor performance and debug unexpected behavior](#monitor-and-debug)
- [Design patterns for common journey use cases](#journey-examples)
---
## How journeys evaluate
Journeys run on a **fixed schedule**. Members enter, move, exit, and sync **only** when the journey runs.
Use journeys for lifecycle programs like welcome flows, nurture tracks, and re-engagement. Do not think of them as an instant event-response system.
Think of a journey as a repeated evaluation of current warehouse data. On each run, Hightouch checks who should enter, who should advance, who should exit, and which sends should happen. That makes timing consistent across the full flow — but it also means journeys are not real time.
### What happens on each run
At each scheduled evaluation, Hightouch:
1. Checks the Start tile and adds members who newly qualify
2. Evaluates every active member's current tile and advances them if conditions are met
3. Checks global exit criteria and removes qualifying members
4. Fires all `Send to destination` syncs for members who reached or remain at sync tiles
5. Logs progress for reporting
Each journey run is one full evaluation.
### Why timing feels different from real-time systems
Some lifecycle tools react as soon as a user takes an action. Journeys work differently:
- **Delays resolve on runs, not on timers.** A "1-day delay" means the member advances on the first run after 24 hours have elapsed. If the journey runs every 6 hours, actual wait time could be anywhere from 24 to ~30 hours.
- **Exit criteria are checked on runs.** If a user makes a purchase between two runs, they stay in the journey until the next evaluation detects the change.
- **Hold until is schedule-checked.** A Hold until tile is not a real-time event listener. Hightouch checks the condition at each run and advances members when it becomes true.
- **Syncs fire on runs.** A member reaching a sync tile does not send immediately. The send happens when that journey run processes the tile.
### How sync timing follows the journey schedule
Syncs inside a journey run when the journey evaluates — they do not follow their own independent schedules. However, the underlying sync must still be enabled and correctly configured. A paused or misconfigured sync can block expected behavior even if the journey runs normally.
If a journey shows members reaching a sync tile but the sync reports zero
records, check that the sync is not paused and that its configuration (model,
field mappings, Liquid templates) is valid.
---
## Plan your journey before you build
Most journey confusion comes from building before planning. These decisions affect when members enter, how long delays last, when exits happen, and when data syncs to destinations. Make them before you open the canvas.
### Choose the right entry method
| Use this entry type | When the start condition is... | Example |
| ------------------- | --------------------------------------- | ------------------------------------------------------------------- |
| **Enters audience** | A user matches an audience right now | Users who haven't purchased in 30 days |
| **Performs events** | A user did something at a specific time | Users who abandoned a cart, created an account, or submitted a form |
Audience-based entry is simpler and works well when the qualifying condition is a state. Event-based entry is better when the qualifying condition is an action with a timestamp, especially when the journey should start relative to that action.
### Choose the right schedule
The evaluation schedule is the most important journey setting because it determines the precision of everything else: when entries are detected, when delays resolve, when exits are checked, and when syncs run.
- **Choose the schedule based on the fastest response time you need.** If you need to send within an hour of a trigger event, use an hourly or sub-hourly schedule.
- **Design delays as multiples of your schedule.** A 1-day delay on an hourly schedule resolves predictably. A 30-minute delay on a daily schedule resolves no differently than a 1-minute delay — both advance on the next daily run.
- **More frequent runs give you more precise timing, but they also run more warehouse queries.** Balance precision against query volume and warehouse cost.
### Choose the right sync mode
How you configure the sync on a `Send to destination` tile determines whether members receive one-time messages or maintain ongoing membership in a destination list.
| Use this mode | When you want to... | Recommended sync mode |
| ------------- | -------------------------------------------------------------------------------- | --------------------- |
| **Trigger** | Send a one-time message (email, SMS, push) per member per step | Insert |
| **Cohort** | Maintain a destination list or audience that reflects current journey membership | Upsert or Mirror |
Detailed sync mode guidance, including what to avoid, is in [Sync configuration for journeys](#choosing-sync-configurations-for-journeys).
If you need destination-specific filtering (consent, privacy, business rules) that should not change audience membership, configure [destination rules](/customer-studio/destination-rules) separately. Destination rules filter records at sync time, so a journey can behave correctly while the destination receives fewer rows.
### When to use one journey vs. multiple
| Situation | Use one journey or multiple? | Example |
| -------------------------------------------------------- | ---------------------------- | ---------------------------------------------------------------------------------- |
| Linear flow with few branches | One journey | Welcome journey: one delay, one branch, two outcomes |
| All paths share the same entry and schedule | One journey | Trial onboarding where everyone enters from the same event |
| Easy to follow on one canvas | One journey | Post-purchase follow-up with one entry, one delay, one branch |
| Different use cases need different schedules | Multiple journeys | Cart abandonment runs hourly; winback runs daily |
| You want to update flows independently | Multiple journeys | Onboarding and feature-adoption evolve on separate timelines |
| Canvas is too complex to reason about | Multiple journeys | Split welcome, activation, re-engagement, upsell, and churn into separate journeys |
| Different lifecycle stages need different entry criteria | Multiple journeys | One journey for account creation, another for 30-day inactivity |
### What to validate before launch
Before starting a journey, confirm:
- The entry audience or event produces the members you expect
- Delays are multiples of the evaluation schedule
- Exit criteria use stable fields that won't cause unexpected removals
- Sync tiles are configured with the right mode (trigger vs. cohort) and the underlying syncs are enabled
- You've run a simulation to verify branching logic
For guidance on which feature should handle timing, overlap, consent, and measurement, see [Plan your campaign workflow](/customer-studio/campaign-workflow).
---
## Required setup
Journeys rely on the parent model defined in your [schema](/customer-studio/schema). Your parent model must come from a supported warehouse source and use the [Lightning sync engine](/syncs/lightning-sync-engine).
Journeys send data to destinations through [syncs](/syncs/overview): sending messages, updating records, or managing audience membership. Connect your [destinations](/syncs/create-your-first-sync#connect-a-destination) and configure syncs before publishing a journey.
### Supported sources
Journeys requires the [Lightning sync engine](/syncs/lightning-sync-engine), which is available for a specific set of warehouse sources. For the current list, see [Lightning sync engine supported sources](/syncs/lightning-sync-engine).
If your source is not listed in the Lightning sync engine docs, Journeys is
not currently supported for that source. If you see the message "This source
does not support journeys" in the app,{" "}
{" "}
to confirm compatibility.
---
## Create a journey
### Step 1: Create the shell
1. Go to **Customer Studio → [Journeys](https://app.hightouch.com/journeys?from=docs)**.
2. Select **Add journey**.
3. In the modal, select a **parent model**, enter a **journey name**, and add an optional **description**.
4. Select **Create journey**.
You are taken to the journey canvas in **Draft** mode.
---
### Step 2: Configure the Start tile
The Start tile controls who enters the journey, how often they can re-enter, and whether they are excluded because of other journeys. This is one of the most important setup steps.
Select the **Start** tile to open its configuration drawer, then select an entry type:
- [**Enters audience**](#audience-based-entry) — members enter when they join an audience
- [**Performs events**](#event-based-entry) — members enter when they perform an event
The entry method cannot be changed after the journey is activated. Choose
carefully before launching.
#### Audience-based entry
1. Select an **entry audience**.
2. (Optional) Configure **exclusions**:
- **Don't exclude any profiles** — the default. Members can be in multiple journeys simultaneously.
- **Exclude profiles who are currently in any other journey** — prevents overlap across all journeys.
- **Exclude profiles who are currently in specific journey(s)** — prevents overlap with selected journeys only.
Exclusions prevent members from entering this journey if they are already in another one. They apply at entry time, not later in the flow. Exclusions are scoped to journeys on the same parent model.
3. Select a **re-entry** behavior:
- **Only one entry** — the default. Each member can enter the journey once, ever.
- **Specific number of entries** — each member can enter up to N times.
- **Unlimited entries** — no cap on re-entry.
You can also set a **re-entry delay** to prevent members from re-entering too soon after their last exit.
4. Select **Update** to save.
#### Event-based entry
1. Select the **event** that triggers entry.
Only events with assigned primary keys can be used to enter a journey. A
warning appears if the event model lacks a primary key.
2. (Optional) Enable **Include past events** and specify a lookback window. When disabled, only events occurring after the journey starts qualify.
3. Configure **exclusions** (same options as audience-based entry).
4. Select a **re-entry** behavior (same options as audience-based entry).
5. If you selected multiple entries, configure **simultaneous entry** behavior. This controls whether a member can be in the journey more than once at the same time.
- **Not allowed** — profiles must finish the journey before starting again
- **Specific number of simultaneous entries**
- **Profile can be in the journey unlimited times simultaneously**
{" "}
The re-entry delay only applies when the member is not currently in the
journey. Simultaneous entries can still happen if the member is already in the
journey.
6. Select **Update** to save.
---
### Step 3: Add tiles to the canvas
Use the **Add to canvas** panel to drag tiles onto the canvas, or select the `+` below any tile.
Available tiles: `Time delay`, `Hold until`, `Segment`, `A/B Split`, `Send to destination`, `Send email`, `Send SMS`, `Send push`, `Set variable`, and `Assign coupon`.
#### Canvas interactions
- **Pan** by selecting empty space and dragging, or scrolling on a trackpad
- **Zoom** with the +/- buttons or pinch gestures
- **Move a tile** by selecting and dragging it
- **Delete a tile** by hovering and selecting the trash icon (the Start tile cannot be deleted)
- **Duplicate a tile** by hovering and selecting the copy icon
- **Select multiple tiles** by holding **Shift** while selecting and dragging around a group
- **Clean up** the layout by selecting the clean-up icon at the top left of the canvas
- **Toggle compact/detail mode** to expand or collapse tile details
---
### Step 4: Configure each tile
Each tile opens a configuration drawer when selected.
#### Time delay
Pauses each member before advancing. Time delay supports three modes:
- **Duration** — wait a fixed amount of time (e.g., 2 hours, 3 days)
- **Specific date** — wait until a calendar date and time
- **Date from a profile property** — wait until a date stored on the member's profile, with an optional plus/minus offset (e.g., 2 days before `subscription_renewal_date`)
In all three modes, members advance at the **next journey run after the delay condition is met** — not at the exact moment. If your journey runs every 6 hours and you set a 1-day duration delay, actual wait time ranges from 24 to ~30 hours.
For duration delays, set the duration equal to or longer than the evaluation
interval. A 15-minute delay on an hourly schedule advances on the next hourly
run — the same as a 59-minute delay. Sub-interval delays do not add meaningful
precision.
#### Hold until
Waits for a condition to be met, with a maximum wait time. Hold until creates two paths:
- **Rule met** — the member advances down this path when the condition is satisfied
- **Rule not met** — the member follows this path if the condition is still not met when the maximum wait time expires
Hold until is **schedule-checked, not real-time**. Hightouch evaluates the condition at each journey run. It should not be treated as an event listener that fires the moment something happens.
Examples of hold conditions:
- Order completed within last 7 days
- `loyalty_tier = Gold`
- Purchase amount > 50
#### Segment
Branches the journey based on member attributes, traits, audience membership, event history, or relationships.
**Branches evaluate top to bottom. The first matching branch wins.** If a member could match multiple branches, they follow the first one. Design branch conditions to be mutually exclusive when possible to avoid ambiguity.
#### A/B Split
Randomly assigns members to branches for controlled experimentation.
Configure the number of branches, percentage allocation per branch, and group names (e.g., `Treatment` and `Control`).
Avoid changing split percentages after the journey goes live. Changing
allocations mid-flight can compromise experimental validity.
**Measurement:** You can attach an experiment to a split tile to measure the impact of different branches on key metrics. In the split tile's details drawer, select **Create experiment** under **Measurement configuration** to get started.
Once created, configure your primary metric, optional secondary metrics, and measurement window. Results are accessible from the split tile or under **Intelligence → Experiments**. See [Experiments](/campaign-intelligence/experiments) for details.
#### Send to destination
Triggers a sync to a destination. You can select an existing sync or create a new one from the tile.
Syncs inside a journey run when the journey evaluates — they do not follow their own independent schedules. Choose trigger syncs for one-time sends and cohort syncs for maintained destination lists. See [Sync configuration for journeys](#choosing-sync-configurations-for-journeys) for details.
#### Set variable
Context variables let you set and carry values as a member progresses through the journey. They're useful for tagging which path a member followed, passing journey state to destinations, or branching on journey-specific data later in the flow.
Define context variables in [journey settings](#variables), then assign their values in Set variable tiles throughout the journey.
**Example:** Track whether a member was routed through a VIP path and sync that flag to a destination.
1. Define a context variable `vip_user` in the journey's settings.
2. Add a Segment tile that branches on loyalty level.
3. On the high-loyalty branch, add a Set variable tile that sets `vip_user` = `true`. On other branches, set `vip_user` = `false`.
4. In a `Send to destination` tile, map `vip_user` as a synced field so the destination receives the journey-state flag.
After a journey runs, you can view context variable values in the member details panel.
Set variable tiles write to the journey variables defined in **Settings →
Variables**. The underlying data model uses the term "context variables" — you
may see this in warehouse log tables and API responses.
#### Assign coupon
Assigns a coupon code from a configured coupon pool to each member who reaches the tile. Use this for promotions that require unique, single-use codes.
See [Coupon assignment](/customer-studio/coupons) for setup and pool configuration.
Coupon assignment tiles cannot be used in event-triggered journeys. Use an
audience-based entry instead.
---
## Edit a journey
Select **Edit** at the top right of the canvas to enter edit mode.
Editing a live journey affects members currently in the flow. If you add,
remove, or reconfigure tiles, members in delays or in-flight branches continue
from their current position — but behavior after the edit may change. For
complex changes, consider creating a new journey and draining the old one
instead of editing in place.
---
## Journey settings
In edit mode, the **Settings** panel appears on the right. Select **Edit** to update journey name, description, evaluation schedule, exit criteria, variables, and alerting.
### Evaluation schedule
The evaluation schedule controls how frequently the journey runs. This is the single most important setting because it determines the precision of entry detection, delay resolution, exit checking, and sync triggering.
| Schedule type | Description | Example |
| --------------------- | ------------------------ | ------------------------------ |
| **Interval** | Repeating fixed interval | Every 1 hour, every 30 minutes |
| **Custom recurrence** | Specific days and times | Weekdays at 9 AM |
**Rules of thumb:**
- Choose the schedule based on the tightest send precision you need. If a 6-hour window is acceptable, a 6-hour interval works. If you need hour-level precision, use an hourly schedule.
- Design all delays as multiples of the schedule. A 2-hour delay on an hourly schedule resolves predictably. A 2-hour delay on a daily schedule resolves the same as a 1-minute delay.
- More frequent runs increase warehouse query volume. Balance precision against cost.
**First run timing:**
- **Interval schedules** run a few seconds after you select **Start**.
- **Custom recurrence** runs at the next matching time.
### Exit criteria
Exit criteria remove members from the journey when a condition becomes true — for example, when a user makes a purchase, unsubscribes, or joins a different audience.
Exit conditions are checked during each scheduled run, before a member moves through the next step in the journey. A member can be removed mid-run before a later step sends or advances them. Exit conditions are not checked in real time — if a user completes the exit action between runs, they remain in the journey until the next evaluation detects the change.
Avoid using the same volatile field for both entry and exit criteria without
considering how it changes. If the entry audience is "users who haven't
purchased in 30 days" and the exit condition is "user has purchased," a single
purchase both removes the user from the entry audience and triggers exit. This
is often correct — but if the field changes frequently (e.g., a rolling
window), members can churn in and out unexpectedly. Use stable,
one-directional fields for exit criteria when possible.
**Debugging exits:** If members exit unexpectedly, check the member details panel to see which run removed them and what exit condition matched. Compare the exit timestamp to data changes in your warehouse to confirm the condition was met.
### Variables
Define context variables in the **Variables** tab. Each variable you define here becomes available in [Set variable tiles](#set-variable) throughout the journey.
### Alerting
Alerting notifies you when a journey run fails. Journeys inherit default alert settings from your workspace, and you can override them at the journey level.
**Recipients:** Alerts can be delivered by email, Slack, PagerDuty, SMS, webhook, or any channel your workspace has configured. Select **Add new recipient** to configure delivery.
**Triggers:** Specify when alerts fire. Journeys currently support alerts for fatal errors — runs that fail during execution. If no trigger is configured, alerts appear in the Hightouch UI only.
**Notification behavior:** You receive one alert per status change, not per run. A run fails → one warning alert. The next run also fails → no additional alert. The next run succeeds → one recovery alert.
**Global defaults** are configured under **Integrations → Alerting → Journeys**. To learn more, see [Sync alerting](/syncs/alerting#overview).
---
## Test and validate before launch
Testing a journey involves three distinct steps, each validating different things. Treating simulation as a full end-to-end test is the most common validation mistake.
| Validation step | What it validates | What it does NOT validate |
| ------------------------- | ------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------- |
| **Simulation** | Branching logic, segment conditions, entry criteria | Delay timing, sync payloads, cross-journey interactions, destination platform behavior |
| **Test sync** | Payload shape, field mappings, Liquid templates, destination formatting | Journey logic, entry/exit behavior, delay timing |
| **Live small-cohort run** | End-to-end behavior including real timing, real syncs, and real destination delivery | Nothing — this is the full validation |
### Run a simulation
Simulation shows how members move through the journey's logic without sending data to any destinations. All time delays are skipped.
1. Open your journey and select **Run test**.
2. Set the **Start date & time**:
- **Now** simulates new entries based on current data.
- A **past date/time** tests event-based journeys using historical events with timestamps after the selected start time.
3. Choose a **test size**:
- **Single event** — simulates the path for one member
- **Set of events** _(default)_ — simulates all qualifying members, useful for validating branch percentages
4. Select **Run test**. The canvas shows how members enter, which branches they follow, and how many reach each step.
Simulation does not affect journey membership, logs, syncs, or the evaluation
schedule. It is a safe, read-only validation step.
#### Simulation limitations
Simulation validates how the configured logic behaves from the chosen start point. It does **not**:
- Replay future journey runs day by day — it evaluates all logic in a single pass with delays skipped
- Trigger syncs or send any data to destinations
- Model cross-journey interactions, exclusions, or priority-list behavior
- Reflect destination platform rules (consent filtering, suppression, deduplication)
Use simulation to answer "does the branching logic work?" — not "will the full campaign behave correctly over time?"
### Run a test sync
For audience-based journeys, you can test how data is sent from `Send to destination` tiles without modifying journey logic. Test syncs validate that the correct members and fields reach the destination in the expected format.
**Triggering a test sync:**
- **From a simulation:** Use members from the simulated path as the basis for the test sync — this ensures the selected members actually reach the sync tile.
- **From a sync tile directly:** Select members from the entry audience. These members are not guaranteed to reach the sync tile in a real run.
- **From a live journey:** Select members who have already passed through the tile.
**Test data:** Test syncs use real journey members. You can override key values (such as email addresses) to prevent test data from reaching real recipients.
- **Upload test users** via CSV — any fields left blank use original values
- **Select existing rows** — search for specific members or choose a random member
Select **Run test** to trigger the sync. Review the results to confirm field mappings and payload formatting.
Override recipient fields (such as email) to avoid sending to real recipients
during testing. Some fields may appear redacted in the UI — if not edited,
`<Redacted by Hightouch>` is used when syncing.
### Validate with a live small cohort
After simulation confirms logic and test syncs confirm payloads, validate end-to-end behavior with a small live audience.
1. Create a narrow test audience (e.g., internal users or a small segment of real users).
2. Configure the journey to use this audience for entry.
3. Start the journey and monitor the first several runs.
4. Check destinations to confirm data arrived correctly.
5. After validation, update the entry audience to your production audience.
This is the only way to validate real timing, real exit behavior, real sync execution, and real delivery to destinations. If you use [destination rules](/customer-studio/destination-rules), validate them separately — they filter records at sync time and can cause the destination to receive fewer rows than the journey processed.
---
## Publish, pause, resume, and turn off
### Activate
Select **Activate** to start the journey. Interval schedules begin evaluating within seconds. Custom recurrence schedules begin at the next matching time.
### Pause
Select **Pause** to stop all evaluations and sync triggering. Members remain in their current positions. No members enter, advance, exit, or sync while the journey is paused.
### Resume
Select **Resume** to restart evaluation without losing member progress. Members continue from where they were when the journey was paused.
### Turn off
Select **Turn off** to stop the journey permanently. You must choose how to handle members currently in the flow:
- **Existing members should complete the journey first** _(default)_ — the journey enters **Draining** state. No new members enter, but existing members continue advancing and syncing until they finish or exit. The journey transitions to **Off** automatically once no members remain.
- **Hard stop** — all members are removed on the next evaluation run.
Draining is the safest option when retiring or replacing a journey. It prevents disruption to members who are mid-flow — they complete their remaining steps, receive any pending messages, and exit naturally.
### Trigger a manual run
For active journeys, you can trigger a manual evaluation run outside the regular schedule. This is useful for testing configuration changes or processing members immediately after a data refresh without waiting for the next scheduled run.
### Reset a journey
For inactive journeys (Off or Draft), you can reset the journey to clear all member progress, run history, and log data. This returns the journey to a clean state as if it had never run.
Resetting a journey permanently deletes all member data, run history, and
journey log table contents. This cannot be undone.
---
## Monitor and debug
### Journey performance
Once a journey is running, the canvas updates to show how members move through each step. Line thickness reflects volume. Hover over any line to see counts.
The **Performance** panel shows high-level metrics across the full history of the journey (unless you reset it):
- **In progress** — members currently waiting in a tile
- **Entered** — total members who have ever entered
- **Met exit criteria** — members removed by global exit rules
- **Finished** — members who reached the final step
Use these metrics to answer the most common monitoring questions: who entered, where people are waiting, where people exited, and whether branch distribution looks plausible.
### Tile-level performance
Select any tile to view step-specific metrics:
- **Currently in tile** — members waiting at this step (commonly nonzero for Time delay and Hold until)
- **Entered tile** — members who have ever reached this step
- **Met exit criteria** — members removed from this tile by exit rules
- **Advanced from tile** — members who moved to the next step
For Segment, A/B Split, and Hold until tiles, you can view per-branch counts and percentages.
### Member-level details
Select a count in the Performance panel to open the member drawer, which shows a sample of 100 random members. You can search by primary key.
Select a member to view their complete attributes and a step-by-step visualization of their path through the journey. This is the fastest way to debug why a specific user entered, exited, or followed an unexpected branch.
### Journey logs
Hightouch writes journey data to tables in your warehouse under the `HIGHTOUCH_PLANNER` schema for reporting, debugging, and custom analytics beyond what the UI provides. Workspace-level read-only views in `hightouch_audit` aggregate data across all journeys.
For full schema details, see [Journey tables](/developer-tools/warehouse-tables#journey-tables) and [Journey views](/developer-tools/warehouse-tables#journey-views).
---
## Journey examples
These examples show how to design journeys for common lifecycle use cases. Each focuses on the key decisions: entry method, schedule, sync mode, exit criteria, and branching.
### Cart abandonment with staged escalation
**Goal:** Send a sequence of messages to users who abandon a cart, escalating from email to SMS, and stop if they purchase.
**Design decisions:**
- **Entry:** Event-based (`cart_abandoned` event) if your warehouse captures cart events with timestamps. Audience-based (`abandoned cart` audience) if you define abandonment as a state rather than an event.
- **Schedule:** Hourly or every 2 hours, since cart abandonment is time-sensitive.
- **Exit:** Purchase event or audience membership change. Set exit criteria so members leave as soon as a purchase is detected on the next run.
- **Flow:** Start → 2-hour delay → Segment by email eligibility → Send email (trigger/insert) → 1-day delay → Segment by SMS eligibility → Send SMS (trigger/insert) → End.
- **Sync mode:** Trigger with Insert for both send steps. Each member should receive each message at most once.
With an hourly schedule, the 2-hour delay resolves within 2–3 hours. If you
use a daily schedule instead, the delay resolves on the next daily run — which
could be up to ~26 hours.
### Welcome journey with state carried through the flow
**Goal:** Send different onboarding messages to new users vs. reactivated users, and tag which path each member followed.
**Design decisions:**
- **Entry:** Event-based (`account_created` or `first_login` event).
- **Schedule:** Hourly or more frequent if near-real-time welcome messages matter.
- **Flow:** Start → Set variable tile assigns `user_type` based on whether the member has prior activity → Segment by `user_type` → New user branch (welcome series) / Reactivated user branch (re-engagement series) → Send to destination with `user_type` mapped as a field.
- **Sync mode:** Trigger with Insert for welcome messages.
- **Variables:** Define a `user_type` context variable. Assign it in Set variable tiles on each branch. Sync it to the destination so the receiving tool knows which path the member followed.
### Re-engagement with fixed evaluation rhythm
**Goal:** Re-engage users who haven't interacted in 60 days with a multi-touch sequence over two weeks.
**Design decisions:**
- **Entry:** Audience-based (`inactive 60+ days` audience).
- **Schedule:** Daily. Re-engagement is not time-sensitive enough to justify hourly evaluation.
- **Exit:** Purchase, login, or opt-out. Use a stable field — "last activity date" works better than a rolling window that could cause churn.
- **Flow:** Start → 3-day delay → Send email 1 → 4-day delay → Send email 2 → 7-day delay → Send final email → End.
- **Sync mode:** Trigger with Insert for emails. If you also want to maintain a "re-engagement" audience in your ad platform during the journey, add a parallel cohort sync (Upsert or Mirror) that reflects current membership.
For campaign patterns that combine journeys, destination rules, priority lists, and experiments — see [Campaign patterns](/customer-studio/campaign-workflow#campaign-patterns).
---
## Best practices
**Start narrow and prove value.** Begin with one well-defined journey — a cart abandonment flow or a welcome series. Validate that the timing, branching, and destination behavior work correctly before building complex multi-journey programs.
**Create new journeys instead of heavily editing live ones.** Major structural changes to a live journey affect members mid-flow in ways that are difficult to predict. For significant redesigns, create a new journey, drain the old one, and transition members to the new flow.
**Plan overlap management for interacting journeys.** When running multiple lifecycle programs, use journey exclusions, priority lists, and staggered schedules to prevent conflicts. See [Manage overlap across programs](/customer-studio/campaign-workflow#manage-overlap-across-programs) for detailed guidance.
### Priority lists and entry audiences
If your entry audience is inside a [priority list](/customer-studio/priority-lists), priority is respected when determining who enters. A member who qualifies for a higher-priority audience does not enter a journey tied to a lower-priority one until they drop out of the higher-priority audience.
### Sync execution inside journeys
**Failed syncs and retries:** If a record fails to sync, Hightouch retries on subsequent runs. For cohort syncs, retries stop when the record is removed from the sync. For trigger syncs, retries continue until the record syncs successfully.
**Sync sequencing:** All `Send to destination` tiles trigger at the same time — the end of each evaluation run. If you need strict sequencing between two syncs, add a Time delay tile between them so they fire on separate runs.
### Choosing sync configurations for journeys
Use these tables when you need to understand how each sync mode behaves inside a journey, or when troubleshooting unexpected sync behavior. For a quick summary, see [Choose the right sync mode](#choose-the-right-sync-mode).
#### Trigger mode syncs
Best for one-time messages (email, SMS) where you want one event per member per step.
| Sync mode | Behavior | Recommendation |
| ---------- | ----------------------------------------------------------------------------------------------------------------- | --------------- |
| **Insert** | Members entering or re-entering the tile during a run are synced. Previously synced members are ignored. | **Recommended** |
| Upsert | New and updated members are synced, which can cause duplicate sends. | Not recommended |
| Update | Only previously synced members are updated. New members are ignored unless they already exist in the destination. | Not recommended |
| Mirror | All members who have ever passed through the tile are synced every run. | Not recommended |
| Diff | One file per operation (enter, change, remove). | Not recommended |
#### Cohort mode syncs
Best for maintaining a destination audience or list that reflects current journey membership.
| Sync mode | Behavior | Removal behavior | Recommendation |
| ---------- | ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ |
| **Upsert** | New members sync. Changes to existing members sync. | Members are removed based on journey configuration. | **Recommended** |
| **Mirror** | All current members sync every run — no diff performed. | Members are removed based on journey configuration. | **Recommended** when destination should exactly match journey membership |
| Update | Only existing destination members are updated. New members are ignored. | Members are removed, but `deleteMode` is usually set to clear (attributes removed, member remains). | Not recommended |
| Insert | New members sync. Changes and removals are ignored. | No removal occurs. | Not recommended |
| Diff | One file per operation. | No removal occurs. | Not recommended |
Avoid `deleteMode: do nothing` with cohort syncs. Removals will not be
applied, and the destination list will grow indefinitely.
---
## Troubleshooting
If your journey run fails because of warehouse SQL issues (for example, invalid identifiers, unresolved columns, or missing objects), see [Resolve SQL compilation errors](/models/sql-compilation-errors).
Most troubleshooting issues trace back to three things: evaluation schedule,
data freshness in the warehouse, and Start tile configuration. Check these
first.
#### Members are not entering the journey
**Why this happens:** Entry is checked at each evaluation run. If the entry criteria aren't met at run time — because the audience hasn't refreshed, the event hasn't been ingested, or the member is excluded — no entry occurs.
**How to verify:**
1. Confirm the Start tile configuration: entry source, filters, exclusions, and re-entry settings.
2. For event entry, verify the event model has a primary key, the correct event is selected, and the lookback window (if enabled) covers the relevant time range.
3. Check the journey's evaluation schedule and last run time — if the journey hasn't run recently, no entries can occur.
4. Preview the entry audience to confirm it contains the members you expect.
5. Check whether members are entering and immediately meeting exit criteria, which would make it look like no one entered.
#### Delays or Hold until behavior seems inconsistent
**Why this happens:** Delays and hold conditions are evaluated on runs, not continuously. A delay doesn't start a countdown timer — it marks when the member entered the tile and advances them on the first run after the delay period has elapsed. This means actual wait times align to the journey schedule.
**How to verify:**
1. Compare the delay duration to the evaluation interval. A 30-minute delay on a daily schedule resolves on the next daily run, not 30 minutes later.
2. Check whether multiple delay or hold tiles are chained, causing timing to compound across runs.
3. If tighter timing is needed, increase the journey's evaluation frequency.
#### Members are not exiting when expected
**Why this happens:** Exit criteria are checked on runs. If the exit-triggering action (a purchase, an opt-out) happens between runs, the member stays in the journey until the next evaluation detects the change in the warehouse.
**How to verify:**
1. Compare the action timestamp (purchase, opt-out) to the journey's run history. If the action occurred after the last run, the member hasn't been evaluated yet.
2. Review the exit criteria definition in journey settings — confirm the correct fields and logic are configured.
3. Check data pipeline latency. If the warehouse hasn't refreshed since the action occurred, exit criteria can't match.
4. Use the member details panel to see which run removed a member and what condition matched.
#### Members are re-entering unexpectedly
**Why this happens:** Re-entry settings on the Start tile control how many times a member can enter. If set to multiple or unlimited entries, members re-enter whenever they meet entry criteria again.
**How to verify:**
1. Check the Start tile's re-entry and simultaneous entry settings.
2. For event-based entry, confirm whether repeated qualifying events are expected or indicate a data issue.
3. If re-entry is intentional but happening too frequently, add or increase the re-entry delay.
#### Sync tile shows members but sync reports zero records
**Why this happens:** The journey moved members to the sync tile, but the underlying sync didn't process them. This is usually a sync configuration issue, not a journey issue.
**How to verify:**
1. Confirm the sync is not paused. A paused sync does not process records even when the journey triggers it.
2. Check that the sync's model is correctly linked to the journey output.
3. Validate any Liquid templates in the sync configuration — a malformed template can cause the sync to produce zero rows.
4. Review the journey run history to confirm members reached the tile during a completed run, not during a simulation.
#### Tile counts don't match destination counts
**Why this happens:** The journey processed the expected number of members, but the destination received fewer. The discrepancy typically comes from the destination side, not the journey.
**How to verify:**
1. Check whether the destination applies additional filtering: consent rules, suppression lists, deduplication, or audience caps.
2. If the sync uses cohort mode with removal on exit and the tile is the final step, members may be removed before the destination processes them. Add a Time delay tile after the sync tile so members remain in the sync for at least one run.
3. Check for a `Users might never be synced` warning on the sync tile, which indicates the configuration may prevent records from reaching the destination.
4. Review [destination rules](/customer-studio/destination-rules) or [priority lists](/customer-studio/priority-lists) that may narrow synced records.
#### Journey run fails
**Why this happens:** A journey run can fail if the underlying warehouse query errors out — for example, due to invalid identifiers, missing objects, permission changes, or query timeouts. Network connectivity issues between Hightouch and your warehouse can also cause failures.
**How to verify:**
1. Check the journey's run history for the specific error message.
2. For SQL-related errors (invalid identifiers, unresolved columns, missing objects), see [Resolve SQL compilation errors](/models/sql-compilation-errors).
3. Verify that warehouse credentials and permissions haven't changed. The service account Hightouch uses must have read access to the parent model's underlying tables.
4. If the failure is intermittent, check whether your warehouse has concurrency limits or query timeout settings that the journey run is hitting.
5. Configure [alerting](#alerting) to get notified on failures so you can respond quickly.
#### Members are receiving duplicate sends
**Why this happens:** Duplicate sends typically come from sync mode misconfiguration rather than a journey logic issue. In trigger mode, Upsert syncs resend to members whose attributes have changed, even if they've already been sent to. Re-entry settings can also cause a member to pass through the same `Send to destination` tile multiple times.
**How to verify:**
1. Check the sync mode on the `Send to destination` tile. For one-time messages, use **Trigger with Insert** — this syncs each member once per tile visit and ignores previously synced members.
2. Check the Start tile's re-entry settings. If members can re-enter, they pass through `Send to destination` tiles again on each entry.
3. For cohort syncs, confirm `deleteMode` is configured — without it, members accumulate in the destination and may be reprocessed.
4. Review the member details panel to see whether the member entered the tile once or multiple times.
#### Members follow an unexpected branch
**Why this happens:** Segment tiles evaluate branches top to bottom and route members to the first matching branch. If conditions overlap, members may consistently route to an earlier branch even though they also qualify for a later one.
**How to verify:**
1. Open the Segment tile and review branch order. Branches are evaluated from top to bottom — the first match wins.
2. Check whether branch conditions overlap. If a member qualifies for multiple branches, they always take the highest one.
3. Use the member details panel to see which branch a specific member took and when.
4. If the issue only appears in live runs but not simulation, check data freshness — the warehouse data at run time may differ from what you tested against.
#### Simulation results differ from live runs
**Why this happens:** Simulation validates branching and conditions in a single pass with delays skipped. Live runs involve real timing, real data freshness, real exit evaluation, cross-journey interactions, and destination platform behavior — none of which simulation models.
**Recommended approach:**
1. Use simulation to validate branching and conditions.
2. Use test syncs to validate field mappings and payload formatting.
3. Use a small live cohort to validate end-to-end behavior.
4. When comparing live results to expectations, check data freshness — warehouse data may have changed between when you ran the simulation and when the live journey evaluated.
---
## Journey notes
Notes let you leave context about goals, tile behavior, or intentional edge cases directly on the canvas. They help teammates understand complex journeys without digging through configuration drawers.
You can add notes in three ways:
1. Drag a note from the tile library onto the canvas or onto a specific tile.
2. Add a note directly on the tile.
3. Open the tile drawer and add a tile-specific note.
Notes carry over when a journey is created from a template. The journey creator appears as the author of template-inherited notes.
---
## Journey templates
Journey templates let you save and reuse common journey structures so teams can build consistent lifecycle flows without starting from scratch. See [Journey templates](/customer-studio/templates#journey-templates).
You can also apply [sync templates](/customer-studio/templates#sync-templates) to journey syncs to standardize destination configuration across teams. Most sync template changes propagate to derived journey syncs, but Journey Setting Configurations (Trigger/Cohort mode) do not propagate. If you change the sync mode on a template, update the journey syncs individually.
---
## What's next
- [Campaign workflow](/customer-studio/campaign-workflow) — manage overlap, priority, and coordination across multiple journeys and audiences.
- [Priority lists](/customer-studio/priority-lists) — control which audiences and journeys take precedence when members qualify for multiple programs.
- [Warehouse tables reference](/developer-tools/warehouse-tables#journey-tables) — full column-level schemas for journey log, metadata, and context tables.
- [Destination rules](/customer-studio/destination-rules) — apply frequency caps, suppression, and channel preferences to journey sends.
- [Experiments](/customer-studio/experiments) — measure journey effectiveness with holdout groups and split tests.
---
## Lookalike audiences
**URL:** https://hightouch.com/docs/customer-studio/lookalike-audiences
**Description:** Find users who resemble your best-performing audience segments using Hightouch's native lookalike modeling.
**Section:** Customer Studio
| | |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Audience** | Marketers and data teams |
| **Prerequisites** |
A defined [schema](/customer-studio/schema) with at least one parent model (for example, `Users`).
[Predictive inputs](/customer-studio/predictive-traits#configure-predictive-inputs) configured in your schema (user properties and key events).
An existing audience to use as your source (seed) audience.
|
_Lookalike audiences let you find users who are similar to a high-value segment — without requiring a specific conversion event or time window — so you can expand your reach to the people most likely to engage._
---
## What you'll learn
- [Understand how lookalike audiences work](#overview)
- [Create a lookalike audience trait](#create-a-lookalike-audience-trait)
- [Analyze lookalike results](#analyze-lookalike-results)
- [Use lookalike traits in audiences](#use-lookalike-traits-in-audiences)
- [Compare native lookalikes vs. ad-platform lookalikes](#native-vs-ad-platform-lookalikes)
---
## Overview
Predictive traits like purchase propensity require you to define a specific outcome event and prediction window. **Lookalike audiences** take a different approach: instead of predicting a future action, they measure how similar each user is to the users in a source audience you define.
This is useful when you want to expand a proven segment — such as your highest-LTV customers, most engaged subscribers, or users who completed a multi-step onboarding — without needing to identify the exact event that makes them valuable.
Lookalike audiences are part of the [predictive traits](/customer-studio/predictive-traits) suite. They reuse the same predictive inputs (user properties and key events) and are exposed as traits in the audience builder.
### How it works
When you create a lookalike audience trait, Hightouch:
1. Analyzes the users in your **source audience** to build a profile of their shared characteristics using your configured predictive inputs.
2. Scores every user in the **match pool** (all users in your parent model, or a filtered subset) based on how closely they resemble that profile.
3. Returns a **similarity percentile** for each user, where higher percentiles indicate greater similarity to the source audience.
The result is a trait you can use like any other — filter on it in audience definitions, sync it to destinations, or combine it with other conditions.
### Use cases
- **Expand proven segments** — Create a lookalike trait using your top-LTV audience as the source audience, filter for the top 10% similarity percentile in a new audience, and sync that audience to ad platforms for prospecting campaigns that mirror your best customers.
- **Prospecting without conversion data** — Use a well-defined audience (for example, "engaged enterprise users") as the source audience for a lookalike trait, then build an audience of similar users to target with outreach — no single conversion event required.
- **Cross-sell targeting** — Set buyers of one product line as the source audience, create a lookalike trait, and build an audience of high-similarity users who haven't purchased that line yet to sync to an ESP for cross-sell email campaigns.
- **Re-engagement** — Create a lookalike trait based on your most active users, then build an audience of high-similarity but low-activity users and sync them to a push notification or email destination for re-engagement.
## Create a lookalike audience trait
To create a lookalike audience trait:
1. Go to **Customer Studio → Traits**.
2. Click **Create → New trait**.
3. In the **Method** step, select a **parent model** (for example, `Users`).
4. Under **Calculation method**, select **Lookalike Audience**.
5. Click **Continue**.
6. In the **Calculation** step, select the audience you want to mimic. This is the **source audience** — the group of users the model will use as the reference profile. Choose a well-defined audience that represents the type of user you want to find more of.
7. (Optional) Add filters to restrict the **match pool** to specific users. For example, you might filter to users where `device = desktop` or limit scoring to a specific region.
8. Set the **update schedule** to control how often the model retrains and refreshes similarity scores.
9. Click **Continue to finalize**. In the **Finalize** step, name your trait, optionally add a description, review the calculation summary, and click **Create trait**.
The model trains using your configured predictive inputs and generates a similarity percentile for each user. Training time varies based on the size of your data.
Choose a source audience that is well-defined and large enough for the model
to learn meaningful patterns. Very small source audiences (under a few hundred
users) may produce less reliable similarity scores.
## Analyze lookalike results
After the model finishes training, the **Prediction analysis** tab on the trait detail page shows how well the model distinguishes source audience users from the rest of the match pool.
The analysis page displays the total number of users scored, the source audience name and size, and the last run date. Below these stats, you can explore results at different similarity thresholds.
### Percentile buckets
The analysis page shows preset similarity buckets — Top 10%, Top 20%, and Top 50% — along with a Custom option. Each bucket displays the number of users in that tier and how much more similar they are to the source audience compared to the average user. Use these to gauge how concentrated similarity is at the top of the distribution.
### Similarity score chart
A chart plots each user's similarity score against their percentile ranking. You can adjust the percentile range to focus on a specific slice of users — for example, the 80th to 100th percentile — to understand how scores are distributed among your highest-similarity users.
### AUC
AUC (Area Under the Curve) measures how well the model separates source audience users from the match pool. It is computed on a held-out test split. A score closer to 100% indicates stronger separation.
### Evaluation metrics
The evaluation metrics table shows Precision, Recall, and Lift at multiple cutoff thresholds (Top 0.1%, Top 1%, Top 5%, Top 10%, Top 25%). These metrics are computed on a held-out test split where a sample of the source audience is mixed into the match pool.
- **Precision** — Of the top N% most similar users, the fraction that are in the heldout source audience.
- **Recall** — Of all heldout source audience users, the fraction that appear in the top N% most similar users.
- **Lift** — How much more likely the top N% of users are to be in the heldout source audience compared to random selection.
## Use lookalike traits in audiences
After the model finishes training, the lookalike trait appears in the Traits list and can be used like any other trait.
1. Go to **Customer Studio → Audiences** and create a new audience.
2. Add a filter using your lookalike audience trait.
3. Set a **percentile threshold** to control how similar users must be to qualify. For example, filtering for the top 10% returns users who most closely resemble your source audience.
4. (Optional) Combine the lookalike filter with other conditions — demographics, behavioral traits, or other predictive traits — to further refine the audience.
Once defined, you can sync the audience to any connected destination for activation: ad platforms, ESPs, CRMs, or any other tool in your stack.
Lookalike similarity scores update on a schedule. Between updates, new users
or behavior changes won't be reflected until the next model run.
## Native vs. ad-platform lookalikes
Many ad platforms (Meta, Google, and others) offer their own lookalike or similar-audience features. Hightouch's native lookalike audiences complement these by giving you more control:
| | **Native lookalikes (Hightouch)** | **Ad-platform lookalikes** |
| ----------------- | ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------- |
| **Data used** | All user properties and behavioral events in your warehouse | Only data the platform has collected |
| **Where usable** | Any destination — ads, email, CRM, push, and more | Only within that ad platform |
| **Transparency** | You control the source audience, see similarity scores, and can combine with other filters | Platform controls the algorithm; limited visibility into how similarity is determined |
| **Cross-channel** | Same lookalike audience can be synced everywhere | Must be rebuilt per platform |
You can use both approaches together: build a refined seed audience in Hightouch using native lookalike scores, then sync that seed to an ad platform to create a platform-native lookalike on top.
## Related articles
- [Predictive traits](/customer-studio/predictive-traits)
- [Catalog recommendations](/customer-studio/catalog-recommendations)
- [Traits](/customer-studio/traits)
- [Audiences](/customer-studio/usage)
---
## OneTrust Snowflake Native App
**URL:** https://hightouch.com/docs/customer-studio/onetrust
**Description:** This guide explains how to use consent data collected and stored in OneTrust, and passed to Hightouch via the OneTrust Snowflake Native App. Once the app is configured, Hightouch can reference this data in subsets, destination rules, and sync behavior to enforce consent across downstream tools.
**Section:** Customer Studio
| | |
| ----------------- | -------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Platform admins |
| **Prerequisites** | [Set up for platform admins and data teams →](/customer-studio/quick-start#setup-for-platform-admins-and-data-teams) |
_This guide explains how to use consent data collected and stored in OneTrust, and passed to Hightouch via the OneTrust Snowflake Native App. Once the app is configured, Hightouch can reference this data in subsets, destination rules, and sync behavior to enforce consent across downstream tools._
---
## What you'll learn
After reading this article, you will be able to:
- Understand how the OneTrust Snowflake Native App makes consent data available in Hightouch.
- Connect OneTrust data stored in Snowflake to your Customer Studio schema.
- Filter audiences by consent status using Subsets.
- Enforce consent compliance across destinations with Destination Rules.
- Configure syncs to remove unconsented records automatically.
- Model various consent and preference types to support advanced use cases.
---
## Overview
If you’re using OneTrust [Universal Consent and Preference Management](https://www.onetrust.com/products/universal-consent-and-preference-management/) (UCPM) to manage customer consent and communication preferences, you can use the [OneTrust Snowflake Native App](https://www.onetrust.com/news/onetrust-accelerates-access-to-trusted-customer-data-with-snowflake-native-app-for-consent-management/) to automatically sync consent and preference data into your Snowflake warehouse. This keeps your data warehouse up-to-date in near real-time with the latest opt-ins, opt-outs, and preference changes. Snowflake’s data governance controls are used to mask rows and filter columns to ensure that specific roles can only access the data they are permissioned for.
Once centralized, consent becomes available in Hightouch, where it can be used to:
* Filter audiences with [Subsets](/customer-studio/subsets)
* Block unconsented records with [Destination Rules](/customer-studio/destination-rules)
* Remove records from downstream tools (Destinations) when consent is revoked
This workflow ensures that downstream tools only receive and act on data from users who have given the appropriate consent, automating compliance and improving campaign targeting.
## Setup
To start using OneTrust consent data in Hightouch:
1. [**Install and configure the OneTrust Snowflake Native App**](https://my.onetrust.com/s/article/UUID-2f18f280-ec76-c4c5-2419-73615c2ad4dd?language=en_US&topicId=0TO1Q000000ItSRWA0) in your Snowflake account.
- This streams real-time consent and preference updates from OneTrust into a Snowflake table in your warehouse.
- You can read OneTrust's complete documentation Universal Consent and Preferences Management [here](https://my.onetrust.com/s/article/UUID-9f6e9cab-3016-5dd0-342f-7f03287c358d?language=en_US)
2. [**Set up Snowflake as a Source**](/sources/snowflake) **in Hightouch**
- Follow Snowflake source setup instructions to connect your warehouse and make the OneTrust data accessible to Hightouch.
3. [**Create a Parent Model**](/customer-studio/schema#define-the-parent-model) **in Customer Studio Schema**
- Go to **Customer Studio → Schema**
- Click **Create parent model** and select your primary user table (e.g., `users`)
4. [**Add your consent table as a related model**](/customer-studio/schema#add-related-models)
- Depending on your OneTrust setup, you may choose to perform some transformations on this table or create a new view before exposing it to Hightouch.
- Click on the **+** button next to the parent model
- Select the consent table (e.g., `DATASUBJECT_CONSENT_VIEW`)
- Define the join condition between your parent model and this consent table
- Choose the appropriate relationship type
- If you only have one “consent purpose” in OneTrust, you can use a 1:1 relationship
- If you have different “consent purposes,” you should choose 1:Many, as OneTrust stores different consent types in different rows
5. [**Select relevant consent columns**](/customer-studio/schema#add-related-models)
- Once the tables are joined, select the consent fields you want to expose in your schema—such as `email_opt_in`, `sms_opt_in`, or `ad_sharing_opt_in`.
These fields will then be available throughout Hightouch to:
* Filter audiences using Subsets
* Enforce rules with Destination Rules
* Configure sync behavior based on consent status
## Enforcing consent in Hightouch
### 1. Filter audience segments by consent with Subsets
Use [Subsets](/customer-studio/subsets) to define reusable filters based on consent preferences. You can apply subsets across audiences, journeys, and syncs to ensure only users who meet consent criteria are included.
**Example use case:**
Create a subset that includes only records where email_opt_in \= true. You can use the default `Email` subset in pre-initialized `Consent management` category.
Once created, this subset can be:
_ **Applied to multiple audiences** to enforce consistent filtering across marketing use cases.
_ **Set as mandatory** for specific user groups (e.g., your email marketing team), so they cannot build audiences without referencing the required consent field.
\* Restricted to certain values, ensuring end users can only select approved options (e.g. forcing the selection of “Opted In” from a consent dropdown).
This helps prevent accidental inclusion of unconsented users in downstream tools.
### 2. Suppress unconsented records with Destination Rules
Use [Destination Rules](/customer-studio/destination-rules) to define conditions under which Hightouch will block records from syncing to specific destinations.
**Example rule:**
Block all records where `sms_opt_in` \!= `true` from syncing to your SMS tool.
This acts as a fallback safeguard to prevent unintentional delivery of unconsented data.
### 3. Remove data when consent is revoked
When a user revokes consent, Hightouch can remove their data from downstream tools—but only if your model is filtered by consent **and** your sync is configured to support deletions.
#### Removal options
**1. Recurring syncs**
On the next scheduled run, Hightouch will remove any records that no longer match your model’s filter.
**Requirements:**
_ Your model or audience includes a consent-based filter (e.g., `email_opt_in` = `TRUE`).
_ Use a [sync mode](/syncs/types-and-modes#sync-modes) that supports removals
\* Enable record removal in the sync configuration
**2. Triggered deletions (real-time)**
Hightouch will automatically configure a sync as Triggered if your model and destination support real-time updates (e.g., using a CDC or event stream).
If your sync is a Triggered Sync:
* You’ll see the option:
“**Delete records when they no longer match the model**”
* Enable this to remove records from the destination as soon as a consent change is detected (e.g., `email_opt_in` becomes `FALSE`)
## Modeling consent and preference data
The OneTrust Snowflake Native App can sync a wide range of consent and preference fields into your warehouse—not just binary opt-ins. Modeling these fields effectively allows you to enforce consent logic across multiple tools using Hightouch.
When defining your [schema](/customer-studio/schema), consider including:
* **Channel-specific consent fields**
**Example:** `email_opt_in`, `sms_opt_in`, `push_opt_in`
These allow you to enforce channel-level rules in Destination Rules or Subsets.
* **Communication frequency preferences**
**Example:** `email_frequency`= `weekly`
You can use these to suppress over-messaging by filtering for lower-frequency segments.
* **Content or category preferences**
**Example:** `category_preference`= `mens_clothing`
Supports targeting users based on stated interests.
* **Consent for data sharing**
**Example:** `ad_sharing_platform_opt_in`= `true`
Ensures records are only sent to third-party platforms (e.g., Meta, Google) when appropriate consent exists.
## Related documentation
**Use OneTrust data in Hightouch:**
- [Subsets](/customer-studio/subsets)
- [Destination Rules](/customer-studio/destination-rules)
**Modeling consent data:**
- [Plan your data model](/customer-studio/data-model)
- [Define your schema](/customer-studio/schema)
**Event-based consent management:**
- [Consent Manager for Events](/events/consent/consent-manager)
- [OneTrust for Events](/events/consent/onetrust)
**OneTrust documentation:**
- [Universal Consent and Preferences Management](https://my.onetrust.com/s/article/UUID-9f6e9cab-3016-5dd0-342f-7f03287c358d?language=en_US)
---
## Customer Studio overview
**URL:** https://hightouch.com/docs/customer-studio/overview
**Description:** Customer Studio is where you build audiences, orchestrate campaigns, and activate customer data from your warehouse — without writing SQL.
**Section:** Customer Studio
| Audience | How you'll use this article |
| -------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| **Marketing** | Understand what Customer Studio does and how you'll use it day-to-day to build audiences, run campaigns, and measure results. |
| **Data teams** | Understand the schema and source setup your marketers depend on, and how Customer Studio connects to the rest of the Hightouch platform. |
_Customer Studio is a visual workspace for building audiences and orchestrating campaigns using data from your warehouse. Marketers define who to target, how to reach them, and when — then sync audiences to ad platforms, CRMs, and messaging tools without writing SQL._
---
## What you'll learn
- [What Customer Studio is and what problems it solves](#overview)
- [How to access Customer Studio](#access-customer-studio)
- [Core features: audiences, campaigns, and measurement](#core-features)
- [How features work together: schema → audiences → syncs, plus governance and orchestration](#how-features-work-together)
- [How to get started, by role](#get-started)
- [When to use Customer Studio](#when-to-use-customer-studio)
- [How Customer Studio connects to other Hightouch products](#how-customer-studio-fits-into-hightouch)
---
## Overview
Customer Studio gives marketers direct access to warehouse data through a visual interface. Instead of requesting lists from data teams or rebuilding segments in every downstream tool, you define audience logic once and activate it everywhere.
Customer Studio handles two related jobs:
- **Audience building** — Define who belongs in a segment using filters, traits, and event conditions. Preview audience size and composition, then sync membership to destinations like ad platforms, CRMs, and email tools.
- **Campaign orchestration** — Coordinate how audiences are activated across channels. Build multi-step journeys, manage priority when audiences overlap, run experiments, and measure lift.
Because Customer Studio connects directly to your data warehouse, audiences stay current without manual exports or CSV uploads. Data is not copied or stored unless explicitly configured.
[**Explore the interactive architecture diagram**](https://architecture.hightouch.com/?from=docs&product=customer-studio&utm_source=hightouch-docs&utm_medium=referral&utm_campaign=architecture-explorer&utm_content=customer-studio) — see how data flows and where Customer Studio fits in your stack.
#### Example: Suppress recent buyers from acquisition campaigns
Say your paid media team runs prospecting campaigns on Meta and Google. Without suppression, you spend budget showing ads to people who already converted. In Customer Studio, you build an audience of customers who purchased in the last 7 days, then sync that audience as a suppression list to both ad platforms. The audience updates automatically as new purchases land in your warehouse.
#### Example: Welcome flow triggered by sign-up
When a new user signs up, you want to send a welcome email immediately, wait three days, then check whether they completed onboarding. If they haven't, send a reminder. If they have, add them to your regular newsletter audience. In Customer Studio, you build this as a journey: an entry trigger based on sign-up events, a delay tile, a conditional branch, and sync tiles that deliver to your email tool at each step.
---
## Access Customer Studio
1. Go to [**app.hightouch.com**](https://app.hightouch.com?from=docs).
2. In the left sidebar, click [**Customer Studio**](https://app.hightouch.com/audiences?from=docs).
---
## Core features
Customer Studio is organized around four areas. Data teams and platform admins define and govern the workspace — [schema](/customer-studio/schema), [traits](/customer-studio/traits), [row-level access](/customer-studio/subsets), and [destination rules](/customer-studio/destination-rules). Marketers self-serve on audiences, campaigns, and measurement within that configured surface.
### Build audiences
Define who belongs in a segment using the [visual audience builder](/customer-studio/usage) or the [AI Agent](/customer-studio/agents). Combine filters on user attributes, behavioral events, and computed [traits](/customer-studio/traits) — then preview size and composition before activating.
### Activate audiences
[Sync audiences](/customer-studio/syncs) to ad platforms, CRMs, email tools, and other destinations on a schedule or in real time. Use [destination rules](/customer-studio/destination-rules) to control when and how each destination receives data.
### Orchestrate campaigns
Build multi-step [journeys](/customer-studio/journeys) with audience- or event-based entry, syncs, delays, conditional branches, and scheduled evaluation — for welcome sequences, winback campaigns, nurture tracks, and lifecycle messaging. [Plan your campaign workflow](/customer-studio/campaign-workflow) to decide which features handle timing, overlap, consent, and measurement. Use [priority lists](/customer-studio/priority-lists) to manage overlap when users qualify for more than one audience, and [experiments](/customer-studio/experiments) to run holdout tests and measure incremental lift.
### Measure and validate
Use [insights](/customer-studio/insights) to analyze audience composition and overlap, [audience dashboards](/customer-studio/audience-overviews) to monitor membership trends, and [sampling](/customer-studio/sampling) to spot-check audience members before activating. Save configurations with [templates](/customer-studio/templates) and capture point-in-time records with [snapshots](/customer-studio/audience-snapshots).
---
## How features work together
The core Customer Studio workflow flows in one direction: **schema → audiences → syncs**. Governance and orchestration layers sit on top.
Your [schema](/customer-studio/schema) defines what data is available. Marketers build [audiences](/customer-studio/usage) by filtering on schema columns, [traits](/customer-studio/traits), and events. Audiences are delivered to destinations through [syncs](/customer-studio/syncs).
From there, two layers shape what actually gets delivered:
- **Orchestration** — [Journeys](/customer-studio/journeys) coordinate multi-step campaigns across audiences. [Priority lists](/customer-studio/priority-lists) resolve overlap so each user lands in only their highest-priority audience. [Experiments](/customer-studio/experiments) split audiences into holdout and treatment groups to measure lift.
- **Governance** — [Destination rules](/customer-studio/destination-rules) control which destinations an audience can sync to and under what conditions. [Row-level access](/customer-studio/subsets) scopes which records specific teams can see and sync. [OneTrust integration](/customer-studio/onetrust) enforces consent before data leaves the warehouse.
Changes at one layer affect everything downstream. A schema change — like removing a column or modifying a relationship — can break traits, audiences, and journeys that depend on it. Check the [Activity tab](/customer-studio/schema#review-model-activity) on any schema model to see what depends on it before making changes.
[AI Decisioning](/ai-decisioning/overview) and [Intelligence](/campaign-intelligence/overview) consume Customer Studio outputs. AI Decisioning uses audiences as its eligible population and optimizes per-user message selection. Intelligence measures campaign outcomes using the same warehouse data and audience definitions.
---
## Get started
Customer Studio requires initial setup from data teams before marketers can use it. Some responsibilities — like schema changes, trait creation, and governance rules — are ongoing.
### For data teams and platform admins
Complete these steps to enable marketers. Steps 1–3 are required; the rest can be added iteratively as your team scales.
| Step | What to do | Why |
| ---- | ----------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 1 | [Connect your warehouse](/sources/overview) | Customer Studio reads from your data warehouse. At least one source must be connected. |
| 2 | [Add destinations](/destinations/overview) | Set up the tools where audiences will be delivered (ad platforms, CRMs, email tools). |
| 3 | [Plan your data model](/customer-studio/data-model) and [define your schema](/customer-studio/schema) | Decide on your grain and model topology, then create the parent, related, and event models marketers use to build audiences. |
| 4 | [Add general models](/customer-studio/general-models) (optional) | Make additional warehouse tables available for [agent](/agents/overview) queries without adding them to your audience schema. |
| 5 | [Enable Lightning Sync Engine](/syncs/lightning-sync-engine) (recommended) | Recommended for all supported sources. Required for [Journeys](/customer-studio/journeys), [Sampling](/customer-studio/sampling), [Warehouse Sync Logs](/syncs/warehouse-sync-logs), and other advanced features. |
| 6 | [Enable sampling](/customer-studio/sampling) (optional) | Speed up audience previews on large datasets. Requires Lightning Sync Engine. |
Once the schema is configured, marketers can start building audiences
immediately. Data teams continue to manage schema updates, traits, governance
rules, and consent configuration as the workspace evolves.
### For marketers
Once your data team has configured the schema, start by [creating your first audience](/customer-studio/usage). You can also [describe a segment in plain language](/customer-studio/agents) and let the AI Agent suggest audience logic you can review and refine.
From there, [sync your audience](/customer-studio/syncs) to a destination, then explore [journeys](/customer-studio/journeys), [experiments](/customer-studio/experiments), and [insights](/customer-studio/insights) as your campaigns grow. If you need a computed value that isn't in your data — like lifetime value or days since last purchase — ask your data team to [create a trait](/customer-studio/traits).
---
## When to use Customer Studio
Customer Studio is the right starting point when you need to build audiences from warehouse data and activate them in downstream tools. It's also where you orchestrate campaign logic like journeys, priority, and experimentation.
| Scenario | Recommended approach |
| ---------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| Build audiences from warehouse data and sync to ad platforms, CRMs, or email tools | **Customer Studio** |
| Orchestrate multi-step lifecycle campaigns (welcome, winback, nurture) | **Customer Studio** [Journeys](/customer-studio/journeys) |
| Run holdout tests and measure incremental lift | **Customer Studio** [Experiments](/customer-studio/experiments) |
| Optimize which message, channel, and timing each user receives | [AI Decisioning](/ai-decisioning/overview) — uses Customer Studio audiences as input |
| Measure campaign performance across channels with unified metrics | [Intelligence](/campaign-intelligence/overview) — draws from the same warehouse data and audiences |
| Personalize onsite or in-app experiences in real time | [Real-Time Personalization](/real-time/overview) |
| Collect behavioral events from your app or website | [Hightouch Events](/events/overview) — events feed into Customer Studio as audience filters |
---
## How Customer Studio fits into Hightouch
Customer Studio is the audience and orchestration layer of the Hightouch platform. It reads from your warehouse data and delivers audiences to the tools where campaigns run.
#### What Customer Studio uses
- **Models from your [connected source](/sources/overview)** — Data teams define SQL or visual models that map warehouse tables (users, events, purchases) into the schema Customer Studio reads from.
- **Resolved identities from [Identity Resolution](/identity-resolution/overview)** — If you use identity resolution, the Golden Record model can serve as the parent model for your schema, giving marketers a deduplicated view of each customer.
- **Real-time events from [Hightouch Events](/events/overview)** — Streaming events like page views, purchases, and app actions can be used as audience filters and journey triggers.
#### What Customer Studio sends
- **Audiences to [destinations](/destinations/overview)** — Syncs deliver audience membership and traits to CRMs, ad platforms, email tools, and other destinations on a schedule or in real time. [Realtime audiences](/real-time/overview) evaluate streaming event behavior alongside warehouse-backed conditions and forward entry and exit changes to downstream tools as they happen.
- **Audiences to [AI Decisioning](/ai-decisioning/overview)** — AI Decisioning agents use Customer Studio audiences as their eligible population, then optimize which messages each user receives.
- **Analytics to [Intelligence](/campaign-intelligence/overview)** — Experiment results, audience composition data, and campaign metrics in Intelligence draw from the same warehouse data and audiences defined in Customer Studio.
---
## Manage data governance
Customer Studio includes tools for controlling what data reaches destinations and who can access it.
- **[Subsets](/customer-studio/subsets)** — Scope which records can be synced to certain destinations by region, subscription status, or compliance flags.
- **[OneTrust integration](/customer-studio/onetrust)** — Automatically exclude opted-out users from audience syncs using your OneTrust consent data.
- **[Data masking](/customer-studio/data-masking)** — Redact or hash PII fields before they reach destinations.
- **[Configure Schema Git Sync](/customer-studio/schema-setup)** — Version-control your schema configuration with Git Sync.
---
## Supported sources
Only the following sources support Customer Studio:
- Snowflake
- Databricks
- Google BigQuery
- Amazon Redshift\*
- Amazon Athena
- Azure Synapse
- MS SQL Server
- PostgreSQL
- Greenplum
- Microsoft Fabric
_\* Customer Studio may not work with [Amazon Redshift Spectrum](https://docs.aws.amazon.com/redshift/latest/dg/c-getting-started-using-spectrum.html) due to its [limitations around nested data](https://docs.aws.amazon.com/redshift/latest/dg/nested-data-restrictions.html). To mitigate these, you can try to query a [materialized view](https://docs.aws.amazon.com/redshift/latest/dg/materialized-view-overview.html) during your [parent model setup](/customer-studio/schema#define-the-parent-model)._
Some advanced features — including [journeys](/customer-studio/journeys),
[predictive traits](/customer-studio/predictive-traits), [audience
snapshots](/customer-studio/audience-snapshots), and [identity
resolution](/identity-resolution/overview) — require specific sources. Check
individual feature docs for source requirements, or
contact support to confirm compatibility with your warehouse.
---
## Permissions
Access to Customer Studio features is controlled through custom roles. Admins can configure Customer Studio-specific grants in the **Customer Studio** tab of the custom role builder, including access to parent model data, audience creation, journey configuration, and journey publishing. See [Roles](/workspace-management/roles#customer-studio) for the full list of grants.
---
## Predictive traits
**URL:** https://hightouch.com/docs/customer-studio/predictive-traits
**Description:** Predictive traits use machine learning to forecast user behavior, giving you forward-looking signals like purchase propensity or churn risk.
**Section:** Customer Studio
| | |
| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers and analytics/data engineers |
| **Prerequisites** |
A defined schema with at least one parent model (for example, `Users`).
Event data that reflects the outcome you want to predict (for example, purchases, cart completions).
(Optional) Connected destinations for syncing audiences.
|
## Overview
When defining audiences, you sometimes want to act on signals about what customers are likely to do next, not just what they have already done.
**Predictive traits** extend Customer Studio traits with AI-powered scores that estimate the likelihood of a user performing a future action.
For example:
- **Purchase propensity:** probability that a user will make a purchase within the next 7 days
- **Cart completion propensity:** probability that a user who started checkout will complete their purchase
- **Churn prediction:** probability that a user will _not_ perform a key event (such as a purchase or login) within a given timeframe
Predictive traits are refreshed automatically on a schedule so your campaigns always use the latest predictions.
The predictive traits suite also includes [catalog recommendations](/customer-studio/catalog-recommendations) for personalized item-level recommendations and [lookalike audiences](/customer-studio/lookalike-audiences) for finding users similar to your best segments.
Predictive traits are built on the same machine learning models used in
Hightouch [**AI Decisioning**](/ai-decisioning/overview). They are
automatically trained on your historical event data and do not require custom
ML expertise.
## Configure predictive inputs
Before you create a predictive trait, you must define **predictive inputs** in your schema. Predictive inputs are the user properties and event data that models use for training and scoring.
To configure predictive inputs:
1. Go to **Customer Studio → Schema**.
2. Select your parent model (for example, `Users`).
3. Open the **Predictive Inputs** tab.
4. Under **User Properties**, select the columns from this model that can help identify patterns — for example, `age`, `country`, `device`, `plan_tier`.
5. Under **Key Events**, add the event models that are relevant to the prediction target. Events must have a primary key configured to be available for selection.
Once saved, these inputs are reusable across multiple predictive traits, catalog recommendations, and lookalike audiences.
## Create a predictive trait
To create a predictive trait:
1. Go to **Customer Studio → Traits**.
2. Click **Create → New trait**.
3. In the **Method** step, select a **parent model** (for example, `Users`).
4. Under **Calculation method**, select **Prediction**.
5. Click **Continue**.
6. In the **Calculation** step, configure the prediction settings. The remaining settings on this page are optional.
Under **What action do you want to predict?**, select the event to forecast (for example, `Purchases`). Only events with primary keys that are configured as predictive inputs are shown.
Under **What kind of prediction is this?**, set the prediction timeframe — for example, "Event Performed within the next 7 Day(s)." Optionally, enable **This is a churn predictor** to score users based on the likelihood of _not_ performing the event within the timeframe.
(Optional) Under **Who should this prediction apply to?**, add filters to restrict which users are scored — for example, based on something they've already done, like signing up or viewing a page.
Under **How often should scores be updated?**, set the refresh cadence. More frequent updates keep scores fresh but use more warehouse resources.
Click **Continue**.
7. In the **Finalize** step, name your trait, optionally add a description, review the calculation summary, and click **Create trait**.
The system trains a model using your data and generates predicted scores for each user. Training typically takes several hours but will vary based on warehouse size and the amount of data being processed.
## View and use predictive traits
After training completes, predictive traits appear in the Traits list with type **Predictive**.
You can:
- **Preview results** in the trait details page
- **Use in audiences** by filtering on score thresholds or percentile ranges
- **Sync predictions** to destinations such as ad platforms, ESPs, or CRMs for activation
## Analyze predictions
After the model finishes training, the **Prediction analysis** tab on the trait detail page displays performance metrics. The page shows the total number of users scored and the last run date. Below these stats, you can explore results at different score thresholds.
### Percentile buckets
The analysis page shows preset score buckets — Top 20%, Bottom 80%, Top 50%, and a Custom option. Each bucket displays the number of users in that tier and their conversion rate. Use these to gauge how concentrated predictive value is at the top of the distribution and to choose effective thresholds for targeting.
### Model metrics
The analysis page displays three model performance metrics:
- **AUC (Area Under the Curve)** — Measures how well the model separates users who will perform the predicted action from those who won't. Values closer to 100% indicate stronger separation.
- **Log Loss** — Measures how confident and accurate the model's predictions are. Lower values are better — they mean the model's probability estimates closely match reality.
- **Average Precision** — Measures the quality of the model across all confidence thresholds. A higher value means the model maintains good precision across a range of recall levels.
### Top Feature Importance
The analysis page also lists the features that contribute most to the model's predictions, ranked by their share of total importance. This helps you understand which user properties and events drive the model's scoring.
Use percentile thresholds to build high-value segments. For example, create an
audience of “Top 20% purchase propensity” users and sync them to ad channels
for efficient spend.
## Use predictive traits in audiences
You can use predictive traits when defining audiences the same way you use other traits.
1. Go to **Customer Studio → Audiences** and click **Add audience**.
2. In the audience definition, select a trait.
3. Set the score range or percentile range you want to include (for example, 80–100%).
4. (Optional) Combine the predictive filter with other attributes, such as `country = US` or `plan_type = paid`.
Predictive traits can be layered with demographic or behavioral conditions, giving you highly targeted segments. For example:
- **High propensity to purchase + specific geography** (US, Canada)
- **Cart completion propensity + recent site activity** (visited in the last 7 days)
Predictive traits update on a schedule. Between updates, some people may
complete the action you’re predicting (like making a purchase) but their score
won’t change until the next update. To keep your audience focused on people
who haven’t yet acted, **add a filter to exclude people who have already
performed the predicted event** (e.g, `Purchases = 0` or `Purchases (in last 7
days) = 0`). This helps make sure your campaign only includes users who are
still likely to take action.
## Sync predictive audiences
Once you define an audience with predictive traits, you can sync it to any connected destination the same way you sync other audiences.
For example, you might sync:
- **High purchase propensity** users to Google Ads or Meta Ads for paid acquisition
- **High cart completion propensity** users to an ESP for checkout reminder emails
When syncing, predictive scores are included alongside user attributes so downstream tools can use them directly.
After syncs run, you can monitor performance and health in the **Syncs > Overview** tab.
You don’t need to configure anything special for predictive traits when
syncing--they behave like any other trait and can be mapped directly to
destination fields.
## Troubleshooting
If a predictive trait run fails, Hightouch assigns an error code that identifies the cause. The table below covers all predictive trait error codes. For a complete reference of all Hightouch error codes, see [Error codes](/developer-tools/error-codes#predictive-traits-errors).
| Error code | What it means | How to resolve |
| :--------------------------------- | :------------------------------------------------------------------------------ | :-------------------------------------------------------------------------------------------------------------------- |
| `NO_USERS_IN_INPUT_FILE` | The training audience returned no users. | Check that the parent audience or model query returns rows before training starts. |
| `NO_USER_ROWS_AFTER_FILTERS` | Users were present initially, but predictive filters removed all of them. | Loosen the filters or verify that event and property conditions match real data. |
| `NO_TRAINING_DATA_FILES` | Hightouch couldn't generate the training dataset. | Confirm that the configured events return data and the event model is queryable. |
| `LABEL_FREQUENCY_TOO_LOW` | There aren't enough examples of the target outcome to train the model reliably. | Increase the lookback window, reduce filters, or choose a higher-volume outcome event. |
| `NO_POSITIVE_EXAMPLES` | The training data contains zero positive examples of the target event. | Check that the outcome event is occurring for users in scope. |
| `TRAINING_MATRIX_DOWNLOAD_FAILED` | An internal failure occurred while retrieving the training artifact. | Retry the run. If it persists, contact support — this is typically not customer-fixable. |
| `NO_KEY_EVENTS` | No feature-generation events are configured. | Add at least one event input in the predictive trait configuration before retraining. |
| `EVENT_MODEL_MISCONFIGURED` | The event model configuration is invalid. | Confirm the event model exists, has a valid primary key, and includes all referenced columns. |
| `EVENT_QUERY_FAILED` | Hightouch couldn't query event data from the warehouse. | Check warehouse permissions, model SQL, and whether the referenced tables still exist. |
| `SOURCE_ACCOUNT_LOCKED` | The warehouse user account is locked. | Wait for the lockout to clear, or have your warehouse admin re-enable the account. |
| `UNKNOWN_EVENT_IN_FILTER` | A predictive filter references an event Hightouch can't resolve. | Check for renamed, deleted, or misconfigured events in the predictive trait setup. |
| `INVALID_EVENT_PROPERTY_CONDITION` | An event-property filter is malformed. | Verify the filter syntax and confirm you're using valid string property names. |
| `NO_EVENT_DATA_FILES` | No event data was produced for training or inference. | Confirm the configured events have recent data and event pulls are succeeding. |
| `TRAINING_ERROR` | An internal training failure occurred. | Retry once. If it repeats, contact support. |
| `CACHED_EVENT_PULL_DISABLED` | A required event pull is disabled. | Re-enable the event pull in your schema configuration and rerun the predictor. |
| `MALFORMED_EVENT_DATA` | Numeric event fields contain invalid values. | Clean the source data so numeric columns contain only valid numbers (no strings, nulls in numeric-only fields, etc.). |
| `MODEL_QUERY_FAILED` | Hightouch couldn't query the parent model data needed for the run. | Check that the parent model still exists and all referenced columns are present. |
## Related articles
- [Catalog recommendations](/customer-studio/catalog-recommendations)
- [Lookalike audiences](/customer-studio/lookalike-audiences)
- [Traits](/customer-studio/traits)
- [Audiences](/customer-studio/usage)
- [Experiments](/customer-studio/experiments)
- [AI Decisioning](/ai-decisioning/overview)
---
## Priority lists
**URL:** https://hightouch.com/docs/customer-studio/priority-lists
**Description:** Control which audience takes precedence when users qualify for multiple campaigns, so each person receives only the highest-priority message.
**Section:** Customer Studio
| | |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers, data teams |
| **Prerequisites** | [At least two audiences](/customer-studio/usage) built using the same [parent model](/customer-studio/schema#define-the-parent-model) |
_Priority lists control which audience takes precedence when users qualify for more than one. When audiences overlap, Hightouch syncs each user only to their highest-priority audience — preventing duplicate messages and ensuring each person receives the most relevant campaign._
---
## What you'll learn
- [When to use a priority list vs. audience exclusions](#when-to-use-a-priority-list)
- [Create a priority list](#create-a-priority-list) and set audience order
- [How priority lists affect journeys and insights](#how-priority-lists-affect-other-features)
---
## Overview
Users often qualify for multiple audiences based on buying behavior, demographics, or lifecycle stage. If you run campaigns targeting each of those audiences, the same person can receive multiple messages — a winback email, a loyalty offer, and a re-engagement ad all in the same week.
Priority lists solve this by assigning a rank to each audience. When a user qualifies for more than one audience in a priority list, Hightouch syncs them only to the highest-ranked audience. Priority is evaluated at sync time, so as users move in and out of audiences, their priority assignment updates automatically.
### When to use a priority list
Use a priority list when you have audiences that overlap and you want to guarantee each user receives only one campaign. This is common for lifecycle campaigns where a user might qualify for multiple stages simultaneously (for example, both a "trial expiring" and "hasn't activated" audience).
If you just need to remove specific users from an audience — like excluding recent purchasers from a prospecting campaign — use [audience exclusions](/customer-studio/usage) instead. Exclusions remove users from a single audience; priority lists rank users across multiple audiences.
---
## Create a priority list
1. Go to **Customer Studio → [Priority lists](https://app.hightouch.com/priority-lists?from=docs)**.
2. Click **Add priority list**.
3. Enter a **Name** and select a **Parent model**.
All audiences in the priority list must share this parent model.
4. Click **Add audiences** to open the audience picker.
5. Drag audiences into the desired priority order. The audience at the top has the highest priority.
6. Click **Save changes**.
You can return to this page anytime to reorder audiences or add new ones.
### Constraints
- Each audience can belong to only one priority list. If you need an audience in a second list, create a duplicate audience.
- All audiences in a priority list must use the same parent model.
---
## How priority lists affect other features
### Journeys
If a [journey's](/customer-studio/journeys) entry audience belongs to a priority list, the priority is respected when determining who enters the journey. For example, if Audience B is ranked below Audience A and a user qualifies for both, the user enters only Audience A's journey. They won't enter Audience B's journey until they drop out of Audience A while still qualifying for Audience B. See [Priority lists and entry audiences](/customer-studio/journeys#priority-lists-and-entry-audiences) for details.
### Insights
[Audience insights](/customer-studio/insights) — including breakdowns and overlap analysis — take priority list membership into account. When you view overlap between two audiences that share a priority list, the numbers reflect the post-priority assignment, not raw qualification.
---
## Get started with Customer Studio
**URL:** https://hightouch.com/docs/customer-studio/quick-start
**Description:** This quick-start guide walks you through setup (for platform admins and data teams) and day-to-day usage (for marketers), with links to each key step.
**Section:** Customer Studio
| | |
| ----------------- | ------------------------------------------------- |
| **Audience** | Platform admins and data teams, marketers |
| **Prerequisites** | At least one connected source and one destination |
Customer Studio helps you build, activate, and orchestrate campaign-ready audiences directly from your warehouse.
This guide covers:
| Section | What you'll learn |
| ----------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
| [**Setup for platform admins and data teams**](#setup-for-platform-admins-and-data-teams) | Connect your data schema, apply consent and governance rules, and enable features like sampling and holdout logs. |
| [**Day-to-day usage for marketers**](#how-marketers-use-customer-studio) | Use the visual interface to build audiences, sync to tools, analyze performance, and run campaigns. |
---
## Setup for platform admins and data teams
Before marketers can build and sync audiences in Customer Studio, your workspace must be connected to the right data and destinations. This setup is typically done once per workspace.
### Connect sources and destinations
| Step | What to do | Why |
| ---- | -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| 1 | [Connect your warehouse](/sources/overview) | Data sources must be connected so models and traits can be defined. |
| 2 | [Add destinations](/destinations/overview) | Set up tools where audiences will be delivered (e.g., CRMs, ESPs, ad platforms). |
| 3 | [Enable Lightning Sync Engine (recommended)](/syncs/lightning-sync-engine) | Recommended for all supported sources. Required for Journeys, Sampling, Warehouse Sync Logs, and other advanced features. |
Syncing audiences in Customer Studio requires both a source (data warehouse)
and at least one destination.
### Configure your data schema
| What to do | Why |
| -------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| [Define the schema](/customer-studio/schema) | Connect models for users, events, and related data so they're available in the audience builder. |
---
### Optional setup
#### Enable observability and optimization
| What to do | Why |
| ---------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| [Enable sampling](/customer-studio/sampling) | Speed up previews for large models by materializing a subset of data for audience and trait previews. |
| [Enable holdout group logs](/customer-studio/experiments#holdout-group-logs) | Track unsynced audience rows in your warehouse after each sync for auditing. |
#### Manage governance and consent
| What to do | Why |
| ------------------------------------------------------------- | ------------------------------------------------------------------------- |
| [Apply destination rules](/customer-studio/destination-rules) | Control sync behavior, redaction, and opt-outs. |
| [Filter with subsets](/customer-studio/subsets) | Narrow audience delivery on a per-destination basis. |
| [Manage consent](/customer-studio/onetrust) | Respect user consent preferences by syncing only users who have opted in. |
| [Mask sensitive data](/customer-studio/data-masking) | Redact or hash PII fields before they reach destinations. |
| [Configure Schema Git Sync](/customer-studio/schema-setup) | Version-control your schema configuration with Git Sync. |
---
## How marketers use Customer Studio
Once the schema is set up, marketers can use the visual interface to build and activate audiences without SQL.
### Define audiences
Define who qualifies for a campaign using the visual builder or the AI Agent.
| What to do | Why |
| ------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------- |
| [Create audiences with the visual builder](/customer-studio/usage) | Use filters, traits, and events to define your target group. |
| [Build audiences with the AI Agent](/customer-studio/agents) | Describe the segment you want in plain language and let the Agent build the audience logic for you. |
| [Define traits](/customer-studio/traits) | Create calculated fields like LTV or churn risk that you can reuse as filters across audiences. |
### Activate audiences
Send users to your tools on a schedule or in real time.
| What to do | Why |
| ------------------------------------------------- | ------------------------------------------------------------------------ |
| [Sync audiences to tools](/customer-studio/syncs) | Deliver audience data to destinations like Klaviyo, Meta, or Salesforce. |
### Analyze and validate
Check audience size, composition, and performance to ensure your segments behave as expected.
| What to do | Why |
| -------------------------------------------------------------------- | ----------------------------------------------------------- |
| [Preview with insights](/customer-studio/insights) | Check size and breakdowns by trait or property. |
| [Spot-check with sampling](/customer-studio/sampling#using-sampling) | Review a sample of audience members. |
| [Run experiments](/customer-studio/experiments) | Use experiments to test variations or apply control groups. |
| [Measure test results](/campaign-intelligence/experiments) | Use charts to evaluate experiment results. |
| [View audience dashboards](/customer-studio/audience-overviews) | Monitor audience health and membership trends at a glance. |
### Orchestrate campaigns
Manage flows, priority, and message timing across audiences.
| What to do | Why |
| ----------------------------------------------------------- | ---------------------------------------------------------------- |
| [Create journeys](/customer-studio/journeys) | Build multi-step campaigns with syncs, waits, and conditions. |
| [Manage campaign priority](/customer-studio/priority-lists) | Use priority lists to handle audience overlaps. |
| [Assign coupons](/customer-studio/coupons) | Distribute unique coupon codes to audience members at sync time. |
### Manage and reuse audiences
Keep audiences consistent, auditable, and reusable.
| What to do | Why |
| -------------------------------------------------------- | ------------------------------------------------------------------------- |
| [Use templates](/customer-studio/templates) | Save standard audience or sync configurations for reuse. |
| [Capture snapshots](/customer-studio/audience-snapshots) | Record audience membership at a point in time for auditing or comparison. |
---
## Sampling
**URL:** https://hightouch.com/docs/customer-studio/sampling
**Description:** Sampling materializes a subset of a model's rows to make model previews faster.
**Section:** Customer Studio
| | |
| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers, Platform admins |
| **Prerequisites** |
A defined [data schema](/customer-studio/schema)
Warehouse [source](/getting-started/concepts#sources) with write permissions
|
_Sampling speeds up audience and trait previews in Customer Studio by materializing a small, representative subset of your schema models into your data warehouse. For large datasets, this optimization can significantly reduce query times and warehouse compute during campaign building._
---
## What you'll learn
After reading this article, you'll be able to:
- Explain how sampling works and when to use it
- Enable and configure sampling for parent and related models
- Use sampled models for faster trait and audience previews
- Monitor and troubleshoot sampling jobs
---
## Overview
**Sampling** creates a reduced version of your schema models by materializing a subset of rows into dedicated tables in your data warehouse. This allows Hightouch to query smaller, faster subsets of data when generating previews in Customer Studio.
## How it works
Here’s how it works:
1. A configurable percentage of rows is sampled from your [**parent model**](/customer-studio/schema#define-the-parent-model).
2. For each sampled parent row, corresponding rows in [**related**](/customer-studio/schema#add-related-models) and [**event models**](/customer-studio/schema#add-event-models) are also materialized.
3. The sampled tables are stored in the [**Hightouch Planner**](/syncs/lightning-sync-engine#warehouse-schemas) schema of your warehouse.
4. Sampling runs on a configurable schedule in the background.
This setup allows previews (like [traits](/customer-studio/traits) or [audiences](/customer-studio/usage)) to query a representative subset of your data—ideal for faster iteration and testing.
Sampling requires write access to your warehouse and only works if the
Lightning sync engine is enabled.
## Supported warehouses
Currently, sampling is supported on the following sources:
- [Amazon Redshift](/sources/amazon-redshift)
- [Databricks](/sources/databricks)
- [Fabric](/sources/fabric)
- [Google BigQuery](/sources/google-bigquery)
- [Microsoft SQL Server](/sources/sqlserver)
- [PostgreSQL](/sources/postgresql)
- [Snowflake](/sources/snowflake)
- [Trino](/sources/trino)
The sampled tables will be materialized in the [Hightouch Planner schema](/syncs/lightning-sync-engine#warehouse-schemas).
## Set up sampling
To enable sampling:
1. Go to a [**parent model**](/customer-studio/schema#define-the-parent-model) in your schema
2. Go to the `Sampling` tab
3. Toggle sampling **on**
| Disabled | Enabled |
| :-------------------------------------------------------------: | :-----------------------: |
|  |  |
You can then configure:
- **Sample size**: Determines the percentage of your parent model (and any related models) that will be sampled.
- **Sample frequency**: Determines how often the models get resampled. This schedule also applies to sampling for related models as well unless they're overridden.
- **Related models**: This table lists all of the related events and models. Tick the checkbox next to each related model you want to sample.
By default, each model will inherit the sampling schedule configured for the parent model. You can override the schedule if you wish.
**_Last run_**: This table lists information about the most recent run for each sampled model. If the sampling job has failed, hover over the status to see the error message.
## Related model
Sampling for [related models](/customer-studio/schema#add-related-models) is controlled through its parent model. However, you can check if the related model is being sampled by going to the Sampling tab:
## Using sampling
After sampling has been enabled and at least one run has successfully completed, you will be able to use the sampled model for certain preview features, such as the ones below.
- [**Audience**](/customer-studio/usage): A toggle will be present that allows you to use the sampled model. The result will also indicate that an approximation was returned.
- [**Trait**](/customer-studio/traits): A similar toggle will show up next to the preview button.
---
## Define your schema
**URL:** https://hightouch.com/docs/customer-studio/schema
**Description:** Create parent, related, and event models so marketers can build audiences in Customer Studio.
**Section:** Customer Studio
| | |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Audience** | Data or analytics engineers, platform admins, technical marketers |
| **Prerequisites** |
A connected data source (e.g., Snowflake, BigQuery, Databricks)
Technical understanding of your data and its relationships
|
_This page covers how to create and configure schema models in Customer Studio. If you haven't decided on your grain, model topology, or naming conventions yet, start with [Plan your data model](/customer-studio/data-model)._
---
## What you'll learn
After reading this article, you'll know how to:
- [Create parent, related, and event models](#create-models)
- [Configure columns](#configure-columns) — visibility, aliases, privacy, semantics, and suggestions
- [Define relationships](#define-relationships) — types, merge columns, and through relationships
- [Label your models](#label-your-models) so the audience builder uses intuitive language
- [Validate your schema](#validate-your-schema) before building production audiences
- [Troubleshoot common schema issues](#troubleshoot-schema-issues) like type mismatches and broken relationships
---
## Overview
Before marketers can build audiences visually, data teams configure the schema in Customer Studio. The schema defines which models, relationships, and events are available in the audience builder.
In Customer Studio, you'll create:
- [**Parent model**](/customer-studio/schema#define-the-parent-model): The base entity audiences are built from (for example, `Users`)
- [**Related models**](/customer-studio/schema#add-related-models): Additional context joined to the parent (for example, `Households`)
- [**Event models**](/customer-studio/schema#add-event-models): Timestamped behaviors (for example, `Product Viewed`)
Once configured, marketers can build audiences using warehouse data. You can also add [**general models**](/customer-studio/general-models) to make additional warehouse tables available for [agent](/agents/overview) queries without including them in the audience schema. If marketers need a reusable calculated value that isn't available as a column — like lifetime value or days since last purchase — create a [trait](/customer-studio/traits).
---
## Create models
### Define the parent model
The parent model is the core dataset for audience building — most commonly a `Users` table, but it could also be `Accounts`, `Households`, or `Devices`.
Before choosing a table, decide what one row should represent. This is your
targeting grain — see [Plan your data
model](/customer-studio/data-model#choose-the-right-grain) for guidance.
**Requirements:**
- One row per entity (for example, one row per user)
- A stable, unique **primary key** (for example, `user_id`)
- Fields useful for filtering (for example, `email`, `region`, `created_at`)
#### Create a parent model
1. Go to **Customer Studio → [Schema](https://app.hightouch.com/schema-v2/view?from=docs)**.
2. Click **Create parent model**.
3. Select a modeling method:
- Table selector (default)
- SQL query
- dbt model
- dbt Cloud model
- Looker model
- Sigma model
4. Preview the results, then click **Continue**.
5. Configure the model:
| Field | Description | Example |
| ------------------------------------------------------------------------- | --------------------------------- | -------------------------------------- |
| **Name** | Display name in Customer Studio | `Users` |
| **Description** | Context for your team | `Represents end users of the platform` |
| [**Primary key**](/models/creating-models#unique-primary-key-requirement) | Unique ID column | `user_id` |
| **Primary label** | Display name in audience previews | `full_name` or `email` |
| **Secondary label** | Additional preview context | `signup_date` |
**Column suggestions** may be available depending on your workspace
configuration. When enabled, they help marketers discover useful filter values
in the audience builder.
6. Click **Create parent model**.
---
### Add related models
Related models join to the parent via a foreign key and provide additional filtering context—like purchases, subscriptions, or devices.
**Examples:**
- Households linked to People by `household_id`
- Subscriptions linked to Companies by `company_id`
#### Create a related model
1. Go to **Customer Studio → [Schema](https://app.hightouch.com/schema-v2/view?from=docs)**.
2. Click the **+** icon next to your parent model.
3. Select **Create related model**.
4. Choose your table or modeling method and preview results.
5. Configure the relationship:
| Field | Description | Example |
| -------------------------- | ---------------------------- | ----------------------- |
| **Name** | Related model name | `Households` |
| **Description** | Context for marketers | `User purchase records` |
| **Relationship** | Cardinality (`1:many`, etc.) | `1:many` |
| **Foreign key (join key)** | Column joining models | `user_id` |
To match on multiple columns (for example, `product_id` and `location_id`), enable **multiple join keys** in the relationship configuration.
The primary key uniquely identifies rows. The join key connects tables.
6. (Optional) If you want to add columns from the parent model onto the related model, turn on [**Merge columns**](#merge-columns-across-models).
7. Click **Create related model**.
---
### Add event models
Event models represent timestamped behaviors like page views, logins, or checkouts. These power both real-time and historical audience logic.
**Requirements:**
- At least one **timestamp column** (for example, `created_at`)
- A **foreign key** linking to the parent model (for example, `user_id`)
#### Create an event model
1. Go to **Customer Studio → [Schema](https://app.hightouch.com/schema-v2/view?from=docs)**.
2. Click the **+** icon next to your parent model.
3. Select **Create related event**.
4. Choose your table or modeling method and preview results.
5. Click **Continue**.
6. Configure the event:
| Field | Description | Example |
| -------------------------- | -------------------------------------- | --------------------------- |
| **Name** | Event name | `Product Viewed` |
| **Description** | Context for marketers | `Tracks user product views` |
| **Timestamp column** | When the event occurred | `created_at` |
| **Event type** | Category (`Generic`, `Checkout`) | `Generic` |
| **Primary key** | For referencing or triggering Journeys | `event_id` |
| **Relationship** | Cardinality to parent | `1:many` |
| **Foreign key (join key)** | Column joining event with parent | `user_id` |
The primary key uniquely identifies a row. The join key links it to another
model. Often, the **primary key in the parent model** becomes the **join key
in a related model**, but they serve distinct purposes.
7. (Optional) If you want to add columns from the parent model onto the event model, turn on [**Merge columns**](#merge-columns-across-models).
8. Click **Create event**.
If all events live in a single table, you can create multiple event models by filtering by event type (for example, `Page View` and `Add to Cart`).
#### Name events for marketers
Use a consistent, human-readable naming pattern. In most cases, `Object + Verb` works well because it groups related actions together in the audience builder:
- `Product Viewed`, `Cart Abandoned`, `Checkout Started`, `Subscription Renewed`
Avoid names that expose implementation details instead of business meaning, like `track_event_v2` or `page_view_prod`. If multiple events come from one event stream, split them into separate event models when marketers will think of them as different behaviors.
---
## Configure columns
Use the `Columns` tab to control which fields appear in Customer Studio and how marketers interact with them.
1. Open the model you want to update.
2. Go to the `Columns` tab.
### Enable or exclude columns
Use the toggle next to each column to control whether it appears in Customer Studio.
Disable columns when:
- the field isn't useful for filtering (for example, internal IDs)
- the field adds noise for marketers
- you don't want it available in audience filters
If a field is useful only for sync mappings or operational workflows, consider hiding it from the audience builder even if you keep it available elsewhere in the schema configuration.
### Rename columns using aliases
Aliases control how columns appear in the audience builder.
For example:
- `geo_region` → `Region`
- `created_at` → `Signup date`
To set an alias, hover over the column and click the pencil icon.
### Set privacy levels for columns
Privacy levels control how column values appear in places like audience previews and profile exploration, while keeping the column available for filtering.
Use privacy levels for sensitive fields like:
- email addresses
- phone numbers
- internal identifiers
Each column in the `Columns` tab has a **Privacy** dropdown. The default value is `Unreviewed`. Select the appropriate privacy level for each column based on its sensitivity.
Set privacy levels before marketers begin building audiences. As a starting point:
- Mark email addresses and phone numbers as **Redacted** if marketers need to filter on them but should not see raw values in previews.
- Mark internal IDs as **Sync-only** or **Sync-blocked** when they are operationally useful but not meaningful for audience building.
- Mark fields as **Approved** only when values are safe to display in previews and profile exploration.
If you're unsure, start more restrictively and loosen visibility later.
### Configure case sensitivity
When enabled for your workspace, new string columns added to a schema model default to **case-sensitive** matching. This means filters like `equals` and `contains` in the audience builder distinguish between uppercase and lowercase values (for example, `"Nike"` and `"nike"` are treated as different values).
This applies to columns added after case-sensitivity was enabled for your workspace. Existing columns retain their previous behavior unless you update them.
If your audience filters depend on case-insensitive matching, review new
columns and adjust settings as needed. Case sensitivity is configured per
column in the `Columns` tab.
### Add descriptions and metadata
Aliases control how columns appear in the audience builder, but marketers also need to understand what values mean. Where possible, add descriptions in your modeling layer (for example, dbt column descriptions) so Customer Studio inherits business context directly. When that's not practical, use the description field on the model.
For columns that need extra context, document:
- **Allowed values** for enums and status fields (for example, `active`, `churned`, `trial`)
- **Units** for numeric fields (for example, whether `amount` is in dollars or cents)
- **Null behavior** — whether null means unknown, not applicable, or genuinely missing
- **Boolean semantics** — whether `true` is a raw data flag or derived logic
### Enable filter value suggestions
Suggestions help marketers choose common values from a dropdown when filtering (for example, brands like `Nike` or `Adidas`).
As a rule of thumb, enable suggestions for columns with fewer than 100 distinct values — fields like `region`, `plan_tier`, or `status`. High-cardinality columns (for example, email addresses or order IDs) don't benefit from suggestions and add unnecessary compute during refreshes.
Suggestions are off by default to reduce query overhead. Enable them selectively where preloaded dropdown values help marketers build filters faster. You can control how often suggestions refresh from the model's [Configuration](#manage-your-schema) settings. For most teams, a weekly refresh is sufficient once the schema is stable — schedule refreshes outside peak analytics hours when possible.
### Refresh columns from your source
If you add or remove columns in your warehouse, you can refresh the columns available in the model.
1. Open the model.
2. Click the three-dot menu.
3. Select **Refresh columns available in source**.
---
## Define relationships
Use the `Relationships` tab to define how models connect in your schema. These relationships power cross-model filtering in the audience builder.
1. Open the model you want to update.
2. Go to the `Relationships` tab.
### Relationship types
Relationships define how rows connect across models:
| Relationship | Use when... | Example |
| :----------- | :-------------------------------------- | :------------------------------------ |
| `1:many` | One parent maps to many related records | One user has many purchases |
| `1:1` | One parent maps to one related record | A company has one active subscription |
| `many:1` | Many records map to one shared record | Many users belong to one household |
Use `1:many` for most behavioral data (for example, purchases, page views). Use `many:1` when referencing shared entities like plans or organizations.
### Validate before saving a relationship
Before saving any relationship, confirm the following:
- Join keys use the **same data type** on both sides. A type mismatch (for example, `integer` on one side and `bigint` on the other) can cause silent join failures.
- The relationship reflects the **actual business cardinality**, not just whatever setting lets it save. A `1:1` relationship on data that's actually `1:many` produces wrong audience counts.
- If using **multiple join keys**, every key pair must match for a row to join. A partial match returns no results.
- After editing or recreating a relationship, re-check any dependent **traits, audiences, or journeys** that referenced it. Recreating a relationship generates a new internal ID, and existing references keep pointing at the old one until you update them.
### Add a relationship
1. Click **Add relationship**.
2. Choose the relationship type (for example, `1:many` or `many:1`).
3. Select the model you want to connect to.
4. Choose join keys for each model.
5. Click **Save**.
### Edit an existing relationship
To update a relationship:
1. Go to the `Relationships` tab.
2. Select the relationship you want to edit (for example, `Purchase History`).
3. Update join keys, the connected model, or cardinality.
4. Click **Save changes**.
### Use multiple join keys
Multiple join keys let you match models on **two or more columns**. This is helpful when you need compound keys (for example, `product_id` and `location_id`).
#### Enable multiple join keys
1. Open the model and go to the `Relationships` tab.
2. Select the relationship you want to edit.
3. Toggle **Multiple join keys** ON.
4. Select the additional join key columns.
5. Click **Save changes**.
### Merge columns across models
Merge columns let you surface fields from a parent model into a related or event model, so marketers can filter on those fields without switching models.
This is useful when:
- the related/event model doesn't include marketer-friendly fields like region or plan name
- you want to reduce duplication across traits or audiences
- marketers need both behavior (events) and customer attributes in the same workflow
Merged columns appear as **read-only** copies in the target model and are filterable like native fields.
#### Merge column directionality
Merge columns copy fields from one side of a relationship into the other. Whether you can merge depends on the cardinality:
- In a **`1:1`** relationship, you can merge in **either direction**.
- In a **`many:1`** relationship, you can merge columns from the `1` side into the `many` side — for example, merging Household columns into Users when many users belong to one household.
- In a **`1:many`** relationship, the merge toggle is **not available**. To merge in this direction, view the relationship from the other model where it appears as `many:1`.
- The **Merge columns** toggle appears on the relationship configuration page. If you don't see it, check the cardinality — merge is only available on `1:1` and `many:1` relationships.
Use merge columns for frequently used context that helps marketers filter more naturally — like adding `region` or `plan_tier` onto purchase records. Don't merge everything from adjacent models; keep it to fields marketers actually need in filters.
If a value only exists on a related or event model and you need it in sync field mappings, you have three options: merge it onto the parent, create a [trait](/customer-studio/traits) that rolls it up, or move the value upstream to the parent model directly.
**Examples:**
- **Users → Purchases** — Merge `geo_region` from Users so marketers can filter Purchases by customer location.
- **Products → Product Viewed events** — Merge `brand` or `category` from Products so marketers can build audiences based on product attributes.
#### Enable merge columns
1. Open the model and go to the `Relationships` tab.
2. Select the relationship you want to edit.
3. Under **Merge columns**, toggle **Merge columns** ON.
4. Choose the columns you want to merge, then click **Save changes**.
If you don't see the merge columns toggle, check the relationship
configuration and cardinality. Merge columns require a relationship where
Hightouch can resolve a single value per row.
### Through relationships
Use a **direct relationship** whenever two models can be joined directly. Use a through relationship only when the model you need is reachable only through an intermediate table. Through relationships are more brittle — if any link in the chain is deleted, recreated, or has its join key changed, the entire path can break without a clear error message.
Through relationships let you connect two models through an intermediate model (a linking table). This is useful for many-to-many scenarios.
**Example:** Users → Memberships → Subscriptions
- Users join to Memberships on `user_id`
- Memberships join to Subscriptions on `subscription_id`
This enables filters like "users with an active subscription" without duplicating logic.
#### Set up a through relationship
Through relationships are configured in the `Relationships` tab of your parent model.
1. Create the two direct relationships:
- Users → Memberships (`1:many`)
- Memberships → Subscriptions (`many:1`)
2. Select your parent model, then open the `Relationships` tab.
3. Click **Add through relationship**.
4. Under **Access**, select the model you want to access (for example, `Products`).
5. In the through dropdown, choose the indirect path that connects your parent model to the target model (for example, `Purchases` → `Products`).
6. Click **Save**.
---
## Label your models
Schema labels tell Hightouch what business entity each model represents, so the audience builder can use more intuitive language. When labels are applied, filters and [insights](/customer-studio/insights) adjust automatically — for example, a People-labeled parent model with a Household-labeled related model lets marketers build audiences using both individual and household-level data.
Label models based on the business entity they represent, not the warehouse table they came from:
- `People` for individual users, leads, or contacts
- `Households` for shared consumer entities
- `Accounts` for companies or organizations
If a model could reasonably be interpreted in multiple ways, choose the label that best matches how marketers will use it in audience building. When you label a model as `Households` or `Accounts`, you'll also need to select a name column (for example, last name for households, company name for accounts).
---
## Validate your schema
After creating your models and relationships, verify the schema works as expected before building production audiences.
### Check primary key uniqueness
Each row in your parent model should have a unique primary key value. Duplicate keys cause audience membership to be evaluated multiple times for the same entity, leading to inflated counts and unexpected sync behavior. Run a quick query in your warehouse to confirm there are no duplicates:
```sql
SELECT primary_key_column, COUNT(*)
FROM your_parent_table
GROUP BY primary_key_column
HAVING COUNT(*) > 1
```
### Confirm join key types match
Open each relationship and verify that the join key columns use the same data type on both sides. A type mismatch (for example, `integer` on the parent and `bigint` on the related model) can cause silent join failures. See [Keep data types consistent](/customer-studio/data-model#keep-data-types-consistent) for details.
### Test with a simple audience
Create a small test audience using one or two filters and check that the results make sense:
- Does the audience count match what you'd expect from a direct warehouse query?
- Can you preview individual members and verify their attributes look correct?
- Do filters on related and event models return the right subset of parent rows?
If counts are off, the issue is usually a grain mismatch or a misconfigured join key. See [Troubleshoot schema issues](#troubleshoot-schema-issues) for common causes.
### Verify through-relationship chains
If you use [through relationships](#through-relationships), test each chain end-to-end. A change to any link in the chain (for example, deleting and recreating one of the intermediate relationships) can break the full path without an obvious error.
---
## Troubleshoot schema issues
#### Audience counts are unexpectedly high or low
**Cause:** The parent model grain doesn't match what you expect. If the parent model is at the account grain but you expected user-level targeting, every qualifying account counts as one row — regardless of how many users belong to it.
**Resolution:** Open the parent model and check what one row represents. If the grain is wrong, switch the parent model to a user-level table or restructure the schema. See [Choose the right grain](/customer-studio/data-model#choose-the-right-grain) for guidance.
#### Relationship setup fails with a type error
**Cause:** A mismatch between the primary key type on one model and the foreign key type on another. For example, an `integer` primary key on the parent and a `bigint` foreign key on a related model can cause join failures.
**Resolution:** Cast both columns to the same type upstream in your warehouse, then open the affected model and select **Refresh columns available in source** from the three-dot menu.
#### Traits or audiences break after schema changes
**Cause:** Renaming columns, changing join keys, or modifying relationships can break traits, audiences, or journeys that reference the old configuration. This is especially common after Git Sync updates that change relationship IDs.
**Resolution:**
1. Go to **Customer Studio → Schema** and check the `Activity` tab on the affected model to see what changed.
2. Open the broken trait or audience and update it to reference the current column or relationship.
3. If the issue is a stale relationship ID (common after Git Sync changes), delete and recreate the relationship.
#### A field exists in the warehouse but doesn't appear in the audience builder
**Cause:** The column may not be added to the schema, may be disabled, or may be on a related or event model instead of the parent model.
**Resolution:** Check these in order:
- Open the model, select the three-dot menu, and choose **Refresh columns available in source**.
- In the `Columns` tab, check that the toggle for the column is on.
- If the field is on a related or event model, it's available for filtering but won't appear directly in sync field mappings unless exposed as a merge column or trait. See [Choose the right model type](/customer-studio/data-model#choose-the-right-model-type) for guidance on where data should live.
#### Errors containing "failed to find direct relationship by ID"
**Cause:** A trait, audience, or journey references a relationship ID that no longer exists. This typically happens when a relationship is deleted and recreated (which generates a new ID), or after a Git Sync update changes relationship IDs.
**Resolution:**
1. Open the affected trait or audience.
2. Remove the filter or condition that references the old relationship.
3. Re-add it using the current relationship from the schema.
4. If the relationship was deleted, recreate it in **Customer Studio → Schema** first.
#### Errors containing "bad relationship data"
**Cause:** This family of errors means the relationship record exists but is inconsistent or incomplete. You may see variants like "bad relationship data, cycle detected," "bad relationship data, no rows returned," or "bad relationship data, no path." Common causes include a partial save that was interrupted, or a relationship that was modified without saving all required fields.
**Resolution:** Delete the broken relationship and recreate it from scratch. Verify the join key columns exist on both models and use the same data type before saving.
#### Through-relationship chain breaks after a link changes
**Cause:** Through relationships depend on a chain of direct relationships. If any link in the chain is deleted, recreated, or has its join key changed, the through relationship can break without a clear error message.
**Resolution:**
1. Open the parent model's `Relationships` tab and check whether the through relationship still shows the correct path.
2. If the path is broken, delete the through relationship and recreate it after verifying both direct relationships in the chain are intact.
#### Upstream column changes aren't reflected in Customer Studio
**Cause:** Customer Studio doesn't automatically detect new or renamed columns in your warehouse.
**Resolution:**
1. Open the affected model in **Customer Studio → Schema**.
2. Select the three-dot menu and choose **Refresh columns available in source**.
3. Update any column aliases if the underlying column name changed.
---
## Manage your schema
### Edit a model
To manage models in your schema:
1. Go to **Customer Studio → [Schema](https://app.hightouch.com/schema-v2/view?from=docs)**.
2. Select a model to open its details page.
3. Use the tabs across the top to configure the model.
If your team uses Git for version control, you can manage your Customer Studio
schema—including parent models, related models, event models, and
relationships—using [Schema Git
Sync](/extensions/git-sync#customer-studio-schema). Note that some
UI-configured settings are not synced. See [Git Sync limitations for Customer
Studio
schema](/extensions/git-sync#git-sync-limitations-for-customer-studio-schema)
for details.
### Delete a model
1. Open the model you want to remove.
2. Click the three-dot menu, then select **Delete**.
Deleting a model removes it from the audience builder. Any audiences using it
will break and must be updated.
### Enrich match rates
Match Booster enhances match rates by enriching your schema with additional identifiers from Hightouch's identity graph.
1. Open your **parent model**.
2. Go to the `Match Booster` tab.
3. Toggle **Enable Match Booster**.
4. Select one or more identifier columns (for example, email or phone).
5. Click **Initialize Match Booster** to start enrichment.
Learn more: [Match Booster overview](/match-booster/overview)
### Speed up previews with sampling
Sampling creates a smaller subset of your model data to improve performance when previewing audiences.
1. Open your model.
2. Go to the `Sampling` tab.
3. Turn on sampling and configure the settings.
4. Click **Save & run sampling**.
Learn more: [Sampling](/customer-studio/sampling)
### Review model activity
Use the `Activity` tab to review recent changes and understand how a model is being used.
You can track:
- recent updates to the model definition
- audiences, traits, or syncs that reference the model
- usage history for debugging and cleanup
### Update model settings
Use the `Configuration` tab to manage core model settings such as primary key, preview labels, and column suggestion refresh interval.
#### Column suggestion refresh interval
Hightouch can automatically refresh filter suggestions so dropdown values stay up to date as your warehouse data changes.
1. Open your model.
2. Go to the `Configuration` tab.
3. Under **Column suggestion refresh interval**, choose how often suggestions should update.
This setting controls how frequently Hightouch updates **suggested dropdown values** in the audience builder.
It does **not** affect:
- live audience results
- sync behavior
- warehouse queries (these always use your source-of-truth data)
Refreshing columns updates the fields and **dropdown values** shown in
filters. It does not change your warehouse data or audience results.
---
## What's next?
After defining your schema, choose the next step based on your role.
#### If you're setting up agents
Add warehouse tables that agents should be able to query but that don't belong in the audience schema:
- [**General models**](/customer-studio/general-models)
Make tables like sales data, product catalogs, or competitive benchmarks available for agent queries.
#### If you're managing governance and delivery
Set up rules that control how data flows to destinations and how consent is enforced:
- [**Destination rules**](/customer-studio/destination-rules)
Define sync behavior, privacy policies, and delivery restrictions per destination.
- [**Subsets**](/customer-studio/subsets)
Apply destination-specific filters (for example, opt-outs) without changing core audience logic.
- [**OneTrust Snowflake Native App**](/customer-studio/onetrust)
Enforce consent policies using Snowflake-native integration with OneTrust.
#### If you need reusable calculated values
If marketers need values that aren't stored as columns — like total spend, days since last login, or most frequent category — create them as traits:
- [**Traits**](/customer-studio/traits)
Build reusable computed fields that appear in audience filters and sync field mappings.
For guidance on when to use a trait vs. keeping logic upstream, see [When to model something as a trait](/customer-studio/data-model#when-to-model-something-as-a-trait).
#### If you're a marketer
Once the schema is in place, you can begin building audiences:
- [**Audiences**](/customer-studio/usage)
Use filters, traits, and events to define dynamic segments directly from your data.
---
## Configure Schema Git Sync
**URL:** https://hightouch.com/docs/customer-studio/schema-setup
**Description:** Connect your Customer Studio schema to a Git repository for version control, consistency, and collaboration across environments.
**Section:** Customer Studio
| | |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | IT, DevOps, Data Engineering, or Analytics |
| **Prerequisites** |
Access to a Hightouch workspace
A Git repository (GitHub, GitLab, etc.) for schema storage
[Git Sync](/extensions/git-sync) enabled in your workspace
Appropriate permissions for Git and Hightouch
|
_**Schema Git Sync** extends [Git Sync](/extensions/git-sync#customer-studio-schema), Hightouch’s integration for version-controlling resources in Git, to include **Customer Studio schema models, events, and relationships**. Use it to keep schemas consistent across environments (for example, staging and production) and to version, review, and recover changes directly from your Git repository._
## Overview
**Schema Git Sync** connects your Customer Studio schema—parent, related, and event models, plus their relationships—to a Git repository.
Whenever you update your schema in Hightouch, the changes are automatically written to YAML files in Git, and vice versa.
This gives your team all the benefits of **version control** for your customer data model:
- Every schema change is captured as a Git commit.
- You can roll back to earlier versions if needed.
- Teams can review and approve schema updates using pull requests.
- Environments like _staging_ and _production_ can stay perfectly aligned.
Think of it as turning your schema into **infrastructure as code**: you define your data model in Hightouch, but track, share, and deploy it safely through Git.
For authentication, repository setup, and permissions, see the [main Git Sync guide →](/extensions/git-sync#setup).
---
## Setup overview
Schema Git Sync setup involves these main steps:
1. [Set up and enable Git Sync in your workspaces](#step-1--set-up-workspaces)
2. [Create your Git repository](#step-2-create-your-git-repository)
3. [Connect Git to each workspace](#step-3-connect-git-to-each-workspace)
4. [Configure branches and paths](#step-4-configure-file-paths-branches)
5. [Build your schema in Customer Studio](#step-5-create-your-schema)
6. [Sync schema changes to Git](#step-6-sync-schema-to-git)
7. [Validate and troubleshoot syncs](#validation-checklist)
For repository authentication and credential setup, see the [main Git Sync guide →](/extensions/git-sync#setup).
---
## Step 1 – Set up workspaces
Git Sync is behind a feature flag. Ask your Hightouch account team to enable
it in each workspace you plan to sync.
Create or identify the Hightouch workspaces you’ll connect to Git.
Each workspace typically represents an environment (for example, _staging_ and _production_).
Clone your data source into each workspace to maintain identical schemas
across environments.
---
## Step 2 – Set up your Git repository
Create a dedicated Git repository for your schema files.
Hightouch uses this repo to store YAML definitions for all models, events, and relationships.
**Automatically generated repository structure:**
```bash
/syncs/ # YAML files for all syncs
/models/ # YAML files for all models
/schema/ # Customer Studio schema files
├── event-models/ # Event model definitions
├── related-models/ # Related model definitions
├── parent-models/ # Parent model definitions
├── model-relationships/ # Relationship model definitions
manifest-*.yaml # Auto-generated source/destination listings
```
For detailed repo setup and authentication, follow the [Git Sync configuration
steps](/extensions/git-sync#configure-git-sync).
---
## Step 3: Connect Git to each workspace
Configure your Git repository connection from Hightouch.
1. In Hightouch, go to **Integrations → [Extensions](https://app.hightouch.com/extensions?from=docs)** and select **Version control with Git**.
2. Under **Overview**, click **Configure extension**.
3. Under **Git credentials**, click **Set up** and select your Git service (e.g., GitHub App).
4. Follow the [**Git Sync Authentication instructions**](/extensions/git-sync#authenticate-to-git) to authenticate and grant Hightouch permission to your repository.
Once connected, Hightouch displays your sync status and automatically detects inbound and outbound changes.
{" "}
Branches do not need to exist beforehand. Hightouch creates them automatically
during setup.{" "}
---
## Step 4: Configure branches (optional)
### Use the same branch (recommended)
For most teams, point all workspaces at the same branch (for example, `main`). Both workspaces read from and write to the same YAML files, and [alias files](/extensions/git-sync#aliases) handle any differences in source or destination IDs. This is the simplest setup and works well when all workspaces share the same schema and parent model structure.
### Use separate branches
Only use separate branches if your environments intentionally diverge — for example, if your staging workspace includes additional parent models or experimental relationships that should not reach production yet.
**Example mapping (for divergent environments):**
| Environment | Workspace | Git Branch |
| ----------- | ---------------------- | ------------------- |
| Staging | `hightouch-staging` | `schema-staging` |
| Production | `hightouch-production` | `schema-production` |
Promote changes by merging from your staging branch into production via pull request. Use [CI checks](/extensions/git-sync#validating-changes) to validate changes before they reach production.
For a full walkthrough of connecting staging and production workspaces to the
same repo, including alias files and branch strategies, see [Connect staging
and production workspaces
→](/extensions/git-sync#connect-staging-and-production-workspaces). For more
about branch and path setup, see [Configure Git Sync
→](/extensions/git-sync#configure-git-sync).
---
## Step 5: Create your schema
Build your schema models in Customer Studio.
**Define:**
- **Parent models** (e.g., `Users` table)
- **Related models** (e.g., `Purchases`, `Subscriptions`)
- **Event models** (e.g., `Page Views`, `Clicks`)
- **Relationships** between models
Each model and relationship automatically becomes a YAML file in your Git repository under `/schema/`.
**Example workflow:**
1. Create a `Users` parent model.
2. Add a related model such as `Purchases`.
3. Define a relationship between them (e.g., “Users → Purchases”).
4. Save — Hightouch commits each definition as YAML in Git.
Follow step-by-step instructions to define your schema in Customer Studio:
[Define data schema →](/customer-studio/schema)
---
## Step 6 – Sync your schema to Git
Once Git Sync is enabled and YAML generation is turned on, Hightouch automatically performs an **initial full sync** of your schema. This first sync creates YAML files for your models, events, relationships, alias files, and manifest files in your Git repository.
After the initial setup, any changes you make in Customer Studio—such as creating or editing models or relationships—are automatically committed to Git within about 30 seconds.
You can monitor progress in the **Runs** tab of the Git Sync extension or by checking your Git commit history.
### How syncing works
- **Initial sync:** Runs automatically when Git Sync is first enabled.
- **Automatic updates:** Every schema change in Customer Studio commits automatically to Git within about 30 seconds.
- **Two-phase commits:** Hightouch commits models first, then relationships, to prevent dependency errors.You don’t need to manage this sequencing manually; Hightouch handles it automatically during both manual and automatic syncs.
- **Full resyncs:** You can trigger a full resync manually if you change repository or branch settings, or to troubleshoot issues.
Large initial syncs may take 10–20 minutes depending on schema size.
### What your YAML files look like
Once synced, each schema component appears as a YAML file in your repository under `/schema/`. The structure of each file depends on the model type.
For full YAML field references, see the [Customer Studio Schema section of the Git Sync guide](/extensions/git-sync#customer-studio-schema). Key things to know:
- **Parent models** follow the [standard model schema](/extensions/git-sync#model-schema), with optional `primaryLabel` and `secondaryLabel` fields.
- **Related models** follow the [standard model schema](/extensions/git-sync#model-schema).
- **Event models** follow the [standard model schema](/extensions/git-sync#model-schema), with a required `eventTimestampColumn` field.
- **Relationships** have a more complex schema depending on whether they are direct or [through relationships](/extensions/git-sync#model-relationship-schemas).
The easiest way to learn the YAML schema for any model type is to create it in
the Hightouch UI first and inspect the resulting file in your Git repository.
Field names in YAML files are case-sensitive. If a column name in your
warehouse changes casing (for example, `user_id` to `User_Id`), Git Sync
treats these as different fields and may cause unexpected behavior.
---
## Step 7 - Two-phase commit process
Schema Git Sync automatically commits schema changes in two stages to prevent dependency errors.
| Phase | Description |
| --------------------- | -------------------------------------------------------------------------- |
| **1 – Models** | Hightouch commits model definitions first to register columns and sources. |
| **2 – Relationships** | Relationships commit after models exist, ensuring valid references. |
This process happens automatically during both manual and automatic syncs.
---
## Validation Checklist
Before your first sync:
**Feature setup**
- Git Sync feature flag is enabled for each workspace.
- Each workspace includes an alias file (`aliases-{workspace-slug}.yaml`).
- Branches or paths are unique only if environments diverge (for example, different parent models).
**Schema setup**
- All workspaces use identical model slugs if they share the same schema YAMLs.
- Source and destination slugs can differ; Hightouch manages these automatically through [alias files](#resource-slug-consistency-and-aliases).
- Data sources are cloned consistently across environments.
{" "}
If these prerequisites aren’t met, your sync may fail with `error serializing resource`.{" "}
{" "}
If model slugs don’t match across workspaces, Hightouch can’t link shared schema
YAMLs correctly. Alias files handle differences for sources and destinations, but
not for models.{" "}
---
## Resource slug consistency and aliases
Hightouch uses **resource slugs**—unique identifiers for each model, source, or destination—to link schema YAML files in Git with the corresponding resources in each workspace connected to Git Sync.
To maintain consistency across environments:
- **Model** and **event contract** slugs must be identical across workspaces sharing the same repository.
- **Source** and **destination** slugs can differ between environments because Hightouch automatically handles those differences through **alias files**.
Alias files act as a translation layer between environments. When your staging and production workspaces use different data sources or destinations, these files ensure the same schema YAMLs continue to work correctly in both.
### How alias files work
Each workspace connected to Git Sync automatically gets its own alias file in your Git repository:
```yaml
aliases-{workspace-slug}.yaml
```
{" "}
You don’t need to create these files manually—Hightouch automatically generates
an empty alias file if one doesn’t exist.{" "}
If your workspaces use different data sources or destinations, (for example, `snowflake-dev` in staging vs. `snowflake-prod` in production), you can edit these files to define the correct mappings for each environment.
**Example structure:**
```yaml
sources:
main-database: snowflake-dev
destinations:
crm-platform: salesforce-sandbox
```
- **Keys** (for example, `main-database`) are shared alias names referenced in your schema YAMLs.
- **Values** (for example, `snowflake-dev`) are the actual resource slugs in that specific workspace.
This lets all environments share the same schema YAMLs while Hightouch automatically resolves the right connections in each workspace.
#### Example: staging vs. production
```yaml
# aliases-staging.yaml
sources:
main-database: snowflake-dev
destinations:
crm-platform: salesforce-sandbox
# aliases-production.yaml
sources:
main-database: snowflake-prod
destinations:
crm-platform: salesforce-prod
```
Both workspaces use the same schema YAMLs, while each resolves to its own data sources and destinations.
---
## Troubleshooting
If a sync fails with an error like:
```go
error serializing resource
```
#### Meaning
Git Sync tried to serialize (save) a schema component to YAML but couldn’t find a matching resource in the workspace.
#### Common causes
- Mismatched model or source slugs between environments
- Missing or incorrect alias mappings
- Workspace connected to the wrong Git branch
- Schema synced before alias setup
#### Fix checklist
1. Verify model and source slugs match across environments.
2. Confirm alias files map correctly to workspace resource IDs.
3. Ensure each workspace is using its correct Git branch.
4. Trigger a Full resync after correcting slugs or aliases.
For more help, see the [Git Sync troubleshooting guide](/extensions/git-sync#faq) or contact Hightouch Support.
---
## Best practices
- Plan workspace and branch structure before syncing.
- Test schema changes in staging before merging to production.
- Use pull requests to review schema changes before deployment.
- Keep model and event contract (event schema) slugs identical across all workspaces sharing the same repository.
- Periodically audit alias files and manifests after cloning or renaming resources.
- Document your branch-to-workspace mapping for future maintainers.
{" "}
For schema modeling guidance, see [Schema Management in Customer Studio](/customer-studio/schema).{" "}
---
## Related resources
- [Git Sync Overview](/extensions/git-sync)
- [Schema management in Customer Studio](/customer-studio/schema)
- [Sync monitoring](/customer-studio/syncs#monitor-syncs#monitor-syncs)
- [Warehouse sync logs](/syncs/warehouse-sync-logs)
---
## Subsets
**URL:** https://hightouch.com/docs/customer-studio/subsets
**Description:** Subsets allow organizations to segment their total addressable audience data based on its attributes and then grant employees access to that data based on the segment.
**Section:** Customer Studio
| | |
| ----------------- | -------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Platform admins |
| **Prerequisites** | [Set up for platform admins and data teams →](/customer-studio/quick-start#setup-for-platform-admins-and-data-teams) |
_Subsets restrict which rows of customer data users can access in Customer Studio. Admins define subset filters based on attributes like region, brand, or consent type, then assign access to specific teammates. All audiences built by those teammates are automatically scoped to the rows they’re permitted to see._
---
## What you'll learn
After reading this article, you’ll be able to:
- Understand how subsets restrict data access in Customer Studio
- Create and configure subset categories and subsets
- Assign users to specific subsets
- Apply subsets in the audience builder and enforce base filters
---
## Overview
Subsets allow organizations to segment their total addressable audience data
based on its attributes and then grant employees access to that data based on
the segment. For example, some organizations may have a Global Customers table
but only want employees to be able to access data in Hightouch from the region
in which they work. By creating a subset for each region, companies can assign
specific employees to each region and limit the data each employee can view and
activate.
### Common use cases for subsets
- **Consent**: Some companies may collect consent or opt-ins for different marketing use
cases. Companies can use subsets to create different filters for each
consent type. For example, lifecycle marketers may be assigned to an “Email
Opt-Ins” subset, while performance marketers may be assigned to a subset for
“Targetable Customers.”
- **Region**: In multinational corporations, subsets can help scope employee data access
to data within a particular geography or region.
- **Brand**: For companies with multiple business units, brands, or product lines,
subsets can be applied so that employees can only create audiences with data
scoped to their specific business unit, brand, or product.
### Subset categories vs. subsets
A [parent model](/customer-studio/schema) can be segmented with subset categories and subsets. In the above
example, region is the subset category and each region (North America, EMEA,
APAC) is the subset.
When building an audience, you can select more than one subset category and more
than one subset within each subset category. A user must be part of any subset
within a subset category, and satisfy all subset categories to be part of the
audience.
## Set up subsets
### 1. Navigate to the Governance page
Navigate to **Customer Studio > Governance**, then open the [**Subset categories** tab](https://app.hightouch.com/governance/subset-categories?from=docs).
### 2. Select a parent model
Select your **parent model** at the top.
### 3. Create a subset category
Click **Add subset category**, name the category (e.g., `Region`, `Consent`), and optionally add a description.
Once created, you'll see the category appear in the list.
## 4. Set category requirements (optional)
Click into the new category and toggle **Required** if you want users to be required to select a subset from this category when creating an audience.
## 5. Add subsets to the category
Click **Add subset** to define specific filters for that category (e.g., `Australia`, `Canada`, `Email Opt-In`, `Targetable Customers`).
1. Name your subset
2. Click **Add filter** to define who is included
3. Select which users should have access
To create your subset, you can mix and match different conditions to segment
your parent model to a specific filter. Similar to building an audience, the
builder provides five types of filters you can use
with nested Boolean (`AND` / `OR`) logic.
- Property conditions
- Related model conditions
- Event conditions
- Audience conditions
- Trait conditions
For help using the builder UI, see [Add filters →](/customer-studio/usage#step-2-add-filters).
## 5. Grant access to users
Once your filter conditions have been created, select which users should have
access to this subset.
## 6. Save subset
Click **Create subset**
## Use subsets in audience creation
When creating or editing an audience, users will see any required categories and can choose one or more subsets they have access to. Subset filters will be applied as base filters on top of the audience definition.
If a user does not have access to any subsets in a required category, they won't be able to build audiences using that parent model.
## Related articles
- [Create audiences →](/customer-studio/usage)
- [Destination rules →](/customer-studio/destination-rules)
---
## Sync audiences
**URL:** https://hightouch.com/docs/customer-studio/syncs
**Description:** Sync audiences to downstream destinations using Hightouch's sync templates.
**Section:** Customer Studio
| | |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers, Platform admins |
| **Prerequisites** |
A saved [audience](/customer-studio/usage)
At least one [destination](/getting-started/concepts#destinations)
|
_Once you’ve [built and saved an audience](/customer-studio/usage), you can sync it to destinations—like CRMs, ad platforms, or email service providers. Hightouch keeps those destinations up to date by automatically adding or removing users as the audience changes._
---
## What you'll learn
After reading this article, you’ll know how to:
- Sync a saved audience to a connected destination
- Configure sync frequency, identifier fields, and removal settings
- Monitor and manage sync status
- Apply sync templates to save time
---
## Overview
Once you've [created an audience](/customer-studio/usage), the last step is to sync it to your desired destination(s).
For example, say you've created an audience of users who've added a particular product type to their cart without purchasing it. You likely want to sync this audience to ad platforms like Meta or Google so you can target them with personalized ads about the product in their cart.
Sync configuration for an audience is the same as any other sync configuration. You need to [map fields](/syncs/mapping-data), [schedule](/syncs/schedule-sync-ui) the sync, and [set up alerts](/syncs/alerting) as desired.
Most syncs run on a set schedule and send the full audience to your destination (e.g., hourly CRM updates or daily data warehouse exports). For event-driven campaigns—like retargeting users after a checkout or suppressing ads after a purchase—Hightouch also supports real-time syncs that deliver users the moment they enter or exit an audience.
{" "}
To learn more about syncing audiences in **real time**, see [Real-time audiences](/real-time/overview).{" "}
## Add a sync to an audience
To create a sync for an existing audience:
1. Go to **Customer Studio → [Audiences](https://app.hightouch.com/audiences?from=docs)**
2. Open a saved audience and click **Add sync**
3. Select a **destination**
Refer to your chosen [Destination docs](/destinations/acoustic) for specific configuration details.
For more information on creating syncs, check out the [sync configuration docs](/syncs/overview).
## Monitor syncs
After setup, you can monitor and manage your syncs from the Syncs tab of each audience.
Each sync shows:
- **Destination name**
- **Sync status** (e.g., active, paused, failed, warning)
- **Last run time**
- **Sync frequency**
- **Matched users** — an approximate count of users successfully matched in the destination
- **Last updated date**
- A **button to manually run the sync**
You can pause, resume, or edit any sync at any time.
## Use sync templates
If your workspace includes [**sync templates**](/customer-studio/templates#sync-templates), you can apply one when creating a new sync. Templates help standardize common configurations across destinations or teams.
Templates may pre-fill values like:
- Destination
- Identifier field
- Field mappings
- List name
- Removal behavior
{" "}
Learn how to create and apply [Sync templates](/customer-studio/templates#create-a-sync-template).{" "}
## Related articles
- [Create audiences →](/customer-studio/usage)
- [Real-time audiences →](/real-time/overview)
- [Syncs overview →](/syncs/overview)
---
## Templates
**URL:** https://hightouch.com/docs/customer-studio/templates
**Description:** Templates let you reuse common logic across audiences, syncs, and journeys.
**Section:** Customer Studio
| | |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Audience** | Platform admins, Marketers |
| **Prerequisites** |
A saved [audience](/customer-studio/usage)
A [configured sync](/customer-studio/syncs)
Or a [saved journey](/customer-studio/journeys)
|
_Templates let you reuse common logic across audiences, syncs, and journeys--helping you reduce manual setup—especially for recurring targeting strategies or multi-team workflows._
---
## What you'll learn
After reading this article, you'll know how to:
- Create audience, sync, and journey templates
- Apply templates to new audiences, syncs, or journeys
- Standardize workflows across teams
---
## Types of Templates
Templates are reusable components in Customer Studio. There are three types:
| Template type | What it stores | Where it’s used |
| ------------------------------------------------------------------------------------------------ | ------------------------------------------------------ | ---------------- |
| [**Audience templates**](http://hightouch.com/docs/customer-studio/templates#audience-templates) | Filter conditions, traits, and event logic | Audience builder |
| [**Sync templates**](http://hightouch.com/docs/customer-studio/templates#sync-templates) | Destination, identifiers, sync frequency, and behavior | Sync setup |
| [**Journey templates**](http://hightouch.com/docs/customer-studio/templates#journey-templates) | Workflow steps and logic | Journey builders |
Templates are workspace-wide and can be created by platform admins or users
with appropriate permissions.
---
## **Use cases**
| Use case | Example |
| --------------------------- | --------------------------------------------------------------------------------- |
| **Standardized targeting** | Create an audience template for “High-value cart abandoners” used across channels |
| **Faster setup** | Prefill sync fields for destinations with consistent matching settings |
| **Cross-team coordination** | Let marketers safely reuse logic built by analysts or engineers |
---
## Audience templates
### Create an audience template
1. Go to **Customer Studio → Templates → [Audience templates](https://app.hightouch.com/templates/audience-templates?from=docs)**
2. Click **Add audience template**
3. Select a parent model
4. Define filters, traits, or events like you would when [building a normal audience](/customer-studio/usage)
- Leave blank fields where customization is expected—users will be prompted to fill these when applying the template
5. Name the template and optionally add a description
6. Click **Finish**
### Apply an audience template
To use an audience template:
1. Go to **Customer Studio → [Audiences](https://app.hightouch.com/audiences?from=docs)**
2. Click **Add Audience** and select a parent model
3. Open the 3-dot menu and click **Add a template** (or select a template beneath **Start with an audience template**)
4. Select a template with a matching parent model
5. Fill in any required blanks (highlighted in blue)
6. Save and finish editing as needed
You can apply more than one audience template to the same audience. Filter
logic from each will be combined.
---
## Sync templates
When managing many audiences, it can become tedious to recreate the same sync configuration each time. For example, you may want to sync several [Lookalike audiences](https://hightouch.com/blog/driving-paid-media-roas-through-audience-activation#lookalike-audiences) to Meta. Hightouch offers a tool called **sync templates** to streamline your audience sync setup.
Sync templates let you define sync configuration settings _once_ for a particular destination. You create the sync template and then use it to add syncs for each audience requiring the same configuration settings and schedule. You can also use multiple templates for an audience if you're sending it to several destinations.
### Create a sync template
1. Go to **Customer Studio → Templates → [Sync templates](https://app.hightouch.com/templates/sync-templates?from=docs)**
2. Click **Add sync template**
3. Choose a **parent model**
4. Select a **destination** and configure the sync settings.
Follow individual [destination configuration instructions →](http://hightouch.com/docs/destinations/acoustic)
5. Click **Save**
You can now use the sync template for multiple audience syncs.
### Apply a sync template during audience creation
1. Build a new audience and proceed through:
- **Select parent model**
- **Define audience**
- **Configure audience details**
2. On the **Sync to destination (optional)** step, select one or more matching sync templates
3. Click **Finish**, or **Save audience without syncs** to skip for now
### Apply a sync template to an existing audience
Audiences can only use sync templates derived from the same parent model as
the audience.
1. Go to **Customer Studio → [Audiences](https://app.hightouch.com/audiences?from=docs)**
2. Select an audience
3. Click **Add** and choose **Sync** from the dropdown
4. In the modal, select one or more **Sync templates**
5. Click **Add syncs**
The sync inherits configuration and scheduling from the template. If the template changes, those changes propagate to all derived syncs.
### Require sync templates for audience activation
Workspace admins can optionally enforce the use of sync templates for audience-based activation.
To enable this:
1. Go to **Settings → Workspace → Workspace**.
2. Turn on **Require sync templates for audience activation**.
3. Click **Save changes**.
When this setting is enabled:
- Users can **only** add syncs to audiences using **sync templates**.
- Users **cannot detach** audience syncs from their templates.
- If no sync templates exist for a workspace, users won’t be able to add new audience syncs until at least one template is created.
This setting is useful when you want to standardize identifiers, enforce consistent mapping/consent handling, and prevent one‑off audience sync configurations.
### Apply a sync template in journeys
1. Build your journey and add a **Send to destination** tile to your canvas.
2. When configuring your sync, select a template from the dropdown
3. Fill in any of the fields that were left blank in the template, or make edits to fields that are unlocked.
4. Save your sync and journey.
This sync inherits configuration from the template. If the template changes, those changes will propagate to the journey syncs.
Journey Setting Configurations (Trigger or Cohort mode) on the template do
**not** propagate to the syncs using the template. If you make a change to
this setting, you'll need to update the syncs individually.
### Unlock sync template fields
You can configure templates to leave certain fields “unlocked,” allowing users to customize them per sync.
Supported destinations include:
- [Adobe Campaign Classic](/destinations/adobe-campaign-classic)
- [Amazon Ads](/destinations/amazon-ads)
- [Braze](/destinations/braze)
- [Braze Cohorts](/destinations/braze-cohorts)
- [Criteo](/destinations/criteo)
- [Display & Video 360](/destinations/display-video)
- [Meta Custom Audiences](/destinations/facebook)
- [Google Ads](/destinations/google)
- [Google Ad Manager 360](/destinations/gam360)
- [Google Partner Audiences](/destinations/google-partner-audiences)
- [Google Sheets](/destinations/google-sheets)
- [HTTP Request](/destinations/http-request)
- [Iterable](/destinations/iterable)
- [LinkedIn](/destinations/linkedin)
- [Marigold Engage](/destinations/marigold-engage)
- [Marigold Engage Plus](/destinations/marigold-engage-plus)
- [Magnite](/destinations/magnite)
- [Marketo](/destinations/marketo)
- [Microsoft Bing Ads](/destinations/bingads)
- [Pinterest Ads](/destinations/pinterest-ads)
- [Reddit Ads](/destinations/reddit-ads)
- [S3](/destinations/s3)
- [Salesforce Marketing Cloud](/destinations/sfmc)
- [Snapchat](/destinations/snapchat)
- [Talon.One](/destinations/talon-one)
- [TikTok](/destinations/tiktok)
- [The Trade Desk](/destinations/tradedesk)
- [Upland Waterfall](/destinations/upland-waterfall)
- [X Ads](/destinations/x)
- [Yahoo](/destinations/yahoo)
- [Yahoo DataX](/destinations/yahoo-datax)
**To unlock a field:**
1. Open the sync template configuration
2. Toggle **Lock this field** off
Some fields are dependent on others. Hover over the toggle to see dependencies.
Once the template is attached to an audience, you can override unlocked fields.
You can update these overrides at any time from the sync configuration.
---
## Journey templates
### Create a journey template
1. Go to **Customer Studio → Templates → [Journey templates](https://app.hightouch.com/templates/journey-templates?from=docs)**
2. Click **Add a journey template**
3. Select a parent model
4. Enter a template name and optionally add a description
5. Click **Create template**
### Use a journey template
1. Go to **Customer Studio → [Audiences](https://app.hightouch.com/audiences?from=docs)**
2. Select an audience
3. Click **Add** and choose **Journey** from the dropdown
4. In the modal, enter a **name** and optional **description**
5. Click **continue**
You’ll be taken to the journey builder with the selected template pre-applied.
---
## Best practices
- Use templates for commonly used audiences, especially suppression groups or VIP cohorts
- Leave placeholders in audience templates where campaign owners should customize filters
- Use sync templates to enforce consistent identifiers and opt-out handling across destinations
- Name templates clearly by purpose (e.g., “Klaviyo cart abandon sync” or “Loyalty email segment”)
---
## Related articles
- [Create an audience →](/customer-studio/usage)
- [Set up a sync →](/customer-studio/syncs)
- [Build a journey →](/customer-studio/journeys)
---
## Traits
**URL:** https://hightouch.com/docs/customer-studio/traits
**Description:** Create reusable calculated values — like lifetime value, purchase count, or days since last order — and use them in audience filters and sync field mappings.
**Section:** Customer Studio
| | |
| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Data teams, marketers |
| **Prerequisites** |
A [defined schema](/customer-studio/schema) with at least one parent model.
(Optional) Connected destinations for syncing audiences.
|
_Traits are reusable calculated values you can build on top of your existing data. Use them when you need a value that isn’t already stored as a field in your tables — like lifetime value, number of purchases, or days since last order. Once created, a trait becomes available in the audience builder so your team can use it in filters without recreating the same logic each time._
---
## What you’ll learn
- [What traits are and when to use them](#when-to-use-a-trait)
- [Who creates and maintains traits](#who-creates-and-maintains-traits)
- [How to create a trait](#create-a-trait)
- [How to use traits in audience filters](#create-audiences-with-traits)
- [How to sync trait values to destinations](#enrich-audiences-with-traits)
---
## When to use a trait
Use a trait when the value you need for filtering or syncing doesn’t exist as a column on your parent model and needs to be computed from related or event data. Common examples include total spend, days since last purchase, most viewed product category, and number of support tickets.
Traits are the right choice when multiple audiences will use the same calculation — define it once and reuse it across audiences. If the logic is a core business metric your team governs centrally (like official LTV or churn score), keep it upstream in your warehouse instead and expose it as a parent model column.
For a full comparison of traits vs. other modeling options, see [Choose the right model type](/customer-studio/data-model#choose-the-right-model-type) and [When to model something as a trait](/customer-studio/data-model#when-to-model-something-as-a-trait).
---
## Who creates and maintains traits
Traits are typically created and maintained by data teams or schema owners, since they involve choosing the right source columns and aggregation logic. If you’re a marketer and the trait you need doesn’t exist, work with your data team to decide whether it should be added as a trait or modeled upstream.
---
## Trait types
| **Trait Type** | **Description** | **Example Usage** |
| ------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
| [**Aggregation**](/customer-studio/traits#aggregation-traits-sum-average) | Calculates the `sum` or `average` of a numeric field across a related or event model. | Total amount a user has spent across all purchases (`order_value`) |
| [**Count**](/customer-studio/traits#count-traits) | Counts the number of records that match optional filters. | Number of emails a user opened in the past 30 days |
| [**Occurrence**](/customer-studio/traits#occurrence-traits-first-last-most-frequent) | Returns the first, last, most frequent, or least frequent value from a column in related data. | Most frequently viewed product category or last brand interacted with |
| [**List**](/customer-studio/traits#list-traits) | Returns an array of values or objects from related or event records, with optional sorting, slicing, and filters. | List of SKUs a user has browsed or added to cart in the past 7 days |
| [**SQL Aggregation**](/customer-studio/traits#sql-aggregation-traits) | Uses SQL to calculate a value across related records. Supports advanced logic like JSON or arrays. | JSON object of purchased products with price and quantity, for use in dynamic email personalization |
| [**SQL Formula**](/customer-studio/traits#sql-formula-traits) | Computes logic using SQL on model-level fields without joins. Ideal for string manipulation or flags. | Boolean trait for whether a user is in a loyalty program (`membership_tier = 'Gold'`) |
Traits can be defined on any type of model, though in most use cases they are defined on the parent model.
---
## Create a trait
You can create a trait in one of three ways:
**Option 1:**
1. Go to **Customer Studio → [Traits](https://app.hightouch.com/traits?from=docs)**
2. Click the **Create** dropdown and select **New trait**
**Option 2:**
Select **Start with a template** to use a predefined calculation
**Option 3:**
Create a custom trait while building an audience using the Custom traits filter group.
---
## Configure traits
### Step 1: Method
In the **New trait** flow, the first step is choosing the logic type.
1. Select a model. Typically this will be your parent model, but you can also create traits on a related or event model.
2. Select the calculation method.
**Available calculation methods:**
- **Aggregation:** Sum or average across rows
- **Count:** Total number of matching events
- **Occurrence:** First, last, most frequent, or least frequent value
- **List:** Return an array of values or objects
- **SQL Aggregation:** SQL for grouping/array logic
- **SQL Formula:** SQL expressions based on parent properties
---
### Step 2: Calculation
In Step 2, you'll configure the logic for your selected calculation method. The fields shown depend on the method you chose in Step 1 (e.g., Aggregation, Count, Occurrence, SQL, etc.).
For most methods, you’ll need to:
- **Select a related or event model**
Choose the dataset you're calculating across (e.g., Purchases, Sessions, Page Views)
- **Add optional filters**
Filter the input records before applying your calculation (for example, revenue greater than 500 or event_type = "checkout")
- **Complete [method-specific settings](/customer-studio/traits#method-specific-settings)**
Filters can include field-based logic (for example, revenue greater than 500) and are applied before aggregation.
Use the **Preview results** button to validate before moving to the final step.
---
### Method specific settings
#### Aggregation traits (Sum, Average)
**Use case:** Compute the sum or average of a numeric column across related or event data.
You’ll configure:
- **Related or event model** (e.g. `Purchases`)
- **Filter conditions** (optional)
- **Aggregation type**: `Sum` or `Average`
- **Column to aggregate** (e.g. `REVENUE`)
#### Count traits
**Use case:** Count the number of records (e.g. purchases, sessions, clicks) that meet a condition.
You’ll configure:
- **Related or event model**
- **Optional filters**
- **Count by**: Select a column (optional — defaults to row count)
#### Occurrence traits (First, Last, Most frequent)
**Use case:** Return the first, last, or most frequent value of a column from a related model.
You’ll configure:
- **Related or event model**
- **Optional filters**
- **Occurrence type**: `First / Min`, `Last / Max`, `Most frequent`, or `Least frequent`
- **Trait value**: Column to return (e.g. `product_name`)
- **Order by**: Determines ordering for first/last
#### List traits
Contact your Customer Success team to enable returning list of objects. By
default, only list of values are supported.
**Use case:** Return an array of values or objects for use in downstream tools (e.g. product IDs, brand names).
You can configure:
- **Column to list**
- Select one column (e.g. `product_id`) to return a list of values
- Select multiple columns to return a list of objects. The object key is the column alias; change the alias to control the output key name in the object.
- **Optional filters** to control which rows are included
- **Deduplication toggle**: Check to skip duplicate values
- **Order by** and **Sort direction**
- **Optional slice controls**: Optionally, choose which part of the list to return (e.g. first 3)
**Examples:**
- Value list: `["product_1", "product_2", "product_3"]`
- Object list: `[{"id": "abc", "expiration": "2024-10-01"}, {"id": "xyz", "expiration": "2024-11-15"}]`
For value lists, if using "Deduplication", "Order by" must match the selected
column. For object lists, "Order by" and "Deduplication" cannot be used
together.
#### SQL Aggregation traits
**Use case:** Build advanced calculations using SQL across related models.
You’ll configure:
- **SQL expression**, using `{{ column "" }}` syntax
- **Default value** for null or missing results
- **Property type** (e.g. `String`, `Number`, `Boolean`)
#### SQL Formula traits
**Use case:** Compute logic based on fields from the trait model itself using SQL. Does not join to related models.
You’ll configure:
- **Formula expression** (no `{{ }}` syntax—just reference columns directly)
- **Property type**
In SQL Formula traits, use direct field names (e.g. `ltv`, `email_verified`)
instead of `{{ column }}` syntax.
---
### Step 3: Finalize
1. Give your trait a clear name and optional description. This name will appear when filtering in the audience builder.
2. Review the summary shown in plain language.
3. Click **Create trait** to finish.
The new trait is now associated with the trait model and can be used in audiences or, if the trait model is a parent model, synced to destinations.
---
## View and manage traits
After creation, traits appear in the **Traits table**, where you can:
- View name, the model the trait belongs to, calculation type, and the model used to calculate the trait
- Edit or archive traits
Use the tabs to switch between:
- **Traits:** All saved traits
- **Templates:** Reusable blueprints
- **Archived:** Deactivated traits
---
## Trait configuration and usage
Click into any trait to view:
- The **Configuration** tab: shows logic summary and allows edits
- The **Usage** tab: lists audiences where this trait is used
---
## Trait templates
Trait templates let you define reusable logic that can be used as a base for other traits. You can create templates once and generate new traits from them, saving time and ensuring consistency across teams.
### Create a new template
To create a template:
1. Go to **Customer Studio → [Traits](https://app.hightouch.com/traits?from=docs)**
2. Click the **Create** dropdown and select **New template**
The flow for creating a template is the same as creating a standard trait:
- Choose a trait model
- Select a calculation method
- Configure filters and logic
- Finalize the template by naming and saving it
Once saved, the template appears under the **Templates** tab.
{" "}
Updating a trait template will also update all traits created from it—unless those
traits have overridden the base logic. For example, if you change a template’s
aggregation type from `Sum` to `Count`, any traits built directly from that template
will adopt the new `Count` method automatically.{" "}
### Create a trait from a template
Once you’ve created a template, you can use it to quickly generate new traits with consistent logic.
To create a trait from a template:
1. Go to **Customer Studio → [Traits](https://app.hightouch.com/traits?from=docs)**
2. Click the **Create** dropdown and select **Start with a template**
3. In the modal, select a template and click **Choose template**.
3. In the **Choose a template modal**, select your desired template from the list.
- You’ll see a summary of the calculation logic, including parent and related models.
- Click **Preview results** to validate the output.
- Click **Choose this template** to continue.
4. On the next screen, you'll see the template logic pre-populated.
- (Optional) Add additional filter conditions.
- Enter a **Name** and optional **Description**.
- Click **Preview results** to validate trait output.
- Click **Save** to finalize.
Once saved, this trait will be available in the Traits tab and can be used for audience filters and destination syncs.
{" "}
Changes made to the base template will automatically update all traits created
from it—unless local filters or logic were modified when the trait was saved.{" "}
---
## Create audiences with traits
Once you've created a trait, you can use it for filtering within your audiences. Traits filters are under **Traits** in the filters dropdown.
There are three kinds of traits you can use in your audience filter:
- Trait templates
- Traits
- Custom traits
### Use trait templates in the audience
1. Click **Add filter**
2. Navigate to **Traits** and select a trait template.
For example, if you created an LTV trait, you can use it to filter for "All users that have spent more than $100."
3. [Optional] You can also add filters to apply before Hightouch calculates trait values. For example, you can restrict the LTV calculation to purchases where the `Brand` was Nike directly in the audience builder.
4. [Optional] You can save the trait filter as a new [trait enrichment](#enrich-audiences-with-traits). To do so, click the actions button and "Save trait enrichment"
### Use traits in the audience
1. Click **Add filter**
2. Navigate to **Custom traits** and select a trait
3. [Optional] You can edit the trait configuration by clicking **Edit**. Updating these filters will not affect the original trait.
4. [Optional] If you edit the trait and want to save it as a new trait, click the actions button and "Save trait."
### Create a custom trait within the audience
Maybe you don't have a suitable trait or trait template to act as a filter. Or, maybe you just want to experiment with a trait filter without necessarily creating one for the parent model.
You can create a **custom trait** inline within the audience query builder.
1. Click **Add filter**
2. Navigate to **Custom traits** and click **Create a custom trait**
3. Select a related model
4. Select the configuration for your trait
5. [Optional] If you decide that this custom trait is useful enough to persist it for the entire parent model, click the actions button and "Save trait."
Doing so converts it into a new trait that can be synced.
---
## Enrich audiences with traits
Traits defined on the parent model are automatically available for syncing to destinations. Alternatively, you can create **trait enrichments** which are traits that are specific to an audience.
Trait enrichments are based on a parent trait which are the trait templates you've defined on the parent model.
1. Go to the **Traits** tab of the audience you want to add enrichments to.
2. Click **Create trait**.
3. Enter a **Name** and select the **Parent trait**.
4. [Optional] Add conditions to apply on top of the parent trait.
Once created, these enrichment traits appear in the Audience preview and sync configuration like any other field.
---
## Event trait enrichments
You can also create traits from events in the audience builder. These automatically become [trait enrichments](#enrich-audiences-with-traits), meaning you can view them in the preview and sync their values. One common use case for event traits is to capture information about abandoned carts.
1. To create a new event trait enrichment, click **Add trait** on an event condition in the audience builder.
You can only access the **Add trait** option from an event filter once you've
saved the audience.
2. Give the event trait enrichment a descriptive **Name**.
3. Select the aggregation you want to use, for example, **Last**.
4. Select the information you want to capture about the event. For example, you may want to capture the `Product_ID`, `SKU`, or `Brand` of the final product a user added to their cart before abandoning it.
---
## SQL Aggregation traits
Custom SQL traits let you write custom aggregations that the [default aggregations](#trait-types) don't cover. To build a SQL trait, [create a trait](#create-a-trait) and follow these additional instructions:
1. Select **SQL** as the **Aggregation** type.
2. Enter a raw SQL aggregation. For example, to create a `SUM` trait with just SQL, you would enter `SUM({{ column "price" }})`.
Column references must use the syntax shown in the SQL editor: double curly braces around the keyword `column` followed by the column name in quotes. Column names are case-sensitive.
3. Enter a default value for when the SQL returns no rows.
4. Define the **Property type** of the SQL aggregation result.
### JSON aggregations
JSON aggregations are a helpful way to aggregate the raw data in a related model. For example, imagine you want to get a list of all product IDs users have purchased. You can do this with the following SQL trait:
```
ARRAY_AGG(DISTINCT {{column "product_id"}})
```
### CASE statements
`CASE` statements are a useful way to create friendly options for aggregations. For example, imagine you want to bucket customers into `low`, `medium`, and `high` LTV. You can do this with the following SQL trait:
```
CASE
WHEN SUM( {{column "price"}} ) > 100 THEN 'high'
WHEN SUM( {{column "price"}} ) > 50 THEN 'medium'
ELSE 'low'
END
```
---
## SQL Formula traits
Formula traits allow you to reference the columns, merged columns, and other traits belonging to the trait model to build inline transformations. Unlike Custom SQL Traits, there is no aggregation against a single related model. Instead, you can reference columns from a multitude of sources.
This lets you compose columns directly within Hightouch.
1. Select **Formula** as the **Aggregation** type.
2. Enter raw SQL. To reference a column or trait, type the name for the autocompletion dropdown to show up. Then, select it to populate the editor as shown above.
Here are a few examples of SQL queries that can be done using formula traits:
- Combine multiple columns into one: `CONCAT({{"first_name"}}, ' ', {{"last_name"}})`.
- Use a string column to create a boolean flag: `CASE WHEN {{"merged.pets.animal"}} LIKE 'dog' THEN true ELSE false END`
- Reference other traits: `CASE WHEN {{"trait.lifetime_total_value"}} > 1000 THEN 'high' ELSE 'low' END`.
**Column references** in formula traits use special syntax. Use the SQL editor's autocompletion feature to write column references.
References to non-existent columns or traits are shown with a red pill.
3. Define the **Property type** of the SQL aggregation result.
## Trait value suggestions
**Most Frequent**, **First**, **Last**, [**Custom SQL**](#custom-sql-traits), and [**Formula**](#formula-traits) traits that result in a string type support trait value suggestions. When enabled this provides **suggested values** for the visual audience builder. For example, if you're creating an audience you want to filter on a "Most Recent City" trait, turning on suggestions populates a dropdown with values—San Francisco, Los Angeles, etc.—found in the dataset.
### Enabling suggestions
Trait value suggestions can be enabled when creating or viewing a trait.
## Traits on event or related models
You can define traits on related or event models (e.g. Purchase, Session) as well as the parent model (e.g. User). This enables traits that compute within the scope of a related row and can be referenced in audience filters.
**Example use cases**
- Per purchase, how many items over 1K do I have? Filter for users with purchases that include 3 items over $1000.
- I only have a timestamp column on my event. How many events occurred on the weekend vs the weekday? Filter for users with purchases that occurred on a weekend.
**Key notes**
- Traits "live" on the event or related model: the event or related model is the trait model, and the trait is calculated relative to it.
- In non-formula traits you can aggregate across models related to trait models. In formula traits, you can use the trait model's columns directly in the formula.
- You can’t sync these traits directly. They can only be filtered.
- Preview rows show the join key and, for events, timestamp and primary key if available.
---
## Create audiences
**URL:** https://hightouch.com/docs/customer-studio/usage
**Description:** Build and activate customer segments using Hightouch's visual audience builder.
**Section:** Customer Studio
| | |
| ----------------- | ------------------------------------------------------------------- |
| **Audience** | Marketers |
| **Prerequisites** | Completed [schema setup](/customer-studio/schema) by your data team |
_Audiences define the group of users or accounts you want to target in your campaigns. Using the visual audience builder, marketers can filter, preview, and activate audiences without writing SQL._
---
## What you'll learn
After reading this article, you’ll know how to:
- Create a new audience in Customer Studio
- Apply filters to define and refine audience membership
- Preview and validate audience results
- Limit and control audience size
- Analyze and reuse audiences
- Sync audiences to destinations
---
## Overview
The **audience builder** lets you visually define and activate segments of users or accounts. It’s designed for marketers to build precise audiences using data from your warehouse, preview the results, and activate them directly in campaigns.
You can filter audiences based on properties, events, relationships, and traits, then preview and activate them directly in your marketing and advertising tools.
Want to skip manual filter configuration? Use the [**AI
Agent**](/customer-studio/agents) to build and analyze audiences using natural
language. Describe the segment you need, and the Agent suggests audience logic
you can review and refine.
Once created, audiences can be:
- Analyzed in [Insights](/customer-studio/insights)
- Used in [Journeys](/customer-studio/journeys)
- [Split](/customer-studio/experiments) for experiments
- [Synced](/customer-studio/syncs) to destinations
Audiences are built from [models](/customer-studio/schema#schema-setup)
configured by your data team. These models define the source data you’ll use
to segment users, accounts, or other entities.
---
## Audience types (batch vs real-time)
Hightouch supports both **batch** and **real-time** audience evaluation.
| Type | Description | Example use |
| ------------- | ---------------------------------------------- | --------------------------------------------------- |
| **Batch** | Evaluated on a schedule or manually refreshed. | Weekly “High-LTV Customers” campaign |
| **Real-time** | Evaluated continuously as new events arrive. | “Users who searched flights in the last 10 minutes” |
Use **batch audiences** when you need stable segments that update on a predictable cadence, such as lifecycle or nurture campaigns.
Use [**real-time audiences**](/real-time/overview) when you want to respond instantly to user behavior like triggering an ad, notification, or personalization within seconds of an event.
---
## Create and refine audiences (core workflow)
### Step 1: Create a new audience
1. Go to **Customer Studio → [Audiences](https://app.hightouch.com/audiences?from=docs)**.
2. Click **`Add Audience`**.
3. Select a [**Parent Model**](../schema#define-the-parent-model) to segment (for example, `Users`, `Accounts`, or `Devices`).
4. Name your audience (for example, _High-Value Customers_) and add an optional description.
---
### Step 2: Apply filters
Start from an [audience template](/customer-studio/templates) or create filters from scratch.
| Filter type | Use it to filter by… | Example filters |
| -------------------- | ----------------------------------------------------------------------- | ------------------------------------- |
| **Properties** | Columns on the parent model | `email contains "@gmail.com"` |
| **Relations** | Linked records (purchases, accounts) | `has Purchase where amount > 100` |
| **Events** | Actions users performed | `Product Viewed within last 30 days` |
| **Audiences** | Membership in another audience | `is in VIP Members` |
| **Journeys** | Participation in a journey | `currently in Abandoned Cart Journey` |
| **Traits** | Custom or computed fields | `customer_score > 80` |
| **Trait comparison** | Compare two trait or column values against each other (feature-flagged) | `lifetime_value > avg_order_value` |
If you find yourself recreating the same logic across multiple audiences,
convert that logic into a reusable [trait](/customer-studio/traits). Traits
help maintain consistent segmentation.
#### Combine filters with AND/OR logic
Use **AND**, **OR**, and nested condition groups to express complex logic.
**Example:**
Include users who made a purchase **or** added to cart, **and** belong to the Gold tier.
| Group | Condition |
| -------------- | ----------------------------------------------------------------------- |
| (A) | `has Purchase where amount > 100` OR `added to cart within last 7 days` |
| (B) | `loyalty_tier equals "Gold"` |
| Audience logic | (A) AND (B) |
#### Nested condition groups in related model filters
When filtering on related models, you can nest AND/OR condition groups inside the related model filter itself. This lets you express logic like "has a purchase where (amount > 100 AND category = 'Electronics') OR (amount > 50 AND category = 'Accessories')."
To add a nested group, open the **...** menu on a related model filter and select **Add "OR" group** to create a nested condition group within the filter block.
Unexpectedly small results often indicate a one-to-many relationship mismatch
or overlapping event time windows. Check your schema relationships and event
time filters.
---
### Step 3: Preview and validate your audience
After defining filters, preview your audience to confirm that your logic returns the expected members.
- Click **`Calculate size`** to estimate audience size.
- Click the member count (for example, `110 members`) to preview sample records.
Use [Sampling](/customer-studio/sampling) to test filters and preview large
datasets without querying your entire warehouse.
#### Limit audience size
Limit audience size to focus on your highest-value members, sample randomly, or run controlled experiments on an audience subset.
1. Click the **`Limit audience size`** icon.
2. Choose a limiting method:
- **Top** or **Bottom** — select the top or bottom N members sorted by a column (for example, `purchase_amount` or `last_purchase_date`).
- **Random** — select a random sample of the specified size. Random sampling does not require a sort column and returns a different set of members on each evaluation.
You can also limit by **percentile** (for example, top 25th percentile of
spend). The percentile option appears only for numeric columns without value
suggestions enabled.
#### Preview with destination rules
If your workspace uses [destination rules](/customer-studio/destination-rules), you can select a destination to preview how the audience will sync before saving.
---
### Step 4: Analyze your audience
Use **[Insights](/customer-studio/insights)** to analyze composition and overlap.
- Click **`Show Insights`** to open the panel.
- **Overlap** compares shared members with another audience.
- **Breakdown** displays distribution by trait, geography, or behavior.
---
### Step 5: Save and reuse audiences
Click **`Save audience`** to finalize your setup.
#### Reuse audiences with templates
To create repeatable logic, apply an **[audience template](/customer-studio/templates)** directly in the builder.
Templates let you standardize approved audience definitions for faster setup.
#### Clone audiences
To duplicate an existing audience, open the overflow menu and click **`Clone`**.
You can modify filters, rename the copy, and choose which syncs to duplicate.
---
### Step 6: Sync to destinations
After saving, activate your audience by syncing to tools such as CRMs, ESPs, or ad platforms.
Learn more in [Sync audiences](/customer-studio/syncs).
Run a test sync to a sandbox destination before enabling production syncs.
This helps validate column mappings and audience behavior without affecting
live campaigns.
---
## Advanced filtering and configuration
Use these optional tools to create more sophisticated or large-scale audiences.
### Filter using related / event models
If your data includes relationships (for example, `Users → Orders`), you can build filters across connected models. This lets you target users based on behaviors or attributes in other tables.
**Example**
"Target users who placed an order worth more than $100."
If related models or expected fields are missing, your data team may need to
adjust [schema
relationships](/customer-studio/schema#define-relationships) or
[merge columns](/customer-studio/schema#merge-columns-across-models).
---
### Compare traits and columns in filters
Trait comparison lets you filter audiences by comparing two column or trait values against each other, rather than comparing a column against a static value. This is useful when the threshold for inclusion depends on per-user data rather than a fixed number.
For example, you can build a filter like `lifetime_value > avg_order_value` to find users whose lifetime value exceeds their average order — without hardcoding a dollar amount.
When adding a filter condition, you can select a column or trait as the comparison value instead of entering a static value. Choose the second column or trait to compare against, and the filter evaluates each user's row individually.
---
### Filter using nested related / event models
#### Overview
Nested filtering lets you filter audiences using data across multiple 1:many relationships (e.g. `User → Purchases → Items`). This allows you to express logic like:
"Target users who have a purchase in the past 30 days where that purchase includes an item in the 'Skincare' category."
You can traverse **up to three levels** of related or event models to capture richer, real-world relationships.
#### When to use it
Use nested filtering when you need to build conditions on data that is in a downstream event or related model. Nested filtering is especially useful when you want to combine conditions within sub-models with conditions on the current model.
#### Example
**Schema:** `User → Purchases → Items`
**Goal:** Find users who have made a purchase in the last 30 days that included a 'Toy' item.
#### How it works
Each nested block evaluates conditions within its own level.
- The outer block ( `Has ≥1 CompletedCheckout`) evaluates purchases belonging to the user.
- The inner block (`PurchaseItems include item\_type \= 'Toy'`) evaluates items belonging to each purchase.
In this example, a user qualifies if **at least one** purchase matches the nested condition. You can also use "Number of" filters at both levels (e.g., "≥3 purchases where for each such purchase, ≥2 items match").
This is different from universal logic (e.g., finding users where all purchases included a 'Toy'). Universal logic queries can be performed via [traits](/customer-studio/traits).
#### Limitations
- Only supports traversal up to **three levels deep** from the parent model.
Nested filtering is an advanced feature intended for users needing to access
multiple levels of models. Contact your Customer Success team to enable nested
filtering for your workspace.
---
### Filter nested JSON data
Some columns contain nested JSON objects or arrays (type `unknown JSON`).
You can filter inside these structures using **dot-path notation** to access specific fields.
For example:
- `booking.product.geo.city` filters by a nested property inside an object.
- `booking.0` filters by the first item in an array.
This is useful when event data or object payloads store multiple attributes within a single column.
JSON filtering is supported on **parent**, **event**, and **related** models.
**Example JSON**
```json
{"inventory":{"on_order":50,"stock":750,"warehouse":"WHB"}, "dimensions":[9, 8, 7]}
{"inventory":{"on_order":10,"stock":120,"warehouse":"WHA"}, "dimensions":[5, 4, 3]}
```
If the column is on a parent model:
If the column is on an event model:
**How to filter**
Use dot-path notation to specify the exact field you want to filter on. Hightouch supports both object and array traversal for nested JSON fields.
By default, JSON columns are treated as type unknown with the exists operator. You can manually set the data type—string, number, or boolean—to unlock the operators that make sense for that field.
All standard operators supported in the audience builder (for example, equals, contains, greater than, exists) are available, but only if they’re compatible with the selected type.
**Preview**
Depending on your dataset size and how recent your events are (if filtering event data), you may use the [member preview](/customer-studio/usage#validate-audience-size-and-results) to confirm that your JSON paths and filters return the expected results.
JSON filtering is an advanced feature intended for users who understand their
data structure. If you’re unsure which JSON fields to target, ask your data
team for help or contact your Customer Success team to enable JSON column
filtering for your workspace.
---
### Drag and drop values
For certain [textual](#textual-operators) and [numerical](#numerical-operators) operators, you can upload a list of values directly into your filter using **drag and drop** instead of entering them manually.
This feature saves time when working with long lists, such as customer IDs, organization IDs, or email addresses, so you don’t have to copy and paste each value.
You can drag and drop `.txt` or `.csv` files containing **comma-separated values (CSV)** directly into the filter’s value field.
**Example use case**
- You have a CSV file with hundreds of organization IDs.
- Instead of entering each ID manually, drag and drop the file into the input box of a property filter.
- Hightouch automatically uploads and parses the list of values for you.
**Supported operators**
| **Data type** | **Supported operators** |
| ------------- | ----------------------------------------------------- |
| **Textual** | contains · does not contain · equals · does not equal |
| **Numerical** | equals · does not equal |
---
### Operator reference
Each filter condition in the audience builder uses a set of **comparison operators** to determine whether records meet your criteria.
The available operators depend on the **data type** (text, number, Boolean, or timestamp).
Use this table to confirm which operators are available for each data type.
| **Data type** | **Supported operators** |
| ------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| **Textual** | equals · does not equal · contains · does not contain · starts with · ends with · exists · does not exist |
| **Numerical** | equals · does not equal · greater than · less than · greater than or equal to · less than or equal to · exists · does not exist |
| **Boolean** | equals · does not equal · exists · does not exist |
| **Timestamp** | equals · before · after · within · not within · between · anniversary · exists · does not exist |
The **anniversary** operator checks whether a date falls on an anniversary of
the current day—useful for birthdays, renewals, or membership milestones.
#### Textual operators
When filtering on textual data (for example, product brand, location, or email domain), you can use:
- equals
- does not equal
- contains
- does not contain
- starts with
- ends with
- exists
- does not exist
**Behavior details**
The **contains** operator performs a fuzzy match—it translates to the SQL
`LIKE` operator with the value wrapped in `% %`. The **equals** operator looks
for an exact match (also using `LIKE`, but without surrounding wildcards).
#### Numerical operators
When filtering on numerical data (for example, price, score, or lifetime value), you can use:
- equals
- does not equal
- greater than
- less than
- greater than or equal to
- less than or equal to
- exists
- does not exist
You can use the [drag and drop](#drag-and-drop-values) feature with the following operators: **equals** and **does not equal.**
When using **greater than**, **less than**, **greater than or equal to**, or **less than or equal to**, you can enter:
- an **absolute numerical value** (for example, `purchase_amount > 100`), or
- a **percentile** (for example, “top 50th percentile of lifetime value”).
To use the **percentile** option, make sure the column has **value suggestions turned off** in the [schema configuration](/customer-studio/schema#enable-filter-value-suggestions).
Percentile filtering is helpful when audience size, rather than a fixed
number, is your targeting driver (for example, “top 10% of users by spend”).
#### Timestamp operators
When filtering on timestamps (for example, purchase date or last activity), you can use:
- equals
- before
- after
- within
- not within
- between
- anniversary
- exists
- does not exist
You can choose between **rolling** or **calendar** time periods for relative filters.
**Examples:**
- To find users active in the last 60 days → use “within previous 2 months” (rolling window).
- To find users active during a specific range → use “within previous 2 calendar months.”
If today is June 3, this filters for dates between April 1 and May 31.
If a column contains timezone data, that timezone is used when calculating
date and time values. If not, Hightouch uses the warehouse’s default timezone
for both stored values and current time calculations.
Operators vary by data type. If you don’t see an operator available for a
specific column, check the column’s configured data type in your [schema
setup](/customer-studio/schema#configure-columns). For example, `string` types support
text-based filters, while `number` types unlock percentile and inequality
operators.
---
### Configure null-value behavior
When building filters, you may encounter `null` data (for example, missing country values).
Admins can control how nulls behave for **“not”** operators in [Workspace Settings](https://app.hightouch.com/settings/workspace?from=docs).
**Options:**
- **Treat null values like any other value (default):**
`Country does not equal USA` → includes `Country = null`
- **Use special behavior for null values:**
`Country does not equal USA` → excludes `Country = null`
---
## Experimentation and governance
### Split audiences for experiments
Divide an audience into random or rule-based groups for testing.
| Split type | Description |
| ---------------------- | ------------------------------------------ |
| **A/B Split** | Divide members randomly between two syncs. |
| **Multi-branch Split** | Route members to multiple destinations. |
Each profile is assigned once, based on its primary key.
Changing split percentages can reassign users—use a [snapshot](/customer-studio/audience-snapshots) to preserve consistency.
---
### How audiences stay up to date
Audiences update automatically as your source data changes so campaigns always reach the right people.
| Behavior | Description |
| --------------------- | ----------------------------------------------------------------------------------- |
| **Automatic updates** | New data or events trigger re-evaluation. |
| **Manual refresh** | Trigger a full recalculation from the overflow menu. |
| **Snapshots** | Freeze membership at a point in time to measure lift or maintain fixed test groups. |
---
### Segment by households or accounts
In some workspaces, audience data is organized around more than one entity type, such as: **People**, **Households**, or **Accounts**. When you enable [schema labeling](/customer-studio/schema#label-your-models), the audience builder automatically reflects these relationships.
This allows you to create audiences that consider shared characteristics or behaviors across related entities, rather than limiting segmentation to individuals.
With schema labels applied, filters adjust automatically so you can build audiences using data from connected entities. For instance, you can define a **People** audience that includes or excludes members based on traits of their associated **Household** or **Account**.
**Examples:**
- **Household segmentation:** Include people in households that meet a shared condition (for example, at least one member identifies as female or holds an active subscription).
- **Account segmentation:** Include people whose associated account shows recent activity or engagement (for example, a recent purchase or contract renewal).
Schema labeling enables you to build audiences using both individual and
group-level data. This supports coordinated targeting and measurement when
multiple related entities—like people within households or employees within
accounts—affect campaign outcomes.
---
## Example use cases
### Abandoned checkout retargeting
**Goal:** Retarget users who started checkout but didn’t complete.
**Filters**
- Event: `Checkout started` within last 24 hours
- Event: `Order completed` not performed in same window
- Trait: `LTV > 200`
- Property: `Email is not null`
**Sync to**
- Meta Custom Audiences
- Klaviyo List
Combine this audience with a [subset](/customer-studio/subsets) or
[destination rule](/customer-studio/destination-rules) to exclude opt-outs.
---
### Real-time travel personalization
**Goal:** Personalize offers for users who searched flights or hotels in the last 24 hours.
**Filters**
- Event: `search` where `category = flight OR hotel`
- Event: `search_time` performed within 24 hours
- Trait: `loyalty_status = Gold`
Enable **real-time evaluation**, split 80/20 for testing, and sync to ad platforms.
Learn more in [Same-session audiences](/real-time/overview).
---
## Best practices
| Goal | Recommendation |
| ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| **Ensure consistent segmentation** | Build complex logic once as reusable [traits](/customer-studio/traits). |
| **Avoid audience drift** | Use [snapshots](/customer-studio/audience-snapshots) to lock membership for control groups. |
| **Simplify experimentation** | Keep [experiment](/customer-studio/experiments) audiences clearly named (for example, `retargeting_a` and `retargeting_b`). |
| **Test before activation** | Use [sampling](/customer-studio/sampling) to preview results on smaller datasets. |
---
## Troubleshooting
| Issue | Likely cause | Resolution |
| ----------------------------------- | ------------------------------------- | -------------------------------------------- |
| Audience size changes unexpectedly | Real-time evaluation enabled | Use snapshots to lock membership |
| Filters return no results | Wrong relationship or null handling | Check schema configuration and null settings |
| Members appear in both split groups | Split percentages edited after launch | Avoid editing live splits |
| Cannot assign sync | Missing permissions | Contact your workspace admin |
If you see warehouse SQL errors (for example, unresolved columns, invalid identifiers, or object authorization failures), use [Resolve SQL compilation errors](/models/sql-compilation-errors).
### Common issues and detailed resolutions
### Filter value suggestions not appearing
**Cause:** Column suggestions are not enabled, or the suggestion cache is stale.
**Resolution:** Ensure you’ve [enabled column suggestions](/customer-studio/schema#enable-filter-value-suggestions). If suggestions are enabled but missing, refresh them in the filter UI.
### Error: column "…" does not exist in \_ht_with_column_casts
**Cause:** A model column was deleted but is still cached in Hightouch.
**Resolution:** Preview and re-save the model configuration to refresh cached columns.
### Error: [UNRESOLVED_COLUMN.WITH_SUGGESTION] … cannot be resolved
**Cause:** A filter or audience condition references a column that is not present in the model output (for example, `segment_value`), often due to a renamed or removed column or stale filter configuration.
**Resolution:**
1. Open the parent model and confirm the referenced column exists in the selected columns.
2. Preview and re-save the model to refresh available columns in Customer Studio.
3. Update audience filters that reference the missing column, or add the column back to the model.
4. Re-run audience preview to confirm the error is gone.
If audience sizes or syncs change unexpectedly, review your{" "}
schema relationships and{" "}
sync schedules regularly to ensure alignment.
---
## What’s next?
After creating audiences, you can activate and measure them using these related features:
- [Sync audiences to destinations](/customer-studio/syncs)
- [Build and analyze audiences with the AI Agent](/customer-studio/agents)
- [Add traits for filtering](/customer-studio/traits)
- [Orchestrate campaigns with journeys](/customer-studio/journeys)
- [Reuse audiences with templates](/customer-studio/templates)
---
## Agent alerting
**URL:** https://hightouch.com/docs/ai-decisioning/agent-alerting
**Description:** Set up automated notifications for agent run failures and recommendation volume drops
**Section:** AI Decisioning
| | |
| ----------------- | ------------------------------------------------------------------------------------------------ |
| **Audience** | Marketers, data teams |
| **Prerequisites** |
A running [agent](/ai-decisioning/agents)
|
*Agent alerting notifies you about engine run failures, long-running or stalled agents, and unexpected drops in recommendation volume before they impact your campaigns.*
---
### What you'll learn
After reading this article, you'll know how to:
- [Understand alert triggers](#alert-triggers)
- [Configure alert severity](#alert-severity)
- [Choose notification channels](#notification-channels)
- [Set up alerting on an agent](#set-up-alerting)
---
## Overview
AI Decisioning agents support automated alerting for engine runs — the daily runs that generate and schedule AID messages for your users. The alerting system for AID agents follows the same model used for [syncs](/syncs/alerting) and journeys in Hightouch.
Every agent comes pre-configured with a default run failure monitor. No setup is required for monitoring to begin.
To access alerting, open an agent and go to the **Alerting** tab.
---
## Alert triggers
There are four alert triggers available for AID agents:
**Agent run failure:** Triggers when one or more consecutive engine runs fail. Each engine run is responsible for generating and scheduling the AID messages sent to your users, so a failure means messages may not be produced or delivered for that cycle. The alert fires regardless of which pipeline stage caused the failure and links directly to the specific failed run, taking you to the **View details** page on the [Health tab](/ai-decisioning/agent-health#view-details).
From View details, you can see each pipeline stage and its status, which stage the run failed at, and an error message explaining what went wrong. For more information, see [Agent health → View details](/ai-decisioning/agent-health#view-details).
**Agent run duration:** Triggers when an agent has been running for longer than a threshold you define (in minutes). Use this to detect runs that are taking unusually long, which may indicate a warehouse performance issue or a bottleneck in the pipeline.
**Agent idle time:** Triggers when no new agent run has started within a specified number of minutes. Use this to detect situations where your agent has stopped running entirely, which could mean a scheduling issue or an upstream dependency failure.
**Low number of recommendations generated:** Triggers when the number of interactions (messages) generated in an engine run falls below a threshold you define. Use this to catch unexpected drops in decision volume that may not surface as an outright failure but could signal an upstream data issue or audience eligibility problem.
---
## Alert severity
Each trigger can be configured with a severity level:
- **Critical:** For high-urgency issues that require immediate attention, such as a run failure.
- **Warning:** For lower-urgency signals, such as a drop in recommendation volume that may warrant investigation but is not blocking.
- **Custom:** Set separate thresholds for Critical and Warning severity on the same trigger. For example, fire a Warning if fewer than 500 interactions are generated and a Critical if fewer than 100.
---
## Notification channels
Alerts can be routed to the following destination types: Slack, Email, PagerDuty, SMS, and Webhook.
Each recipient can be scoped to receive only Critical alerts or all alerts.
---
## Set up alerting
1. Go to any AID agent and select the **Alerting** tab.
2. Review the pre-configured **Agent run failure** monitor. Adjust the severity or switch to a Custom threshold if needed.
3. Click the **+** button under **Recipients** to add a notification channel (Slack, Email, PagerDuty, SMS, or Webhook).
4. Optionally, configure additional triggers such as **Agent run duration**, **Agent idle time**, or **Low number of recommendations generated** with your preferred thresholds and severity.
5. Select **Save changes**. Alerts fire on the next qualifying run.
---
## What's next
- [Agent health: Monitor pipeline runs and delivery rates →](/ai-decisioning/agent-health)
- [Insights: Measure agent and message performance →](/ai-decisioning/insights)
- [Agents: Configure goals, audiences, and scheduling →](/ai-decisioning/agents)
---
## Agent health
**URL:** https://hightouch.com/docs/ai-decisioning/agent-health
**Description:** Monitor agent pipeline runs, channel delivery rates, and outcome data freshness
**Section:** AI Decisioning
| | |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers, data teams |
| **Prerequisites** |
A running [agent](/ai-decisioning/agents)
[Send event tracking](/ai-decisioning/configuration#send-event-tracking) configured for channel delivery metrics
|
*The Health tab gives you real-time visibility into your agent's pipeline runs, channel delivery rates, and the freshness of the outcome data powering optimization.*
---
### What you'll learn
After reading this article, you'll know how to:
- [Monitor message generation runs](#message-generation)
- [Diagnose pipeline failures](#view-details)
- [Track channel delivery success rates](#channel-send-success)
- [Troubleshoot low delivery rates](#troubleshoot-low-delivery-rates)
- [Check outcome data freshness](#outcome-health)
---
## Overview
The `Health` tab provides operational visibility into your agent from a single view. No additional setup is required — the tab automatically pulls data from your existing agent configuration, runs, and event models.
To access it, open an agent and go to the **Health** tab.
---
## Message generation
The **Message generation** section shows the current and historical state of your agent's daily runs — the runs that schedule and produce AID messages.
**Current run:** If a generation is in progress, you can see when it started, how long it has been running, and its current pipeline step.
**Last run:** Shows when the most recent generation completed, how long it took, and how many recommendations were produced.
**Historical runs:** A table of recent runs with start time, duration, run status, and recommendation count.
**Failed runs:** If a run encountered an error, it is flagged in red. Click **View details** to see per-stage durations and actionable error messages.
### View details
Click **View details** on any run to open a per-stage pipeline breakdown. This view is useful for diagnosing run failures, identifying bottlenecks, and understanding where time is spent across the generation process.
Each run includes the following pipeline phases:
- **Cache Interactions** — Loads recent interaction data
- **Assign User Groups** — Assigns users to treatment or holdout groups
- **Pull Warehouse** — Retrieves data from your warehouse
- **Generate Interactions** — Produces message recommendations for eligible users
- **Write Warehouse** — Writes results back to your warehouse
- **Compact Interactions** — Cleans up interaction records
- **Trigger Analytics** — Updates reporting and analytics data
Each phase shows its duration and status (`Completed`, `Pending`, or `Failed`). Stages that haven't started yet show as `Pending`.
When a run fails, the error message includes guidance on what went wrong and what you can do to fix it. Errors fall into two categories:
- **External issues (actionable by you):** Problems originating from your data or configuration. For example, a column may have been dropped or renamed in your warehouse while your parent model, user feature matrix, audience, or event goals still reference it. Update the affected configuration and re-run the agent.
- **Internal issues:** Problems on Hightouch's side. Hightouch receives internal alerting for all agent run failures and proactively investigates. You can also reach out to your Hightouch contact for support.
If a run fails at **Pull Warehouse**, check that your data warehouse is running and accessible, and verify that your connection credentials in Hightouch are correct.
A common cause of run failures is a column being dropped or renamed in your warehouse while your audience, feature matrix, event models, or parent model still reference it. If you see a missing column error, check each of those configurations for stale references, update them to reflect the current schema, and re-run the agent.
---
## Channel send success
The **Channel send success** section shows how successfully your agent's messages are being delivered to users, broken down by channel (Email, SMS, Push, and others).
- **Delivery success rate:** The percentage of scheduled messages confirmed sent by your ESP, based on the send event model configured for each channel.
- **Raw counts on hover:** Hover over the percentage to see exact sent versus scheduled message counts (for example, "34,189,852 sent / 36,293,161 scheduled").
- **Destination:** Shows which integration destination is handling delivery for that channel.
- **Time filter:** Use the dropdown in the top-right corner to filter delivery success by **All time**, **Last 7 days**, **Last 30 days**, or **Last 90 days**.
If send success is not configured for a channel, click **Open send success insights** or go to [Insights → Send success](/ai-decisioning/insights) to configure it.
Send success tracking requires send events to be linked to your channels. See [Send event tracking](/ai-decisioning/configuration#send-event-tracking) for setup instructions.
### Troubleshoot low delivery rates
If you see unexpected delivery numbers, here are the most common causes:
- **0% with a warning ("Send event model may not be set up correctly"):** The send event model for that channel likely needs to be reviewed. Confirm that the event model tracking send confirmations from your ESP is correctly configured and scoped to the relevant business units or campaign identifiers for your AID campaigns.
- **Below 50% (highlighted in red):** This threshold flags channels that may have a misconfigured event model or a delivery issue worth investigating. This is particularly common during initial setup, when send tracking is first being validated.
---
## Outcome health
The **Outcome health** section tracks the freshness of the event data powering each of your agent's configured outcomes (goals). This helps you catch reverse ETL pipeline issues before they affect agent optimization.
- **Recent timestamp:** Outcome event data is flowing correctly.
- **Stale or missing timestamp ("--"):** Event data has not been loaded recently. Investigate whether the reverse ETL pipeline feeding that outcome has stalled or broken.
Each outcome's "Last data loaded" timestamp reflects when event data was most recently ingested, not when the agent last ran. If you see a stale timestamp, check your reverse ETL sync for that outcome before the next agent run.
---
## Troubleshooting
| Issue | Likely cause | Resolution |
|---|---|---|
| Channel send success shows 0% with a warning | The send event model for that channel is not configured correctly. | Confirm the event model tracking send confirmations from your ESP is correctly configured and scoped to the relevant business units or campaign identifiers for your AID campaigns. |
| Outcome health shows a stale or missing timestamp | The reverse ETL pipeline feeding that outcome has stalled or broken. | Check the reverse ETL sync for the affected outcome and resolve any pipeline issues before the next agent run. |
| Pipeline run fails at Pull Warehouse | Your data warehouse is unreachable or credentials are invalid. | Verify that your warehouse is running and accessible, and confirm your connection credentials in Hightouch are correct. |
| Pipeline run fails with a missing column error | A column was dropped or renamed in your warehouse while your parent model, audience, feature matrix, or event models still reference it. | Update the affected configurations to reflect the current schema, then re-run the agent. |
---
## What's next
- [Alerting: Get notified about run failures and volume drops →](/ai-decisioning/agent-alerting)
- [Insights: Measure agent and message performance →](/ai-decisioning/insights)
- [Agents: Configure goals, audiences, and scheduling →](/ai-decisioning/agents)
---
## AI Decisioning agents
**URL:** https://hightouch.com/docs/ai-decisioning/agents
**Description:** Define campaign goals, audiences, and messages AID can send
**Section:** AI Decisioning
| | |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers, business users |
| **Prerequisites** |
[Configuration →](/ai-decisioning/configuration)
[Prepare data for AID →](/ai-decisioning/prepare-data)
At least one connected destination ([SFMC](/ai-decisioning/sfmc), [Braze](/ai-decisioning/braze), [Iterable](/ai-decisioning/iterable))
|
_Agents are the foundation of AI Decisioning. Each agent defines who to target, what to optimize for, and when messages can be sent._
---
### What you'll learn
After reading this article, you’ll know how to:
- [Create a new agent](#create-a-new-agent)
- [Configure your agent](#configure-your-agent)
- [Understand how agents use messages](#messages)
- [Manage offers to factor cost and value into decisions](#offers)
- [Monitor agent health and pipeline runs](/ai-decisioning/agent-health)
- [Set up alerting for agent issues](/ai-decisioning/agent-alerting)
- [Translate content variants across locales](#localization)
- [Monitor and manage active agents](#monitor-and-manage-agents)
---
## Overview
An **Agent** in AI Decisioning combines your audience, messages, and goals into a single adaptive campaign.
Agents use reinforcement learning to decide which message, timing, and channel will drive the best outcome for each user. They operate continuously, learning from real user outcomes such as clicks, purchases, or sign-ups.
Each new message helps the model understand which creative and timing combinations perform best for different audiences.
Agents don’t replace your content or rules—they learn within the goals,
audiences, and guardrails you define.
---
## Create a new agent
1. Go to `AI Decisioning →` [`Agents`](https://app.hightouch.com/ai/flows?from=docs).
2. Click `+ Add agent`.
3. Enter a name that reflects your campaign’s purpose, such as `Win-back customers` or `Subscription renewals`.
4. Choose a target audience from [Customer Studio → Audiences](/customer-studio/usage).
5. (Optional) Add a **holdout group** to measure incremental lift (for example, `20%`).
6. Click `Add agent`.
---
## Configure your agent
After creation, click your agent to open the `Configuration` tab.
From here, you can set audience eligibility, messages, goals, Smart Suppression, and scheduling preferences.
---
### Step 1: Set audience eligibility
This section shows the audience your agent targets and any experimental setup, such as a holdout group.
| Setting | Description |
| ------------------- | ----------------------------------------------------------------- |
| **Target audience** | The audience defined in Customer Studio. |
| **Holdout group** | The control group used to measure true lift (for example, `20%`). |
Holdouts let you compare users who receive messages with users who don’t, so
you can measure incremental lift.
---
### Step 2: Add and manage goals
Goals define what your agent optimizes toward.
Each goal represents a measurable event (like a purchase or click) and includes a priority level—helping AID decide which outcomes matter most.
To add a goal:
1. In your agent’s `Configuration` tab, under `Goals`, click `+ Add goal`.
2. Configure the goal.
- Select an event model (for example, `Purchase event`, `Viewed Product`, or `Redeemed offer`).
- (Optional) Add a filter to refine the event, such as `Performed → Where property is campaign = Summer Sale`.
- (Optional) Select related collections (for example, `Live Offers`, `In-stock Products`) to attribute conversions to catalog items.
3. Click `Continue` to move to the `Impact` step.
- Choose whether to include this goal in decisioning or track it for measurement only.
- Select a classification:
- `Best` — Primary success metric
- `Very good` / `Good` — Positive supporting outcomes
- `Bad` / `Very bad` / `Worst` — Negative outcomes such as `Unsubscribe`
- (Optional) Add a weight column to scale results by value (for example, `Price` or `Order Value`).
4. Click `Continue` to finalize.
- Enter a clear, descriptive goal name (for example, `Redeemed offer`).
- Click `Add goal` to save.
Goals replace the older “Outcomes” field from earlier versions of AI
Decisioning. Each agent learns continuously based on these metrics.
Start with one clear `Best` goal (for example, purchase) before adding
supporting or negative signals.
---
### Step 3 (optional): Configure scheduling
Use the `Scheduling` panel to control how often and when your agent sends messages.
| Setting | Description |
| ------------------ | -------------------------------------------------------------------------- |
| **Send frequency** | Limits how often a user can receive messages (for example, `3x per week`). |
| **Quiet hours** | Prevents sends during off-hours (for example, weekends or late nights). |
| **Blackout dates** | Pauses messaging during specific holidays or events. |
Use scheduling rules to maintain consistency across agents. Combine with Smart
Suppression to control both *when* and *whether* messages are sent.
---
### Step 4 (optional): Enable Smart Suppression
Smart Suppression predicts which messages are most likely to drive incremental conversions and holds back the rest. It uses the [goals](/ai-decisioning/agents#step-2-add-and-manage-goals) you’ve already configured to evaluate each send.
1. In **Configuration**, scroll to **Smart suppression (optional)**.
2. Select **Enabled**.
3. Set the percentage of eligible users to send to (for example, send messages to the top **67%**).
4. Select **Save changes**. A predicted performance curve appears once the model has enough data, typically after about two weeks.
For detailed guidance, see [Smart Suppression → Enable Smart
Suppression](/ai-decisioning/smart-suppression#enable-smart-suppression).
---
### Step 5 (optional): Advanced settings
Use `Advanced` settings to connect your agent with your organization’s downstream campaign tracking.
| Setting | Description |
| ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| **Customer managed campaign attribution** | Enter one or more campaign IDs from your ESP or internal tracking system to tie AID results to your own reporting. |
Campaign IDs help maintain consistent reporting between Hightouch and your
connected destinations. For example, you can enter Braze, Iterable, or SFMC
campaign identifiers to align AID outcomes with your existing analytics.
---
## Monitor and manage agents
Once your agent is running, open the `Overview` tab to review live performance and experiment results.
The Overview includes several key panels:
| Section | Description |
| ------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Eligible users** | Shows the number of users who currently qualify for your agent’s audience and can receive messages, excluding holdouts and ineligible users. |
| **Performance overview** | Shows conversion and engagement metrics for each goal, including the current **winning group**, **holdout**, and **treatment** performance with confidence intervals. |
| **Sends summary** | Displays total sends and messages scheduled for today and tomorrow. |
| **Sends over time** | Charts daily send volume to help you spot delivery trends and campaign pacing. |
| **Sends breakdown by message** | Compares message-level send volume and engagement across your agent. |
Use this view to monitor how your messages are distributed, which goals are performing best, and how holdout results compare to treatment performance.
Performance updates automatically as your agent runs. Confidence levels appear
once enough event data has been collected for statistical analysis.
Use [Insights](/ai-decisioning/insights) for deeper breakdowns of message
performance, goal impact, and timing analysis across agents.
---
## Messages
**Messages** define what an agent can send and where those messages are delivered.
Each message connects an agent to a destination (such as Braze, Iterable, or Salesforce Marketing Cloud). Agents evaluate messages continuously to determine which content, timing, and channel perform best for each user.
Message setup includes:
- Connecting the agent to a destination
- Adding optimized content and variants
- Applying message- and variant-level guardrails
For step-by-step instructions, see [Messages →](/ai-decisioning/messages).
---
## Offers
The `Offers` tab lets you create and manage offers that agents can factor into their decisioning. When an offer is attached to a message, the agent weighs the message's expected value (value minus cost, weighted by conversion likelihood) alongside engagement signals when choosing what to send.
To create an offer:
1. Open an agent and go to the `Offers` tab.
2. Click `+ Add offer`.
3. Configure the offer:
- **Name**: A short, human-readable name (for example, `$10 off first order`).
- **Description** _(optional)_: Additional context about the offer.
- **External ID** _(optional)_: A stable identifier in your source system. Required for CSV upserts to match existing offers.
- **Tags** _(optional)_: Key-value attributes to help organize and categorize offers.
- **Offer type**: **Static** offers apply the same cost to every user. **Column** offers read per-user values from warehouse columns.
- **Cost value**: A fixed cost associated with each conversion (for example, $10 off).
- **Cost multiplier**: A discount rate applied to each conversion's value (for example, `0.2` = 20% discount).
4. Click `Add offer`.
Exactly one of cost value or cost multiplier must be non-zero.
Once created, attach offers to individual messages from the message [Configuration tab](/ai-decisioning/messages#offers). Each message can have at most one attached offer, and not all messages require one.
Use offers when your agent needs to balance engagement with cost efficiency —
for example, choosing between a high-discount winback offer and a lower-cost
loyalty reminder.
---
## Health
The **Health** tab gives you real-time visibility into your agent's pipeline runs, channel delivery rates, and outcome data freshness. From a single view, you can monitor message generation status, diagnose pipeline failures, and catch data issues before they affect optimization.
For full details, see [Agent health →](/ai-decisioning/agent-health).
---
## Localization
The `Localization` tab lets you translate content variants to a user's preferred language. When enabled, the agent serves localized versions of your message content based on each user's locale. Localization is opt-in per agent — existing agents keep working as-is until you enable it.
To configure localization:
1. Open an agent and go to the `Localization` tab.
2. Toggle on **Enable localization**.
3. Configure the following settings:
- **User locale field**: Select the column from your parent model that contains each user's locale (for example, `locale` or `language`).
- **Locales**: Define the locales supported by this agent. Suggestions are pulled from values that already exist in your warehouse, but you can add any string (for example, `en-US`, `fr-FR`, or a non-ISO value like `mx`).
- **Default locale**: The locale displayed in the UI. It's used for users whose locale is missing or unsupported when the missing-locale behavior is set to **Use default locale**.
- **Missing locale**: Choose what the system should do when a user doesn't qualify for any locale:
- **Use default locale** — Send the default locale content
- **Send nothing** — Skip the user
- **Raise an error** — Flag the issue for review
Locale values in your `User locale field` column must exactly match the
locales you configure here, including case and separators. A user whose value
doesn't match any configured locale is treated as missing a locale.
Start with a small number of high-value locales and expand as you validate
translation quality and conversion impact. Adding a locale later doesn't
retroactively translate existing content — each localized variant needs a
value for the new locale before it's eligible to send in that language.
Once localization is enabled, translate individual message variants on the [Content tab](/ai-decisioning/messages#localize-message-content) of each message.
---
## Alerting
The **Alerting** tab lets you set up automated notifications for engine run failures, long-running or stalled agents, and unexpected drops in recommendation volume. Every agent comes pre-configured with a default run failure monitor — no setup is required for monitoring to begin. You can add notification channels (Slack, Email, PagerDuty, SMS, or Webhook) and configure severity levels and thresholds.
For full details, see [Agent alerting →](/ai-decisioning/agent-alerting).
---
## Troubleshooting
| Issue | Likely cause | Resolution |
| ------------------------------------- | ---------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| Agent isn’t sending messages | The target audience is empty, or the destination connection is paused. | Check audience eligibility in `Configuration → Eligibility`, and confirm destination status under `Integrations`. |
| Messages not appearing in ESP | Message or automation identifier mismatch. | Verify the correct destination and automation identifier when adding messages. |
| No performance data visible | The agent is new, paused, or running only a holdout group. | Wait 24–48 hours after activation, or adjust your holdout percentage to include active treatments. |
| Smart Suppression chart missing | Suppression is off or not enough send data has been collected. | Enable `Smart suppression` and allow at least 2 weeks of message activity for predictions to populate. |
| Goals not updating or showing results | The event model or schema changed in your workspace. | Reopen the goal configuration, reselect the event model, and save the update so the agent can rebuild its data relationships. |
If you recently updated goals, audiences, or connected destinations, allow
time for the agent to refresh data before performance metrics stabilize.
---
## What’s next
- [Messages: Create and manage message variants →](/ai-decisioning/messages)
- [Agent health: Monitor pipeline runs and delivery rates →](/ai-decisioning/agent-health)
- [Agent alerting: Get notified about run failures and volume drops →](/ai-decisioning/agent-alerting)
- [QA guide: Validate messages before launch →](/ai-decisioning/qa-guide)
- [Smart Suppression: Control send volume based on predicted impact →](/ai-decisioning/smart-suppression)
- [Tags: Add and analyze message tags →](/ai-decisioning/tags)
- [Insights: Measure agent and message performance →](/ai-decisioning/insights)
- [Collections: Attach item recommendations to messages →](/ai-decisioning/collections)
- [Workspace configuration: Global defaults, channels, scheduling, and feature matrices →](/ai-decisioning/configuration)
---
## Braze integration with AID
**URL:** https://hightouch.com/docs/ai-decisioning/braze
**Description:** Learn how to connect Hightouch AI Decisioning to Braze to deliver personalized messages in real time via Campaigns and Canvases.
**Section:** AI Decisioning
| | |
| ----------------- | ---------------------------------------------------- |
| **Audience** | Technical marketers, marketing and lifecycle teams |
| **Prerequisites** | [Prepare data for AID](/ai-decisioning/prepare-data) |
_Connect Braze to AI Decisioning so agents can deliver personalized emails, SMS, push notifications, and in-app messages through your existing Braze Campaigns and Canvases._
---
## What you'll learn
In this article, you'll learn how to:
- [Set up Braze as a destination](#1-set-up-braze-as-a-destination)
- [Add channels in AI Decisioning](#2-add-channels-in-ai-decisioning)
- [Add messages from Braze](#3-add-messages)
- [Configure Braze Campaigns and Canvases for AID](#4-configure-braze-campaigns-and-canvases-for-aid)
- [Use dynamic variables in Braze templates](#use-variables-in-braze)
---
## Overview
Delivering relevant and timely customer experiences in Braze requires up-to-date, contextual data. By integrating Hightouch's AI Decisioning (AID) with Braze, you can:
- Automatically trigger Braze Campaigns or Canvases based on predictions and behaviors surfaced by AID Agents
- Personalize message content using dynamic variables from your data warehouse
- Centralize targeting and decision logic in Hightouch, while using Braze to handle message delivery
## 1. Set Up Braze as a Destination
Already configured Braze? Even if you’ve previously set Braze up as a
destination, you’ll still need to apply the required permissions for AID.
To [connect Braze to Hightouch](/destinations/braze), you’ll need:
- An **API key** with the correct permissions
- Your **Braze region** identifier
### Step 1: Create API Key
**In Braze:**
1. Go to **Settings > APIs and Identifiers**
2. Click **Create API Key**
### Step 2: Set required permissions
1. Assign the following permissions:
| Sync Type | Required Permissions |
| --------- | ---------------------------------------------------------------------------------------------------- |
| Campaigns | `campaigns.trigger.send`, `campaigns.list`, `campaigns.details`, `campaigns.trigger.schedule.create` |
| Canvas | `canvas.trigger.send`, `canvas.list`, `canvas.details` |
| Messages | `messages.send` |
| Templates | `templates.email.list`, `templates.info` |
### Step 3: Find your Braze region
Use the table below to find your Braze region:
| URL | Region |
| ------------------------------ | ------------------ |
| https://dashboard-01.braze.com | `iad-01.braze.com` |
| https://dashboard-02.braze.com | `iad-02.braze.com` |
| https://dashboard-03.braze.com | `iad-03.braze.com` |
| https://dashboard-04.braze.com | `iad-04.braze.com` |
| https://dashboard-05.braze.com | `iad-05.braze.com` |
| https://dashboard-06.braze.com | `iad-06.braze.com` |
| https://dashboard-07.braze.com | `iad-07.braze.com` |
| https://dashboard-08.braze.com | `iad-08.braze.com` |
| https://dashboard-01.braze.eu | `fra-01.braze.eu` |
| https://dashboard-02.braze.eu | `fra-02.braze.eu` |
### Step 4: Connect to Hightouch
1. Go to **Integrations > Destinations**
2. Click **Add destination**
3. Select **Braze** and click **Continue**
4. Paste your API key and select your Region.
5. Click **Test Connection** to verify setup. You should see `Success` for Campaigns and Canvases.
6. Click **Continue →**.
7. Optional: configure alerts. You can also do this later in **Integrations > Alerting**.
8. Finalize destination settings and click **Finish**.
## 2. Add channels in AI Decisioning
Once Braze is set up as a destination, you can add **channels**. Channels define how Hightouch sends messages to Braze—via Campaigns or Canvases.
### Set up a channel
1. Go to `AI Decisioning → Configuration → Channels`.
2. Click `Add channel`.
3. Choose a channel type: Email, SMS, Push, or In-App.
4. Select **Braze** as the destination.
5. Choose a [sync type](/ai-decisioning/braze#sync-type):
- **Campaign**: One-time, real-time sends used for email, push, and SMS
- **Canvas**: Journey flows used for triggering in-app messages
6. Select an identifier (typically `external_id` or `email`).
7. Click `Add channel`,
8. Select your new channel and click `Save changes`.
### Sync type comparison
The **Sync Type** determines how Hightouch sends messages through Braze. Here’s a breakdown of the available options:
| Channel | Campaign Trigger | Canvas Trigger |
| ------- | ---------------- | -------------- |
| Email | ✅ (recommended) | ✅ |
| SMS | ✅ (recommended) | ✅ |
| Push | ✅ (recommended) | ✅ |
| In-App | ❌ | ✅ |
| **Sync type** | Best for | Message timing | Dynamic vars |
| -------------------- | --------------------------------------------------------- | -------------- | ------------------------------------ |
| **Campaign trigger** | Immediate, single-send campaigns for Email, SMS, and Push | Immediate | Fully supported |
| **Canvas trigger** | In-App Messages | Immediate | Limited (no API-triggered overrides) |
## 3. Add messages
After you add channels in AI Decisioning, you can add **messages** to your agents. Each message connects an agent’s decision to a Braze Campaign or Canvas.
For full details on message configuration, variants, and tags, see [Messages →](/ai-decisioning/messages).
### Add a message from Braze
1. Go to `AI Decisioning → Agents` and select your agent.
2. Open the `Messages` tab.
3. Click `Add messages`, then select **Add messages individually**.
4. Select your configured Braze **channel and destination** tile (for example, **Push - Braze (Campaign)**).
5. Under **Braze campaign**, select the campaign or canvas you created in Braze.
6. Enter a **message name** and click `Add messages`.
You can repeat this process for each channel:
- [Email via campaign trigger](/ai-decisioning/braze#campaign-trigger)
- [SMS via campaign trigger](/ai-decisioning/braze#campaign-trigger)
- [Push via campaign trigger](/ai-decisioning/braze#campaign-trigger)
- [In-app messages via canvas trigger](/ai-decisioning/braze#canvas-trigger)
### Test a message
1. Open the message and go to the `Content` tab.
2. Under **Optimized content**, confirm your variants are configured. If your message uses [dynamic variables](/ai-decisioning/braze#use-variables-in-braze), set values for testing.
3. In the **Preview** panel, verify the rendered message looks correct.
4. Click `Send test` to send the message to a test user in Braze.
#### Test requirements
Your test will only succeed if:
- The Braze Campaign or Canvas is **Active** (not a draft or scheduled).
- **Frequency Capping** is turned **off** in Braze.
- The test user exists in Braze and has the appropriate **identifier** (for example, `user_id`, `external_id`, `email`).
- The user meets eligibility for the channel (for example, email opted-in, has a push token).
To test accurately, use a Braze user whose `user_id` or `external_id` matches the identifier configured for the channel. You can confirm delivery in Braze by checking message logs or the user’s messaging history.
## 4. Configure Braze Campaigns and Canvases for AID
Before adding a message in Hightouch, you need to configure or create the corresponding Campaign or Canvas in Braze.
### For Campaign Triggers
[Create a Campaign](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns/creating_campaign) in Braze that supports API triggering.
**In Braze**
1. Go to **Campaigns > Create Campaign**
2. Choose your channel (Email, SMS, Push) and complete Campaign Details
- [Create an email campaign](https://www.braze.com/docs/user_guide/message_building_by_channel/email/html_editor/creating_an_email_campaign/#step-1-choose-where-to-build-your-message)
- [Create an SMS campaign](https://www.braze.com/docs/user_guide/message_building_by_channel/sms_mms_rcs/sms/create)
- [Create a push campaign](https://www.braze.com/docs/user_guide/message_building_by_channel/push/creating_a_push_message)
3. On the **Schedule Delivery** step, select [API Triggered](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns/delivery_types/api_triggered_delivery) as the Delivery Type
4. In **Target Audiences**, set a broad audience or include your test user via filters
5. In **Assign Conversions**, optionally define conversion goals
6. On the **Review** step, ensure your campaign is:
- **Active** (not saved as Draft)
- Has **Frequency Capping disabled** if applicable
- Displays a full message preview
7. Note the **Campaign ID** for reference in Hightouch
**In Hightouch**
8. Select the **Campaign** when [adding a message](/ai-decisioning/braze#3-add-messages).
9. [Test the message](/ai-decisioning/braze#test-a-message) to confirm delivery.
### For Canvas Triggers
[Create a Canvas](https://www.braze.com/docs/user_guide/engagement_tools/canvas/create_a_canvas/create_a_canvas) with an API entry event
**In Braze**
1. Go to **Canvases > Create Canvas**
2. Fill out **Canvas Details**, and note the **Canvas ID**
3. On the **Entry Schedule** step, choose [API Triggered](https://www.braze.com/docs/user_guide/engagement_tools/canvas/create_a_canvas/create_a_canvas/?tab=scheduled%20delivery)
4. In **Target Audience**, optionally add filters or include a known test user
5. In **Send Settings**, confirm eligibility settings (e.g., subscription status, quiet hours, frequency capping)
6. Use the **Build Canvas** step to drag in components (like message or delay)
7. Use the **Summary** screen to review your configuration before launch
**In Hightouch**
8. Select the **Canvas** when [adding a message](/ai-decisioning/braze#3-add-messages).
9. [Test the message](/ai-decisioning/braze#test-a-message) to confirm delivery.
AID cannot override message properties in Canvas messages. Use default
template values instead.
## Use variables in Braze
Hightouch variables allow you to personalize Braze messages using live warehouse data, any variable you’ve defined within the message such as subject line variations, or different media assets or content blocks in your Braze library.
### Access variables
1. Open a message in Hightouch and go to the `Content` tab.
2. Under **Optimized content**, expand a content type (for example, **title** or **body**) and add your variants.
3. Click the `...` menu on a variant and select `Copy code snippet` to get the variable code, for example: `{{ api_trigger_properties.${hightouch}.recommendation.variables.subject }}`.
4. In Braze, paste the variable tag into the corresponding field in your message template (for example, the **Subject** field).
### Important notes
- **Campaign triggers**: Variables are fully supported.
- **Canvas triggers**: Only template-level defaults can be used—API-triggered variables won’t override content.
---
## Troubleshooting
| Issue | Likely cause | Resolution |
| -------------------------------- | ------------------------------------------------- | ------------------------------------------------------------------------------- |
| Message fails to send | API errors or misconfigured destination | Check Hightouch logs for API errors. |
| Test user not receiving messages | Identifier mismatch | Ensure your identifier matches a valid `external_id` or user property in Braze. |
| Canvas sends not triggering | Canvas not published or entry event misconfigured | Confirm the Canvas is published and the API entry event is configured. |
---
## What’s next
- [Messages: Create and manage message variants →](/ai-decisioning/messages)
- [QA guide: Validate messages before launch →](/ai-decisioning/qa-guide)
- [Agents →](/ai-decisioning/agents)
- [Insights →](/ai-decisioning/insights)
---
## Collections
**URL:** https://hightouch.com/docs/ai-decisioning/collections
**Description:** Dynamically recommend personalized products or content
**Section:** AI Decisioning
**Audience**: Business users (marketers, lifecycle managers)
**Prerequisites**: [Set up an Agent →](/ai-decisioning/agents)
---
By the end of this article, you’ll understand:
- What a collection is and how it enhances personalization
- When and why to use collections in an AI Decisioning campaign
- How to create and configure a collection in Hightouch
- Where to use collections inside your actions
---
## What is a collection?
A **collection** is a dynamic list of items — like products, content, or offers — that AID can use to personalize messages. Collections are pulled from catalogs you’ve already synced to Hightouch.
Instead of hardcoding one product into a message, a collection **allows AID to select the best-fit item(s) for each user**, based on behavior, preferences, or campaign performance.
## When to use collections
Use collections when:
- You want to test which items perform best in a message
- You’re sending product recommendations (e.g., top sellers, “just for you”)
- You want dynamic inserts like:
- “Recommended for you” product grids
- Personalized blog/article carousels
- Contextual discounts or promo bundles
You **don’t need a collection** if your message has fixed content that doesn’t change by user.
## Navigating to Collections
To configure a collection:
1. Go to **AI Decisioning > Collections**
2. Click **Add collection**
3. Follow the three-step builder:
- **Select** a catalog
- **Filter** the catalog
- **Finalize** configuration
## How to create a collection
### Step 1: Select a catalog
Choose a data source that contains the items you want to recommend — for example, your synced `Products` or `Content` catalog.
Catalogs are created in Customer Studio. If you haven’t created a catalog yet,
navigate to **Customer Studio > Schema > Create > Catalog**.
### Step 2: Filter the catalog
Apply filters to limit which items the AI can select from. Common filters include:
- In-stock only
- Specific category or brand
- Minimum discount threshold
If you skip this step, the AI will consider all items in the catalog.
#### Example: “In-stock spring accessories over $20”
Let’s say you want to create a collection of spring accessories that are currently available and cost more than $20. This could be used in a win-back email or seasonal campaign to re-engage lapsed shoppers with trending items.
You would set up your filters like this:
- `category` **is equal to** `Accessories`
- `season` **is equal to** `Spring`
- `inventory_status` **is equal to** `In Stock`
- `price` **is greater than** `20`
This ensures the AI only recommends:
- Spring-ready accessories
- That are in stock
- And priced above $20
Use this type of collection in a personalized message like: "We picked these
trending spring accessories just for you."
### Step 3: Finalize configuration
1. Name your collection.
- Use clear, descriptive names that reflect what the collection includes and how it’ll be used—like “In-stock Spring Accessories” or “Top Picks for Win-back.” This makes it easy to recognize and reuse collections across campaigns.
3. Configure callback behavior.
- Sometimes the ideal item might not be available at the time of send. You can choose:
- **Do not trigger the action** if the item isn’t available
- **Send random items** as a fallback
You can return to a collection later and adjust filters or fallback rules.
## Eligibility (item-level targeting)
You can click into individual collections to add eligibility filters. Eligibility filters let you control which users are eligible to receive specific items from a collection. This ensures users only see recommendations that are relevant and appropriate for them.
You’ll define logic such as:
- `field` **is equal to** `static value`
- `field` **is greater than/less than** a defined value
#### Example:
- Only recommend size-specific SKUs to users who’ve bought that size
- Show luxury items only to loyalty-tier Platinum customers
| **Goal** | **Eligibility Filter Setup** |
| ------------------------------------------ | ---------------------------------------------------- |
| Show only women’s products | `user.gender` is equal to `female` |
| Recommend premium items to high spenders | `user.lifetime_value` is greater than `500` |
| Target users who’ve viewed sunglasses | `user.last_viewed_category` is equal to `sunglasses` |
| Limit discounted items to bargain shoppers | `user.segment` is equal to `Value Seeker` |
| Personalize by location | `user.region` is equal to `Midwest` |
You can stack multiple filters together using AND conditions, just like shown in the screenshot.
Use eligibility filters to avoid sending irrelevant product recommendations,
especially in campaigns where precision and context matter (like loyalty,
retention, or upsell flows).
## Using collections in actions
Once your collection is created, you can apply it inside an action:
1. Go to **AI Decisioning > Agents >** _Select an agent_ **> Configuration >** _Select an Action_
2. Navigate to the [Advanced Configuration](/ai-decisioning/agents#advanced-configuration) tab
3. Select a collection (e.g., “In-stock Products”)
4. Define how many items to recommend (e.g., 4 items per message)
Use a standard format for your product templates to make rendering easier
across channels.
## Preview and QA
1. Go to **AI Decisioning > Agents >** _Select an agent_ **> Configuration >** _Select an Action_
2. Navigate to the **Content** tab to [preview your content](/ai-decisioning/agents#preview-your-content):
- Confirm that the correct number of items are inserted
- Test fallback logic
- Ensure templates display well with various item combinations
## What’s next
[→ Insights: Analyze what’s working and optimize performance](/ai-decisioning/insights)
---
## Workspace configuration
**URL:** https://hightouch.com/docs/ai-decisioning/configuration
**Description:** One-time setup to define settings and defaults that power AI Decisioning across your Hightouch workspace
**Section:** AI Decisioning
| | |
| ----------------- | --------------------------------------------------------------------------------------------------------- |
| **Audience** | Platform admins (workspace settings), Marketers (agent and message settings) |
| **Prerequisites** |
Access to [AI Decisioning](/ai-decisioning/overview)
At least one agent created
|
_Define how AI Decisioning is configured across workspace, agent, and message levels._
---
## What you'll learn
- [Workspace-level configuration — Set global defaults](#workspace-configuration)
- [Agent configuration — Configure targeting, goals, and delivery](#agent-configuration)
- [Message configuration — Define tags, catalog recommendations, and delivery controls](#message-configuration)
For details on agent-level tabs such as Offers, Health, Localization, and Alerting, see [Agents →](/ai-decisioning/agents). For message-level configuration, see [Messages →](/ai-decisioning/messages#message-configuration), including how to [translate message content](/ai-decisioning/messages#localize-message-content) for each locale.
---
## Overview
Configuration in AI Decisioning spans multiple levels to balance global guardrails with agent-level strategy and message-level relevance.
Configuration is structured as:
- **Workspace configuration** – Shared defaults and constraints that apply across all agents, including the global user population, time zone handling, channels, scheduling limits, and Insights inputs.
- **Agent configuration** – Controls how individual agents operate within those boundaries, including eligibility, goals, scheduling rules, and suppression logic.
- **Message configuration** – Fine-tunes how specific messages behave using tags, collections, and advanced delivery controls. Messages inherit their goals from the parent agent.
Together, these layers determine which users can be evaluated, when messages can be sent, and how learning occurs based on delivery and performance data.
---
## Workspace configuration
Use workspace configuration to define the **global defaults and constraints** that apply across all agents in your Hightouch workspace.
**How to access workspace configuration**
1. Open the Hightouch app at https://app.hightouch.com.
2. Navigate to **AI Decisioning** in the left-hand navigation.
3. Select **Configuration**.
### General
#### Parent model (global user eligibility)
The **parent model** defines the complete set of users that AI Decisioning can consider across all agents. Every agent audience is evaluated **within** this model.
- **Recommended:** Use a broad, inclusive model such as `Users`
- Users not included in the parent model are **never eligible** for AI Decisioning actions, even if they match agent-level criteria
Learn more about defining parent models and preparing your data schema: [Prepare data for AID →](/ai-decisioning/prepare-data)
---
#### Time zone defaults
Configure time zone defaults to evaluate scheduling rules, frequency limits, and quiet hours using each user’s local time.
You can configure:
- **User time zone field** – A field from the parent model (for example, `user.time_zone`) formatted as a UTC offset (such as `+09:00`) or IANA string (such as `America/Los_Angeles`)
- **Fallback time zone** – Used when a user does not have a valid time zone value
Time zone settings are used to personalize send windows and enforce quiet
hours in each user’s local time.
---
#### Additional user columns
Additional user columns allow you to expose **merged or derived fields** from your data model for use in personalization and Insights. These columns extend the parent model without duplicating schema structure.
---
### Channels
Channels determine which messaging destinations are available to AI Decisioning agents and messages.
Supported channel types typically include:
- **Email**
- **SMS**
- **Push notifications**
- **In-app messaging** (inbox, popups, etc.)
- **Custom destinations**, such as S3 buckets or internal APIs
---
#### Active channels
Add channels to make them available when configuring messages within agents.
---
#### Send event tracking
Send event tracking configures how message sends are tracked from your warehouse for attribution matching. This is required for [Send success](/ai-decisioning/insights#send-success) and [Send comparison](/ai-decisioning/insights#send-comparison) data to appear in Insights, and for the [Channel send success](/ai-decisioning/agents#channel-send-success) section in the agent Health tab to report delivery status.
To configure send event tracking for a channel:
1. Navigate to **AI Decisioning → Configuration → Channels**.
2. Click on an active channel to open its settings.
3. Under **Send event tracking**, configure the following:
- **Event model**: Select an [event model](/customer-studio/schema#add-event-models) that tracks when messages are sent to users (for example, `Purchase Line Items` or a dedicated sends table).
- **Filter send events** _(optional)_: Add filters to narrow which events count as sends (for example, `Performed → Purchase Line Items → Where property is...`).
- **Campaign ID column** _(optional)_: Select the column that contains the campaign ID for matching sends to specific messages.
- **Metadata columns** _(optional)_: Track boolean flags like `is_clicked` or `is_delivered` from your sends table. Click `+ Add column` to include additional metadata.
- **Timestamp offset (hours)** _(optional)_: An offset to apply to event timestamps, useful for timezone corrections. Defaults to `0`.
- **Attribution window (minutes)** _(optional)_: The maximum time window in minutes for matching sends to recommendations. Defaults to `120`.
Once configured, the **Send Event** column in the Active channels table shows which event model is linked to each channel.
Without send event tracking, Insights views like Send success and Send
comparison will show `Not configured` for the channel. Configure at least one
send event per channel to enable delivery monitoring.
---
#### Channel-level eligibility
Channel-level eligibility restricts which users are allowed to receive messages through a specific channel.
Common use cases include:
- Enforcing opt-in requirements for SMS or push
- Applying region-based or compliance-driven delivery rules
Channel-level eligibility is evaluated **in addition to** agent- and message-level eligibility.
---
#### Archived channels
Archived channels are unavailable for new messages but remain visible for historical reference.
---
### Scheduling
#### Global frequency limit
The global frequency limit caps the total number of messages a user can receive across all agents.
---
#### Agent frequency limit
Configure agent frequency limits to apply stricter caps than the global limit.
---
#### Channel frequency limit
Apply channel frequency limits in addition to global and agent-level frequency limits.
Use channel frequency limits to:
- Enforce channel-specific fatigue controls (for example, stricter limits for SMS)
- Apply consistent limits across all agents using the same channel
- Layer channel limits on top of global and agent-level frequency caps
Channel frequency limits are evaluated in addition to global and agent frequency limits.
---
#### Quiet hours
Quiet hours prevent messages from being sent during restricted times.
They are:
- Configured per channel
- Evaluated using each user’s local time zone
- Applied across all agents and messages using that channel
---
### Agent prioritization
Use agent prioritization to control which agent takes precedence when a user is eligible for multiple agents at the same time.
When overlap occurs, prioritization prevents agents from competing for the same user during the same evaluation window.
When conflicts occur:
- Agents are evaluated based on configured priority
- Higher-priority agents take precedence
- Lower-priority agents may be skipped during the same evaluation window
#### Agent weights
Agent weights indicate the relative priority of each agent when eligibility overlaps.
Adjust weights to bias delivery toward a specific agent when users show similar likelihood to convert across multiple agents.
#### Simulated performance
Simulated performance shows expected agent performance based on historical data.
Use this view to understand how prioritization and weighting decisions may affect outcomes before changing live delivery.
---
### Feature Matrix
A **feature matrix** defines which user attributes and behavioral data from your data warehouse AI Decisioning uses during evaluation. Feature matrices connect your parent model and related models to source columns that the decision engine can reference. They also control which attributes are available for breakdowns and analysis in [Insights](/ai-decisioning/insights).
To create a feature matrix:
1. Navigate to **AI Decisioning → Configuration → Feature Matrix**.
2. Click `+ New feature matrix`.
3. Configure the matrix:
- **AI Decisioning source**: The data warehouse connection (for example, Snowflake).
- **Parent model**: The parent model from your data warehouse. To use a different parent model, update it on the AI Decisioning configuration page.
- **Name**: A descriptive name for the feature matrix (for example, `Feature Matrix 1`).
4. Click `Create feature matrix`.
Once created, the feature matrix displays all available **models** (parent, related, and event models) in the left panel. Select a model to view and configure its **source columns**, including:
- **Source Column**: The column name from your data warehouse
- **Type**: The data type (for example, `String`, `Number`, `Boolean`)
- **Cast To**: Optional type casting to override the original type
- **Dependency**: Whether the column depends on other columns
- **Insights**: Whether the column is available for analysis in [Insights](/ai-decisioning/insights)
- **Null Handling**: How null values are treated
Each feature matrix card shows the number of features selected and how many agents use it.
For a conceptual overview of how AI Decisioning uses customer features, see [User feature matrix →](/ai-decisioning/user-feature-matrix).
---
## Agent configuration
Agent configuration controls **who an agent targets, when it can act, and what it optimizes for**.
Agent configuration builds on the global defaults set in workspace configuration and applies only to the selected agent.
To learn more about how agents work overall, see [Agents](/ai-decisioning/agents).
**How to access agent configuration**
1. Open the Hightouch app at https://app.hightouch.com.
2. Go to **AI Decisioning** > **Agents**.
3. Select an agent from the list.
4. Open the **Configuration** tab.
### Eligibility
Define which users can enter the agent by selecting a target audience and applying eligibility filters.
Eligibility determines the _scope_ of users the agent can evaluate before considering goals, scheduling, or suppression.
### Goals
Select goals to define which outcomes the agent should prioritize when choosing messages.
Goals guide how the agent evaluates success across eligible users and messages.
### Scheduling
Configure scheduling to control when the agent is allowed to send messages.
Scheduling settings apply additional constraints on top of workspace-level and channel-level rules.
### Smart Suppression
Use Smart Suppression to limit sends to users with the highest predicted value.
This setting helps reduce low-impact messaging by focusing delivery on users most likely to respond positively.
To learn more, see [Smart suppression](/ai-decisioning/smart-suppression).
### Advanced
Use advanced settings to support experimentation, exclusions, and specialized agent behavior.
These settings are optional and typically used in more complex setups.
---
## Message configuration
Message configuration controls **how individual messages behave once a user is eligible**.
**How to access message configuration**
1. Open the Hightouch app at https://app.hightouch.com.
2. Go to **AI Decisioning** > **Agents**.
3. Select an agent, then open the **Messages** tab.
4. Select a message and open its **Configuration** tab.
### Tags
Add tags to describe message intent, tone, or structure and group similar messages for learning and analysis.
Tags help organize messages and provide consistent context when comparing performance across content, variants, and agents.
To learn more about defining and managing tags, see [Tags](/ai-decisioning/tags).
### Goals
Messages inherit their goals from the parent agent. The **Goals** section in message configuration displays each inherited goal along with a performance rating (for example, `Best`, `Very good`, `Good`, `Worst`) that reflects how well the message contributes to each goal relative to other messages in the agent.
Goals are configured at the agent level and cannot be edited from the message configuration tab. See [Agents](/ai-decisioning/agents#step-2-add-and-manage-goals) for details on goal configuration.
### Catalog recommendations (collections)
Use catalog recommendations to dynamically include products or content at send time.
You can you tailor messages to each user without creating separate variants.
To learn how to define and manage collections, see [Collections](/ai-decisioning/collections).
### Advanced
Use advanced message settings to apply additional eligibility filters, configure campaign attribution, and control delivery behavior.
These settings refine how and when a specific message is sent.
---
## What’s next
- [Prepare data for AID →](/ai-decisioning/prepare-data)
- Configure a destination for AI Decisioning:
- [Salesforce Marketing Cloud](/ai-decisioning/sfmc)
- [Braze](/ai-decisioning/braze)
- [Iterable](/ai-decisioning/iterable)
---
## Content Analysis & Suggestions
**URL:** https://hightouch.com/docs/ai-decisioning/content-analysis
**Description:** Analyze message structure, similarity, and performance signals to understand why content works and how to improve it.
**Section:** AI Decisioning
| | |
|----------------------|------------------------------------------------------------------------------------------------------|
| **Audience** | Marketers |
| **Prerequisites** |
Access to [AI Decisioning](/ai-decisioning/overview)
[Tags](/ai-decisioning/tags) configured for at least one message or variant.
|
*Content Analysis & Suggestions helps you understand how your messages relate to each other, why certain content performs, and where you can improve your creative library. It builds on your tag framework and provides insights to guide ongoing content planning.*
---
### Learning objectives
After reading this article, you’ll know how to:
* [Find Content Analysis and Content Suggestions](#where-to-access-content-analysis--suggestions)
* [Review message uniqueness before launch](#review-message-uniqueness-before-launch)
* [Interpret weekly Content Suggestions](#interpret-weekly-content-suggestions)
* [Ask follow-up questions using Hightouch Agents](#ask-follow-up-questions-with-hightouch-agents)
---
## Overview
Content Analysis & Suggestions helps you understand *why* some messages perform better than others. Instead of only seeing which message won, it highlights the elements—such as tone, structure, or value proposition—that influenced performance.
Inside each message, **Content Analysis** shows whether your content is unique before activating an agent. After your messages start running, the weekly **Content Suggestions** report in [**Insights**](/ai-decisioning/insights) helps you decide when to add new content, refresh existing messages, or retire content that’s no longer effective.
Together, these tools help you understand what’s working, avoid unintended duplication, and plan what to create next.
---
## Before you start
Complete these steps before using Content Analysis & Suggestions:
1. **Create a message.**
Build or update message content inside an agent.
See: [Add and manage messages →](/ai-decisioning/agents#add-and-manage-messages)
2. **Add or refine tags.**
Apply message and variant tags to describe key attributes such as tone, value proposition, or offer.
See: [AID Tags →](/ai-decisioning/tags)
### Content Analysis depends on your tags
Content Analysis uses your message and variant tags to understand tone, structure, and strategic intent.
Consistent tagging helps the system:
- Identify similar messages
- Detect overlap or redundancy
- Surface content gaps
- Provide more accurate recommendations
---
## How to use Content Analysis & Suggestions
Follow this workflow to evaluate and improve your content over time:
1. **Review Content Analysis before launch.**
In the message’s `Content` tab, check whether the message is unique and not unintentionally duplicating other messages in your library.
2. **Activate your agent.**
When your content looks distinct, enable the agent so your messages begin generating performance data.
3. **Review weekly Content Suggestions.**
In `Insights → Summary`, check for declining engagement, content gaps, or recommended improvements.
4. **Use Agents to explore insights.**
From the Summary tab, ask follow-up questions—such as “Why is this message similar to others?” or
“Which tones performed best this week?”—to understand results in plain language.
5. **Update or create new content.**
Refresh, retire, or expand content based on the insights and recommendations you discover.
---
## Review message uniqueness before launch
Content Analysis summarizes the creative attributes of a message and shows how it relates to other messages in your content library.
### View Content Analysis inside a message
Navigate to:
[`AI Decisioning → Agents`](https://app.hightouch.com/ai/flows/?from=docs) `→ Select an agent → Messages → Select a message → Content`
You'll see:
### **Tag summary**
All manual and AI-generated tags applied to the message.
### **Similar messages**
Messages with similar tone, structure, framing, value proposition, or language patterns.
Expand each item to see why the messages match.
### **Differentiation indicators**
Signals that help you determine whether your content is:
- **Distinct** — introduces new strategic or creative value
- **Similar** — overlaps with existing content
- **Highly similar** — potentially redundant or interchangeable
Use Content Analysis before activating an agent to confirm each message adds something unique.
---
## Interpret weekly Content Suggestions
Weekly suggestions appear in the [**Summary**](https://hightouch.com/docs/ai-decisioning/insights#1-summary) view of Insights:
[`AI Decisioning → Insights → Summary`](https://app.hightouch.com/ai/insights/summary/?from=docs)
You can review this report during weekly planning to decide what to adjust or create next.
The report includes:
### **Performance highlights**
Overall trends across themes, tones, or formats.
### **Content gaps**
Areas where your content library lacks variety.
### **Content to refresh**
Messages that are declining in engagement.
### **Recommended actions**
Suggestions such as:
- Refreshing visuals or copy
- Adding a new variant
- Retiring a low-differentiation message
- Expanding topics that perform well
Content Suggestions refresh automatically each week based on updated performance data.
---
## Ask follow-up questions with Hightouch Agents
From the Summary tab, select:
**Ask a follow-up question** or **Start a new chat about this report**
[**Agents**](/agents/overview) help you explore insights in plain language.
You can ask:
- “Why is this message similar to others?”
- “Which motivators performed best last month?”
- “Where are our biggest content gaps?”
- “What themes should we prioritize next?”
Learn more [**Agents overview →**](/agents/overview)
Use Agents to validate ideas or explore trends as you build your next set of content variants.
---
## Troubleshooting
| Issue | Likely cause | Resolution |
|--------|--------------|------------|
| Similar messages not appearing | Message is new or missing tags | Add or refine [tags](/ai-decisioning/tags) |
| Weekly report missing | Summary not configured | Contact support |
| Recommendations feel too general | Limited content variety or inconsistent tag usage | Refine tags or add more content |
| Fatigue insights unavailable | Not enough message sends | Allow more time for performance data to accumulate |
Content Suggestions rely on recent performance. Newly created messages may need time before insights appear.
---
## What’s next?
* [AID Tags →](/ai-decisioning/tags)
* [QA guide →](/ai-decisioning/qa-guide)
* [Hightouch Agents overview →](/agents/overview)
* [AID Insights →](/ai-decisioning/insights)
* [AID Agents →](/ai-decisioning/agents)
---
## Insights
**URL:** https://hightouch.com/docs/ai-decisioning/insights
**Description:** Analyze, learn, and optimize AI Decisioning performance
**Section:** AI Decisioning
| | |
| ----------------- | ---------------------------------------------------- |
| **Audience** | Marketers, lifecycle leads, and campaign analysts |
| **Prerequisites** | [At least one running Agent](/ai-decisioning/agents) |
_Insights helps you measure and understand how your AI Decisioning agents perform. Explore which messages, audiences, and timing strategies drive your key outcomes and use those findings to improve results over time._
---
### What you'll learn
After reading this article, you’ll know how to:
- [Navigate the Insights dashboard and its views](#overview)
- [Interpret metrics like conversion, lift, and engagement](#insights-views)
- [Explore audience, creative, and timing performance](#explore-each-view)
- [Apply insights to improve future campaign results](#apply-insights-to-improve-performance)
---
## Overview
**Insights** is the analytics hub for AI Decisioning. It shows you not only **how your agents perform**, but also **why**, so you can make data-informed creative and scheduling decisions.
Use Insights:
- **After launching an agent** to track performance in real time
- **During optimization** to identify top-performing segments or messages
- **Before making changes** to content or targeting
- **For reviews and reporting** to measure lift and summarize learnings for stakeholders
Hightouch automatically prioritizes the most relevant charts so you can focus
on the trends that matter most.
---
## Navigate to Insights
In the Hightouch app, go to `AI Decisioning` → [`Insights`](https://app.hightouch.com/ai/insights/?from=docs)
---
## Insights views
Each view highlights a different aspect of campaign performance.
| **View** | **Purpose** | **Example questions** |
| ---------------------------- | --------------------------------------------------------------------------------- | -------------------------------------------------------------- |
| **Summary** | Provides a top-level report of results, trends, and creative takeaways. | What themes or tones are performing best right now? |
| **Creative performance** | Compares message-level metrics like sends, conversions, and engagement. | Which message or subject line is driving the most conversions? |
| **User breakdowns** | Shows conversion performance by audience segment. | Which demographic or loyalty tier performs best? |
| **Personalization insights** | Highlights which offers or creative types perform best for specific groups. | What incentives resonate most with younger users? |
| **Timing** | Displays when users are most likely to engage, using a day/time heatmap. | What send windows generate the highest conversion rates? |
| **Personalization Matrix** | Displays a heatmap of conversion rates across user segments and creative options. | Where does personalization have the most impact? |
| **Send success** | Shows delivery status by channel and highlights configuration gaps. | Are my channels successfully delivering messages? |
| **Send comparison** | Compares send volume and performance across messages, channels, or time periods. | How do sends differ across messages or time periods? |
---
## Explore each view
### 1. Summary
Get an at-a-glance view of your agent’s overall performance and top insights.
The **Summary** includes:
- **Performance highlights** showing top-performing creative and audience signals
- **Patterns marketers should know**, based on behavior and conversion data
- **Creative examples** that illustrate what’s driving performance
Use this view to guide weekly reviews or summarize campaign results for your team.
---
### 2. Creative performance
Compare all message variants side by side across your key performance metrics.
| **What you might see** | **What it means** | **What to do next** |
| ---------------------------- | ----------------------------------- | ----------------------------------------- |
| High sends, low conversions | Overexposed audience or weak offer | Refresh content or narrow targeting |
| High clicks, low conversions | Link or landing page misalignment | QA destination or adjust CTA |
| High unsubscribes | Message fatigue or off-tone content | Pause or rewrite underperforming messages |
Sort the table by your main goal (for example, `Second purchase`) to identify
which creative has the strongest impact.
---
### 3. User breakdowns
Understand which user segments are driving conversions and which are underperforming. Breakdowns include attributes like age, region, loyalty tier, and device type.
| **What you might see** | **What it means** | **What to do next** |
| ----------------------------- | -------------------------------- | --------------------------------------- |
| Gen Z users underperform | Tone or visuals may not resonate | Test shorter, more visual creative |
| Loyalty users convert 2× more | Offer relevance is high | Expand similar campaigns to other tiers |
| Mobile engagement is lower | Template or UX issue | Review layout and links on mobile |
Charts are sorted automatically by AI to surface the most relevant differences
first.
---
### 4. Personalization insights
Discover which creative elements perform best for specific user segments. This view helps you understand how **audience attributes** and **creative choices** interact to drive conversions.
At the top of the view, select:
- A **key conversion event** to define _what success looks like_ (for example, `Second purchase`)
- **Creative dimensions** to compare _what_ performs best (such as subject lines, incentive types, or tags)
- **User dimensions** to compare _who_ it performs best for (such as age buckets, loyalty tiers, or device types)
Insights are automatically sorted to surface the most meaningful performance differences for the selected configuration.
| **What you might see** | **What it means** | **What to do next** |
| ---------------------------------------------- | --------------------- | -------------------------------------- |
| Discounts lift users aged 18–24 | Price sensitivity | Prioritize discounts for this segment |
| Loyalty users respond to bundles | Value-driven behavior | Feature multi-product or tiered offers |
| Push outperforms SMS for high churn-risk users | Channel preference | Shift allocation toward push |
Use personalization insights to inform message tags, variant strategy, and
audience splits in [Agents → Messages](/ai-decisioning/agents#messages).
---
### 5. Timing
Analyze when users are most likely to convert or engage, visualized as a heatmap by day and time of day.
| **What you might see** | **What it means** | **What to do next** |
| ----------------------------- | ----------------------------- | ------------------------------------ |
| Evenings convert best midweek | Post-work engagement | Shift sends later in the day |
| Weekend conversion spikes | Browsing or planning behavior | Test Saturday or Sunday reminders |
| Monday performance drops | Inbox saturation | Schedule key sends later in the week |
Use **Timing** results to refine [Scheduling and quiet
hours](/ai-decisioning/agents#step-3-optional-configure-scheduling) for your
agent.
---
### 6. Personalization Matrix
The **Personalization Matrix** displays a heatmap of conversion rates across user segments and creative options, highlighting where personalization has the most impact.
At the top of the view, select:
- **Outcome name**: The goal to measure (for example, `Purchases`)
- **Creative dimension**: The creative variable to compare (for example, `Incentive Type`)
- **User dimension**: The user attribute to segment by (for example, `Age Bucket`)
The heatmap shows conversion rates at each intersection, colored by intensity. Toggle between **Conversion rates** and **Interaction effects** to see raw performance or the incremental impact of personalization.
| **What you might see** | **What it means** | **What to do next** |
| ----------------------------------------------------------------------- | --------------------------------------------- | -------------------------------------------------- |
| `fixed_discount` converts 5.96% for 25–34 year olds but 1.59% for 55–64 | Strong age-based preference for discount type | Personalize incentive by age segment |
| `free_gift` performs consistently across all age groups | Broad appeal, low personalization value | Use as a safe default when segment data is limited |
| `pct_discount` underperforms across most segments | Weak creative option overall | Consider retiring or reworking the variant |
Use the Personalization Matrix to identify which creative-audience
combinations to prioritize, and feed those insights back into your [variant
strategy](/ai-decisioning/messages#variants) and [tags](/ai-decisioning/tags).
---
### 7. Send success
The **Send success** view shows whether your channels are successfully delivering messages. It displays each channel connected to the agent, its configuration status, and message-level match rates between recommendations and confirmed sends.
All send data on this page comes from the send event table configured in your
schema. If that table is incomplete — for example, missing your AID campaigns
or not kept up to date — match rates and send counts will appear lower than
expected. Always verify your send event table is correctly defined before
drawing conclusions from these charts.
A **channel configuration** card at the top of the page shows how each enabled channel is set up — its destination, send event model, and campaign ID column. If a channel is missing a send event or campaign ID column, a warning icon appears next to it. These are the most common reasons match rate looks low or attribution is incomplete.
The **Match rate by message** grid at the bottom of the page is interactive. You can filter by channel type, sort by value to find best- or worst-performing messages, click a message name to open it directly in Hightouch, or download a CSV for offline analysis.
Each chart on the page displays its numerator and denominator (for example, "Matched sends / Recommendations") underneath the title. Hover any data point to see the absolute counts behind the ratio.
| **What you might see** | **What it means** | **What to do next** |
|------------------------|------------------|---------------------|
| `Not configured` warning on a channel | Send events are not linked to this channel | Set up [send event tracking](/ai-decisioning/configuration#send-event-tracking) for the channel |
| Low match rate across all messages | Send event table may be incomplete or outdated | Verify the send event table includes your AID campaigns |
| Low match rate on a single message | That message's campaign ID may not be attributed correctly | Check the campaign ID column in your channel's attribution config |
Send success tracking requires send events to be linked to your channels. See
[Send event tracking](/ai-decisioning/configuration#send-event-tracking) for
setup instructions. Without this, send success and send comparison data will
not populate.
---
### 8. Send comparison
The **Send comparison** view lets you compare send patterns between treatment and customer\_managed groups. Use it to confirm the experiment is shifting volume where it should and to spot channel- or timing-level differences.
Send comparison includes charts for weekly sends per user broken down by experiment group and channel type, by experiment group and campaign group (this agent, other agents, customer\_managed), and heatmaps of send count by time of day and day of week. Each heatmap cell shows its share of the group's total so you can spot whether AI Decisioning is shifting send timing.
| **What you might see** | **What it means** | **What to do next** |
|------------------------|------------------|---------------------|
| Treatment sends more on push, less on email | AI Decisioning is reallocating channel mix | Review whether the shift aligns with your channel strategy |
| Send timing differs between groups | AI Decisioning is optimizing delivery windows | Compare with [Timing insights](#5-timing) to confirm the pattern |
| Volume is flat across experiment groups | The experiment may not be shifting behavior yet | Check that the agent has had enough time to learn and that campaign IDs are configured correctly |
Like Send success, the Send comparison view requires send events to be
configured on your channels. See [Send event
tracking](/ai-decisioning/configuration#send-event-tracking) for setup
instructions.
---
## Apply insights to improve performance
Once you’ve reviewed your results, use them to guide your next optimization cycle.
| **If you see...** | **Then...** |
| -------------------------------------------- | ------------------------------------------------- |
| High lift for a segment | Create a dedicated agent or audience to scale it |
| Flat or declining conversions | Revisit your goals, tags, or creative mix |
| Frequent “bad” outcomes (e.g., unsubscribes) | Adjust Smart Suppression or reduce frequency |
| Specific time windows outperforming others | Update Scheduling settings to match user behavior |
Combine insights from **User breakdowns**, **Creative performance**,
**Timing**, and the **Personalization Matrix** to identify your most impactful
opportunities.
---
## Example weekly optimization checklist
✔️ Review **Summary** for top and bottom performers
✔️ Check unsubscribe and fatigue signals in **Creative performance**
✔️ Compare lift vs. holdout for each goal or segment
✔️ Identify 1–2 test ideas (e.g., new variant or timing window)
✔️ Check **Send success** to confirm channels are delivering correctly
✔️ Review **Send comparison** to verify experiment groups are shifting volume as expected
✔️ Share findings with your team to inform future content and campaign planning
---
## Inspector
**URL:** https://hightouch.com/docs/ai-decisioning/inspector
**Description:** View and QA delivery behavior for individual users
**Section:** AI Decisioning
| | |
|----------------------|-----------------------------------------------------------------|
| **Audience** | Marketers, business users |
| **Prerequisites** | [At least one Agent →](/ai-decisioning/agents) |
*Inspector helps you understand how messages are delivered to individual users. Use it to QA campaigns, troubleshoot delivery behavior, and explain outcomes with confidence.*
---
## What you’ll learn
After reading this article, you’ll know how to:
- [Navigate the Inspector interface](#navigate-to-inspector)
- [Review a user’s journey across agents](#review-a-users-experience)
- [Understand message decisions and suppression](#understand-message-decisions)
- [Use Inspector for QA, troubleshooting, and reviews](#when-to-use-inspector)
---
## Overview
**Inspector** helps you understand how messages are delivered to an individual user.
Instead of aggregated metrics, Inspector shows *exactly what happened* for a specific person—step by step—across one or more agents. This makes it easy to validate campaign behavior and explain outcomes with confidence.
In Inspector, you can review a user’s **full journey across agents**, including:
- When the user becomes eligible for an agent
- Which messages were evaluated and which were sent
- When sends were skipped due to guardrails or Smart Suppression
- Which content and variants were selected
- Why a specific decision was made at that moment
Inspector is most useful during **campaign setup, QA, troubleshooting, and reviews**, when you need to confirm that agents are behaving as expected for real users.
It complements [**Insights**](/ai-decisioning/insights), which focuses on performance trends and lift across many users, by providing a clear, user-level explanation of delivery behavior.
---
## Navigate to Inspector
In the Hightouch app, go to:
`AI Decisioning` → [`Inspector`](https://app.hightouch.com/ai/inspector?from=docs)
From here, you can:
- Filter by **All agents** or a specific agent
- Search for a user
- Step forward or backward between users
- Review each user’s experience over time
---
## Review a user’s experience
Inspector is organized into three main areas that work together to explain delivery behavior.
---
### User attributes
The left panel shows the **full set of user attributes available to agents**.
These attributes help explain:
- Eligibility and agent entry
- Message selection and personalization
- Suppression or exit behavior
This view represents the broader user context agents can use when evaluating decisions across the user’s journey.
The attributes shown here represent **everything the agent knows** about the user.
Not every attribute directly influences every message decision.
---
### Experience timeline
The center panel shows a chronological timeline of the user’s journey across agents.
You may see events such as:
- **Eligible for agent**
- **Wait periods** between evaluations
- **Messages sent**, including channel and agent
- **Skipped sends**, including Smart Suppression
- **Agent exit** events
- **Outcome events**, such as clicks or purchases (when available)
This timeline makes it easy to understand both **what happened** and **what didn’t happen**, in context.
---
### Message details
Select any message card in the timeline to open the message details drawer.
This view explains **what was sent** and **why it was selected**.
The drawer includes several tabs:
| Tab | Purpose |
|-----|--------|
| `Preview` | View the rendered message content as delivered to the user (when supported). |
| `Data` | Inspect the payload sent to the destination, including variables and metadata. |
| `Reasoning` | Understand why this message was selected, including key attributes that influenced the decision. |
| `Internal` | View internal identifiers and diagnostic details used for support and debugging. |
Use these views together to validate content, confirm personalization logic, and explain message decisions.
---
## Understand message decisions
After reviewing a user’s overall journey, use the `Reasoning` tab in the message details drawer to understand **why a specific message was chosen at that moment**.
The `Reasoning` tab surfaces the key inputs and signals that influenced the agent’s decision.
When reviewing a message’s reasoning, you’ll see:
- A **subset of user attributes** that most influenced selection
- Whether the decision was:
- **Personalized** — selected because it is expected to perform well for this user based on similar users and prior outcomes
- **Exploratory** — selected to gather more data and learn how it performs
- The **delivery channel** used for the message
- Context about **how the agent evaluated available options** at decision time
This view is especially useful when you need to explain **why one message or variant was chosen over another**, or when validating that agents are behaving as expected before scaling a campaign.
The attributes shown in the left panel represent the **full user context** available to agents. The `Reasoning` tab highlights only the attributes that were most relevant for this specific decision.
---
## When to use Inspector
Use Inspector when you need a **clear, user-level explanation** of how an agent behaved for a specific person.
Inspector is best suited for questions like:
- *Why did this user receive this message?*
- *Why was a send skipped or suppressed?*
- *What personalization or attributes influenced the decision?*
- *How did this user’s experience unfold across agents?*
Common scenarios include:
| Scenario | How Inspector helps |
|--------|---------------------|
| QA a new agent | Preview upcoming messages and confirm eligibility before launch |
| Troubleshoot skipped sends | Identify Smart Suppression, wait periods, or exit conditions |
| Validate personalization | Confirm variables, recommendations, and attributes used |
| Review user journeys | See how experiences unfold across multiple agents |
| Stakeholder reviews | Walk through real user examples to explain outcomes |
For performance trends, lift analysis, and insights across many users, use [Insights →](/ai-decisioning/insights).
---
## What’s next
- [Agents → Configure audiences, messages, and goals](/ai-decisioning/agents)
- [QA guide → Channel-specific checklists for email, SMS, and push](/ai-decisioning/qa-guide)
- [Insights → Analyze performance and lift](/ai-decisioning/insights)
- [Smart Suppression → Control low-impact sends](/ai-decisioning/smart-suppression)
---
## Iterable integration with AI Decisioning
**URL:** https://hightouch.com/docs/ai-decisioning/iterable
**Description:** Learn how to connect Iterable to AI Decisioning in Hightouch to send personalized, real-time messages across email, SMS, push, and in-app channels.
**Section:** AI Decisioning
| | |
|----------------------|------------------------------------------------------------------------------------------------------|
| **Audience** | Technical marketers, marketing and lifecycle teams |
| **Prerequisites** | [Prepare data for AID →](/ai-decisioning/configuration#prepare-data-for-aid) |
*Connect Iterable to AI Decisioning so agents can deliver personalized emails, SMS, push notifications, and in-app messages through your existing Iterable Triggered Campaigns.*
---
## What you'll learn
In this article, you'll learn how to:
- [Set up Iterable as a destination](#1-set-up-iterable-as-a-destination)
- [Add channels in AI Decisioning](#2-add-channels-in-ai-decisioning)
- [Add messages from Iterable](#3-add-messages)
- [Configure Iterable Triggered Campaigns for AID](#4-configure-iterable-triggered-campaigns-for-aid)
- [Use dynamic variables in Iterable templates](#use-variables-in-iterable)
---
## Overview
Delivering relevant and timely customer experiences in Iterable requires up-to-date, contextual data. By integrating Hightouch's AI Decisioning (AID) with Iterable, you can:
- Automatically trigger Iterable campaigns based on predictions and behaviors surfaced by AID Agents
- Personalize message content using dynamic variables from your data warehouse
- Centralize targeting and decision logic in Hightouch, while using Iterable to handle message delivery
## 1. Set Up Iterable as a Destination
Follow the [destination setup guide](/destinations/iterable) up to **Sync Configuration**. Once connected, Iterable will be available as a destination in AI Decisioning.
---
## 2. Add Channels in AI Decisioning
Once Iterable is set up as a destination, you can add **channels**. Channels define how Hightouch sends messages to Iterable via Triggered Campaigns.
### Set up a channel
1. Go to `AI Decisioning → Configuration → Channels`.
2. Click `Add channel`.
3. Choose a channel type:
- **Email** – for triggered email sends
- **SMS** – for triggered text messages
- **Push** – for mobile push notifications
- **In-App** – for in-app messaging (popups, inbox cards, etc.)
4. Select **Iterable** as the destination.
5. Under **Identifier type**, choose how to match users:
- **Email** – match by email address
- **User ID** – match by unique ID
6. Choose the identifier field from your model (for example, `user.email` or `user.id`).
7. Click `Add channel`, then `Save changes`.
### Identifier type comparison
| Identifier | Description | Use when |
|------------|-------------|----------|
| Email | Matches users based on email address | Campaigns primarily use email for targeting |
| User ID | Matches users using their unique ID | Source data uses internal IDs or email isn't guaranteed |
Both identifier types support all channel types (Email, SMS, Push, and In-App).
---
## 3. Add Messages
After you add channels in AI Decisioning, you can add **messages** to your agents. Each message connects an agent's decision to an Iterable Triggered Campaign.
For full details on message configuration, variants, and tags, see [Messages →](/ai-decisioning/messages).
### Add a message from Iterable
1. Go to `AI Decisioning → Agents` and select your agent.
2. Open the `Messages` tab.
3. Click `Add messages`, then select **Add messages individually**.
4. Select your configured Iterable **channel and destination** tile (for example, **Email - Iterable**).
5. Select an Iterable template.
6. Enter a **message name** (for example, `Cart Abandonment SMS`) and click `Add messages`.
When you create a message, Hightouch **clones** the selected Iterable template and creates a new Triggered Campaign. Hightouch uses this new campaign to send messages via API—not the original template. See [How Iterable templates and campaigns work with Hightouch](#how-iterable-templates-and-campaigns-work-with-hightouch) for details.
You can repeat this process for each channel type:
| Channel type | Campaign type | Description |
|--------------|---------------|-------------|
| Email | Triggered | Real-time personalized emails |
| SMS | Triggered | Time-sensitive text alerts |
| Push | Triggered | Native app push notifications |
| In-App | Triggered | Contextual popups and messages |
### Test a message
1. Open the message and go to the `Content` tab.
2. Under **Optimized content**, confirm your variants are configured. If your message uses [dynamic variables](#use-variables-in-iterable), set values for testing.
3. In the **Preview** panel, verify the rendered message looks correct.
4. Click `Send test` to send the message to a test user in Iterable.
---
## How Iterable Templates and Campaigns Work with Hightouch
When creating an Iterable message in Hightouch, you're not sending the base template directly. Instead, Hightouch **clones** the template to create a new Triggered Campaign.
### What happens behind the scenes
1. You select an existing Iterable template (by Template ID).
2. When you click **Add messages**, Hightouch:
- Clones the template
- Creates a new **Campaign ID**
3. Hightouch uses the **new campaign ID** to send messages via API—not the original template.
### Why this matters
- Hightouch sends through the **new campaign**, not the original.
- To update message content, edit the new campaign version—not the base template.
### How to find the right campaign
- Open the **Content** tab in the message:
`AI Decisioning → Agents → [Select Agent] → Messages → [Select Message] → Content`
- Click the link next to **Base content** to open the Iterable campaign.
---
## 4. Configure Iterable Triggered Campaigns for AID
Before adding a message in Hightouch, you need to configure or create the corresponding Triggered Campaign in Iterable.
### In Iterable
1. Go to **Messaging > Campaigns**.
2. Click **New Campaign > Triggered Campaign**.
3. Select the message type.
4. Design your template and publish the campaign.
5. Copy the **Campaign ID** or **API Trigger Name**.
### In Hightouch
6. Select the **template** when [adding a message](#3-add-messages).
7. [Test the message](#test-a-message) to confirm delivery.
---
## 5. Use Variables in Iterable Templates
Hightouch variables allow you to personalize Iterable messages using live warehouse data, any variable you've defined within the message such as subject line variations, or different content blocks in your Iterable library.
### Access variables
1. Open a message in Hightouch and go to the `Content` tab.
2. Under **Optimized content**, expand a content type (for example, **subject** or **body**) and add your variants.
3. Click the `...` menu on a content type and select `Copy code snippet` to get the variable code.
4. In Iterable, paste the variable tag into the corresponding field in your message template.
### Important notes
- Variables are fully supported in Triggered Campaigns.
- Make sure your Iterable template references the correct variable syntax from the code snippet.
---
## Troubleshooting
| Issue | Likely cause | Resolution |
|-------|-------------|------------|
| Message fails to send | API errors or misconfigured destination | Check Hightouch logs for API errors. |
| Test user not receiving messages | Identifier mismatch | Verify the identifier (email or user ID) matches a record in Iterable. |
| Campaign sends not triggering | Campaign not published or misconfigured | Confirm the campaign is live and triggerable in Iterable. |
| Variables not rendering | Incorrect syntax or missing field mapping | Double-check variable syntax from `Copy code snippet` and field mapping. |
---
## What's next
- [Messages: Create and manage message variants →](/ai-decisioning/messages)
- [QA guide: Validate messages before launch →](/ai-decisioning/qa-guide)
- [Agents →](/ai-decisioning/agents)
- [Insights →](/ai-decisioning/insights)
---
## Messages
**URL:** https://hightouch.com/docs/ai-decisioning/messages
**Description:** Create, manage, and optimize message variants that AI Decisioning evaluates and improves over time.
**Section:** AI Decisioning
| | |
| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers, business users |
| **Prerequisites** |
At least one [Agent](/ai-decisioning/agents)
A connected destination (Braze, Iterable, or Salesforce Marketing Cloud)
|
_Add Messages to optimize content from your destination templates by defining variants and letting agents control which version is sent, when, and to whom._
---
## What you’ll learn
In this article, you’ll learn how to:
- [Add messages from your destination](#add-a-message)
- [Understand base vs optimized content](#message-content)
- [Create and manage message variants](#add-and-manage-variants)
- [Control variant eligibility with rules and tags](#variant-level-controls)
- [Translate message content for a user's locale](#localize-message-content)
- [Configure message-level settings](#message-configuration)
- [Manage messages at scale with CSV uploads and bulk actions](#manage-messages-at-scale)
## Overview
**Messages** define the creative variations an [agent](/ai-decisioning/agents) can send. Each message is sourced from a destination (such as Braze or Iterable) and can include multiple variants for experimentation.
Agents continuously evaluate messages to determine which variant, timing, and channel perform best for each user.
---
## Add a message
1. Open an agent and go to the `Messages` tab.
2. Click `Add messages`, then select **Add messages individually**.
3. In the **Add messages** window:
1. Select a **channel and destination**.
- A **channel** defines how a message is delivered (for example, email or push), while the **destination** defines where the content is managed (such as Braze or Iterable).
2. Select the destination-specific identifier:
- **Braze**: Choose a campaign
- **Iterable**: Choose a template
- **Salesforce Marketing Cloud**: Choose a triggered send or message
- **Custom (S3)**: Enter a resource identifier
3. Enter a **message name** to identify the message in Hightouch.
4. Click `Add messages`.
The message is added to the agent and appears in the Messages list.
Use message names that describe the intent or use case (for example,
`Cross-sell – accessories email`) to make messages easier to find and analyze
later.
---
### View and manage messages
The `Messages` page provides a centralized view of message configuration and performance for an agent.
From this view, you can:
- Review **message status** (`Enabled` or `Draft`)
- See the number of **variants** per message
- View the **audience size** — the estimated number of eligible users for each message
- View applied **tags**
- Search messages by **name**
- Filter messages by **status**
Select a message to access three views:
| Tab | Purpose |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `Configuration` | Review and manage message settings including tags, offers, collections, and advanced delivery guardrails (send limits, timeline, delivery windows, eligibility, and campaign attribution). Goals inherited from the agent are also visible here. |
| `Usage` | See how often the message is eligible to send and how it is being evaluated by the agent. |
| `Performance` | Review delivery and outcome metrics to understand how the message contributes to agent goals. |
Use these views together to confirm setup and understand how messages perform over time.
---
## Message content
Each message includes two parts: **base content** and **optimized content**.
### Base content
**Base content** is the underlying message content defined in your marketing destination.
Examples include:
- An email built in Iterable, Braze, or Salesforce Marketing Cloud
- A push or in-app message defined in your destination
- Destination-managed layout, variables, or logic
When you add a message, you select the destination and the campaign, template, or send to use. That destination-managed content becomes the base content for the message.
AI Decisioning does not modify your base content. It uses your content as the
foundation for optimized content and variants.
---
### Optimized content
**Optimized content** includes the specific fields within a message that agents can vary and learn from over time.
Common optimized content types include:
- Subject
- Preheader
- Call to action (CTA)
- Other supported message fields
Each optimized content type can include multiple variants, which are evaluated against your agent’s goals.
---
## Variants
A **variant** is one version of an optimized content type (for example, a subject line option).
Variants are the individual options that agents compare and learn from over time.
Variants differ from traditional A/B tests:
- Traffic allocation adapts over time
- There is no fixed test window
- No manual winner selection is required
---
## Add and manage variants
Variants are added inside an individual message, in the `Content` tab.
1. Open an agent and go to the `Messages` tab.
2. Select a message.
3. Open the `Content` tab.
4. Expand an optimized content type (for example, `Subject`).
5. Click `Add variant`.
6. Enter the variant content.
You can add multiple variants to the same optimized content type.
Start with a small number of clear variants that differ meaningfully in tone,
framing, or offer.
---
## Variant-level controls
Each variant includes controls that determine when it is eligible to send.
To manage variant-level settings, open a message and go to the `Content` tab. Under **Optimized content**, click `Edit tags and rules`.
From here, you can add or update:
- Tags
- Audience rules
- Date rules
- Send limits
---
### Audience rules
Audience rules limit a variant to specific users.
Examples:
- Show a loyalty-focused subject line only to returning customers
- Restrict a discount variant to lapsed users
Audience rules apply at the **variant level**, not the message level.
---
### Date rules
Date rules control when a variant is active.
Use date rules to:
- Run seasonal or time-bound messaging
- Automatically retire expired content
- Stage future variants ahead of launch
Expired variants remain visible but are no longer eligible to send.
---
### Send limits
Send limits control how often the same variant can be sent to a single user.
Available options include:
- **No limit**
- **Static limit** (for example, once per user)
- **Time window** (for example, once every 30 days)
Send limits apply to variants individually, not to the entire message.
---
### Tags
Tags describe the intent or attributes of a variant, such as:
- Tone (`friendly`, `urgent`)
- Offer type (`discount`, `free gift`)
- Value proposition (`convenience`, `exclusivity`)
Tags help agents group and analyze content and are used in [Content Analysis](/ai-decisioning/content-analysis) and [Insights](/ai-decisioning/insights).
See [Tags →](/ai-decisioning/tags) for guidance on tagging strategy.
AI Decisioning only compares variants that are eligible at the time of send.
Audience rules, date rules, and send limits determine which variants are
considered.
---
## Localize message content
If your agent's [localization](/ai-decisioning/agents#localization) is enabled, you can translate individual message variants so each user receives content in their own language. You provide the translations — Hightouch doesn't generate them — and AI Decisioning sends the one that matches each user's locale.
Localizing message content requires localization to be enabled first, at the
agent level. See [Enable localization](/ai-decisioning/agents#localization).
### Enable localization on a content type
1. Open a message and go to the `Content` tab.
2. Next to an optimized content type (for example, `subject`), select `···`.
3. Toggle on `Use localization`.
You can mix localized and non-localized content types on the same message. Leave fixed values that don't need translation, such as a promo code or product ID, non-localized.
### Add translations for each variant
Once localization is on for a content type, add a translation for every configured locale on each variant.
- To translate one variant, select the globe icon next to it to open the **Localization** dialog, enter a value for each locale, and select `Apply`.
- To review or edit every variant in one language at a time, use the locale dropdown next to **Optimized content** to switch the table to that language.
Use [CSV bulk-edit](#bulk-edit-translations-with-csv) to add translations for
many variants at once instead of editing them one by one.
### Check translation status
Each localized content type shows a badge summarizing whether its variants are fully translated, and each variant shows its own badge — hover it to see which locales still need a value.
A **Localization incomplete** banner also appears at the top of the message until every localized variant has a translation for every locale.
### Bulk-edit translations with CSV
Translations use the same [CSV upload workflow](#bulk-upload-and-export-messages-csv) as other message content. When localization is enabled, the CSV includes one column per configured locale, and the import review step shows a per-language diff for each row before you save.
### Test translations before launch
Use `Send a test` on a message and select a sample user and a locale. The preview reflects exactly what AI Decisioning sends, so you can confirm each translation independently before launch.
---
## Manage messages at scale
AI Decisioning supports two bulk workflows for managing messages:
- [**CSV upload**](#bulk-upload-and-export-messages-csv) for content and metadata updates
- [**Bulk actions**](#apply-message-guardrails-in-bulk) for applying message guardrails
Use CSV upload for structured content changes and bulk actions for delivery controls.
---
### Bulk upload and export messages (CSV)
Use bulk CSV upload to add or update messages in an agent at scale. This workflow is best for structured, repeatable updates that are easier to manage in a spreadsheet.
#### What you can manage with CSV upload
Using a CSV file, you can:
- Add new messages to an agent
- Update existing messages
- Add or modify:
- Message content
- Variables and variable options
- Message- and variant-level tags
- [Translations](#localize-message-content) for localized content types — the CSV includes one column per configured locale
#### What CSV upload does not support
CSV upload does not support message guardrails, including:
- Eligibility filters
- Send limits
- Campaign attribution
Apply these settings in the product UI using [bulk actions](#apply-message-guardrails-in-bulk).
#### Upload messages with CSV
1. Go to `AI Decisioning → Agents` and select an agent.
2. Open the `Messages` tab.
3. Click `Add messages`, then select `Bulk upload with CSV`.
4. Download the provided CSV template.
5. Edit the template with your updates.
6. Upload the completed CSV to apply changes.
You must use the provided CSV template. Uploads will fail if required columns
are missing or renamed.
---
### Apply message guardrails in bulk
Use bulk actions to apply message-level guardrails to multiple messages at once.
#### Supported bulk guardrails
You can apply:
- Eligibility filters
- Send limits
- Campaign attribution values
#### Apply guardrails in bulk
1. Go to `AI Decisioning → Agents` and select an agent.
2. Open the `Messages` tab.
3. Select one or more messages.
4. Click `Actions`.
5. Choose one of the following:
- `Add eligibility filters`
- `Add send limits`
- `Add campaign attribution`
6. Configure the settings and apply them.
When applying eligibility filters in bulk, the **Update mode** setting controls how the new filters interact with existing values:
- **Add to existing guardrails** — new conditions are added alongside any existing guardrails on each selected message. Existing values are preserved.
- **Replace existing guardrails** — overwrites existing guardrail settings on each selected message with the new values.
Bulk actions update message configuration only. They do not change message
content, variants, or tags.
---
## Message configuration
The `Configuration` tab contains message-level settings that control tagging, offer attribution, collections, and advanced delivery guardrails. It also displays the goals inherited from the parent agent.
To access these settings, open an agent, go to the `Messages` tab, select a message, and open the `Configuration` tab.
---
### Tags
Tags are key-value pairs that describe a message's intent or attributes, such as `motivator = efficiency` or `value_proposition = convenience`.
Tags can be added manually or generated automatically using the **Generate tags with AI** button, which analyzes the message content and suggests relevant tags.
Message-level tags apply to the message as a whole and are separate from [variant-level tags](#tags-1). Both are used in [Content Analysis](/ai-decisioning/content-analysis) and [Insights](/ai-decisioning/insights).
See [Tags →](/ai-decisioning/tags) for guidance on tagging strategy.
---
### Offers
Attach an offer to have the agent factor the message's cost and value into its decisions. Messages with higher expected values (value minus cost, weighted by conversion likelihood) are more likely to be selected. Each message can have at most one attached offer, and not all messages require one.
To attach an offer, click `Add offer` in the **Offer** section and configure the following:
- **Name**: A short, human-readable name (for example, `$10 off first order`).
- **Description** _(optional)_: Additional context about the offer.
- **External ID** _(optional)_: A stable identifier in your source system. Required for CSV upserts to match existing offers.
- **Tags** _(optional)_: Key-value attributes to help organize and categorize offers.
- **Offer type**: **Static** offers apply the same cost to every user. **Column** offers read per-user values from warehouse columns.
- **Cost value**: A fixed cost associated with each conversion (for example, $10 off).
- **Cost multiplier**: A discount rate applied to each conversion's value (for example, `0.2` = 20% discount).
Exactly one of cost value or cost multiplier must be non-zero.
---
### Goals
The **Goals** section displays the goals inherited from the parent agent, along with a performance rating for each goal (for example, `Best`, `Very good`, `Good`, `Worst`). These ratings reflect how well the message contributes to each goal relative to other messages in the agent.
Goals are configured at the agent level and cannot be edited from the message Configuration tab. See [Agents →](/ai-decisioning/agents) for details on goal configuration.
---
### Collections
[Collections](/ai-decisioning/collections) let you attach item recommendations to a message. For example, you can include product recommendations from a "In-stock Products" collection alongside the message content.
To configure collections, select one or more available collections and set how many items to recommend per message (for example, `Recommend 4 item(s) per message`).
Collections are optional. See [Collections →](/ai-decisioning/collections) for details.
---
### Advanced settings
The **Advanced** section contains delivery guardrails that control how, when, and to whom a message is sent. Expand the **Advanced** section to access these settings.
#### Send limits
Send limits control the maximum number of times a message can be sent to the same user.
Available options include:
- **No limit**: The message can be sent without restriction.
- **Static**: A fixed cap (for example, send at most 3 times per user).
- **Time window**: A rolling limit (for example, once every 30 days).
- **Minimum other messages in between**: Requires a set number of other messages to be sent before this message can be sent again.
Message-level send limits apply to all variants of the message. They are separate from [variant-level send limits](#send-limits), which apply to individual variants.
#### Timeline
Set a start and end date for a message. Use timeline settings to stage messages ahead of a launch or automatically stop a message after a campaign ends.
If no dates are set, the message is eligible to send as long as it is enabled.
#### Delivery windows
Delivery windows let you restrict when a message can be sent by specifying allowed days and time slots. This gives you direct control over message timing without relying on complex channel or schedule configurations.
To set a delivery window:
1. Toggle on **Delivery windows**.
2. Select the days and time blocks during which the message is allowed to send.
Time blocks that are not selected are blocked, meaning the agent will not send the message during those windows.
Use delivery windows to send promotional emails only during morning hours or
restrict campaigns to specific days, such as midweek-only sends.
#### Eligibility
Eligibility filters control which users can receive the message. The base eligibility filter is inherited from the agent and cannot be edited at the message level. You can add additional filters to further restrict the audience.
Click `Calculate size` to estimate how many users match the current eligibility criteria.
#### Campaign attribution
Add additional campaign IDs for outcome attribution purposes. This lets you associate message performance with specific campaigns in your destination, even if the message is used across multiple campaigns.
For a step-by-step checklist to verify all Configuration tab settings before
launch, see the [QA guide →](/ai-decisioning/qa-guide).
---
## How Messages connect to Content Analysis
As your agent runs, message and variant data feeds into **Content analysis and suggestions**, where you can:
- Review content similarity before launch
- Identify overlapping or redundant variants
- Understand which themes or tones perform best
- Decide when to add, refresh, or retire content
See [Content analysis and suggestions →](/ai-decisioning/content-analysis) for guidance.
---
## What’s next
- [Agents →](/ai-decisioning/agents)
- [Tags →](/ai-decisioning/tags)
- [Content analysis and suggestions →](/ai-decisioning/content-analysis)
- [QA guide: Validate messages before launch →](/ai-decisioning/qa-guide)
- [Insights →](/ai-decisioning/insights)
---
## AI Decisioning overview
**URL:** https://hightouch.com/docs/ai-decisioning/overview
**Description:** AI Decisioning automates campaign decisions at the individual user level — choosing the best message, channel, and timing for each person based on what's working.
**Section:** AI Decisioning
| Audience | How you'll use this article |
| -------------- | ------------------------------------------------------------------------------------------------------------------- |
| **Marketing** | Understand what AI Decisioning does, how agents work, and how to set up your first adaptive campaign. |
| **Data teams** | Understand what data AI Decisioning needs, how to configure your workspace, and how agents connect to destinations. |
_AI Decisioning (AID) automates campaign decisions at the individual user level. You define agents that combine an audience, goals, and messages — then AID learns which message, channel, and timing perform best for each person._
---
## What you'll learn
- [What AI Decisioning is and what problems it solves](#overview)
- [How agents work](#how-agents-work)
- [Core components](#core-components)
- [How to get started, by role](#get-started)
- [When to use AI Decisioning](#when-to-use-ai-decisioning)
- [How AI Decisioning connects to other Hightouch products](#how-ai-decisioning-fits-into-hightouch)
---
## Overview
Most lifecycle campaigns are built with static rules: if a user does X, send message Y. That can work for simple campaigns, but it becomes harder to manage as audiences, messages, and channels increase.
AI Decisioning uses event data to learn which messages, channels, and timing work best for each user. Instead of writing rules for every scenario, you define what success looks like — purchases, sign-ups, engagement — and AID uses performance data to choose the message, channel, and timing for each person.
The core unit is an **agent**. Each agent combines everything a campaign needs to learn and improve: an audience, goals, messages, scheduling rules, and guardrails. Once running, the agent uses event data as feedback and continuously adjusts which messages, channels, and timing it uses.
Agents operate entirely within the inputs and rules you define. They don't create new content or act outside your configuration.
[**Explore the interactive architecture diagram**](https://architecture.hightouch.com/?from=docs&product=ai-decisioning&utm_source=hightouch-docs&utm_medium=referral&utm_campaign=architecture-explorer&utm_content=ai-decisioning) — see how data flows and where AI Decisioning fits in your stack.
#### Example: Onboarding campaign that learns what converts
You want new users to complete their first purchase within 14 days of signing up. Instead of building a fixed drip sequence, you create an agent with a "first purchase" goal, add several message variants across email and push, and let AID decide which variant and channel each user gets. Over time, the agent learns that discount-forward messages work better for price-sensitive segments while product-focused messages convert power users — without you writing rules for each case.
#### Example: Win-back campaign with smart suppression
Lapsed users haven't engaged in 60 days. You create an agent targeting that audience with a re-engagement goal, add variants with different incentive levels, and enable Smart Suppression to skip users unlikely to respond. AID learns which incentive level and channel re-activates each type of user, while suppression reduces wasted sends.
---
## How agents work
Each agent follows a continuous loop:
1. **Collect inputs** — Audience, goals, messages, and guardrails.
2. **Decide** — Which message to send, when, and through which channel.
3. **Deliver** — Send through your connected destinations (Braze, Iterable, Salesforce Marketing Cloud).
4. **Measure** — Record outcomes such as purchases or clicks.
5. **Learn** — Update future decisions based on what worked.
This process runs automatically for every user. AID uses reinforcement learning to balance exploration (testing new options) with optimization (sending what performs best). Unlike static rules or journeys, AID adjusts automatically to seasonality, creative changes, and audience behavior shifts.
---
## Core components
### Agents and messages
[Agents](/ai-decisioning/agents) are the foundation — each one combines an audience, goals, and messages into a single adaptive campaign. After creating an agent, add [messages](/ai-decisioning/messages) sourced from your destination (Braze, Iterable, or SFMC) and define variants the agent can evaluate. Use [tags](/ai-decisioning/tags) to label messages by creative theme, offer type, or content category so you can compare performance across dimensions.
### Optimization tools
Enable **Smart Suppression** to reduce low-impact sends using predictive performance data. Use [collections](/ai-decisioning/collections) to dynamically recommend products or content from synced catalogs. Run [content analysis](/ai-decisioning/content-analysis) to identify overlap, gaps, and opportunities across your message variants.
### Measurement and QA
Review agent performance with [Insights](/ai-decisioning/insights), which shows conversion breakdowns by audience, creative performance by message or tag, timing insights, and lift metrics. Use the [Inspector](/ai-decisioning/inspector) to preview sends, confirm eligibility, and troubleshoot delivery issues. Before launching, walk through the [QA guide](/ai-decisioning/qa-guide) for channel-specific validation checklists.
---
## Get started
AI Decisioning requires workspace configuration from data teams before marketers can create agents. This setup is typically done once per workspace.
### For data teams and platform admins
Complete these steps in order. Marketers cannot create agents until the workspace is configured and at least one destination is connected.
| Step | What to do | Why |
| ---- | -------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- |
| 1 | [Configure workspace settings](/ai-decisioning/configuration) | Define global eligibility rules, time zones, and channel defaults. |
| 2 | [Prepare data for AID](/ai-decisioning/prepare-data) | Define audience models, structure event data, and prepare tables for decisioning. |
| 3 | Connect a destination: [Braze](/ai-decisioning/braze), [Iterable](/ai-decisioning/iterable), or [Salesforce Marketing Cloud](/ai-decisioning/sfmc) | Connect your ESP or messaging platform so AID can deliver messages. |
Once the workspace is configured and a destination is connected, marketers can
start [creating agents](https://app.hightouch.com/ai/flows?from=docs) immediately.
### For marketers
Once your data team has configured the workspace, start by [creating your first agent](/ai-decisioning/agents). Define a target audience and a single measurable goal — most teams start with one goal like reactivation or first purchase and expand as performance data accumulates.
From there, [add messages](/ai-decisioning/messages) from your connected destination, define variants, and launch. Use the [QA guide](/ai-decisioning/qa-guide) to validate before going live, then monitor performance in [Insights](/ai-decisioning/insights) as your agent learns.
---
## When to use AI Decisioning
Use AI Decisioning for lifecycle or retention campaigns where users may need different messages, channels, or timing based on their behavior. For campaigns where you need full control over sequencing and timing, use [Journeys](/customer-studio/journeys) in Customer Studio instead.
| Scenario | Recommended approach |
| -------------------------------------------------------------------- | -------------------------------------------------------- |
| Automate which message, channel, and timing each user receives | **AI Decisioning** |
| Onboard, win back, cross-sell, or retain with adaptive campaigns | **AI Decisioning** [Agents](/ai-decisioning/agents) |
| Reduce wasted sends by suppressing low-likelihood-to-convert users | **AI Decisioning** Smart Suppression |
| Build audiences from warehouse data and sync to destinations | [Customer Studio](/customer-studio/overview) |
| Orchestrate multi-step campaigns with fixed sequencing and branching | [Journeys](/customer-studio/journeys) in Customer Studio |
| Measure experiment results and track KPIs | [Intelligence](/campaign-intelligence/overview) |
---
## How AI Decisioning fits into Hightouch
AI Decisioning is the optimization layer of the Hightouch platform. It takes audiences and data from your warehouse and learns how to reach each person most effectively through your connected messaging tools.
#### What AI Decisioning uses
- **Audiences from [Customer Studio](/customer-studio/overview)** — Agents target Customer Studio audiences as their eligible population. Audience membership updates automatically as warehouse data changes.
- **Event data from your [connected source](/sources/overview)** — AID uses behavioral events like purchases, clicks, and app opens as feedback signals to learn what drives conversions.
- **Messages from connected destinations** — Messages are sourced from your ESP or messaging platform ([Braze](/ai-decisioning/braze), [Iterable](/ai-decisioning/iterable), or [Salesforce Marketing Cloud](/ai-decisioning/sfmc)). AID evaluates variants and controls which version is sent — it does not modify your base content.
#### What AI Decisioning sends
- **Optimized sends to destinations** — AID delivers the best message, channel, and timing for each user through your connected ESP or messaging platform.
- **Performance data to [Insights](/ai-decisioning/insights)** — Conversion breakdowns, creative performance, timing insights, and lift metrics are available for ongoing analysis and refinement.
---
## Use cases
| Use case | Example goal |
| ------------------------ | -------------------------------------------------------- |
| **Onboarding** | Help new users complete a first purchase or app setup |
| **Win-back** | Re-engage lapsed users with personalized incentives |
| **Cross-sell or upsell** | Recommend complementary products or upgrades |
| **Referral** | Encourage existing customers to invite friends |
| **Retention** | Keep engaged users active with timely, relevant messages |
Each agent focuses on a single measurable outcome and continuously improves performance over time.
---
## Permissions
AI Decisioning uses a dedicated role-based permission system with three roles: Viewer, Content Editor, and Admin. Each role controls access to agents, messages, goals, configuration, and more. See [AI Decisioning permissions](/ai-decisioning/permissions) for the full permission matrix.
---
## Prepare data for AID
**URL:** https://hightouch.com/docs/ai-decisioning/prepare-data
**Description:** Learn how to structure your data models and events so AI Decisioning can target users, personalize content, and optimize for real outcomes.
**Section:** AI Decisioning
**Audience:** Data and analytics engineers should collaborate closely with business stakeholders to identify the data required for AI Decisioning use cases.
**Prerequisite:** [Configuration →](/ai-decisioning/configuration)
---
By the end of this article, you’ll be able to:
- Define the parent, related, and event models AID needs
- Identify which tables to include and how they should be structured
- Build goal-aligned audiences in Customer Studio
---
## Overview
Before [creating agents](?tab=t.n814e49lyqz), you’ll need to prepare the data that AID will learn from. This setup ensures that your AI agents have the full picture: who users are, what they’ve done, and how they’ve responded to marketing. That data is then used to both personalize decisions and measure impact. This includes:
- Defining a global user base (**parent model**)
- Enriching it with related data to give user-level context (**related models**)
- Creating **event models** that reflect the outcomes you want to optimize.
- Building an **audience** to feed into each agent so it knows which users to target.
AID uses your data to:
- **Engineer user features**: Create an understanding of each individual user and detect patterns in behavior across characteristics, traits, events, and responses
- **Train and evaluate models**: Identify which behaviors lead to outcomes like purchases or sign-ups
Learn more about what we do with this data: [Security →](/ai-decisioning/security)
Once your data is structured correctly, you can reuse these models across all of your lifecycle motions that are powered by AID.
## 1. Define your parent model
**Purpose:** Establish the global user base eligible for AID agents
The parent model is your master list of users. Every AID agent will reference an audience that is created from this parent model when defining who is eligible for a set of messages. It should include all users who may become eligible for AID, even if not immediately targeted.
You’ll configure this model in **Customer Studio > Schema**.
**Requirements:**
- One row per user
- A unique identifier (e.g. `user_id`) as each user’s primary key
- Basic user data that can include:
- Personal: First Name, Last Name, Full Name, Gender, Date of Birth, Age, Email
- Location & timezone: City, State/Province, Country, Timezone (UTC Offset or IANA Timezone String)
- Account details: Signup Date, Last Login Date, Account Status, Language preferences, Device type, Operating system
- Demographics: Income Bracket, Education Level, Occupation, Marital Status, Number of Dependents
**Example model:**
Learn more: [Parent models →](/customer-studio/schema#define-the-parent-model)
## 2. Add related models
**Purpose:** Enrich users with attributes that help the model learn
Related models enrich the parent model with extra context the AI can learn from when making predictions and include additional user information like voluntary information, opt-in status, or persona data.
These tables must be joined to the parent model using a foreign key, typically `user_id`.
### Table types to include
You’ll need the following tables that contain information about:
**Required:**
- **Consent flags** for email, SMS, push, in-app
**Recommended:**
- **User context data** from rewards program membership and participation, purchase history and frequency, customer support interactions, etc.
- **Persona voluntary data** from quizzes, surveys (e.g. user goals, intent), etc.
AID requires up-to-date marketing consent to ensure compliance and prevent false negatives in model feedback.
Learn more: [Related models →](/customer-studio/schema#add-related-models)
### 3. Create event models
**Purpose:** Supply the actions and outcomes AID will learn from
Event models capture the actions users take—purchases, clicks, visits, and more. These power both model training and optimization. AID requires raw, unfiltered event logs that contain:
- User behavior on-site or in-app
- Engagement with marketing content
- Transactional conversion events
At **minimum**, your model should be structured as one event instance. For instance, if the goal is to increase subscription orders, each row in the table connected to the event model should reflect one distinct order (one row per user), not a line item in a distinct order (multiple rows per one order per user).
Models must include:
- `user_id`
- `timestamp`
- `event_name` of the campaign goal/outcome (e.g. `order_completed`)
- Relevant outcome metadata (e.g. purchase value, subscription plan details, etc.)
### Table types to include
| Type | Examples | Why it's important |
| ---------------------- | ------------------------------------ | --------------------------------------- |
| **Conversion events** | `order_completed`, `signup` | Define AI Decisioning goals |
| **Engagement signals** | `page_view`, `checkout_started` | Predict user intent |
| **Campaign events** | `email_clicked`, `push_unsubscribed` | Train on content efficacy and responses |
Learn more: [Event models →](/customer-studio/schema#add-event-models)
## 4. Build audiences in Customer Studio
**Purpose:** Define who each agent will target
Audiences are created using the models above. Each AID agent targets one audience, which determines who is eligible to receive messages—and who gets held out for lift analysis. Your starting audience should be **as broad as possible** — if there are certain pockets within that audience that need additional guardrails for consent or compliance, you will be able to configure those filters within the agent at a per-message level.
>Hightouch recommends audiences of at least 500K users in order to build enough training data to optimize for reinforcement learning.
**Your audience should:**
- Include all users AID is allowed to message
- Reflect your lifecycle goal (e.g. win-back, onboarding)
- Mirror consent and eligibility filters between Hightouch and your ESP
**Example: Win-back campaign audience**
- Users who haven’t purchased in the past 12 months
- AND opened a message in the past 6 months
- AND have opted into at least one channel
Learn more: [Create Audiences →](/customer-studio/usage)
## Summary: What data you’ll need
| Purpose | Table Type | Examples |
| ------------------------ | -------------- | --------------------------------------- |
| Define total user base | Parent model | `users` |
| Add traits & permissions | Related models | `user_profiles`, `personas`, `opt_ins` |
| Supply behaviors | Event models | `orders`, `signups`, `page_views` |
| Target users | Audience | Built from the above in Customer Studio |
## What's next
Configure a destination for AID ([SFMC](/ai-decisioning/sfmc), [Braze](/ai-decisioning/braze), [Iterable](/ai-decisioning/iterable))
---
## QA guide
**URL:** https://hightouch.com/docs/ai-decisioning/qa-guide
**Description:** Step-by-step guide to QA email, SMS, and push messages in AI Decisioning — send test emails with specific variants, verify personalization, and confirm targeting before launch.
**Section:** AI Decisioning
| | |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers, campaign managers |
| **Prerequisites** |
At least one [agent](/ai-decisioning/agents) with [messages](/ai-decisioning/messages) configured
A connected destination ([Braze](/ai-decisioning/braze), [Iterable](/ai-decisioning/iterable), or [Salesforce Marketing Cloud](/ai-decisioning/sfmc))
|
*Use this checklist to QA your messages before launching an AI Decisioning agent.*
---
## Why QA in AI Decisioning
Your team has likely already QA'd creatives before adding them to AI Decisioning. This guide covers the additional QA you should run inside AID — specifically for things AID introduces on top of your existing creative workflow:
- **Variants** — AID can serve different subject lines, images, CTAs, and more. You need to confirm every option renders correctly.
- **Payload fields** — AID passes user data (like first name or product recommendations) to your messaging platform. You need to confirm these populate in the rendered message.
- **Targeting and send rules** — AID applies audience eligibility, send limits, and timing at the message, variant-type, and individual-variant levels. You need to confirm these match your campaign intent.
- **UTM parameters** — AID constructs links with tracking parameters. You need to confirm URLs resolve correctly and UTMs are intact.
Run through this checklist whenever you launch a new agent, add or update messages, or change targeting rules.
---
## How to preview and test variants
A single message in AID can have multiple **variant types** (for example, subject line, image, CTA). Each variant type can have multiple **options**. To QA your message, you preview and test each option individually.
For each variant option:
1. **Preview the variant** — In your message’s **Content** tab, expand the variant type you want to test (for example, **subject**) and click the **eye icon** next to the option you want to preview. The **"Now previewing"** banner confirms the option is selected.
2. **Send a test** — Click **Send test**. In the dialog that appears, enter the identifier for your test user (for example, an email address or user ID) and click **Send test**. The test message uses exactly the variant options you’ve selected.
3. **Check the test message** — Open the test email, SMS, or push notification and verify the content renders as expected. Use the [channel-specific checklists](#email-qa) below to know what to look for.
4. **Repeat for each variant option** until you’ve tested every option across all variant types.
5. After testing all variants, switch to the **Configuration** tab to review message-level settings like send limits, timing, and audience eligibility. See the [channel-specific checklists](#email-qa) below for the full list of configuration checks.
"Now previewing" is not just a visual preview. It determines what gets sent in your test message.
---
## Email QA
Preview and test each variant option in the **Content** tab, then confirm your send rules in the **Configuration** tab.
Go to: `AI Decisioning → Agents → [Your Agent] → Messages → [Your Message]`
### Content tab
If your message uses variants, [preview and test each option](#how-to-preview-and-test-variants) as you work through the checks below.
- [ ] [Sender and metadata](#sender-and-metadata)
- [ ] [Subject line and preheader](#subject-line-and-preheader)
- [ ] [Body content](#body-content)
- [ ] [Links and CTAs](#links-and-ctas)
- [ ] [Footer](#footer)
- [ ] [Variant-type send limits](#variant-type-send-limits)
- [ ] [Individual variant eligibility](#individual-variant-eligibility)
#### Sender and metadata
* Confirm the **sender email address** matches what recipients should see in their inbox.
#### Subject line and preheader
* Confirm the subject line and preheader render correctly for each variant.
* Confirm personalization tokens (first name, dynamic fields, etc.) populate with real user data — not placeholder text or blanks.
#### Body content
* Confirm the content matches your intended messaging and layout.
* Confirm images render correctly and include alt text.
* Confirm dynamic or conditional content behaves as expected for different audience segments.
#### Links and CTAs
AID constructs the final URL including UTM parameters, so check the full link — not just the display text.
* Confirm all links and buttons lead to the correct destination.
* Confirm UTM parameters and tracking links are intact.
#### Footer
* Confirm the footer displays correctly.
* Confirm unsubscribe, privacy, and legal links work.
#### Variant-type send limits
These let you set send limits per variant type. For example, you can cap how often a specific subject line is sent to the same user. To access these, click **Send limit** at the top of a variant type section.
* Confirm send limits are set correctly for each variant type.
#### Individual variant eligibility
These let you restrict specific variant options to certain audience segments. For example, you can limit a "VIP exclusive" subject line to loyalty-tier users. To access these, hover over the **audience icon** to the right of a variant option and click **Add eligibility filters**.
* Confirm audience eligibility filters are correct for each variant option.
### Configuration tab
Expand the **Advanced** section.
- [ ] [Message-level settings](#message-level-settings)
- [ ] [Message status](#message-status)
#### Message-level settings
These apply to all variants of this message.
* **Send limit**: How often this message can be sent to the same user. Confirm this matches your campaign cadence (for example, 1 send every 30 days). See [Send limits](/ai-decisioning/messages#send-limits-1).
* **Timeline**: Start and end dates for the message. Confirm these are set or intentionally left open. See [Timeline](/ai-decisioning/messages#timeline).
* **Delivery windows**: If enabled, confirm the allowed days and time blocks match your intended send schedule. See [Delivery windows](/ai-decisioning/messages#delivery-windows).
* **Audience eligibility**: The base audience filter inherited from your agent. You can’t edit it here, but confirm it’s correct. See [Eligibility](/ai-decisioning/messages#eligibility).
* **Campaign attribution**: If set, confirm the campaign IDs are correct for outcome tracking. See [Campaign attribution](/ai-decisioning/messages#campaign-attribution).
#### Message status
Confirm the message is set to **Enabled** before your agent goes live.
You can send a **test** even if a message is **disabled**.
However, once AID is live, agents will **NOT** send a message unless it is **enabled**.
---
## SMS QA
SMS has tighter constraints than email — character limits, carrier filtering, and compliance requirements. Preview and test each variant option in the **Content** tab, then confirm your send rules in the **Configuration** tab.
Go to: `AI Decisioning → Agents → [Your Agent] → Messages → [Your Message]`
### Content tab
If your message uses variants, [preview and test each option](#how-to-preview-and-test-variants) as you work through the checks below.
- [ ] [Sender](#sender)
- [ ] [Title and body content](#title-and-body-content)
- [ ] [Links and actions](#links-and-actions)
- [ ] [Compliance](#compliance)
- [ ] [Variant-type send limits and eligibility](#variant-type-send-limits-and-eligibility)
#### Sender
* Confirm the **sender ID or phone number** appears as expected.
#### Title and body content
* Confirm the text is accurate, correctly formatted, and not truncated. SMS messages over 160 characters split into multiple segments, which can affect readability and cost.
* Confirm personalization tokens populate with real user data.
* Confirm dynamic content renders correctly for different audience segments.
#### Links and actions
* Confirm all links work and track correctly.
* Confirm shortened URLs or deep links open as expected.
#### Compliance
* Confirm opt-out instructions are included in every variant (for example, "Reply STOP to unsubscribe").
* If your messages are subject to TCPA or other regulations, confirm required disclosures are present.
#### Variant-type send limits and eligibility
If your message has variants, confirm send limits and audience eligibility filters are set correctly at the variant level. See the [email QA section](#variant-type-send-limits) for details on where to find these settings.
### Configuration tab
Expand the **Advanced** section.
* Confirm message-level send limits, timing, delivery windows, audience eligibility, and campaign attribution are correct. See [Message configuration →](/ai-decisioning/messages#message-configuration) for details on each setting.
* Confirm the message is set to **Enabled** before your agent goes live.
You can send a **test** even if a message is **disabled**.
However, once AID is live, agents will **NOT** send a message unless it is **enabled**.
---
## Push notification QA
Push notifications have tighter display constraints than email — titles and body text can be truncated differently on iOS and Android. Preview and test each variant option in the **Content** tab, then confirm your send rules in the **Configuration** tab.
Go to: `AI Decisioning → Agents → [Your Agent] → Messages → [Your Message]`
### Content tab
If your message uses variants, [preview and test each option](#how-to-preview-and-test-variants) as you work through the checks below.
- [ ] [Sender and app info](#sender-and-app-info)
- [ ] [Notification content](#notification-content)
- [ ] [Links and actions](#links-and-actions-1)
- [ ] [Variant-type send limits](#variant-type-send-limits-1)
- [ ] [Individual variant eligibility](#individual-variant-eligibility-1)
#### Sender and app info
* Confirm the app name and sender ID are correct.
* Confirm icons or media display correctly.
#### Notification content
* Confirm the title and body display as expected.
* Confirm personalization tokens populate with real user data.
* If possible, test on both iOS and Android. iOS typically truncates titles around 40 characters and bodies around 110 characters. Android varies by device.
#### Links and actions
Deep links open specific screens in your app (for example, a product page or cart). If misconfigured, the user may land on the home screen instead.
* Test all deep links by tapping the notification on a test device and confirming you reach the correct screen.
* If you have fallback URLs for users without the app, test those as well.
#### Variant-type send limits
These let you set send limits per variant type. For example, you can cap how often a specific notification title is sent to the same user.
* Confirm send limits are set correctly for each variant type.
#### Individual variant eligibility
These let you restrict specific variant options to certain audience segments. For example, you can limit a promotional notification to users who have the latest app version.
* Confirm audience eligibility filters are correct for each variant option.
### Configuration tab
Expand the **Advanced** section.
- [ ] [Message-level settings](#message-level-settings-1)
- [ ] [Message status](#message-status-1)
#### Message-level settings
These apply to all variants of this message.
* Confirm send limits, timing, delivery windows, audience eligibility, and campaign attribution are correct. See [Message configuration →](/ai-decisioning/messages#message-configuration) for details on each setting.
#### Message status
Confirm the message is set to **Enabled** before your agent goes live.
You can send a **test** even if a message is **disabled**.
However, once AID is live, agents will **NOT** send a message unless it is **enabled**.
---
## Troubleshooting
If something isn't working during QA, check here for the most common issues.
Go to: `AI Decisioning → Agents → [Your Agent] → Messages → [Your Message]`
### Message test not received
If your test send doesn’t arrive:
1. **Confirm the message is enabled.**
2. **Check the identifier.**
* Confirm you entered the correct identifier (for example, email, user ID, or subscriber key).
* It must match both your channel configuration and the value shown in the test send.
3. **Confirm the identifier exists in your destination.**
* The user must already exist in your email or messaging platform.
4. **Select a user in preview (SFMC).**
* Go to **Settings → Select user** before sending a test.
### Personalization not working
If personalization appears blank or incorrect:
1. **Check your channel payload configuration.**
* Confirm the fields you’re using are included in the channel payload (these are the fields sent with each message).
2. **Check your user data model.**
* Fields like `first_name` and `last_name` must exist on your main user model (AID parent model).
* If they’re on a related model, you can either:
* Merge the model into the main user model (so all fields are available in one place), or
* Add the fields through **Configuration → Additional user columns** (to include specific fields without merging everything)
### Variants not showing
If your test email shows the same content regardless of which variant you select, the destination template is likely missing the required code snippet.
* Confirm the code snippet is added to your destination template.
* See the variable setup guide for your destination: [Braze](/ai-decisioning/braze#use-variables-in-braze), [Iterable](/ai-decisioning/iterable#5-use-variables-in-iterable-templates), or [Salesforce Marketing Cloud](/ai-decisioning/sfmc#use-variables-in-sfmc).
This is a one-time setup step. Once the snippet is in your template, it applies to all messages and variants.
### Variant images not displaying
If images appear broken or don't load in your test email:
* Confirm the code snippet is added to your destination template.
* Confirm all image URLs are publicly accessible — open each URL in an incognito browser window to verify.
---
## Next steps
Once you've completed QA, confirm your [agent](/ai-decisioning/agents) is enabled and ready to launch.
---
## Get started with AI Decisioning
**URL:** https://hightouch.com/docs/ai-decisioning/quick-start
**Description:** This quick-start guide walks you through initial setup (for admins and data teams) and day-to-day usage (for marketers), with links to each key step.
**Section:** AI Decisioning
| | |
| ----------------- | ----------------------------------------------------- |
| **Audience** | Platform admins and data teams, marketers |
| **Prerequisites** | [AI Decisioning overview →](/ai-decisioning/overview) |
**AI Decisioning (AID)** helps teams automate and improve campaign decisions at the individual user level. You define **agents** that decide who to target and what to optimize for, then add **messages** so agents can test and deliver different versions of your content. Over time, agents learn which message, timing, and channel perform best for each user.
This guide covers:
- [**Technical setup (admins and data teams) →**](#technical-setup-admins-and-data-teams)
Configure AI Decisioning settings, prepare data for decisioning, review data handling, and connect destinations.
- [**Day-to-day usage for marketers →**](#define-agents)
Define agents, add and optimize messages, validate delivery, and measure performance.
---
## Technical setup (admins and data teams)
Before marketers can create agents or messages, your workspace must be configured with the right data models and connected destinations.
Most of this setup is completed once per workspace.
### Configure workspace and data
| Step | What you’ll do | Article |
| ------------------------ | --------------------------------------------------------------------------------- | ---------------------------------------------------- |
| **Configure settings** | Define global eligibility rules, time zones, and channel defaults. | [Configuration →](/ai-decisioning/configuration) |
| **Prepare data for AID** | Define audience models, structure event data, and prepare tables for decisioning. | [Prepare data for AID](/ai-decisioning/prepare-data) |
### Connect destinations
| Step | What you’ll do | Article |
| ------------------------ | ------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| **Connect destinations** | Connect your ESP or messaging platform so AID can deliver messages. | [Braze](/ai-decisioning/braze), [Iterable](/ai-decisioning/iterable), or [Salesforce Marketing Cloud](/ai-decisioning/sfmc) |
After technical setup is complete, marketers can manage agents and messages
directly from [**AI Decisioning →
Agents**](https://app.hightouch.com/ai/flows?from=docs).
---
## Define agents
Agents define _who_ to target and _what_ to optimize for. Each agent combines an audience, goals, and decisioning rules into a single adaptive campaign.
### Create and configure an agent
| Step | What you’ll do | Article |
| --------------------------------------- | ------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| **Create an agent** | Define the target audience and campaign goals. | [Agents →](/ai-decisioning/agents) |
| **Configure scheduling** | Set send frequency, quiet hours, and blackout dates. | [Agents → Configure scheduling](/ai-decisioning/agents#step-3-optional-configure-scheduling) |
| **Enable Smart Suppression (optional)** | Reduce low-impact sends across the entire agent. | [Smart Suppression →](/ai-decisioning/smart-suppression) |
| **Add collections (optional)** | Define dynamic sets of products or content the agent can recommend. | [Collections →](/ai-decisioning/collections) |
Most teams start with a single agent focused on one goal (for example,
reactivation or purchase) and expand as performance data accumulates.
---
## Add and optimize messages
Once an agent is defined, add messages from your destination and define variants the agent can evaluate and learn from.
### Create messages and variants
| Step | What you’ll do | Article |
| -------------------------- | ---------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| **Add messages** | Start from destination templates and connect them to your agent. | [Messages → Add a message](/ai-decisioning/messages#add-a-message) |
| **Create variants** | Define multiple versions of fields like subject lines or CTAs. | [Messages → Add and manage variants](/ai-decisioning/messages#add-and-manage-variants) |
| **Apply tags and rules** | Control variant eligibility and classify creative attributes. | [Tags →](/ai-decisioning/tags) |
| **Review content quality** | Identify overlap, gaps, and opportunities to improve creative. | [Content analysis and suggestions →](/ai-decisioning/content-analysis) |
Messages are sourced from your destination. AI Decisioning does not change
your base content—it evaluates variants and controls which version is sent.
---
## Measure and learn
After your agent is running, validate delivery and review performance to refine your strategy over time.
### Validate delivery and analyze results
| Step | What you’ll do | Article |
| ----------------------- | ----------------------------------------------------------------- | ---------------------------------------- |
| **QA messages** | Run through channel-specific checklists before launching. | [QA guide →](/ai-decisioning/qa-guide) |
| **Inspect delivery** | Preview sends, confirm eligibility, and troubleshoot issues. | [Inspector →](/ai-decisioning/inspector) |
| **Analyze performance** | Compare results across agents, messages, variants, and audiences. | [Insights →](/ai-decisioning/insights) |
Review Insights regularly to adjust goals, suppression thresholds, and
creative as agents continue learning.
---
## Salesforce Marketing Cloud integration with AI Decisioning
**URL:** https://hightouch.com/docs/ai-decisioning/SFMC
**Description:** Learn how to connect Salesforce Marketing Cloud (SFMC) to AI Decisioning in Hightouch to send personalized, real-time messages across email, SMS, push, and in-app channels.
**Section:** AI Decisioning
| | |
|----------------------|------------------------------------------------------------------------------------------------------|
| **Audience** | Technical marketers, marketing and lifecycle teams |
| **Prerequisites** | [Prepare data for AID →](/ai-decisioning/configuration#prepare-data-for-aid) |
*Connect Salesforce Marketing Cloud to AI Decisioning so agents can deliver personalized emails, SMS, push notifications, and in-app messages through your existing SFMC Triggered Sends and Journeys.*
---
## What you'll learn
In this article, you'll learn how to:
- [Set up SFMC as a destination](#1-set-up-sfmc-as-a-destination)
- [Add channels in AI Decisioning](#2-add-channels-in-ai-decisioning)
- [Add messages from SFMC](#3-add-messages)
- [Configure SFMC Triggered Sends and Journeys for AID](#4-configure-sfmc-triggered-sends-and-journeys-for-aid)
- [Use dynamic variables in SFMC templates](#use-variables-in-sfmc)
---
## Overview
Hightouch's AI Decisioning (AID) helps you deliver real-time, personalized messages based on live customer context and predictions. By connecting AID with Salesforce Marketing Cloud (SFMC), you can:
- Automatically trigger messages from AID Agents via SFMC Triggered Sends or Journeys
- Personalize SFMC templates using dynamic variables from Hightouch—no custom code required
- Centralize targeting and decision logic in Hightouch, while using SFMC to handle message delivery
## 1. Set Up SFMC as a Destination
Already configured SFMC? Even if you've previously set SFMC up as a destination, you'll still need to apply the specific permissions and scopes outlined below.
### Why these permissions matter
To send messages through SFMC using AID, Hightouch needs access to certain channels, content, and user data. These permissions allow Hightouch to:
- Trigger messages via Triggered Send and Journey Builder
- Read and update subscriber data
- Access and personalize email, SMS, push, and in-app templates
### Connect SFMC and grant permissions
**In SFMC:**
1. [Create an installed package in SFMC](/destinations/sfmc#create-installed-package-in-sfmc) until you reach **Set Server-to-Server Properties**.
2. Under **Channels**, enable the permissions for each message type you want Hightouch to use (read, write, send, publish, or post).
3. Click **Save**.
**In Hightouch:**
4. Continue setting up the SFMC destination until you reach the [Custom Scopes](/destinations/sfmc#custom-scopes) step.
5. Under **Advanced permissions > Custom Scopes**, paste in the following:
`email_read email_write email_send accounts_read data_extensions_read data_extensions_write journeys_read journeys_write journeys_aspr journeys_execute list_and_subscribers_write list_and_subscribers_read push_read push_write push_send`
6. Click **Test Connection** to verify that setup is complete.
## 2. Add Channels in AI Decisioning
Once SFMC is set up as a destination, you can add **channels**. Channels define how Hightouch sends messages to SFMC—via Triggered Sends or Journeys.
### Set up a channel
1. Go to `AI Decisioning → Configuration → Channels`.
2. Click `Add channel`.
3. Choose a channel type: **Email**, **SMS**, **Push**, or **In-App**.
4. Select **Salesforce Marketing Cloud** as the destination.
5. Select a [sync type](#sync-type-comparison):
- **Triggered Send** – sends a 1:1 message in real time
- **Journey entry** – sends as part of a journey workflow
- **Data Extension** *(advanced)* – batch sends managed within SFMC
6. Choose an **Identifier** that matches the Subscriber Key field in SFMC (for example, email or user ID).
7. Click `Add channel`, then `Save changes`.
### Sync type comparison
The **Sync Type** determines how Hightouch sends messages through SFMC:
| Channel | Triggered Send | Journey | Data Extension |
|---------|:--------------:|:-------:|:--------------:|
| Email | ✅ | ✅ | ✅ |
| SMS | ❌ | ✅ | ✅ |
| Push | ✅ | ✅ | ✅ |
| In-App | ❌ | ✅ | ✅ |
| Sync type | Best for | Setup notes |
|------------------------|---------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Triggered Send** | Real-time, event-based sends tied to a single user action | Requires send classification: use **Commercial** for marketing content (users can opt out) or **Transactional** for system notifications (users can't opt out). |
| **Journey entry** | Automated, multi-step campaigns (for example, onboarding, retargeting) | Select the **Synced user columns** (for example, email, name, product interest) you want to sync. These fields are available inside the journey's Data Extension for personalization or decision splits. |
| **Data Extension** *(advanced)* | Batch sends, scheduled campaigns, and advanced setups where you need full control over delivery logic | A [Data Extension](https://help.salesforce.com/s/articleView?id=mktg.mc_es_data_extension_data_relationships_classic.htm&type=5) is a table in SFMC that stores subscriber or message data. When you choose this sync type, Hightouch writes data directly to a Data Extension, and you manage the send logic (timing, audience filters, scheduling) within SFMC. This gives you the most flexibility but requires more setup on the SFMC side. Choose **Insert** to append new rows or **Mirror** to keep the Data Extension in sync with your warehouse. |
---
## 3. Add Messages
After you add channels in AI Decisioning, you can add **messages** to your agents. Each message connects an agent's decision to a specific template in SFMC.
For full details on message configuration, variants, and tags, see [Messages →](/ai-decisioning/messages).
### Add a message from SFMC
1. Go to `AI Decisioning → Agents` and select your agent.
2. Open the `Messages` tab.
3. Click `Add messages`, then select **Add messages individually**.
4. Select your configured SFMC **channel and destination** tile (for example, **Email - Salesforce Marketing Cloud**).
5. Select the appropriate **template** or [event identifier](https://help.salesforce.com/s/articleView?id=mktg.mc_jb_create_journey_specific_entry_event.htm&type=5) from SFMC.
6. Enter a **message name** and click `Add messages`.
You can repeat this process for any channel and sync type. Continue the setup process in the channel-specific sections below:
- [Send emails via Triggered Send](#send-emails-via-triggered-send)
- [Send emails via Journey](#send-emails-via-journey)
- [Send emails via Data Extension](#send-emails-via-data-extension)
- [Send SMS via Journey](#send-sms-text-messages-via-journey)
- [Send SMS via Data Extension](#send-sms-text-messages-via-data-extension)
- [Send push (outbound) via Triggered Send](#send-push-outbound-notifications-via-triggered-send)
- [Send push (app inbox) via Triggered Send](#send-push-app-inbox-notifications-via-triggered-send)
- [Send push via Journey](#send-push-notifications-via-journey)
- [Send push via Data Extension](#send-push-notifications-via-data-extension)
- [Send in-app via Journey](#send-in-app-notifications-via-journey)
- [Send in-app via Data Extension](#send-in-app-notifications-via-data-extension)
### Test a message
1. Open the message and go to the `Content` tab.
2. Under **Optimized content**, confirm your variants are configured. If your message uses [dynamic variables](#use-variables-in-sfmc), set values for testing.
3. In the **Preview** panel, verify the rendered message looks correct.
4. Click `Send test` to send the message to a test user.
---
## 4. Configure SFMC Triggered Sends and Journeys for AID
Before adding a message in Hightouch, you need to configure the corresponding Triggered Send or Journey in SFMC.
### Send emails via Triggered Send
1. [Add a channel](#set-up-a-channel) with the sync type set to **Triggered Send**.
2. [Add a message](#add-a-message-from-sfmc) from your SFMC email template.
3. Close the window that opens and return to the **Messages** tab.
4. Click the link next to **Base content** to go to SFMC.
5. This will prompt you to log in to SFMC and should open the **Email Studio > Interactions** page.
6. Click **Triggered Sends**.
7. Select the send name that corresponds with your newly created message.
8. Click **Start/Restart**.
9. Return to **Hightouch** and open your message.
10. In the Content tab, [set the variables](#use-variables-in-sfmc) and click **Send Test**.
### Send emails via Journey
1. [Add a channel](#set-up-a-channel) with the sync type set to **Journey entry**.
2. [Add a message](#add-a-message-from-sfmc) from your SFMC email template.
3. In the Content tab, [set the variables](#use-variables-in-sfmc) and click **Send Test**.
### Send emails via Data Extension
1. [Add a channel](#set-up-a-channel) with the sync type set to **Data Extension**.
2. Choose **Insert** (append new rows) or **Mirror** (keep in sync with your warehouse).
3. [Add a message](#add-a-message-from-sfmc) from your SFMC email template.
4. In SFMC, configure your send logic (automation, query activity, or triggered automation) to read from the Data Extension that Hightouch populates.
### Send SMS text messages via Journey
1. [Add a channel](#set-up-a-channel) with the sync type set to **Journey entry**.
2. [Add a message](#add-a-message-from-sfmc) from your SFMC SMS template.
3. In the **Content** tab, [set the variables](#use-variables-in-sfmc) and click **Send Test**.
### Send SMS text messages via Data Extension
1. [Add a channel](#set-up-a-channel) with the sync type set to **Data Extension**.
2. Choose **Insert** or **Mirror** mode.
3. [Add a message](#add-a-message-from-sfmc) from your SFMC SMS template.
4. In SFMC, configure your send logic to read from the Data Extension that Hightouch populates.
### Send push (outbound) notifications via Triggered Send
#### Enable the push channel
1. [Add a channel](#set-up-a-channel) with the sync type set to **Triggered Send**.
#### Configure your push template in SFMC
**In SFMC:**
2. Go to **Mobile Studio > MobilePush**.
3. Click **Create Message**.
4. Select the **Outbound** template and click **Next**.
5. Name your message and select your app.
6. Select **API Triggered** as the Send Method.
7. Select **Alert** as the Push Method and click **Next**.
You can only trigger Alert Only or Inbox Only messages, not Alert + Inbox.
8. Create the template for your push alert.
9. **Save** your message.
10. Click **Send** in the top right-hand corner.
11. An activation window should pop up with an API key. Copy this API key to make it easier to find the template in Hightouch in a later step.
#### Add a message in Hightouch
**In Hightouch:**
12. [Add a message](#add-a-message-from-sfmc) from your SFMC push template.
#### Preview content
Hightouch automatically creates the variables for push notification title, subtitle, and body. The values you supply here are used directly in the push notification without any need for code snippets in the template. If these are left blank, Hightouch uses the default values in the template.
Hightouch uses the override capabilities in the API. If even one variable has a value supplied, the default values in the template for **all** variables get overridden. Make sure to supply at least the default value for all variables, even if you only want AID to vary one.
#### Send tests
13. To test push notifications, paste the contact key you use in your SFMC contacts.
14. Personalization strings can be supplied directly in the values for the variables without any need for code snippets in the template.
### Send push (app inbox) notifications via Triggered Send
#### Enable the push channel
1. [Add a channel](#set-up-a-channel) with the sync type set to **Triggered Send**.
#### Configure your push template in SFMC
**In SFMC:**
2. Go to **Mobile Studio > MobilePush**.
3. Click **Create Message**.
4. Select the **Inbox** template and click **Next**.
5. Name your message and select your app.
6. Select **API Triggered** as the Send Method.
7. Select **Alert** as the Push Method and click **Next**.
You can only trigger Alert Only or Inbox Only messages, not Alert + Inbox.
8. Create the template for your push alert.
9. **Save** your message.
10. Click **Send** in the top right-hand corner.
11. An activation window should pop up with an API key. Copy this API key to make it easier to find the template in Hightouch in a later step.
#### Add a message in Hightouch
**In Hightouch:**
12. [Add a message](#add-a-message-from-sfmc) from your SFMC push template.
### Send push notifications via Journey
1. [Add a channel](#set-up-a-channel) with the sync type set to **Journey entry**.
2. [Add a message](#add-a-message-from-sfmc) from your SFMC push template.
3. In the **Content** tab, [set the variables](#use-variables-in-sfmc) and click **Send Test**.
### Send push notifications via Data Extension
1. [Add a channel](#set-up-a-channel) with the sync type set to **Data Extension**.
2. Choose **Insert** or **Mirror** mode.
3. [Add a message](#add-a-message-from-sfmc) from your SFMC push template.
4. In SFMC, configure your send logic to read from the Data Extension that Hightouch populates.
### Send in-app notifications via Journey
#### Enable the in-app channel and add a message
1. [Add a channel](#set-up-a-channel) with the sync type set to **Journey entry**.
2. [Add a message](#add-a-message-from-sfmc) from your SFMC in-app template.
#### Link your message to an SFMC Journey
In these next steps, you will link your newly created message to an associated Journey in SFMC.
**In Hightouch:**
3. Go to the **Content** tab within your message to add the variables you would like Hightouch to control.
**In SFMC:**
4. Go to **Journey Builder > Journey Builder**.
5. Click **Create New Journey**.
6. Select **Multi-Step Journey**.
7. Select **API Event** as the entry source. Drag it onto the Journey Builder.
8. Click the grey **API Event** icon in the Journey Builder. This will open a list of API Events.
9. Select the message name you chose when adding the message in Hightouch.
Hightouch automatically creates the required Data Extension with the event so you don't have to set that up.
10. Click **Done**.
11. Next, drag **In-App Message** to the Journey Builder.
12. Click the grey **In-App Message** icon in the Journey Builder.
13. **Select** (or create) your desired in-app message template.
14. In the **In-App Message Activity Summary**, edit the required settings for **Message Configuration** and **Display Trigger**. Optionally, configure **Display Options** and **Advanced Options** as desired. See [SFMC documentation](https://help.salesforce.com/s/articleView?id=mktg.mc_jb_configure_inapp_in_journey_builder.htm&type=5) for more details.
15. Optional: If you would like Hightouch to control aspects of the In-App Message, follow the instructions for [using variables in SFMC templates](#use-variables-in-sfmc).
- Paste the variable code snippets from Hightouch into SFMC:
- Select a message in **Message Definition**
- Click **Edit Selected Message**
16. Paste the copied code snippets from Hightouch into the **Content** tab that opens.
17. **Save** your changes.
18. Optional: To continuously test sends, click the **Gear icon > Journey Settings** and select re-entry any time.
19. **Activate** your journey.
**In Hightouch:**
20. Return to **Hightouch** and refresh the Content tab within your message.
21. Preview your content to ensure everything is accurate.
22. You can now **Send test** to subscribers.
### Send in-app notifications via Data Extension
1. [Add a channel](#set-up-a-channel) with the sync type set to **Data Extension**.
2. Choose **Insert** or **Mirror** mode.
3. [Add a message](#add-a-message-from-sfmc) from your SFMC in-app template.
4. In SFMC, configure your send logic to read from the Data Extension that Hightouch populates.
---
## Use Variables in SFMC
Hightouch variables allow you to personalize SFMC messages using live warehouse data, any variable you've defined within the message such as subject line variations, or different content blocks.
### Access variables
1. Open a message in Hightouch and go to the `Content` tab.
2. Under **Optimized content**, expand a content type (for example, **subject** or **body**) and add your variants.
3. Click the `...` menu on a content type and select `Copy code snippet` to get the variable code.
4. Paste the code into your SFMC template. These snippets render with real values for each user when the message is sent.
### Important notes
- **Triggered Sends**: Variables are fully supported.
- **Journeys**: Variables are available via Data Extension fields synced by Hightouch.
---
## Troubleshooting
| Issue | Likely cause | Resolution |
|-------|-------------|------------|
| Message fails to send | API errors or misconfigured destination | Check Hightouch logs for API errors. |
| Test user not receiving messages | Identifier mismatch | Verify the identifier matches a valid Subscriber Key in SFMC. |
| Triggered Send not firing | Send not started | Confirm the Triggered Send is in **Running** status in Email Studio. |
| Journey sends not triggering | Journey not activated or entry event misconfigured | Confirm the Journey is activated and the API entry event is configured. |
| Variables not rendering | Incorrect syntax or missing field mapping | Double-check variable syntax from `Copy code snippet` and field mapping. |
---
## What's next
- [Messages: Create and manage message variants →](/ai-decisioning/messages)
- [QA guide: Validate messages before launch →](/ai-decisioning/qa-guide)
- [Agents →](/ai-decisioning/agents)
- [Insights →](/ai-decisioning/insights)
---
## AID Tags
**URL:** https://hightouch.com/docs/ai-decisioning/tags
**Description:** Plan, apply, analyze, and manage message tags to compare creative performance across AI Decisioning agents.
**Section:** AI Decisioning
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Marketers (with support from data or platform teams) |
| **Prerequisites** |
Access to [AI Decisioning](/ai-decisioning/overview)
At least one agent, message, or variant created
|
*Tags help you organize messages and variants, compare creative performance, and apply consistent structure across your AI Decisioning workspace.*
---
## What you'll learn
After reading this article, you'll know how to:
* [Plan a tagging framework](#plan-your-tagging-framework) before you start
* [Add tags](#tag-messages-and-variants) for messages and variants
* [Analyze performance](#analyze-tag-performance) using tags in Insights
* [Standardize, govern, and optimize tags](#manage-and-govern-your-tags) across your workspace
---
## Overview
Tags in **AI Decisioning** classify and group [message variants](/ai-decisioning/messages#add-and-manage-variants) so you can compare how different creative strategies perform across your agents.
Each tag consists of a **key** and a **value**. The key defines a category, such as `tone` or `product`, and the value describes a specific attribute, such as `playful` or `sunglasses`. Together, these pairs describe both what your message says and how it says it.
| Tag type | Description | Example use |
|-----------|--------------|-------------|
| **Message tags** | Describe the overall goal, topic, or focus of a message | `Summer Promo`, `Loyalty Push`, `Flight Offers` |
| **Variant tags** | Describe specific creative differences within a message | `Emotional`, `Discount CTA`, `Travel Deals` |
Once applied, tags let you break down performance in [Insights](/ai-decisioning/insights) to identify which themes, tones, offers, or structures drive engagement. Tags are most useful when multiple messages or variants share the same concepts, because that overlap makes it easier to compare results and identify patterns.
Consistent tagging also helps AI Decisioning learn from past performance, improving future recommendations and experiments.
---
## Plan your tagging framework
Before you start tagging, decide what you want to learn from your campaigns — such as which **offers**, **value propositions**, or **tones** drive engagement. Planning your framework upfront helps you tag consistently from the start.
### Decide what to tag
Your tags should capture:
- **What the content says** — for example, product, offer, or audience
- **How it says it** — for example, tone, urgency, or format
**Guidelines**
- Keep tag names short and consistent
- Use one value per tag type, such as one `Tone` or one `Offer`
- Avoid duplicate concepts, such as `Incentive` and `Offer`
- Don't recreate system fields like `channel`, `message`, or `subject line`
- Create tags for any dimension you want to measure or control, such as business unit or product line
Tags describe **content attributes**, not user traits. It's okay if they overlap. For example, a message might use the tag `Career = Marketer` while a user profile also includes a `career` trait.
### Choose message tags vs. variant tags
Use **message-level tags** for broad attributes that apply to the entire message, such as product, campaign theme, or business unit. Use **variant-level tags** for finer creative differences, such as subject line tone, CTA style, or emoji usage.
### Start small, then expand
| Tag type | Recommended range | Notes |
|-----------|------------------|-------|
| **Keys** | 3–8 | Focus on the main dimensions you want to measure, such as tone, offer, or motivator. |
| **Values** | 3–6 per key | Each value should apply to multiple messages. Avoid long lists of rarely used values. |
As you add new tags, Insights automatically updates results for past agents that use them.
### Best practices
| Goal | Recommendation |
|------|----------------|
| **Stay consistent** | Use a standard naming pattern, such as `Tone`, `Offer`, or `Audience`. |
| **Avoid duplicates** | Regularly review tags in the **Global Tag Manager** to remove similar values. |
| **Be specific** | Use clear labels like `CTA: Learn More` instead of generic names like `Email Test`. |
| **Align with goals** | Create tags that reflect what you want to measure, such as conversion or engagement. |
| **Use AI to assist** | Generate tags quickly, then refine them for accuracy. |
| **Review performance** | Use [Insights](/ai-decisioning/insights) to identify which tags drive results. |
Keep a shared reference document for your team's approved tags, such as `Promo Type`, `Tone`, and `Audience`, to maintain consistency across agents.
### Example tag structures
#### Financial company example
| Category | Key | Example values |
|-----------|-----|----------------|
| **Product** | Product type | Checking, Savings, Credit Card, Mortgage |
| **Product** | Feature | No ATM Fees, Auto Transfers, 2% Cash Back |
| **Expression & Style** | Tone | Playful, Urgent, Reassuring, Empowering |
| **Expression & Style** | Motivator | Save Money, Earn Rewards, Gain Control, Simplify Life |
| **Expression & Style** | Narrative | Problem–Solution, Benefit-Focused, Storytelling |
| **Incentive** | Value prop | Low Fees, High Yield, Personalized Offers |
| **Incentive** | Offer | 10% Off, $100 Bonus, Free Trial |
| **Incentive** | CTA | Learn More, Apply Now, Get Started |
| **Creative Structure** | Message length | Short, Medium, Long |
| **Creative Structure** | Urgency | Low, Medium, High |
#### Streaming service example
| Category | Key | Example values |
|-----------|-----|----------------|
| **Product** | Subscription plan | Basic, Standard, Premium, Bundle |
| **Product** | Feature | Ad-Free, Offline Download, 4K, Multi-Screen |
| **Expression & Style** | Tone | Inspirational, Humorous, Reassuring, Playful |
| **Expression & Style** | Motivator | Entertainment, Discovery, Relaxation, Family Fun |
| **Expression & Style** | Genre | Action, Comedy, Drama, Kids, Documentary |
| **Incentive** | Value prop | Free Trial, Exclusive Content, Personalized Recommendations |
| **Incentive** | Offer | 1 Month Free, 20% Off Annual Plan, Bundle Deal |
| **Incentive** | CTA | Watch Now, Start Free Trial, Subscribe Today |
| **Creative Structure** | Message length | Short, Medium, Long |
| **Creative Structure** | Emoji usage | None, Light, Heavy |
| **Business Unit** | Region | US, GB, FR, DE, AUS, NZ, JP |
Use these examples as a starting point. Choose the categories that fit your business, then expand over time.
---
## Tag messages and variants
Once you've planned your tag structure, you can start applying tags while building or reviewing agents. Use the Agent view to tag messages in context, or the [Global Tag Manager](#manage-tags-in-global-tag-manager) to manage tags across your workspace.
| Method | Best for |
|---------|-----------|
| **From an Agent** | Adding or editing tags while building or reviewing a specific message |
| **From Global Tag Manager** | Creating, standardizing, or bulk-editing tags across multiple agents |
### Message tags
**Message tags** classify the overall purpose or focus of a message. You can add them directly from the message configuration view inside an agent.
1. Go to `AI Decisioning → Agents → Select an Agent`.
2. In the `Messages` tab, choose a message.
3. Open the `Configuration` tab.
4. Under `Tags`, use the dropdowns to select existing tags or click `+ Add tag` to create new ones.
5. Click `Save changes` to apply.
**Examples**
- `Product: Bluelight`
- `Motivator: Wellness`
- `Incentive Type: Free gift`
- `Value Proposition: Convenience`
You can also generate message tags with AI by clicking `Generate tags with AI` under the tag list.
---
### Variant tags
**Variant tags** help you compare creative differences within a single message, such as tone, offer, structure, or subject line style.
1. Go to `AI Decisioning → Agents → Select an Agent`.
2. In the `Messages` tab, choose a message.
3. Open the `Content` tab.
4. Under `Optimized content`, click `Edit tags and rules`.
5. Hover over the tag icon next to a variant and click `+ Add content tags`.
6. In the `Edit content tags` modal, use the dropdown to select existing tag values or type to create new ones.
7. Click `Apply` to save your changes.
**Examples**
- `Length: Short`
- `Tone: Savings focused`
- `Urgency: Normal`
- `Has emoji: Yes`
Add variant tags while testing subject lines, images, or CTAs. These tags connect performance differences directly to specific creative choices in [Insights](/ai-decisioning/insights).
---
## Analyze tag performance
Use tags to understand how creative themes perform across agents and messages.
### Insights
Once your messages and variants are tagged, you can measure how each tag performs across agents in **Insights**.
1. Go to `AI Decisioning → Insights`.
2. Select the `Creative performance` view.
3. In the `Break down by` dropdown, choose a tag key such as `incentive_type`, `tone`, or `motivator`.
If your workspace uses these tag keys consistently, they'll appear as available breakdown fields.
The table shows how each tag value performs across your agents and messages, including:
- **Sends** — total number of messages delivered for each tag value
- **Key conversion events** — such as purchases, sign-ups, or other tracked actions
- **Engagement metrics** — including click-through rate (CTR), open rate, or conversion rate
**Example**
Breaking down by `incentive_type` reveals which offers drive the strongest results. For example, `fixed_discount` may outperform `free_shipping` or `free_gift` in conversion rate, helping you understand which incentive types resonate most across audiences.
#### Drill-down views
You can also **drill down** into a specific message or variant to explore deeper insights:
- **Personalization insights** — Compare how engagement differs by user attributes such as device type, region, or affinity
- **Timing insights** — Identify when messages perform best by day or time of day
Use the **Insights** tab to evaluate your content strategy at scale. Tag-driven breakdowns connect creative attributes — such as incentive type, tone, or motivator — to measurable performance outcomes.
---
### Content analysis
Each message also includes **content-level insights** to help you understand relationships between creatives.
In the message's `Content` tab, scroll to **Content analysis** to view:
- **Tags** — Applied tags such as `Motivator: Efficiency` or `Value Proposition: Convenience`
- **Similar messages** — Messages with similar structure, tone, or intent
You can expand each comparison to see why messages are related. This helps you identify patterns in your creative library and understand how messages may influence model learning.
**Content analysis** focuses on creative similarity and reuse, while the **Insights** tab focuses on quantitative performance and impact. Use both to understand what works and why.
---
## Manage and govern your tags
As your tagging system grows, use these tools to keep tags consistent, apply them efficiently, and extend their use beyond analysis.
### Manage tags in Global Tag Manager
The Global Tag Manager (`AI Decisioning → Tags`) provides a centralized view of all tags in your workspace. Changes made here apply globally to every agent that uses the tag key.
| Action | Description |
|--------|-------------|
| **View tags** | Review all tag keys, such as `product`, `tone`, or `incentive_type`, and their allowed values. |
| **Create tags** | Define new tag keys and allowed values for messages or variants. |
| **Edit tag values** | Rename or remove values to keep naming consistent. *(Tag keys can't be renamed — delete and recreate if needed.)* |
| **Add tag instructions** | Specify when a tag or value should be applied to guide consistent tagging and improve AI suggestions. |
| **Bulk apply tags with AI** | Use AI to classify many messages at once for a selected tag key. |
| **Check consistency** | Confirm agents use the same key–value pairs, such as `tone = playful` or `product = bluelight`. |
---
### Add tag instructions
In the Global Tag Manager, you can add **tag-level** and **value-level instructions** that define when the tag — and each of its values — should be applied.
Clear instructions improve consistency across teams and help AI apply tags more accurately.
#### What to include in tag instructions
Effective tag instructions answer one question:
**When should this tag or value be used?**
Good instructions help people tag consistently and give AI the context it needs to distinguish between similar values.
#### Writing effective instructions
When writing tag or value instructions:
- Describe **when to apply** the tag or value
- Explain **how it differs** from other values
- Avoid generic definitions or restating the tag name
- Use short, directive language
#### Example: tag-level instructions
**Tag:** `incentive_type`
**Good instruction**
Use this tag to classify the primary incentive offered in the message. Select the value that best represents the main benefit emphasized to the user.
**Weak instruction**
Indicates the type of incentive.
#### Example: value-level instructions
Value-level instructions are most useful when values are similar or easily confused, messages include multiple incentives or concepts, or you use AI-assisted tagging.
**Tag:** `incentive_type`
- **`discount`**
Use when the message emphasizes a price reduction, percentage off, or limited-time deal.
- **`free_gift`**
Use when the message highlights a free item added to a purchase, regardless of purchase amount.
- **`free_shipping`**
Use when the message emphasizes waived shipping costs as the primary incentive.
---
### Use AI to suggest and apply tags
AI can help you tag faster, whether you're working on a single message or tagging across your workspace.
#### Generate tags for individual messages or variants
When editing a message or variant, click `Generate tags with AI` to get suggestions based on your content.
1. Click `Generate tags with AI`.
2. Review the AI-generated tags.
3. Edit or remove any that don't fit your naming conventions.
**Example suggestions**
- Message: *"Earn double points on every booking this weekend"* → AI suggests `Loyalty`, `Limited Time Offer`
- Variant: *"Book now to unlock rewards faster"* → AI suggests `Urgent CTA`, `Rewards Focused`
AI-suggested tags do not apply automatically. Always review suggestions before saving to ensure accuracy and consistency.
#### Bulk-apply tags in the Global Tag Manager
For tagging many messages at once, hover over a tag key and click the **magic wand** button to apply tags with AI in the Global Tag Manager.
---
### Set send frequency caps using tags
You can also use tags to apply **send frequency caps** more granularly. This helps you control how often users receive messages within specific categories, such as by business unit or product line.
For example, if your agent includes messages across multiple business units, you can tag each message with a `Business Unit` value such as `Insurance` or `Banking`, then set separate frequency caps for each one. This helps prevent users from receiving too many messages from the same category, even when the overall agent-level cap has not been reached.
You can also set send frequency caps at the **channel level** (email, SMS, push, and so on). You do not need to tag whether a message is email, push, or SMS because AI Decisioning identifies the channel automatically.
---
## Troubleshooting
| Issue | Likely cause | Resolution |
|--------|---------------|------------|
| Tags not appearing in Insights | Agents not synced or tags missing | Check that each message or variant includes at least one tag and that Insights is enabled. |
| Duplicate tags | Inconsistent naming, such as `Loyalty` vs. `Loyalty Campaign` | Delete duplicates in **Global Tag Manager**. |
| Unclear AI suggestions | Inconsistent message naming or unclear tag definitions | Review your tag list, instructions, and message copy for clearer context. |
Tag visibility in Insights depends on your workspace's sync schedule. If you recently applied tags, wait for the next sync cycle before results appear.
---
## What's next?
After tagging your AID messages and variants, explore related features:
* [Analyze results in Insights →](/ai-decisioning/insights)
* [Inspect message performance →](/ai-decisioning/inspector)
* [Create Agents to automate decisions →](/ai-decisioning/agents)
* [Set up Collections for grouped experimentation →](/ai-decisioning/collections)
---
## Configure Match Booster on a model
**URL:** https://hightouch.com/docs/match-booster/configure-model
**Description:** Enable Match Booster on a model or parent model, map identifiers, configure enrichment settings, and run the initial enrichment.
**Section:** Match Booster
| | |
| ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Data teams, platform admins |
| **Prerequisites** | A [model](/models/creating-models) or Customer Studio [parent model](/customer-studio/schema) with at least one [supported identifier](#match-keys-reference), and Match Booster enabled on your workspace |
_Configure Match Booster on a model to enrich your first-party identifiers with additional emails, phone numbers, device IDs, and other identifiers from Hightouch's third-party identity graph._
---
## Overview
This page covers the model side of Match Booster setup — enabling enrichment, mapping your identifier columns, configuring settings, and running the initial enrichment. After enrichment completes, [enable Match Booster in your syncs](/match-booster/enable-in-syncs).
### Where Match Booster lives in Hightouch
You can enable Match Booster in two places, depending on how you build your audiences:
| Approach | Best for | Where to configure |
| ------------------------ | ----------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| Enable on a model | Syncs built directly from a model in Activation | [**Activation > Models**](https://app.hightouch.com/models?from=docs), select a model, then open the **Match Booster** tab |
| Enable on a parent model | Customer Studio audiences built from the schema | [**Customer Studio > Schema**](https://app.hightouch.com/schema?from=docs), select a parent model, then open the **Match Booster** tab |
---
## Enable Match Booster on the model
1. Open the model or parent model where you want to enable Match Booster:
- For models: go to [**Activation > Models**](https://app.hightouch.com/models?from=docs). Select the model and open the **Match Booster** tab.
- For parent models: go to [**Customer Studio > Schema**](https://app.hightouch.com/schema?from=docs). Select the parent model and open the **Match Booster** tab.
2. Toggle **Enable Match Booster** on.
This unlocks the expansion options, identifier mapping, and enrichment schedule sections.
---
## Select expansion options
Configure optional expansion behavior before mapping match keys.
- **Enable Household Expansion** — Enable this if you want to target, suppress, or measure at the household level instead of just the individual. Household Expansion matches a single record to a residential address in the identity graph and expands targeting to all people and devices in that household. To use Household targeting, you must also select the Household targeting level during [sync configuration](/match-booster/enable-in-syncs).
- **Enable CTV Matching** — Enable this if you run ads on connected TV platforms (Roku, LG, Samsung, Vizio). CTV Matching translates your first-party identifiers into CTV IDs and IP addresses that CTV ad platforms use for matching.
See [Targeting levels](/match-booster/targeting-levels) for guidance on when to use person-level versus household-level matching.
---
## Map match keys
Under **Which identifiers should be used to link your model to Hightouch's identity graph?**, map your match keys by selecting a warehouse **Column name** and an **Identifier type** for each identifier you want to use.
1. Select the identifiers you want to match into the identity graph. Common mappings include:
- Email (raw or hashed — Hightouch hashes raw values automatically)
- Phone number (raw)
- Name and address fields (for name/address matching)
- Device IDs, IP addresses, or mobile ad IDs (for anonymous or CTV use cases)
2. For name and address matching, you must provide **first name**, **last name**, and at minimum the **first address line** and **postal code**. Including city and state improves accuracy.
3. Select **Save** when your mappings are complete.
Select the magic wand icon to automatically map match keys based on your
column names.
### Match keys reference
Match keys determine which records can be matched, what additional identifiers Match Booster can add, and what enriched identifiers are available in your destination syncs.
#### Known users
Use known-user matching when you have user-level identifiers like email, phone, MAIDs, or postal address information.
**Supported inputs:**
| Match key | Supported format | Notes |
| ------------------- | ----------------------- | ------------------------------------------------------------------- |
| Email | Plain text, MD5, SHA256 | |
| Phone | Plain text | |
| MAIDs | Plain text | Mobile advertising IDs |
| Full postal address | Plain text | First name, last name, street 1, street 2, city, state, postal code |
**Enriched identifiers added:**
| Identifier | Notes |
| ---------------- | ------ |
| Email | Hashed |
| Phone | Hashed |
| MAIDs | |
| IP address (CTV) | |
| First name | |
| Last name | |
| City | |
| State | |
| Postal code | |
#### Anonymous users
Use anonymous matching when you have IP address data but no known user identifiers.
**Supported inputs:**
| Match key | Supported format | Notes |
| ---------- | ---------------- | --------------------------- |
| IP address | Plain text | Used for anonymous matching |
**Enriched identifiers added:**
| Identifier | Notes |
| ------------ | ------ |
| Email | Hashed |
| Mobile ad ID | |
#### Conversion events
Use conversion matching when you sync conversion records and want ad platforms to match more of those events.
**Supported inputs:**
| Match key | Supported format | Notes |
| --------- | ----------------------- | ----- |
| Email | Plain text, MD5, SHA256 | |
| Phone | Plain text | |
**Enriched identifiers added:**
| Identifier | Notes |
| ----------- | ------ |
| Email | Hashed |
| Phone | Hashed |
| First name | |
| Last name | |
| City | |
| State | |
| Postal code | |
---
## Identifier normalization
If you pre-hash email or phone values, normalize your plaintext identifiers using the rules below so your hashes match what Hightouch requires. EU enrichment requires stricter normalization than US enrichment because the EU identity graph stores values in a different format.
### Email
In all regions, lowercase the email before hashing.
For US enrichment, also trim leading and trailing whitespace.
For EU enrichment, also:
- Remove all whitespace from the email.
- Remove accents (for example, `é → e`, `ü → u`).
- Replace these special characters: `ß → ss`, `œ → oe`, `æ → ae`, `þ → th`.
- For Gmail addresses (`@gmail.com`), remove dots and `+tag` suffixes from the part before the `@`.
Examples (plaintext → value to hash):
- `Marie.Dupré@example.com` → `marie.dupre@example.com`
- `weiß@example.de` → `weiss@example.de`
- `John.Doe+promo@gmail.com` → `johndoe@gmail.com`
### Phone
For EU enrichment, format phone numbers as E.164 (with the leading `+`, for example `+447700900123`). This formatting is required for both plaintext and pre-hashed phone values.
### Postal codes and addresses
In all regions, lowercase names and address fields (street, city, state).
For postal codes:
- **US enrichment** — Keep only the digits (5-digit ZIP, 3-digit ZIP prefix, or ZIP+4).
- **EU enrichment** — Remove whitespace but keep letters and numbers. For example, UK `SW1A 1AA` becomes `sw1a1aa`.
---
## Set enrichment schedule
Configure how frequently Hightouch enriches your model against the identity graph.
The **Enrichment schedule** section lets you set a recurring interval (for example, every 1 day). Schedules are capped at 28 days to meet data retention requirements.
Enrichment creates a secure mapping between your hashed primary keys and the third-party identity graph without storing raw PII.
---
## Add enrichment filters (parent models only)
On parent models, you can use enrichment filters to define a subset of records to enrich. Common filters include:
- Limiting enrichment to specific regions
- Enforcing consent rules by enriching only consented users
---
## Run initial enrichment
Select **Initialize Match Booster** to run the first enrichment.
The initial run performs a full enrichment and may take longer for large datasets. Subsequent runs only enrich new or updated records.
Initialization is a one-time step that can take up to 72 hours. Select **Email
me when enrichment is complete** to avoid monitoring progress manually.
---
## Next steps
Configuring Match Booster on a model is step one. Enrichment does not flow to ad platforms until you enable Match Booster on a sync.
After the initial enrichment completes:
- [**Enable Match Booster in syncs**](/match-booster/enable-in-syncs) — Turn on Match Booster in your destination syncs to start sending enriched identifiers to ad platforms.
---
## Enable Match Booster in syncs
**URL:** https://hightouch.com/docs/match-booster/enable-in-syncs
**Description:** Turn on Match Booster in supported destination syncs to send enriched identifiers for higher audience reach.
**Section:** Match Booster
| | |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Audience** | Marketers, platform admins |
| **Prerequisites** | Match Booster [enabled and enriched on a model](/match-booster/configure-model), and a sync to a [supported destination](/match-booster/overview#supported-destinations) |
_After Match Booster enrichment has completed on a model, enable it in syncs to send enriched identifiers to supported advertising destinations._
---
## Overview
When you enable Match Booster on a sync, Hightouch adds the matched identifiers from the model's enrichment to the data it sends to the ad platform. Before you begin, [configure Match Booster on a model](/match-booster/configure-model) and wait for the initial enrichment to complete.
You can only enable Match Booster on syncs to [supported
destinations](/match-booster/overview#supported-destinations). The toggle does
not appear for unsupported destinations.
---
## Enable Match Booster in a sync
1. Go to [**Activation > Syncs**](https://app.hightouch.com/syncs?from=docs). Select an existing sync or create a new one.
2. Open the **Configuration** tab.
3. In the field mapping section, toggle **Enable Match Booster for higher audience reach** on.
4. Choose a [targeting level](#choose-a-targeting-level).
5. Review which identifiers Hightouch will send. Hightouch automatically selects the best available enriched identifiers for each destination. Some destinations allow you to remove specific identifiers if needed.
6. Select **Save changes**.
You do not need to manually map enriched Match Booster fields. Hightouch
automatically sends the appropriate identifiers based on what the destination
supports.
If you just enabled Match Booster for the first time, wait for the initial
enrichment to complete before running your sync. You can check enrichment
status on the model's **Match Booster** tab.
---
## Choose a targeting level
When you enable Match Booster on a sync, select a targeting level that matches your campaign goals:
| Targeting level | What it does | When to use it |
| --------------- | ---------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Person** | Targets individuals based on person-level identifier matching. | Default for most campaigns. Use for precision targeting and suppression. |
| **Household** | Expands targeting to all people and devices at the same residential address. | Use when the product or service is used by the entire household (retail, streaming, automotive, telecom). Requires [Household Expansion](/match-booster/configure-model#select-expansion-options) enabled on the model. |
| **CTV** | Targets consumers on connected TV devices. | Use for CTV campaigns on Roku, LG, Samsung, and Vizio. Requires [CTV Matching](/match-booster/configure-model#select-expansion-options) enabled on the model. |
Targeting levels cannot be changed after the sync is enabled. To change the targeting level for an audience, create a new sync with the same configuration and set the targeting level before activating it.
See [Targeting levels](/match-booster/targeting-levels) for detailed guidance on person-level versus household-level matching.
---
## Monitor match rates
Hightouch surfaces match rates for syncs to paid advertising destinations, whether or not Match Booster is enabled.
To view match rates for a sync:
1. Go to [**Activation > Syncs**](https://app.hightouch.com/syncs?from=docs).
2. Select the sync you want to review.
3. Open the **Overview** tab.
4. In the right-hand panel, find **Matched users** and click **See details**.
The details panel shows:
- Audience health status and monitoring state
- Number of matched users in the ad platform audience
- A link to view your audience directly in the ad platform
Match rates can take up to 48 hours to stabilize after the initial sync,
depending on the ad platform. Allow time for results to settle before
evaluating performance.
---
## Match Booster common questions
**URL:** https://hightouch.com/docs/match-booster/faqs
**Description:** Answers to common questions about Match Booster identifier formats, enrichment timing, data storage, and match rate behavior.
**Section:** Match Booster
| | |
| ------------ | --------------------- |
| **Audience** | Data teams, marketers |
_Answers to frequently asked questions about identifier formats, enrichment timing, data handling, and match rate behavior._
---
### What format should source identifiers use?
Match Booster supports hashed (SHA256 or MD5) identifiers for emails and phone numbers. Hashed values should be lowercased and trimmed before hashing.
Other PII identifiers (name, address, device IDs) should be in raw, unhashed format. Hightouch normalizes these identifiers — for example, by lowercasing and removing spaces — to ensure they match data in the identity graph.
For the rules that apply to each identifier, see [Identifier normalization](/match-booster/configure-model#identifier-normalization).
---
### How should I model multiple identifiers of the same type?
Consolidate all values for one identifier type in a single row. Capture multiple entries as an array. If you have just one value per identifier, a string is appropriate.
For example:
| First Name | Email | Phone | uuid |
| ---------- | --------------------------------------- | ----------------------------------- | ---- |
| Tyler | `"tyler@work.com"` | `["555-123-4567" , "999-555-1234"]` | 1 |
| Amina | `["amina@work.com", "amina@gmail.com"]` | `"555-567-8309"` | 2 |
Hightouch formats the data to match the requirements of ad platform APIs.
---
### When does enrichment take place? Does Hightouch store a copy of my data?
After you configure Match Booster on a model, Hightouch runs a comparison of your selected identifiers against the identity graph and looks for exact matches. Hightouch uses a hashed version of your model's primary key to track matched rows.
Hightouch never stores your proprietary data. See [Data storage and privacy](/match-booster/overview#data-storage-and-privacy) for details on how data flows.
---
### How long does model enrichment take?
After configuring your model and selecting which identifiers to use, you can expect enrichment to take a few hours and up to one day. This applies to both the initial enrichment and subsequent runs.
The initial calibration (initialization) can take up to 72 hours.
---
### How often do enriched models refresh?
You choose how frequently your models are refreshed using the enrichment schedule. Schedules are capped at 28 days to meet data retention requirements.
---
### How can I check when my model was last enriched?
You can find enrichment status on the model's **Match Booster** tab, including:
- Enrichment schedule
- Last enrichment time
- Next scheduled enrichment
---
### Can I manually trigger an enrichment?
Yes. Select **Initialize Match Booster** (for first-time runs) or save your configuration to trigger enrichment from the model's **Match Booster** tab.
---
### Why do my match rates appear lower with Match Booster?
After the initial sync to an advertising platform, match rates can take up to 48 hours to stabilize depending on the platform. Allow enough time for results to calculate before comparing boosted versus non-boosted performance.
---
### Why is Snapchat reporting greater than 100% match?
Snapchat reports match rates differently than other platforms. If you send multiple identifiers for the same user, Snapchat initially reports separate matches, which can result in a match rate greater than 100%. Snapchat deduplicates and resolves multiple matches to a single user over time. Check back in a day or two to see the stabilized match rate.
---
## Set up Match Booster
**URL:** https://hightouch.com/docs/match-booster/implementation
**Description:** Configure Match Booster to enrich your data with additional identifiers and improve match rates.
**Section:** Match Booster
This page has moved. See [Configure Match Booster on a model](/match-booster/configure-model) for model-side setup and [Enable Match Booster in syncs](/match-booster/enable-in-syncs) for sync-side activation.
---
## Matching methods
**URL:** https://hightouch.com/docs/match-booster/matching-options
**Description:** Understand how Match Booster matches consumer data, work data, address data, and anonymous visitor data against its identity graph.
**Section:** Match Booster
| | |
| ------------ | --------------------- |
| **Audience** | Data teams, marketers |
_Match Booster takes your first-party identifiers across multiple data types and resolves them to people and devices in its identity graph, so ad platforms can match more of your records._
---
## Overview
Match Booster supports four types of input data, each suited to different use cases. The type of data you provide determines which records can be matched and what enriched identifiers Match Booster can append.
For details on configuring these inputs, see [Map match keys](/match-booster/configure-model#map-match-keys).
---
## Consumer data
Consumer data matching enriches your first-party records with additional emails, phone numbers, device IDs, and cookie IDs. Use this method when you collect emails or phone numbers from consumers and want ad platforms to match more of those records.
This is the most common matching method and works well for targeting, suppression, and lookalike campaigns.
---
## Work data (B2B)
Match Booster matches B2B employees to consumer profiles and devices. If you collect work emails from prospects or have purchased a list of business contacts, work data matching enriches those records with consumer emails, phone numbers, device IDs, and cookie IDs that ad platforms are more likely to match on.
This is useful when your first-party data is primarily work-context identifiers that have low match rates in consumer-oriented ad platforms.
---
## Address data
Match Booster matches first-party name and address data to person-level identifiers in its identity graph, including hashed emails, phone numbers, and device IDs.
To use address matching, provide at minimum: first name, last name, first address line, and postal code. Including city and state improves accuracy. See [Match keys reference](/match-booster/configure-model#match-keys-reference) for format requirements.
---
## Anonymous web visitor data (IP address)
Match Booster matches anonymous visitor IP addresses to known consumers and devices using reverse IP lookup. When you provide a first-party IP address, Hightouch filters the data to verify it comes from a residential IP address. If it does, Hightouch's identity graph matches that IP address to the people and devices associated with that household and sends those identifiers to ad platforms.
This method is designed for web retargeting scenarios where you have visitor session data with IP addresses but no known user identifiers.
---
## Match Booster offerings
**URL:** https://hightouch.com/docs/match-booster/offerings
**Description:** Learn about the audience boosting, conversion event boosting, anonymous visitor boosting, and CTV device boosting offerings.
**Section:** Match Booster
| | |
| ------------ | --------------------- |
| **Audience** | Marketers, data teams |
_Match Booster includes four offerings that address different activation scenarios: audience boosting, conversion event boosting, anonymous visitor boosting, and CTV device boosting._
---
## Audience boosting
Audience boosting matches more of your customer records to users across ad platforms, including walled gardens, DSPs, SSPs, CTV platforms, and publisher-direct destinations.
Hightouch can match audiences across consumers, B2B employees, households, and devices.
**Use cases:**
- **Custom targeting** — Help ad platforms match more of your audience so campaigns reach a larger share of the segment.
- **Suppression** — Improve suppression lists so ad platforms can exclude more existing customers from acquisition campaigns.
- **Lookalikes** — Give ad platforms a larger matched seed audience to build more representative lookalike models.
- **Bid optimization** — Help ad platforms distinguish new versus returning customers for bid strategy adjustments.
- **Offer optimization** — Match more of the qualified audience for special offers, pricing, or discounts, such as loyalty members who qualify for discounted rates on search platforms.
---
## Conversion event boosting
Conversion event boosting helps ad platforms match more of the online and offline conversion events you send, so more conversions are attributed to ad interactions.
Conversion event boosting does not cause the same event to be sent to the ad
platform multiple times.
**Use cases:**
- **Attribution and measurement** — Match more conversions to ad impressions and clicks for more accurate attribution and incrementality measurement.
- **Budget allocation** — When more conversions are attributed, ad platforms can make better budget allocation decisions across ads, ad groups, and campaigns.
- **Conversion optimization** — Send more complete conversion data to the ad platform's bidding and targeting systems.
---
## Anonymous visitor boosting
Anonymous visitor boosting resolves anonymous visitors via IP addresses. You may be collecting user events and their associated IP addresses, even when the visitor does not provide personally identifiable information like email or phone. Hightouch's anonymous identity graph uses reverse IP lookup to identify emails and device IDs associated with an IP address, then sends those identifiers to ad platforms for retargeting.
Hightouch rigorously cleans observed linkages between IP addresses, hashed
emails, and MAIDs to eliminate public locations.
**Use cases:**
- **Web retargeting** — Retarget anonymous visitors to your web properties by resolving their IP addresses to matchable identifiers.
---
## CTV device boosting
CTV device boosting targets consumers on connected TV devices like Roku, LG, Samsung, and Vizio. Most connected TV platforms rely on CTV IDs and IP address matching to target viewers with ads. Hightouch's identity graph translates your first-party identifiers into CTV IDs and IP addresses that these platforms use for matching.
Hightouch rigorously cleans observed linkages between IP addresses, hashed
emails, and MAIDs to eliminate public locations.
**Use cases:**
- **User acquisition** — Acquire new app installs and subscribers through personalized CTV ads and more effective suppression.
- **Subscription upsell** — Upsell customers from a free to paid tier on your CTV application.
- **Tune-in** — Drive consumers to your app for specific shows, movies, or live events by displaying native ads on smart TV home screens.
---
## Match Booster overview
**URL:** https://hightouch.com/docs/match-booster/overview
**Description:** Help ad platforms match more of your customers by safely adding more identifiers to the data you sync from Hightouch.
**Section:** Match Booster
| Audience | How you'll use this article |
|----------|----------------------------|
| **Marketing** | Learn what Match Booster does, when to use it, and how it can help improve campaign reach and measurement. |
| **Data teams** | Learn what data Match Booster uses, how it adds identifiers to syncs, and which destinations support it. |
*Match Booster helps ad platforms recognize more of your customers by adding additional identifiers, such as alternate emails, phone numbers, or device IDs, to the audiences and conversion events you sync from Hightouch.*
---
## What you'll learn
- [What Match Booster does and how it works](#overview)
- [Core features: audience, conversion, anonymous visitor, and CTV boosting](#core-features)
- [How to set up Match Booster](#how-to-set-up-match-booster)
- [When to use Match Booster](#use-cases)
- [How Match Booster fits into Hightouch](#how-match-booster-fits-into-hightouch)
- [Data storage and privacy](#data-storage-and-privacy)
- [Supported destinations](#supported-destinations)
---
## Overview
Ad platforms need to recognize your customers before they can include them in campaigns or connect their actions back to your ads.
When you send an audience or conversion event to an ad platform, Hightouch includes identifiers from your data, such as email addresses, phone numbers, or device IDs. The ad platform compares those identifiers to the users it knows about.
Sometimes, the platform can't find a match. For example, you might send one email address for a customer, but the ad platform knows that same person by a different email, phone number, or device ID. When that happens, the customer may not be included in your audience, and their conversion may be harder to measure.
Match Booster helps solve this by adding more identifiers to the records you sync. Hightouch checks the identifiers in your model against its licensed third-party data and finds related identifiers — such as alternate emails, phone numbers, or device IDs — for the same person, household, or device. The ad platform then has more ways to recognize each customer.
Match Booster adds these identifiers when Hightouch sends data to the destination. The added identifiers are not written back to your warehouse.
[**Explore the interactive architecture diagram**](https://architecture.hightouch.com/?from=docs&product=match-booster&utm_source=hightouch-docs&utm_medium=referral&utm_campaign=architecture-explorer&utm_content=match-booster)
See how data flows and where Match Booster fits in your stack.
---
## How to set up Match Booster
Setting up Match Booster requires two steps: first configure enrichment on a model, then enable it in your syncs. Enrichment must complete before syncs can send boosted identifiers.
See [Get started with Match Booster](/match-booster/quick-start) for the step-by-step walkthrough.
---
## Core features
Match Booster includes four offerings, each designed for a different activation scenario:
- **Audience boosting** — Help ad platforms match more of the people in the audiences you sync.
- **Conversion event boosting** — Help ad platforms match more conversion events to ad interactions for better attribution and optimization.
- **Anonymous visitor boosting** — Reach website visitors who haven't logged in by resolving IP addresses to matchable identifiers.
- **CTV device boosting** — Translate first-party identifiers into CTV IDs and IP addresses for connected TV campaigns.
See [Offerings](/match-booster/offerings) for detailed use cases and examples.
---
## How Match Booster fits into Hightouch
Match Booster works between your Hightouch model and your destination sync.
It does not change your model, your audience rules, or the source data in your warehouse. Instead, it adds more identifiers to the records Hightouch sends to ad platforms.
That means you still define your audiences the same way. Match Booster gives the destination more information to improve matching.
#### What Match Booster uses
- **Your models and parent models** — Match Booster reads the identifiers you choose from the model you configure it on. These can include email, phone number, name, address, device ID, or IP address.
- **Hightouch's licensed third-party data** — Hightouch maintains a licensed identity graph sourced from vetted data vendors. Hightouch rigorously cleans and validates this data — filtering out public locations, checking recency and frequency of observed linkages, and prioritizing verified records — before using it to find additional identifiers that may belong to the same person, household, or device.
#### What Match Booster sends
- **Additional identifiers in syncs** — When a sync runs, Hightouch adds the matched identifiers to the data it sends to the destination. The added identifiers are not written back to your warehouse.
- **Match rate data** — Hightouch shows match rate information on the sync overview page so you can compare performance with and without Match Booster.
### Match Booster vs. Identity Resolution
Match Booster and [Identity Resolution](https://hightouch.com/platform/identity-resolution) help with different parts of customer data matching:
- **Identity Resolution** helps you combine customer records across your own datasets. For example, it can help determine that two records in your warehouse belong to the same person.
- **Match Booster** helps ad platforms recognize more of the records you send from Hightouch by adding third-party identifiers to the data you sync.
You can use them together. Use Identity Resolution to create cleaner, more complete customer profiles in your warehouse. Then use Match Booster to help ad platforms recognize more of those customers when you send them.
---
## Use cases
| Use case | Example |
| --- | --- |
| **Custom audience targeting** | Add more identifiers to a high-value customer audience so Meta can recognize more of the people in that segment. |
| **Suppression** | Improve a "purchased in the last 30 days" suppression list so ad platforms can exclude more existing customers from acquisition campaigns. |
| **Conversion optimization** | Add more identifiers to offline conversion events so Google Ads can match more conversions to ad clicks. |
| **Lookalike seeds** | Help an ad platform match more people in your seed audience, giving it a larger group to use when building a lookalike audience. |
| **Anonymous retargeting** | Use IP address data to find related emails or device IDs for anonymous website visitors, then sync those identifiers to ad platforms for retargeting. |
| **CTV targeting** | Translate first-party identifiers into connected TV identifiers so you can reach customers on platforms like Roku, LG, Samsung, and Vizio. |
| **Retail media monetization** | List enriched audiences on DSP marketplaces without giving up revenue to a middleman, using Match Booster's zero-fee activation model. |
---
## Data storage and privacy
Match Booster is built on a warehouse-native architecture that eliminates the need for data replication. Your data is never persisted on Hightouch infrastructure, besides a hash of your model's primary key.
### How data flows
Hightouch uses data sharing to match its third-party identity graph to your first-party data and maintains a pseudonymized mapping of the identity graph on its own premises. When a sync runs, Hightouch queries the third-party identifiers and adds them to your first-party data at sync time.
This means you never have to store regulated third-party data in your infrastructure, and Hightouch never has to store your first-party data on its premises. Your data is not stored at rest outside of your own infrastructure.
Match Booster enriches records with two categories of additional identifiers:
- **Offline identifiers** — such as email addresses and phone numbers.
- **Online identifiers** — such as mobile device IDs.
### Benefits of this architecture
- **Security** — Your first-party data is never stored with an intermediary, reducing security and privacy risk.
- **Direct data access** — Match Booster queries your warehouse directly. When new data is added to your first-party data, it is available immediately with no extra pipelines.
- **Faster refreshes** — Hightouch supports enrichment on a daily or custom schedule, so you don't have to wait weeks for data to be onboarded.
- **Simpler workflow** — Hightouch acts as an enrichment and pass-through layer. You don't need to maintain segments or audiences in a separate onboarding tool.
- **No media-based activation fees** — There are no revenue share, CPM, or percentage-of-media fees for sending enriched data to ad platforms, DSPs, SSPs, or publisher-direct destinations.
---
## Regional coverage
Match Booster currently supports the following region:
- **North America (NAMER)** — United States
Contact your account representative for the latest regional availability.
**EU enrichment** is available in early access. Contact your account
representative to enable it. See [Identifier
normalization](/match-booster/configure-model#identifier-normalization) for
the rules that apply when EU is enabled.
---
## Supported destinations
You can use Match Booster with the following destinations.
**Custom audiences**
- [Amazon Ads DSP](/destinations/amazon-ads)
- [Criteo](/destinations/criteo)
- [Digital Turbine](/destinations/digital-turbine)
- [Dynata](/destinations/dynata)
- [Meta Custom Audiences](/destinations/facebook)
- [Google Ads](/destinations/google)
- [Google Display & Video 360](/destinations/display-video)
- [InMobi](/destinations/inmobi)
- [iSpot.tv](/destinations/ispot-tv)
- [LiveIntent](/destinations/liveintent)
- [LG Ad Solutions](/destinations/lg-ad-solutions)
- [LinkedIn Ads](/destinations/linkedin)
- [Microsoft Bing Ads Audiences](/destinations/bingads)
- [Pinterest Ads](/destinations/pinterest-ads)
- [Reddit Ads](/destinations/reddit-ads)
- [Rokt](/destinations/rokt)
- [Roku Ads](/destinations/roku-ads)
- [Samsung Ads](/destinations/samsung-ads)
- [Snapchat](/destinations/snapchat)
- [StackAdapt](/destinations/stackadapt)
- [Taboola](/destinations/taboola)
- [TikTok](/destinations/tiktok)
- [The Trade Desk](/destinations/tradedesk)
- [X](/destinations/x)
- [Vistar Media](/destinations/vistar-media)
- [Vizio Ads](/destinations/vizio-ads)
- [Yahoo DSP](/destinations/yahoo)
**Conversion events**
- [Meta Conversions](/destinations/meta-conversions)
- [Google Ads](/destinations/google)
- [Pinterest Ads](/destinations/pinterest-ads)
- [TikTok](/destinations/tiktok)
The list of supported destinations is growing. if there are additional ad platforms you want to use with Match Booster.
---
## Get started with Match Booster
**URL:** https://hightouch.com/docs/match-booster/quick-start
**Description:** Set up Match Booster in two steps: configure enrichment on a model, then enable it in your syncs.
**Section:** Match Booster
Setting up Match Booster is a two-step process. Both steps are required — enrichment must complete before syncs can send boosted identifiers.
---
## Step 1: Configure enrichment on a model
**Who:** Data teams or platform admins
Before enabling Match Booster, confirm that your model or parent model includes at least one supported identifier (email, phone, MAID, IP address, or name + postal address).
Then, on the model's **Match Booster** tab:
1. Enable Match Booster and map your identifiers.
2. Optionally enable Household Expansion or CTV Matching.
3. Set the enrichment schedule.
4. Run the initial enrichment (one-time, can take up to 72 hours).
Wait for the initial enrichment to complete before moving to Step 2. Select
**Email me when enrichment is complete** during initialization so you don't
have to check manually.
See [Configure Match Booster on a model](/match-booster/configure-model) for the full walkthrough.
---
## Step 2: Enable Match Booster in your syncs
**Who:** Marketers or platform admins
Once enrichment has completed, open a sync to a [supported destination](/match-booster/overview#supported-destinations) and:
1. Toggle **Enable Match Booster** on in the sync's **Configuration** tab.
2. Choose a targeting level: Person, Household, or CTV.
3. Review the mapped identifiers — Hightouch automatically selects the best available enriched identifiers for the destination.
See [Enable Match Booster in syncs](/match-booster/enable-in-syncs) for the full walkthrough.
---
## Monitor match rates
After a sync runs with Match Booster enabled, check match rates on the sync's **Overview** tab under **Matched users** to assess the impact of enrichment.
Match rates can take up to 48 hours to stabilize after the initial sync, depending on the ad platform.
See [Monitor match rates](/match-booster/enable-in-syncs#monitor-match-rates) for details.
---
## Targeting levels
**URL:** https://hightouch.com/docs/match-booster/targeting-levels
**Description:** Choose between person-level and household-level targeting in Match Booster to match your campaign goals.
**Section:** Match Booster
| | |
| ------------ | --------------------- |
| **Audience** | Marketers, data teams |
_Match Booster supports targeting at the person level and household level. Targeting levels are set per sync and determine how Match Booster groups matched identifiers._
---
## Overview
Person-level matching is the default targeting level. Household-level targeting must be enabled in the model's Match Booster configuration (via [Household Expansion](/match-booster/configure-model#select-expansion-options)) before it can be applied to a sync.
Targeting levels cannot be changed after the sync is enabled. If you want to change the targeting level for an audience, create a new sync with the same configuration and set the targeting level before activating it.
---
## Person-level matching
Person-level matching links your customer and device records to a single individual. Each record maps to one person in the identity graph.
Match Booster finds deterministic linkages between your first-party identifiers and people in its third-party identity graph. When matching to devices, Match Booster can isolate the most likely user of a device or broaden the scope by including shared devices in the match criteria (for example, a shared iPad or tablet).
---
## Household-level matching
Household-level matching uses Household Expansion to match a single record to a residential address in the identity graph and then expands targeting to reach all of the people and devices in that household.
Household Expansion is especially effective for brands and media networks whose products are purchased or used by an entire household:
- **Retail marketing and retail media networks** — Loyalty memberships are often shared across family members. Household Expansion helps target, suppress, and monetize the entire household based on past purchase behavior.
- **Streaming TV** — Household members typically share a single streaming login. Household Expansion targets every member of the household for more effective audience matching and suppression.
- **Automotive** — Vehicle purchases are often family decisions. Household Expansion targets all members of the household based on purchase behavior.
- **Telecom and utilities** — Services like phone, internet, and HVAC are sold at the household level. Household Expansion matches a single customer record to the entire household for more effective suppression and targeting.
Consider creating separate syncs for person-level and household-level
audiences so you can compare performance across targeting strategies.
---
## Consent Manager
**URL:** https://hightouch.com/docs/events/consent/consent-manager
**Description:** Hightouch Events Consent Manager UI
**Section:** Hightouch Events
Hightouch Events offers the Consent Manager UI for managing user consent and tracking preferences. Users can opt in/out of specific categories of data collection, and those preferences are synced to your configured destinations.
## Prerequisites
In order to use Consent Manager UI, you must use the Browser (Javascript) SDK snippet on your site.
Note: `e.load('WRITE_KEY')` is removed from the snippet since the Consent Manager will load the Browser SDK for you.
```html
```
## Installation
The Consent Manager is available via [NPM](https://npmjs.org/@ht-sdks/consent-manager). It can be installed on any site using a `
```
```jsx
return (
)
}
```
## API
For a full list of features and options, see the [README](https://github.com/ht-sdks/consent-manager).
---
## Hightouch Events + OneTrust
**URL:** https://hightouch.com/docs/events/consent/onetrust
**Description:** Hightouch Events with OneTrust consent integration
**Section:** Hightouch Events
Hightouch Events offers a pre-built integration for managing user consent with OneTrust.
## Prerequisites
Make sure to setup your consent categories in OneTrust before integrating with Hightouch Events.
The integration expects the [OneTrust Banner SDK](https://my.onetrust.com/articles/en_US/Knowledge/UUID-66bcaaf1-c7ca-5f32-6760-c75a1337c226) to be available in order to interact with OneTrust. This library derives the group IDs that are active for the current user from the `window.OneTrustActiveGroups` object provided by the OneTrust SDK ([more info](https://my.onetrust.com/articles/en_US/Knowledge/UUID-66bcaaf1-c7ca-5f32-6760-c75a1337c226)).
## Installation
The Hightouch Events + OneTrust integration is available via [NPM](https://npmjs.org/@ht-sdks/events-sdk-js-consent-wrapper-onetrust). It can also be installed using a `
```
## API
For a full list of features and options, see the [README](https://github.com/ht-sdks/events-sdk-js-mono/tree/master/packages/consent/consent-wrapper-onetrust).
---
## Manage contracts
**URL:** https://hightouch.com/docs/events/contracts/management
**Description:** Define, validate, and version event schemas using contracts to ensure reliable event data.
**Section:** Hightouch Events
| Audience | How you’ll use this article |
| --------------------- | -------------------------------------------------------------------------------------------------------- |
| **Marketing teams** | Understand how contracts protect downstream tools from incomplete or inconsistent event data. |
| **Data teams** | Define required event properties, manage schema versions, and control how undeclared fields are handled. |
| **Engineering teams** | Implement JSON Schema validation, configure enforcement rules, and manage context field validation. |
## Overview
Contracts define the events and fields Hightouch expects from an event source, so you can catch incomplete or inconsistent data before it reaches your warehouse.
Contracts have two components:
- A **contract event** defines the name, type, description, and validation schema for a single event (for example, `Order Completed`).
- A **contract** is a collection of contract events that can be applied to one or more event sources (for example, `E-commerce Events`).
Each contract event belongs to a single contract. A contract can be applied to multiple event sources.
---
## Create a contract
To create a new contract:
1. Go to **Event collection → Contracts**.
2. Click **New contract**.
3. Complete the setup flow:
### 1. Name & description
Provide a contract name and optional description.
### 2. Select sources
Choose the event sources this contract should monitor.
Hightouch will validate events coming from these sources against the contract schema.
### 3. Enforcement
Configure how Hightouch handles events that don’t match the contract.
#### Undeclared event types
When an event type is sent that is not defined in the contract:
- **Allow** — Sync the event with a warning.
- **Block** — Prevent the event from syncing.
After saving, you can begin adding contract events.
---
## Add a contract event
To define an event within a contract:
1. Open your contract.
2. Click **Add event**.
3. Configure the event in the **Configuration** section:
- **Type** (for example, `Track`)
- **Name** — must exactly match the event name sent in your code (including capitalization and spacing).
---
### 1. Define the event schema
When creating a contract event, the most important step is defining its **validation schema**.
Under **Schema**, define the structure of the event payload.
You can:
- Use the visual schema builder
- Toggle **Edit as JSON** to directly modify the schema
- Create multiple schema versions
Hightouch uses [JSON Schema Draft 7](https://json-schema.org/draft-07/json-schema-release-notes). This supports advanced validation, including [validating field values with regular expressions](https://json-schema.org/understanding-json-schema/reference/regular_expressions).
#### Define event properties
Under **Properties**, you can:
- Add fields using **Add property**
- Set field types (for example, `String`, `Number`, `Array`)
- Mark fields as **Required**
- Define nested objects or array element types
These rules determine whether an event is considered valid.
#### Configure context validation
Events include a `context` object, which contains metadata such as device information, page details, or SDK-provided fields.
Under **Context**, you can define validation rules for context fields.
#### Include built-in context fields
Enable **Include built-in context fields** to automatically allow [standard SDK fields](/events/event-spec#automatically-captured-fields) (like device info, page data, IP address, etc.) without having to define them manually.
If you define a context field that shares a name with a built-in one, your definition takes precedence.
---
### 2. Manage schema versions
Contracts support **multiple versions of the same event schema**.
Versioning is useful when it is difficult to migrate your codebase to use the latest payload. For example, when an iOS app takes a while to be updated across all devices.
After creating your first event, you can create additional versions of the
schema. The first version is the "default" version, which applies to events that
do not specify a version in the payload. It is recommended that your events
follow an incrementing pattern (for example, v1, v2, v3), although not strictly
required.
In the **Schema** section, you can:
- Create a new version using the **+** button
- Switch between versions using the version dropdown (for example, `v2`, `default`)
- Edit versions independently
When invoking an event, you can specify the schema version via:
```
client.track('Order Completed', {
...
}, {
context: {
htevents: {
schemaVersion: "v2"
}
}
});
```
---
### 3. Enforcement behavior for fields
Contracts define what happens when an event contains:
- **Undeclared properties** (fields not defined in the schema)
- **Invalid fields** (missing required fields or incorrect types)
#### Undeclared fields
For undeclared fields within an event, you can choose:
- **Allow** — Sync the event with a warning.
- **Filter** — Sync the event but remove undeclared fields.
- **Block** — Prevent the event from syncing.
#### Invalid fields
For invalid fields, you can choose:
- **Allow** — Sync the event with a warning.
- **Block** — Prevent the event from syncing.
These settings apply to both event properties and context fields.
---
## Handle violations
**URL:** https://hightouch.com/docs/events/contracts/violations
**Description:** Understand the types of contract violations and configure how Hightouch enforces them.
**Section:** Hightouch Events
## Overview
A **violation** occurs when an event sent through an event source does not satisfy the contract applied to that source.
Violations help you identify incomplete, inconsistent, or unexpected event data before it impacts downstream tools.
Contracts match events exactly. The event name, field names, and field types
in an incoming event must match the contract as written, including
capitalization and spaces. A contract is a precise rule for the event data,
not a loose set of naming guidelines.
Events can cause three types of violations:
- An **undeclared event** violation occurs when a specific event is not
specified in the contract
- e.g. `Button Clicked` event is sent to Hightouch, but it is not in the
E-commerce Events contract
- An **undeclared field** violation occurs when a field in an event is not
specified in the validation schema
- e.g. `user_name` field is sent to Hightouch in the `Order Completed`
payload, but it is not specified in the validation schema
- An **invalid field** violation occurs when a field does not pass the schema
validation defined by JSON Schema rules
- e.g. `userId` is sent as a `string` when the schema defines it as a `number`
All violations are written to a separate `event_violations` table in your
destination, regardless of how violation enforcement is configured.
This table has the schema:
- **payload**: JSONB
- **event_name**: string (derived from payload)
- **event_type**: string (derived from payload)
- **schema_version**: string (derived from payload)
- **has_undeclared_schema**: boolean
- **has_undeclared_fields**: boolean
- **has_schema_violation**: boolean
- **violations**: string[]
### **Violation enforcement**
Hightouch allows you to configure how to handle violations at both the contract
and event level. Depending on the cleanliness of your current event tracking, you may want
to allow violations to be written to your main event tables.
This table shows how Hightouch can handle each violation type. The
bolded actions represent the default behavior.
| Invalid fields | Undeclared field | Undeclared event |
| :------------: | :--------------: | :--------------: |
| **Allow** | **Allow** | **Allow** |
| Block | Block | Block |
| | Filter | |
**Allow** writes the event and records the violation. **Block** rejects the event. **Filter** drops only the undeclared fields and writes the rest of the event.
Configure how to handle undeclared events at the contract level.
Schema violations and undeclared fields can be configured at an event-level on the event schema page.
## Diagnose why an event was blocked or flagged
When an event doesn't arrive the way you expect, it usually maps to one of the three violation types above. Most of the time the cause is a small mismatch between what the SDK sends and what the contract declares, not a missing contract.
Start with the `event_violations` table or the [debugger](/events/sources/overview#debugger) to see which violation fired, then check the matching cause below.
### Event name doesn't match the contract
A small difference between the event name in your tracking code and the event name in the contract registers as an **undeclared event**. The match is exact, so these all count as different events:
- `Order Completed` and `order completed` (capitalization)
- `Order Completed` and `Order Completed` (extra space)
- `Order Completed ` and `Order Completed` (trailing space)
Confirm the event name in your code matches the contract character for character. If you block undeclared events, a mismatch blocks the event.
### Fields are in the wrong place
Hightouch validates a field by where it sits in the event, not only by its name. A field that the contract expects under `context` but the SDK sends under `properties` registers as an **undeclared field** in `properties`, and as a missing field in `context`. Page and screen metadata are common cases, since some of it belongs in `context` rather than `properties`.
Check that each field is in the section the contract declares.
### Field type doesn't match
A field sent as the wrong type registers as an **invalid field** violation. A common case is an identifier sent as a string when the contract defines it as a number, or the reverse. Align the type in your tracking code with the type in the schema.
### Every event is blocked or the schema appears empty
If a whole source is blocked, check the contract before the events. An empty schema combined with settings that block undeclared events and fields blocks every incoming event, because nothing matches a contract that declares nothing. Confirm the contract has its events and fields defined, then confirm enforcement is set the way you intend.
When undeclared events are blocked and no matching schema exists, the raw violation recorded for the event is `Schema not found`. You'll see this text in the `event_violations` table and in the debugger's event detail. The debugger's blocked-state badge shows a different message in its tooltip — `Undeclared events are blocked in your contracts settings` — which explains the enforcement decision rather than the raw violation.
---
## Understand event types and payload structure
**URL:** https://hightouch.com/docs/events/event-spec
**Description:** Learn how Hightouch Events structures identify, track, page, screen, and group events—including required fields, identity behavior, and example payloads.
**Section:** Hightouch Events
| Audience | How you’ll use this article |
|--------|-----------------------------|
| **Marketing teams** | Understand which event types exist and what they represent. |
| **Data teams** | Validate event payloads and understand how events map to tables. |
| **Engineering teams** | Implement SDKs and ensure correct identity and schema usage. |
## Overview
Hightouch Events uses a standard event structure for identifying users, tracking actions, and recording page and screen views, while letting your team control event structure and identity behavior.
This article defines the **event tracking specification** used by Hightouch:
- Supported event types
- Required and optional fields
- Identity behavior across anonymous and known users
- Shared payload structure
- Example JSON payloads
Use this guide when implementing SDKs, reviewing event data, or validating incoming events.
---
## Supported event types
Hightouch supports the following event types:
| Event type | Purpose |
|----------|---------|
| [`identify`](#identify-events) | Identify a known user and set user-level traits |
| [`track`](#track-events) | Record an action a user performed |
| [`page`](#page-events) | Record a browser page view |
| [`screen`](#screen-events) | Record a mobile app screen view |
| [`group`](#group-events) | Associate a user with a group or account |
Each event type builds on a shared base structure and adds its own required fields.
---
## Shared event structure
All event types share a common base structure, so Hightouch can validate, govern, and activate events consistently across destinations.
### Required fields
- `type` — The event type (`identify`, `track`, `page`, `screen`, `group`)
- `messageId` — A unique identifier for the event
- At least one identifier: `anonymousId` or `userId`
- `originalTimestamp` — When the event occurred
- `sentAt` — When the event was sent by the SDK or API
### Optional fields
- `context` — Automatically collected metadata (device, OS, SDK version, etc.)
- `integrations` — Destination-specific flags
- `traits` or `properties` — Metadata attached to the event:
- **Traits** describe the user (for example, email or plan)
- **Properties** describe the action or context (for example, price or product ID)
---
### Identify events
**Identify events** capture data about *who* a user is. They are used to:
- Associate anonymous activity with a known user
- Set or update user-level traits
- Link events across devices and sessions
For example, you might use Identify events to power follow-up email campaigns, segment users by location or plan, or link anonymous activity to authenticated behavior.
Users often generate events *before* they log in or sign up. During this time, events are associated with an `anonymousId`.
When you later send an Identify event that includes both a `userId` and the same `anonymousId`, subsequent events from that device include both identifiers. You can then join earlier anonymous-only events to the known user in your warehouse models using the shared `anonymousId`.
---
#### Components of Identify events
- **`anonymousId`**
A device- or browser-level identifier generated by the SDK to associate events from the same user before they are identified. This value is included automatically in other event types (e.g., Track events).
An `anonymousId` typically persists across multiple sessions on the same device or browser.
- **`userId`**
A stable, persistent identifier for a known user (for example, it's common to use an ID generated by your database).
Avoid using mutable values such as email addresses.
- **`traits`**
Optional user attributes, such as email, name, or age.
#### Example identify event payload
```json
{
"type": "identify",
"messageId": "111e984d-c93c-444c-b29c-f499a117c500",
"traits": {
"email": "kevin@hightouch.io"
},
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"originalTimestamp": "2023-09-22T18:05:47.063Z",
"sentAt": "2023-09-22T18:05:47.064Z"
}
```
---
### Track events
**Track events** capture data about *what* a user did, such as completing a purchase or clicking a button.
Additional details about the action are included in the `properties` object. For example, for a `Purchase Complete` event, you may want to include
the `product ids` of the purchased products.
When events are stored in the warehouse, the `event` field is used to generate [table
names](./warehouse-schema#per-event-tables) in the warehouse. Additionally, the
`properties` field is autoconverted into
[columns](./warehouse-schema#per-event-tables) in the warehouse.
#### Example track event payload
```json
{
"type": "track",
"messageId": "7b1e2872-4ebb-4779-b9ad-e7aa1a8eab82",
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"event": "Purchase completed",
"properties": {
"cart_total": 24.99
},
"originalTimestamp": "2023-09-22T18:12:00.584Z",
"sentAt": "2023-09-22T18:12:00.585Z"
}
```
### Page events
**Page events** track the webpages users interact with. In most situations, you
should automatically call the page load event when a page is loaded.
For single-page applications, trigger page events on route changes.
#### Example page event payload
```json
{
"type": "page",
"messageId": "bb9fcbbd-9e19-4bfd-a726-7b1e7e44f8ba",
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"properties": {
"title": "Home"
},
"originalTimestamp": "2023-09-22T18:05:47.061Z",
"sentAt": "2023-09-22T18:05:47.063Z"
}
```
### **Screen events**
**Screen events** are the equivalent of page views for _mobile apps_. Like
page events, you should automatically call the screen event when a mobile
screen view is loaded.
#### Example screen event payload
```json
{
"type": "screen",
"messageId": "bb9fcbbd-9e19-4bfd-a726-7b1e7e44f8ba",
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"properties": {
"title": "Home"
},
"originalTimestamp": "2023-09-22T18:05:47.061Z",
"sentAt": "2023-09-22T18:05:47.063Z"
}
```
### **Group events**
Group events let you associate users with a larger group. For example, a user
may be part of an organization, or family.
The `traits` field is optional, and can include information on traits on the
group. For example, if linking a user to a company, you may include traits on
the company such as the company size or location.
#### Example group event payload
```json
{
"type": "group",
"messageId": "d87596da-5f1e-40f4-a28a-7099290548b7",
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"groupId": "456",
"traits": { "company_name": "Hightouch" },
"originalTimestamp": "2023-09-22T18:46:23.098Z",
"sentAt": "2023-09-22T18:46:23.099Z"
}
```
### Automatically captured fields
Hightouch SDKs automatically populate common context fields about the device, environment, and session. The table below shows which fields are captured by each SDK.
| CONTEXT FIELD | Javascript | iOS | Android | Description |
| --- | --- | --- | --- | --- |
| app.name | | ✅ | ✅ | Name of the application. |
| app.version | | ✅ | ✅ | Version of the application. |
| app.build | | ✅ | ✅ | Build of the application. |
| campaign.name | ✅ | | | Name of the campaign. |
| campaign.source | ✅ | | | Source of the campaign. |
| campaign.medium | ✅ | | | Medium of the campaign. |
| campaign.term | ✅ | | | Term of the campaign. |
| campaign.content | ✅ | | | Content of the campaign. |
| device.type | | ✅ | ✅ | Type of the device. |
| device.id | | ✅ | ✅ | ID of the device. |
| device.advertisingId | | ✅ | ✅ | Advertising ID of the device. |
| device.adTrackingEnabled | | ✅ | ✅ | If ad tracking is enabled on the device. |
| device.manufacturer | | ✅ | ✅ | Manufacturer of the device. |
| device.model | | ✅ | ✅ | Model of the device. |
| device.name | | ✅ | ✅ | Name of the device. |
| library.name | ✅ | ✅ | ✅ | Name of the library. |
| library.version | ✅ | ✅ | ✅ | Version of the library. |
| ip | ✅ | ✅ | ✅ | Current user’s IP address. |
| locale | ✅ | ✅ | ✅ | Locale string of the user. |
| network.bluetooth | | | ✅ | Bluetooth information. |
| network.carrier | | ✅ | ✅ | Carrier information about the network connection. |
| network.cellular | | ✅ | ✅ | Cellular information about the network connection. |
| network.wifi | | ✅ | ✅ | WiFi information about the network connection. |
| os.name | | ✅ | ✅ | Name of the operating system. |
| os.version | | ✅ | ✅ | Version of the operating system. |
| page.path | ✅ | | | Path of the current page in the browser. |
| page.referrer | ✅ | | | Referrer of the current page in the browser. |
| page.search | ✅ | | | Search of the current page in the browser. |
| page.title | ✅ | | | Title of the current page in the browser. |
| page.url | ✅ | | | URL of the current page in the browser. |
| screen.density | | | ✅ | Density of the device’s screen. |
| screen.height | | ✅ | ✅ | Height of the device’s screen. |
| screen.width | | ✅ | ✅ | Width of the device’s screen. |
| traits | | ✅ | ✅ | Traits of the user. |
| userAgent | ✅ | | ✅ | User agent of the device making the request. |
| userAgentData | ✅ | | | Information about the user’s timezone. |
| timezone | | ✅ | ✅ | Information about the user’s timezone. |
- IP Addresses are captured by Hightouch's servers when a message is received for client-side events and is not captured by Hightouch's libraries.
- userAgentData is only collected if the Client Hints API is available on the browser.
---
## Event streaming
**URL:** https://hightouch.com/docs/events/event-streaming
**Description:** Stream events from Hightouch to downstream tools in real time for activation, personalization, and time-sensitive workflows.
**Section:** Hightouch Events
| Audience | How you’ll use this article |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| **Marketing teams** | Understand the end-to-end event streaming workflow and how event data powers real-time activation, personalization, and experimentation. |
| **Data teams** | Learn how streamed events are filtered, mapped, validated, and delivered to downstream tools alongside warehouse-based workflows. |
| **Engineering teams** | Follow the recommended implementation path, from source setup and validation to configuring event streaming syncs and monitoring delivery. |
## What you’ll learn
By the end of this article, you’ll be able to:
1. [Decide when to use event streaming](#when-to-use-event-streaming)
2. [Understand how event streaming works](#how-event-streaming-works)
3. [Set up an event streaming sync](#set-up-event-streaming)
4. [Configure event types and filters](#configure-streamed-events)
5. [Map event and user data to a destination](#field-mappings)
6. [Monitor delivery and troubleshoot errors](#errors)
7. [Set up alerting](#alerting)
---
## Overview
**Event streaming** sends events collected by Hightouch directly to destinations, without going through the warehouse. Events arrive **in seconds** — useful when timing matters, like triggering a message, updating consent, or powering real-time personalization.
Event streaming is a type of [**Event
Sync**](/syncs/types-and-modes#event-syncs) that takes events from an Event
Source and syncs them to an Event Destination in real-time.
---
## When to use event streaming
Use event streaming when your workflow depends on **something that just happened**.
Examples:
- **Post-purchase messaging** — send an upsell or confirmation immediately after checkout
- **Real-time ad spend control** — pause ads as soon as they exceed their budget
- **Lifecycle triggers** — follow up right after a demo request, new sign-up, or missed call
- **Consent and contact updates** — sync a user's consent or contact status the moment it changes
### When to use reverse ETL instead
Reverse ETL is a better fit when timing is less critical and **context matters more than immediacy**.
Use reverse ETL when you need:
- **Warehouse-enriched or aggregated data**
(joins across tables, LTV, days since last purchase, ML scores)
- **Audiences built from historical behavior**
(for example, users who purchased 3+ times in the last 90 days)
- **Ongoing monitoring and governance at scale**
(shared models and metrics used across many workflows)
For example, reverse ETL works better for:
- Conversion events that benefit from complete identity and revenue context
- Audience membership that should change over time, not from a single event
- Features or scores computed from many events (churn risk, power users, high-value cohorts)
→ Learn more about [Reverse ETL syncs](/syncs/types-and-modes).
---
## How event streaming works
With event streaming, events flow through Hightouch and are forwarded to destinations in near real time.
At a high level, each streamed event follows this path:
1. The event is received from an **event source** (SDK, HTTP API, streaming source, or webhook).
2. (Optional) **Contracts** validate the event’s structure, naming, and required fields.
3. (Optional) An **Event Function** transforms or enriches the payload.
4. **Filters** decide whether the event should be forwarded for a given event type.
5. **Field mappings** shape event and user data to match the destination schema.
6. The event is delivered to the **destination**, with automatic retries on failures.
This flow happens **per event**, rather than on a schedule. For warehouse-based, batch processing, use event storage and reverse ETL instead of event streaming.
---
### Optional validation and transformation
You can optionally add additional controls to this flow:
- **Data contracts**
Validate event structure, naming, and required fields before events are delivered.
→ See [Managing contracts](/events/contracts/management)
- **Event functions**
Modify or enrich events in-flight before they are sent to the destination.
→ See [Functions overview](/events/functions/overview)
These steps are optional and are commonly introduced as your event implementation grows.
---
## Set up event streaming
### Prerequisites
Before you begin, make sure you have:
- An **event source** that’s already set up and receiving events
- A **destination** where you want to forward those events
---
### Step 1: Create a new event sync
1. Go to **`Event Collection → Event Syncs`**
2. Click **Add sync**
---
### Step 2: Select an event source
Choose the source that emits the events you want to stream.
If you don’t see the expected source, confirm that events are being received:
1. Go to **`Event Collection → Event Sources`**
2. Select the source
3. Open the **Debugger** tab
4. Trigger the event in your app or site and verify it appears in the debugger
---
### Step 3: Select an event destination
Event syncs can either **store events** or **forward events**, depending on your use case:
- **Event storage** destinations write events to your warehouse
- **Event forwarding** destinations send events to downstream tools in real time
Select a destination under **Event forwarding**.
Use the **Docs** link in the UI to review destination-specific requirements.
---
### Step 4: Name and create the sync
Enter a clear, descriptive name for the sync, then create it.
Once created, you’ll be taken to the sync’s detail page.
---
## Configure streamed events
Event configuration happens **inside the sync**, not during creation.
**Set in:**
`Event Collection → Event Syncs → [Select sync] → Configuration`
---
### Event types
Each destination supports one or more **standard event types** (for example, a purchase event) and **custom event types**.
You can configure **multiple custom event types** per realtime sync. This lets you forward different event categories (for example, `Product Viewed`, `Cart Updated`, and `Wishlist Added`) through a single sync, each with its own field mappings and configuration.
We recommend always configuring at least one **custom event type** so you don’t unintentionally drop events.
---
### Referencing fields
In [**event filters**](#event-filters) and [**field mappings**](#field-mappings), you can reference fields on the event payload to filter on and match to fields in the destination object. Keep in mind:
- Reference nested fields using dot notation, for example `properties.price` or `event`.
- If a destination expects a specific data type, use **Liquid templates** to cast or format values, for example `{{ row['properties'].price | cast : 'number'}}`.
- If a field name contains a period (`.`) or backslash (``), escape it with a backslash.
- You do not need to escape spaces.
- The `event` property refers to the event’s name for track events.
- The `type` property refers to the SDK and API event type (`track`, `identify`, `page`, `screen`, `group`).
Use the **Debugger** in your event source to inspect real payloads while designing filters and mappings.
---
### Event filters
Event filters determine which incoming events are sent as each event type.
For example, when configuring a purchase event, you might filter for only purchase-related events.
---
### Field mappings
Field mappings control **what data is sent to the destination** and **where it appears**.
Mappings are configured per event type and grouped by purpose in the UI.
**Set in:**
`Event Collection → Event Syncs → [Select sync] → Configuration → [Select event type]`
---
### Map required event fields
This section maps core event data to fields **required by the destination**.
Required fields (such as items, total, currency, or timestamp) must be mapped before the sync can run successfully. If required fields are missing or mis-typed, the destination may reject events.
---
### Sync additional user fields (optional)
This section lets you send **user-level data** along with each event, such as email or subscription status.
Use this when:
- The destination updates user profiles based on events
- You want events to enrich user data in real time
---
### Sync custom event fields (optional)
This section controls which **event-specific details** are included beyond required fields.
For custom events, we recommend sending the **entire `properties` object** so downstream tools receive full context.
---
### Choosing source values
When mapping fields or creating filters, you choose **where each value comes from**.
You can use:
- **Field** – Pull a value directly from the event payload (for example, `properties.total`)
- **Static value** – Send the same value for every event
- **Template** – Format or transform values
- **Create an array** – Combine one or more fields into an array
These options let you adapt your data to match what the destination expects without changing how events are tracked.
---
### (Advanced) Edit JSON mode
Within a sync's **`Configuration`** tab, you also have the option to modify the underlying sync configuration directly. This mode is intended for **advanced use cases** where the standard UI does not provide enough flexibility.
Use Edit JSON mode when you need to:
- Make bulk or highly repetitive field mappings
- Reference fields or structures not exposed in the UI
- Apply destination-specific configuration that can’t be expressed visually
Most users should configure event streaming using the event filters and field mappings in the UI.
→ See [**Referencing fields**](#referencing-guide).
---
## Errors
Each event sync includes built-in visibility and controls:
- **Overview**: event volume and delivery status
- **Errors**: failed and retried events
- **Alerting**: notifications for failures or latency issues
Failed events are retried automatically before being archived.
When troubleshooting, check:
- Filters (events may be excluded before mapping)
- Required mappings (missing required fields)
- Destination credentials and rate limits
- Any Event Functions attached to the sync
---
## Alerting
Alerting helps you proactively monitor event streaming health.
You can configure alerts to notify your team when:
- Event delivery fails or error rates increase
- Retry thresholds are exceeded
- Latency increases beyond acceptable limits
Alerts are optional but recommended for production workflows.
---
## Supported destinations
Any destination in the Hightouch catalog that supports **event forwarding** can be used with event streaming. Check each destination’s documentation for:
- Supported event types
- Required fields
- Rate limits and identity requirements
---
## First-party tracking
**URL:** https://hightouch.com/docs/events/first-party-tracking
**Description:** Send browser events through your own domain to improve client-side event coverage.
**Section:** Hightouch Events
First-party tracking sends browser events through your own domain instead of directly to Hightouch. This improves event coverage when browsers or privacy tools restrict third-party requests, such as Safari's Intelligent Tracking Prevention (ITP). You set it up by serving the SDK and collecting events from a domain you control.
First-party tracking has three parts. The first two are proxies you put in front of Hightouch; the third is a small service you host to issue cookies that survive browser privacy limits:
- **Proxy the JavaScript SDK** so the library loads from your domain instead of `cdn.hightouch-events.com`. See [Proxying the JavaScript SDK](#proxying-the-javascript-sdk).
- **Proxy events** so the SDK sends collected events to your domain instead of `hightouch-events.com`. See [Proxying events](#proxying-events).
- **Issue first-party, HTTP-only cookies** from a service on your domain so anonymous and user identifiers persist past Safari's cookie limits. See [Intelligent Tracking Prevention](#intelligent-tracking-prevention).
The first two parts improve delivery against ad blockers and third-party request blocking. The third keeps identity stable for returning visitors on browsers that enforce ITP. You can set them up independently, but most teams that need ITP mitigation do all three.
## Proxying
To use first-party tracking, set up a proxy so clients send their event data to your domain rather than directly to Hightouch.
### Prerequisites
You'll need a certificate for whatever domain you plan to use for the proxy.
This guide uses [Amazon Cloudfront](https://aws.amazon.com/cloudfront), but any CDN or reverse proxy will work.
### Proxying the JavaScript SDK
Out of the box, the JavaScript SDK is served from cdn.hightouch-events.com. But you can serve the JavaScript SDK from your own domain instead.
First, create a [Cloudfront distribution](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-web-creating-console.html) with the following settings:
| Setting | Value |
| ------------------------------ | -------------------------------------------------------------- |
| Origin domain | `cdn.hightouch-events.com` |
| Origin path | Leave this blank |
| Name | Give your distribution a unique name |
| Enable Origin Shield | `No` |
| Compress objects automatically | `Yes` |
| Viewer protocol policy | `HTTPS only` |
| Allowed HTTP methods | `GET, HEAD` |
| Restrict viewer access | `No` |
| Cache key and origin requests | `Cache policy and origin request policy` |
| Cache policy | `CachingOptimized` |
| Origin request policy | `AllViewerExceptHostHeader` |
| Response headers policy | None |
| Function associations | No associations |
| Alternate domain name (CNAME) | The domain you want to expose to clients |
| Custom SSL certificate | Select a certificate that corresponds to the CNAME you entered |
| Supported HTTP versions | Enable `HTTP/2` and `HTTP/3` |
| Default root object | Leave this blank |
| Standard logging | `Off` |
| IPv6 | `On` |
Then, point your clients to the CNAME you configured. If using the JavaScript snippet, you can do this by replacing `cdn.hightouch-events.com` in the body of the snippet with your CNAME.
### Proxying events
By default, events are sent to `hightouch-events.com`. But you can configure Hightouch to send events to your own domain instead.
First, create a [Cloudfront distribution](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-web-creating-console.html) with the following settings:
| Setting | Value |
| ------------------------------ | -------------------------------------------------------------- |
| Origin domain | `us-east-1.hightouch-events.com` |
| Origin path | Leave this blank |
| Name | Give your distribution a unique name |
| Enable Origin Shield | `No` |
| Compress objects automatically | `Yes` |
| Viewer protocol policy | `HTTPS only` |
| Allowed HTTP methods | `GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE` |
| Restrict viewer access | `No` |
| Cache key and origin requests | `Cache policy and origin request policy` |
| Cache policy | `CachingDisabled` |
| Origin request policy | `AllViewer` |
| Response headers policy | None |
| Function associations | No associations |
| Alternate domain name (CNAME) | The domain you want to expose to clients |
| Custom SSL certificate | Select a certificate that corresponds to the CNAME you entered |
| Supported HTTP versions | Enable `HTTP/2` and `HTTP/3` |
| Default root object | Leave this blank |
| Standard logging | `Off` |
| IPv6 | `On` |
Then, set your SDK's `apiHost` setting to the CNAME you configured. For example, if you are using the Browser SDK:
```js
HtEventsBrowser.load(
{ writeKey: "WRITE_KEY" },
{ apiHost: "events.example.com" }
);
```
---
## Intelligent Tracking Prevention
_Intelligent Tracking Prevention_ (ITP) is an automatic feature of the Safari web browser, designed to limit user tracking. More specifically, the program is designed to prevent analytics libraries from using client-side Javascript to correlate user sessions. ITP configures the Safari browser in the following ways:
1. Third-party cookies are blocked by default.
2. First-party cookies (and localStorage) have a default expiry of 7 days.
3. A user revisiting a website will restart the 7 day expiry.
Unfortunately, most analytics libraries persist session information by:
1. Storing an identifier in a Cookie.
2. Storing an identifier in localStorage.
A legacy analytics installation, one not working to mitigate ITP, will collect less accurate session data for Safari users. For example, a single user visiting a website on both 01/01/2023 and 01/15/2023 will look like two different users. After Safari deletes the first cookie on 01/08/2023, the web SDK will be forced to make a new cookie and a new identifier for the session occurring on 01/15/2023. Downstream, your warehouse will likely see these sessions as two different users. This simple inaccuracy may impact otherwise useful metrics generated from analytics tracking.
### Mitigating ITP
When properly configured, Hightouch Events can store session identifiers outside of ITP's 7 day expiry. It does this by using first party, HTTPOnly cookies. These are not subject to Safari's 7 day expiry. These types of cookies are generally used for logging into a website and are less likely to be limited, even when dealing with anonymous users.
These "server cookies" must follow several rules put in place by ITP:
1. The server providing the HTTPOnly cookie must be on the same domain as the website.
2. If the server is on a subdomain of the website, its IP address must match the IP address that served the main HTML document.
In practice, this means that ITP mitigation requires customers to host some sort of web service responsible for returning HTTPOnly cookies to the browser. The service does not need to be complex; Hightouch has numerous example servers. The most important part of this **HTTPCookieService** is that it must be a first party service: routing a subdomain via DNS will not suffice.
In order for the HTTPCookieService to serve first party, HTTPOnly cookies, you'll need **_one of_** the following:
A **webserver** that handles both your HTML documents and your API. This would mean building the HTTPCookieService functionality into your regular webserver. As an example, you might already have a Java Spring server for handling regular website requests.
A **reverse proxy** that can forward path-dependent requests to different servers. This requires proxying your domain to both an HTTPCookieService and your regular webserver. As an example, you might already have something like NGINX that proxies HTML requests to one place and API requests to another.
A **CDN** that can run programmatic logic when matching certain path-dependent requests. If you already have a CDN, you may be able to add custom handlers without standing up new servers. This can be achieved with a variety of technologies like Amazon's Cloudfront, Lambda@Edge, or API Gateway. Cloudflare Workers can also work.
### Client SDK setup
Once your HTTPCookieService is reachable on your domain, you may configure your Hightouch Events SDK to make use of it when generating Cookies:
```Javascript
const htevents = HtEventsBrowser.load(
{ writeKey: ''},
{
apiHost: "us-east-1.hightouch-events.com", // HtEvents API remains the same
httpCookieServiceOptions: {
clearUrl: '/ht/clear', // route hosted on *your* domain and infra
renewUrl: '/ht/renew', // route hosted on *your* domain and infra
}
},
)
htevents.identify('hello world')
document.body?.addEventListener('click', () => {
htevents.track('documentBodyClick')
})
```
### Server setup
The Events SDK expects to interact with a customer's HTTPCookieService that implements a specific spec for two routes. You can name the endpoints whatever you want, as long as you configure the client SDK to call those endpoints.
### An API for creating server and browser cookies
This route should look for the following **browser** cookies (from Events SDK):
- `request.headers.get("Cookie")["htjs_anonymous_id"]`
- `request.headers.get("Cookie")["htjs_user_id"]`
This route should return these values as **server** cookies:
- `response.cookie("htjs_anonymous_id_srvr", anonVal, {httpOnly:true, ...})`
- `response.cookie("htjs_user_id_srvr", userIdVal, {httpOnly:true, ...})`
If there are no browser cookies found, return any server cookies as **browser** cookies:
- `anonVal = request.headers.get("Cookie")["htjs_anonymous_id_srvr"]`
- `userIdVal = request.headers.get("Cookie")["htjs_user_id_srvr"]`
- `response.cookie("htjs_anonymous_id", anonVal, ...)`
- `response.cookie("htjs_user_id", userIdVal, ...)`
### An API for clearing server cookies
This route should look for **server** cookies and clean them:
- `res.cookie("htjs_anonymous_id_srvr", "", {maxAge: 0, httpOnly:true, ...});`
- `res.cookie("htjs_user_id_srvr", "", {maxAge: 0, httpOnly:true, ...});`
### API spec
The spec of the actual `request` and `response` payloads are kept intentionally vague. The spec should fit a variety of server environments.
The Events SDK only requires that the server: A) handles cookies and B) returns a `200` status code.
### Server examples
- [Express.js and NGINX](https://github.com/ht-sdks/events-sdk-js-mono/blob/master/packages/browser/src/core/http-cookies/server-examples/node-express-js.md)
- [Next.js and Vercel](https://github.com/ht-sdks/events-sdk-js-mono/blob/master/packages/browser/src/core/http-cookies/server-examples/node-next-js.md)
- [AWS Lambda and API Gateway](https://github.com/ht-sdks/events-sdk-js-mono/blob/master/packages/browser/src/core/http-cookies/server-examples/node-aws-lambda.md)
### More information
- Safari: https://webkit.org/blog/9521/intelligent-tracking-prevention-2-3/
---
## Create a function
**URL:** https://hightouch.com/docs/events/functions/getting-started
**Description:** Create, test, and connect a function to transform events before they sync to a destination.
**Section:** Hightouch Events
Write JavaScript to modify event properties, validate fields, or enrich event data on a [Hightouch Events](/events/overview) sync. This guide walks through creating, testing, and connecting a function.
### Creating a function
To create a new function, go to **Event Collection → Functions** and
select **Create function**.
When you create a function, you’ll start with a default function definition,
which you can edit to add custom transformation logic.
Keep these requirements in mind:
- All code must have a `transformEvents` function definition, which receives a
batch of events, and returns a batch of transformed events
- Output events must conform to the core [Hightouch Event spec](/events/event-spec)
- Event batches can include multiple event types (so make sure to check
`event.type` if you need to handle event types differently)
- Batch sizes are variable based on sync volume and event size
- Functions time out after 5 seconds of execution time
- Events can be filtered out by omitting them from the function return
### Testing
Test the function works as expected by using the test panel in the function
code editor view.
Modify the input in the test panel, and select **Test**. Results will be
displayed in the output panel, along with any `console` logs. If the function
errors during execution, the error and stack trace will be displayed.
### Save & connect
To save a function, hit **Save changes** at the bottom of the code editor.
Upon save, if the function is connected to any event syncs, the new function
code will immediately be used.
To connect a function to a sync, use the **Connected syncs** tab, where you can
connect or disconnect a function from a sync. When a function is connected
to a sync, events in the target event sync will immediately begin to be transformed by
the function.
Note: a function can be connected to multiple syncs, but a sync can only have
one function connected to it at a time.
### Runtime and dependencies
Functions are executed in a Node.js 22 runtime environment.
Additional available modules:
- [Crypto](https://developer.mozilla.org/en-US/docs/Web/API/Crypto) is available as `crypto`
- [Fetch_API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) is available as `fetch`
### Debugging
Errors from functions will be displayed in the event sync error logs, along
with sample event payloads that received that error.
### Other notes
- Functions are executed after contract validation (if a contract is defined
for the respective event source). This means events blocked by contracts
are not able to be transformed using functions.
- Use `Promise.all` to run async code in parallel, reducing the chance of
hitting a timeout
- If a function errors, the entire batch is failed and will be retried
based on the sync retry configuration
---
## Handling webhooks
**URL:** https://hightouch.com/docs/events/functions/handling-webhooks
**Description:** Use functions to reshape webhook payloads before Hightouch writes them to your warehouse or sends them to destinations.
**Section:** Hightouch Events
Use functions to reshape webhook payloads before Hightouch writes them to your warehouse or sends them to destinations.
These webhooks usually contain event data that are helpful for building a full
profile of your users. For example, Shopify has webhooks for when users check
out, and Iterable has webhooks for when users open emails.
To set this up, create a [webhook source](/events/sources/webhook)
and add it to the tool that emits events. You can then process the raw webhook
data using functions.
For example, your code might look something like:
```js
async function transformEvents(events) {
return events.map((event) => ({
...event,
type: "track",
event: event.event,
messageId: event.messageId,
properties: {
...event.properties.body.dataFields,
email: event.properties.body.email,
},
anonymousId: event.properties.body.userId
? null
: event.properties.body.email,
userId: event.properties.body.userId,
}));
}
```
---
## Functions overview
**URL:** https://hightouch.com/docs/events/functions/overview
**Description:** Transform, validate, or enrich events with custom JavaScript before they sync to a destination.
**Section:** Hightouch Events
Functions let you transform events before they sync to a destination. Write JavaScript to modify event properties, validate fields, or enrich event data on any [Hightouch Events](/events/overview) sync.
### Features
- Modify, validate, enrich, or filter events with custom logic before they sync to destinations
- Write transformations in modern JavaScript
- Code, test, and connect functions to syncs within the Hightouch app
- Transform batches of events at once
### Use cases
- **Enrich event data.** Call external APIs to add data to your event payload. For example, add geolocation fields based on IP addresses.
- **Data cleaning and normalization.** Encrypt sensitive information, normalize complicated data types, or remove PII from event payloads.
- **Data validation.** Write advanced validation logic so your event data conforms to your spec.
- **General transformations.** Reshape an event before sending it to a destination.
### Getting started
Use functions to transform events in any event sync: create a function, test the code, and connect it to a sync. [Read the guide](/events/functions/getting-started) to get started.
---
## Plan your Events implementation
**URL:** https://hightouch.com/docs/events/implementation-plan
**Description:** Decide what to track, how to identify users, how strictly to validate events, how to handle coverage gaps, and where events should go.
**Section:** Hightouch Events
| | |
| ----------------- | ------------------------------------------------------------------------ |
| **Audience** | Data teams, analytics teams, engineers, and marketers who use event data |
| **Prerequisites** | Familiarity with [how Events works](/events/overview) |
_Plan these decisions before you instrument Hightouch Events. They are harder to change after events start flowing because your warehouse tables, contracts, and destinations depend on them._
---
## What you'll decide
- [Event model](#design-your-event-model) — what to track and how to name and structure events
- [Identity](#plan-identity) — how to recognize people and connect their activity over time
- [Contract strategy](#decide-your-contract-strategy) — what to validate and how strictly
- [Coverage](#plan-for-coverage) — how reliable client-side collection needs to be
- [Delivery](#choose-how-events-reach-destinations) — where events go and how each destination should receive them
---
## Design your event model
Decide which actions to track and how each event should be named and structured. These choices affect your warehouse tables, contracts, audiences, reports, and destinations.
- **Track what you'll use.** Tie each event to a real use case, such as an audience, journey, report, or destination workflow. Tracking everything "just in case" creates noise and tables no one uses.
- **Name events consistently.** Pick one convention, such as `Object Action` (`Order Completed`), and use it everywhere. Event names become warehouse table names, and contracts match them exactly. Inconsistent names can create duplicate tables or break validation.
- **Separate properties from traits.** Properties describe the action, such as cart total. Traits describe the person, such as email or plan type. Adding person attributes to every event can make tables harder to manage.
- **Decide required and optional properties.** Define which fields each event must include so contracts can validate them later.
- **Keep field types consistent.** A price should always be a number. A flag should always be a boolean. Hightouch infers each column's type from the values it receives, so mixed types can cause rejected or changed data.
Go deeper: [event types and payload structure](/events/event-spec), [warehouse schema](/events/warehouses/schema).
---
## Plan identity
Decide how to recognize people before and after they are known, and how their activity should connect to one profile.
- **Choose a stable `userId`.** Use an identifier that does not change over a person's lifetime, such as your internal database ID. Avoid email addresses, phone numbers, and session IDs because they can change or rotate. Send email and phone as traits instead.
- **Connect anonymous and known activity.** A visitor starts with an `anonymousId`. Call `identify` when they sign in so their pre-login and post-login activity can be connected.
- **Decide how profiles unify.** Joining one person's anonymous and known activity can happen in a warehouse model. Unifying many identifiers across devices, sessions, and sources into one profile is the job of Identity Resolution. [Customer Studio](/customer-studio/overview) can then use that identity layer for traits and audiences.
- **If you're migrating, preserve identity behavior.** Recreate your current tool's identity priority logic with identifier rules so people resolve consistently while both systems run in parallel.
---
## Decide your contract strategy
Decide which event rules to enforce and how strict those rules should be. A [data contract](/events/contracts/management) validates the event name, field names, and field types exactly as written, including capitalization and whitespace.
- **Choose an enforcement level for each violation type.** Hightouch can allow the event and flag the violation, block the event, or filter out the invalid fields. The default is to allow.
- **Start permissive, then tighten.** Use a permissive setup first so you can see violations without dropping data. Tighten enforcement once event names and field types are stable.
- **Prioritize important events.** Add contracts for your core, high-value events before covering edge cases.
Go deeper: [managing contracts](/events/contracts/management), [handling violations](/events/contracts/violations).
---
## Plan for coverage
Decide how complete your client-side event collection needs to be and which events need a more reliable path. Installing the SDK does not guarantee that every browser event will be captured.
- **Expect some client-side loss.** Browser events can be blocked by ad blockers, privacy settings such as Safari ITP, and cookie limits. This is part of client-side tracking, not necessarily a setup issue.
- **Use first-party and server-side collection where needed.** Proxy SDK traffic through your own domain with [first-party tracking](/events/first-party-tracking). Send business-critical events, such as purchases and signups, through [server-side collection](/events/sdks/http) so they do not depend on the browser.
- **Choose which events must be reliable.** Route those server-side from the start.
Go deeper: [first-party tracking](/events/first-party-tracking).
---
## Choose how events reach destinations
Decide where each event should go and how each destination should receive it.
- **Warehouse, streaming, or both.** Write events to your warehouse for analytics, modeling, audience building, and long-term reuse. Stream events directly to a tool only when the workflow is time-sensitive, such as triggering a message as soon as an action happens. Many teams warehouse all events and stream only the events that need to be immediate.
- **Event-shaped or profile-shaped data.** A destination can receive data as events, which represent actions over time, or as profile properties, which update attributes on a person. Choose based on what the destination should do with the data. Sending events when you meant to update profiles, or the reverse, is a common setup mistake.
Go deeper: [write events to your warehouse](/events/warehouses/warehouse), [event streaming](/events/event-streaming).
---
## Build your implementation
Once you've made the decisions above, follow these steps to set up Events. If you're migrating from another tool, follow the [Migration guide](/events/migration-guide/overview) instead — it adds the work of running both systems in parallel, proving your data matches, and cutting over without downtime.
1. **Connect your sources.** Instrument your app, site, or backend to send events to Hightouch using an SDK, the HTTP API, or a streaming source. See [Collect events](/events/sources/overview).
2. **Send events to your warehouse.** Configure a warehouse destination so events land in a consistent schema you can query and model. See [Send events to your warehouse](/events/warehouses/warehouse).
3. **Govern your events with contracts.** Define [data contracts](/events/contracts/management) for your core events so bad data is caught before it reaches your warehouse. Start permissive and tighten as your event names and types stabilize.
4. **Validate the data.** For a fresh build, confirm events arrive as expected: inspect live events in the [debugger](/events/sources/overview#debugger) and the [event catalog](/events/sources/overview#event-catalog), and check that your warehouse tables and columns match what you sent (see [Warehouse schema](/events/warehouses/schema)). If you're migrating, validate parity against your previous tool instead — see [Validate your migration](/events/migration-guide/validating-your-migration).
5. **Set up monitoring.** Add volume, violation, and sync alerts so you catch issues early. See [Monitor events](/events/monitoring).
Depending on your use cases, you may also:
- **Stream events to destinations** for time-sensitive workflows, alongside or instead of the warehouse. See [Stream events to destinations](/events/event-streaming).
- **Transform events** in flight to rename properties, filter, or enrich them. See [Transform events with functions](/events/functions/overview).
- **Improve coverage** for browser events by [proxying through your own domain](/events/first-party-tracking) and moving critical events to [server-side collection](/events/sdks/http).
## What's next
- **Migrating** from another tool? Follow the [Migration guide](/events/migration-guide/overview).
- Once events are flowing, [build audiences in Customer Studio](/customer-studio/overview) from your event data.
---
## Step 2: Implementing Hightouch Events SDKs
**URL:** https://hightouch.com/docs/events/migration-guide/implementing-hightouch-events
**Description:** How to switch to Hightouch SDKs, including a wrapper function that sends events to both tools so you can validate during the migration.
**Section:** Hightouch Events
Hightouch Events provides SDKs across web, mobile, and server-side languages. This step covers switching from your current event tracking SDK (such as Segment's or RudderStack's) to Hightouch Events.
Hightouch Events is backward compatible with common tracking APIs like those of Segment and RudderStack, so the initial code changes are usually straightforward. If you're coming from a platform with a different SDK shape, such as Amplitude or mParticle, expect more re-instrumentation — see its platform guide for what differs.
To compare data side by side during the migration, implement a wrapper
function that sends each event to both your current platform and Hightouch.
You'll see this in practice in the JavaScript example below.
## Install the SDKs
Install the Hightouch SDK for each source you're migrating. Installation is the same as for a fresh implementation — see [Connect event sources](/events/sources/overview) and the SDK reference for your platform, such as [Browser](/events/sdks/browser#installation), [Node.js](/events/sdks/nodejs), or [Python](/events/sdks/python). Use the write key from your Hightouch event source.
Because the SDKs accept the same calls as your current tool, the migration-specific work isn't the install — it's sending each call to both tools so you can compare data. That's what the rest of this step covers.
## Implement standard tracking calls (track, page, and identify)
During this step, you'll update your event method calls to also call the matching Hightouch SDK method.
The example below shows a wrapper function that calls the `track` method of your prior tool and Hightouch Events. Both calls take the same parameters and include a `migrationId` that lets you validate data during the migration.
```jsx
function track(eventName, properties) {
const context = { migrationId: getMigrationId() };
analytics.track(eventName, properties, context);
htevents.track(eventName, properties, context);
}
function getMigrationId() {
const timestamp = Date.now();
const randomness = Math.floor(Math.random() * 1000);
return `${timestamp}-${randomness}`;
}
```
This wrapper function does the following:
- Generates a unique `migrationId` for each event
- Sends the event to your current tracking system
- Sends the same event to Hightouch Events
- Includes the `migrationId` with both events for later comparison and deduplication
Attach the `migrationId` in the event's `context`, as the examples here do, so it lands consistently across event types. In the warehouse, Hightouch flattens `context` fields to `context_` columns and snake-cases the key, so `migrationId` becomes the **`context_migration_id`** column. The validation and unification queries in later steps use `context_migration_id` for this reason. (If your event storage destination isn't configured to explode `context` into columns, the value stays nested under the `context` JSON instead.)
## Handle identify and page view events
Create similar wrapper functions for other common event types based on their parameters and type signatures. For example, for an `identify` method in the Browser SDK,
```jsx
function identifyUser(userId, traits) {
const context = { migrationId: getMigrationId() };
analytics.identify(userId, traits, context);
htevents.identify(userId, traits, context);
}
```
## Migrate server-side events
Follow a similar pattern for server-side event tracking. Below, we show an example of using the [Python SDK](/events/sdks/python) with a wrapper function.
```python
analytics.write_key('YOUR_SEGMENT_WRITE_KEY')
htevents.write_key('YOUR_HIGHTOUCH_WRITE_KEY')
def track_event(user_id, event_name, properties):
context = {'migrationId': str(uuid.uuid4())}
# Track with Segment
analytics.track(user_id, event_name, properties, context=context)
# Track with Hightouch
htevents.track(user_id, event_name, properties, context=context)
```
## Update analytics code across your repositories
If your codebase historically directly calls Segment's (or another provider's) library to trigger events, use your preferred method for making bulk code changes across your repository or repositories, whether it's using `grep` / `find` and `sed`, find and replace across a workspace in your editor, or other tools to replace track, identify, group, page, and screen calls with the wrapper functions.
Make sure that you don't just update the analytics code with the wrapper function(s), but also update library imports across files that call the new wrapper function.
If you've already implemented wrapper functions and have modified them using the preceding migration examples, you should be all set.
## Testing the implementation
After implementing the wrapper functions:
1. Test thoroughly in a development or staging environment.
2. Verify that events are being received by both your current system and Hightouch Events.
3. Check that the `migrationId` is correctly included in events from both systems.
## Anonymous IDs and automatically collected information
Hightouch Events’ Browser, iOS, and Android SDKs automatically collect a range of information. Read [the docs](/events/event-spec#automatically-captured-fields) for the full list of fields.
Anonymous-ID continuity depends on the SDK. On **iOS**, Hightouch Events reuses the anonymous ID that Segment or RudderStack already assigned, so returning users keep their session instead of starting fresh. On **Android**, the SDK stores the anonymous ID under its own keys and doesn't read the previous provider's value, so those users get a new anonymous ID at cutover. Rely on `userId` continuity to connect known users across the switch on every platform.
## Optional: Gradual rollout
Depending on the number of your event sources, your event volume, and whether any of them power critical downstream services, you can consider implementing a gradual rollout strategy. This can help build the migration team's confidence in Hightouch Events and make the migration process more manageable by constraining event volume and sources in the first migration efforts.
1. Start using Hightouch Events with a small percentage of your user base or a specific segment. Some organizations migrate a portion of traffic to start with, while others migrate sources related to a particular platform (such as mobile apps) before migrating additional platforms.
2. Monitor for any issues or discrepancies.
3. Gradually increase the percentage of users or sources on the new system.
By following these steps, you'll have successfully implemented the Hightouch SDK alongside your existing event tracking system. This dual-tracking approach allows for thorough validation and comparison in the next phases of the migration process.
As you instrument, set up [data contracts](/events/contracts/management) for your core events to catch bad data before it reaches your warehouse. Version your contract events so older app versions keep validating during the rollout, and start permissive — allow violations with a warning, then tighten as your data stabilizes.
In the next step, you'll [validate your migration](validating-your-migration) to confirm that Hightouch Events captures data correctly and consistently with your previous system.
---
## How to migrate to Hightouch Events
**URL:** https://hightouch.com/docs/events/migration-guide/overview
**Description:** Learn how to migrate to Hightouch Events from another event collection solution.
**Section:** Hightouch Events
This guide walks through migrating from your current event tracking tool, such as Segment or RudderStack, to Hightouch Events. It covers how to plan the migration, run both systems in parallel, validate the data, and cut over your destinations — keeping your existing workflows running throughout. The steps below follow that order.
If you're coming from a specific platform, read the platform guide alongside this roadmap for the details that differ: Segment, RudderStack, mParticle, or Amplitude.
Before you start, make the foundational decisions in [Plan your Events implementation](/events/implementation-plan) — your event model, identity, contract strategy, coverage, and delivery. Those apply to any implementation; this guide covers what's specific to a migration. For how Events works and where your data lives, see the [Events overview](/events/overview).
## What you'll learn
Follow these steps:
1. **[Preparing for migration](preparing-for-migration)**: Scope your migration, decide on your data model and tracking plan, and estimate the time and resources required.
2. **[Implementing Hightouch Events SDKs](implementing-hightouch-events)**: Switch to Hightouch SDKs with an analytics wrapper function to support validation.
3. **[Validating your migration](validating-your-migration)**: Verify your setup and ensure data quality matches or exceeds your previous platform.
4. **[Unifying historical and new data](unifying-data)**: Deduplicate and unify your historical data with new events collected by Hightouch.
5. **[Updating downstream applications and destinations](updating-downstream-applications)**: Configure or set up the applications that consume your event data.
You'll also set up [data contracts](/events/contracts/management) to keep your event data consistent as it flows in, and [monitoring](/events/monitoring) to catch volume drops, violations, and sync errors. Define contracts as you instrument your events, and turn on monitoring before you start shifting traffic.
Each step keeps your existing data flowing while you move collection to Hightouch Events, so you can validate the new setup before you turn off your current tool.
---
## Step 1: Preparing for migration
**URL:** https://hightouch.com/docs/events/migration-guide/preparing-for-migration
**Description:** Scope your migration, plan your data model and tracking strategy, work through the data readiness checklist, and estimate the resources needed to migrate to Hightouch Events.
**Section:** Hightouch Events
Make the foundational decisions in [Plan your Events
implementation](/events/implementation-plan) first — event model, identity,
contracts, coverage, and delivery apply to any implementation. This step
covers what's specific to a migration: scope, parallel-run setup, and a
readiness checklist.
Before you migrate, plan the scope, owners, timeline, and validation approach. Most migrations involve:
1. **Form a migration team and communication plan**: Gather a cross-functional team, including data and analytics engineers, analysts, and stakeholders that will carry out the migration and verify data quality. Ensure that you inform affected teams of upcoming changes and provide a channel for concerns or questions.
2. **Determine the scope of your migration**: Decide whether you're modifying your data model during the migration or keeping it the same, whether you're merging Hightouch Event data with historical data, and whether you want to replace existing event tables or create new ones.
3. **Map downstream destinations**: Document any data destinations, dashboards, reports, or other consumers of event data that may need to be updated or validated as part of the migration.
4. **Establish a timeline**: Set realistic deadlines for each migration phase based on available personnel.
## Determine your migration scope
### Data model and tracking plan
Decide on your approach to migration. There are two main options:
1. **Simple migration**: Keep your existing data model and event structure as much as possible. This makes validation and updating downstream applications and destinations more straightforward, and may require little or no data wrangling to fit new event data into the existing data model.
2. **Complex migration**: Address issues or concerns in your data model during the migration process. Changes during a migration process could also include re-instrumenting events for a new tracking plan or integrating new tooling.
While these changes can have positive impacts on long-term data quality, each change increases the complexity of the migration and should be evaluated for return on the increased implementation time.
For most organizations, **start with a simple migration**. Minimizing changes increases the likelihood of timely completion and cuts down on the roadblocks that can develop through substantial changes in the data model.
If data quality is a significant issue, a migration to Hightouch Events can be a good occasion to build for long-term data quality and usability.
### Historical data handling
During planning, you should decide where you want the new event data to appear in your warehouse and whether you intend to union the historical and new data.
Regardless of your choice on unioning, we recommend writing the new Hightouch Event data into a separate schema in your warehouse. This makes it easier to isolate the events while you test and validate your migration, comparing them to historical data. We'll discuss where and how to unify historical and new data in [Step 5](unifying-data) of the guide.
### Map sources and destinations
During planning, identify:
- **Sources**: All the applications you collect events from that you want to migrate to Hightouch Events.
- **Destinations**: All the places that should receive or consume event data, including your warehouse, BI and analysis tools, and marketing and advertising platforms.
While most organizations send their events into their warehouse, then reverse ETL (rETL) data to other tools, Hightouch Events also supports [streaming events](/events/event-streaming) directly to other destinations.
It's worthwhile to create a table or other structure to map each source to its destinations and downstream consumers along with the relevant transfer method (rETL or streaming).
Once your team has decided on any changes to the data model and mapped data pipelines from sources to destinations, you can plan for any updates that will be needed for each downstream consumer or tool.
## Data readiness checklist
Use this checklist to confirm your data is ready before you start implementing. Each area is a planning input for a later step in this guide—the goal here is to have decisions and owners in place, not to do the work yet.
| Area | What should be ready | Owners |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------- |
| Migration scope | A decision on a [simple vs. complex migration](#data-model-and-tracking-plan), including whether the tracking plan and event model stay the same | Data lead, analytics lead, product owner |
| Source inventory | A complete list of web, mobile, server, and other [sources](#map-sources-and-destinations) to migrate, each with an owner | Engineering, analytics |
| Warehouse landing | A [separate schema](#historical-data-handling) for new Hightouch data during validation, with access for the people who will validate it | Data engineering |
| Event schema mapping | A field-level map of event names, properties, and types from your current provider to Hightouch | Analytics engineering |
| Identity plan | Agreement on your canonical identifier and anonymous ID behavior. | Data engineering, product analytics |
| Data contracts | Initial [contracts](/events/contracts/management) for core events, with a planned enforcement level | Engineering, analytics |
| Validation plan | Queries, sample events, a comparison window, and acceptable mismatch reasons. See [Step 3](validating-your-migration). | Analytics, data engineering |
| Historical data | A decision on lookback window, target tables, transformations, and deduplication. See [Step 4](unifying-data). | Data engineering |
| Downstream consumers | A ranked inventory of destinations, dashboards, and reports that depend on event data. See [Step 5](updating-downstream-applications). | Analytics, marketing ops, business owners |
| Monitoring and cutover | Alert thresholds, a cutover date, and a rollback plan. See [Monitor events](/events/monitoring). | Engineering, analytics |
If you're migrating from a specific platform, the Segment, RudderStack, mParticle, and Amplitude guides note what differs for each of these areas.
You're ready to move from planning to implementation when your scope is documented, every source and downstream consumer is inventoried, your landing schema and key event mappings exist, your identity and contract expectations are set, and your validation, historical, and cutover plans are written. At that point, implementation becomes execution instead of discovery.
## Estimate the time to migrate
Once you've broken the migration into the phases on the roadmap, you can estimate effort based on personnel, number of events, and number of sources and destinations.
Our estimates below—and designated personnel—are based on our experience
helping customers migrate to Hightouch Events. Every organization is
different, though, and migration time can vary significantly based on
organizational and technical factors.
For organizations with 10 to 30 event sources, we often see 4 to 6 calendar weeks for a migration. Organizations with 70 to 100 sources can migrate over 8 to 12 weeks.
1. **Planning and preparation**: 1 week effort for a **data engineer** and **data analyst** for research, planning, and documenting the plan, though this scales with changes to the data model and tracking plan.
2. **SDK implementation**: 1 week (or less) for an engineer to implement, test, and deploy new SDK code and initial data contracts.
3. **Configuring destinations**: 1 week (or less) for a data analyst to configure, test, and do initial monitoring.
4. **Validating and unioning data**: 1 week for a data analyst, though this scales with number and complexity of events and destinations and how you're unioning historical data.
5. **Updating downstream applications and deprecating your prior event collection tool**: 1 week or less for an engineer and a data analyst to update downstream data consumers, replace analytics code, re-validate data, and sunset your prior event collection tool.
Remember, these are rough estimates. Your actual timeline may vary based on the complexity of your current setup, the scope of changes you're implementing, and the number of dedicated personnel you have working on the migration.
Preparing and planning sets up the technical phases that follow. In the next step, you'll [switch SDKs](implementing-hightouch-events) to start collecting data with Hightouch Events.
---
## Step 4: Unifying historical and new data
**URL:** https://hightouch.com/docs/events/migration-guide/unifying-data
**Description:** Deduplicate and combine your historical data with new events collected by Hightouch so analytics stay continuous.
**Section:** Hightouch Events
After Hightouch Events is collecting data, decide how to combine new events with historical data from your previous provider. Most migrations need this step so dashboards, models, and destinations keep a continuous event history. (If you're rebuilding your data model or want a fresh start, you can skip it.)
## Plan your unification strategy
Before you begin, consider the following:
1. **Time frame**: Based on your use cases, decide how far back you need to unify data (for example, 1 year, 2 years, or all historical data). Your decided time frame affects processing time and storage requirements and ensures you have enough historical context for your analyses.
2. **Data volume**: Assess the volume of historical data you need to unify. This could impact processing time and storage costs and may influence your choice of unification method.
3. **Schema differences**: Identify any differences between your old and new event schemas if you haven't in a prior step. If there are changes, this will inform any necessary data transformations during the unification process.
4. **Storage considerations**: Determine where the unified data will be stored (for example, an existing or new table in your data warehouse). Your decision can affect query performance, how you update downstream destinations, and may have cost implications.
You have a few options for where the unified data lives: the previous provider's tables, the Hightouch tables, or a separate unified table or view. **During validation, keep Hightouch data in its own schema** so you can compare it against your historical data without modifying either set. For the long-term unified dataset, many teams use a separate unified table or view. In some warehouses, you may also choose to backfill historical rows into the Hightouch event tables after validation, but treat that as an implementation choice rather than the default recommendation.
## Union historical and new data
To keep your analytics continuous for data consumers, union the historical Segment data with the new Hightouch Events data.
Your process will depend on where you've decided to store the unioned data and any needed data transformations. We've sketched in SQL here for inserting 2 years of historical data from a prior provider (in this case Segment) into the Hightouch tables with minimal transformation.
In this particular case, we're only going to insert events from Segment that don't have a `context_migration_id`, since any dual-tracked event already exists in the Hightouch tables. The `context_migration_id` column only exists when your event storage destination writes context fields as columns; if it stores context as JSON, extract the value from the JSON column instead.
```sql
-- Insert historical data from Segment to Hightouch identifies table
INSERT INTO HIGHTOUCH.identifies (
id, anonymous_id, user_id, timestamp, email, name, context_migration_id
)
SELECT
s.id, s.anonymous_id, s.user_id, s.timestamp, s.email, s.name, s.context_migration_id
FROM SEGMENT.identifies s
WHERE s.context_migration_id IS NULL
AND s.timestamp >= DATEADD(day, -730, CURRENT_DATE()); -- Adjust time frame as needed
```
Modify this for table names and to include any traits or other properties you need. Repeat similar INSERT statements for `tracks`, `pages`, and other relevant tables, mapping the columns appropriately.
If there are differences between Segment and Hightouch schemas, you may need to transform the data during insertion. If you are migrating from Segment, for example, the schemas are quite similar and should only require basic SQL transformations, if any.
The example above uses Segment table and column names to keep it concrete. The approach is the same for any source—substitute your provider's names. See the RudderStack and mParticle guides for platform-specific notes.
## Validate the unified dataset
We've previously validated that Hightouch Events is collecting an equivalent volume of events with matching values. For peace of mind, after unifying the de-duped data, you can perform some basic validation steps:
1. **Check event counts**: Ensure the total event count makes sense based on the historical data and new Hightouch events.
```sql
SELECT
DATE_TRUNC('day', timestamp) as date,
COUNT(*) as event_count,
COUNT(DISTINCT user_id) as user_count
FROM HIGHTOUCH.tracks
GROUP BY 1
ORDER BY 1;
```
2. **Verify data continuity**: Look for any unexpected drops or spikes in event volume, especially around the migration period.
3. **Compare with Segment data**: Run a comparison query to ensure all historical data was properly migrated.
4. **Sample data**: Manually review a sample of events from different periods to ensure data format and content consistency.
By following these steps, you'll create a unified dataset in your Hightouch tables that combines your historical Segment data with new events from Hightouch Events. This approach ensures data continuity and provides a single source of truth for all your event data.
Remember to adjust the SQL queries as needed to match your specific schema and table names. Also, consider running these operations in batches if you're dealing with a large volume of historical data.
In the next section, you'll [update downstream applications and destinations](updating-downstream-applications) so each consumer reads from Hightouch Events.
---
## Step 5: Updating downstream applications and destinations
**URL:** https://hightouch.com/docs/events/migration-guide/updating-downstream-applications
**Description:** How to update the applications that consume your event data so each one reads from Hightouch Events after the migration.
**Section:** Hightouch Events
After validating and unifying your data, update the tools that consume event data so each one reads from Hightouch Events or your unified event tables. This is the final step of the migration.
It is possible to run this step in parallel with earlier validation to shorten the migration, but you may need to repeat some steps depending on any changes made during validation or unioning with historical data.
Before you cut over, make sure [monitoring and alerts](/events/monitoring) are in place so you catch volume drops, contract violations, or sync errors during the transition.
## Re-check your inventory of downstream applications
In your planning, you likely created an initial list of applications consuming event data. Take this opportunity to review and update that list in light of the migration so far.
Make sure your list includes any systems, tools, and processes that consume your event data, including:
- Analytics platforms (for example, Google Analytics, Amplitude)
- Marketing and advertising tools (for example, Meta Ads, Google Ads)
- Customer engagement platforms (for example, Braze, [Customer.io](http://customer.io/))
- Business intelligence tools (for example, Tableau, Looker)
- Custom internal dashboards and applications
Rank your downstream systems based on their criticality to your business operations to plan a phased approach to the updates if necessary.
## Update data sources
For each downstream system:
1. **Identify current data source**: Determine how the system is currently receiving event data (for example, reverse ETL, streaming, or direct SDKs).
2. **Update configurations or set up new syncs**: Depending on the downstream tool, you might:
- Create new or update existing Hightouch syncs
- Set up or modify streaming destinations in Hightouch
- Modify configurations in the downstream tool to point to the unified events data.
3. **Verify data flow**: Confirm that events are flowing correctly into the updated system.
## Adapt to schema changes
If your migration to Hightouch Events involved changes to your event schema:
1. **Document changes**: Create a clear mapping between your old and new event schemas.
2. **Update data models**: Modify any data models or transformations in your downstream systems to align with the new schema.
3. **Revise queries**: Update SQL queries, API calls, or other data retrieval methods to reflect the new event structure.
## Validate metrics and reports
For each updated system:
1. **Reconcile key metrics**: Compare critical metrics between the old and new data sources to ensure consistency.
2. **Update dashboards**: Revise any dashboards or standard reports to use the new data source and schema.
3. **Cross-check historical data**: Verify that historical data is accurately represented in updated reports.
## Handle data discrepancies
If you encounter differences between your old and new systems:
1. **Investigate root causes**: Determine if differences are due to improved data collection, schema changes, or potential issues.
2. **Document explanations**: For any persistent differences, document the reasons and communicate these to stakeholders.
3. **Adjust baselines**: If necessary, establish new baselines for key metrics based on the new data from Hightouch Events.
## Manage the transition period
If you're executing your migration in a linear fashion, you'll carry out this work as a whole after validating and confirming your new event data and cutting off your prior event collection tool.
However, if you're gradually shifting traffic over to Hightouch Events, you might decide to start transitioning downstream applications to use the new data at the same time and run in parallel with your prior tool and historical. If you do so, have a clear date planned for switching all downstream applications over to Hightouch events data and communicate changes to stakeholders as you transition.
## Gather feedback from users
As teams start working with the new event data, gather their feedback and refine your collection and activation setup. Early questions often surface schema or mapping gaps that are easier to fix before you turn off your previous provider.
## Turn off your previous provider
Once you're confident in your Hightouch Events setup, you can begin sunsetting your previous provider. We recommend the process below.
1. **Gradual cutover**: Incrementally increase the proportion of traffic sent only to Hightouch Events.
2. **Monitor closely**: Watch for any unexpected changes in data patterns or downstream system behavior.
3. **Final switch**: When ready, update your tracking code to send events only to Hightouch Events. With a wrapper function, this is a one-line change.
4. **Decommission old system**: Follow your previous provider's instructions for account closure and data handling.
This step completes your migration. Review your event data flow regularly afterward to keep data quality and downstream consumers in good shape as your tracking plan changes.
## What's next
Your collection now runs on Hightouch Events. Keep these running to maintain data quality and activate your events:
- **[Data contracts](/events/contracts/management)** — enforce your event schema and catch violations as your tracking plan evolves.
- **[Monitor events](/events/monitoring)** — keep volume, violation, and sync alerts running.
- **[Troubleshoot events](/events/troubleshooting)** — diagnose missing events, blocked events, or unexpected warehouse columns.
- **[Build audiences in Customer Studio](/customer-studio/overview)** — activate your event data downstream.
For the complete set of building blocks that make up an Events implementation, see [Plan your implementation](/events/implementation-plan#build-your-implementation).
---
## Step 3: Validating your migration
**URL:** https://hightouch.com/docs/events/migration-guide/validating-your-migration
**Description:** Compare Hightouch Events data with your previous event tracking platform before cutover.
**Section:** Hightouch Events
After Hightouch Events is running alongside your existing system, compare the two data streams before you cut over.
There are two main steps in validation:
1. Verify that event data is flowing from Hightouch Events with the correct properties and data types.
2. Check that the data from Hightouch Events is comparable to the data from your existing provider in volume and in values, such as user IDs and event properties.
Set up [monitoring and alerts](/events/monitoring) while both systems run, so you catch volume drops or contract violations early instead of discovering them at cutover.
## Verify your setup
1. **Check event reception.**
- Confirm that both your current system and Hightouch Events receive events.
- Use the [debugger](/events/sources/overview#debugger) to view incoming events in real time.
2. **Verify event structure.**
- Ensure all expected properties are present in the Hightouch events.
- Check that data types are correct (for example, numbers aren't being sent as strings).
3. **Test all event types.**
- Manually trigger each type of event (page views, user identifications, custom events) in your application.
- Verify that they appear correctly in both systems.
## Check data quality
Compare the data between your current platform and Hightouch Events to make sure that Hightouch Events is instrumented correctly and sending the data you expect. We recommend checking both event volume and values.
If you're maintaining your data model during your migration or only making minor changes, validating data values will be more straightforward. Significant changes to the data model during migration will make validation more complex and time-consuming.
### Compare event volume
Let's assume that you're migrating from Segment to Hightouch Events, and have data flowing from Segment into tables per Segment's schema and from Hightouch Events into tables per Hightouch's schema. We'll also assume events from during the migration carry the `migrationId` attached by the wrapper in Step 2, which Hightouch writes to the `context_migration_id` column.
The examples below use Segment table and column names to keep them concrete. The approach is identical for any source—if you're migrating from RudderStack or mParticle, substitute your provider's table and column names. See the RudderStack and mParticle guides for platform-specific notes.
We could use the following query—or something adjusted to your warehouse and setup—to look at the count of `identifies` events within the last 7 days. While we expect to see roughly the same count of events, there can be some variation—we'll cover why differences can occur later in this step of the guide.
```sql
WITH segment_identifies AS (
SELECT DATE(timestamp) AS event_date, context_migration_id, COUNT(*) AS segment_count
FROM SEGMENT.identifies
WHERE timestamp >= DATEADD(day, -7, CURRENT_DATE())
GROUP BY DATE(timestamp), context_migration_id
),
hightouch_identifies AS (
SELECT DATE(timestamp) AS event_date, context_migration_id, COUNT(*) AS hightouch_count
FROM HIGHTOUCH.identifies
WHERE timestamp >= DATEADD(day, -7, CURRENT_DATE())
GROUP BY DATE(timestamp), context_migration_id
)
SELECT
COALESCE(s.event_date, h.event_date) AS event_date,
s.context_migration_id,
s.segment_count,
h.hightouch_count,
s.segment_count - h.hightouch_count AS count_difference,
CASE
WHEN s.segment_count = h.hightouch_count THEN 'Match'
ELSE 'Mismatch'
END AS comparison_result
FROM segment_identifies s
FULL OUTER JOIN hightouch_identifies h ON s.event_date = h.event_date AND s.context_migration_id = h.context_migration_id
WHERE s.context_migration_id IS NOT NULL OR h.context_migration_id IS NOT NULL
ORDER BY event_date, context_migration_id;
```
### Compare property values
We also need to validate that Hightouch Events is collecting the same values as your prior provider.
The query below looks at a selection of properties relevant to `identifies` calls and compares between the two platforms over a 7-day period.
You can modify this query to examine different properties, use a different time window, or look at data in a narrower set of dates.
```sql
WITH segment_data AS (
SELECT
id AS segment_id,
context_migration_id,
anonymous_id,
user_id,
timestamp,
email,
name
FROM SEGMENT.identifies
WHERE timestamp >= DATEADD(day, -7, CURRENT_DATE())
),
hightouch_data AS (
SELECT
id AS hightouch_id,
context_migration_id,
anonymous_id,
user_id,
timestamp,
email,
name
FROM HIGHTOUCH.identifies
WHERE timestamp >= DATEADD(day, -7, CURRENT_DATE())
)
SELECT
s.context_migration_id,
COALESCE(s.user_id, h.user_id) AS user_id,
s.segment_id,
h.hightouch_id,
s.anonymous_id AS segment_anonymous_id,
h.anonymous_id AS hightouch_anonymous_id,
s.timestamp AS segment_timestamp,
h.timestamp AS hightouch_timestamp,
CASE WHEN s.email = h.email THEN 'Match' ELSE 'Mismatch' END AS email_comparison,
CASE WHEN s.name = h.name THEN 'Match' ELSE 'Mismatch' END AS name_comparison,
s.email AS segment_email,
h.email AS hightouch_email,
s.name AS segment_name,
h.name AS hightouch_name
FROM segment_data s
FULL OUTER JOIN hightouch_data h ON s.context_migration_id = h.context_migration_id
WHERE
s.email != h.email
OR s.name != h.name
OR s.anonymous_id != h.anonymous_id
OR s.context_migration_id IS NULL
OR h.context_migration_id IS NULL
ORDER BY s.timestamp DESC
LIMIT 100; -- Limit to 100 rows for a manageable sample
```
## What differences to expect between your old provider and Hightouch Events
Event volume should be approximately the same, but
some discrepancies between systems are normal. Deployment rollout, ad blockers, and network errors could all cause events to appear in one tool but not another, leading to differences in volume.
1. **Timing differences**: Events may be processed in slightly different orders or with small time variations.
2. **Dropped events**: Network issues might cause events to be lost in one system but not the other.
3. **Duplicate events**: Some events might be sent twice in edge cases (for example, page reloads).
4. **Blocking behavior**: Other providers might not block events as strictly as Hightouch does when running type checks.
Hightouch Events should collect the same values as your previous event collection provider, though there may be minor variations in [automatically collected fields](/events/event-spec).
In the next section, we'll explore how to [unify your historical data with new Hightouch data](unifying-data).
---
## Monitor events
**URL:** https://hightouch.com/docs/events/monitoring
**Description:** Monitor event volume, detect contract violations, and alert on sync issues so you can catch problems in your event tracking setup early.
**Section:** Hightouch Events
Monitoring and alerts help your team find, diagnose, and fix issues in your event tracking setup before they affect downstream data. This applies to any Events implementation, whether you're building fresh or migrating from another tool.
Hightouch has three features that help engineering and analytics teams manage event collection and activation:
1. Per-source volume graph
2. In-app debugger
3. Per-sync configurable alerts
The examples below show how to use each one.
## Monitor event volume
Teams can get a quick overview of event volume over time with Hightouch's built-in visualization. Each event source page shows a graph of event volume at 1, 3, and 7 day time periods.
This visual representation helps you quickly identify unusual spikes or drops in event volume, which could indicate issues with data collection or unexpected changes in user behavior.
Manual checks are useful during setup, but most teams should also configure automated alerts for noticeable drops in event volume.
Events uses Hightouch's configurable alerting system in conjunction with event-specific triggers to send messages by email, SMS, Slack, or PagerDuty. You can configure an alert so that a team member or Slack channel is notified when event traffic drops below a particular level, so your team can investigate quickly.
## Detect event violations
Hightouch also supports alerting based on your event contracts. Contracts can send invalid events to a violations table in your warehouse. That sync to the warehouse can send an alert when violations exceed a threshold in a given time period for particular [types of violations](/events/contracts/violations).
To understand a particular set of violations, use the [debugger](/events/sources/overview#debugger) on each event source. The debugger lets you inspect individual events and their data to see why they failed schema or contract checks. This event-level view complements volume monitoring and gives teams the detail they need to fix errors.
## Alert on sync issues
Hightouch supports row and sync-level alerting for reverse ETL syncs, and treats event syncs the same way. You can set up alerts for an elevated row rejection rate, such as a percentage of rows failing during a sync to your warehouse. You can also be notified when a series of consecutive syncs fails with fatal errors. Both alerts let your engineering and analytics teams investigate and fix sync errors quickly.
Some advanced alerting capabilities may require enablement for your workspace.
If you don't see the event-specific alert triggers described here,{" "}
{" "}
to enable them.
## Review and maintain regularly
Automated monitoring and alerts catch most issues, but regular human review still matters. Review data quality monthly or quarterly, and update contracts as you add new events or properties to your tracking plan.
Hightouch's monitoring and alerts help you maintain data quality and spot changes in your collection process and user behavior over time.
---
## Events overview
**URL:** https://hightouch.com/docs/events/overview
**Description:** Hightouch Events collects customer behavior from websites, mobile apps, and backend systems, then stores it in your warehouse for analytics, activation, and real-time workflows.
**Section:** Hightouch Events
| Audience | How you'll use this article |
| --------------- | ------------------------------------------------------------------------------------------------------------ |
| **Marketing** | Understand what event data is used for and how it supports activation, personalization, and experimentation. |
| **Data teams** | Learn how events are collected, structured, stored, and reused across analytics and activation workflows. |
| **Engineering** | Understand the event model, ingestion paths, and how Events fits into your data architecture. |
_Hightouch Events collects customer behavior from websites, mobile apps, and backend systems, then stores that data in your warehouse. Teams can use the same event data to analyze behavior, build audiences, trigger journeys, and power real-time workflows._
Events runs in US East (`aws-us-east-1`), EU West (`aws-eu-west-1`), and Asia
Pacific – Mumbai (`aws-ap-south-1`).{" "}
{" "}
to request another region.
---
## What you'll learn
- [What Events does](#overview)
- [What an event is](#what-an-event-is)
- [How an event moves through Hightouch](#how-an-event-moves-through-hightouch)
- [Where your data lives](#your-warehouse-is-the-source-of-truth)
- [Core features: sources, destinations, and syncs](#core-features)
- [When to use Events](#when-to-use-events)
- [How Events connects to other Hightouch products](#how-events-fits-into-hightouch)
---
## Overview
Hightouch Events lets you collect customer behavior from your website, app, or backend systems and store it in your warehouse. Once events are flowing, you can use that data to build audiences, trigger journeys, measure campaigns, and power real-time personalization.
This page explains what an event is, how Events works, and where it fits alongside your other tools.
---
## What an event is
An event is a record of a single customer action, such as viewing a product, completing a purchase, or signing up. Every event carries three things:
- **Identity** — who acted, as an `anonymousId` (a device or browser, tracked before someone is known) and, once known, a `userId`.
- **Properties or traits** — the details. Properties describe the action (a cart total). Traits describe the person (an email, a plan).
- **Context** — metadata the SDK collects automatically, such as device, page URL, or campaign parameters.
Events come in five types — `identify`, `track`, `page`, `screen`, and `group` — each for a different kind of action. See [event types and payload structure](/events/event-spec) for the full field reference.
---
## How an event moves through Hightouch
An event moves through three stages. Knowing where each happens tells you what you own and what Hightouch handles.
1. **You send the event.** You instrument your app, site, or backend with a Hightouch SDK, the HTTP API, or a streaming source like Kafka or Google Pub/Sub. This is your code, and your data quality starts here.
2. **Hightouch validates and structures it.** Hightouch checks the event against any [data contract](/events/contracts/management) you've defined, optionally transforms it with a [function](/events/functions/overview), and puts it into a consistent shape.
3. **Hightouch delivers it.** Hightouch writes the event to [your warehouse](/events/warehouses/warehouse), streams it to [downstream tools](/events/event-streaming), or both.
You own the instrumentation and the warehouse. Hightouch owns receiving, validating, structuring, and delivering.
---
## Your warehouse is the source of truth
Hightouch writes events into your own warehouse as tables — `identifies`, `tracks`, `pages`, `screens`, and `groups` — and new fields you send become new columns automatically. Events doesn't keep a separate copy of your customer profiles. Your warehouse holds the event data and is the source of truth; other Hightouch products read from it rather than from a separate store.
Because your tables mirror the data you send, consistent event names, property names, and types matter — the warehouse reflects your input rather than cleaning it. See [Warehouse schema](/events/warehouses/schema) for the table reference.
If you're coming from a tool that stored and served customer profiles for you, this is the largest shift to plan for. The logic that lived in that tool moves to your warehouse: Identity Resolution unifies a person's identifiers into one resolved identity, and [Customer Studio](/customer-studio/overview) builds traits and audiences on top.
---
## Core features
### Event sources
[Event sources](/events/sources/overview) are where events come from. Hightouch provides SDKs for [browser](/events/sdks/browser), [iOS](/events/sdks/ios), [Android](/events/sdks/android), [Node.js](/events/sdks/nodejs), and other platforms, plus an [HTTP API](/events/sdks/http) for server-side ingestion and connectors for streaming platforms like [Kafka](/events/sources/kafka) and [Google Pub/Sub](/events/sources/google-pubsub).
### Event destinations
Event destinations are where events go. You can [write events to your warehouse](/events/warehouses/warehouse) for analytics and modeling, or [stream events to downstream tools](/events/event-streaming) like your email platform, CRM, or analytics service.
### Event syncs
Event syncs define how events move between sources and destinations — which events are sent, how they're filtered, and how data is mapped.
### Governance and transformation
Use [data contracts](/events/contracts/management) to enforce event schemas and catch violations before bad data reaches your warehouse. Use [functions](/events/functions/overview) to update events before they reach a destination — for example, rename properties, filter events, or add fields to the event data.
---
## Get started
Events is usually set up by engineering and data teams. Start with [Plan your implementation](/events/implementation-plan) to make the decisions every setup depends on — your event model, identity, contracts, coverage, and delivery. Then follow the path that matches your situation:
- **Migrating** from another event collection tool, such as Segment or RudderStack? Most teams adopting Events are. The [Migration guide](/events/migration-guide/overview) keeps your existing data flowing while you switch: you run both tools in parallel, confirm the new data matches, and cut over downstream tools deliberately.
- **Implementing fresh**, with no existing event collection? Follow the [build steps in Plan](/events/implementation-plan#build-your-implementation) — connect your sources, send events to your warehouse, define contracts, validate, and monitor — with no parallel run or cutover to manage.
---
## When to use Events
Use Events when you need to collect behavioral data from your app, website, or backend systems and store it in your warehouse. For activating warehouse data that's already been collected and modeled, use Reverse ETL or Customer Studio instead.
| Scenario | Recommended approach |
| --------------------------------------------------------------- | ----------------------------------------------------------------------------------- |
| Collect behavioral data from websites, apps, or backend systems | **Events** |
| Write event data to your warehouse for analytics and modeling | **Events** [warehouse destinations](/events/warehouses/warehouse) |
| Stream events to downstream tools in real time | **Events** [event streaming](/events/event-streaming) |
| Enforce event schemas and catch data quality issues | **Events** [data contracts](/events/contracts/management) |
| Build audiences using behavioral event data | [Customer Studio](/customer-studio/overview) — uses Events data as audience filters |
| Respond to user behavior during an active session | [Real-Time](/real-time/overview) — evaluates audience membership using live events |
| Sync modeled warehouse data to business tools | [Reverse ETL](/getting-started/concepts) |
---
## How Events fits into Hightouch
Events is the collection layer of Hightouch's composable CDP. It feeds a pipeline that turns raw behavior into unified profiles and activation, with your warehouse at the center:
1. **Collect** — Events captures behavior from your app, website, and backend through SDKs, the [HTTP API](/events/sdks/http), and streaming sources like [Kafka](/events/sources/kafka) and [Google Pub/Sub](/events/sources/google-pubsub).
2. **Store** — Events writes that behavior to [your warehouse](/events/warehouses/warehouse) in a consistent schema. The warehouse is the source of truth for every stage that follows.
3. **Unify** — Identity Resolution links a person's identifiers across devices, sessions, and sources into one resolved identity.
4. **Model** — [Customer Studio](/customer-studio/overview) builds traits, audiences, and journeys on that data, without SQL.
5. **Activate** — send the result to your tools with [Reverse ETL](/getting-started/concepts), evaluate it live with [Real-Time Personalization](/real-time/overview), or optimize per person with [AI Decisioning](/ai-decisioning/overview).
You don't need every stage. Events plus your warehouse already gives you queryable behavioral data for analytics; identity unification, modeling, and activation layer on as you need them. For time-sensitive cases, Events can also [stream events](/events/event-streaming) straight to a destination, bypassing the warehouse round trip.
---
## Permissions
Access to Events resources is controlled through custom roles. Admins can configure Events-specific grants in the **Events** tab of the custom role builder, including workspace-level creation grants (for creating new event sources, destinations, contracts, and functions) and per-resource grants (for managing individual resources).
Events permissions require Permissions V2. To create or manage an event sync,
users need permissions on **both** the event source and the event destination.
See [Roles](/workspace-management/roles#events) for the full list of Events grants.
---
## Android SDK
**URL:** https://hightouch.com/docs/events/sdks/android
**Description:** Hightouch's Android SDK for collecting events
**Section:** Hightouch Events
The Android SDK makes it easy to track events from Java and Kotlin Android applications.
## Installation
The Android SDK is hosted on [Jitpack](https://jitpack.io/#ht-sdks/events-sdk-android/).
You may install it via Gradle by adding it to your `settings.gradle` and `build.gradle`.
```
// settings.gradle
allprojects {
repositories {
maven { url 'https://jitpack.io' }
}
}
```
```
// build.gradle
dependencies {
implementation 'com.github.ht-sdks.events-sdk-android:analytics:0.+'
}
```
## Initialization
You should initialize the SDK in a central location and store it in a variable
for use across your application. For example, you may want to initialize it in
your application's `onCreate` callback. After initialization, you can use the
SDK anywhere you import the `Analytics` class via the singleton instance.
**Kotlin example**:
```
val analytics = Analytics.Builder(context, "YOUR_WRITE_KEY")
.defaultApiHost("us-east-1.hightouch-events.com/v1")
.trackApplicationLifecycleEvents()
.build()
Analytics.setSingletonInstance(analytics)
```
**Java example**:
```
Analytics analytics = new Analytics.Builder(context, "YOUR_WRITE_KEY")
.defaultApiHost("us-east-1.hightouch-events.com/v1")
.trackApplicationLifecycleEvents()
.build();
Analytics.setSingletonInstance(analytics);
```
**Configuration options**:
| Option | Type | Description |
| --------------------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------- |
| `defaultApiHost` | String | The API host to send the tracked events to. |
| `flushQueueSize` | int | The number of events to queue before sending events to the Hightouch API. Defaults to 20. |
| `flushInterval` | int | The maximum amount of time (in seconds) to store events locally before forwarding to Hightouch. Defaults to 30 seconds. |
| `trackApplicationLifecycleEvents` | n/a | Automatically track application lifecycle events (like opening the application). |
| `recordScreenViews` | n/a | Automatically track screen views. |
| `trackDeepLinks` | n/a | Automatically track deep links. |
### Context
By default, Hightouch automatically collects contextual metadata with each event, such as the device operating system. This context is forwarded with each event.
#### Modify global context
To customize the default context sent with every event, use the `getAnalyticsContext()` method. This allows you to add or remove metadata that will be included with all subsequent events.
For example, to set UTM campaign parameters globally:
```
AnalyticsContext context = Analytics.with(context).getAnalyticsContext();
context.putValue("custom_key", "custom_value");
```
This applies the campaign data to every future event in the session.
For more information, see:
- [AnalyticsContext.java](https://github.com/ht-sdks/events-sdk-android/blob/main/analytics/src/main/java/com/hightouch/analytics/AnalyticsContext.java)
- [Android SDK README – Context](https://github.com/ht-sdks/events-sdk-android#context)
#### Override context per event
You can override the context for an individual `track()` or `identify()` call by passing an `Options` object as the third argument. This is useful when you want to apply custom context — like UTM campaign parameters — to a single event.
```
ValueMap campaign = new ValueMap()
.putValue("utm_source", "adwords")
.putValue("utm_medium", "cpc")
.putValue("utm_campaign", "product_launch");
ValueMap contextMap = new ValueMap().putValue("campaign", campaign);
Options options = new Options().setContext(contextMap);
Analytics.with(context).track("Product Viewed", null, options);
```
This will attach campaign context only to the specified event.
For implementation details, see:
- [Options.java](https://github.com/ht-sdks/events-sdk-android/blob/main/analytics/src/main/java/com/hightouch/analytics/Options.java)
- [Android SDK README – Context](https://github.com/ht-sdks/events-sdk-android#context)
## API
### Identify
The `identify` method sends an `identify` event. It accepts user id and trait
information for the current user -- if you only have partial information, you
can just supply a single parameter.
**Java method signature**:
```
Analytics.with(context).identify("userId", new Traits().putEmail("email"), null);
```
**Kotlin method signature**:
```
Analytics.with(context).identify("userId", Traits().putEmail("email"), null);
```
**Method parameters**:
| Parameter | Type | Description |
| --------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------- |
| `userId` | String | The user's persistent ID |
| `traits` | Traits (optional) | Additional traits about the user, such as `email` and `name`. |
| `options` | Options (optional) | Overrides to the event's context may be passed through the `options` object. This includes setting custom fields like [campaign data](#override-context-per-event). *Note that this overrides the default context.* |
### Track
The `track` method sends a `track` event. The event name is required, and
additional properties that describe the event may be supplied.
**Java method signature**:
```
Analytics.with(context).track("eventName", new Properties().putValue("custom", "property"), null);
```
**Kotlin method signature**:
```
Analytics.with(context).track("eventName", Properties().putValue("custom", "property"), null);
```
**Method parameters**:
| Parameter | Type | Description |
| ------------ | --------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `name` | String | The name of the event. |
| `properties` | Properties (optional) | Additional properties about the event, such as `product_id`. |
| `options` | Options (optional) | Overrides to the event's context may be passed through the `options` object. Note that this overrides the default context. |
### Screen
The `screen` method sends a `screen` event. The screen title is required, and
additional information about the screen's category and properties may be
supplied.
**Java method signature**:
```
Analytics.with(context).screen("category", "name", new Properties().putValue("custom", "property"), null);
```
**Kotlin method signature**:
```
Analytics.with(context).screen("category", "name", Properties().putValue("custom", "property"), null);
```
**Method parameters**:
| Parameter | Type | Description |
| ------------ | ----------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `category` | String (optional if name is supplied) | The screen's category. For example "Docs" |
| `title` | String (optional if category is supplied) | The screen's name. For example "Getting started" |
| `properties` | Properties (optional) | Additional properties about the event, such as `from_link`. |
| `options` | Options (optional) | Overrides to the event's context may be passed through the `options` object. Note that this overrides the default context. |
### Group
The `group` method sends a `group` event.
The id identifying the user's group is required. Additional traits on the group
can be provided.
**Java method signature**:
```
Analytics.with(context).group("groupId", new Traits().putValue("custom", "property"), null);
```
**Kotlin method signature**:
```
Analytics.with(context).group("groupId", Traits().putValue("custom", "property"), null);
```
**Method parameters**:
| Parameter | Type | Description |
| --------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------- |
| `groupId` | String | The id for the group. |
| `traits` | Traits (optional) | Additional traits about the group, such as `company_name`. |
| `options` | Options (optional) | Overrides to the event's context may be passed through the `options` object. Note that this overrides the default context. |
### Flush
The Android SDK buffers events locally before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and makes the tracking
non-blocking.
To force the local buffer to be sent to Hightouch immediately call the `flush`
method.
**Method signature**:
```
Analytics.with(context).flush()
```
### Reset
The `reset` method resets the `identify` and `group` for the local session.
The `reset` method should be called when users log out. This way, if the user
logs back in with another account, the userIds and traits from the different
sessions remain separate.
**Method signature**:
```
Analytics.with(context).reset()
```
### Opt out
The `optOut` method disabled event tracking. When called, events are not sent
to Hightouch's servers.
Note that the opt out persists across device reboots. To opt back in to
tracking, pass `false` to `optOut`.
**Method signature**:
```
Analytics.with(context).optOut(true)
```
---
## Browser (Javascript) SDK
**URL:** https://hightouch.com/docs/events/sdks/browser
**Description:** Hightouch's browser Javascript SDK for collecting events
**Section:** Hightouch Events
The Browser SDK makes it easy to track events in webpages.
## Installation
The Browser SDK can be installed by loading the Javascript via a `script` tag, or directly bundled via a `npm` package.
To install using a `script` tag, add the following snippet to your site, replacing WRITE_KEY with the write key from your Event Source:
```
```
To install using `npm`:
1. Install the SDK by running `npm install @ht-sdks/events-sdk-js-browser`
2. Import and initialize the SDK with the following:
```jsx
{ writeKey: "WRITE_KEY" },
{ apiHost: "us-east-1.hightouch-events.com" }
);
```
## API
### Identify
The `identify` method is used to send an `identify` event.
If `identify` is called multiple times for the same user, the `traits` are
merged together.
**Method signature**:
```
htevents.identify(userId, [traits], [context], [callback])
```
**Method parameters**:
| Parameter | Type | Description |
| ---------- | -------- | ---------------------------------------------------------------------------------------------------------------- |
| `userId` | String | The user's persistent ID. |
| `traits` | Object | Additional traits about the user, such as `email` and `name`. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `callback` | Function | A callback that's invoked after the event is sent. |
### Track
The `track` method is used to send a `track` event.
**Method signature**:
```
htevents.track(event, [properties], [context], [callback])
```
**Method parameters**:
| Parameter | Type | Description |
| ------------ | -------- | ---------------------------------------------------------------------------------------------------------------- |
| `event` | String | The event name (for example "Checked Out"). |
| `properties` | Object | Additional properties about the event, such as `product_id`. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `callback` | Function | A callback that's invoked after the event is sent. |
### Page
The `page` method is used to send a `page` event.
**Method signature**:
```
htevents.page(category, name, [properties], [context], [callback])
```
**Method parameters**:
| Parameter | Type | Description |
| ------------ | -------- | ------------------------------------------------------------------------------------------------------------------------ |
| `category` | String | The page's category. For example, "Docs". |
| `name` | String | The page's name. For example, "Getting started". |
| `properties` | Object | Additional properties about the event, such as `url`. By default, the SDK autocollects properties such as page referrer. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `callback` | Function | A callback that's invoked after the event is sent. |
### Group
The `group` method sends a `group` event.
**Method signature**:
```
htevents.group(groupId, [traits], [context], [callback])
```
**Method parameters**:
| Parameter | Type | Description |
| ---------- | -------- | ---------------------------------------------------------------------------------------------------------------- |
| `groupId` | String | The id for the group. |
| `traits` | Object | Additional traits about the group, such as `company_name`. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `callback` | Function | A callback that's invoked after the event is sent. |
### Reset
The `reset` method resets the `identify` and `group` for the local session.
Specifically, it resets the `anonymousId`, `userId`, `groupId`, and `traits`.
The `reset` method should be called when users log out. This way, if the user
logs back in with another account, the userIds and traits from the different
sessions remain separate.
**Method signature**:
```
htevents.reset()
```
## Single page applications
While Single Page Apps (SPAs) are great for many reasons, they do require some extra consideration in order to set up client-side tracking than with a traditional webpage.
By default, the Hightouch htevents.js library doesn't generate or store the referrer value. Instead, the referrer value you see in the payload is the value returned by `document.referrer` directly from the browser, and the URL value is the canonical URL on the page.
When a user navigates between pages on an SPA website, there won't be a referrer because there is no concept of a new page since it's all a single page load. This means that the referrer will always be the same as it was on the first page call where someone was first directed to your site. However, in order to circumvent this, you can manually set the referrer and URL in your Hightouch calls by updating the context object.
For example, a Page call with the referrer and URL manually set looks like this:
```js
htevents.page({
referrer: "https://hightouch.com/",
url: "https://hightouch.com/pricing/?ref=nav",
});
```
A Track call with these fields manually updated looks like this:
```js
htevents.track(
"Example Event",
{},
{
page: {
referrer: "https://hightouch.com/",
url: "https://hightouch.com/pricing/?ref=nav",
},
}
);
```
### Tracking emulated page views
Your application should update the URL in the address bar to emulate traditional webpage navigation. Full page requests aren't made in most of these instances since the resources are loaded on initial page load. This means that the Page call in the traditional htevents.js snippet won't fire again as a user navigates around your site.
You should still place the snippet in the head of your site, but you should remove the Page call and fire it whenever you're emulating a page load. Hightouch recommends that you call [Page](/events/sdks/browser#page) from the same block of logic that updates the view and URL path like below:
```js
// The new view has been called to render
htevents.page("Home");
```
To track more than the page field, pass those fields in as additional properties. Hightouch recommends that you use variables to set information about page properties, rather than hard-coding. In most SPA frameworks, you can automate this by attaching the Page call to the routing service.
## QueryString API
The Browser SDK can trigger `track` and `identify` events based on the URL query string. You can use this when tracking email click-throughs, social media clicks, and digital advertising.
**Query parameters**:
| Parameter | Description | Triggers |
| --------------------- | --------------------------------------- | ------------------------------------------------------------------------------------------ |
| `ajs_uid` | The `userId` for the `identify` event. | This triggers an `identify` event. |
| `ajs_event` | The `name` for the `track` event. | This triggers a `track` event. |
| `ajs_aid` | The `anonymousId` for the user. | This sets the `anonymousId`, by itself does not trigger any event. |
| `ajs_prop_` | A property to set on the `track` event. | This sets a property on the resulting `track` event, by itself does not trigger any event. |
| `ajs_trait_` | A trait to set on the `identity` event. | This sets a trait on the resulting `identify` event, by itself does not trigger any event. |
### Example
The following URL:
```
https://hightouch.com/?ajs_uid=123&ajs_event=Clicked%20Email&ajs_aid=abc123&ajs_prop_emailCampaign=First+Touch&ajs_trait_name=John
```
would result in these calls:
```js
htevents.identify("123", { name: "John" });
htevents.track("Clicked Email", { emailCampaign: "First Touch" });
htevents.user().anonymousId("abc123");
```
### Configuration
The `useQueryString` option allows you to control the behavior of the query parameters. For example, you can entirely disable query string processing by setting `useQueryString` to `false`:
```js
htevents.load("WRITE_KEY", {
useQueryString: false,
});
```
You can also keep query string processing on, but enforce validation rules. For example:
```js
htevents.load("WRITE_KEY", {
useQueryString: {
// set a pattern for anonymous id
aid: /([A-Z]{10})/,
// set a pattern for user id
uid: /([A-Z]{6})/,
},
});
```
## Cross Domain Tracking
Using the [QueryString API](#querystring-api) you can easily track users across different domains. For example, if you have the following sites:
- Site 1: `www.landing-page.com`
- Site 2: `www.product-page.com`
To track a user who starts on Site 1 and navigates to Site 2, you just need to pass the `userId` from Site 1 to Site 2 using the `ajs_uid` query parameter.
```
https://www.product-page.com/?ajs_uid=123
```
This will automatically trigger an `identify` event on Site 2 with `userId: 123`.
If you want to track anonymous users across domains, you can access the `anonymousId` on Site 1 using:
```js
htevents.user().anonymousId();
```
You can then pass the `anonymousId` from Site 1 to Site 2 using the `ajs_aid` query parameter. Note that the `ajs_aid` parameter does not automatically trigger an `identify` event.
## Autocollected fields
The browser SDK autocollects the following fields within the `context` of each event:
| Field | Description |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| `page.path` | The current page's sanitized path, i.e. with search parameters and the site domain removed. |
| `page.referrer` | The referrer to the current page. An empty string denotes a direct referrer. |
| `page.search` | The search parameter of the current page, if any |
| `page.title` | The current page's title |
| `page.url` | The current page's full URL |
| `sessionId` | An identifier for the visitor's current session. |
| `userAgent` | The raw user agent string |
| `userAgentData` | Parsed information on the user agent. This includes whether the visitor's device is a mobile device, and the platform of the device. |
| `locale` | The browser's locale |
| `library` | Information about the SDK version used to send the event. |
| `timezone` | The visitor's local timezone. |
## Session tracking
A session is a group of "uninterrupted" events that were performed by the same visitor.
The session resets once the session's timeout has expired, and automatically refreshes when a new event is sent.
For example, if the visitor views a couple pages, and then returns to your site after a day, the next day's events will be considered a new session.
The browser SDK automatically tracks a `sessionId` field to track what events occurred in the same session. If two events occured in the same session, they
will have the same session ID.
The session timeout can be adjusted from the default of **30 minutes** when the SDK is initialized:
```js
e.load("YOUR_WRITE_KEY", {
apiHost: "us-east-1.hightouch-events.com",
user: {
sessions: {
autoTrack: true,
// Set the timeout to 60 minutes.
timeout: 60 * 60 * 1000,
},
},
});
```
## Campaign Attribution (UTM)
Hightouch automatically captures UTM parameters that you pass in URLs. You can also manually pass UTMs into the `context.campaign` field.
UTM parameters are only used when linking to your site from outside of your domain. When a visitor arrives to your site using a link containing UTM parameters, Hightouch's Browser SDK will automatically parse the URL query string and store UTM parameters within the `context.campaign` object as outlined here. These parameters do not persist to subsequent calls unless you pass them explicitly.
**UTM parameters**:
| Parameter | Mapping | Description |
| -------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `utm_campaign` | `campaign.name` | This is the name of your campaign. All marketing activities that support this campaign need to have the same `utm_campaign` so that downstream analysis to measure performance for this specific campaign can be done off this primary key. |
| `utm_medium` | `campaign.medium` | How the traffic is coming to your site. Is it through email, a display ad, or an online forum? This ensures downstream analysis can easily see which channel performs the best.
eg. `email`, `paid-display`, `paid-social`, `organic-social` |
| `utm_source` | `campaign.source` | Where the traffic is specifically coming from. You can be specific here. This ensures downstream analysis can measure which specific source brings the most conversions.
eg. `x`, `facebook`, `adroll` |
| `utm_content` | `campaign.content` | For multiple calls to action on a single page, `utm_content` indicates which one. For example, on a website, there may be three different display ads. While the link on each display ad will have the same `utm_campaign`, `utm_medium`, and `utm_source`, the `utm_content` will be different.
eg. `banner`, `left-side`, `bottom-side` |
| `utm_term` | `campaign.term` | This is the parameter suggested for paid search to identify keywords for your ad. If you’re using Google Adwords and have enabled “autotagging”, then you don’t need to worry about this. Otherwise, you can manually pass the keywords from your search terms through this parameter so that you can see which keywords convert the most. Note that this parameter is reserved explicitly for search.
eg. `toast`, `butter`, `jam` |
If you’d like UTM parameters to persist in subsequent calls, you’ll need to manually set those fields in `context.campaign`. For example:
```js
htevents.page(
"My Page",
{},
{
campaign: {
name: "TPS Innovation Newsletter",
source: "Newsletter",
medium: "email",
term: "tps reports",
content: "image link",
},
}
);
```
## Cookies and LocalStorage
Hightouch stores cookies to track information about website visitors across multiple visits.
- `htjs_user_id`: ID tracking known or identified users as defined [here](/events/event-spec#identify-events#identify-events).
- `htjs_anonymous_id`: ID tracking anonymous website visitors as defined [here](/events/event-spec#identify-events#identify-events).
- `htjs_group_id`: ID tracking the group a user is associated with as defined [here](/events/event-spec#identify-events#group-events).
- `htjs_sesh`: Information about the user's current session as defined [here](/events/sdks/browser#session-tracking).
In addition to tracking this information in cookies, we use localStorage to track this as well. In localStorage, we also track:
- `htjs_user_traits`: Traits defined about the user being tracked
- `htjs_group_properties`: Properties defined about the group being tracked
For more information on measures to increase persistence of cookies, please refer to our guide on [proxying and server-side cookies](/events/first-party-tracking).
---
## C# SDK
**URL:** https://hightouch.com/docs/events/sdks/csharp
**Description:** Hightouch's C# SDK for collecting events
**Section:** Hightouch Events
The C# SDK makes it easy to track events from all C#/.NET applications.
## Installation
The C# SDK can be installed from [NuGet](https://www.nuget.org/packages/Hightouch.Events.CSharp):
```sh
dotnet add package Hightouch.Events.CSharp
```
To initialize the C# SDK in your application, create a new `Analytics` instance:
```c#
using Hightouch.Events;
using Hightouch.Events.Serialization;
var configuration = new Configuration("WRITE_KEY",
apiHost: "https://us-east-1.hightouch-events.com",
flushAt: 1);
var analytics = new Analytics(configuration);
```
## API
### Identify
The `analytics.Identify` method sends an [`identify`](../event-spec#identify-events) event.
**Example usage**:
```c#
analytics.Identify("user-123", new JsonObject {
["username"] = "MisterWhiskers",
["email"] = "hello@test.com",
["plan"] = "premium"
});
```
### Track
The `analytics.Track` method sends a [`track`](../event-spec#track-events) event.
**Example usage**:
```c#
analytics.Track("View Product", new JsonObject {
["productId"] = 123,
["productName"] = "Striped trousers"
});
```
### Page
The `analytics.Page` method sends a [`page`](../event-spec#page-events) event.
**Example usage**:
```c#
analytics.Page("PageName", new JsonObject {
["productSlug"] = "example-product-123"
});
```
### Screen
The `analytics.Screen` method sends a [`screen`](../event-spec#screen-events) event.
**Example usage**:
```c#
analytics.Screen("ScreenName", new JsonObject {
["productSlug"] = "example-product-123"
});
```
### Group
The `analytics.Group` method sends a [`group`](../event-spec#group-events) event.
**Example usage**:
```c#
analytics.Group("user-123", new JsonObject {
["username"] = "MisterWhiskers",
["email"] = "hello@test.com",
["plan"] = "premium"
});
```
### Flush
The C# SDK buffers events locally, either in-memory or on disk, before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and makes the tracking non-blocking.
To force the local buffer to be sent to Hightouch immediately call the `Flush()`
method. `Flush()` should be called when shutting down your app to make sure no events are lost.
**Example usage**:
```c#
analytics.Flush()
```
---
## Flutter SDK
**URL:** https://hightouch.com/docs/events/sdks/flutter
**Description:** Hightouch's Flutter SDK for collecting events
**Section:** Hightouch Events
The Flutter SDK makes it easy to track events from all types of Flutter applications.
## Installation
The Flutter SDK is hosted on [pub.dev](https://pub.dev/packages/hightouch_events).
You may install the SDK by adding it to your `pubspec.yaml` file:
```sh
flutter pub add hightouch_events
```
## Initialization
You should initialize the SDK in the top-level app state and store it in a variable
for use across your application. For example, you may want to initialize it in
your app delegate.
**Example**:
```
final analytics = createClient(Configuration("WRITE_KEY"));
```
**Configuration options**:
| Option | Type | Description |
| --------------------------------- | ------ | ---------------------------------------------------------------------------------------------------------------- |
| `writeKey` | String | Your Hightouch write key. |
| `apiHost` | String | The API host to send the tracked events to. |
| `trackApplicationLifecycleEvents` | Bool | Whether to automatically track application lifecycle events (like opening the application). Defaults to `false`. |
## API
### Identify
The `analytics.identify` method sends an [`identify`](../event-spec#identify-events) event.
**Example usage**:
```
analytics.identify(userId: "user-123", userTraits: UserTraits(email: "user@example.com"));
```
### Track
The `analytics.track` method sends a [`track`](../event-spec#track-events) event.
**Example usage**:
```
analytics.track("View Product", properties: {"productId": 123});
```
### Screen
The `analytics.screen` method sends a [`screen`](../event-spec#screen-events) event.
**Example usage**:
```
analytics.screen("Home");
```
### Group
The `analytics.group` method sends a [`group`](../event-spec#group-events) event.
**Example usage**:
```
analytics.group("group-123", groupTraits: GroupTraits(name: "Group 123"));
```
### Flush
The Flutter SDK buffers events locally before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and makes the tracking
non-blocking.
To force the local buffer to be sent to Hightouch immediately call the `flush()`
method.
### Reset
The `reset` method resets the `identify` and `group` for the local session.
Specifically, it resets the `anonymousId`, `userId`, `referrer`, and `traits`.
The `reset` method should be called when users log out. This way, if the user
logs back in with another account, the userIds and traits from the different
sessions remain separate.
---
## Go SDK
**URL:** https://hightouch.com/docs/events/sdks/go
**Description:** Hightouch's Go SDK for collecting events
**Section:** Hightouch Events
The Go SDK makes it easy to track events from Go applications.
## Installation
The Go SDK can be installed from [GitHub](https://github.com/ht-sdks/events-sdk-go):
```
go get github.com/ht-sdks/events-sdk-go
```
To initialize the Go SDK in your application, create a new `Client` instance:
```
package main
"github.com/ht-sdks/events-sdk-go"
)
func main() {
client, _ := htevents.NewWithConfig("WRITE_KEY", htevents.Config{
Endpoint: "https://us-east-1.hightouch-events.com"
})
defer client.Close()
}
```
## API
### Identify
The `htevents.Identify` message sends an `identify` event.
Unlike the [Browser SDK](./browser), it does **not** persist the user ID and
traits locally, so user IDs must be explicitly added to other events. This is
because server side events are usually servicing many different users at once.
**Example usage**:
```
client.Enqueue(htevents.Identify{
UserId: "123",
Traits: htevents.Traits{
"location": "San Francisco",
},
})
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `UserId` | string | The user's persistent ID |
| `AnonymousId` | string | The user's anonymous ID |
| `Traits` | Traits | Additional traits about the user, such as `email` and `name`. |
| `Context` | Context | Overrides to values in the event `Context`. By default, `Context` contains information autocollected by the SDK. |
| `Timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Track
The `htevents.Track` message sends a `track` event.
**Example usage**:
```
client.Enqueue(htevents.Track{
Event: "Order completed",
UserId: "123",
Properties: htevents.Properties{
"total": 29.99,
},
})
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `UserId` | string | The user's persistent ID |
| `AnonymousId` | string | The user's anonymous ID |
| `Event` | string | The name of the event. |
| `Properties` | Properties | Additional properties about the event, such as `product_id`. |
| `Context` | Context | Overrides to values in the event `Context`. By default, `Context` contains information autocollected by the SDK. |
| `Timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Page
The `htevents.Page` message sends a `page` event.
**Example usage**:
```
client.Enqueue(htevents.Page{
Name: "Getting started",
UserId: "123",
Properties: htevents.Properties{
"total": 29.99,
},
})
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `UserId` | string | The user's persistent ID |
| `AnonymousId` | string | The user's anonymous ID |
| `Name` | string | The page's name. For example "Getting started" |
| `Properties` | Properties | Additional properties about the event, such as `url`. |
| `Context` | Context | Overrides to values in the event `Context`. By default, `Context` contains information autocollected by the SDK. |
| `Timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Screen
The `htevents.Screen` message sends a `screen` event.
**Example usage**:
```
client.Enqueue(htevents.Screen{
Name: "Getting started",
UserId: "123",
Properties: htevents.Properties{
"total": 29.99,
},
})
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `UserId` | string | The user's persistent ID |
| `AnonymousId` | string | The user's anonymous ID |
| `Name` | string | The page's name. For example "Getting started" |
| `Properties` | Properties | Additional properties about the event, such as `url`. |
| `Context` | Context | Overrides to values in the event `Context`. By default, `Context` contains information autocollected by the SDK. |
| `Timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Group
The `htevents.Group` message sends a `group` event.
**Example usage**:
```
client.Enqueue(htevents.Group{
GroupId: "G-1",
UserId: "123",
Traits: htevents.Traits{
"company_location": "San Francisco",
},
})
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `UserId` | string | The user's persistent ID |
| `AnonymousId` | string | The user's anonymous ID |
| `GroupId` | string | The id for the group. |
| `Traits` | Traits | Additional traits about the group, such as `company_name`. |
| `Context` | Context | Overrides to values in the event `Context`. By default, `Context` contains information autocollected by the SDK. |
| `Timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Close
The Go SDK buffers events locally before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and makes the tracking
non-blocking.
To force the local buffer to be sent to Hightouch immediately call the `Close`
method. `Close` should be called when shutting down your app to make sure no events are lost.
**Example usage**:
```
client.Close()
```
---
## HTTP API
**URL:** https://hightouch.com/docs/events/sdks/http
**Description:** Hightouch's HTTP API endpoint for collecting events
**Section:** Hightouch Events
## HTTP API Source
Hightouch's HTTP Tracking API enables you to record data from any website or application.
## Headers
### Authentication
To authenticate Hightouch's tracking API, send your projects Write Key along with a request. Authentication uses HTTP Basic Auth.
To do this, you should include an Authorization header that includes `Basic` followed by your base64 encoded write key. This looks like this:
`Authorization: Basic YmJjYmU434YTNmRlNWRkYjM3MDUzMDRlMzN2JjM2NiY2RjYWJmZTA5YTA3ZGYxWNkNjQ5Zg==`
## Content-Type
Ensure your content-type header is set to 'application/json'
## Identify
**Identify events** capture data about _who_ your users are. This user context is vital for understanding and targeting your users. For example, you might use the collected identifiers in followup email campaigns, to segment your users by location, or to link anonymous and authenticated events.
_Example `identify` call:_
```
POST https://us-east-1.hightouch-events.com/v1/identify
```
```
{
"type": "identify",
"messageId": "111e984d-c93c-444c-b29c-f499a117c500",
"traits": {
"email": "kevin@hightouch.io"
},
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"originalTimestamp": "2023-09-22T18:05:47.063Z",
"sentAt": "2023-09-22T18:05:47.064Z"
}
```
## Track
**Track events** capture data on _what_ the user is doing. Track events can capture a broad range of actions, such as clicking a button, completing a purchase, or creating a new account.
_Example `track` call_
```
POST https://us-east-1.hightouch-events.com/v1/track
```
```
{
"type": "track",
"messageId": "7b1e2872-4ebb-4779-b9ad-e7aa1a8eab82",
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"event": "Purchase completed",
"properties": {
"cart_total": 24.99
},
"originalTimestamp": "2023-09-22T18:12:00.584Z",
"sentAt": "2023-09-22T18:12:00.585Z"
}
```
## Page events
**Page events** track the webpages users interact with. In most situations, you should automatically call the page load event when a page is loaded.
For single page apps, you should simulate page events in your routing layer.
_Example `page` call_
```
POST https://us-east-1.hightouch-events.com/v1/page
```
```
{
"type": "page",
"messageId": "bb9fcbbd-9e19-4bfd-a726-7b1e7e44f8ba",
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"properties": {
"title": "Home"
},
"originalTimestamp": "2023-09-22T18:05:47.061Z",
"sentAt": "2023-09-22T18:05:47.063Z"
}
```
## Screen events
**Screen events** are the equivalent of page views for _mobile apps_. Like page events, you should automatically call the screen event when a mobile screen view is loaded.
_Example `screen` call_
```
POST https://us-east-1.hightouch-events.com/v1/screen
```
```
{
"type": "screen",
"messageId": "bb9fcbbd-9e19-4bfd-a726-7b1e7e44f8ba",
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"properties": {
"title": "Home"
},
"originalTimestamp": "2023-09-22T18:05:47.061Z",
"sentAt": "2023-09-22T18:05:47.063Z"
}
```
## Group events
Group events let you associate users with a larger group. For example, a user may be part of an organization, or family.
The `traits` field is optional, and can include information on traits on the group. For example, if linking a user to a company, you may include traits on the company such as the company size or location.
_Example `group` call_
```
POST https://us-east-1.hightouch-events.com/v1/group
```
```
{
"type": "group",
"messageId": "d87596da-5f1e-40f4-a28a-7099290548b7",
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"groupId": "456",
"traits": { "company_name": "Hightouch" },
"originalTimestamp": "2023-09-22T18:46:23.098Z",
"sentAt": "2023-09-22T18:46:23.099Z"
}
```
---
## iOS SDK
**URL:** https://hightouch.com/docs/events/sdks/ios
**Description:** Hightouch's iOS SDK for collecting events
**Section:** Hightouch Events
The iOS SDK makes it easy to track events from Swift and Objective-C applications.
## Installation
The iOS SDK is hosted on GitHub at
`git@github.com:ht-sdks/events-sdk-swift.git` under the name
`events-sdk-swift`.
You may install the SDK by adding it to your `package.swift` file, or adding it
as a Package Dependency via Xcode.
## Initialization
You should initialize the SDK in a central location and store it in a variable
for use across your application. For example, you may want to initialize it in
your app delegate.
**Swift example**:
```
let configuration = Configuration(writeKey: "YOUR_WRITE_KEY")
.apiHost("us-east-1.hightouch-events.com/v1")
analytics = Analytics(configuration: configuration)
```
**Objective-C example**:
```
@import Hightouch
HTConfiguration *config = [[HTConfiguration alloc] initWithWriteKey:@"YOUR_WRITE_KEY"];
config.apiHost = "us-east-1.hightouch-events.com/v1";
analytics = [[HTAnalytics alloc] initWithConfiguration: config];
```
**Configuration options**:
| Option | Type | Description |
| --------------------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------- |
| `writeKey` | String | Your Highotuch write key. |
| `apiHost` | String | The API host to send the tracked events to. |
| `flushAt` | Int | The number of events to queue before sending events to the Hightouch API. Defaults to 20. |
| `flushInterval` | Int | The maximum amount of time (in seconds) to store events locally before forwarding to Hightouch. Defaults to 30 seconds. |
| `trackApplicationLifecycleEvents` | Bool | Whether to automatically track application lifecycle events (like opening the application). Defaults to `true`. |
## API
### Identify
The `identify` method sends an `identify` event. It accepts user id and trait
information for the current user -- if you only have partial information, you
can just supply a single parameter.
**Method signature**:
```
func identify(userId: String, traits: T)
func identify(traits: T)
func identify(userId: String)
```
**Method parameters**:
| Parameter | Type | Description |
| --------- | ------- | ------------------------------------------------------------- |
| `userId` | String | The user's persistent ID |
| `traits` | Codable | Additional traits about the user, such as `email` and `name`. |
### Track
The `track` method sends a `track` event. The event name is required, and
additional properties that describe the event may be supplied.
**Method signature**:
```
func track(name: String)
func track(name: String, properties: P?)
```
**Method parameters**:
| Parameter | Type | Description |
| ------------ | ------- | ------------------------------------------------------------ |
| `name` | String | The name of the event. |
| `properties` | Codable | Additional properties about the event, such as `product_id`. |
### Screen
The `screen` method sends a `screen` event. The screen title is required, and
additional information about the screen's category and properties may be
supplied.
**Method signature**:
```
func screen(title: String, category: String? = nil)
func screen(title: String, category: String? = nil, properties: P?)
```
**Method parameters**:
| Parameter | Type | Description |
| ------------ | ------- | ----------------------------------------------------------- |
| `title` | String | The screen's name. For example "Getting started" |
| `category` | String | The screen's category. For example "Docs" |
| `properties` | Codable | Additional properties about the event, such as `from_link`. |
### Group
The `group` method sends a `group` event.
The id identifying the user's group is required. Additional traits on the group
can be provided.
**Method signature**:
```
func group(groupId: String)
func group(groupId: String, traits: T?)
```
**Method parameters**:
| Parameter | Type | Description |
| --------- | ------- | ---------------------------------------------------------- |
| `groupId` | String | The id for the group. |
| `traits` | Codable | Additional traits about the group, such as `company_name`. |
### Flush
The iOS SDK buffers events locally before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and makes the tracking
non-blocking.
To force the local buffer to be sent to Hightouch immediately call the `flush`
method.
**Method signature**:
```
func flush()
```
### Reset
The `reset` method resets the `identify` and `group` for the local session.
Specifically, it resets the `anonymousId`, `userId`, `referrer`, and `traits`.
The `reset` method should be called when users log out. This way, if the user
logs back in with another account, the userIds and traits from the different
sessions remain separate.
**Method signature**:
```
func reset()
```
---
## Java SDK
**URL:** https://hightouch.com/docs/events/sdks/java
**Description:** Hightouch's Java SDK for collecting events
**Section:** Hightouch Events
The Java SDK makes it easy to track events from Java applications.
## Installation
The Java SDK is available via
[JitPack](https://jitpack.io/#ht-sdks/events-sdk-java/).
MavenGradle
```
jitpack.iohttps://jitpack.iocom.github.ht-sdks.events-sdk-javaanalyticsLATEST
```
```
allprojects {
repositories {
...
maven { url 'https://jitpack.io' }
}
}
dependencies {
...
compile 'com.github.ht-sdks.events-sdk-java:analytics:+'
}
```
To initialize the Java SDK in your application, create an `Analytics` instance:
```
public class Main {
public static void main(String... args) throws Exception {
final Analytics analytics =
Analytics.builder(WRITE_KEY)
.endpoint(API_ENDPOINT)
.build();
}
}
```
## API
### Identify
The `IdentifyMessage` sends an `identify` event.
Unlike the [Browser SDK](./browser), it does **not** persist the user ID and
traits locally, so user IDs must be explicitly added to other events. This is
because server side events are usually servicing many different users at once.
**Example usage**:
```
analytics.enqueue(IdentifyMessage.builder()
.userId("123")
.traits(ImmutableMap.builder()
.put("name", "John Doe")
.put("email", "test@example.com")
.build()
)
);
```
**Method parameters**:
| Parameter | Type | Description |
|---------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `traits` | Map | Additional traits about the user, such as `email` and `name`. |
| `context` | Map | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Track
The `TrackMessage` sends a `track` event.
**Example usage**:
```
analytics.enqueue(TrackMessage.builder("Item Purchased")
.userId("123")
.properties(ImmutableMap.builder()
.put("revenue", 39.95)
.put("shipping", "2-day")
.build()
)
);
```
**Method parameters**:
| Parameter | Type | Description |
|---------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `event` | String | The name of the event. |
| `properties` | Map | Additional properties about the event, such as `product_id`. |
| `context` | Map | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Page
The `PageMessage` sends a `page` event.
**Example usage**:
```
analytics.enqueue(PageMessage.builder("Schedule")
.userId("123")
.properties(ImmutableMap.builder()
.put("category", "Sports")
.put("path", "/sports/schedule")
.build()
)
);
```
**Method parameters**:
| Parameter | Type | Description |
|---------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `name` | String | The page's name. For example "Getting started" |
| `properties` | Map | Additional properties about the event, such as `url`. |
| `context` | Map | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Screen
The `ScreenMessage` sends a `screen` event.
**Example usage**:
```
analytics.enqueue(ScreenMessage.builder("Schedule")
.userId("123")
.properties(ImmutableMap.builder()
.put("category", "Sports")
.put("path", "/sports/schedule")
.build()
)
);
```
**Method parameters**:
| Parameter | Type | Description |
|---------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `name` | String | The screen's name. For example "Getting started" |
| `category` | String | The screen's category. For example "Docs" |
| `properties` | Map | Additional properties about the event, such as `url`. |
| `context` | Map | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Group
The `GroupMessage` sends a `group` event.
**Example usage**:
```
analytics.enqueue(GroupMessage.builder("group123")
.userId("123")
.traits(ImmutableMap.builder()
.put("name", "Hightouch")
.put("size", 50)
.build()
)
);
```
**Method parameters**:
| Parameter | Type | Description |
|---------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `groupId` | String | The id for the group. |
| `traits` | Map | Additional traits about the group, such as `company_name`. |
| `context` | Map | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Flush
The Java SDK buffers events locally before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and makes the tracking
non-blocking.
To force the local buffer to be sent to Hightouch immediately call the `flush`
method. `flush` should be called when shutting down your Java app to
make sure no events are lost.
**Example usage**:
```
analytics.flush()
```
---
## Node.js SDK
**URL:** https://hightouch.com/docs/events/sdks/nodejs
**Description:** Hightouch's Node.js SDK for collecting events
**Section:** Hightouch Events
The Node.js SDK makes it easy to track events from Node.js applications.
## Installation
The Node.js SDK is available via NPM.
To install the Node.js SDK, run:
```
npm install @ht-sdks/events-sdk-js-node
```
Initialize the SDK exactly once and export it:
```
// initialize once
writeKey: "HT_WRITE_KEY",
host: "https://us-east-1.hightouch-events.com",
});
```
Then, import it into the rest of your app, as needed:
```
// import your already initialized instance into the rest of your app
// example usage in an express like framework
app.post('/login', (req, res) => {
htevents.identify({
userId: req.body.userId,
previousId: req.body.previousId
})
res.sendStatus(200)
})
```
The `HtEvents` constructor accepts the following parameters:
| Parameter | Type | Description |
| ---------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `writeKey` | String | Key that corresponds to your Hightouch source |
| `host` | String | The base URL of the API. Default: "https://us-east-1.hightouch-events.com" |
| `path` | String | The API path route. Default: "/v1/batch" |
| `maxRetries` | Number | The maximum number of attempts when flushing a batch. Default: 3 |
| `maxEventsInBatch` | Number | The maximum number of messages to enqueue before flushing. Default: 15 |
| `flushInterval` | Number | The maximum amount of time (in milliseconds) to store events locally before forwarding to Hightouch. Defaults to 10000. |
| `httpRequestTimeout` | Number | The maximum number of milliseconds to wait for an http request. Default: 10000 |
| `disable` | Boolean | Disable the analytics library. All calls will be a noop. Default: false |
## API
### Identify
The `identify` method sends an `identify` event.
Unlike the [Browser SDK](./browser), it does **not** persist the user ID and
traits locally, so user IDs must be explicitly added to other events. This is
because server side events are usually servicing many different users at once.
**Example usage**:
```
htevents.identify({
userId: "123",
traits: {
location: "San Francisco",
}
})
```
The first argument to the method is an object representing the identify event to be sent. The second argument is an optional callback that's invoked after the event is sent.
```
htevents.identify(message, [callback])
```
The message object may include the following fields:
| Parameter | Type | Description |
| ------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------- |
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `traits` | Object | Additional traits about the user, such as `email` and `name`. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to the past. By default, this is autoset to the current date. |
### Track
The `track` method sends a `track` event.
**Example usage**:
```
htevents.track({
userId: "123",
event: "Order completed",
properties: {
total: 29.99,
}
})
```
The first argument to the method is an object representing the track event to
be sent. The second argument is an optional callback that's invoked after the
event is sent.
```
htevents.track(message, [callback])
```
The message object may include the following fields:
| Parameter | Type | Description |
| ------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `event` | String | The name of the event. |
| `properties` | Object | Additional properties about the event, such as `product_id`. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Page
The `page` method sends a `page` event.
**Example usage**:
```
htevents.page({
userId: "123",
category: "Docs",
name: "Getting started",
})
```
The first argument to the method is an object representing the page event to
be sent. The second argument is an optional callback that's invoked after the
event is sent.
```
htevents.page(message, [callback])
```
The message object may include the following fields:
| Parameter | Type | Description |
| ------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `category` | String | The page's category. For example "Docs" |
| `name` | String | The page's name. For example "Getting started" |
| `properties` | Object | Additional properties about the event, such as `url`. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Screen
The `screen` method sends a `screen` event.
**Example usage**:
```
htevents.screen({
userId: "123",
category: "Docs",
name: "Getting started",
})
```
The first argument to the method is an object representing the screen event to
be sent. The second argument is an optional callback that's invoked after the
event is sent.
```
htevents.screen(message, [callback])
```
The message object may include the following fields:
| Parameter | Type | Description |
| ------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `category` | String | The screen's category. For example "Docs" |
| `name` | String | The screen's name. For example "Getting started" |
| `properties` | Object | Additional properties about the screen. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Group
The `group` method sends a `group` event.
**Example usage**:
```
htevents.group({
userId: "123",
groupId: "456",
traits: {
company_location: "San Francisco",
},
})
```
The first argument to the method is an object representing the group event to
be sent. The second argument is an optional callback that's invoked after the
event is sent.
```
htevents.group(message, [callback])
```
The message object may include the following fields:
| Parameter | Type | Description |
| ------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `groupId` | String | The id for the group. |
| `traits` | Object | Additional traits about the group, such as `company_name`. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
---
## PHP SDK
**URL:** https://hightouch.com/docs/events/sdks/php
**Description:** Hightouch's PHP SDK for collecting events
**Section:** Hightouch Events
The PHP SDK makes it easy to track events from PHP applications.
## Installation
The PHP SDK can be installed via [Composer](https://packagist.org/packages/ht-sdks/events-sdk-php):
```sh
composer require ht-sdks/events-sdk-php
```
To initialize the PHP SDK in your application:
```php
use Hightouch\Hightouch;
Hightouch::init('WRITE_KEY', [
'host' => 'https://us-east-1.hightouch-events.com',
]);
```
## API
### Identify
The `identify` method sends an `identify` event.
Unlike the [Browser SDK](./browser), it does **not** persist the user ID and
traits locally, so user IDs must be explicitly added to other events. This is
because server side events are usually servicing many different users at once.
**Example usage**:
```php
Hightouch::identify([
'userId' => '123',
'traits' => [
'location' => 'San Francisco',
],
]);
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | string | The user's persistent ID |
| `anonymousId` | string | The user's anonymous ID |
| `traits` | array | Additional traits about the user, such as `email` and `name`. |
| `context` | array | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | int | The UNIX timestamp (seconds) of the message. When backfilling, this can be a time in the past. By default it uses `time()`. |
### Track
The `track` method sends a `track` event.
**Example usage**:
```php
Hightouch::track([
'userId' => '123',
'event' => 'Order Completed',
'properties' => [
'total' => 29.99,
],
]);
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | string | The user's persistent ID |
| `anonymousId` | string | The user's anonymous ID |
| `event` | string | The name of the event. |
| `properties` | array | Additional properties about the event, such as `product_id`. |
| `context` | array | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | int | The UNIX timestamp (seconds) of the message. When backfilling, this can be a time in the past. By default it uses `time()`. |
### Page
The `page` method sends a `page` event.
**Example usage**:
```php
Hightouch::page([
'anonymousId' => 'anonymous-id',
'name' => 'events-sdk-php',
'category' => 'docs',
'properties' => [
'path' => '/docs/libraries/php/',
],
]);
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | string | The user's persistent ID |
| `anonymousId` | string | The user's anonymous ID |
| `name` | string | The page's name. For example "Getting started" |
| `properties` | array | Additional properties about the event, such as `url`. |
| `context` | array | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | int | The UNIX timestamp (seconds) of the message. When backfilling, this can be a time in the past. By default it uses `time()`. |
### Screen
The `screen` method sends a `screen` event.
**Example usage**:
```php
Hightouch::screen([
'userId' => '123',
'name' => 'Getting started',
'properties' => [
'path' => '/getting-started',
],
]);
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | string | The user's persistent ID |
| `anonymousId` | string | The user's anonymous ID |
| `name` | string | The page's name. For example "Getting started" |
| `properties` | array | Additional properties about the event, such as `url`. |
| `context` | array | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | int | The UNIX timestamp (seconds) of the message. When backfilling, this can be a time in the past. By default it uses `time()`. |
### Group
The `group` method sends a `group` event.
**Example usage**:
```php
Hightouch::group([
'userId' => '123',
'groupId' => 'G-1',
'traits' => [
'company_location' => 'San Francisco',
],
]);
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | string | The user's persistent ID |
| `anonymousId` | string | The user's anonymous ID |
| `groupId` | string | The id for the group. |
| `traits` | array | Additional traits about the group, such as `company_name`. |
| `context` | array | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | int | The UNIX timestamp (seconds) of the message. When backfilling, this can be a time in the past. By default it uses `time()`. |
### Flush
The PHP SDK buffers events locally before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and improves performance.
To force the local buffer to be sent to Hightouch immediately call the `flush`
method. `flush` should be called when shutting down your app to make sure no events are lost.
**Example usage**:
```php
Hightouch::flush();
```
---
## Python SDK
**URL:** https://hightouch.com/docs/events/sdks/python
**Description:** Hightouch's Python SDK for collecting events
**Section:** Hightouch Events
The Python SDK makes it easy to track events from Python applications.
## Installation
The Python SDK can be installed via
[pypi.org](https://pypi.org/project/events-sdk-python/).
```
pip install events-sdk-python
```
To initialize the Python SDK in your application, import `htevents` and set the `write_key`:
```
htevents.write_key = 'WRITE_KEY'
```
## API
### Identify
The `identify` method sends an `identify` event.
Unlike the [Browser SDK](./browser), it does **not** persist the user ID and
traits locally, so user IDs must be explicitly added to other events. This is
because server side events are usually servicing many different users at once.
**Example usage**:
```
htevents.identify('user123', {
'email': 'test@example.com',
'name': 'John Doe',
})
```
**Method parameters**:
| Parameter | Type | Description |
|----------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `user_id` | str | The user's persistent ID |
| `anonymous_id` | str | The user's anonymous ID |
| `traits` | dict | Additional traits about the user, such as `email` and `name`. |
| `context` | dict | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | datetime | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Track
The `track` method sends a `track` event.
**Example usage**:
```
htevents.track('user123', 'Order completed', {
'product_id': '123',
})
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `user_id` | str | The user's persistent ID |
| `anonymous_id` | str | The user's anonymous ID |
| `event` | str | The name of the event. |
| `properties` | dict | Additional properties about the event, such as `product_id`. |
| `context` | dict | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | datetime | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Page
The `page` method sends a `page` event.
**Example usage**:
```
htevents.page('user123', 'Docs', 'Python', {
'url': 'http://example.com/docs/python'
})
```
**Method parameters**:
| Parameter | Type | Description |
|----------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `user_id` | str | The user's persistent ID |
| `anonymous_id` | str | The user's anonymous ID |
| `category` | str | The screen's category. For example "Docs" |
| `name` | str | The page's name. For example "Getting started" |
| `properties` | dict | Additional properties about the event, such as `url`. |
| `context` | dict | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | datetime | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Screen
The `screen` method sends a `screen` event.
**Example usage**:
```
htevents.screen('user123', 'Docs', 'Python', {
'url': 'http://example.com/docs/python'
})
```
**Method parameters**:
| Parameter | Type | Description |
|----------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `user_id` | str | The user's persistent ID |
| `anonymous_id` | str | The user's anonymous ID |
| `category` | str | The screen's category. For example "Docs" |
| `name` | str | The screen's name. For example "Getting started" |
| `properties` | dict | Additional properties about the event, such as `url`. |
| `context` | dict | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | datetime | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Group
The `group` method sends a `group` event.
**Example usage**:
```
htevents.group('user123', 'group123', {
'company_name': 'Initech',
})
```
**Method parameters**:
| Parameter | Type | Description |
|----------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `user_id` | str | The user's persistent ID |
| `anonymous_id` | str | The user's anonymous ID |
| `group_id` | str | The id for the group. |
| `traits` | dict | Additional traits about the group, such as `company_name`. |
| `context` | dict | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | datetime | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Flush
The Python SDK buffers events locally before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and makes the tracking
non-blocking.
To force the local buffer to be sent to Hightouch immediately call the `flush`
method. `flush` should be called when shutting down your Python app to
make sure no events are lost.
**Example usage**:
```
htevents.flush()
```
---
## React Native SDK
**URL:** https://hightouch.com/docs/events/sdks/react-native
**Description:** Hightouch's React Native SDK for collecting events
**Section:** Hightouch Events
The React Native SDK makes it easy to track events in React Native
applications. The SDK supports both iOS and Android applications.
## Installation
The React Native SDK is distributed via `npm`.
Install the required dependencies into your React Native project via `npm`:
```
npm install --save \
@ht-sdks/events-sdk-react-native \
@ht-sdks/sovran-react-native \
react-native-get-random-values \
@react-native-async-storage/async-storage
```
### iOS
On iOS, you will also need to install the native modules by running:
```
npx pod-install
```
### Android
The SDK requires additional permissions on Android in order to populate the
device context for events.
Add these permissions to your app by updating your `AndroidManifest.xml`:
```
```
## SDK Initialization
Once the dependencies are installed, you can import the tracking client into
your application.
```
const htClient = createClient({
writeKey: 'API_KEY'
});
```
The client can either be passed around explicitly throughout your app, or
exposed through a [React context
hook](https://react.dev/learn/passing-data-deeply-with-context). To pass the
context through a Context, use `AnalyticsProvider` and `useAnalytics`.
```
// The Child component can retrieve the htClient by calling `useAnalytics()`.
const App = () => (
);
```
The client may be configured with the following options:
| Parameter | Type | Description |
| ------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `writeKey` | String | The write key used to identify the source in Hightouch. |
| `collectDeviceId` | Boolean | Whether to autocollect a device ID via the Android DRM API. The ID is stored in `context.device.id` on all events. This is only supported on Android, and is disabled by default. |
| `debug` | Boolean | Whether to generate debug logs. Defaults to `true`. |
| `logger` | Logger | Custom logger instance to redirect internal logging from the library. |
| `flushAt` | Number | Maximum number of events to accumulate before sending to the Hightouch API. Defaults to `20`. |
| `flushInterval` | Number | Maximum number of seconds to hold events before sending to the Hightouch API. Defaults to `30`. |
| `flushPolicies` | Array | Optional advanced customization for how events are buffered before sending to the Hightouch API. Mutually exclusive with `flushAt` and `flushInterval` — if set, those properties are ignored. |
| `maxBatchSize` | Number | The maximum number of events to send in one HTTP request to the Hightouch API. You may want to adjust this if your events are extremely large. Defaults to `1000`. |
| `trackAppLifecycleEvents` | Boolean | Whether to automatically track lifecycle events like `Application Installed`. Defaults to `false`. |
| `trackDeepLinks` | Boolean | Whether to automatically track deep links. Only supported for Android. Defaults to `false`. |
| `proxy` | String | Used to override the URL used for sending events to Hightouch. This should contain the full path, e.g. `https://us-east-1.hightouch-events.com/v1/batch` |
## Manual tracking API
### Identify
The `identify` method is used to send an `identify` event.
If `identify` is called multiple times for the same user, the `traits` are
merged together.
**Method signature**:
```
htClient.identify(userId, [traits])
```
**Method parameters**:
| Parameter | Type | Description |
| --------- | ------ | ------------------------------------------------------------- |
| `userId` | String | The user's persistent ID. |
| `traits` | Object | Additional traits about the user, such as `email` and `name`. |
### Track
The `track` method is used to send a `track` event.
**Method signature**:
```
htClient.track(event, [properties])
```
**Method parameters**:
| Parameter | Type | Description |
| ------------ | ------ | ------------------------------------------------------------ |
| `event` | String | The event name (for example "Checked Out"). |
| `properties` | Object | Additional properties about the event, such as `product_id`. |
### Screen
The `screen` method is used to send a `screen` event.
**Method signature**:
```
htClient.screen(name, [properties])
```
**Method parameters**:
| Parameter | Type | Description |
| ------------ | ------ | -------------------------------------------------- |
| `name` | String | The screen's name. For example, "Getting started". |
| `properties` | Object | Additional properties about the event. |
### Group
The `group` method sends a `group` event.
**Method signature**:
```
htClient.group(groupId, [traits])
```
**Method parameters**:
| Parameter | Type | Description |
| --------- | ------ | ---------------------------------------------------------- |
| `groupId` | String | The id for the group. |
| `traits` | Object | Additional traits about the group, such as `company_name`. |
### Reset
The `reset` method resets the `identify` calls for the local session.
Specifically, it resets the `anonymousId`, `userId`, and `traits`.
The `reset` method should be called when users log out. This way, if the user
logs back in with another account, the userIds and traits from the different
sessions remain separate.
**Method signature**:
```
htClient.reset()
```
### Flush
The `flush` method forces the SDK to send all queued events to the Hightouch API immediately, rather than waiting for the next automatic flush cycle.
**Method signature**:
```
await htClient.flush()
```
Call `flush` after mission-critical events like purchases or post-login `identify` calls to make sure those events are delivered before the user closes or backgrounds the app.
```ts
// Ensure a purchase event is sent immediately
htClient.track("Order Completed", { orderId: "12345", total: 59.99 });
await htClient.flush();
```
#### Flush behavior
The SDK queues events locally and sends them to the Hightouch API in batches. By default, the SDK flushes when either of these conditions is met:
- **Count threshold** (`flushAt`): The queue accumulates 20 events (configurable via the `flushAt` option in `createClient`).
- **Time interval** (`flushInterval`): 30 seconds have elapsed since the last flush (configurable via the `flushInterval` option in `createClient`).
Events on the queue are persisted to `AsyncStorage`, so they survive app restarts. Only events that are successfully delivered are removed from the queue. Failed events are retried on the next flush cycle.
#### Flush policies
For more control over when batches are sent, you can pass custom flush policies via the `flushPolicies` option in `createClient`. The SDK ships with the following built-in policies:
- `CountFlushPolicy(limit)` — flushes after a specified number of events accumulate. This is the policy behind the `flushAt` default.
- `TimerFlushPolicy(intervalMs)` — flushes on a recurring time interval (in milliseconds). This is the policy behind the `flushInterval` default.
- `BackgroundFlushPolicy()` — flushes automatically when the app is sent to the background. This is useful for ensuring events are sent before the OS suspends your app.
- `StartupFlushPolicy()` — triggers a single flush immediately when the client starts. This is useful for sending any events that were queued from a previous session.
```ts
createClient,
CountFlushPolicy,
TimerFlushPolicy,
BackgroundFlushPolicy,
StartupFlushPolicy,
} from "@ht-sdks/events-sdk-react-native";
const htClient = createClient({
writeKey: "API_KEY",
flushPolicies: [
new CountFlushPolicy(10), // Flush every 10 events
new TimerFlushPolicy(15 * 1000), // Flush every 15 seconds
new BackgroundFlushPolicy(), // Flush when the app is backgrounded
new StartupFlushPolicy(), // Flush any queued events on startup
],
});
```
`flushPolicies` is mutually exclusive with `flushAt` and `flushInterval`. If
you provide `flushPolicies`, the SDK ignores `flushAt` and `flushInterval`
entirely and uses only the policies you specify. The default count and timer
policies are not included automatically, so add them explicitly if you still
want that behavior.
You can also add or remove flush policies at runtime:
```ts
const backgroundPolicy = new BackgroundFlushPolicy();
// Add a policy after initialization
htClient.addFlushPolicy(backgroundPolicy);
// Remove a policy later
htClient.removeFlushPolicy(backgroundPolicy);
```
## Anonymous ID management
Every device that uses the SDK is assigned an `anonymousId` — a UUID that the SDK auto-generates on first launch and persists in `AsyncStorage`. The `anonymousId` is attached to every event, allowing Hightouch to stitch anonymous activity to a known user once `identify` is called.
The anonymous ID and other user info are accessible through the `userInfo` property on the client.
### Get the anonymous ID
To read the current `anonymousId`, call `userInfo.get()`. Pass `true` for a safe async read that waits for storage to be ready:
```ts
// Async (recommended) — waits for storage to be fully restored
const { anonymousId } = await htClient.userInfo.get(true);
// Sync — returns the current in-memory value immediately
const { anonymousId } = htClient.userInfo.get();
```
This is useful when you need to pass the anonymous ID to another system or include it in a deep link for cross-platform identity resolution.
### Set or override the anonymous ID
To replace the auto-generated `anonymousId` with your own value, use `userInfo.set()`:
```ts
await htClient.userInfo.set((state) => ({
...state,
anonymousId: "my-custom-anonymous-id",
}));
```
Common use cases for overriding the anonymous ID include:
- **Migrating from another vendor.** If your app previously used a different analytics SDK, you can carry over the existing anonymous ID so that pre- and post-migration events are attributed to the same device.
- **Using your own device identifier.** If your app already generates a stable device ID, you can use it as the anonymous ID to simplify identity resolution.
Set the anonymous ID before any `identify` or `track` calls so that all events
share the same identifier from the start of the session.
### Relationship between anonymousId and userId
Before `identify` is called, all events carry only the `anonymousId`. Once you call `identify(userId)`, subsequent events carry both the `anonymousId` and the `userId`. Hightouch uses both values to stitch anonymous and identified activity together during identity resolution.
Calling `reset()` generates a new `anonymousId` and clears the `userId` and `traits`, starting a fresh anonymous session. If you want to keep the current `anonymousId` while clearing the user, pass `false`:
```ts
// Reset everything including anonymousId (default)
await htClient.reset();
// Clear userId and traits but keep the current anonymousId
await htClient.reset(false);
```
## Out of the box events
### Lifecycle events
When `trackAppLifecycleEvents` is enabled, Hightouch automatically tracks the following events:
- `Application Installed` -- Emitted when the app is first opened after a new install.
- `Application Updated` -- Emitted when the app is first opened after upgrading from a previous version.
- `Application Opened` -- Emitted whenever the app is opened (including when it's resuming from the background).
- `Application Backgrounded` -- Emitted whenever the app is backgrounded.
## Advertising identifiers
To collect IDFAs and AAIDs for advertising identification, you may use the
`@ht-sdks/events-sdk-react-native-plugin-advertising-id` and
`@ht-sdks/events-sdk-react-native-plugin-idfa` packages.
These plugins load native modules that pull the local advertising ID.
### Android AAID
Once setup, the AAID will be stored in `context.device.advertisingId`
1. Add the Android AAID plugin to your project.
```
npm install @ht-sdks/events-sdk-react-native-plugin-advertising-id
```
2. Add the plugin to your code by adding the following code after initializing your client.
```
htClient.add({plugin: new AdvertisingIdPlugin()});
```
3. Update your Android application manifest to include your [AdMob app ID](https://developers.google.com/admob/android/quick-start).
```
```
### iOS IDFA
Once setup, the IDFA will be stored in `context.device.advertisingId`
1. Add the iOS IDFA plugin to your project.
```
npm install @ht-sdks/events-sdk-react-native-plugin-idfa
```
2. Add the plugin to your code by adding the following code after initializing your client.
```
htClient.add({plugin: new IdfaPlugin()});
```
3. Update your `Info.plist` to set
[NSUserTrackingUsageDescription](https://developer.apple.com/documentation/bundleresources/information_property_list/nsusertrackingusagedescription).
This description will be displayed when prompting users for permission to
track their IDFA.
## Tips and troubleshooting
### Missing events
If events are not appearing in Hightouch, work through the following steps:
1. **Upgrade to the latest SDK version.** Run `npm install @ht-sdks/events-sdk-react-native@latest` to pick up the most recent bug fixes and delivery improvements.
2. **Add a manual `flush` after critical events.** The SDK buffers events for up to 30 seconds by default. Although queued events are persisted and retried on the next app launch, calling `await htClient.flush()` immediately after high-value events like purchases or sign-ups ensures they are delivered promptly rather than waiting for the next flush cycle or app session.
3. **Add a `BackgroundFlushPolicy`.** This ensures the queue is flushed whenever the app moves to the background, covering cases where users leave the app between automatic flush cycles. You can add it at runtime without replacing your existing policies:
```ts
htClient.addFlushPolicy(new BackgroundFlushPolicy());
```
4. **Enable debug logging.** Set `debug: true` in `createClient` (this is the default) and check device logs for errors during event delivery.
5. **Verify your write key.** Confirm that the `writeKey` in `createClient` matches the write key shown on your event source in Hightouch.
---
## Ruby SDK
**URL:** https://hightouch.com/docs/events/sdks/ruby
**Description:** Hightouch's Ruby SDK for collecting events
**Section:** Hightouch Events
The Ruby SDK makes it easy to track events from Ruby applications.
## Installation
The Ruby SDK can be installed via
[RubyGems.org](https://rubygems.org/gems/events-sdk-ruby/). You may add the SDK
to your Gemfile if you're using Bundler, or install it directly:
```
gem install events-sdk-ruby
```
To initialize the Ruby SDK in your application, create a new Analytics instance:
```
require 'hightouch'
analytics = Hightouch::Analytics.new(
write_key: WRITE_KEY,
host: 'us-east-1.hightouch-events.com'
)
```
## API
### Identify
The `identify` method sends an `identify` event.
Unlike the [Browser SDK](./browser), it does **not** persist the user ID and
traits locally, so user IDs must be explicitly added to other events. This is
because server side events are usually servicing many different users at once.
**Example usage**:
```
analytics.identify(
user_id: "123",
traits: {
location: "San Francisco",
}
)
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `user_id` | String | The user's persistent ID |
| `anonymous_id` | String | The user's anonymous ID |
| `traits` | Hash | Additional traits about the user, such as `email` and `name`. |
| `context` | Hash | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Track
The `track` method sends a `track` event.
**Example usage**:
```
analytics.track(
user_id: "123",
event: "Order completed",
properties: {
total: 29.99,
}
)
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `user_id` | String | The user's persistent ID |
| `anonymous_id` | String | The user's anonymous ID |
| `event` | String | The name of the event. |
| `properties` | Hash | Additional properties about the event, such as `product_id`. |
| `context` | Hash | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Page
The `page` method sends a `page` event.
**Example usage**:
```
analytics.page(
user_id: "123",
category: "Docs",
name: "Getting started",
)
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `user_id` | String | The user's persistent ID |
| `anonymous_id` | String | The user's anonymous ID |
| `name` | String | The page's name. For example "Getting started" |
| `properties` | Hash | Additional properties about the event, such as `url`. |
| `context` | Hash | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Screen
The `screen` method sends a `screen` event.
**Example usage**:
```
analytics.screen(
user_id: "123",
category: "Docs",
name: "Getting started",
)
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `user_id` | String | The user's persistent ID |
| `anonymous_id` | String | The user's anonymous ID |
| `name` | String | The screen's name. For example "Getting started" |
| `category` | String | The screen's category. For example "Docs" |
| `properties` | Hash | Additional properties about the event, such as `url`. |
| `context` | Hash | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Group
The `group` method sends a `group` event.
**Example usage**:
```
analytics.group(
user_id: "123",
group_id: "456",
traits: {
company_location: "San Francisco",
},
)
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `user_id` | String | The user's persistent ID |
| `anonymous_id` | String | The user's anonymous ID |
| `groupId` | String | The id for the group. |
| `traits` | Object | Additional traits about the group, such as `company_name`. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Flush
The Ruby SDK buffers events locally before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and makes the tracking
non-blocking.
To force the local buffer to be sent to Hightouch immediately call the `flush`
method. `flush` should be called when shutting down your Ruby app to
make sure no events are lost.
**Example usage**:
```
analytics.flush()
```
---
## Google PubSub
**URL:** https://hightouch.com/docs/events/sources/google-pubsub
**Description:** Hightouch and Google PubSub integration
**Section:** Hightouch Events
## Overview
Hightouch supports streaming events from Google Pub/Sub in realtime.
Simply configure your Google Pub/Sub source in the Hightouch UI, and Hightouch will automatically extract, transform, and load your data into Hightouch Events. Your data can then be synced to your warehouse or streamed to our catalog of realtime destinations.
## Setup
To get started, [**create an event source**](https://app.hightouch.com/events/sources/new?from=docs), select **Google Pub/Sub** and follow the steps below.
### Configure your source
Configure the following fields in Hightouch:
- **Credentials**: GCP credentials for connecting to Pub/Sub.
- **Subscription**: The full name of the Pub/Sub subscription to consume events from.
- **Project ID**: The Google Cloud Project ID.
- **Regional API Endpoint**: [List of endpoints](https://cloud.google.com/Pub/Sub/docs/reference/service_apis_overview#Pub/Sub_endpoints)
### Grant permissions
Hightouch believes in the principle of least privilege. We ask for no more permissions than necessary. For the Google Pub/Sub event source, Hightouch requires `roles/Pub/Sub.subscriber` and `roles/Pub/Sub.viewer`. Ensure you run the command in your Google Cloud Console to grant Hightouch's service account the correct permissions.
### Message format
Hightouch Events supports PubSub messages encoded using Avro and Protobuf as long as a [PubSub Schema](https://cloud.google.com/pubsub/docs/schemas) is configured and associated with the topic. Otherwise, messages need to be encoded as JSON.
## Event Structure
All events are ingested into Hightouch as `track` events. The Pub/Sub message `id`, `data`, and `attributes` are exposed as custom `properties` on the `track` event. Metadata such as `subscription` are also exposed in the `context.Pub/Sub` object.
For example, the following Pub/Sub message:
```json
{
"id": "123",
"data": {
"user_id": "user_123",
"amount": 123.45
},
"attributes": {
"a1": "abc"
}
}
```
Will be ingested into Hightouch as:
```json
{
"type": "track",
"event": "Google Pub/Sub Event",
"properties": {
"id": "123",
"data": {
"user_id": "user_123",
"amount": 123.45
},
"attributes": {
"a1": "abc"
}
},
"context": {
"Pub/Sub": {
"subscription": "projects/my-project/subscriptions/my-subscription"
}
},
"timestamp": "2024-07-29T14:52:37.648Z"
}
```
### Field mappings
You can configure how Hightouch transforms messages from your Pub/Sub topic to Hightouch [`track` events](/events/event-spec#track-events). Mappings can be configured for the following fields:
- `event`
- `userId`
- `anonymousId`
- `messageId`
- `timestamp`
You can also transform your events using Functions in order to standardize the schema or enrich its contents.
### Schema Enforcement
[Event Contracts](/events/contracts/management) can be attached to Google Pub/Sub sources just like any other Event Source.
## Tips and troubleshooting
---
## Kafka (Apache, Confluent, & Redpanda)
**URL:** https://hightouch.com/docs/events/sources/kafka
**Description:** Hightouch integration with Apache Kafka, Confluent Cloud, and Redpanda
**Section:** Hightouch Events
## Overview
Hightouch supports streaming events from **Apache Kafka**, **Confluent Cloud**, and **Redpanda** clusters in real time.
All three are **Kafka-compatible platforms**:
- **Apache Kafka**: the open-source distributed event streaming platform.
- **Confluent Cloud**: a fully managed Kafka-as-a-service offering with enterprise features.
- **Redpanda**: a high-performance, Kafka API–compatible platform designed as a drop-in replacement.
Since they all speak the **Kafka protocol**, setup and event ingestion in Hightouch are nearly identical.
Once connected, Hightouch automatically extracts, transforms, and loads your event data into Hightouch Events. Your data can then be synced to your warehouse or streamed to our catalog of real-time destinations.
## Setup
To get started, [**create an event source**](https://app.hightouch.com/events/sources/new?from=docs) and select either:
- **Kafka** for Apache Kafka
- **Confluent** for Confluent Cloud
- **Redpanda** (for Redpanda clusters)
Both sources use the same configuration flow.
### Choose connection type
### Configure your source
The required fields are the same across Apache Kafka, Confluent, and Redpanda:
- **Broker URL**:
- Kafka: hostname:port for your brokers.
- Confluent: provided in your Confluent cluster settings.
- Redpanda: hostname:port for your Redpanda brokers.
- **Topic**: The topic to consume events from. Messages must be JSON-formatted.
- **Authentication**:
- Kafka: Multiple auth methods supported.
- Confluent: Use SASL/SSL credentials from Confluent Cloud.
- Redpanda: Uses Kafka-compatible auth methods configured in your cluster.
### Message format
Hightouch Events requires Kafka messages be encoded as JSON.
## Event Structure
All events from Kafka, Confluent, and Redpanda are ingested into Hightouch as `track` events.
- Message `key`, `value`, and `headers` → `properties`
- Metadata such as `topic`, `partition`, and `offset` → `context.kafka`
For example, the following Kafka message:
```json
{
"partition": 1,
"offset": 123,
"timestamp": 1721659050648,
"key": "k1",
"value": {
"user_id": "user_123",
"amount": 123.45
},
"headers": [
{
"key": "h1",
"value": "abc"
}
]
}
```
Will be ingested into Hightouch as:
```json
{
"type": "track",
"event": "Kafka Event",
"properties": {
"key": "k1",
"value": {
"user_id": "user_123",
"amount": 123.45
},
"headers": {
"h1": "abc"
}
},
"context": {
"kafka": {
"topic": "my-topic",
"partition": 1,
"offset": "123"
}
},
"timestamp": "2024-07-22T14:52:37.648Z"
}
```
### Field mappings
You can configure how Hightouch transforms messages from your Kafka topic to Hightouch [`track` events](/events/event-spec#track-events). Mappings can be configured for the following fields:
- `event`
- `userId`
- `anonymousId`
- `messageId`
- `timestamp`
You can also use Functions to standardize schemas or enrich event data.
### Schema Enforcement
[Event Contracts](/events/contracts/management) can be attached to Kafka, Confluent, and Redpanda sources just like any other Event Source.
## Tips and troubleshooting
---
## Event sources overview
**URL:** https://hightouch.com/docs/events/sources/overview
**Description:** Monitor, validate, and govern events as they enter Hightouch.
**Section:** Hightouch Events
| Audience | How you’ll use this article |
| --------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| **Marketing teams** | Understand where events come from, how they’re monitored, and how event health impacts real-time activation and personalization. |
| **Data teams** | Learn how incoming events are validated, inspected, governed, and made reliable for downstream use. |
| **Engineering teams** | Monitor event volume, debug payloads, validate schemas, and configure alerts to catch issues early. |
## What you’ll learn
By the end of this article, you’ll be able to:
1. [Add or configure an event source](#add-or-configure-an-event-source)
2. [Monitor event volume and health](#monitoring-and-overview)
3. [Explore events in the event catalog](#event-catalog)
4. [Set up alerting for event behavior](#alerting)
5. [Inspect events using the debugger](#debugger)
6. [Configure SDK for event source](#setup)
---
## Overview
An **event source** represents a system that sends events into Hightouch, such as a web app, mobile app, backend service, or data pipeline.
Event sources are the starting point for all event-based workflows in Hightouch, including event storage, event streaming, and downstream activation.
---
#### Supported SDKs
Hightouch provides several SDKs (software development kits) that you can deploy to your app or site in order to start collecting events. There are separate SDKs for:
- [Browser (Javascript)](/events/sdks/browser)
- [Node.js](/events/sdks/nodejs)
- [Ruby](/events/sdks/ruby)
- [Go](/events/sdks/go)
- [PHP](/events/sdks/php)
- [Java](/events/sdks/java)
- [Python](/events/sdks/python)
- [iOS](/events/sdks/ios)
- [Android](/events/sdks/android)
- [React Native](/events/sdks/react-native)
- [Flutter](/events/sdks/flutter)
- [C# (.NET)](/events/sdks/csharp)
For server-side ingestion without an SDK, use the [HTTP API](/events/sdks/http). You can also collect events directly from [Kafka](/events/sources/kafka), [PubSub](/events/sources/google-pubsub), and [Webhooks](/events/sources/webhook), and inspect events in a web-based debugger to confirm you're capturing them correctly from each source.
Please note that there is a 96 kilobytes size limit for each event that's collected.
#### Regions
Events runs in US East (`aws-us-east-1`), EU West (`aws-eu-west-1`), and Asia Pacific – Mumbai (`aws-ap-south-1`). See the [Events overview](/events/overview) for the current list and how to request another region.
---
## Add or configure an event source
To add or manage an event source:
1. Go to [https://app.hightouch.com/?from=docs](https://app.hightouch.com/?from=docs)
2. Navigate to **`Event Collection → Event sources`**
3. Click **Add source** to create a new event source, or
4. Select an existing source from the list to begin monitoring, debugging, or configuration
---
## Monitoring and overview
The **Overview** tab provides a high-level view of event activity for a single source.
From this view, you can:
- Monitor **total event volume** over time
- Identify gaps, drops, or spikes in traffic
- See whether any **alerts** are currently active
Use the time range controls (`24h`, `3d`, `7d`) to analyze recent changes or longer-term trends.
This view is typically used to answer questions like:
- _Is this source actively sending events?_
- _Did volume change after a release or deployment?_
- _Are any alerts firing that require investigation?_
---
## Event catalog
The **Event catalog** lists all event types received from the source, along with their recent volume and configuration status.
For each event, you can:
- View the **event name and type** (for example, `Track`, `Identify`, or `Group`)
- Monitor **event volume** over time
- Configure **data contracts** to validate schema and required fields
- Set up **alerts** for volume-based changes
New event types appear in the catalog after they are received for the first time. This may take a few minutes after initial instrumentation.
Use the event catalog to:
- Understand which events are actively being sent
- Identify high- or low-volume events
- Decide where schema validation or alerting is needed
---
## Alerting
Alerting helps you detect unexpected changes in event behavior.
Alerts are configured at the **event level** from the **Event catalog**.
**Navigate to alerting:**
1. Go to **`Event Collection → Event sources`**
2. Select an event source
3. Open the **`Event catalog`** tab
4. Find the event you want to monitor
5. Click **Configure** in the **Alerts** column
This opens the event’s **Alerting** panel, where you can review alert history, define triggers, and configure recipients.
From this panel, you can configure alerts to notify your team when:
- Event volume exceeds a high threshold
- Event volume drops below a minimum threshold
Alerts can be sent to external notification channels, such as email or messaging tools, so teams can respond quickly to issues.
Alerting is commonly used to:
- Detect broken or stalled event pipelines
- Catch unintended increases in event volume
- Monitor critical events that power downstream workflows
---
## Debugger
The **Debugger** lets you inspect individual events as they arrive from the source.
From the debugger, you can:
- View recently received events and timestamps
- Search events by name, type, or payload content
- Inspect the **full JSON payload** for a selected event
- Pause or clear the event stream during debugging
The debugger is especially useful during:
- Initial SDK or API setup
- Instrumentation changes
- Troubleshooting missing or malformed events
If no events appear, confirm that the SDK or integration is installed correctly and that the source is actively sending data.
---
## Setup
Use the **Setup** tab to install and configure the SDK or integration for this event source. This view provides source-specific installation instructions, including required keys, endpoints, and example code.
---
## Next steps
- [**Send events to your warehouse**](/events/warehouses/warehouse) once events are arriving
- [**Build your implementation**](/events/implementation-plan#build-your-implementation) — the full set of steps from collection through monitoring
---
## Troubleshoot events
**URL:** https://hightouch.com/docs/events/troubleshooting
**Description:** Diagnose common Events problems — missing events, blocked or flagged events, unexpected warehouse columns, rejected writes, and data that doesn't match your previous tool.
**Section:** Hightouch Events
_Start here when something looks wrong with your event data. Each section starts from a symptom, points to the likely cause, and links to the fix._
Start with these two tools:
- The [debugger](/events/sources/overview#debugger) shows individual events as they arrive, so you can confirm exactly what Hightouch received.
- The `event_violations` table in your warehouse records every event that failed contract validation, along with the reason. See [handling violations](/events/contracts/violations).
---
## Events aren't arriving, or volume is lower than expected
Work through these in order:
- **Confirm receipt in the debugger.** If events don't appear there, the problem is upstream of Hightouch — the SDK isn't firing, or the write key or region is wrong. Check that you're using the write key from your event source.
- **Expect some client-side loss.** Browser events are dropped by ad blockers, privacy settings such as Safari ITP, and cookie limits. This is inherent to client-side collection, not a setup error. Reduce it with [first-party tracking](/events/first-party-tracking), and move business-critical events to [server-side collection](/events/sdks/http). See [Plan for coverage](/events/implementation-plan#plan-for-coverage).
- **Check whether a contract is blocking events.** If enforcement is set to block, undeclared or invalid events are dropped from your main tables. See the next section.
## An event is blocked or flagged
A contract [violation](/events/contracts/violations) means an event didn't satisfy the contract applied to its source. The match is exact, so the usual causes are small mismatches:
- The event name doesn't match the contract, including capitalization and whitespace.
- A field is in the wrong section — for example, sent under `properties` when the contract expects it under `context`.
- A field's type doesn't match the schema.
- The contract is empty while set to block undeclared events, so everything is blocked.
Work through these with [Diagnose why an event was blocked or flagged](/events/contracts/violations#diagnose-why-an-event-was-blocked-or-flagged), which maps each cause to its fix.
## Unexpected or extra columns in the warehouse
Your warehouse tables mirror the data you send — Hightouch structures it but doesn't clean it. Extra or odd columns usually trace back to the input:
- A misspelled or inconsistently named property creates a new column whenever it varies.
- Malformed URL or campaign parameters — for example, a missing `=` in a UTM tag — land as unexpected fields.
- Nested objects are flattened into separate columns (`company.name` becomes `company_name`).
Validate the inbound data before changing anything in the warehouse. See [Warehouse schema](/events/warehouses/schema) for how columns and types are created.
## An event was rejected and not written
If an event's value doesn't match the existing column type — for example, a string where the column is a number — Hightouch can't write it. It [retries the event for 48 hours](/events/warehouses/error-handling), so correcting the type in your warehouse lets the event sync without data loss. See [Error handling](/events/warehouses/error-handling).
## Data doesn't match my previous tool
During a migration, some difference between systems is normal — timing, deduplication, ad blockers, and Hightouch's stricter blocking all cause minor variation. Before treating a gap as a bug, see [what differences to expect](/events/migration-guide/validating-your-migration#what-differences-to-expect-between-your-old-provider-and-hightouch-events) and the parity checks in [Validate your migration](/events/migration-guide/validating-your-migration).
---
## Still stuck?
Set up [monitoring and alerts](/events/monitoring) so you catch issues early, and if you can't resolve a problem.
---
## Error handling
**URL:** https://hightouch.com/docs/events/warehouses/error-handling
**Description:** Understand how Hightouch handles events that fail to write to your warehouse.
**Section:** Hightouch Events
Sometimes Hightouch can't write an event to your warehouse. This can happen
for a variety of reasons; for example, the warehouse credentials may be
invalid, or your event schema may have evolved to have incompatible types.
When it happens, Hightouch retries the event and stores unresolved failures so you can replay them later.
## Viewing sync errors
Sync errors can be found under the sync run page.
The run summary page provides a quick summary of the types of errors that are
keeping your data from being synced.
For each error, you can see how many events it's affecting, as well as
individual examples of bad events.
## Retrying data
Invalid data is automatically retried for at least 48 hours. If the data issue
has still not been resolved, the invalid events are stored so that they can be
replayed at a later date.
Replaying events is done via the "Archived Failures" screen.
### Fixing column types
To change an existing column type, you can create a temporary version of the
column with the new format.
For example, the following query may used on Snowflake to change the `price`
column on the `product_ordered` table from an `INT` to a `FLOAT`:
```
BEGIN;
ALTER TABLE product_ordered ADD COLUMN price_float FLOAT;
UPDATE product_ordered SET price_float=price;
ALTER TABLE product_ordered DROP COLUMN price;
ALTER TABLE product_ordered RENAME COLUMN price_float TO price;
COMMIT;
```
---
## Warehouse schema
**URL:** https://hightouch.com/docs/events/warehouses/schema
**Description:** How Hightouch structures event data in your warehouse — the tables it creates, how fields become columns, and which timestamps to use.
**Section:** Hightouch Events
Hightouch automatically creates and manages tables for all event types. New
tracked fields are automatically imported into the warehouse without additional
data engineering work.
Use this page to understand which tables Hightouch creates, how event fields become columns, and which timestamps to use for analysis.
All the tables for an event source are written into the same schema in the
warehouse. By default, this schema name is autogenerated based on the event
source. However, the destination schema can be configured in the destination
settings.
## Tables
### Identifies
All `identify` events are written into the warehouse in the `identifies` table.
| Column | Description |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `id` | The `messageId` from the event |
| `anonymous_id` | The anonymous ID from the event |
| `user_id` | The user ID from the event |
| `` | The user traits provided in the `identify` request. Each key in the `traits` object is converted into a separate column. |
| `context_` | Additional context about the request. Often automatically collected by the tracking SDKs. |
| `timestamp` | The clock-skew adjusted timestamp this event occurred. In most cases, this column should be used for timestamp analysis. |
| `received_at` | When Hightouch received the event. Used for calculating `timestamp`. |
| `sent_at` | When the tracking SDK sent the event. Used for calculating `timestamp`. |
| `original_timestamp` | The client-side timestamp for when the tracking SDK sent the event. Used for calculating `timestamp`. |
### Tracks
All `track` events are written into the warehouse in the `tracks` table.
| Column | Description |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `id` | The `messageId` from the event |
| `anonymous_id` | The anonymous ID from the event |
| `user_id` | The user ID from the event |
| `context_` | Additional context about the request. Often automatically collected by the tracking SDKs. |
| `event_text` | The event name. |
| `event` | The snake-cased version of the event name. This matches the table name autocreated by Hightouch. |
| `timestamp` | The clock-skew adjusted timestamp this event occurred. In most cases, this column should be used for timestamp analysis. |
| `received_at` | When Hightouch received the event. Used for calculating `timestamp`. |
| `sent_at` | When the tracking SDK sent the event. Used for calculating `timestamp`. |
| `original_timestamp` | The client-side timestamp for when the tracking SDK sent the event. Used for calculating `timestamp`. |
#### Per-event tables
By default, Hightouch creates a separate table for each Track
event type. When these per-event tables are enabled, the event properties are
written as separate columns within the table.
When per-event tables are turned off, the events `properties` column is added
to the `tracks` table so that custom properties are still accessible.
### Pages
All `page` events are written into the warehouse in the `pages` table.
| Column | Description |
| -------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `id` | The `messageId` from the event |
| `anonymous_id` | The anonymous ID from the event |
| `user_id` | The user ID from the event |
| `context_` | Additional context about the request. Often automatically collected by the tracking SDKs. |
| `` | The page properties provided in the `page` request. Each key in the `properties` object is converted into a separate column. |
| `timestamp` | The clock-skew adjusted timestamp this event occurred. In most cases, this column should be used for timestamp analysis. |
| `received_at` | When Hightouch received the event. Used for calculating `timestamp`. |
| `sent_at` | When the tracking SDK sent the event. Used for calculating `timestamp`. |
| `original_timestamp` | The client-side timestamp for when the tracking SDK sent the event. Used for calculating `timestamp`. |
### Screens
All `screen` events are written into the warehouse in the `screens` table.
| Column | Description |
| -------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `id` | The `messageId` from the event |
| `anonymous_id` | The anonymous ID from the event |
| `user_id` | The user ID from the event |
| `context_` | Additional context about the request. Often automatically collected by the tracking SDKs. |
| `` | The screen properties provided in the `screen` request. Each key in the `properties` object is converted into a separate column. |
| `timestamp` | The clock-skew adjusted timestamp this event occurred. In most cases, this column should be used for timestamp analysis. |
| `received_at` | When Hightouch received the event. Used for calculating `timestamp`. |
| `sent_at` | When the tracking SDK sent the event. Used for calculating `timestamp`. |
| `original_timestamp` | The client-side timestamp for when the tracking SDK sent the event. Used for calculating `timestamp`. |
### Groups
All `group` events are written into the warehouse in the `groups` table.
| Column | Description |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `id` | The `messageId` from the event |
| `anonymous_id` | The anonymous ID from the event |
| `user_id` | The user ID from the event |
| `group_id` | The group ID from the event |
| `context_` | Additional context about the request. Often automatically collected by the tracking SDKs. |
| `` | The group traits provided in the `group` request. Each key in the `traits` object is converted into a separate column. |
| `timestamp` | The clock-skew adjusted timestamp this event occurred. In most cases, this column should be used for timestamp analysis. |
| `received_at` | When Hightouch received the event. Used for calculating `timestamp`. |
| `sent_at` | When the tracking SDK sent the event. Used for calculating `timestamp`. |
| `original_timestamp` | The client-side timestamp for when the tracking SDK sent the event. Used for calculating `timestamp`. |
## FAQ
### How are the warehouse column types determined?
Hightouch determines what types to use for new columns based on the event
values. Event values are categorized into one of the following types:
- String
- Integer
- Float
- Timestamp
- Boolean
Timestamps must be sent as ISO-8601 string.
### What happens if my event types change?
When possible, Hightouch attempts to convert the values in the event payload
into the existing types in the warehouse. For example, if an event contains a
Integer value for a field, but the current type in the warehouse is a String,
the Integer is automatically converted.
If a conversion can't be safely done without losing data, Hightouch doesn't
upload the event, and [retries it for 48 hours](./error-handling). This way, if
you fix the type in the warehouse, Hightouch will automatically sync the event.
### How are complex objects handled?
When tracking events with Objects as custom properties, Hightouch flattens
nested columns into a flat schema.
For example, the following properties:
```json
{
"company": {
"name": "Hightouch",
"location": "San Francisco"
}
}
```
Would get imported into the warehouse as:
| company_name | company_location |
| ------------ | ---------------- |
| Hightouch | San Francisco |
---
## Write events to your warehouse
**URL:** https://hightouch.com/docs/events/warehouses/warehouse
**Description:** Configure an event storage destination so Hightouch can write event data to your warehouse.
**Section:** Hightouch Events
| Audience | How you’ll use this article |
| --------------------- | -------------------------------------------------------------------------------------------- |
| **Marketing teams** | Understand where event data lives and how it becomes available for activation and analytics. |
| **Data teams** | Configure warehouse storage, schema behavior, and table structure for event data. |
| **Engineering teams** | Set up secure warehouse connections and validate end-to-end event delivery. |
## Overview
To write event data collected by Hightouch Events to your warehouse, you must configure an **event storage destination**.
An event storage destination defines:
- **Where** events are written (Snowflake, BigQuery, Redshift, etc.)
- **How** event tables and schemas are created
- **How** event properties and context fields are stored
Once configured, Hightouch automatically creates and manages event tables in your warehouse as new events and fields are received.
Event storage is **optional but strongly recommended** if you want to use
events for analytics, modeling, or warehouse-based activation.
You can also use Hightouch Events in a [**streaming‑only**
mode](/events/event-streaming) to forward events directly to downstream tools
without writing them to the warehouse.
---
## Supported event storage destinations
Hightouch supports the following event storage destinations:
- Snowflake
- BigQuery
- Redshift
- Databricks
- Microsoft Fabric
- PostgreSQL
- Amazon S3
Each destination supports the same core event schema, with destination-specific configuration options.
---
## Step-by-step: Set up an event storage destination
### 1. Add a destination
1. In the left navigation, go to **`Event Collection → Event destinations`**
2. Click **`Add destination`**
---
### 2. Select a destination type
On the **Select a destination** step:
1. Under **Event storage**, choose your warehouse (for example, **Snowflake**)
2. Click the destination tile to continue
---
### 3. Connect the destination
On the **Connect destination** step, enter the required warehouse credentials.
For example, for Snowflake, you’ll configure:
- **Account identifier**
- **Warehouse**
- **Database**
- **Authentication method** (RSA key recommended)
You’ll also upload credentials and validate that Hightouch can connect successfully.
For credential and permission setup, see the destination-specific guides:
The destination docs linked below cover both credential setup and
**model-based sync configuration**.
For **event destinations**, only the credential/permission setup applies. Skip
sections like **Sync configuration**, **Add sync**, and any model mapping or
sync mode instructions.
- [Snowflake](/destinations/snowflake#connect-to-snowflake)
- [BigQuery](/destinations/google-bigquery#connect-to-google-adh-via-bigquery)
- [Databricks](/destinations/databricks#connect-to-databricks)
- [PostgreSQL](/destinations/postgres#connect-to-postgresql)
- [Amazon S3](/destinations/s3#connect-to-amazon-s3)
Click **`Continue`** once the connection details are complete.
---
### 4. Finalize the destination
On the **Finalize** step:
1. Review the destination details
2. Click **`Finish`**
Once finalized, the destination becomes available for event storage.
---
## What happens after setup
After your event storage destination is created and used in an event sync:
- Hightouch begins writing incoming events to your warehouse
- Schemas and tables are created automatically (see: [**Warehouse schema**](/events/warehouses/schema))
- New event fields are added as columns as they appear
You can manage this behavior using the destination’s **Configuration** and **Schema settings** tabs.
---
## Configuration after initial setup
Navigate to:
**`Event Collection → Event destinations → [Select destination]`**
### Configuration tab
The **Configuration** tab contains the core warehouse connection settings, including:
- Account and authentication details
- Warehouse, database, and role configuration
- Credentials and key management
Use this tab if you need to:
- Rotate credentials
- Update authentication methods
- Change the warehouse or database used for writes
---
### Schema settings tab
The **Schema settings** tab controls how Hightouch writes event data into your warehouse.
From here, you can configure:
- **Schema name**
Customize the schema Hightouch uses. By default, this is derived from the event source name.
- **Per-event tables**
When enabled, Hightouch creates a separate table for each `track` event type.
When disabled, all track events are written to a shared `tracks` table.
- **Convert properties and context fields to columns**
When enabled, event properties and context fields are written as individual columns.
When disabled, they are stored as JSON.
- **Write Page and Screen events to the Tracks table**
Useful if you want all user interactions in a single table.
These settings apply to all events written to this destination.
---
## Validating event storage
After setup, you should:
1. Send test events from an event source
2. Confirm tables and schemas appear in your warehouse
3. Verify event fields match your tracking spec
You can also inspect events in the debugger before they’re written to storage.
→ See [**Warehouse schema**](/events/warehouses/schema) for a full breakdown of tables and columns.
---
## Next steps
- Understand how events are written to your warehouse: [Warehouse schema](/events/warehouses/schema)
- Review retries, failures, and archived events: [Error handling](/events/warehouses/error-handling)
- Forward events to downstream tools: [Event streaming](/events/event-streaming)
---
## Trigger syncs with Airflow
**URL:** https://hightouch.com/docs/extensions/airflow
**Description:** Learn how to trigger syncs with Hightouch's Airflow Operator.
**Section:** Extensions
## Overview
Hightouch provides an [Airflow Operator](https://airflow.apache.org/docs/apache-airflow/stable/concepts.html#operators) for triggering syncs via Airflow. The [Git repository](https://github.com/hightouchio/airflow-provider-hightouch) contains the latest code as well as [example DAGs](https://github.com/hightouchio/airflow-provider-hightouch#examples) to help you get started.
## Setup
### Create an API key
The first step is to create a Hightouch API key in your Hightouch workspace settings.
### Add the Airflow connection
1. In Airflow, create a new connection by going to **Admin > Connections** and clicking the **+** button to add a new connection.
2. Enter `hightouch_default` as the **Conn ID**, select **HTTP** as the **Conn Type**, and set the host to `https://api.hightouch.com`. Optionally, enter a **Description**.
3. Enter the **API key** you created in the **Password** field.
### Set sync schedule type
### Install the Airflow package
While the details will vary for your particular Airflow installation, the simplest way to install our package is through pip.
```bash
pip install airflow-provider-hightouch
```
### Add the HightouchTriggerSyncOperator to your DAG
The last step is to add the operator to a DAG and enter the sync ID of the sync you want to trigger. When the task is run, Airflow sends a call to the Hightouch API to trigger a run which completes asynchronously.
Alternatively, you can choose to send a synchronous request and await the results of the sync before completing the task.
For a full description of the operator parameters, [view the source code in the GitHub repository](https://github.com/hightouchio/airflow-provider-hightouch/blob/main/airflow_provider_hightouch/operators/hightouch.py).
```python
from airflow_provider_hightouch.operators.hightouch \
import HightouchTriggerSyncOperator
with DAG(....) as dag:
...
# This task submits the request but doesn't wait for completion of the sync before completing.
my_async_task = HightouchTriggerSyncOperator(
task_id="run_my_sync", sync_id=1234
)
# This task waits for the sync status to complete or error before completing.
# Warnings can be treated as errors or successes depending on your use case.
my_sync_task = HightouchTriggerSyncOperator(
task_id="run_sync", sync_id=1234, synchronous=True, error_on_warning=True
)
```
---
## Connections Overview
**URL:** https://hightouch.com/docs/extensions/connections
**Description:** Learn how to use Connections in Hightouch to import data from third-party applications into your data warehouse.
**Section:** Extensions
| | |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Data engineers, analysts, and technical marketers |
| **Prerequisites** |
Access to a supported data warehouse (e.g., Snowflake, BigQuery, Databricks, or Postgres)
Admin or API access to the external application you plan to connect
|
_Use Connections to bring data from external tools--such as Salesforce, Klaviyo, or Braze--into your data warehouse for analysis and activation._
---
## Learning Objectives
After reading this article, you will be able to:
- Understand when and why to use Connections.
- Create and configure a new Connection in Hightouch.
- Select which tables and columns to import using the schema picker.
- Schedule or manually run Connection syncs.
- Manage and monitor existing Connections from your workspace.
---
## Overview
Connections let you bring data from third-party applications, such as Salesforce, Klaviyo, or Braze, into your data warehouse.
While [destinations](/destinations/overview) send data _out_ of Hightouch to external tools, **Connections** bring data _in_. This allows you to centralize and analyze information from multiple systems within your warehouse, supporting downstream workflows in Hightouch.
You can use Connections to:
- Import data from marketing, sales, and engagement platforms
- Enrich warehouse data with external attributes and activity
- Support analysis, segmentation, and activation across Customer Studio and other Hightouch features
Each Connection uses a secure API or OAuth authentication flow and provides schema controls so you can decide which data to include and how it’s stored.
---
## Prerequisites
Before creating a Connection, make sure you have:
- **Access permissions** for both Hightouch and your target application (for example, admin-level or API key access)
- **Warehouse permissions** to write tables or manage schemas
- **API credentials** (if required by your integration). Some tools, like Klaviyo or Iterable, require API keys while others authenticate through OAuth.
---
## Create a Connection
### Access Connections
To create or manage Connections, go to **Data Management → Connections** in your Hightouch workspace or visit [app.hightouch.com/connections](https://app.hightouch.com/connections?from=docs).
From this page, you can view existing connections or click **Add connection** to start a new one.
---
### 1. Select a data source
Choose the external application you want to import data from.
---
### 2. Select a warehouse
Choose the warehouse where you want to store your imported data.
---
### 3. Configure connection
Authenticate your selected tool using OAuth or API credentials. Read documentation for your specific [destination](/destinations/overview) for detailed instructions.
After entering your credentials, click **Test connection** to verify that Hightouch can connect successfully before continuing.
---
### 4. Manage schema
After connecting, Hightouch will detect the schema from your external application.
Use the **visual schema picker** to select which tables and columns to import into your warehouse.
---
### 5. Finalize and schedule
Before completing setup, decide how schema changes should be handled over time:
- **Sync all:** Include all new schemas, tables, and columns automatically
- **Sync new columns only:** Add new columns to existing tables but skip new tables or schemas
- **Do not sync changes:** Keep your schema fixed and ignore new data structures
Then, choose how and when your Connection should sync:
- **Manual:** Run the sync manually in the app or through the API
- **Interval:** Schedule syncs to run at a set interval (for example, every hour)
- **Custom recurrence or Cron expression:** Define your own schedule
- **dbt Cloud or Fivetran:** Trigger syncs automatically after jobs finish in these tools
When finished, click **Finish** to complete setup.
For integration-specific setup instructions, such as authentication details,
required permissions, or field mappings, see the relevant [Destination
documentation](/destinations/overview) for your tool.
---
## Manage and Edit Connections
After creating a Connection, you can manage it from the **Connections** page in your workspace.
From here, you can:
- View recent connection runs and their status
- Edit how schema changes are handled
- Update selected tables and columns when new fields become available
- Modify your sync schedule
Hightouch tracks connection activity and run results automatically, so you can monitor data freshness and resolve issues directly from the UI.
---
## Related articles
- Learn more about integration setup in [Destination documentation](/destinations/overview)
- Visit [Alerting](/syncs/alerting) to configure sync alerts for your Connections
- Review the [Syncs Overview](/syncs/overview) to understand how incoming data supports downstream activation
---
## Trigger syncs with Dagster
**URL:** https://hightouch.com/docs/extensions/dagster
**Description:** Learn how to trigger syncs and gain visibility into sync execution trends with Hightouch's Dagster extension.
**Section:** Extensions
## Overview
Hightouch provides a Dagster extension to trigger syncs and gain visibility into sync execution trends. The [Git repository](https://github.com/hightouchio/dagster-hightouch) contains the latest code.
## Setup
### Create an API key
The first step is to create a Hightouch API key in your Hightouch workspace settings.
### Install library
The simplest way to install the Hightouch Dagster extension is by using `pip`.
```bash
pip install dagster-hightouch
```
### Configure library
To configure the library, provide your Hightouch API key as a resource in the `resources.py` file:
```python
# resources.py
from dagster_hightouch.resources import ht_resource as hightouch_resource
ht_resource = hightouch_resource.configured(
{"api_key": "555555-4444-3333-2222-1111111111"},
)
```
### Set sync schedule type
### Schedule syncs
You can schedule syncs by using the `hightouch_sync_op` function with the sync IDs of the syncs you want to schedule.
This example creates a job to trigger two syncs—one for Salesforce account objects and one for organization objects—every thirty minutes.
```python
# The hightouch_sync_op calls the Hightouch API to trigger a sync, and then polls the API until the sync either completes or returns an error.
from dagster import ScheduleDefinition, get_dagster_logger, job
from dagster_hightouch.ops import hightouch_sync_op
from .resources import ht_resource
# Set sync IDs as constants
SFDC_ACCOUNTS_SYNC_ID = "23620"
SFDC_ORGS_SYNC_ID = "39619"
# Define configured sync ops
run_ht_sync_accounts = hightouch_sync_op.configured(
{"sync_id": SFDC_ACCOUNTS_SYNC_ID}, name="hightouch_sfdc_accounts"
)
run_ht_sync_orgs = hightouch_sync_op.configured(
{"sync_id": SFDC_ORGS_SYNC_ID}, name="hightouch_sfdc_organizations"
)
# Create a job with the defined resources, specifying the dependencies
@job(
resource_defs={
"hightouch": ht_resource,
}
)
def ht_sfdc_job():
ht_orgs = run_ht_sync_orgs(start_after=ht_contacts)
run_ht_sync_accounts(start_after=ht_orgs)
# Schedule it to run as often as you like with a cron expression
# This example shows every 30 min
every_30_schedule = ScheduleDefinition(job=ht_sfdc_job, cron_schedule="*/30 * * * *")
```
To learn more, check out the [blog post](https://hightouch.com/blog/introducing-the-hightouch-dagster-integration) about the extension.
---
## Datadog
**URL:** https://hightouch.com/docs/extensions/datadog
**Description:** The Datadog extension allows you to receive real-time metrics and events about your syncs
**Section:** Extensions
## Overview
Hightouch integrates with Datadog to help you gain greater observability into your syncs and monitor their health. Once you've authorized access to Datadog in Hightouch, we send real-time sync metrics and events to your Datadog account. You can then aggregate these statistics to gain insight into the health of your syncs.
## Setup
You need your [Datadog site](https://docs.datadoghq.com/getting_started/site/) and [API key](https://docs.datadoghq.com/account_management/api-app-keys/) to set up the extension. You may want to create a new API key for Hightouch. Make sure that the API key creator in Datadog has the necessary write permissions for [custom metrics](https://docs.datadoghq.com/metrics/custom_metrics/) (`Metric Tags Write`, `Host Tags Write`, `Metrics Metadata Write`) and [events](https://docs.datadoghq.com/events/) (`Event Config Write`).
Make sure to use the **API Key** itself and not the Key ID. Also ensure you use an API Key and not an Application Key to avoid the `Datadog configuration is invalid` error. You can find more information about the difference between these keys in [Datadog's documentation](https://docs.datadoghq.com/account_management/api-app-keys/).
1. Go to the [**Monitoring** section](https://app.hightouch.com/extensions/monitoring?from=docs) of the **Extensions** page and select **Configure extension**.
2. Select your Datadog [**site**](https://docs.datadoghq.com/getting_started/site/) and enter your **API Key**.
3. Click **Save**.
Once you've saved your Datadog credentials in Hightouch, your syncs automatically start emitting metrics and events to Datadog.
Hightouch metrics and events don't immediately show up in Datadog, but only once sync runs are triggered.
## Emitted metrics and events
Hightouch emits both [metrics](https://docs.datadoghq.com/metrics/types/) and [events](https://docs.datadoghq.com/events/). Metrics are best suited for dashboards and statistical aggregations, while events are best for filtering. Don't hesitate to if there's another metric or event you're interested in using.
### Metrics
#### `hightouch.sync.row_processed`
This metric is a [`COUNT`](https://docs.datadoghq.com/metrics/types/?tab=count) type metric and is incremented for each row processed in the sync.
It provides the following tags:
- `sync` - The sync slug in Hightouch.
- `workspace` - The slug of the Hightouch workspace that contains the sync.
- `op` (added/changed/removed) - The operation performed: whether the row was added, changed, or removed in the query.
- `success` (true/false) - Whether the row was successfully synced to the destination.
#### `hightouch.sync.complete`
This metric is a [`COUNT`](https://docs.datadoghq.com/metrics/types/?tab=count) type metric and is incremented each time the sync completes.
It supports the following tags:
- `sync` - The sync slug in Hightouch.
- `workspace` - The slug of the Hightouch workspace that contains the sync.
- `status` (success/failed/warning/cancelled) - The overall status of the sync run.
#### `hightouch.sync.query_time`
This metric tracks your syncs' querying times.
It supports the following tags:
- `sync` - The sync slug in Hightouch.
- `workspace` - The slug of the Hightouch workspace that contains the sync.
#### `hightouch.sync.execution_time`
This metric tracks your syncs' execution times.
It supports the following tags:
- `sync` - The sync slug in Hightouch.
- `workspace` - The slug of the Hightouch workspace that contains the sync.
#### `hightouch.sync.total_time`
This metric tracks your syncs' total run times.
It supports the following tags:
- `sync` - The sync slug in Hightouch.
- `workspace` - The slug of the Hightouch workspace that contains the sync.
### Events
#### `hightouch.sync.complete`
Hightouch emits this event each time a sync completes. It's identical to the metric of the same name.
It supports the following tags:
- `sync` - The sync slug in Hightouch.
- `workspace` - The slug of the Hightouch workspace that contains the sync.
- `status` (success/failed/warning/canceled) - The overall status of the sync run.
## Use cases
With just a few metrics, you can build complex dashboards and monitors. For example, you can create dashboards that target specific syncs that are critical to your operations and report the total number of operations or top syncs that have warnings or failures.
You can [import the JSON](https://docs.datadoghq.com/dashboards/#copy-import-or-export-dashboard-json) for a basic dashboard using this [sample code](https://gist.github.com/PedramNavid/71f73c98d98373053674f19b8239b06c) to get started.
### Completed sync status
One common use case is to build a basic dashboard that tracks completed syncs by status over time.
The preceding chart uses the [`hightouch.sync.complete` metric](#hightouchsynccomplete) and sums the counts by `status`. You can filter further by including only statuses that are `status:success` or `NOT status:success`.
### Anomaly detection
Anomaly detection can help flag significant changes, like many users suddenly being deleted or created. You can use the Datadog extension for anomaly detection in added, changed, or removed rows.
#### Anomaly detection setup
1. In Datadog, create a [metric monitor](https://docs.datadoghq.com/monitors/create/types/metric/) with the **Anomaly Detection** method.
2. Define the metric by selecting the [`hightouch.sync.row_processed`](##hightouchsyncrow_processed) metric. Specify the `op_type`, for example, `removed`, and `sync` name. Select the metric aggregation as `sum by` and enter `sync`.
3. In [**Alert Conditions**](https://docs.datadoghq.com/monitors/create/types/metric/?tab=threshold#set-alert-conditions), set the **Alert threshold** and **Warning threshold** to values that make sense for your use case.
4. (Optional) You can inject the sync name into the monitor's subject or message body using `{{sync.name}}`.
### Other use cases
You're not limited to the preceding examples. Here are some other use cases to consider:
- Metric monitors for when the number of failed rows exceeds a threshold
- Metric monitors for when a critical sync hasn't run over a specified time period
---
## dbt Cloud - Trigger syncs and sync models
**URL:** https://hightouch.com/docs/extensions/dbt-cloud
**Description:** The dbt Cloud extension lets you both schedule Hightouch syncs to run as soon as a dbt Cloud job completes, and to sync models from dbt into Hightouch. Relying on dbt Cloud for sync scheduling ensures that Hightouch performs syncs when fresh data is available.
**Section:** Extensions
This extension requires a paid Hightouch plan
dbt requires you to have a paid plan to use this extension
The dbt Cloud extension allows you to:
- Schedule Hightouch syncs to run as soon as a dbt Cloud job completes. Relying on dbt Cloud for sync scheduling ensures that Hightouch performs syncs when fresh data is available.
- Sync models from dbt into Hightouch
Extension setup requires connecting your dbt account to Hightouch.
## Connect to dbt Cloud
Open the [dbt Cloud configuration page](https://app.hightouch.com/extensions/dbt-cloud/configuration?from=docs) in your workspace and enter the **dbt Cloud URL**. Depending on your setup, you need to insert:
- for regular dbt Cloud setups, a [default access URL](https://docs.getdbt.com/docs/cloud/about-cloud/access-regions-ip-addresses)
- for accounts with cell-based account prefixes, you can find the custom [unique access URL](https://docs.getdbt.com/docs/cloud/about-cloud/access-regions-ip-addresses#api-access-urls) in dbt on the **Account settings** page (below the **Account information** pane)
These are the most commonly used default access URLs:
- `https://cloud.getdbt.com`, for the US
- `https://emea.getdbt.com`, for EMEA
Then navigate to dbt Cloud and generate a [service account token](https://docs.getdbt.com/docs/dbt-cloud-apis/service-tokens) or a [personal access token (PAT)](https://docs.getdbt.com/docs/dbt-cloud-apis/user-tokens). In Hightouch, enter your token in the **dbt Cloud API token** field in the **Configuration** tab of the dbt Cloud extension. Click **Connect** to complete your connection configuration.
A **Member-level or higher** [service account token](https://docs.getdbt.com/docs/dbt-cloud-apis/service-tokens#team-plans-using-service-account-tokens) is required.
Read-only tokens are no longer supported because the dbt Cloud integration uses webhooks, which require elevated permissions.
If you previously used a read-only dbt Cloud token, you must update to a **Member** token (or higher). Without this change, Hightouch syncs triggered by dbt Cloud will fail.
To update your dbt Cloud API token in Hightouch, edit your credentials within the extension and set a new API token. This won't reset the schedule configuration of syncs that use the [dbt Cloud scheduling](#configure-your-sync-schedule) method, nor are any synced models removed.
## Configure sync schedules
When creating or modifying a sync you can select **dbt Cloud** as a schedule type.
1. Go the **Schedule** tab of overview page of the sync to schedule. Select **dbt Cloud** as the schedule type.
2. Select the appropriate dbt **Account** and **Job**.
3. Click **Save**.
Your sync is now scheduled to run whenever your chosen dbt Cloud jobs complete.
The sync won't get triggered if your dbt Cloud job fails.
## dbt Cloud model sync
To sync models from dbt Cloud to Hightouch, the extension requires a **Member-level or higher** service account token as described
in the preceding section. A webhook ID should display next to your account status.
### Connecting a source
On the [extension configuration page](https://app.hightouch.com/extensions/dbt-cloud/configuration?from=docs) there is a list of sources from your workspace within Hightouch that can be
connected to dbt Cloud. Choose one that you would like to configure and click "Connect to dbt."
Configure the dbt Cloud account and job from the dropdowns, then choose "Create Configuration"
Select a dbt job that has a "Compile" step (`compile`, `build`, `run`) or no model artifacts get produced.
### Syncing models
Once configured, the "dbt to Hightouch" status shows "Waiting to be synced." If your job in dbt Cloud has previous
runs you can click the "refresh" button to manually sync models from the job's result. Otherwise, run the job within dbt
Cloud and the result should trigger the update in Hightouch.
After a successful run, dbt Cloud models are available within Hightouch.
A dbt Cloud job can fail and still produce model artifacts which sync to Hightouch.
### dbt Cloud model sync advantages
Along with the [advantages of using dbt models](/extensions/dbt-models#dbt-model-advantages) that dbt Core brings,
dbt Cloud brings more helpful features:
- dbt Mesh: reference models in other projects (`{{ ref('customers', 'contact_info') }}`)
- "Versionless" (now called "Latest") - always runs your compile job with the latest dbt version
- Private dependencies
- Database adapters that Hightouch doesn't support for dbt Core (such as Fabric)
---
## Create models using dbt models and analyses
**URL:** https://hightouch.com/docs/extensions/dbt-models
**Description:** Connect Hightouch to your dbt project's Git repository and build Hightouch models using existing dbt models and analyses.
**Section:** Extensions
Now you can sync models from [dbt Cloud](/extensions/dbt-cloud).
This extension requires a paid Hightouch plan
Hightouch can communicate with two distinct Git repository types. The dbt extension interacts with Git repositories that contain dbt projects. These are separate from the Git repository used for [Git Sync](/extensions/git-sync) to version control your Hightouch settings.
If you've invested in building models with [dbt](https://www.getdbt.com/), you likely want to use those models as the source of truth for your downstream data needs. At Hightouch, we use dbt to build most of our models for analytics and reporting.
Our dbt integration lets you define Hightouch models using your dbt models and [analyses](https://docs.getdbt.com/docs/build/analyses).
## Supported dbt versions
Hightouch supports dbt Core 1.10 and newer. You can set the version on the dbt setup page. If you need support for a version not listed, .
If you update your dbt version, return to Hightouch to update your dbt version selection. Once you've made the update, it may take up to 15 minutes for the changes to take effect.
## Connect to your Git repository
Navigate to the [dbt extension configuration](https://app.hightouch.com/extensions/dbt-models/configuration?from=docs) page and enter your Git credentials under **Credentials**.
If you're using GitHub, it's best to use [Hightouch's GitHub App](https://github.com/apps/hightouch-connect/) for the most up-to-date feature support.
To set it up, choose **GitHub App** as the **Git Service**, and then select **Configure GitHub App**.
This prompts you to install the Hightouch Connect app in the relevant GitHub repository.
For other Git services, you can choose to use either **SSH** or **HTTP** protocol. SSH protocol requires you provide an **SSH private key** and HTTP requires a **username** and **token**.
See [GitHub documentation](https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/creating-a-personal-access-token) or [GitLab documentation](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) for instructions on creating personal access tokens.
## Connecting a source
You can use dbt models and analyses from your Git repository for any [source](https://app.hightouch.com/sources?from=docs) in Hightouch. On the [extensions configuration page](https://app.hightouch.com/extensions/dbt-models/configuration?from=docs), there is a list of sources from your workspace within Hightouch that can be connected to dbt models. Choose one that you would like to configure and click "Connect to dbt."
## Configure dbt model selector
When connecting dbt sources for the first time or making changes to your dbt model sync configuration, you usually only need to click **Save** and not **Full Resync** for Hightouch to create or update your configuration.
The **Full Resync** action is only necessary when making significant changes in your dbt Git repository. It restarts the dbt compilation from a fresh state and can take a significant amount of time. After clicking this button, wait for the updated dbt models to compile before making any changes to your configuration.
The default schema is the schema where dbt [materializes](https://docs.getdbt.com/docs/build/materializations) your tables to unless otherwise specified in project file. Some common examples are `public`, `production`, `dbt_production`.
Hightouch lets you optionally specify [dbt selectors](https://docs.getdbt.com/reference/node-selection/syntax#examples) like `tag:hightouch`. Otherwise, we use `*` and sync _all_ dbt models to Hightouch.
You can also specify which version of dbt you want to use. By default, Hightouch uses the most recent version available in the drop-down menu.
Hightouch's dbt integration automatically looks for your `dbt_project.yml` file in the repository directory. If your project file is named something else, for example, `dbt_project.yaml`, or you have multiple `dbt_project.yml` files, you can specify a relative path to your project file, for example, `./hightouch/dbt/dbt_project.yaml`.
After saving your settings, it may take a one to two minutes for Hightouch to sync your dbt models and for the sync status to turn green.
## Select a model
Once you've configured the dbt model selector, you can use dbt models and analyses in the Hightouch [model creation process](/models/creating-models). Select the dbt model tab in the query explorer to view all your available dbt models and analyses.
The compiled SQL shown in the preview is what's ran against your source. We do not leverage the output table written by dbt.
### Model column descriptions
Hightouch pulls in column descriptions from dbt.
You can see them in the **Columns** tab of a model's overview page.
## Unsupported features
Hightouch doesn't support the following features:
- Private dependency packages
- Pre- and post-hooks (limited support, this may work depending on the hook)
- [Python models](https://docs.getdbt.com/docs/build/python-models)
Hightouch compiles the project using `dbt compile` to resolve your data warehouse's schema and generate compiled SQL for preview purposes. When running the model in Hightouch, we query the materialized table directly.
## dbt model advantages
When deciding between Hightouch's [table selector](https://hightouch.com/docs/models/table-selector) and our [dbt model selector](https://hightouch.com/docs/extensions/dbt-models), the advantages of leveraging dbt models might not be immediately clear, since many dbt models are often materialized as tables and views in a [Source](https://hightouch.com/docs/getting-started/concepts#sources).
However, there are a few key benefits of building Hightouch Models using the dbt model selector:
- It provides a clean, information-rich modal with metadata about the underlying model, including its unique ID, description, and repository link.
- It allows you to automatically pull in [column descriptions](https://hightouch.com/docs/extensions/dbt-models#model-column-descriptions) into Hightouch for data governance/organization.
- It allows you to leverage our [dbt observability layer](https://hightouch.com/docs/extensions/dbt-observability), which includes dbt exposures and CI checks.
- Not all dbt resources materialize as tables/views in the warehouse. For example, dbt analyses will not form tables/views at all.
## dbt observability layer
Hightouch lets you [validate dbt model changes](/extensions/dbt-observability#dbt-ci-checks) as they relate to Hightouch syncs and observe [how Hightouch syncs are using your dbt models](/extensions/dbt-observability#exposure-syncs). Check out our [dbt observability layer documentation](/extensions/dbt-observability) for more information.
## Tips and troubleshooting
Hightouch compiles the project using [`dbt compile`](https://docs.getdbt.com/reference/commands/compile) to resolve your data warehouse's schema and generate compiled SQL for your model. If you encounter error messages on your dbt extension configuration page, try running [`dbt compile`](https://docs.getdbt.com/reference/commands/compile) locally on your dbt project to see if the errors still occur.
### Credential errors
If your Git credentials are out of date, you won't be able to connect your dbt models and analyses.
We recommend using [Hightouch's GitHub app](#connecting-to-your-git-repository) for your credentials if you're using GitHub.
### Duplicated mapping key error
You may receive a `duplicated mapping key` error under your dbt model sync status if there are duplicate keys in your `dbt_project.yml` file.
To resolve the error, you can either remove the duplicates or rename duplicated keys.
### The YAML property file is missing a version tag
This dbt compilation error can occur if you use the same Git repository for [Git Sync](/extensions/git-sync) and [dbt models sync](#configure-dbt-model-selector).
To solve this, make sure that you're using two different Git repositories for these extensions.
Otherwise, if you would like to use the same repository, you can specify a custom path in the [Git Sync configuration](/extensions/git-sync#configure-git-sync).
---
## dbt observability layer
**URL:** https://hightouch.com/docs/extensions/dbt-observability
**Description:** See which Hightouch syncs are using your dbt models from your dbt docs and lineage graph.
**Section:** Extensions
Hightouch provides two separate features to monitor your dbt models as they relate to Hightouch syncs:
- [dbt CI checks](#dbt-ci-checks)
- [dbt exposure sync](#dbt-exposure-sync)
You can use either one or both of these features together. Both require that you've [set up dbt-based models](/extensions/dbt-models) in Hightouch.
## dbt CI checks
The dbt CI checks feature doesn't currently work with [Customer Studio](/customer-studio/overview) models.
You can use Hightouch's **dbt CI checks** to validate changes in your dbt project Git repository. This feature checks your GitHub pull requests to ensure they don't break any existing, [enabled](/syncs/overview#individual-sync-overview-page) dbt-based syncs in Hightouch.
To enable this, toggle on **GitHub Pull Request checks** from your dbt connected source's [configuration page](https://app.hightouch.com/extensions/dbt-models/configuration?from=docs) and then select **Save**. You need to do this for every connected source you want to enable CI checking on.
Once enabled, Hightouch runs a CI check on any pull request made to the dbt project's Git repository. If a breaking change is detected—for example, deleting, renaming, or moving a model used in a sync—the check fails.
Clicking into the details of a failed CI check shows which sync the breaking change effects.
The CI check passes if the pull request doesn't affect any [enabled](/syncs/overview#individual-sync-overview-page) Hightouch syncs.
Though dbt CI checks work similarly to [version control with Git
Sync](/extensions/git-sync#validating-changes), these are separate features
and aren't dependent on one another.
## dbt exposure sync
Hightouch exposure sync lets you see which Hightouch syncs are using your dbt models via your dbt docs and lineage graph.
Exposure sync works by defining and updating exposures in your chosen dbt repository anytime a Hightouch sync that uses a dbt model is added or modified.
### Turn on exposure sync
You can enable exposure sync by navigating to **dbt Models** on the [**Extensions page**](https://app.hightouch.com/extensions?from=docs) and selecting your **dbt Connected Source**. From there, activate the **dbt Exposures Sync** toggle and click **Save**.
You should see exposures starting to sync within a few minutes.
In your dbt repository, you should see a new file called `hightouch_sync_exposures.yml`. This file updates every time a Hightouch sync that uses a dbt model is added, modified or disabled.
Exposures will be created for syncs using audiences created with Customer Studio where the [parent model](/customer-studio/schema#define-the-parent-model) is a dbt model. If an audience references [related or event models](/customer-studio/schema#related-models-and-events) that are built with dbt models, this information won't be included in the exposure file. If you'd like to request this functionality, please .
## Tips and troubleshooting
### Git credentials
If your Git credentials are out of date, you won't be able to connect your dbt models and analyses. We recommend using [Hightouch's GitHub app](https://github.com/apps/hightouch-connect/) for your credentials if you're using GitHub.
The Hightouch GitHub app must have commit access to your chosen dbt branch to create exposures. If Hightouch is unable to push to your branch you may have to update your branch protection rules in your dbt GitHub repository.
### dbt build errors
The Hightouch exposures file is intended to stop you from breaking Hightouch syncs as a result of deleted dbt models. If you need to delete a model defined in the Hightouch exposures file, you can remove the Hightouch exposures file at any point—Hightouch will automatically recreate it.
### Your local changes to the following files would be overwritten by checkout
This error generally occurs if the **exposure sync branch** in your linked Git repository has diverged from your main branch.
Once you update your exposure branch so that it's up-to-date with your main one, it should resolve the error. You can find more information on how you can resolve this type of error in this [Stack Overflow article](https://stackoverflow.com/questions/22424142/error-your-local-changes-to-the-following-files-would-be-overwritten-by-checkou).
### Remote rejected, protected branch hook declined
You may receive this error if you have [branch protections](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches) on the **branch** you selected for you [exposure sync configuration](#turn-on-exposure-sync).
To avoid this, you can select a branch to use for exposure syncs specifically.
Then, you can open a pull request from the dbt exposure sync branch to your main dbt branch.
This allows the exposure sync with Hightouch to operate smoothly while also allowing branch protection rules to be applied to the main branch.
You can read more about a similar error in [this section](/extensions/git-sync#protected-branch-update-failed) of the Git Sync page.
---
## Trigger syncs with Fivetran
**URL:** https://hightouch.com/docs/extensions/fivetran
**Description:** The Fivetran extension lets you schedule Hightouch syncs to run as soon as a Fivetran job completes. Relying on Fivetran dbt transformations and connectors for sync scheduling ensures that Hightouch performs syncs when fresh data is available.
**Section:** Extensions
The Fivetran extension lets you schedule Hightouch syncs to run as soon as a Fivetran job completes. Relying on Fivetran dbt transformations and connectors for sync scheduling ensures that Hightouch performs syncs when fresh data is available.
Extension setup requires two steps:
- Connecting your Fivetran account to Hightouch
- Configuring your Hightouch syncs to trigger with Fivetran
## Connect to Fivetran
Navigate to your Fivetran dashboard and [retrieve your API Key and Secret](https://fivetran.com/docs/rest-api/getting-started#gettingstarted).
The API Key and Secret must belong to a user with the [Account Administrator role](https://fivetran.com/docs/getting-started/fivetran-dashboard/account-management/role-based-access-control#rolesinourrbacmodel) or you'll receive [an error](#forbidden-access-denied-error) when trying to save your configuration.
In Hightouch, enter your credentials in the [Fivetran configuration page](https://app.hightouch.com/extensions/fivetran/configuration?from=docs) and click **Connect**.
Behind the scenes, Hightouch adds a new [signed](https://fivetran.com/docs/rest-api/webhooks#signing) webhook to your Fivetran account.
## Configure your sync schedule
Now, when creating or modifying a sync you can select **Fivetran** as a schedule type.
1. Go the **Schedule** tab of overview page of the sync you want to schedule. Select **Fivetran** as the schedule type.
2. Select the appropriate **Fivetran workspace**.
3. Select one or more Fivetran connectors or Fivetran-managed dbt Core transformations as the **Fivetran jobs** to trigger the sync.
3. Click **Save**.
Your sync should is now scheduled to run whenever you chosen Fivetran jobs complete.
## Tips and troubleshooting
### Forbidden access denied error
Ensure your Fivetran account has the [Account Administrator role](https://fivetran.com/docs/getting-started/fivetran-dashboard/account-management/role-based-access-control#rolesinourrbacmodel).
Otherwise, you might encounter this error when saving your Fivetran configuration in Hightouch:
`Forbidden access denied. Required permission: ACCOUNT_SECURITY_MANAGE`
---
## Git Sync
**URL:** https://hightouch.com/docs/extensions/git-sync
**Description:** Version-control supported Hightouch resources in a Git repository.
**Section:** Extensions
{/* */}
Git Sync connects your Hightouch workspace to a Git repository so you can version-control supported Hightouch resources.
With Git Sync, you can:
- Create and edit multiple Hightouch models and syncs in code
- Review changes through Git commits and file diffs
- Roll resources back to a previous version
- Keep staging and production workspaces aligned through shared YAML files
Git Sync is bidirectional:
- Changes made in the Hightouch UI are written to YAML files in Git. These changes are marked as `OUTBOUND`.
- Changes made to YAML files in Git are applied back to Hightouch. These changes are marked as `INBOUND`.
Each update to a supported resource creates a commit in your configured repository.
Git Sync currently supports models (including Customer Studio schema models), syncs, and event contracts. Git Sync does not currently support Customer Studio audiences or syncs created from those audiences.
Inbound changes made in your Git Sync repository don't require approval from
another team member, even if your workspace uses [approval
flows](/workspace-management/approval-flows).
Git Sync supports most Git providers, including GitHub, Bitbucket, GitLab, and private [on-premise](#tunneling) repositories. In most cases, changes sync between Hightouch and Git in about one minute.
Hightouch can connect to Git repositories for two different purposes.
Git Sync uses a Git repository to version-control Hightouch resources. This is separate from Git repositories used to [leverage dbt models](/extensions/dbt-models).
---
## Setup
To set up Git Sync, complete these steps:
1. Create or choose a Git repository.
2. Authenticate Hightouch with your Git provider.
3. Configure Git Sync in Hightouch.
### Create or choose a Git repository
Use a Git repository to store the YAML files that represent your supported Hightouch resources.
If you are creating a new repository, GitHub repositories are the easiest to start with because they can use the Hightouch GitHub app.
If you create an empty repository, make an initial commit, such as adding a README file. This creates the branch Hightouch needs to connect to.
### Authenticate to Git
After you have a repository, configure your Git credentials in Hightouch.
1. Go to the [Git Sync configuration page](https://app.hightouch.com/extensions/git-sync/configuration?from=docs).
2. Under **Configuration > Git credentials**, select **Set up**.
3. Choose your Git provider and authentication method.
If you use GitHub, we recommend using the [Hightouch GitHub app](https://github.com/apps/hightouch-connect/) for the most up-to-date feature support.
Regardless of provider, grant Hightouch access only to the repository or repositories you want to use with Git Sync.
#### Authenticate to a GitHub Enterprise repository
If you use GitHub Enterprise with a self-hosted server:
1. Under **Git service**, select **Other**.
2. For **Username**, enter your full GitHub Enterprise username. For example, `hightouch123@github.hightouch.com`.
3. For **Token**, enter a [GitHub access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) tied to that username.
4. Make sure the token has access to the repository or repositories Hightouch should use.
#### Authenticate to an AWS CodeCommit repository
If you use AWS CodeCommit, you must use SSH to authenticate.
1. Configure your AWS CodeCommit repository for SSH by following [AWS's setup instructions](https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-ssh-unixes.html).
2. In Hightouch, select **Other** as the **Git service**.
3. Select **SSH** as the **Protocol**.
4. Under **SSH Private key**, enter the private key you generated when configuring AWS CodeCommit.
5. Select **Save**.
6. For **Repository**, enter the full SSH URL, including the AWS SSH key ID.
The AWS SSH key ID is different from the general access keys in your AWS account.
You can copy your URL from these [instructions in AWS's docs](https://docs.aws.amazon.com/codecommit/latest/userguide/how-to-connect.html).
### Configure Git Sync
After you authenticate to Git, enable Git Sync for your workspace.
1. Go to [Integrations > Extensions](https://app.hightouch.com/extensions/git-sync/configuration?from=docs).
2. Open the **Version control with Git** extension.
3. On the **Configuration** tab, enable **Git Sync**.
4. Select the **Repository** and **Branch** Hightouch should read from and write to.
5. Optionally, enter a **Path** where Hightouch should create folders for Git Sync files.
6. Select **Save**.
If you enter a path, Hightouch creates the `models`, `syncs`, and other supported folders inside that subfolder. If you don't enter a path, Hightouch saves Git Sync files in the root of your repository.
After you save the configuration, Hightouch creates the configured folder structure if needed. Git Sync then starts syncing supported resources between Hightouch and Git.
To start the syncing process immediately, select **Full resync**.
This pushes all supported workspace resources to Git, then pulls changes from Git back to your workspace. Because this happens in that order, a full resync can overwrite Git changes that have not synced to Hightouch yet.
For more information, see [What does the Full resync button do?](#what-does-the-full-resync-button-do).
Git Sync does not support deletions from Git.
If you delete a YAML file in Git, the corresponding model or sync remains in Hightouch. The file is not recreated in Git until the next change is made in Hightouch or you trigger a [full resync](#what-does-the-full-resync-button-do).
---
## Use Git Sync with branch protection
If your repository uses branch protection, you may need to let the Hightouch Connect app bypass specific branch rules. The Hightouch Connect app can only push to a protected branch if you add it to the bypass list under GitHub rulesets.
To configure this in GitHub:
1. Go to **Settings > Rules > Rulesets**.
2. Add the Hightouch Connect app to the **Bypass list**.
3. Configure the rest of the ruleset to enforce your branch protection requirements.
GitHub supports branch protection in both **Settings > Branches** and **Settings > Rules > Rulesets**.
The Hightouch Connect app only works with branch protection when you add it to the **Bypass list** under **Rulesets** and define the rest of your branch protection rules there. It does not work as a force-push app under **Branches**.
---
## Connect staging and production workspaces
You can use Git Sync to keep staging and production workspaces aligned through the same Git repository. When multiple workspaces connect to the same repository, [alias files](#aliases) handle environment-specific source and destination IDs.
### Use the same branch
For most teams, the simplest setup is to point both workspaces to the same repository and branch, such as `main`.
1. In each workspace, go to the [Git Sync configuration page](https://app.hightouch.com/extensions/git-sync/configuration?from=docs).
2. Select the same repository and branch.
3. Share model and sync YAML files between both workspaces.
4. Create [alias files](#aliases) for each workspace to map shared aliases to the correct source and destination IDs.
For example, both workspaces can reference `aliased-postgres` in their model files. The staging alias file maps `aliased-postgres` to `staging-postgres`, while the production alias file maps it to `production-postgres`.
For a full staging-to-production example, see [Aliases](#aliases).
### Use separate branches
For a more controlled promotion workflow, use a separate branch for each environment.
1. Point your staging workspace to a development branch, such as `develop`.
2. Point your production workspace to your production branch, such as `main`.
3. Test and iterate in staging. Changes sync to the development branch.
4. When ready, open a pull request to merge the development branch into the production branch.
5. After the pull request is merged, Git Sync picks up the changes in your production workspace.
Use [CI checks](#validating-changes) to validate changes before they reach production.
### Understand what syncs across environments
Git Sync can version-control and replicate the following resources across environments:
- Models
- Syncs
- Event contracts
- Customer Studio schema models, including parent, related, and event models
- Customer Studio schema relationships
Git Sync does not replicate everything. Configure the following manually in each workspace:
- Customer Studio traits, audiences, and syncs from audiences
- Workspace-level settings, such as approval flows, spaces, and environments
- Column-level schema settings configured in the UI, such as redaction, column suggestions, and display configuration
For Customer Studio schema setup instructions, see [Configure Schema Git Sync](/customer-studio/schema-setup).
---
## Schemas
Git Sync represents supported Hightouch resources as YAML files in your Git repository.
Field names in YAML files are case-sensitive. For example, a field named
`Email` is not the same as `email`. This applies to all field references,
including source column names, primary keys, and field mappings. If you rename
a column in your warehouse changing only the casing, Git Sync treats it as a
different field, which can cause Git Sync errors or unexpected behavior.
---
### Model schema
Models use the following schema:
```yaml
name: > the model name/slug. unique per model
source: > the name/slug of the source. the slug is visible on the source page
type: > one of: raw_sql, table, or dbt_model
dbtModel: > null or the name of the dbt model to select, written as `model..`
rawSql: > null or the sql to use, for example, select * from albums
tableName: > null or the table name to use
isSchema: > used internally by Hightouch, this should always be false
primaryKey: > primary key column for this model, for example, album_id
```
#### Model field reference
| Field | Required | Type | Valid values | Notes |
| --- | --- | --- | --- | --- |
| `name` | Yes | string | Any unique string | Acts as the model slug. Must be unique across all models. |
| `source` | Yes | string | Source slug | Find the slug on the source's settings page in Hightouch. |
| `type` | Yes | string | `raw_sql`, `table`, `dbt_model` | Determines which of the next three fields is used. |
| `rawSql` | Conditional | string or null | Valid SQL | Required when `type: raw_sql`. Set to `null` otherwise. |
| `tableName` | Conditional | string or null | Table reference | Required when `type: table`. Set to `null` otherwise. |
| `dbtModel` | Conditional | string or null | `model..` | Required when `type: dbt_model`. Set to `null` otherwise. |
| `isSchema` | Yes | boolean | `false` | Always set to `false` for standard models. |
| `primaryKey` | Yes | string | Column name | Must match an existing column in the model's query results. |
The `name` field doubles as the YAML filename. If you rename a model, rename
the file to match — otherwise Git Sync creates a new model instead of updating
the existing one.
---
### Sync schemas
Sync schemas vary by destination. Because each destination has different configuration options, we recommend creating syncs in the Hightouch UI first. You can then use the generated YAML file as a starting point for updates or new syncs. The YAML filename is the sync name.
```yaml
model: > model name/slug for this sync.
destination: > destination name/slug for this sync
config:
> destination dependent configuration for this sync.
schedule: > type of schedule to use. the format here varies based on the
schedule selected, we recommend using the UI as a starting point.
schedulePaused: > boolean, where this sync is paused, or enabled.
```
#### Sync field reference
| Field | Required | Type | Valid values | Notes |
| --- | --- | --- | --- | --- |
| `model` | Yes | string | Model slug | Must match the `name` of an existing model YAML file. |
| `destination` | Yes | string | Destination slug | Find the slug on the destination's settings page or in the [manifest file](#manifests). |
| `config` | Yes | object | Destination-specific | Structure varies by destination. Create a sync in the UI first to see the generated config shape. |
| `schedule` | Yes | object | Schedule object | Format depends on schedule type (interval, cron, etc.). Use a UI-created sync as a reference. |
| `schedulePaused` | Yes | boolean | `true`, `false` | Controls whether the sync is [enabled or disabled](/syncs/overview#individual-sync-overview-page). |
The safest way to learn the `config` structure for a destination is to create
a sync manually in the Hightouch UI, then inspect the generated YAML file in
your Git repository. Use that file as a template for additional syncs to the
same destination.
Avoid modifying YAML files while a sync is actively running. Pause the sync
in the Hightouch UI before committing structural changes to prevent unexpected
behavior.
---
### Customer Studio Schema
Git Sync supports version-controlling your Customer Studio schema—including parent models, related models, event models, and relationships. For full setup instructions, see [Configure Schema Git Sync](/customer-studio/schema-setup).
Support for Customer Studio schema models is currently a beta feature. Please if you're interested in this feature.
#### Parent model schema
[Parent models](/customer-studio/schema#define-the-parent-model) live in the `schema/parent-models` directory. They have the same schema as [regular models](#model-schema), with the following additions:
```yaml
primaryLabel: "primary_label_column" # Optional
secondaryLabel: "secondary_label_column" # Optional
```
#### Related model schema
[Related models](/customer-studio/schema#add-related-models) live in the `schema/related-models` directory. They have the same schema as [regular models](#model-schema).
#### Event model schema
[Event models](/customer-studio/schema#add-event-models) live in the `schema/event-models` directory. They have the same schema as [regular models](#model-schema), with the following addition:
```yaml
eventTimestampColumn: "column_name" # Required
```
#### Model relationship schemas
[Model relationships](/customer-studio/schema#define-relationships) live in the `schema/model-relationships` directory.
The schema for model relationships depends on whether the relationship is direct or through other relationships.
Direct relationships look like this:
```yaml
from:
relationshipName: "" # Relationship name for from -> to
slug: "model_slug"
mergeColumns: false # or true
to:
relationshipName: "" # Relationship name for to -> from
slug: "model_slug"
mergeColumns: false
cardinality: "1:1" # One of: "1:1", "1:many", "many:1"
mapping:
- fromColumn: ""
toColumn: ""
```
Through relationships look like this:
```yaml
name: "relationship name"
from:
slug: "model_slug"
to:
slug: "model_slug"
through:
# List of relationship slugs that constitute this through relationship.
- relationship_slug_1
- relationship_slug_2
```
---
### Git Sync limitations for Customer Studio schema
Git Sync currently supports version-controlling model definitions and relationship structure. However, certain schema configurations set in the Hightouch UI are not synced across environments.
For example, Git Sync does **not** copy:
- Column-level settings (for example, redaction, column suggestions, or display configuration set in the UI)
- Merge column selections configured in the UI
- Column aliases
If you migrate a schema to another environment using Git Sync, you may need to reconfigure these settings manually after deployment.
Field names are also case-sensitive in YAML. If a column name changes only in casing (for example, `user_id` to `User_Id`), Git Sync treats these as different fields. Make sure column references in your YAML files match the exact casing used in your warehouse.
→ For full setup instructions, see [Configure Schema Git Sync](/customer-studio/schema-setup).
---
### Event Contract Schema
#### Event Contracts
[Event Contracts](/events/contracts/management) have a simple, consistent schema:
```yaml
name: > name of the event contract
description: > optional: description for the contract
onUndeclaredSchema: > optional: one of "ALLOW_EVENT" (default) or "BLOCK_EVENT"
```
The name of the file is the slug of the event contract, which is derived from the name of the contract.
#### Events
[Event Schemas](/events/contracts/management#event-definition) are YAML files that define the allowed format of events sent to Hightouch.
Each file represents a single version of a single event within a contract.
The files are located in the `events/contracts` directory, within a subdirectory named after the contract.
The filename is the slug of the event, which is derived from the name or type, the contract slug, and the version.
The schema for an event is as follows:
```yaml
type: > one of "track", "identify", "group", "page", "screen", or "alias"
name: > for track events, the event name; otherwise should not be present
version: > optional: the version of the event; defaults to "default"
onSchemaViolation: > optional: one of "ALLOW_EVENT" (default) or "BLOCK_EVENT"
onUndeclaredFields: > optional: one of "ALLOW_EVENT" (default), "BLOCK_EVENT", or "OMIT_FIELDS"
includeBuiltInContext: > optional: one of "true" (default), or "false"
schema: |
Schema object for validating the event, adhering to the JSON Schema spec
Usually the root of this schema is an object which contains a nested object named properties
```
In the following example, we have a Sample Event that contains a single optional "id" field:
```yaml
# events/contracts/web/sample-event-web-default.yaml
name: Sample Event
onSchemaViolation: ALLOW_EVENT
onUndeclaredFields: ALLOW_EVENT
schema:
type: object # The root of our schema is an object
properties: # Objects contain a "properties" element in the JSON Schema spec
properties: # We define a single child named properties
type: object # Which is an object
properties: # Finally, the properties our child object contains
id: # Which is only an optional id field
type: string # Which is of type string
type: track
version: default
```
This corresponds to the following JSON in the in-app JSON Editor:
```json
{
"type": "object",
"properties": {
"properties": {
"type": "object",
"properties": {
"id": {
"type": "string"
}
}
}
}
}
```
Because event schemas may get complex, using Hightouch [CI checks](#validating-changes) is a great way to validate your changes
before merging them into your repository.
---
### Manifests
Hightouch creates a manifest file at the root of your Git repository for each connected workspace. **Manifest files** list all sources and destinations in a workspace, making it easier for you to find their IDs to add to sync and model files or reference in [aliases](#aliases).
Manifest filenames follow this format: `manifest-.yaml`. A manifest file follows this format:
```yaml
# READ ONLY FILE
# Changes made here will be ignored. Please use the Hightouch dashboard to create and edit sources and destinations.
sources:
production-postgres:
name: Production postgres
type: postgres
snowflake-243-tenant:
name: Snowflake EU instance
type: snowflake
destinations:
production-salesforce:
name: Production Salesforce
type: salesforce
```
Sources and destinations are listed by their ID. In the example file above, the Postgres source's ID is `production-postgres` and the Salesforce destination's ID is `production-salesforce`.
Manifest files are **read-only**. Git Sync overwrites any changes made to
manifest files. Use the Hightouch dashboard to create and edit sources and
destinations.
---
### Aliases
Managing multiple Hightouch workspaces from the same Git repository can be difficult because source and destination IDs can differ between workspaces. **Alias files** let you create custom IDs for sources and destinations in a given workspace allowing model and sync files to work across different workspaces.
Hightouch creates an empty alias file in your Git repository for each connected workspace. Alias filenames follow this format: `aliases-.yaml`.
You can then add sources and destinations to the alias file following this format:
```yaml
# e.g. aliases-hightouch-production-workspace.yaml
sources:
my-source-alias: my-actual-source-id
othersourcealias: another-workspace-source
destinations:
my-destination-alias: existing-destination
```
Each key is an alias and each value should be an actual source or destination ID. You can find source and destination IDs in the [manifest file](#manifests) in the root of your Git repository.
Aliases should be unique across sources and destinations in the same
workspace. In other words, you can't use the same alias for multiple resources
in the same alias file.
#### Example alias usage
Consider a workspace, `acme-staging`, with a model `stocking-list.yaml` file like this:
```yaml
name: Stocking list
source: staging-postgres
type: table
tableName: public.workspaces
isSchema: false
primaryKey: id
```
Another workspace called `acme-production` is connected to the same repository but can't use this model during Git Sync. That's because the `acme-production` workspace doesn't have a source with the ID `staging-postgres`. Instead, `acme-production` uses a Postgres source with the ID `production-postgres`.
You can use aliases so that both workspaces can use the same model `stocking-list.yaml` file.
1. Add an alias to the `acme-staging` workspace alias file:
```yaml
# aliases-acme-staging.yaml
sources:
aliased-postgres: staging-postgres
```
This creates the custom ID `aliased-postgres` that points to the source with the ID `staging-postgres`.
2. Add an alias to the `acme-production` workspace alias file:
```yaml
# aliases-acme-production.yaml
sources:
aliased-postgres: production-postgres
```
This creates the custom ID `aliased-postgres` that points to the source with the ID `production-postgres`. For the next step, it's important that both alias files use the same alias name. Since these aliases exist in different workspaces, they are still unique within their respective workspaces.
3. Update the `source` value of `stocking-list.yaml` to `aliased-postgres`.
```diff
name: Stocking list
- source: staging-postgres
+ source: aliased-postgres
type: table
tableName: public.workspaces
isSchema: false
primaryKey: id
```
Now, during Git Sync, both workspaces resolve `aliased-postgres` to their respective Postgres slugs and Git Sync completes for both staging and production workspaces.
---
## Validate changes
You can use Hightouch CI checks to validate changes in your Git Sync repository before merging them.
To enable CI checks, turn on **GitHub Pull Request checks** in your Git Sync [configuration page](https://app.hightouch.com/extensions/git-sync/configuration?from=docs).
After CI checks are enabled, Hightouch runs a check on each pull request made to the connected Git repository. If a breaking change is detected, such as deleting a model used by a sync, the CI check fails. Open the failed check details to see which sync is affected.
If the pull request does not affect any Hightouch syncs, the CI check passes.
### Custom YAML validation in CI
In addition to Hightouch's built-in CI checks, you can add your own YAML validation step to your CI pipeline to catch syntax errors and missing required fields before a pull request is merged.
Here's an example using a simple validation script with `yq` (a YAML processor available in most CI environments):
```bash
#!/bin/bash
# validate-git-sync-yaml.sh
# Validates model and sync YAML files for required fields.
set -euo pipefail
ERRORS=0
# Validate model files
for file in models/*.yaml; do
[ -f "$file" ] || continue
for field in name source type primaryKey; do
if ! yq -e ".$field" "$file" > /dev/null 2>&1; then
echo "ERROR: $file is missing required field '$field'"
ERRORS=$((ERRORS + 1))
fi
done
done
# Validate sync files
for file in syncs/*.yaml; do
[ -f "$file" ] || continue
for field in model destination config schedule schedulePaused; do
if ! yq -e ".$field" "$file" > /dev/null 2>&1; then
echo "ERROR: $file is missing required field '$field'"
ERRORS=$((ERRORS + 1))
fi
done
done
if [ "$ERRORS" -gt 0 ]; then
echo "$ERRORS validation error(s) found."
exit 1
fi
echo "All YAML files passed validation."
```
This script checks for required fields only. Hightouch's built-in CI checks
handle deeper validation, such as verifying that referenced models and
destinations exist. Use both for the most coverage.
---
## Review change history
Git Sync gives you a changelog through commit history. You can also use
[audit logs](/workspace-management/audit-logs) to review changes in Hightouch.
You can use your Git repository to see who created or modified supported resources and when.
To show the correct user for Hightouch-created commits, the user's Hightouch email must match their GitHub email.
In GitHub, open the repository's **Commits** tab or file history to review changes.
Open a commit to see the exact line-by-line diff.
---
## Use tunneling for private Git repositories
If your Git repository is hosted on an on-premise server without public internet access, use [tunneling](/security/ssh-tunneling) to connect Hightouch to your Git server.
1. Go to the [**Tunnels** tab](https://app.hightouch.com/settings/tunnels?from=docs) on the **Settings** page.
2. Create a tunnel or reverse tunnel that connects to your Git server. For more information, see the [tunneling documentation](/security/ssh-tunneling).
3. Go to the [Git Sync configuration page](https://app.hightouch.com/extensions/git-sync/configuration?from=docs).
4. Under **Configuration > Git credentials**, select **Manage**.
5. Add your Git credential and select the correct tunnel.
We recommend using the SSH protocol and `ssh_privatekey` to connect to private Git repositories.
Basic authentication can have issues forwarding authentication headers through an SSH tunnel and HTTP redirects. For more information about SSH authentication, see the [GitHub SSH documentation](https://docs.github.com/en/authentication/connecting-to-github-with-ssh).
Hightouch does not support `ssh_privatekey` with passphrases.
---
## Next steps
After Git Sync is configured, you can use Git workflows to manage supported Hightouch resources. For example, you can:
- Create a new model by copying an existing model file and updating the name and query
- Create a new model and sync in the same pull request
- Use [CI checks](#validate-changes) to validate changes before merging them
- Use [aliases](#aliases) to share model and sync files across staging and production workspaces
---
## FAQ
### What does the Full resync button do?
Selecting **Full resync** runs an outbound sync for all supported resources in the workspace. This pushes all supported resources from Hightouch to Git, then pulls changes from Git back to your workspace.
If your Git repository has changes that have not synced to Hightouch yet, **Full resync** can overwrite those changes.
{/* */}
### Recreate a deleted YAML file
Git Sync does not support deletions from Git. If you delete a YAML file in Git, the corresponding model or sync remains in Hightouch. The file is not recreated in Git until the next change is made in Hightouch or you trigger a [full resync](#what-does-the-full-resync-button-do).
To recreate a deleted YAML file, select **Full resync**.
You can also recreate the file manually by adding a YAML file with the same filename and [schema](#schemas) as the original file. Hightouch recognizes that the file refers to an existing model or sync and associates it with the existing resource.
### Protected branch update failed
If you selected a protected branch, Git Sync setup might fail with this error:
```
Protected branch update failed - Changes must be made through a pull request
```
To resolve the error, use one of these options:
- Select a branch that is not protected, then open pull requests in GitHub to merge updates into the protected branch.
- Follow the instructions in [Use Git Sync with branch protection](#use-git-sync-with-branch-protection) to let the Hightouch Connect app work with your protected branch.
---
## Create models using Looker Looks
**URL:** https://hightouch.com/docs/extensions/looker-models
**Description:** Connect Hightouch to your Looker instance and build Hightouch models using existing Looks.
**Section:** Extensions
If you've already invested in modeling business data in [Looker](https://www.looker.com/), you may want to use your Looks as the basis for [Hightouch models](/getting-started/concepts#models).
When you import a model from Looker, Hightouch converts your Look into a SQL query and runs it directly against your data warehouse. As you publish changes in Looker, your Hightouch model updates to reflect the latest version of your Look.
This extension uses Looker as a *modeling method*, not a *data source*. In
other words, Looker provides the SQL query, not the query results. Before
enabling this extension, you must first [create a
source](/syncs/create-your-first-sync#connect-a-source) in Hightouch that
connects to the same data warehouse used by Looker.
## Supported sources
You can create models based on Looker Looks for the following sources:
- [Amazon Redshift](/sources/amazon-redshift)
- [Amazon Athena](/sources/amazon-athena)
- [Google BigQuery](/sources/google-bigquery)
- [Greenplum Database](/sources/greenplum)
- [PostgreSQL](/sources/postgresql)
- [Snowflake](/sources/snowflake)
## Connect to Looker
In Hightouch, go to the [**Extensions** page](https://app.hightouch.com/extensions?from=docs) and select [**Looker**](https://app.hightouch.com/extensions/looker/configuration?from=docs).
Enter the following fields:
- (Optional) **Tunnel** - If you've previously set up an [SSH tunnel](/security/ssh-tunneling) and want to use it to connect to Looker, you can select it here.
- **URL** - Hightouch uses this URL to connect to your Looker instance. The format is often `https://.looker.com`. Refer to [Looker's documentation](https://docs.looker.com/reference/api-and-integration/api-getting-started#looker_api_path_and_port) if you have a non-standard Looker setup.
- **Client ID** - You can find this in your [Looker API 4 credentials](https://docs.looker.com/reference/api-and-integration/api-auth).
- **Client secret** - You can find this in your [Looker API 4 credentials](https://docs.looker.com/reference/api-and-integration/api-auth).
If you need to create credentials, refer to Looker's documentation on [authentication with an SDK](https://docs.looker.com/reference/api-and-integration/api-auth#authentication_with_an_sdk).
## Select a Look
After configuring your Looker credentials, you can create a model using a [source that supports Looker](#supported-sources). Select your source and then choose **Import Model from Looker** as your modeling method.
On the next screen, you can select which Look you want to base your model on.
## Tips and troubleshooting
If you encounter any error messages during Look-based model setup, confirm the following about your credentials:
### Permissions
You must be a Looker admin to retrieve your Looker connection URL. Check [Looker's documentation on roles, permissions, and access](https://docs.looker.com/admin-options/settings/roles) for more information.
### Looker URL
Check you're using the correct URL by consulting [Looker's API connection troubleshooting guide](https://docs.looker.com/reference/api-and-integration/api-troubleshooting#endpoint_unreachable).
### Common error: 404 Looker is unavailable
When the Looker extension can’t reach your Looker instance (or the API credentials are invalid/expired), Looker may return an HTML 404 page instead of a JSON API error. In Hightouch this can appear as:
> Look not found. Has it been deployed to production?
> Looker Not Found (404) Looker is unavailable.
> If you typed in a URL, double-check the spelling.
> This may also be due to a temporary condition such as an outage, scheduled maintenance or upgrade.
If you see this for all Looker-based models:
- Check the **Looker API 4 client ID/secret** in **Extensions → Looker** (rotate if expired).
- Confirm the **Looker URL** is correct.
- Verify your Looker instance is reachable and not under maintenance.
---
## Trigger syncs with Mage
**URL:** https://hightouch.com/docs/extensions/mage
**Description:** Learn how to trigger syncs with Mage.ai.
**Section:** Extensions
## Overview
Mage provides a native `HightouchClient` to trigger Hightouch syncs. Check out the [Mage's integration documentation](https://docs.mage.ai/integrations/hightouch) for usage details.
## Setup
To use Mage to trigger syncs, you need a Hightouch API key.
### Create an API key
The first step is to create a Hightouch API key in your Hightouch workspace settings.
### Set sync schedule type
## Usage
You can then trigger a sync with Mage's `HightouchClient`:
```python
from mage_ai.services.hightouch.hightouch import HightouchClient
client = HightouchClient(config=dict(api_key='api_key'))
syncs = client.list_syncs() # Get all the syncs
sync_id = syncs['data'][0]['id'] # Get sync id
client.sync_and_poll(sync_id) # Trigger sync and poll status
```
If successful, the output looks like this:
```bash
Polling Hightouch Sync [id]. Current status: querying. 0% completed.
Polling Hightouch Sync [id]. Current status: success. 100% completed.
Sync request status: success. Polling complete
```
---
## PagerDuty
**URL:** https://hightouch.com/docs/extensions/pagerduty
**Description:** The PagerDuty extension lets Hightouch notify your on-call staff about failed queries and syncs.
**Section:** Extensions
The PagerDuty extension lets Hightouch notify your on-call staff about failed queries and syncs.
Once you've authorized access to PagerDuty, you must [set up alerts for each sync](/syncs/alerting#sync-alert-setup) that you want to send alerts through PagerDuty on.
## Setup
1. Create a PagerDuty Events API V2 extension and copy its API Key:
- In PagerDuty, go to the **Integrations** tab in the **Services** page. Select **+ Add another Integration**.
- Select **Events API V2** integration and click **Add**.
- Back in the **Integrations** tab, copy the **Integration Key**.
2. In Hightouch, go to the [**Extensions** page](https://app.hightouch.com/extensions?from=docs) and select [**Alerting**](https://app.hightouch.com/extensions/alerting?from=docs). Open the [**Configuration** tab](https://app.hightouch.com/extensions/alerting/configuration?from=docs).
3. Click **Set up** under PagerDuty and paste the **PagerDuty API Key** you previously copied.
4. In the [**Configuration** tab](https://app.hightouch.com/extensions/alerting/configuration?from=docs), click **Add alert**.
5. In the modal that appears, enter a descriptive name for your alert, select **PagerDuty** as the **Platform** and select the **Credentials** you just entered.
6. Set the **Default Behaviors**, and optionally enter a **Minimum Alert Interval**.
- The **Default Behaviors** can be **fatal error** and/or **row error**.
- A **fatal error** is an error communicating with either a source or a destination. it's 'fatal' because the sync can't even run.
- A **row error** means a problem with a particular row or rows. The sync works, but one or more rows failed.
- Setting the **Minimum Alert Interval** lets you receive alerts at spaced intervals—once every hour, for instance. If you don't set this, you receive alerts every time your selected error occurs. You must enter the Minimum Alert Interval in minutes.
7. Once you've entered your alert preferences, click **Create**.
8. [Set up alerts for each sync](/syncs/alerting#sync-alert-setup) that you want to send alerts through PagerDuty on.
Now when a sync attempt fails, Hightouch sends an alert to your PagerDuty service.
---
## Trigger syncs with Prefect
**URL:** https://hightouch.com/docs/extensions/prefect
**Description:** Learn how to trigger syncs with Prefect's Hightouch Tasks.
**Section:** Extensions
## Overview
Prefect provides native **tasks** to interact with Hightouch. Check out the [Git repository](https://github.com/PrefectHQ/prefect-hightouch) for examples and [Prefect's API reference](https://docs-v1.prefect.io/api/latest/tasks/hightouch.html#hightouchrunsync) for usage details.
## Setup
To use Prefect's tasks you need:
- A Hightouch API key
- The sync IDs of the syncs you want to trigger using Prefect
### Create an API key
The first step is to create a Hightouch API key in your Hightouch workspace settings.
### Set sync schedule type
## Usage
Refer to Prefect's [Hightouch Git repository](https://github.com/PrefectHQ/prefect-hightouch) for the most up-to-date examples and instructions.
---
## Create models using Sigma
**URL:** https://hightouch.com/docs/extensions/sigma
**Description:** Connect Hightouch to your Sigma instance and build Hightouch models using existing workbooks.
**Section:** Extensions
If you've already invested in modeling business data in [Sigma](https://www.sigmacomputing.com/), you may want to use your Sigma workbooks as the basis for [Hightouch models](/getting-started/concepts#models).
When you import a model from Sigma, Hightouch converts your workbook element into a SQL query and runs it directly against your data warehouse. As you publish changes in Sigma, your Hightouch model updates to reflect the latest version of your Sigma workbook.
This extension uses Sigma as a *modeling method*, not a *data source*. In
other words, Sigma provides the SQL query, not the query results. Before
enabling this extension, you must first [create a
source](/syncs/create-your-first-sync#connect-a-source) in Hightouch that
connects to the same data warehouse used by Sigma.
## Supported sources
You can create models based on Sigma workbooks for the following sources:
- [Amazon Redshift](/sources/amazon-redshift)
- [Databricks](/sources/databricks)
- [Greenplum Database](/sources/greenplum)
- [Google BigQuery](/sources/google-bigquery)
- [PostgreSQL](/sources/postgresql)
- [Snowflake](/sources/snowflake)
## Connect to Sigma
In Hightouch, go to the [**Sigma configuration page**](https://app.hightouch.com/extensions/sigma/configuration?from=docs) in **Extensions**.
Enter the following fields:
- **Sigma cloud** - Follow [these instructions](https://help.sigmacomputing.com/hc/en-us/articles/360037430573-Technical-Specifications#h_01EJF3BT46HVZ8YMNFQ149SD0A) to find your Sigma cloud.
- **Client ID** - Follow [these instructions](https://help.sigmacomputing.com/hc/en-us/articles/4408555307027-Get-an-API-Token-and-Client-ID) to find your client ID.
- **Client secret** - Follow [these instructions](https://help.sigmacomputing.com/hc/en-us/articles/4408555307027-Get-an-API-Token-and-Client-ID) to create a secret tied to your client ID. Make sure to create an **API token** rather than **Embed secret**.
## Build models
After configuring your Sigma credentials, you can create a model using a [source that supports Sigma](#supported-sources). Select your source and then choose **Import Model from Sigma** as your modeling method.
On the next screen, you can select which Sigma **Workbook**, **Page**, and **Element** to build your Hightouch model off of. Click **View SQL** to see the query that Hightouch executes on your data warehouse. Make sure to click **Preview** to preview your results before finishing your model setup.
## Tips and troubleshooting
### Worksbooks aren't visible in Hightouch
Hightouch only displays [published](https://help.sigmacomputing.com/hc/en-us/articles/1500010372322-Workbook-Lifecycle-Explore-Draft-and-Publish#h_01F6JP3G04NC4APKVE1DQ7ESH8) Sigma workbooks, to which the credentials [you entered into Hightouch](#connect-to-sigma) have access.
If you don't see all the workbooks you expect in Hightouch, make sure:
- The workbook is published (not in draft mode).
- The user whose credentials you used to [connect to Sigma](#connect-to-sigma) has at least **view** access to the workbook; if they don't, share it with them and give them at least view access.
---
## Docs for AI tools
**URL:** https://hightouch.com/docs/developer-tools/agent-resources
**Description:** Search and read Hightouch documentation from AI tools via MCP, plain-text files, and agent skills.
**Section:** Developer tools
## Overview
Hightouch publishes documentation in formats that AI tools can read directly, so they have accurate and current information when answering questions or helping you build integrations. Choose the format that fits how your tool works:
| Format | Best for | How it works |
| :------------------------------------ | :-------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------- |
| [MCP server](#mcp-server) | Targeted lookups from tools that support MCP | The tool searches for relevant sections and reads only the pages it needs. Uses fewer tokens because it doesn't load everything upfront. |
| [Plain-text files](#plain-text-files) | Broad context loading or tools without MCP support | Paste a URL to give your tool an index of all pages or the full content of every page in one file. |
| [Skills](#skills) | Implementation tasks (API calls, SDK setup, sync configuration) | A structured instruction file that helps tools work with the right endpoints, patterns, and parameters. |
## MCP server
For AI tools that support [Model Context Protocol](https://modelcontextprotocol.io) (MCP), Hightouch publishes an MCP endpoint with two tools: one to search across all public docs by keyword, and one to read the full content of a specific page. The tool retrieves only the sections relevant to your question rather than loading all documentation at once, which means faster responses and lower token usage.
This is a **docs-only** MCP server for searching and reading Hightouch
documentation — it doesn't connect to your Hightouch workspace or take actions
in your account. For an AI assistant that can build audiences, design
journeys, and generate emails, ad creatives, and campaign concepts directly in
your workspace, see the [Hightouch MCP](/ai-integrations/mcp) instead.
The search index is rebuilt on every deployment, so results always reflect the latest content.
### Endpoint
```
https://hightouch.com/docs/api/mcp
```
### Available tools
| Tool | Description |
| :------------ | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `search_docs` | Full-text search across all public docs with per-heading granularity. Returns matching sections with title, URL, content snippet, and section hierarchy. Supports filtering by product area. |
| `read_doc` | Returns the full markdown content of a documentation page by path. Use paths from `search_docs` results. |
### Configuration
Any MCP client that supports HTTP transport can connect using the endpoint URL. Below are setup instructions for common tools.
#### Claude
##### Custom connectors (recommended)
**If you're an Owner on a Team or Enterprise plan:**
1. Go to [Organization settings → Connectors](https://claude.ai/admin-settings/connectors).
2. Click **Add**.
3. Hover over **Custom**, then select **Web**.
4. Paste the URL: `https://hightouch.com/docs/api/mcp`.
5. Click **Add** to finish — no OAuth client ID or secret is needed since the endpoint doesn't require authentication.
**If you're a member (not an Owner) on a Team or Enterprise plan:**
1. Ask an Owner to complete the steps above first — members can't add custom connectors directly.
2. Once it's added, go to [Customize → Connectors](https://claude.ai/customize/connectors).
3. Find "Hightouch Docs" in the list (labeled **Custom**).
4. Click **Connect** to authenticate and start using it.
**If you're on an individual Pro, Max, or Free plan:**
1. Go to [Customize → Connectors](https://claude.ai/customize/connectors).
2. Click **+**, then **Add custom connector**.
3. Paste the URL: `https://hightouch.com/docs/api/mcp`.
4. Click **Add** to finish.
Free plan accounts are limited to one custom connector total. If you already
have one configured, remove it before adding this one.
##### Config file (if you can't add custom connectors)
1. Quit Claude Desktop completely.
2. Open the config file:
- macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`
- Windows: `%APPDATA%\Claude\claude_desktop_config.json`
3. Add (or merge into) the `mcpServers` object:
```json
{
"mcpServers": {
"hightouch-docs": {
"type": "url",
"url": "https://hightouch.com/docs/api/mcp"
}
}
}
```
4. Save the file and relaunch Claude Desktop.
5. Confirm the connector is live: open a chat, click the tools/connector icon, and check that "hightouch-docs" appears with `search_docs` and `read_doc` listed.
**Claude Code** — Run the following from your terminal:
```bash
claude mcp add --transport http hightouch-docs https://hightouch.com/docs/api/mcp
```
Verify with `claude mcp list` — you should see `hightouch-docs` with a connected status.
#### Cursor
Add the following to `.cursor/mcp.json` in your project:
```json
{
"mcpServers": {
"hightouch-docs": {
"type": "url",
"url": "https://hightouch.com/docs/api/mcp"
}
}
}
```
#### Other clients
Use the endpoint URL `https://hightouch.com/docs/api/mcp` with Streamable HTTP transport. No authentication is required. Refer to your client's documentation for MCP configuration instructions.
### Example prompts
Once connected, your AI tool queries the docs automatically when you ask questions about Hightouch. Here are some examples to try:
**Sync configuration**
```
How do I set up a sync to Salesforce? What sync modes does it support?
```
```
What's the difference between upsert and mirror sync modes?
```
**Destinations and sources**
```
What permissions do I need to connect a Snowflake source?
```
```
Show me the field mapping options for the Braze destination.
```
**Troubleshooting**
```
I'm getting a "rate limit exceeded" error syncing to HubSpot. What should I check?
```
```
How do I use the Live Debugger to inspect API requests for a sync?
```
**Customer Studio**
```
How do I create a computed trait that counts purchases in the last 30 days?
```
```
What filter types are available in the audience builder?
```
The docs MCP server gives your AI tool access to Hightouch documentation. To
also connect your AI assistant to your Hightouch workspace — to build
audiences, create journeys, or generate content — add the [Hightouch
MCP](/ai-integrations/mcp). You can use both together for documentation
reference and platform actions in the same conversation.
## Plain-text files
Hightouch publishes plain-text versions of the docs that any AI tool can read without parsing web page markup. Both files are regenerated on every deployment, so they always reflect the latest content. Use these when your tool doesn't support MCP or when you want to load broad context in a single request.
[**llms.txt**](https://hightouch.com/docs/llms.txt) is an index of every public page, organized by product area. Use this when you want an AI tool to know what documentation is available.
[**llms-full.txt**](https://hightouch.com/docs/llms-full.txt) contains the full text of every public page in a single file. Use this when you want an AI tool to have complete context about Hightouch.
To use either file, paste the URL into your AI tool:
```
https://hightouch.com/docs/llms.txt
https://hightouch.com/docs/llms-full.txt
```
## Skill file
Hightouch publishes a [skill file](https://hightouch.com/docs/skill.md) that gives AI tools a structured reference for writing code against the Hightouch platform. It covers REST API endpoints and authentication, CLI commands, Events SDK methods and setup snippets, sync modes and schedule types, and core data model concepts (sources, models, destinations, syncs).
Use this when you want an AI tool to generate correct API calls, CLI commands, or SDK integrations without searching the full docs. The file follows the [Agent Skills](https://agentskills.io/) spec.
The skill file is a static reference for developer tooling. To connect an AI
assistant to your Hightouch workspace for building audiences, creating
journeys, or generating content, see [Hightouch MCP](/ai-integrations/mcp).
## Additional resources
- [API overview](/developer-tools/api-guide) — REST API authentication and usage
- [API reference](/api-reference) — Full API endpoint documentation
- [CLI overview](/developer-tools/cli-guide) — Command line interface for Hightouch
- [Events SDK](/events/overview) — Track user events for real-time activation
---
## API overview
**URL:** https://hightouch.com/docs/developer-tools/api-guide
**Description:** Hightouch exposes a REST style API that allows users to interact with resources like syncs, models, sources and destinations.
**Section:** Developer tools
## Overview
Hightouch exposes a REST API that lets users interact with resources like syncs, models, sources, and destinations.
Though the REST API is open to all Hightouch users, we aim to build features so you don't need to rely on it directly. For example, [Git Sync](/extensions/git-sync) lets you manage workspace resources programmatically.
If you're looking to create a custom destination, it's best to use the [HTTP Request destination](/destinations/http-request) as your base. If you find you still need to use the REST API directly, about your use case.
To start using the API, you must first create and retrieve an API key from your workspace.
## Required permissions
To use Hightouch programmatically through the API, you must use an API key created by an Admin user of your Hightouch workspace. See [Roles](/workspace-management/roles#pre-built-roles) for more information.
## Create an API key
Log in to Hightouch as an Admin user of your Hightouch workspace, then follow these steps:
Once you have an API key, you can follow the [API Reference documentation](/api-reference) to start using the API.
## Revoke an API key
Your API key provides read/write access to sensitive Hightouch resources and should be kept secure. If you believe your API key may have been compromised, revoke it immediately.
Log in to Hightouch as an Admin user of your Hightouch workspace, then follow these steps:
1. From the [**API keys** tab](https://app.hightouch.com/settings/api-keys?from=docs) on the **Settings** page, select the API key or keys you want to revoke.
2. Select **Delete selected**.
3. In the modal that appears, select **Delete**.
## Hightouch API Rate Limits
The Hightouch API rate limit is 200 requests per 10 seconds per workspace.
## TLS requirements
The Hightouch REST API requires TLS 1.2 or later. The API rejects requests that use an older protocol version.
The API supports the following cipher suites:
```text
TLS_AES_128_GCM_SHA256
TLS_AES_256_GCM_SHA384
TLS_CHACHA20_POLY1305_SHA256
ECDHE-ECDSA-AES128-GCM-SHA256
ECDHE-RSA-AES128-GCM-SHA256
ECDHE-ECDSA-AES256-GCM-SHA384
ECDHE-RSA-AES256-GCM-SHA384
```
On July 10, 2026, Hightouch removes support for several weaker cipher suites. After that date, the API rejects requests that negotiate any of the following:
```text
ECDHE-ECDSA-AES128-SHA256
ECDHE-RSA-AES128-SHA256
ECDHE-ECDSA-AES256-SHA384
ECDHE-RSA-AES256-SHA384
```
If your API client only supports the deprecated cipher suites, update it
before July 10, 2026 to avoid failed requests. Most modern HTTP libraries and
TLS stacks already negotiate one of the supported suites.
## Tips and troubleshooting
If you encounter issues using the Hightouch API, check our [status page](https://status.hightouch.io/) or for assistance.
### Common Errors
The following error indicates an issue with your API key. Check that the creator of the API key is an Admin user of your Hightouch workspace.
```json
{
"message": "Authentication error",
"details": "No user found"
}
```
---
## CLI overview
**URL:** https://hightouch.com/docs/developer-tools/cli-guide
**Description:** Hightouch provides an intuitive Command Line Interface (CLI) to give users access to resources in the terminal or automation scripts.
**Section:** Developer tools
## Overview
Hightouch provides an intuitive Command Line Interface (CLI) to give users access to resources in the terminal or automation scripts.
You can use the CLI to access the definition of your resources like syncs and models or trigger a sync run from your terminal or automation scripts.
## Quick start
### Install CLI
To install from shell scripts, run:
```shell
# Install the latest version
$ curl -sLf https://getcli.hightouch.io | sh -
# Install specific version
$ curl -sLf https://getcli.hightouch.io | INSTALL_HT_VERSION=v0.1.0 sh -
```
### Login with API key
Next, you need to create a Hightouch API key in your Hightouch workspace settings.
Then you can login with this API key:
```shell
$ ht login
Hightouch API key [************************************]:
```
You can also use your API key with ENV variables.
```shell
HIGHTOUCH_APIKEY=${yourkey} ht COMMANDS
```
## CLI reference
Once you've installed and logged in, you can follow the [CLI Reference documentation](/developer-tools/cli-reference) to start using the CLI.
## Issues
If you have any problems using CLI, please file an issue through [GitHub](https://github.com/hightouchio/cli/issues), or contact us through Slack or.
---
## CLI Reference
**URL:** https://hightouch.com/docs/developer-tools/cli-reference
**Section:** Developer tools
- [`ht destinations`](#ht-destinations)
- [`ht inspect RESOURCE`](#ht-inspect-resource)
- [`ht login`](#ht-login)
- [`ht models`](#ht-models)
- [`ht runs`](#ht-runs)
- [`ht sources`](#ht-sources)
- [`ht syncs`](#ht-syncs)
- [`ht trigger SYNC`](#ht-trigger-sync)
## `ht destinations`
List all destinations in current workspace
```
USAGE
$ ht destinations
DESCRIPTION
List all destinations in current workspace
ALIASES
$ ht destination
EXAMPLES
$ ht destinations
```
_See code: [src/commands/destinations.ts](https://github.com/hightouchio/cli/blob/v0.1.0/src/commands/destinations.ts)_
## `ht inspect RESOURCE`
Inspect resources in the current workspace
```
USAGE
$ ht inspect [RESOURCE] -f
ARGUMENTS
RESOURCE Resource ID
FLAGS
-f, --format= (required) [default: json] Output format(yaml/json)
DESCRIPTION
Inspect resources in the current workspace
EXAMPLES
$ ht inspect model/run-user
```
_See code: [src/commands/inspect.ts](https://github.com/hightouchio/cli/blob/v0.1.0/src/commands/inspect.ts)_
## `ht login`
Login Hightouch with API key
```
USAGE
$ ht login
DESCRIPTION
Login Hightouch with API key
EXAMPLES
$ ht login
```
_See code: [src/commands/login.ts](https://github.com/hightouchio/cli/blob/v0.1.0/src/commands/login.ts)_
## `ht models`
List all models in current workspace
```
USAGE
$ ht models
DESCRIPTION
List all models in current workspace
ALIASES
$ ht model
EXAMPLES
$ ht models
```
_See code: [src/commands/models.ts](https://github.com/hightouchio/cli/blob/v0.1.0/src/commands/models.ts)_
## `ht runs`
List all syncs run for a given sync
```
USAGE
$ ht runs -s [-l ]
FLAGS
-l, --limit= [default: 5] Limit the number of runs to output
-s, --sync= (required) Specify sync name to list runs
DESCRIPTION
List all syncs run for a given sync
ALIASES
$ ht run
EXAMPLES
$ ht runs
```
_See code: [src/commands/runs.ts](https://github.com/hightouchio/cli/blob/v0.1.0/src/commands/runs.ts)_
## `ht sources`
List all sources in current workspace
```
USAGE
$ ht sources
DESCRIPTION
List all sources in current workspace
ALIASES
$ ht source
EXAMPLES
$ ht sources
```
_See code: [src/commands/sources.ts](https://github.com/hightouchio/cli/blob/v0.1.0/src/commands/sources.ts)_
## `ht syncs`
List all syncs in current workspace
```
USAGE
$ ht syncs
DESCRIPTION
List all syncs in current workspace
ALIASES
$ ht sync
EXAMPLES
$ ht syncs
```
_See code: [src/commands/syncs.ts](https://github.com/hightouchio/cli/blob/v0.1.0/src/commands/syncs.ts)_
## `ht trigger SYNC`
Trigger sync to run
```
USAGE
$ ht trigger [SYNC] [-f] [-q]
ARGUMENTS
SYNC Sync ID
FLAGS
-f, --full-sync Whether to run a full sync
-q, --quiet Whether to show run progress
DESCRIPTION
Trigger sync to run
EXAMPLES
$ ht trigger --sync test123
```
_See code: [src/commands/trigger.ts](https://github.com/hightouchio/cli/blob/v0.1.0/src/commands/trigger.ts)_
---
## Error codes
**URL:** https://hightouch.com/docs/developer-tools/error-codes
**Description:** A reference of all Hightouch error codes, what they mean, and how to resolve them.
**Section:** Developer tools
When something fails in Hightouch, the platform assigns an error code that identifies the cause. This page is a complete reference of all Hightouch error codes, organized by where they originate.
Some error codes include troubleshooting links directly in the Hightouch UI. If you don't see a link for your error, use this page to find the code and follow the resolution steps.
Don't see your error code? This page covers Hightouch platform error codes only. Hightouch passes error messages from third-party APIs directly, so if you see a vendor error (like a `400` or `409`), check the [destination's docs page](/destinations) under **Tips and troubleshooting** or use the [live debugger](/syncs/debugger) to inspect the full response.
If your error still isn't covered anywhere, use the **Send feedback** button on this page to let us know and we'll add it.
## Sync request errors
These errors occur during sync planning, before Hightouch sends any data to the destination. They typically indicate a problem with your model, primary key, or warehouse configuration.
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `DUPLICATE_COLUMNS_DETECTED` | Your model query returns duplicate column names. | Alias columns in your model SQL so every selected field has a unique name. |
| `NON_UNIQUE_PRIMARY_KEY` | The selected primary key contains duplicate values. | Update your model so each row has a unique identifier. If you're using a SQL model, add a `DISTINCT` clause or `GROUP BY` on the key column. |
| `UNSUPPORTED_PRIMARY_KEY_TYPE` | The primary key's data type isn't supported for this sync path. | Cast the primary key to a supported type (typically `VARCHAR` or `INTEGER`) in your model SQL. |
| `QUERY_COLUMN_MISSING` | A column referenced in your sync configuration no longer exists in the model query results. | Refresh the model schema in Hightouch and update your [field mappings](/syncs/mapping-data) to use valid columns. |
| `WAREHOUSE_TABLE_MISSING` | A warehouse table required for sync planning doesn't exist. | Confirm the table still exists in your warehouse and that the model points to the correct schema and table name. |
| `PREVIOUS_SYNC_RUN_OBJECT_MISSING` | Stored objects from the previous sync run are missing, usually because cached plan data expired. | Retry the sync. If it persists, trigger a [full resync](/syncs/overview#resync-full-query) to rebuild sync state. |
| `REMOVE_PLAN_INCOMPLETE` | Backfill or removal planning couldn't complete during a planner transition. | Retry the sync after a few minutes. If it continues, contact support. |
| `REMOVE_RETRY_CHANGED_COLUMN_TYPES` | Rejected-row removal retries are unsafe because model column types changed between runs. | Refresh your model schema and trigger a [full resync](/syncs/overview#resync-full-query) from a consistent state. |
| `SORT_RAN_OUT_OF_DISK_SPACE` | Sorting rows exhausted available disk space. | Narrow the model to reduce row volume, or contact support for infrastructure help. |
| `UNSPECIFIED` | Hightouch couldn't classify the failure into a more specific code. | Use the [live debugger](/syncs/debugger) and [warehouse sync logs](/syncs/warehouse-sync-logs) to identify the root cause. |
## Destination errors
### Generic destination errors
These errors can occur with any destination. They indicate a problem with authentication, rate limits, or the destination's availability.
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `DESTINATION_INVALID_GRANT` | The destination's authorization grant expired or was revoked. | Reconnect the destination in Hightouch to refresh authorization. |
| `DESTINATION_INVALID_CREDENTIALS` | The destination credentials are invalid. | Verify the username, password, token, or secret configured for the destination. Reconnect if needed. |
| `DESTINATION_INVALID_API_KEY` | The API key is invalid or expired. | Replace it with a current key that has the required permissions. |
| `DESTINATION_DEFINITION_NOT_FOUND` | Hightouch couldn't load the destination type definition. | Confirm you're using a supported destination type. If the connector appears broken, contact support. |
| `DESTINATION_INTERNAL_ERROR` | The destination returned a generic internal failure or timed out. | Retry the sync. If it continues, check the destination's status page for outages, or contact support. |
| `DESTINATION_RATE_LIMIT` | The destination rate-limited the sync. | Wait for the rate limit window to reset, then retry. To prevent recurrence, reduce [sync frequency](/syncs/schedule-sync-ui) or batch volume. |
| `DESTINATION_MISSING_AD_ACCOUNT` | A required ad account is missing from the destination configuration or inaccessible. | Re-check the destination setup and confirm your account has access to the selected ad account. |
| `DESTINATION_PAYLOAD_TOO_LARGE` | The destination rejected the payload as too large. | Reduce batch size in your sync configuration, or trim the number of mapped fields. |
### Salesforce
For additional Salesforce-specific errors returned by the Salesforce API, see [Salesforce: Tips and troubleshooting](/destinations/salesforce#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `SALESFORCE_INACTIVE_USER` | The Salesforce integration user is inactive. | Reconnect the destination with an active Salesforce user. |
| `SALESFORCE_EXPIRED_TOKEN` | The Salesforce access token expired. | [Reauthorize the destination](/destinations/salesforce#connect-to-salesforce) in Hightouch. |
| `SALESFORCE_INACTIVE_ORG` | The Salesforce org is inactive or inaccessible. | Confirm the org is active and the integration user can log in. |
| `SALESFORCE_RESOURCE_DOES_NOT_EXIST` | A referenced Salesforce object, field, or record doesn't exist. | Check the object name, field name, or record ID in your sync configuration. |
| `SALESFORCE_AUTH_FAILURE` | Salesforce authentication failed. | Re-check credentials, connected app settings, and any IP or login restrictions in Salesforce. |
| `SALESFORCE_REQUESTS_LIMIT_EXCEEDED` | Salesforce API limits were exceeded. | Wait for the limit to reset, or reduce sync frequency and concurrency. See [Salesforce API limits](https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_api.htm). |
| `SALESFORCE_NETWORK_DIFFICULTY` | A network or transport failure prevented the Salesforce request from completing. | Retry the sync and check for temporary Salesforce outages on [Salesforce Status](https://status.salesforce.com/). |
| `SALESFORCE_MAX_CALL_STACK` | Salesforce hit a recursion or call-stack limit during processing. | Review Apex triggers, flows, and automation rules that may be recursively updating records during the sync. See [Salesforce: UNABLE_TO_LOCK_ROW](/destinations/salesforce#unable_to_lock_row) for related guidance. |
| `SALESFORCE_UNABLE_TO_LOCK_ROW` | Salesforce row locking prevented the write because another process is modifying the same record. | Retry later, reduce concurrent updates, or lower your [parallelization setting](/destinations/salesforce#batching-and-parallelization). See [Salesforce: UNABLE_TO_LOCK_ROW](/destinations/salesforce#unable_to_lock_row) for detailed steps. |
### Salesforce Marketing Cloud (SFMC)
For additional SFMC-specific errors, see [SFMC: Tips and troubleshooting](/destinations/sfmc#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `SFMC_DATA_EXTENSION_ADD_UPDATE_ONLY` | This SFMC sync path only supports add and update operations. | Change the sync mode or target a Data Extension that supports the requested operation. |
| `SFMC_IMPORT_FAILED_NO_RECORDS` | The SFMC import failed because no records were present in the batch. | Confirm that the source model returned rows for this sync run. |
### HubSpot
For additional HubSpot-specific errors, see [HubSpot: Tips and troubleshooting](/destinations/hubspot#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `HUBSPOT_RATE_LIMIT_DAILY` | HubSpot's daily API limit was reached. | Wait for the daily limit to reset (typically midnight ET). To prevent recurrence, reduce sync frequency or volume. See [HubSpot API usage guidelines](https://developers.hubspot.com/docs/api/usage-details). |
| `HUBSPOT_RATE_LIMIT_TEN_SECONDLY` | HubSpot's rolling 10-second rate limit was reached. | Hightouch automatically retries these. If the error persists across runs, reduce concurrent syncs to HubSpot. |
| `HUBSPOT_RATE_LIMIT_SECONDLY` | HubSpot's per-second rate limit was reached. | Hightouch automatically retries these. Reduce burst volume if the error recurs. |
| `HUBSPOT_INVALID_REFRESH_TOKEN` | The HubSpot OAuth refresh token is invalid or expired. | Reconnect the HubSpot destination in Hightouch to generate a new token. |
| `HUBSPOT_SYNC_ERROR` | HubSpot returned a generic sync error. | Retry the sync. If it continues, check HubSpot-side logs and object validation rules. |
| `HUBSPOT_CONNECTION_ERROR` | A connection failure prevented the HubSpot request from completing. | Retry and check for temporary HubSpot outages on [HubSpot Status](https://www.hubspotstatusupdates.com/). |
| `HUBSPOT_CONTACT_ALREADY_EXISTS` | A contact with this email already exists in HubSpot. | Review your record matching configuration. HubSpot uses email as the primary unique identifier. See [HubSpot: 409 - Contact already exists](/destinations/hubspot#409---contact-already-exists) for detailed steps. |
| `HUBSPOT_CLOUDFLARE_BLOCKED_IP` | HubSpot's Cloudflare proxy blocked the request IP. | If you use [Hightouch's static IPs](/security/networking#ip-addresses), confirm they aren't blocked by your HubSpot account's network settings. Contact HubSpot support if needed. |
### Google Ads
For additional Google Ads-specific errors, see [Google Ads: Tips and troubleshooting](/destinations/google#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `GOOGLE_ADS_INTERNAL_SERVER_ERROR` | Google Ads returned an internal server error. | Retry the sync after a short delay. Check [Google Ads Status](https://ads.google.com/status/publisher/) for outages. |
| `GOOGLE_ADS_USER_LIST_NOT_FOUND` | The target Google Ads audience doesn't exist. | Confirm the audience still exists in Google Ads and the connected account has access. |
| `GOOGLE_ADS_UNAUTHORIZED` | Google Ads rejected the request due to missing permissions or invalid authorization. | Reauthorize the destination and confirm the connected account has the required access level. |
| `GOOGLE_ADS_USER_LIST_NAME_ALREADY_IN_USE` | An audience with this name already exists in Google Ads. | Use a unique audience name, or connect the sync to the existing audience. |
| `GOOGLE_ADS_CONCURRENT_MODIFICATION` | Google Ads rejected the update because another change is still processing. | Wait a few minutes and retry. Avoid running multiple syncs to the same audience simultaneously. |
| `GOOGLE_ADS_STORE_SALES_DIRECT_DATA_NOT_ALLOWED` | Store sales direct data isn't enabled for this Google Ads account. | Confirm account eligibility and enable the feature in Google Ads. See [Google Ads Store Sales requirements](https://support.google.com/google-ads/answer/7620302). |
| `GOOGLE_ADS_GENERIC_INVALID_ARGUMENT` | Google Ads rejected the request as invalid. | Review required fields, identifiers, and mapped values. Use the [live debugger](/syncs/debugger) to inspect the rejected payload. |
### Braze
For additional Braze-specific errors, see [Braze: Tips and troubleshooting](/destinations/braze#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `BRAZE_SERVICE_UNAVAILABLE` | Braze is temporarily unavailable. | Retry the sync after a short delay. Check [Braze Status](https://braze.statuspage.io/) for outages. |
| `BRAZE_ACCESS_DENIED` | Braze denied access to the API. | Confirm the API key has the required permissions in your Braze dashboard. |
| `BRAZE_INVALID_REQUEST` | Braze rejected the request content or structure. | Check required fields and value formats. Use the [live debugger](/syncs/debugger) to inspect the rejected payload. |
### TikTok
For additional TikTok-specific errors, see [TikTok: Tips and troubleshooting](/destinations/tiktok#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `TIKTOK_GATEWAY_TIMEOUT` | TikTok timed out while processing the request. | Retry the sync after a short delay. |
| `TIKTOK_REJECTED_REPLACE_OP` | TikTok rejected a replace operation because another audience job is still processing. | Wait for the existing job to complete before retrying. |
| `TIKTOK_AUDIENCE_ERROR` | The TikTok audience is invalid or not editable. | Confirm the audience exists in TikTok Ads Manager and supports the requested operation. |
| `TIKTOK_PERMISSION_DENIED` | The connected TikTok account lacks the required permissions. | Reauthorize with a TikTok account that can manage the target advertiser and audience. |
| `TIKTOK_INVALID_ENCRYPTION_VALUE` | TikTok rejected the configured encryption or hashing value. | Verify the expected hashing format in your TikTok sync configuration. |
### Google Sheets
For additional Google Sheets-specific errors, see [Google Sheets: Tips and troubleshooting](/destinations/google-sheets#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `GOOGLE_SHEETS_INVALID_GRANT` | The Google Sheets authorization grant is invalid. | Reauthorize the destination in Hightouch. |
| `GOOGLE_SHEETS_INVALID_DATA_TYPE` | The sync tried to write an unsupported value type to a cell. | Cast complex values (arrays, objects) to plain text or supported scalar types in your model. |
| `GOOGLE_SHEETS_INVALID_RANGE` | The configured write range is invalid. | Check the tab name, cell range, and sheet structure in the destination configuration. |
| `GOOGLE_SHEETS_EXCEEDS_GRID_LIMITS` | The write would exceed the spreadsheet's grid limits (10 million cells). | Reduce the number of rows or columns, or write to a new sheet. |
| `GOOGLE_SHEETS_MISSING_SHEET_ID` | The spreadsheet or sheet identifier is missing from the configuration. | Reopen the destination setup and confirm the selected sheet. |
| `GOOGLE_SHEETS_QUOTA_EXCEEDED` | The Google Sheets API quota was exceeded. | Retry later, or reduce sync frequency. See [Google Sheets API limits](https://developers.google.com/sheets/api/limits). |
| `GOOGLE_SHEETS_ENTITY_NOT_FOUND` | The target spreadsheet or sheet no longer exists. | Confirm the spreadsheet exists and is shared with the connected Google account. |
### Slack
For additional Slack-specific errors, see [Slack: Tips and troubleshooting](/destinations/slack#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `SLACK_NOT_IN_CHANNEL` | The Hightouch Slack app isn't a member of the target channel. | Invite the Hightouch app to the channel, or select a different channel. |
| `SLACK_IS_ARCHIVED` | The target Slack channel is archived. | Unarchive the channel or choose a different destination channel. |
| `SLACK_INVALID_BLOCKS` | Slack rejected the block payload as invalid. | Validate the [Block Kit JSON](https://api.slack.com/tools/block-kit-builder) structure and required fields. |
| `SLACK_NO_TEXT` | The Slack message is missing required text content. | Add message text to your sync configuration, even if blocks are also present. |
| `SLACK_NON_CALLABLE_ITERATOR` | An unexpected payload construction failure occurred. | Retry once. If it persists, contact support. |
| `SLACK_UNEXPECTED_TOKEN` | Slack rejected the payload due to invalid syntax. | Review templated message content and JSON formatting for syntax errors. |
### Other destinations
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `AMPLITUDE_INVALID_JSON_REQUEST` | Amplitude rejected the request payload format. | Review field mappings and ensure the JSON structure matches [Amplitude's API requirements](https://www.docs.developers.amplitude.com/analytics/apis/http-v2-api/). |
| `CITRUS_AD_INVALID_BASE_URL` | The configured CitrusAd base URL is invalid. | Update the destination with the correct API endpoint. |
| `ITERABLE_INVALID_LIST_ID` | The Iterable list ID is invalid. | Confirm the list still exists in Iterable and the configured ID is correct. |
| `ANAPLAN_BAD_MAPPINGS` | The Anaplan field mappings are invalid. | Re-check dimension, line-item, and field mapping setup. See [Anaplan: Tips and troubleshooting](/destinations/anaplan#tips-and-troubleshooting). |
| `ANAPLAN_NULL_POINTER` | The Anaplan request failed with a null-pointer error. | Retry first. If it persists, check mapped objects and source values for missing required fields. |
| `TABOOLA_MISSING_NUMERIC_ACCOUNT_ID` | The Taboola account ID is missing or not numeric. | Update the destination configuration with the correct numeric account ID. |
| `ELASTICSEARCH_INVALID_METADATA_TYPE` | The Elasticsearch metadata type is invalid. | Check mapping configuration and ensure metadata fields use supported types. |
| `FRESHDESK_CLOUDFLARE_IP_BLOCKED` | Freshdesk's Cloudflare proxy blocked the request IP. | Confirm [Hightouch's static IPs](/security/networking#ip-addresses) are allowlisted, or contact Freshdesk support. |
## Rejected-row errors
Rejected-row errors apply to individual records, not the entire sync run. You can inspect rejected rows in the [live debugger](/syncs/debugger) or [warehouse sync logs](/syncs/warehouse-sync-logs).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `NULL_EXTERNAL_ID` | A required external ID is null for one or more rows. | Ensure the mapped identifier column is populated for every record. Update your model to filter out rows with null identifiers. |
| `DUPLICATE_PRIMARY_KEY` | Multiple source rows share the same primary key value. | Deduplicate your model so each primary key appears only once per sync run. |
### LinkedIn
For additional LinkedIn-specific errors, see [LinkedIn: Tips and troubleshooting](/destinations/linkedin#tips-and-troubleshooting).
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `LINKEDIN_INVALID_CONVERSION` | LinkedIn rejected the conversion payload. | Check that all required conversion fields are present and correctly formatted. |
| `LINKEDIN_EVENT_TOO_OLD` | The event timestamp is outside LinkedIn's allowed time window. | Send more recent events. LinkedIn typically requires events within the last 90 days. |
| `LINKEDIN_MISSING_USER_IDS` | Required LinkedIn user identifiers are missing. | Map at least one accepted identifier (email, Google Advertising ID, or Apple IDFA) for each row. |
| `LINKEDIN_MISSING_INVALID_USER_INFO` | LinkedIn user information is missing or invalid. | Review identifier formatting and required match fields. |
| `LINKEDIN_MISSING_IDENTIFIER_OR_NAME` | A required identifier or name value is missing from the row. | Populate the required matching fields before syncing. |
| `LINKEDIN_INVALID_SEGMENT_VALUE` | A segment value (often email) failed LinkedIn's validation. | Check email formatting and ensure values meet LinkedIn's requirements. |
| `LINKEDIN_PERMISSION_DENIED` | The connected LinkedIn account lacks the required permissions. | Reauthorize with a LinkedIn account that can manage the target audience or conversion action. |
| `LINKEDIN_API_ERROR` | LinkedIn returned a generic row-level API failure. | Retry if the issue appears transient. Use the [live debugger](/syncs/debugger) to inspect LinkedIn's response for the affected rows. |
## Predictive traits errors
These errors occur when training or running [predictive trait](/customer-studio/predictive-traits) models. They indicate a problem with training data, event configuration, or warehouse access.
| Error code | What it means | How to resolve |
| :--- | :--- | :--- |
| `NO_USERS_IN_INPUT_FILE` | The training audience returned no users. | Check that the parent audience or model query returns rows before training starts. |
| `NO_USER_ROWS_AFTER_FILTERS` | Users were present initially, but predictive filters removed all of them. | Loosen the filters or verify that event and property conditions match real data. |
| `NO_TRAINING_DATA_FILES` | Hightouch couldn't generate the training dataset. | Confirm that the configured events return data and the event model is queryable. |
| `LABEL_FREQUENCY_TOO_LOW` | There aren't enough examples of the target outcome to train the model reliably. | Increase the lookback window, reduce filters, or choose a higher-volume outcome event. |
| `NO_POSITIVE_EXAMPLES` | The training data contains zero positive examples of the target event. | Check that the outcome event is occurring for users in scope. |
| `TRAINING_MATRIX_DOWNLOAD_FAILED` | An internal failure occurred while retrieving the training artifact. | Retry the run. If it persists, contact support — this is typically not customer-fixable. |
| `NO_KEY_EVENTS` | No feature-generation events are configured. | Add at least one event input in the predictive trait configuration before retraining. |
| `EVENT_MODEL_MISCONFIGURED` | The event model configuration is invalid. | Confirm the event model exists, has a valid primary key, and includes all referenced columns. |
| `EVENT_QUERY_FAILED` | Hightouch couldn't query event data from the warehouse. | Check warehouse permissions, model SQL, and whether the referenced tables still exist. |
| `SOURCE_ACCOUNT_LOCKED` | The warehouse user account is locked. | Wait for the lockout to clear, or have your warehouse admin re-enable the account. |
| `UNKNOWN_EVENT_IN_FILTER` | A predictive filter references an event Hightouch can't resolve. | Check for renamed, deleted, or misconfigured events in the predictive trait setup. |
| `INVALID_EVENT_PROPERTY_CONDITION` | An event-property filter is malformed. | Verify the filter syntax and confirm you're using valid string property names. |
| `NO_EVENT_DATA_FILES` | No event data was produced for training or inference. | Confirm the configured events have recent data and event pulls are succeeding. |
| `TRAINING_ERROR` | An internal training failure occurred. | Retry once. If it repeats, contact support. |
| `CACHED_EVENT_PULL_DISABLED` | A required event pull is disabled. | Re-enable the event pull in your schema configuration and rerun the predictor. |
| `MALFORMED_EVENT_DATA` | Numeric event fields contain invalid values. | Clean the source data so numeric columns contain only valid numbers (no strings, nulls in numeric-only fields, etc.). |
| `MODEL_QUERY_FAILED` | Hightouch couldn't query the parent model data needed for the run. | Check that the parent model still exists and all referenced columns are present. |
---
## Warehouse tables reference
**URL:** https://hightouch.com/docs/developer-tools/warehouse-tables
**Description:** A reference for all the tables and schemas Hightouch creates in your data warehouse, with usage guidance and stability expectations.
**Section:** Developer tools
Hightouch creates and manages tables in your data warehouse to power change data capture, sync logging, audience snapshots, journeys, identity resolution, and more. This page is a single reference for all of these tables: which schema they belong to, which feature creates them, and how you should (and shouldn't) use them.
## Usage guidance
Hightouch writes these tables to your warehouse for observability, debugging, and lightweight reporting. Before building on top of them, keep the following in mind.
### Recommended uses
- **Ad-hoc debugging and auditing.** Query sync logs to investigate failed rows, inspect audience membership changes, or trace a user's journey path.
- **Lightweight reporting.** Build dashboards on top of sync run metadata, audience snapshots, or identity resolution outputs to monitor Hightouch activity.
- **Analytics and activation with Identity Resolution outputs.** The `_resolved`, `_resolved_identifiers`, and `_golden_records` tables are explicitly designed for downstream use. You can query them directly, register them as dbt sources, or sync from them.
### Not recommended
- **Wiring tables into production workflows.** Don't build business-critical pipelines that depend on these tables without . Table schemas may change as Hightouch evolves, and we don't guarantee schema stability for all tables.
- **Writing to Hightouch-managed tables.** Never insert, update, or delete rows in the `hightouch_planner` schema (especially `JOURNEY_LOG` tables and `_plan`/`_rejections` tables). These tables are actively used during sync and journey execution — modifying them can cause unexpected behavior.
- **Depending on internal state tables.** Tables prefixed with `IDR_` or `IDR_BACKUP_` are internal to Identity Resolution. Don't query or reference them in pipelines — they can change without notice.
Documenting a table here doesn't mean Hightouch is committing to a stable
schema contract. If you want to build durable integrations on top of these
tables,{" "}
{" "}
so we can help you find the right approach.
---
## Schemas overview
Hightouch uses up to three schemas in your warehouse, depending on which features you enable:
| Schema | Purpose | Created when |
| -------------------------- | --------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ |
| `hightouch_planner` | Change data capture, audience snapshots, journey logs, Identity Resolution outputs, AI Decisioning tables | [Lightning sync engine](/syncs/lightning-sync-engine) is enabled |
| `hightouch_audit` | Warehouse Sync Logs, holdout group logs, journey views, data extraction reports | Lightning sync engine is enabled; individual features populate specific tables |
| Custom (per Events source) | Event data ([Hightouch Events](/events/warehouses/schema)); schema name is configurable | An Events source writes to a warehouse destination |
---
## hightouch_planner schema
### Lightning sync engine (CDC)
These tables are created automatically when the [Lightning sync engine](/syncs/lightning-sync-engine) is enabled. They're **required** for syncs using the Lightning engine.
These are internal change data capture working tables, so you'll rarely query them directly. The occasional exception is `*_rejections`, which you can inspect to see rows Hightouch dropped during a specific run.
| Table | Key columns | Description |
| -------------- | ----------- | ------------------------------------------------------------------------------------------------------------------ |
| `*_plan` | — | Stores a model's query results for a sync run, used to compute row-level diffs. Table names change with every run. |
| `*_rejections` | — | Stores rows rejected during a sync run. Table names change with every run. |
Hightouch only keeps the two most recent pairs of `_plan` and `_rejections` tables per sync. Because table names change with every run, Hightouch requires write access to the entire schema.
Don't delete tables from `hightouch_planner`. Removing CDC tables breaks
change data capture and requires a [full
resync](/syncs/overview#resync-full-query) to recover.
For full details, see [Lightning sync engine — Warehouse schemas](/syncs/lightning-sync-engine#warehouse-schemas).
### Audience snapshots
**Optional** — must be enabled on the source's **Sync Logs** tab. See [Audience snapshots](/customer-studio/audience-snapshots).
Use this table to see how an audience's membership changed over time — for example, who entered or left a segment between two dates.
| Table | Key columns | Description |
| --------------------- | -------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
| `audience_membership` | `ht_row_id`, `ht_audience_id`, `ht_timestamp`, `ht_event_type`, `ht_split_group` | Tracks audience membership changes over time. Each row records an `enter` or `exit` event for a member. |
### Journey tables
Created **automatically** when a journey is active. See [Journeys — Journey logs](/customer-studio/journeys#journey-logs).
Use these tables when you're analyzing a single journey — for example, tracing where users drop off or auditing the path a specific user took. Each journey gets its own set of tables. To report across all journeys at once, use the combined [journey views](#journey-views) in the `hightouch_audit` schema instead. Those views cover `JOURNEY_LOG` and `JOURNEY_METADATA`; there's no combined view for `JOURNEY_CONTEXT_LOG`, so query the per-journey context tables directly when you need journey variable values across journeys.
Use this table to decide which object to query:
| Goal | Query | Schema |
| ---------------------------------- | ----------------------------------------------------------------------------- | ------------------- |
| Debug or audit a single journey | `JOURNEY_LOG_`, `JOURNEY_METADATA_` | `hightouch_planner` |
| Report across all journeys at once | `JOURNEY_LOG_VIEW_`, `JOURNEY_METADATA_VIEW_` | `hightouch_audit` |
| Read journey variable values | `JOURNEY_CONTEXT_LOG_` (no combined view) | `hightouch_planner` |
Don't write to `JOURNEY_LOG` tables. They're actively used during journey
execution — inserting, updating, or deleting rows can cause the journey to
behave unexpectedly.
The `journey_id` in table names is the journey's UUID with dashes removed.
#### JOURNEY_LOG
`JOURNEY_LOG_` logs the progress of all rows through a given journey. Each time a row enters the journey, moves from one node to another, or exits the journey, Hightouch creates an entry in this table. Since a row can enter a journey more than once, the `row_instance_id` column uniquely identifies each entry instance as it moves through the journey.
Resetting a journey clears its log table.
| Column | Type | Description |
| ----------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `row_id` | string | The primary key from the journey's parent model that this row represents. |
| `row_instance_id` | string | A UUID to uniquely identify a `row_id` each time it enters the journey. |
| `run_id` | string | The ID of the journey run that executed this operation. Internal detail. |
| `from_node_id` | string | The node this action originates from. For moves or exits, it's the node the row moved or exited from. For entries, it's `NULL`. |
| `to_node_id` | string | The node this action targets. For moves and entries, it's the node the row moved into. For exits, it's `NULL`. |
| `timestamp` | timestamp | The effective timestamp of this operation. This doesn't always represent the actual time the operation occurred — for example, if a row moved due to an event, this is the event timestamp rather than when the warehouse query ran. |
| `event_type` | string | The type of event: `moved-to-node`, `entered-journey`, `exited-journey-by-criteria`, or `exited-journey`. |
#### JOURNEY_METADATA
`JOURNEY_METADATA_` contains the journey's graph structure and customer-defined node names. Updated each time a change is saved to the journey.
| Column | Type | Description |
| -------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `journey_id` | text | The ID of the journey. |
| `journey_name` | text | The customer-defined name of the journey. |
| `node_name` | text | The customer-defined name of the node. |
| `node_type` | text | The node type: `entry-cohort`, `entry-event`, `sync`, `segments`, `segment-branch`, `time-delay`, `splits`, `split-branch`, `wait-until-event`, `wait-until-event-branch`. |
| `node_id` | text | The ID of the journey node. |
| `to_nodes` | text[] | An array of the outbound node IDs from the given node. |
#### JOURNEY_CONTEXT_LOG
`JOURNEY_CONTEXT_LOG_` logs the result of journey context [variables](/customer-studio/journeys#variables) for each row.
| Column | Type | Description |
| ----------------- | --------- | ---------------------------------------------------------------------------------------------------------- |
| `row_id` | string | The primary key from the journey's parent model that this row represents. |
| `row_instance_id` | string | A UUID to uniquely identify a `row_id` each time it enters the journey. |
| `run_id` | string | The ID of the journey run that executed this operation. Internal detail. |
| `node_id` | string | The ID of the journey node. |
| `entered_at` | timestamp | When the row entered the "set a variable" node. |
| `inserted_at` | timestamp | Used for prioritizing when multiple context nodes assign the same variable. The most recent value is used. |
| `[variable_name]` | dynamic | One dynamic column per context variable. The type depends on the variable's type. |
### Identity Resolution outputs
Created **automatically** when an identity graph runs. See [Identity Resolution](https://hightouch.com/platform/identity-resolution) for details.
By default, these tables are written to `hightouch_planner`. If you configure an output schema for the identity graph, Hightouch writes them to that schema instead.
Use these tables to power analytics and downstream activation on resolved identities. They're the recommended starting point for anything you build on top of Identity Resolution — query them directly, register them as dbt sources, or sync from them.
| Table | Key columns | Description |
| -------------------------------------- | ---------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| `_resolved` | `ht_id`, `source`, `primary_key`, `latest_timestamp` | Maps every input row to a resolved identity. |
| `_resolved_identifiers` | `ht_id`, `identifier`, `value`, `first_timestamp`, `last_timestamp`, `count` | All identifier values associated with each resolved identity. |
| `_unresolved` | — | Input rows that couldn't be processed, typically due to duplicate primary keys. |
| `_golden_records` | `ht_id`, plus one column per configured field | One row per identity with canonical field values. Only created when [Golden Record](https://hightouch.com/platform/identity-resolution) is enabled. |
These four output tables **are designed for your use**. You can safely query
them, reference them in dbt, or build views on top of them. For long-term
stability, we recommend creating a view or derived table rather than
transforming them in place.
### Identity Resolution internal state
Created **automatically** as part of identity graph processing. These tables support incremental resolution and graph consistency behind the scenes — they aren't meant for your use.
| Table | Description |
| ------------------------ | ----------------------------------------------------------------- |
| `IDR_*` / `IDR_BACKUP_*` | Internal tables for incremental processing and graph consistency. |
Don't query or modify these tables. They're internal implementation details
and can change without notice.
### AI Decisioning
Created **automatically** per agent when using Hightouch-assigned groups. See [AI Decisioning — Group assignment](/ai-decisioning/group-assignment#experiment-groups-table-in-the-warehouse).
Use this table to analyze experiment results — compare how the `treatment`, `holdout`, and `customer_managed` groups performed to measure AI Decisioning's impact.
| Table | Key columns | Description |
| -------------------------------------- | --------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `de_user_experiment_groups_` | `user_id`, `user_hash`, `experiment`, `first_seen_at`, `last_seen_at` | Tracks which experiment bucket (`treatment`, `holdout`, `customer_managed`) each user belongs to over time. |
---
## hightouch_audit schema
### Warehouse Sync Logs
**Optional** — must be enabled per source or per sync. See [Warehouse Sync Logs](/syncs/warehouse-sync-logs).
Use these to audit sync outcomes: investigate why specific rows failed (`sync_changelog`), check the latest status of each row (`sync_snapshot`), or report on run-level metrics like planned, succeeded, and failed counts (`sync_runs`).
| Table | Key columns | Description |
| ---------------- | ----------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| `sync_changelog` | `sync_id`, `row_id`, `op_type`, `status`, `failure_reason`, `fields` | A log of every operation across all sync runs. Each synced row gets one entry per run. |
| `sync_snapshot` | `sync_id`, `row_id`, `op_type`, `status`, `failure_reason`, `fields` | The latest status of each row across all syncs. Replaced after each run. |
| `sync_runs` | `sync_id`, `sync_run_id`, `model_name`, `started_at`, `finished_at`, `status` | Metadata for each sync run, including row counts for planned, attempted, succeeded, and failed operations. |
### Holdout group logs
**Optional** — requires a feature flag and must be enabled on the source's **Sync Logs** tab. See [Experiments — Holdout group logs](/customer-studio/experiments#holdout-group-logs).
Use this table after a campaign to measure holdout performance — compare outcomes for held-out members against those who received the campaign.
| Table | Key columns | Description |
| ------------------ | ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
| `audience_holdout` | `sync_id`, `sync_run_id`, `model_id`, `row_id`, `fields`, `split_group` | Logs rows excluded from a sync due to holdout group membership, for post-campaign analysis. |
### Journey views
Created **automatically** when journeys are active. These are read-only views that combine every journey's [per-journey tables](#journey-tables) into a single workspace-level view. Use them for reporting across all journeys at once, rather than querying and joining individual `JOURNEY_LOG` and `JOURNEY_METADATA` tables. Only those two tables have combined views — for journey variable values, query the per-journey `JOURNEY_CONTEXT_LOG` tables directly.
Find your workspace slug under **Settings → Workspace → Workspace slug**.
Replace dashes with underscores in the slug when querying (for example,
`my-workspace` becomes `my_workspace`).
#### JOURNEY_LOG_VIEW
`JOURNEY_LOG_VIEW_` combines all `JOURNEY_LOG` tables into a single view of row movement across all journeys. Updated each time a journey is added or removed from the workspace.
| Column | Type | Description |
| ----------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `source_table` | string | The name of the table this row came from: `journey_log_`. |
| `row_id` | string | The primary key from the journey's parent model that this row represents. |
| `row_instance_id` | string | A UUID to uniquely identify a `row_id` each time it enters the journey. |
| `run_id` | string | The ID of the journey run that executed this operation. Internal detail. |
| `from_node_id` | string | The node this action originates from. For moves or exits, it's the node the row moved or exited from. For entries, it's `NULL`. |
| `to_node_id` | string | The node this action targets. For moves and entries, it's the node the row moved into. For exits, it's `NULL`. |
| `timestamp` | timestamp | The effective timestamp of this operation. This doesn't always represent the actual time the operation occurred — for example, if a row moved due to an event, this is the event timestamp rather than when the warehouse query ran. |
| `event_type` | string | The type of event: `moved-to-node`, `entered-journey`, `exited-journey-by-criteria`, or `exited-journey`. |
| `journey_id` | text | The ID of the journey. |
#### JOURNEY_METADATA_VIEW
`JOURNEY_METADATA_VIEW_` combines all `JOURNEY_METADATA` tables into a single view of node structures across all journeys. Updated each time a journey runs.
| Column | Type | Description |
| -------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `source_table` | string | The name of the table this row came from: `journey_metadata_`. |
| `journey_id` | text | The ID of the journey. |
| `journey_name` | text | The customer-defined name of the journey. |
| `node_name` | text | The customer-defined name of the node. |
| `node_type` | text | The node type: `entry-cohort`, `entry-event`, `sync`, `segments`, `segment-branch`, `time-delay`, `splits`, `split-branch`, `wait-until-event`, `wait-until-event-branch`. |
| `node_id` | text | The ID of the journey node. |
| `to_nodes` | text[] | An array of the outbound node IDs from the given node. |
### Data extraction
Created **automatically** when data extraction is enabled for a destination. See [Data extraction](/syncs/data-extraction).
Use these to see the audiences Hightouch maintains in each ad platform — query the combined `external_audiences_metadata_v1` view to report on audience sizes and refresh times across destinations.
| Table | Key columns | Description |
| ----------------------------------------------- | --------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- |
| `{destination-type}_{destination-id}_audiences` | Varies by ad platform | One table per enabled destination. Schema mirrors the ad platform's API response. |
| `external_audiences_metadata_v1` (view) | `ad_account_id`, `audience_id`, `audience_name`, `audience_size`, `ht_fetched_at` | A combined, standardized view across all destination tables. |
---
## Events schema
[Hightouch Events](/events/warehouses/schema) writes event data into a **configurable schema** — not `hightouch_planner` or `hightouch_audit`. The schema name defaults to an auto-generated value based on the event source but can be configured in destination settings. Tables are created **automatically** when an Events source writes to a warehouse destination.
Use these tables to analyze collected behavioral data directly in your warehouse — for example, build models on `tracks` events or join `identifies` traits into your audiences.
| Table | Key columns | Description |
| ---------------- | -------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| `identifies` | `id`, `anonymous_id`, `user_id`, ``, `timestamp` | All `identify` events. Trait keys become separate columns. |
| `tracks` | `id`, `anonymous_id`, `user_id`, `event`, `event_text`, `timestamp` | All `track` events. |
| Per-event tables | Varies | A separate table for each track event type, with event properties as columns. Can be disabled. |
| `pages` | `id`, `anonymous_id`, `user_id`, ``, `timestamp` | All `page` events. |
| `screens` | `id`, `anonymous_id`, `user_id`, ``, `timestamp` | All `screen` events. |
| `groups` | `id`, `anonymous_id`, `user_id`, `group_id`, ``, `timestamp` | All `group` events. |
For the full column-level schema, see [Events warehouse schema](/events/warehouses/schema).
---
## Approval flows
**URL:** https://hightouch.com/docs/workspace-management/approval-flows
**Description:** Require approval before changes to models and syncs are published.
**Section:** Workspace management
Approval flows let you stage new resources ([models](/getting-started/concepts#models) and [syncs](/getting-started/concepts#syncs)) and changes to existing resources in **draft mode** until an approver reviews them.
Approval flows apply only to **models** and **syncs**. Other resources, such as [Audiences](/customer-studio/usage) and [Journeys](/customer-studio/journeys), are published immediately.
## Getting started
To enable approval flows:
1. Navigate to [**Settings**](https://app.hightouch.com/settings?from=docs) in the right sidebar.
2. Open the **Workspace** tab.
3. Enable **Require approvals**.
4. Select **Save**.
After you enable approval flows, any role with the `Draft models & syncs` permission can save changes as drafts or submit them for approval — not just the built-in **Workspace draft editor** role.
Whether a user must get approval depends on their role:
- Users with draft-only roles must submit changes for approval before publishing.
- Users with publishing permissions can submit changes for approval or use **Publish now** to publish directly.
- Users with the `Approve models & syncs` permission can review, approve, decline, or withdraw approval requests.
### Require approval for a user
To require a user's model and sync changes to go through approval, assign them a role that enforces drafts.
#### Use the Workspace draft editor role
1. Go to **Settings > Members**.
2. Find the user.
3. Change their role to **Workspace draft editor**.
Users with this role can create and edit models and syncs, but their changes must be approved before they are published.
#### Create a custom draft role
1. Go to **Settings > Roles**.
2. [Create a custom role](/workspace-management/roles#custom-roles).
3. Open the **Sources** or **Destinations** tab.
4. Select `Draft models & syncs` for the relevant resources.
Users with this permission can draft changes but do not see the **Publish now** option.
### Who can publish directly
Some roles can bypass approval and publish directly. For example, users with the **Admin** or **Workspace editor** role can save drafts, submit approval requests, or select **Publish now**.
Any role without the `Draft models & syncs` restriction can self-publish.
The **Workspace draft editor** role is the built-in role whose changes must be approved. It is not the only role that can create drafts. Any role with the `Draft models & syncs` permission can create and edit drafts. Roles that can publish directly can also save drafts voluntarily but still have access to **Publish now**.
If you use approval flows, we recommend the following role structure:
- **Admin** — Can approve changes and manage the workspace.
- **Workspace editor** — Can approve changes and publish directly.
- **Workspace draft editor** — Can create and edit models and syncs, but changes require approval before publishing.
For more advanced setups, [create custom roles](/workspace-management/roles#custom-roles) that include or exclude the `Approve models & syncs` permission on model and sync resources.
## Create new resources
When you create a new model or sync, the available actions depend on your role:
- **Publish now** publishes the resource immediately. Only available to roles that can bypass approval.
- **Submit approval request** sends the draft to one or more approvers for review. You can include a comment.
- **Save as draft** saves the resource without notifying anyone. You can keep editing and submit it for approval later.
If your role requires approval, you do not see **Publish now**. You can only submit the resource for approval or save it as a draft.
If a sync uses a model that is still in draft mode, both the model and the sync must be published before the sync can run.
## Edit existing resources
Editing existing models and syncs uses the same approval options as [creating new resources](#create-new-resources).
The following changes require approval:
| Resource | Changes requiring approval |
| -------- | --------------------------------------------- |
| Model | Query, configuration, and column type changes |
| Sync | Configuration and schedule changes |
If a sync uses a model with unpublished changes, the sync refers to the draft version of the model instead of the published version. Both the model and the sync must be published before the sync can run.
## Approve requests
[Approvers](#who-can-approve-requests) can view approval requests by selecting **View request** on the corresponding resource.
When reviewing a request, approvers can choose one of the following actions:
- **Approve & publish** publishes the new resource or drafted changes.
- **Delete draft** deletes the draft. New resources or drafted changes are discarded and cannot be recovered.
- **Withdraw request** removes the approval request but keeps the draft. Users can continue editing and submit it again later.
Any user with approval permission and access to the resource can approve the request, even if they were not selected as the reviewer.
For sync changes, the approval request shows a comparison of changes to the sync configuration. For model changes, the approval request shows changes to the model query.
### Who can approve requests
The `Approve models & syncs` [permission](/workspace-management/roles#custom-role-grant-reference) controls who can approve requests. By default, all roles [except **Workspace draft editor** and **Workspace viewer**](/workspace-management/roles#differences-when-approval-flows-are-enabled) include this permission:
- **Admin**
- **Workspace editor**
- **Model + sync editor**
- **Sync editor**
- **Source admin**
- **Destination admin**
- **Audience editor**
To grant or remove approval permissions for a [custom role](/workspace-management/roles#custom-roles), add or remove the `Approve models & syncs` permission for the relevant sources or destinations. Users with this permission can use **Approve & publish**, **Delete draft**, and **Withdraw request** when reviewing a draft.
---
## Audit logs
**URL:** https://hightouch.com/docs/workspace-management/audit-logs
**Description:** Hightouch offers audit logs so workspace admins can see what actions users took in-app and when.
**Section:** Workspace management
Hightouch offers audit logs so workspace admins can see what actions users took in-app and when. For example, suppose someone notices that a sync has sent incorrect data to a destination. A workspace admin can use audit logs to review changes made to the sync before the incorrect data appeared. The admin can correct the issue, and—if necessary—implement [controls](/workspace-management/overview#access-management) to avoid future incidents.
## View audit logs
Workspace admins and those [with the appropriate access](#who-can-view-audit-logs) can view logs by going to the [**Audit log** tab](https://app.hightouch.com/settings/audit-log?from=docs) on the **Settings** page. The logs consist of a table of actions beginning with the most recent. Each row represents one action.
Each action includes:
- The **Date** and time, in the viewer's timezone, the action took place
- The **Action** name, for example, "Sync updated"
- The **User** who performed the action
- The **Resource type** of the resource the action was performed on
- The **Resource name** of the resource the action was performed on
Clicking on an action displays the action's full details in JSON format in the right panel. Added lines are highlighted in green, while removed lines are highlighted in red.
## Filter audit logs
You can filter logs by:
- the user who performed the action
- the resource type the action was performed on
- the resource name the action was performed on
- a range of dates
You can select multiple users and resource types to filter by simultaneously.
## FAQ
### Who can view audit logs?
Anyone with read access to the workspace resource can view audit logs.
[Pre-built roles](/workspace-management/roles#pre-built-roles) with these permissions include Admins, Destination Admins, and Source Admins. If you want to create a [custom role](/workspace-management/roles#custom-roles) with the ability to view audit logs, it should include this policy:
```JSON
{
"effect": "allow",
"actions": "read",
"resource": [
"workspace"
]
}
```
### How far back do audit logs go?
Audit logs track up to 90 days of activity.
### What do audit logs include?
Audit logs track user-driven activity vs. programmatic changes. They work by taking a JSON snapshot of the changed resource for every user-driven change. Hightouch uses these snapshots for a before and after comparison for any changes.
A snapshot may include programmatic fields such as `last_run_at`, but Hightouch doesn't append a new audit log every time a field like `last_run_at` is updated since sync runs can be configured to run at regular intervals.
The exception to this is activity driven by [Git Sync](/extensions/git-sync). For example, if a user changes a configuration via [Git Sync](/extensions/git-sync), Hightouch treats that as a user-driven change, and it appears in the audit logs.
For a complete list of tracked actions, see the following [list](#resource-types-and-tracked-actions).
## Resource types and tracked actions
Audit logs show creation, update, and deletion type actions for the following resource types:
| Resource type | Tracked actions |
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Alert** | Creations, updates, or deletions of [alerts](/syncs/alerting) you configure to inform you of syncing issues |
| **API key** | Creations and deletions of API keys _Note: Audit logs don't include API Key usage_ |
| **Approval request** | Actions taken during [approval flows](/workspace-management/approval-flows), including sending a request to review a draft or approving a draft |
| **Audience** | Creations, updates, or deletions of [audience](/customer-studio/overview) configurations |
| **Audience schema** | Creations, updates, or deletions of [parent models, relationships, and event models](/customer-studio/schema) |
| **Cloud credential** | Creations, updates, or deletions of cloud credentials, including changing your [external storage configuration](/security/storage) |
| **Draft** | Creations, updates, or deletions of drafts used in [approval flows](/workspace-management/approval-flows) |
| **Git credential** | Creations, updates, or deletions of Git credentials used for [Git Sync](/extensions/git-sync) |
| **Git Sync configuration** | Creations, updates, or deletions of [Git Sync](/extensions/git-sync) configurations; for example, changing your tracked Git branch |
| **Invite** | Creations and deletions of workspace invitations |
| **Membership** | Creations, updates, or deletions of workspace members, including changing a user's role |
| **Membership request** | Creations and deletions of workspace membership requests |
| **Model** | Creations, updates, or deletions of general model configurations, including updating the model query or primary key |
| **Model column** | Creations, updates, or deletions of model columns |
| **Model preview** | Actions taken while previewing a model, including a user clicking “Preview” while editing a model or running a SQL query against a source during model setup |
| **Model trait definition** | Creations, updates, or deletions of audience traits |
| **Role** | Creations, updates, or deletions of [custom roles](/workspace-management/roles#custom-roles) |
| **Session** | User sign-ins and sign-outs |
| **Source** | Creations, updates, or deletions of source configurations, including updating credentials or adding labels |
| **Sync** | Creations, updates, or deletions of sync configurations, including adding labels, scheduling, and turning a sync on or off |
| **Sync run** | Actions taken while viewing a sync; for example, a user clicking “Run” _Note: Audit logs don't include scheduled sync runs nor API-triggered sync runs_ |
---
## Environments and deployments
**URL:** https://hightouch.com/docs/workspace-management/environments
**Description:** Hightouch supports change management in-app via separate workspaces for different environments, for example, production, staging, development, and sandbox.
**Section:** Workspace management
**Not sure whether you need environments or a separate workspace?**
Environments are the right choice when your teams share the same sources,
destinations, and user roster but need a safe promotion path (dev → staging →
prod). If you need hard isolation — separate sources, separate users, nothing
shared — you need a separate workspace instead. For a side-by-side comparison,
see [Do you need another
workspace?](/workspace-management/workspaces#do-you-need-another-workspace)
Maintaining separate [environments](https://en.wikipedia.org/wiki/Deployment_environment) for different stages of the release process is a common practice across technical domains.
Separating environments minimizes the risk of accidental changes or disruptions to the user-facing experience.
For example, environments can help you avoid overwriting CRM data you didn't mean to, activating a marketing campaign prematurely, or incurring unexpected data warehouse charges.
Generally, production environments are for live, user-facing applications, staging for pre-release testing, and development for experimentation.
You can set up whatever environments your organization requires in Hightouch.
## How do environments work?
Each Hightouch [workspace](/workspace-management/overview#core-concepts) in your organization acts as a separate environment.
You can link resources between workspaces, which allows you to **deploy** [models and syncs](/getting-started/concepts) from one environment to another.
Each workspace can have its own set of [alerts](/syncs/alerting) and can use its own extensions.
Environments integrate seamlessly with other governance features, such as [role-based access controls](/workspace-management/roles) (RBAC) and [approval flows](/workspace-management/approval-flows).
Workspace [permissions](/workspace-management/roles) and [drafts](/workspace-management/approval-flows) are enforced across workspaces when deploying changes.
You can set up your organization so that specific team members have access to certain resources.
For example, some team members, such as your data team, may have permission to set up sources and models.
Others, such as the marketing team, can deploy or draft sync deployments.
### Separate resources for separate environments
Each workspace has its own set of resources, including [sources](/getting-started/concepts#sources), [models](/getting-started/concepts#models), [destinations](/getting-started/concepts#destinations), and [syncs](/getting-started/concepts#syncs).
Hightouch allows you to link resources across workspaces.
For example, you can link a Snowflake playground in your "dev" workspace to a production Snowflake instance in your "prod" workspace,
a Salesforce sandbox to your production Salesforce instance, etc.
Environments allow you to **deploy** changes to models and syncs.
When you deploy a model or sync, Hightouch copies its configuration to another workspace, updating or creating resources where necessary.
Before performing the copy, Hightouch checks that all [underlying resources](#resource-dependencies) are appropriately configured.
For example, you may be experimenting with changes to a Salesforce sync [mapping](/syncs/mapping-data) in your development environment.
When you're happy with your change, you can deploy it to your production environment.
Before copying your changes to a production sync, Hightouch checks that the underlying model and destination have linked counterparts in production.
Keeping resources separate between environments lets you create as many as you want in a development environment without cluttering other environments you may want to keep pristine.
And since changes don't automatically deploy to linked resources,
you can safely delete or update resources without worrying about knock-on effects in other environments.
Each resource also has its own history, which you can inspect, for example, by examining its activity.
### Resource dependencies
While any resources of the same type can be linked, for example, source to source and sync to sync, **you can only deploy models and syncs**.
To deploy models and syncs, you must link their underlying sources and destinations to their relevant counterparts.
Environments apply to **Reverse ETL models and syncs only**. Customer Studio
resources — including audiences, traits, parent models, and schema models —
**cannot be deployed between environments**. Each environment must be
configured and maintained independently for Customer Studio workflows. For
Customer Studio change management, use [Approval
Flows](/workspace-management/approval-flows) instead.
When you [deploy](#deploy-changes) models or syncs, Hightouch checks that the underlying resources match linked counterparts in the target environment.
For example, if you link one **source** to another, you can deploy **models** built off the linked source.
To deploy **syncs**, the underlying linked **models** _and_ **destinations** need to match in both workspaces.
## Environment usage
To deploy changes, you first need to link resources.
You can either link existing sources and destinations on their configuration pages,
or link them as part of the [deployment workflow](#deploy-changes).
You only need to link underlying resources once to allow deployments.
Removing links between resources doesn't impact syncs or models you previously deployed.
### Link resources
To link resources, you need [`create`
permissions](/workspace-management/roles#custom-role-grant-reference) for the
relevant resource type (source or destination) in all workspaces you want to
create the link between. For example, if you want to link two sources, you
need [`create`
permissions](/workspace-management/roles#custom-role-grant-reference) for
sources in both workspaces you want to link them in.
You can link existing sources and destinations to their counterparts in other workspaces.
You can only link models and syncs during the deployment workflow.
When you [deploy](#deploy-changes) a resource for the first time, you create and simultaneously link the resource to a counterpart in the target workspace.
During deployments, Hightouch ensures you've linked the necessary [underlying resources](#resource-dependencies).
If you haven't, you can link them directly from the deployment wizard.
#### Link sources and destinations
To link existing sources and destinations, follow these steps:
1. Go to the resource's overview page and open the **Linking** tab.
2. Select the relevant resource for each workspace you want to link and then click **Link**.
Hightouch only displays eligible resources for linking. To be eligible: - the
resources must be of the same type; for example, they both must be Snowflake -
the user creating the link must be a member of both workspaces - both
workspaces must belong to the same organization
#### Link syncs and models
The only way to link syncs to other syncs and models to other models is by [deploying them](#deploy-changes).
Once deployment creates a new sync or model, it's automatically linked for future deployments.
You cannot deploy to existing syncs or models. If you want to link existing syncs or models from one workspace to another,
delete the development or testing version and recreate it by deploying it from your production environment.
Don't hesitate to if you have any questions or concerns.
### Deploy changes
To deploy changes, you need [permission to at least create
drafts](/workspace-management/roles#differences-when-approval-flows-are-enabled)
for the relevant resource type (model or sync) in the target workspace to
which you want to deploy changes. For example, if you want to deploy a sync,
you need permission to create sync drafts in the workspace you want to deploy
the sync to.
To deploy syncs or models, follow these instructions:
1. From a model or sync's configuration page, click the deploy icon to the left of the **Run** button and select **Deploy**.
2. Select the environment you want to deploy the change to. The first time you deploy a particular resource, Hightouch creates a new resource in the target environment. If you've deployed the resource before, the deployment updates the previously created and linked resource.
3. If the relevant [underlying resources](#resource-dependencies) aren't already linked, follow the steps in the modal to link them.
4. The deployment wizard shows what has changed: added, removed, and updated configurations. Review these to ensure they're the changes you want to deploy.
5. For sync deployments, you can choose whether to deploy its [schedule](/syncs/schedule-sync-ui#schedule-types) as well as the other configuration changes.
6. Click **Deploy**.
7. **Syncs created via deployment are disabled by default.** If you want to enable a deployed sync, make sure to do so from its [overview page](/syncs/overview#individual-sync-overview-page) once you've deployed it.
You can't deploy
[drafted](/workspace-management/approval-flows#editing-existing-resources)
changes on a configuration. You first need to get approval and/or publish
changes to a resource before deploying the changes.
#### Deployment validations
Before deploying changes, Hightouch validates dependencies.
Hightouch blocks deployments until you've resolved all dependency issues.
For example, if you try to deploy a [dbt-based](/extensions/dbt-models) model from your staging to production workspace,
but the [dbt extension](/extensions/dbt-models) isn't set up in production, Hightouch blocks the deployment.
Once you've resolved the issue, for example, by configuring a missing extension, you can retry the deployment.
Deployments will clear or replace any existing [drafts or approval requests](/workspace-management/approval-flows) on a sync or model in a target workspace. You will be warned if a deployment will replace a draft during the validation step.
## FAQ
### What permissions do you need to use environments?
To link resources, you need [`create` permissions](/workspace-management/roles#custom-role-grant-reference) for the relevant resource type in both workspaces you want to create the link between.
To deploy changes, you need [permission to create or update drafts](/workspace-management/roles#differences-when-approval-flows-are-enabled) for the relevant resource type in the target workspace you want to deploy changes to.
### Can you link any resource to another resource of the same type?
You can link sources to other sources, models to other models, destinations to other destinations, and syncs to other syncs.
**Customer Studio resources — including audiences, traits, parent models, and
schema models — cannot be linked or deployed between environments.** Each
environment must be set up and maintained independently for Customer Studio.
For Customer Studio change management, use [Approval
Flows](/workspace-management/approval-flows) instead.
Please if this is something you're interested in.
### How do you link existing models and syncs?
The only way to link syncs to other syncs and models to other models is by [deploying them](#deploy-changes).
Once deployment creates a new sync or model, it's automatically linked for future deployments.
If you want to link existing syncs or models from one workspace to another,
delete the development or testing version and recreate it by deploying it from your production environment.
Don't hesitate to if you have any questions or concerns.
### Can you use environments to copy resources?
You can use [deployments](#deploy-changes) to copy existing models and syncs but not sources and destinations.
To copy an existing model or sync to a different workspace, follow the instructions for [deploying changes](#deploy-changes).
### Can you only deploy changes to production environments?
Deploying changes has no directionality.
You can deploy changes from a testing workspace to a production workspace and vice-versa, as long as the [validations](#deployment-validations) pass.
### Can you use environments and deployments with Git Sync?
If you already use [Git Sync](/extensions/git-sync) to manage your syncs and destinations across workspaces, deploying a sync or model to your target workspace would create its corresponding `yaml` file in the configured Git Sync repository.
Beyond updating `yaml` files, environments and deployments don't interact with your Git Sync repositories.
You should use _either_ Git Sync _or_ the in-app environments and deployments feature for your environment management.
### Can you use environments and deployments with Hightouch Event Collection?
At the moment, it's not possible to create deployments of event resources like event syncs, event sources or event destinations and so it isn't currently possible to separate your Event Collection resources into staging and production environments.
---
## Folders, filters, and labels
**URL:** https://hightouch.com/docs/workspace-management/folders-filters
**Description:** Hightouch provides folders, filters, and labels to organize your workspace.
**Section:** Workspace management
Hightouch offers folders, filters, and labels to help you organize and navigate your workspace, whether you're managing a handful of syncs or thousands. These tools allow you to:
- Create dedicated spaces for specific teams
- Organize marketing campaigns
- Group resources by project, department, or any other dimension
## Folders
You can show or hide the folder sidebar by clicking the arrow next to the main app sidebar.
Folders allow you to group your resources like syncs, models, and audiences. These resources can be placed into folders, and you can nest folders without limits.
Only users with the **Workspace admin** role can modify the folder structure
inside a workspace. Other roles can only add or remove syncs from existing
folders.
### Creating folders
To create a folder, click **New folder** and enter a name for your folder.
### Editing and removing folders
To edit or remove a folder, click the three dots next to the folder name and select an action.
By default, removing a folder won’t delete its contents. The resources will simply become unfiled but still accessible. If you want to delete the folder and its resources together, you’ll need to choose the last option from the dropdown menu.
### Moving resources to folders
1. Select the resources you want to move from the syncs, models, or audiences list.
2. Click **Actions**.
3. Click **Move to folder**.
You can move the resources into an existing folder or create a new one during this process.
### Folder hierarchy
Hightouch enforces a shared folder hierarchy across resources. Models and their attached syncs share the same folder structure, as do audiences and their attached syncs.
When you move a sync, Hightouch automatically moves its associated model or audience to the same folder. This helps keep your folder structure organized and consistent across resource types.
## Filters
Filters allow you to quickly narrow down your list of resources.
Hightouch supports different filters depending on the resource type.
### Sync filters
- **Progress**: whether the sync is currently running
- **Destination**: the business tool where the data is being sent
- **Source**: the origin of the model associated with the sync
- **Label**: any custom key/value labels assigned to the sync
- **Health**: whether the sync is healthy (as determined by its alert triggers)
- **Creator**: the user who originally created the sync
- **Match Booster**: indicates whether identifiers are boosted during the sync process
### Model filters
- **Query type**: how the model is defined (e.g., SQL editor, table selector, dbt, etc.)
- **Source type**: the type of source where the data originates (e.g., Snowflake, Databricks, etc.)
- **Created by**: the user who initially created the model
- **Labels**: any custom key/value labels assigned to the model
- **Duplicate primary keys**: indicates if the model contains any duplicate primary keys
- **Match Boosted**: indicates whether the model's identifiers have been boosted
### Audience filters
- **Parent model**: the parent model from which the audience is derived
- **Source**: the source from which the audience's parent model is derived
- **Created by**: the user who initially created the audience
- **Labels**: any custom key/value labels assigned to the audience
- **Duplicate primary keys**: indicates if the parent model contains any duplicate primary keys
- **Subsets**: the subset applied to the parent model when creating the audience
### Applying filters
Multiple filters can be applied simultaneously to refine your results. Hightouch displays only resources that match all the selected filters. For example, if you apply `Health: Unhealthy` and `Destination: Salesforce`, only syncs that are unhealthy and connected to Salesforce will be shown.
If you're inside a folder, filters apply only to the resources within that folder. To filter across all resources, switch to **All syncs**, **All models**, or **All audiences**.
### Saving filters
You can save a set of filters for reuse by creating a **Saved filter**. After selecting your filters, click **Save as** to store them for future use.
Name your saved filter and select its visibility: private (visible only to you) or shared with the entire workspace.
## Labels
Labels in Hightouch are structured as key/value pairs and offer a flexible way to categorize and manage resources. Unlike folders, which group resources in a strict hierarchy, labels provide a more dynamic approach by allowing you to tag resources with multiple labels. This is especially useful when you want to organize resources across different dimensions that don't fit neatly into a folder structure.
For example, while you might use folders to organize resources by team or department, labels are ideal for tracking specific characteristics like campaign type (e.g., `sms`, `email`, `push`) or brand (e.g., `Brand A`, `Brand B`). Labels allow for more granular filtering and flexible organization across all resources, regardless of their folder placement.
### Applying labels
Select the resources you want to label by checking their boxes. Click the **Actions** button at the top of the page and choose **Add labels** from the dropdown.
In the modal, enter your key/value pairs (e.g., `brand = ACME`). Labels can contain letters, numbers, spaces, underscores, and dashes. Then, click **Save changes**.
### Filtering by label
On any top-level resource page, you can expand the **Labels** section in the filter sidebar and include or exclude specific key/value pairs.
---
## Governance best practices
**URL:** https://hightouch.com/docs/workspace-management/governance
**Description:** Learn when to use governance controls like roles, subsets, Spaces, destination rules, approval flows, and environments to manage access, collaboration, and activation safely.
**Section:** Workspace management
## Overview
**Customer Data Platforms (CDPs)** are often used by many teams across an organization, including:
- Marketers building audiences and launching campaigns
- Data teams managing models, permissions, and warehouse access
- Administrators overseeing workspace setup, security, and compliance
- Regional, brand, or business-unit teams working from shared customer data
- Legal, privacy, or operations teams reviewing how data is used
As more teams begin using the same customer data, governance becomes important.
Without clear governance controls, teams can end up with too much access, send data to the wrong destinations, override each other's work, or launch changes without enough review.
Hightouch helps you prevent that by adding governance controls at the **activation layer**.
Because Hightouch runs directly on top of your warehouse, **your warehouse remains the foundation for data access and security**. Hightouch inherits the controls already defined there — if a table, column, or dataset is restricted in your warehouse (e.g. via BigQuery IAM policies, [Snowflake role grants](/sources/snowflake-rbac), Databricks Unity Catalog, or Redshift permissions), Hightouch respects that access automatically. Hightouch then adds controls for how teams work with that data in the product:
- Who can access a workspace
- What data they can use
- What work they can see
- What can be sent to downstream tools
- Which changes require approval
This guide explains the main governance controls available in Hightouch, why they matter, and when to use them.
[**Explore the interactive architecture diagram →**](https://architecture.hightouch.com/?from=docs&product=governance&utm_source=hightouch-docs&utm_medium=referral&utm_campaign=architecture-explorer&utm_content=governance)
See how governance controls fit into the Hightouch architecture.
---
## How composable governance differs from traditional governance
In traditional customer data platforms, customer data is typically copied into the platform's own infrastructure. That means governance often happens inside a separate system with its own permissions, storage layer, and rules.
Hightouch works differently.
Hightouch connects directly to your existing warehouse and inherits the controls already defined there. If a table, column, or dataset is restricted in your warehouse, Hightouch respects that access. Hightouch then adds additional controls for activation workflows, so teams can use customer data without bypassing the governance your organization already has in place.
In practice, this means **governance in Hightouch is composable**:
- Your warehouse controls the underlying data access
- Hightouch controls how teams activate and distribute that data
- You can apply only the controls you need based on how your organization works
For most teams, this is easier to manage than creating separate systems or duplicating customer data just to enforce access rules.
---
## What governance helps you manage
Hightouch governance controls help answer a few practical questions:
- Who should be able to access this workspace?
- What data should each team be allowed to work with?
- Which audiences and campaign assets should teams be able to see?
- What data should be allowed to leave the platform?
- Which changes should require review before going live?
Each control in Hightouch helps with a different part of that process.
---
## 1. Control who can access a workspace
### Organizations and workspaces
Before configuring workspaces, it helps to understand the two levels of structure in Hightouch: [**Organizations**](/workspace-management/overview#organizations) and [**Workspaces**](/workspace-management/overview#workspaces).
A **Hightouch Organization** is the parent container representing your company as a whole. It is the top-level entity used for user management, billing, and cross-workspace administration. Within an Organization, you create one or more **Workspaces** — each a separate environment with its own sources, destinations, models, syncs, and users.
Two pre-built organization-level groups are available by default:
- **Organization Admin** — full access across all workspaces and org-level settings, including managing SSO configuration, billing, creating new workspaces, inviting users, and creating new user groups
- **Organization Viewer** — read-only access across the organization
Workspace-level roles and groups are separate from org-level groups. A user can be an Org Admin without being a Workspace Admin, and vice versa. In practice, Org Admins are typically members of your IT or data platform team, while day-to-day workspace management is delegated to Workspace Admins within each workspace.
### SSO and SCIM
For teams managing access at scale, Hightouch supports **SAML SSO** through identity providers like Okta and Microsoft Entra ID, plus automatic user provisioning via **SCIM**.
- **SAML SSO** automatically creates Hightouch users on their first login
- **SCIM** synchronizes user account changes from your identity provider to Hightouch automatically — including provisioning, deprovisioning, and group assignments
You configure SSO at the Organization level, then map groups in your identity provider to user groups already defined in Hightouch. When someone joins a team in Okta, they automatically receive the right workspace access and role permissions without manual provisioning.
See:
- [SSO setup guide](/workspace-management/sso)
### Workspace structure
The next governance decision is whether teams should work in the same workspace or in separate ones.
A **workspace** is a separate container with its own sources, destinations, models, syncs, and users. Separate workspaces are useful when you need *complete isolation*, such as:
- Regional or data residency requirements
- Strict separation between staging and production
- Business units that should not share data or configuration
For many organizations, though, separate workspaces are not the best starting point.
If teams share the same underlying customer data, keeping them in one workspace is often easier to manage. It reduces duplicate setup and allows teams to work from the same source of truth while still applying internal governance controls.
Use workspace-level structure when you need hard separation. Otherwise, start with a shared workspace and add more targeted controls inside it.
Create separate workspaces only when required for geo-specific data processing, separate infrastructure, or compliance. For most enterprise customers, a single production workspace is sufficient.
See:
- [Workspaces](/workspace-management/workspaces) — includes a [side-by-side comparison](/workspace-management/workspaces#do-you-need-another-workspace) of workspaces vs. environments
- [Core concepts](/workspace-management/overview)
- [Environments](/workspace-management/environments)
### When to use this
Use separate workspaces when you need complete operational or regulatory isolation.
Use a shared workspace when teams should collaborate from the same models and data, but still need scoped access and control.
---
## 2. Control what data teams can work with
Once users are in a workspace, the next question is: what data should they actually be able to use?
This is where access controls become important for day-to-day work.
For example:
- A regional team may only need access to customers in its market
- A brand team may need to work only with its own customer segment
- Marketers may need to build audiences without seeing raw PII
- Some users may need to review data, while others need to create and launch workflows
Hightouch supports this through **user groups and roles**, **subsets**, and **data masking**.
### User groups and roles
Access control in Hightouch is managed at the **user group** level, not on individual users. A **user group** is a collection of users who share the same permissions. Each group is assigned a role per workspace, which determines what actions members can take.
There are two types of user groups:
- **Pre-built groups** — Organization Admin and Organization Viewer, available by default
- **Custom groups** — created by org admins to reflect different teams, regions, brands, or any other structure that impacts permissions
A user can belong to multiple groups and inherits the combined permissions of all their group memberships. If a user is in both a Viewer group and an Editor group for the same workspace, they receive the higher level of access — Editor.
**Roles** define what actions a group can perform within a workspace. Pre-built roles apply across all sources and destinations:
- **Admin** — full grants, typically assigned to data or IT teams
- **Editor** — can create and configure resources, but cannot delete or update existing source and destination configurations
- **Viewer** — read-only access across the workspace
- **Draft editor** — can create and propose changes, but cannot publish; all changes go through an approval flow
**Custom roles** are scoped to specific sources and destinations rather than applying globally. Permissions for dependent resources — models, syncs, and audiences — are inherited from their associated source and destination. Custom role permissions are configured across six tabs:
- **General** — manage who can create new integrations
- **Sources** — manage rights per source (including [schema management](/customer-studio/schema), which also governs access to traits, sync templates, and governance features like destination rules and subsets)
- **Destinations** — manage rights per destination
- **Customer Studio** — manage rights per parent model and general model
- **Agents** — manage access to Agents, Ad Studio, and Lifecycle Studio
- **Events** — manage creation and per-resource rights for event sources, destinations, contracts, and functions
This means you can create a "Paid Media Marketing" role that has access to Google Ads, Meta, and TikTok destinations but cannot see or touch Salesforce CRM or Braze configurations — enforced at the platform level, not by policy document.
Start with pre-built roles during onboarding. Switch to custom roles to scope access to specific sources and destinations as your usage scales.
See:
- [User groups](/workspace-management/user-management)
- [Roles](/workspace-management/roles)
### Subsets
**Subsets** limit which records a team can access at the row level.
This is useful when multiple teams work in one workspace but should only be able to use part of the overall customer data, such as:
- One region
- One brand
- One business unit
- One consented customer segment
Subsets are enforced at the query level — there is no way to bypass them through the UI or API. Users in the audience builder automatically see only the data their subset allows.
Common subset patterns include consent enforcement (filtering out opted-out customers globally), regional partitions, brand-level segments, and custom suppression lists for legal hold or VIP exclusions.
Subsets are configured under **Governance > Subset Categories** in Customer Studio.
See:
- [Subsets](/customer-studio/subsets)
### Data masking
Data masking helps teams work with customer data without exposing sensitive values in previews, profile views, or record inspection.
Three masking levels are available:
- **Approved** — data is visible and available for use in syncs and filters
- **Redacted** — data is hidden in the UI (profile details, column suggestions, analytics) but remains accessible for syncs and audience filters
- **Blocked** — data is completely restricted from access and syncs
This is useful when users need to build audiences from behavioral or account data, but should not see raw identifiers or sensitive fields.
See:
- [Data masking](/customer-studio/data-masking)
### When to use this
Use these controls when multiple teams share a workspace, but not everyone should have the same level of access to the same data.
---
## 3. Control which campaign assets teams can see
The previous section covered who can access the platform and what underlying data they can query. This section is about something different: which audiences, syncs, and campaign assets different teams can see once they're inside the workspace.
Even if two teams have identical data access, they may have no reason to see each other's audiences, syncs, or journeys. Without that separation, large workspaces become noisy, harder to navigate, and easier to change by mistake.
### Spaces
This is where **Spaces** help.
Spaces let you partition a single workspace into isolated sub-workspaces for different regions, brands, teams, or business units. Within a Space, teams work from the same underlying models, sources, and destinations — but their audiences and syncs are scoped to that Space and are not visible to other teams.
Key behaviors:
- Models, sources, destinations, and schema are **shared** across all Spaces in a workspace
- Audiences and syncs are **scoped** to a specific Space
- Only users with explicit access to a Space can create, view, or edit its resources
- Workspace admins retain cross-Space visibility for governance and troubleshooting
- Access is managed via **user groups**, not individual users
In practical terms, Spaces help answer questions like:
- Which audiences should this team see?
- Which syncs belong to this region?
- How do we keep one team from accidentally editing another team's work?
When enabled, Spaces can be accessed and configured under **Settings → Workspaces → Spaces**.
Spaces is currently in early access and must be enabled for your workspace by your Hightouch team. Contact your Hightouch representative to request access.
See:
- [Spaces](/workspace-management/spaces)
### When to use this
Use Spaces when teams should share the same workspace and data foundation, but should not all see the same campaign assets or operational work.
---
## 4. Control what data can leave Hightouch
One of the most important governance decisions is what happens when customer data is sent to downstream tools or [destinations](/destinations/overview).
Even if the right team built the right audience, you may still need to control:
- Which records can be synced to a destination
- Whether consent rules are enforced
- Which fields are allowed to be mapped
- Whether syncs follow an approved setup pattern
Hightouch supports this through **Destination Rules**, **Data Masking**, **Audience Templates**, and **Sync Templates**.
### Destination Rules
**Destination Rules** automatically filter which records are eligible to sync to a specific destination.
They are evaluated at sync time — they do not affect the underlying audience definition. Think of them as a channel-specific filter that Hightouch applies automatically every time an audience syncs to a given destination.
They are especially useful for enforcing requirements like:
- Email consent
- SMS consent
- Advertising consent
- Regional policy restrictions
- Minimum audience thresholds
- Frequency capping
This helps move consent enforcement out of individual campaign setup and into a reusable governance control that applies regardless of who built the audience or which Space it belongs to.
Destination Rules are configured under **Governance > Destination Rules** in Customer Studio.
See:
- [Destination Rules](/customer-studio/destination-rules)
### Data Masking
**Data Masking** can also be used at the destination layer to ensure that raw sensitive values — such as email addresses or phone numbers — are not mapped to destinations that only need hashed identifiers or anonymized fields.
See:
- [Data masking](/customer-studio/data-masking)
### Audience Templates
**Audience Templates** let admins define reusable audience structures that users can quickly adapt with minimal changes.
Admins set up an audience as usual but leave certain fields blank — those become the fields users fill in when creating an audience from the template. This ensures that baseline segmentation logic, suppression rules, and consent filters are baked into every audience from the start. Marketers can customize within the template's parameters but cannot bypass the foundational logic.
You can also mandate that all new audiences must start from an approved template, ensuring governance guardrails are always in place.
Audience Templates are configured under **Templates > Audience Templates** in Customer Studio.
See:
- [Audience Templates](/customer-studio/templates#audience-templates)
### Sync Templates
**Sync Templates** give teams an approved starting point for destination configuration.
Instead of setting up every sync from scratch, admins can define approved mappings, modes, and field-level permissions ahead of time. For specific fields, admins can use the **editable toggle** to allow users to customize only those fields when creating a sync from a template.
You can also enforce the use of sync templates entirely by enabling the **Require sync templates** toggle in Settings.
This helps teams move faster while reducing setup mistakes and configuration drift.
Sync Templates are configured under **Templates > Sync Templates** in Customer Studio.
See:
- [Sync Templates](/customer-studio/templates#sync-templates)
### When to use this
Use destination-level controls when different downstream tools have different requirements, or when your team wants to enforce safe and repeatable activation patterns.
---
## 5. Organize and navigate workspace resources
As workspaces grow, teams need ways to keep resources organized and easy to find. Hightouch provides three tools for this: **Folders**, **Labels**, and **Filters**.
### Folders
Folders let you organize resources like syncs, models, and audiences into a shared hierarchy. You can group resources in a strict hierarchy or organize by team or department. Folder hierarchy is shared across resource types — moving a sync also moves its associated model to the same folder.
Only workspace admins can modify the folder structure. Other roles can add or remove resources from existing folders.
### Labels
Labels are key/value pairs that let you tag resources across multiple dimensions that don't fit into a folder structure. For example:
- Track campaign type: `type:email`, `type:sms`, `type:push`
- Track brand or region: `brand:brand-a`, `region:eu`
Multiple labels can be applied to a single resource. You can then filter by label combinations in the sidebar.
### Filters
Filters let you narrow the resource list by attributes like sync progress, destination, source, health status, and more. Once you've set up a useful filter combination, you can save it for future use.
See:
- [Folders and labels](/workspace-management/folders-filters)
### When to use this
Use these tools in any workspace with multiple teams or a large number of resources. They don't enforce access — they improve discoverability and reduce noise.
---
## 6. Control which changes require review
Some changes are low risk. Others should not go live without review.
For example:
- Launching a new audience to a paid media platform
- Changing sync mappings for a production destination
- Updating logic that affects a large customer segment
- Allowing less technical users to propose changes without publishing directly
Hightouch supports this through **Approval Flows**, and in some cases through **Environments** for staging and production workflows.
### Approval Flows
Approval Flows let designated users create or edit resources in draft mode, then require review before those changes go live.
To set this up:
1. Enable the **Require Approvals** toggle in **Settings > Workspace**
2. Assign the **Workspace draft editor** role to users who should go through the approval process
3. By default, all roles except Workspace draft editor and Workspace viewer can grant approval — if you need to grant approval permissions to other roles, build a custom role
Approval Flows pair well with Audience Templates and Sync Templates. When both are in place, admins can require that all new audiences start from an approved template — ensuring that baseline segmentation logic, suppression rules, and consent filters are baked in. Marketers can customize within the template's parameters but cannot bypass the foundational logic.
This is useful when you want marketers or operators to move quickly, but still want an admin, data team, or compliance owner to review higher-impact changes first.
See:
- [Approval flows](/workspace-management/approval-flows)
### Environments
Environments apply to **models and syncs only** ([rETL workflows](/getting-started/concepts)). Customer Studio resources — such as audiences, destination rules, and sync templates — cannot be deployed between environments. For Customer Studio change management, use **Approval Flows** instead.
Environments help teams separate testing from production and move changes more intentionally between them. Each workspace acts as a separate environment. You can link workspaces and deploy models and syncs from one environment to another without recreating them from scratch.
Before a deployment executes, Hightouch validates that all underlying resources (sources, destinations, models) match their production counterparts — preventing mismatches that could cause data integrity issues.
See:
- [Environments](/workspace-management/environments)
### Audit visibility
Governance also depends on being able to understand what changed, when, and by whom.
Audit logs are available under **Settings > Audit log** and track up to 90 days of in-app activity. Each log entry records the user, action, resource, and a before/after JSON diff. For longer retention or custom dashboarding, audit data can also be written back to your warehouse via [Warehouse Sync Logs](/syncs/warehouse-sync-logs).
See:
- [Audit logs](/workspace-management/audit-logs)
### When to use this
Use review and change-management controls when multiple teams can affect production workflows, or when your organization needs more formal approval before activation.
---
## Choosing the right governance controls
Not every team needs every governance feature from day one.
A practical way to roll out governance is to start with the controls that match your organization's complexity.
### Smaller teams
Start with:
- Workspace structure
- User groups
- Roles
### Growing teams
Add:
- Subsets
- Spaces (if early access is available)
- Basic destination controls
### Larger or more regulated teams
Consider:
- Custom groups and scoped roles by team, region, or brand
- Subsets by region, brand, or business unit
- Spaces
- Destination Rules
- Audience Templates
- Sync Templates
- Approval Flows
- Environments (for rETL models and syncs only)
- Audit logs
The goal is not to add as many controls as possible. The goal is to give teams the access they need while reducing unnecessary risk.
---
## Recommended governance patterns
### If teams share customer data but work independently
Use:
- One shared workspace
- Roles for action-based access
- Subsets for data access
- Spaces for team-level visibility (if early access is enabled)
### If different regions or entities must be fully separate
Use:
- Separate workspaces
- Separate infrastructure or regional connections as needed
### If consent rules differ by channel
Use:
- Destination Rules
- Sync Templates
### If campaign changes should be reviewed before launch
Use:
- Approval Flows
- Environments (for rETL models and syncs only) for staging and production where needed
---
## Next steps
To put governance into practice, start with the feature docs for the controls you need:
- [Core concepts](/workspace-management/overview)
- [User groups](/workspace-management/user-management)
- [Roles](/workspace-management/roles)
- [SSO setup guide](/workspace-management/sso)
- [Subsets](/customer-studio/subsets)
- [Destination Rules](/customer-studio/destination-rules)
- [Data masking](/customer-studio/data-masking)
- [Audience Templates](/customer-studio/templates#audience-templates)
- [Sync Templates](/customer-studio/templates#sync-templates)
- [Approval flows](/workspace-management/approval-flows)
- [Environments](/workspace-management/environments)
- [Audit logs](/workspace-management/audit-logs)
- [Folders and labels](/workspace-management/folders-filters)
- [Spaces](/workspace-management/spaces) — early access feature; contact your Hightouch team to enable
---
## Core concepts
**URL:** https://hightouch.com/docs/workspace-management/overview
**Description:** Understand the core building blocks of Hightouch — organizations, workspaces, user groups, roles, environments, and Spaces.
**Section:** Workspace management
This page covers core concepts. For prescriptive guidance on when to use each control and how to structure your organization, see [Governance best practices](/workspace-management/governance).
Hightouch is built around a set of core concepts: organizations, workspaces, user groups, roles, environments, and Spaces. Understanding how they relate to each other is the foundation for configuring your workspace correctly.
---
## Organizations
An **organization** is the top-level entity in Hightouch. It represents your entire company and is the container for billing, SSO configuration, and user management. All workspaces belong to a single organization.
Most companies should have exactly one organization. An organization is created automatically when the first person at your company signs up — that person becomes the first organization admin. From there, organization admins can manage billing, configure [SSO and SCIM](/workspace-management/sso), [invite users](/workspace-management/user-management#inviting-users), and [transfer admin access](/workspace-management/user-management#transferring-organization-admin-access) as team members change.
See: [Set up an organization and workspace](/workspace-management/workspaces#first-sign-in)
---
## Workspaces
A **workspace** is a fully isolated partition within an organization. Each workspace has its own sources, destinations, models, syncs, audiences, and user roster. By design, nothing is shared across workspace boundaries — not even the connection to your data source.
Workspaces are commonly used to:
- Separate staging and production environments
- Isolate business units or regions that must not share resources or configuration
For most enterprise customers, a single production workspace is sufficient. Hightouch's internal governance controls — including user groups, roles, subsets, and Spaces — allow different teams, regions, or brands to operate within a shared workspace without interfering with each other.
To learn how to set up a new workspace, see [Workspaces](/workspace-management/workspaces). For guidance on when to create additional workspaces versus using [Environments](/workspace-management/environments), see [Do you need another workspace?](/workspace-management/workspaces#do-you-need-another-workspace)
---
## Environments
An **environment** is a linked copy of a workspace used for change management — typically dev, staging, and production. Environments let you test changes to models and syncs before deploying them to production, without needing a completely separate workspace.
Environments share the same organization, user roster, and governance controls as the parent workspace. Use them when you need a safe promotion path but don't need hard data isolation.
See: [Environments](/workspace-management/environments) | [Workspaces vs environments](/workspace-management/workspaces#do-you-need-another-workspace)
---
## Spaces
A **Space** is a visibility boundary within a single workspace. Spaces let you scope which resources (syncs, models, audiences) a team can see without creating a separate workspace. They're useful when multiple teams share a workspace but shouldn't be distracted by each other's work.
Spaces control visibility, not permissions — roles and user groups still govern what actions a user can take.
See: [Spaces](/workspace-management/spaces)
---
## User groups
A **user group** is a collection of users who share the same permissions. Access control in Hightouch is managed at the group level, not for individual users. This makes it easier to keep permissions consistent and to update access as team structures change.
User groups are created within an organization and can be granted access to one or more workspaces. Users can belong to multiple groups and inherit the combined permissions from all of their assigned groups.
In most cases, user-to-group mappings are managed through your identity provider via [SSO and SCIM](/workspace-management/sso). If SSO is not configured, user groups and memberships can be managed manually in the Hightouch app.
See: [User groups](/workspace-management/user-management)
---
## Roles
A **role** defines what actions a user group can perform within a workspace. User groups aggregate users; roles define the scope of what those users can do.
Hightouch offers two types of roles:
- **Pre-built roles** — apply broadly to all resources in a workspace. These are the right starting point for most teams.
- **Admin** — full grants across the workspace, typically assigned to data or IT teams
- **Editor** — can create and configure resources, but cannot delete or update existing source and destination configurations
- **Viewer** — read-only access across the workspace
- **Draft Contributor** — can create and propose changes, but cannot publish; all changes require approval before going live
- **Custom roles** — provide granular, per-resource control. Custom roles let you define specific permissions for individual sources, destinations, and Customer Studio parent models. Useful when pre-built roles are not flexible enough for a large or complex workspace.
See: [Roles](/workspace-management/roles)
---
## Next steps
- [Workspaces](/workspace-management/workspaces) — create, manage, and delete workspaces
- [Set up user groups and members](/workspace-management/user-management)
- [Configure roles](/workspace-management/roles)
- [Set up SSO and SCIM](/workspace-management/sso)
- [Governance best practices](/workspace-management/governance) — when to use workspaces, subsets, Spaces, destination rules, approval flows, and more
---
## Roles
**URL:** https://hightouch.com/docs/workspace-management/roles
**Description:** Learn how role-based access control works in Hightouch, including pre-built and custom roles, permission types, and best practices for managing workspace access.
**Section:** Workspace management
| Field | Content |
| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Organization admins |
| **Prerequisites** | Organization admin permissions, or membership in a group with access to [Organization settings](https://app.hightouch.com/settings/organization?from=docs). |
## Overview
Hightouch uses **role-based access control (RBAC)** to define what actions users can take across workspaces and resources. The access model works like this:
1. **Users** are people in your organization.
2. **Groups** are the unit of access control — you manage permissions at the group level, not per user.
3. A group gets access to a workspace by being **assigned a role** for that workspace.
4. That role is either **pre-built** (a standard access level) or **custom** (fine-grained grants you configure).
New groups start with no workspace access. A user who isn't in any group — or whose group has no role assigned — can log in but can't see or do anything in a workspace.
---
## How roles work
Hightouch's permissions system is built around **data flow** — who can see data and where it can be sent — rather than micromanaging every model or sync.
### Access is based on sources and destinations
Every sync involves multiple resources. For example, syncing BigQuery → Salesforce includes:
- The **source** (BigQuery)
- The **model** built from BigQuery
- The **destination** (Salesforce)
- The **sync** that connects them
Rather than managing permissions for each model or sync, Hightouch groups access by **source** (where data comes from) and **destination** (where it goes). Models and syncs inherit permissions from these top-level resources.
To create or run a sync, users need permission for **both** the source and the
destination. Having access to one side alone isn't enough.
---
## Pre-built vs. custom roles
Every group needs a role before it can access a workspace. Hightouch offers two kinds:
- **Pre-built roles** — Four standard access levels (Admin, Editor, Draft Editor, Viewer) that apply broadly across all resources in a workspace.
- **Custom roles** — Fine-grained roles where you toggle individual grants per source, destination, product, or resource.
**Start with pre-built roles.** They cover the most common access patterns and make onboarding fast. Only switch to custom roles when a group needs exceptions — like access to specific sources but not others, or Customer Studio permissions without source-level access.
| Use pre-built when... | Use custom when... |
| ---------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| A group should have one standard access level across a workspace | Different teams sharing a workspace need distinct permissions |
| You're onboarding and want to move fast | You need to restrict access to specific sources or destinations |
| Everyone in the group does roughly the same type of work | You use subsets, approval flows, or destination rules and need roles aligned to those controls |
| The group maps cleanly to Admin, Editor, Draft Editor, or Viewer | Marketers need parent model access without source-level permissions |
A good rule of thumb: **pre-built = "this whole group should have one standard
level of access."** **Custom = "this group needs exceptions, scoped resources,
or product-specific permissions."**
---
## Pre-built roles
Pre-built roles are ready-made access levels. They apply to **all** sources, destinations, models, Customer Studio parent models, Agents, and Events resources in a workspace.
There are four pre-built roles:
- **Workspace admin** – Full access to all resources and settings. Can manage credentials, approve changes, and configure everything. Organization admins automatically inherit this role.
- **Workspace editor** – Can create and edit most resources, create new sources and destinations, and trigger syncs. Cannot manage credentials or configure schema.
- **Workspace draft editor** – Can create and edit drafts (where supported) and view data, but cannot trigger syncs, approve changes, or create new sources or destinations.
- **Workspace viewer** – Read-only access. Cannot see row-level data or make changes.
### Pre-built role permissions
The tables below show the default grants for each pre-built role. Viewer has no grants and is not shown.
**General**
| Permission | Admin | Editor | Draft editor |
| ----------------------- | ----- | ------ | ------------ |
| Create new sources | ✅ | ✅ | ❌ |
| Create new destinations | ✅ | ✅ | ❌ |
**Source**
| Permission | Admin | Editor | Draft editor |
| ---------------------- | ----- | ------ | ------------ |
| View row-level data | ✅ | ✅ | ✅ |
| Draft syncs & models | ✅ | ✅ | ✅ |
| Approve syncs & models | ✅ | ✅ | ❌ |
| Configure schema | ✅ | ❌ | ❌ |
| Manage source | ✅ | ❌ | ❌ |
**Configure schema** controls access to Customer Studio schema configuration,
including creating and editing traits, destination rules, and row-level
access. If users with the Editor role need to manage traits, create a [custom
role](#custom-roles) that includes the Configure schema grant. Workspaces
migrated from Hightouch's earlier permissions model may have a legacy
"Workspace Editor" custom role that includes Configure schema. The current
pre-built Editor role does not.
**Destination**
| Permission | Admin | Editor | Draft editor |
| ---------------------- | ----- | ------ | ------------ |
| Trigger syncs | ✅ | ✅ | ❌ |
| Draft syncs & models | ✅ | ✅ | ✅ |
| Approve syncs & models | ✅ | ✅ | ❌ |
| Manage destination | ✅ | ❌ | ❌ |
**Models — Parent models**
Customer Studio does **not** have a "draft audience" concept. Approval flows
do not apply to audience creation, but they may be required for **audience
syncs**.
| Permission | Admin | Editor | Draft editor |
| ------------------------- | ----- | ------ | ------------ |
| View parent model data | ✅ | ✅ | ✅ |
| Create and edit audiences | ✅ | ✅ | ✅ |
| Trigger audience syncs | ✅ | ✅ | ❌ |
| Approve audience syncs | ✅ | ❌ | ❌ |
**Models — General models**
| Permission | Admin | Editor | Draft editor |
| --------------- | ----- | ------ | ------------ |
| View model data | ✅ | ✅ | ✅ |
**Agents**
| Permission | Admin | Editor | Draft editor |
| ---------------- | ----- | ------ | ------------ |
| Access Agents | ✅ | ✅ | ❌ |
| Configure Agents | ✅ | ❌ | ❌ |
**Events**
| Permission | Admin | Editor | Draft editor |
| --------------------------------------- | ----- | ------ | ------------ |
| Create new event sources | ✅ | ✅ | ❌ |
| Create new event warehouse destinations | ✅ | ✅ | ❌ |
| Create new event streaming destinations | ✅ | ✅ | ❌ |
| Create new event contracts | ✅ | ✅ | ❌ |
| Create new event functions | ✅ | ✅ | ❌ |
### Assign a pre-built role
Roles are assigned to **user groups**, not individual users.
1. Go to **Settings**, then **Organization**, then [**Groups**](https://app.hightouch.com/organization/groups?from=docs).
2. Select or create a group.
3. In the **Workspaces** tab, use the **Role** dropdown to assign a pre-built role.
4. **Save changes**.
The **Access** column shows a summary of the group's permissions for each workspace. It gives a helpful overview but doesn't display every grant or cover all resource types.
### When to switch to custom roles
Pre-built roles work well for onboarding and teams with uniform access needs. Consider switching to a custom role when:
- A team needs access to **some** sources or destinations but not others
- Marketers should work only with parent models in Customer Studio without source-level permissions
- You need to align roles with **subsets**, **approval flows**, or **destination rules**
- You want to give an Events team access to event resources without full editor permissions
---
## Custom roles
Custom roles let you define precise permissions for each team, workspace, or use case. Instead of applying a standard access level, you toggle individual **grants** across tabs that mirror the custom role builder in the UI:
| Tab | What it controls |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **General** | Whether users can create new sources or destinations in the workspace. |
| **Sources** | How users interact with connected data sources — view row-level data, manage credentials, create models. |
| **Destinations** | What users can do with destinations — create syncs, trigger sync runs, manage credentials. |
| **Models** | Parent models: access to audiences, journey configuration, and journey publishing. General models: row-level data access for general models and Agents. |
| **Agents** | Access to the Agents, Ad Studio, and Lifecycle Studio sections, and whether users can modify Agents settings. |
| **Events** | Who can create and manage Event resources: sources, streaming destinations, warehouse destinations, contracts, and functions. |
### How to create or edit a custom role
1. Go to **Settings**, then **Organization**, then [**Groups**](https://app.hightouch.com/organization/groups?from=docs).
2. Select or create a group.
3. In the **Workspaces** tab, select `Custom...` from the **Role** dropdown.
4. **Save changes**.
5. Click **Edit custom role** to open the configuration panel. You'll see the tabs listed above.
Changes save automatically when you click **Save changes** in the bottom-right corner.
---
## Custom role grant reference
This section documents every individual grant you can configure when building a custom role. Each subsection corresponds to a tab in the custom role builder.
**Jump to:**
[General](#general) · [Sources](#source) · [Destinations](#destination) · [Models](#models) · [Agents](#agents) · [Events](#events) · [Subsets](#differences-when-subsets-are-enabled) · [Approval flows](#differences-when-approval-flows-are-enabled)
### General
| Grant | What it allows |
| --------------------------- | ------------------------------------------------------------------------------------ |
| **Create new sources** | Create data sources in the workspace and assign access permissions for other groups. |
| **Create new destinations** | Create destinations in the workspace and assign access permissions for other groups. |
Workspace-level resources — such as folders, extensions, custom storage, and
API keys — can only be configured by workspace admins.
### Source
Source-level permissions are designed primarily for Reverse ETL use cases,
where users need the flexibility to run custom SQL when pulling data from the
source. If you're creating a role for marketers in Customer Studio, it's best
to avoid enabling any source grants. Instead, use the [parent model
grants](#parent-models) in the Models tab, which limit access to specific
parent models and subsets, ensuring that marketers work within your
pre-approved Customer Studio schema.
**View source data**
With this grant enabled, users can view row-level data from this source across the Hightouch app. This includes:
- Previewing models
- Viewing row-level data when testing rows during sync creation
- Accessing row-level data in the debugger for historical sync runs
Primary keys and error messages are **not** classified as row-level data to
preserve basic sync observability. If you choose not to assign this grant,
avoid using columns with PII (like email or phone number) as primary keys, as
these fields will be visible to all users across the app.
**Configure models & syncs**
When enabled, this grant allows users to:
- Create, edit, or delete models that pull data from the source
- Use these models for syncing (provided the user has permissions to sync data to destinations)
Users can only edit existing models if they can also edit all of the syncs
that rely on that model.
**Configure schema**
Applicable only to workspaces with Customer Studio. This grant allows users to configure the [schema](/customer-studio/schema) layer, including creating and editing traits, managing destination rules, and configuring row-level access. Without this grant, users can view but not save changes to traits.
**Manage source**
With this grant enabled, users can edit the source configuration, which includes:
- Updating connection details and credentials
- Adjusting the sync engine choice
- Setting source-level defaults for Warehouse Sync Logs
### Destination
| Grant | What it allows |
| ---------------------- | ----------------------------------------------------------------------------------------------------------- |
| **Trigger syncs** | Manually trigger sync runs. Does not allow editing schedules. |
| **Configure syncs** | Create, edit, or delete syncs to the destination and set their schedules. Does not allow manual triggering. |
| **Manage destination** | Modify the destination's configuration, including connection details, credentials, and alert defaults. |
### Models
The Models tab controls permissions for parent models and general models. It has two sub-tabs: **Parent models** and **General models**.
#### Parent models
These grants were previously located in a standalone **Customer Studio** tab.
They now appear under **Models → Parent models**.
Parent model grants function similarly to source-level grants but are scoped to specific parent models within the Customer Studio schema.
**View parent model data**
Enabling this grant allows users to view row-level data from the parent model throughout the Hightouch app. This includes:
- Previewing audiences to see row-level data for individual members
- Viewing row-level data when testing rows during sync creation
- Accessing row-level data in the debugger for historical sync runs
If a user has the **Configure audiences & syncs** grant but lacks the **View
parent model data** grant, they can still preview audiences, but they'll be
limited to aggregate counts and insights, without row-level data for each
audience member.
**Configure audiences & syncs**
When enabled, this grant allows users to:
- Create, edit, or delete audiences that pull data from the parent model
- Use these audiences for syncing (provided the user has permissions to sync data to destinations)
Users can only edit existing audiences if they can also edit all of the syncs
that rely on that audience.
**Configure journeys**
When enabled, this grant allows users to:
- Create, edit, or delete journeys and journey templates.
**Publish journeys**
When enabled, this grant allows users to:
- Launch, pause, or stop live journeys
#### General models
General model grants are applied per model, similar to how source grants are applied per source. You can configure grants for all general models in the workspace (including any created in the future) or for individual models.
| Grant | What it allows |
| ------------------- | ---------------------------------------------------------------------------------------------------------- |
| **View model data** | Access row-level data from the general model. Required for Agents to query the model on the user's behalf. |
Creating and managing general models requires the **Configure schema** grant on the underlying [source](#source).
### Agents
| Grant | What it allows |
| -------------------- | ----------------------------------------------------------------------------------------------------------- |
| **Access Agents** | Access the Agents, Ad Studio, and Lifecycle Studio sections. Without this grant, these sections are hidden. |
| **Configure Agents** | Modify Agents settings, including context, brand, guardrails, and channels. |
In the role builder, these appear as the **Can this role access Agents?** and **Can this role configure Agents?** toggles.
Agents also respects model-level and row-level data controls. Which tables an agent can query depends on the user's permissions for [parent models](#parent-models) and [general models](#general-models) in the Models tab. For row-level access and column access controls, see [Agents data governance](/agents/overview#data-governance).
### Events
Events permissions are organized into two layers: workspace-level **creation grants** that control who can create new Event resources, and **per-resource grants** that control what users can do with specific resources.
Events permissions require Permissions V2.
#### Creation grants
The General sub-tab within Events controls which types of Event resources users can create. Each grant also allows the creator to assign access permissions to other groups.
| Grant | What it allows |
| ------------------------------------------- | ------------------------------------------------------------------------------------- |
| **Create new event sources** | Create event sources and grant other groups permission to manage them. |
| **Create new event warehouse destinations** | Create event warehouse destinations and grant other groups permission to manage them. |
| **Create new event streaming destinations** | Create event streaming destinations and grant other groups permission to manage them. |
| **Create new event contracts** | Create event contracts for defining event schemas and data quality rules. |
| **Create new event functions** | Create event transformation functions. |
#### Per-resource grants
The remaining Events sub-tabs control what users can do with individual event resources. Available grants vary by resource type:
| Resource type | Can edit | Can sync | Can access data | Can run |
| -------------------------- | -------- | -------- | --------------- | ------- |
| **Event sources** | ✅ | ✅ | ✅ | — |
| **Streaming destinations** | ✅ | ✅ | — | — |
| **Warehouse destinations** | ✅ | ✅ | — | ✅ |
| **Functions** | ✅ | — | — | — |
| **Contracts** | ✅ | — | — | — |
- **Can edit** — Modify the resource's configuration.
- **Can sync** — Create and manage syncs involving the resource.
- **Can access data** — View live event data in the source debugger. Only available for event sources.
- **Can run** — Trigger runs. Only available for warehouse destinations.
To create or manage an event sync, a user needs the relevant permissions on
**both** the event source and the event destination. Having access to only one
side is not sufficient.
Workspace admins and built-in workspace editors are automatically granted all
Events creation permissions. Non-admin, non-editor groups default to no Events
access until grants are explicitly configured.
### Differences when subsets are enabled
[**Subsets**](/customer-studio/subsets) let you divide a parent model into smaller, permissioned segments (e.g., by brand, region, or consent type).
They're configured in **Customer Studio**, then **Governance**, then **Subset categories**.
For example:
- Your **EU marketing team** can only build audiences using EU data.
- Your **email marketing team** can only use consented audiences.
When subsets are active, you'll see a separate set of checkboxes that let you apply earlier grants (like **View parent model data** and **Configure audiences & syncs**) to specific subsets. The behavior of these checkboxes varies based on whether the subset category is marked as "required."
- **If the subset category is required,** the user will only be able to configure and use audiences within the selected subset values. (If no values are checked, the user will not be able to configure or use any audiences at all.)
- **If the subset category is not required,** the user can configure and use audiences with the selected subset categories, or without subsets altogether.
### Differences when approval flows are enabled
[**Approval flows**](/workspace-management/approval-flows) add review steps before publishing model or sync changes.
They're enabled in **Settings**, then **Workspace**, then **Configuration**, then **Require approvals**.
When enabled, the **Configure models & syncs** grant splits into:
- **Draft models & syncs** – Create or edit drafts (cannot delete published resources).
- **Approve models & syncs** – Review and approve or deny drafts.
Together, these two grants provide the same functionality as the original **Configure models & syncs** grant.
Approval flows do not apply to audience creation in Customer Studio, as there
is no concept of a draft audience. However, approval may be required to sync
an audience to a destination.
The **Draft** and **Approve** grants are separate and do not overlap. To allow
a user to publish changes without requiring approval, you need to assign both
the **Draft** and **Approve** grants. (Under the hood, the user creates a
draft and then immediately self-approves it.)
To approve a sync, a user needs the **Approve** grant for both the source and
the destination.
---
## Governance in context
RBAC defines **who can act** in a workspace, but it's only _one_ part of Hightouch's governance framework. Roles control user access, but they don't enforce organization-wide policies like redacting PII or honoring consent requirements.
Hightouch provides complementary governance features that work alongside roles:
- **[Destination rules](/customer-studio/destination-rules)** – Control what types of data can be sent to specific destinations (for example, ensuring only consented users are synced to email marketing).
- **[Subsets](/customer-studio/subsets)** – Divide your total addressable customer base into smaller segments, which is especially useful in workspaces that span multiple brands or regions.
- **[Approval flows](/workspace-management/approval-flows)** – Add peer review for sync and model changes before publishing.
- **[Staging environments](/workspace-management/environments)** – Test and validate syncs in a dedicated space before going live to reduce the risk of errors in production.
You can use these features independently or in combination. You don't need to enable every governance feature — especially when you're just getting started. Many are built for large enterprises with complex brand or compliance needs.
If you'd like help designing the right setup,{" "}
{" "}
— we're here to help.
---
## FAQ
### How do I prevent a user from seeing a resource?
In Hightouch, all resources in a workspace are intentionally visible to all workspace members; configuration details are not hidden. Access control determines who can create or modify resources, but it cannot make resources invisible. If you need to restrict visibility of sensitive resources for certain users, consider placing these resources in a separate workspace.
### What happens if a user belongs to more than one group?
Users can belong to multiple groups within the same workspace, allowing them to inherit permissions from more than one role at a time.
This setup is common. When a user is part of multiple groups, they receive the combined permissions of all their group memberships, meaning that they can perform all actions permitted to members of any of those groups.
However, this does not allow them to "mix and match" permissions across groups. In other words, if one group can sync from source A to destination B, and another group can sync from source C to destination D, a user who belongs to both groups can sync from A to B and C to D — but not from A to D or C to B.
---
## SSO setup guide
**URL:** https://hightouch.com/docs/workspace-management/sso
**Description:** Hightouch helps customers manage access control for large organizations via a third-party identity provider.
**Section:** Workspace management
## Overview
Hightouch integrates with identity providers like Okta and Microsoft Entra ID to simplify user management and authentication.
- **SAML SSO** enables users to authenticate through their organization’s identity provider, supporting just-in-time (JIT) provisioning during login.
- **SCIM** automates user management tasks such as group assignments and deactivations based on changes in the identity provider.
Need help with SSO or SCIM setup? Share this page with your IT team.
### Required permissions
To configure these integrations, you'll need the following:
- Admin access to your company's identity provider, which is typically managed by your IT team.
- User membership in the **Organization admins** group within your Hightouch organization.
### Identity model and trust boundaries
- SSO is configured at the **organization level** in Hightouch. This means all workspaces in that organization inherit the same SAML trust and access model.
- Hightouch supports one SAML IdP trust per organization at a time. Updating IdP metadata (like issuer, certificate, or IdP SSO URL) updates that trust for the entire organization.
## Configuring SAML SSO
SAML single sign-on (SSO) enables just-in-time (JIT) provisioning so that Hightouch users are automatically created upon first login. Once SSO is set up, users in your organization can navigate to the [Hightouch login page](https://app.hightouch.com/login?from=docs) and select **Log in with SSO** to authenticate with your identity provider.
When a user attempts to log in, Hightouch sends a SAML authentication request to your identity provider. If the identity provider validates the user’s credentials and confirms that the user is authorized to access Hightouch, the user is logged in. For first-time users, an account is automatically provisioned based on the information returned from the identity provider, such as name and email address. SSO also ensures that user attributes (like email addresses) are updated during login when changes are detected.
Configuration steps for SAML SSO may vary depending on the identity provider used. Below are detailed instructions for setting up SAML with [Okta](#okta) and [Microsoft Entra ID](#microsoft-entra-id) (formerly known as Azure Active Directory). For other identity providers, similar configuration will apply.
For an overview of SSO and SAML concepts, refer to this [introductory video](https://www.youtube.com/watch?v=O1cRJWYF-g4).
Keep your SAML NameID stable over time, especially during IdP migrations or
tenant changes. If the NameID format or source value changes (for example,
from `email` to an employee ID or UUID), users may be created as duplicate
identities in Hightouch.
### Okta
The first step is to create a new SAML application in Okta. You can follow [this guide](https://help.okta.com/oag/en-us/Content/Topics/Access-Gateway/add-app-saml-pass-thru-add-okta.htm) or the steps outlined below:
1. In your Okta dashboard, navigate to **Applications** and select **Create App Integration**.
2. Choose **SAML 2.0** as the sign-in method and click **Next**.
3. Give the application a descriptive **App Name**, such as "Hightouch." You can also add the [Hightouch logo](https://cdn.sanity.io/images/pwmfmi47/production/acbc037f008ebc7c51a8ba6ca32db440aec03ee2-1600x1600.jpg) if you'd like. Then, click **Next**.
4. In Hightouch, visit the **Single sign-on** tab on the **Organization settings** page. Click **Configure SAML SSO** to display a modal that provides the **Hightouch SSO URL** and **Audience URI**.
5. In Okta, configure the SAML settings by entering the **Hightouch SSO URL** from the Hightouch modal as the **Single sign on URL**, and the **Audience URI** as the **Audience URI (SP Entity ID)**. You can leave the remaining fields at their default settings.
6. Under **Attribute Statements**, map the `name` and `email` attributes. For instance, you might map `name` using `String.join(" ", user.firstName, user.lastName)` and `email` as `user.email`. Ensure that these match the properties defined in your Okta instance. You can refer to the Okta user [profile properties](https://developer.okta.com/docs/reference/api/users/#default-profile-properties) if needed. Click **Next** to continue.
7. For the prompt **Help Okta Support understand how you configured this application**, select **I'm an Okta customer adding an internal app**. Click **Finish**.
8. Then, you’ll be taken to the application overview page in Okta. Under **Metadata details**, click **More details**, then copy the **Sign on URL** and download the **Signing Certificate**. These will be needed for Hightouch.
9. In the Hightouch modal from step 4, paste the **Identity provider SSO URL** and upload the certificate. Click **Save** to finalize the connection.
10. In Okta, go to the **Assignments** tab of the application you just created. Assign users or groups to grant them access to Hightouch.
11. At this point, you've completed the basic SAML SSO setup, allowing your users to log in to Hightouch through Okta. However, you'll still need to manually assign permissions for each user after they join your Hightouch organization. To streamline this process, we highly recommend setting up automatic group assignments in Okta. This ensures users have the right access to workspaces and resources as soon as they log in for the first time.
If SSO group mappings are not configured, new users won't be automatically
assigned to any group, which may result in limited access until group
assignments are manually set up.
12. To configure group mappings, navigate to the **General** tab for your Hightouch application in Okta and click the **Edit** button in the **SAML Settings** section.
13. Scroll down to the **Group Attribute Statements** section. Set the attribute name to `groups` and apply the appropriate filter. For instance, to make all Okta groups available in Hightouch, select the **Matches regex** filter and enter `.*`.
If you want to send only specific groups, you can either map them individually or use a filter like **Starts with `Hightouch`**.
14. Next, you'll want to go back to Hightouch and create mappings between the groups from your identity provider and the corresponding user groups in Hightouch. Navigate to **Organization settings** and click on the **Single sign-on** tab.
15. Scroll to the bottom section called **Group mappings**. In this table, each group from your identity provider can be mapped to any number of user groups in Hightouch. (Users can belong to multiple groups, and when they do, they inherit the combined access from all of their assigned groups.)
Group assignments are refreshed only when a user logs in. This means that new
SSO groups will not appear for mapping until after a member belonging to that
group has logged into Hightouch. If you need to synchronize users and groups
without requiring login, consider [enabling SCIM](#configuring-scim).
15. All done! Members of your organization can access Hightouch by selecting **Log in with SSO**. You can also share your workspace's direct **Hightouch login URL**, which is available in the **Single sign-on** tab on the **Organization settings** page.
### Microsoft Entra ID
1. In the Microsoft Entra admin center, go to the **Enterprise applications** screen and select **New application**, then click **Create your own application**.
2. In your newly created app, select **Set up single sign on**.
3. Choose **SAML** as the sign-on method.
4. In Hightouch, open the **Single sign-on** tab in the **Organization settings** page. Click **Configure SAML SSO** to display a modal with the **Hightouch SSO URL** and **Audience URI**.
5. Configure the SAML settings in Microsoft Entra ID by entering the **Hightouch SSO URL** as the **Reply URL (Assertion Consumer Service URL)** and the **Audience URI** as the **Identifier (Entity ID).**
Leave the **Sign on URL** field empty if you plan to use IdP-initiated
sign-on. Alternatively, grab the value from the **Hightouch login URL** field in Hightouch and enter it here.
6. In Microsoft Entra ID, go to the **Attributes & Claims** section and add two claim mappings:
- **Name**: `email`
**Namespace** (optional): `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/email`
- **Name**: `name`
**Namespace** (optional): `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name`
As indicated above, the `Namespace` fields can be left blank.
The **Unique User Identifier (Name ID)** is the stable value the service provider uses to identify a user across login sessions. Set this to a unique, persistent value in Microsoft Entra (e.g.`userprincipalname`) to ensure consistent user recognition.
Example mapping section:
The **Source attributes** you choose for the `name` and `email` claims will depend on your Entra configuration; in the example above, the `displayname` attribute contains the user's **full name**, while the `mail` property contains their **work email**.
7. In the Hightouch modal from step 4, enter Microsoft Entra ID's **Login URL** as the **Identity provider SSO URL**, and upload the **Certificate Base64** as the **x.509 certificate** in Hightouch. Then, click **Save**.
8. At this point, you've completed the basic SAML SSO setup, allowing your users to log in to Hightouch through your identity provider. However, you'll still need to manually assign permissions for each user after they join your Hightouch organization. To streamline this process, we highly recommend setting up automatic group assignments. This ensures users have the right access to workspaces and resources as soon as they log in for the first time.
If SSO group mappings are not configured, new users won't be automatically
assigned to any group, which may result in limited access until group
assignments are manually set up.
9. In the Microsoft Entra admin center, go to **Enterprise applications** and select the Hightouch application you created earlier. Navigate to **Single Sign On configuration** and then click on **Attributes & Claims** again.
10. Select **Add a group claim**.
11. Make sure to choose `Groups assigned to the application` when defining your scope. That means **only** groups that are both assigned to the application and include the logging-in user as a member will be included in the SAML assertion.
If you select a different option, you won’t be able to use `Cloud-only group display names` as the **Source attribute** - this attribute is required to sync human-readable group names to Hightouch, as described in the [Entra documentation](https://learn.microsoft.com/en-us/entra/identity/hybrid/connect/how-to-connect-fed-group-claims#emit-cloud-only-group-display-name-in-token).
12. Next, set `Cloud-only group display names` as the **Source attribute**. This will send the `displayName` for each group associated with both the application and the user.
13. Optionally, you can also configure custom settings, such as filtering for groups whose `Display name` starts with a specific prefix, e.g. `Hightouch`.
For setups that are on-premises only (but synced to Entra) or hybrid (using both on-premises and cloud groups), you have two options for emitting group claims:
1. For on-premises groups, select an appropriate on-premises source attribute (such as `sAMAccountName`). For cloud-only groups, enable the `Emit group name for cloud-only groups` checkbox (see screenshot below). This configuration emits the selected on-prem group attribute for synced groups and the display name for cloud-only groups. Learn more about this setup in the [Entra documentation](https://learn.microsoft.com/en-us/entra/identity/hybrid/connect/how-to-connect-fed-group-claims?WT.mc_id=AZ-MVP-5003833#emit-cloud-only-group-display-name-in-token).
2. You can also choose to emit the `Group ID` for both group types. In this case, Entra group object UUIDs will appear in the **SSO group from identity provider** section of the **SSO mappings** settings in Hightouch.
14. Next, you'll want to go back to Hightouch and create mappings between the groups from Microsoft Entra and the corresponding user groups in Hightouch. Navigate to **Organization settings** and click on the **SSO mappings** tab.
15. On this page, each group from your identity provider can be mapped to any number of user groups in Hightouch. (Users can belong to multiple groups, and when they do, they inherit the combined access from all of their assigned groups.)
Group assignments are refreshed only when a user logs in. This means that new
SSO groups will not appear for mapping until after a member belonging to that
group has logged into Hightouch. If you need to synchronize users and groups
without requiring login, consider [enabling SCIM](#configuring-scim).
16. All done! Members of your organization can access Hightouch by selecting **Log in with SSO**. You can also share your workspace's direct **Hightouch login URL**, which is available in the **Single sign-on** tab on the **Organization settings** page.
Regardless of your exact Entra configuration, make sure that your Entra SAML user and group attribute mappings are aligned with your Entra SCIM user and group attribute mappings.
### Other identity providers
Hightouch supports all major identity providers, including OneLogin, Rippling, Google, Ping, and more.
The setup instructions are generally similar to those provided above for Okta. If you encounter any issues or need assistance, please don't hesitate to reach out to our support team—we're here to help!
## Configuring SCIM
You must [configure SAML SSO](#saml-single-sign-on) before configuring SCIM.
Unlike SAML SSO, which creates and updates user accounts only when users log in, SCIM automatically synchronizes user account changes from your identity provider to Hightouch without requiring user login. This means that adding, updating, or deactivating user accounts in your identity provider happens in the background and is immediately reflected in Hightouch.
To get started, generate a SCIM API token by following these steps:
1. Visit the **Organization settings** page in Hightouch and navigate to the **Single sign-on** tab.
2. Click **Generate SCIM token**.
3. Copy the generated bearer token and click **Save** to activate the token.
Hightouch displays the SCIM token only once, immediately after you generate
it. Once you close the modal, you won't be able to access the same token
again. If you don't copy it when it's first displayed, you'll need to generate
a new token by clicking **Refresh SCIM token**.
### Okta
After generating your SCIM token, follow these steps:
1. Navigate to the Hightouch application within your Okta admin panel.
2. On the **General** tab, locate the **App Settings** section and click the **Edit** button.
3. Check the box labeled **Enable SCIM provisioning**, then click **Save**.
4. Move to the **Provisioning** tab and click **Edit** in the **SCIM Connection** section.
5. Enter `https://api.hightouch.com/api/scim/v2` as the **SCIM connector base URL**.
6. Enter `userName` as the **Unique identifier field for users**.
7. Under **Supported provisioning actions**, ensure that the following options are selected:
- Import New Users and Profile Updates
- Push New Users
- Push Profile Updates
- Push Groups
8. Set the **Authentication Mode** to **HTTP Header**.
9. Paste the SCIM bearer token you generated in Hightouch into the authentication field, ensuring it includes the `Bearer` prefix.
10. Click **Save** to complete the configuration.
11. Finally, make sure to assign the relevant users and groups to the Hightouch application. In Okta, you must do this from both the **Assignments** tab and the **Push Groups** tab.
For additional information, or if your Okta admin panel differs from the steps
above, please refer to [Okta's documentation on SCIM
provisioning](https://help.okta.com/en-us/content/topics/apps/apps_app_integration_wizard_scim.htm?cshid=ext_Apps_App_Integration_Wizard-scim).
### Microsoft Entra ID
After generating your SCIM token in Hightouch, follow these steps:
1. Navigate to the **SAML SSO application** that you created for logging into Hightouch within the Microsoft Entra admin center.
2. Click on the **Provisioning** tab.
3. To configure SCIM for your application, click **Connect your application**.
4. Set the **Tenant URL** to `https://api.hightouch.com/api/scim/v2`.
5. Paste the SCIM bearer token you generated in Hightouch into the **Secret Token** field. Test the connection to ensure it's working, then click **Save**.
6. At this point, SCIM is configured for your application, but it isn't yet enabled. It's also recommended to set up user and group assignments.
7. Cick on the **Provisioning** tab under the **Manage** section. This will open a nested Provisioning section where you can configure **Users** and **Groups** attribute mappings.
8. Select **Provision Microsoft Entra ID Users** to configure SCIM attribute mappings for users.
9. A Hightouch user object has three key properties relevant to SCIM provisioning: _email_, _name_, and _active_. During provisioning, these fields should receive their values from the following SCIM Target Attributes:
- `emails[type eq "work"].value`
- `displayName`
- `active`
While it’s technically possible to use other Target Attributes (for example, sending a value for email under userName), doing so isn't recommended. Using the Target Attributes above ensures that user provisioning behaves as expected within Hightouch.
If your Entra configuration meets the following conditions:
- Emails are stored in the `mail` attribute and are **unique**.
- Full names are stored in the `displayName` attribute.
- You want to control user access by enabling or disabling accounts in Hightouch without immediately deleting them.
You should proceed with the following setup:
- First, uncheck the **userName** field as a required attribute in the **Edit attribute list for customappsso** section under **Show advanced options**, and save the change.
- For `emails[type eq "work"].value` as the **Target attribute**, use `mail` as the **Source attribute**. Set **Match objects using this attribute** to `Yes` and set **Matching precedence** to `1`.
- For `displayName` as the **Target attribute**, use `displayName` as the **Source attribute** as well.
- Leave `active` as the **Target attribute** mapped to the `Switch([IsSoftDeleted], , "False", "True", "True", "False")` formula as the **Source attribute**.
- Leave the **Primary Key** settings unchanged.
**Example of a valid user attribute mapping configuration**:
You can read more about SCIM with Entra - including the `active` / `IsSoftDeleted` behavior, and the meaning of **Match objects using this attribute**, **Matching precedence**, and **Primary Key** - in the [official Microsoft documentation](https://learn.microsoft.com/en-us/entra/identity/app-provisioning/customize-application-attributes).
10. As mentioned earlier, the exact **Source attribute** configuration depends on how your Entra directory is set up. Once this setup is complete, your **User Attributes Mapping** section is ready!
11. The **Group provisioning** section (`Provision Microsoft Entra ID Groups`) does not require any changes. The default configuration works out of the box. If you want to verify it, ensure that the `displayName` **Target attribute** is mapped to the `displayName` **Source attribute**, and that **Match objects using this attribute** is set to `Yes` with a **Matching precedence** of `1` for this mapping. Additionally, the `members` **Target attribute** should be mapped to the `members` **Source attribute**. Optionally, you can delete the `externalId` (**Target attribute**) to `objectId` (**Source attribute**) mapping.
**Example of a valid group attribute mapping configuration**:
As with the earlier example of the SAML group attribute configuration, this example SCIM setup is also optimized for a cloud-only Entra group environment; for an on-premises or hybrid group setup that includes both **on-premises groups synced to Entra** and **cloud-only groups in Entra**, you need to ensure that the **Source attribute** selected during SCIM provisioning for the `displayname` **Target attribute** fits your Entra configuration and also **aligns with your SAML attribute mapping**.
12. When provisioning **groups with members**, keep in mind that users added with this process **will not** automatically receive the intended Hightouch group assignment, as the SSO group mapping between Entra and Hightouch cannot yet exist at provisioning time.
Let’s assume the corresponding Hightouch group already exists. In that case, to ensure the user is correctly assigned to that group in Hightouch:
1. After the initial group-with-members provisioning, set up the relevant Entra → Hightouch SSO group mapping inside Hightouch.
2. Then, have the user log in to Hightouch via Entra using SAML SSO. The group assignment will be applied at that time (assuming the SAML attribute mapping is correctly configured, as [per our documentation](https://hightouch.com/docs/workspace-management/sso#microsoft-entra-id)).
### Other identity providers
Hightouch supports all major identity providers, including OneLogin, Rippling, Google, Ping, and more.
The setup instructions are generally similar to those provided above for Okta. If you encounter any issues or need assistance, please don't hesitate to reach out to our support team—we're here to help!
## Rollout, migration, and diagnostics best practices
These recommendations help reduce lockouts and access regressions during new SSO deployments, IdP migrations, and certificate rotations.
### Before rollout
- Use the **exact** Hightouch values for **SSO URL** and **Audience URI** from the SSO modal. Even minor typos can cause login loops.
- Plan your IdP group taxonomy and mapping strategy before production rollout.
- During initial validation, keep default access minimal and temporary until group mappings are verified.
### During migration or cutover
1. Recreate your SAML app in the new IdP/tenant with the same **Audience URI**, **NameID format/source**, and required claim mappings.
2. Update Hightouch with the new IdP SSO URL and signing certificate.
3. Generate a new SCIM token and connect SCIM in the new IdP app before disabling SCIM in the old app.
4. Validate with a small pilot (2-3 users), then expand to a full rollout.
Keep an admin validation path during cutovers, for example by testing in a
separate browser session before organization-wide rollout.
### Diagnostics
- Test in an incognito/private window to avoid cached session effects.
- Use a SAML tracer tool to inspect incoming NameID and group claims.
- If users can authenticate in your IdP but have no workspaces, verify SSO group mappings and that expected groups have been sent to Hightouch.
SSO disable/rollback is not self-serve in the app. If you need emergency SSO
disablement for recovery, contact Hightouch Support.
## FAQ
### Where can I find my organization identifier to log in with SSO?
After setting up SSO, you can share your workspace's direct **Hightouch login URL**, which is available in the **Single sign-on** tab on the **Organization settings** page.
The organization identifier is the part that appears after `/sso/`.
### I've enabled SSO in my workspace—how do I invite the rest of my team?
Once SSO is enabled, users no longer need a direct invitation to join your Hightouch organization. When they log in through your identity provider for the first time, Hightouch will automatically create an account for them.
### How do I allow or disallow non-SSO logins?
Go to the **Single sign-on** tab in the **Organization settings** screen. Here, you'll find a toggle labeled **Allow inviting users**. We typically recommend disabling this when using SSO to ensure that all users in your organization are managed through your central identity provider.
However, there may be situations where you'd want to allow non-SSO logins for external users, such as contractors or consultants. In these cases, you might choose to leave the setting on.
### When trying to login, I get redirected to app.hightouch.com/login.
If you see the **Log in to Hightouch** page instead of your Hightouch workspace after logging in with SSO, you need to make sure your IT team has appropriately set the **Audience URI** when [configuring SAML SSO for login](#saml-single-sign-on)).
### I don't see any workspaces after accepting an invitation.
There are a few possible reasons for this:
- If your workspace uses SSO, you shouldn't need to accept an invitation. Instead, use your organization's dedicated login link (shared with you directly or accessible through your identity provider). This ensures that you're logging in to the correct organization.
- If you land on the **Welcome to Hightouch** page, you may have accidentally created a separate Hightouch account. Keep in mind that **Log in with Google** and **Log in with Microsoft** do not connect to your company's central identity provider. To access your company's organization, be sure to select **Log in with SSO**.
- Confirm with your team that your account is mapped to at least one user group in Hightouch. Group assignments might be automatically inherited from your identity provider. Without being part of a user group, you won't have access to any workspaces.
### When I log in to Hightouch after configuring SSO, I am shown an error on the log in page.
In your SAML configuration:
- Check that you have mapped attributes for `name` and `email`. See the [SSO setup instructions](#okta-saml-sso-configuration) for more detailed steps.
- Ensure you used the correct Hightouch SAML URL and audience URL provided in your [dashboard](https://app.hightouch.com/settings/sso?from=docs).
If you made changes to the SAML app in your identity provider between uploading your self-serve SAML settings, you can try to re-generate a certification and upload the new app settings on the Hightouch [**SSO** tab](https://app.hightouch.com/settings/sso?from=docs).
### I see an error when logging into Hightouch after configuring SSO.
In your SAML configuration:
- Verify that the attributes for `name` and `email` are correctly mapped. Refer to the [SSO setup instructions](#okta-saml-sso-configuration) for detailed guidance.
- Make sure you are using the correct **Hightouch SSO URL** and **Audience URI** when configuring SAML SSO in your identity provider.
If you made changes to the SAML app in your identity provider after uploading your self-serve SAML settings, try re-generating a certificate and uploading it again.
### I see a duplicate user in Hightouch after logging in for the first time with SSO.
This is expected. SSO users are treated as separate from non-SSO users, so it's possible to have two users with the same email address (but different login methods). If you no longer need the original non-SSO user, you can manually delete it. This will not affect existing syncs or other resources.
### I don't see any groups on the SSO tab of my dashboard.
You will only see groups and their mapped roles on your **SSO** tab if your identity provider has been configured to send the `groups` attribute. Users may need to log out and log back in for changes to take effect.
### My group was updated in my organization's identity provider, but I still belong to the same group in Hightouch.
You will need to log out and log back in for changes to take effect.
### Why can't I remove certain group assignments in the Hightouch app?
Group assignments inherited from your identity provider cannot be overridden. However, you can still manually add additional users beyond those assigned by your identity provider.
### I see a "Matching user not found" error when adding users after enabling SCIM.
When SCIM is enabled, your identity provider needs to be configured to create users in Hightouch. For example, in Okta, you must go to the **Provisioning** tab and ensure your Hightouch integration app has permissions to **Create Users**, **Update User Attributes**, and **Deactivate Users**.
Once these settings are updated, delete any users showing the "Matching user not found" error and re-add them.
---
## Managing users and groups
**URL:** https://hightouch.com/docs/workspace-management/user-management
**Description:** Hightouch xxxx
**Section:** Workspace management
Before you begin, it's essential to understand the structure of a Hightouch organization. Access control is exclusively managed at the group level, so configuring users and groups should be your first step during the onboarding process.
This guide provides an overview of common user management tasks, such as [inviting users](#inviting-users), [assigning them to groups](#assigning-users-to-groups), and [managing group access](#granting-a-group-access-to-a-workspace) to workspaces. Once you've set up users and groups, you can proceed to [define custom roles](/workspace-management/roles) and manage more granular permissions.
If you're planning to connect Hightouch to your company's identity provider
(e.g., Okta), it's recommended to review the [single sign-on (SSO)
documentation](/workspace-management/sso) before inviting other users to your
organization. An identity provider can automate many of the following through
SAML SSO and SCIM.
## Users
To view a list of users in your organization, navigate to the **Organization settings** page and select the **Users** tab. There, you'll find a table of all users in your organization, including their authentication methods and group memberships. You can click on any user in the table to reveal more detailed information.
### Inviting users
If your organization is configured to use SSO, there's no need to manually invite users. Instead, go to **Organization settings** > **Single sign-on** and copy the **Hightouch login URL**. Share this link with your teammates so they can log in through your identity provider—no invite required.
For organizations not using SSO, you can invite others by going to **Organization settings** > **Users** and clicking the **Invite user** button. Only members of the **Organization admins** group can send invites. Make sure to assign the recipient to at least one user group so they receive workspace access upon their first login.
A list of pending invitations is available at the top of **Organization settings** > **Users**, where you can also withdraw invites if necessary. Otherwise, invites will expire automatically after 30 days.
### Removing users
Go to **Organization settings** > **Users**, select the user you wish to remove, and click **Remove user**.
Removing a user doesn't prevent them from rejoining your organization in the
future if they receive another invite or authenticate through your central
identity provider.
## Groups
A **group** is a collection of users. In Hightouch, access control is managed at the group level, making it easy to update permissions for many users at once.
Every Hightouch organization includes two built-in groups that cannot be deleted:
- **Organization admins** have full access to all workspaces and can manage organization-wide settings such as users, groups, billing, SSO, and more.
- **Organization viewers** have read-only access to all workspaces, but they do not have any admin privileges whatsoever.
Each organization must always have at least one admin, while the viewers group is optional and exists purely for convenience.
### Assigning users to groups
User-to-group mappings are best managed by your identity provider (e.g., Okta). However, if SSO group mappings aren't enabled or you need custom groups, you can manually set up user groups and assign users directly within the Hightouch app.
To manage group assignments for a single user, navigate to **Organization settings** > **Users**, select a user from the list, and use the **Assign** toggle for each group to add or remove the user.
To manage multiple members in a group, go to **Organization settings** > **Groups**, choose the group from the sidebar, and select the **Members** tab.
To add more users, click **Add users**; to remove existing users, check the boxes next to their names and click **Remove users**.
If group memberships are controlled by your identity provider via SSO group
mappings, you cannot override or make subtractive changes (e.g., removing
users from groups). However, you can make additive changes, such as assigning
additional members to an existing group.
### Creating a new group
Each group should represent a set of users with similar access to workspaces and their associated resources. For instance, you might create one group for “Data Science” and another for “Growth Marketing”. Groups can also be used to reflect different teams, regions, brands, or any other organizational structure that impacts permissions.
To simplify your onboarding, it's recommended to keep the number of groups to a minimum. Remember, users can belong to multiple groups at the same time, inheriting combined permissions from all their assigned groups. This can help reduce the total number of groups needed in your organization. For example, instead of creating a separate group for “Growth Marketing Analysts”, you might instead assign those users to both the “Data Science” and “Growth Marketing” groups.
To create a new group, go to **Organization settings** > **Groups** and click **Add group**. Name the group and select the members to include.
### Granting a group access to a workspace
By default, new groups don't have access to any workspaces. To grant access, navigate to **Organization settings** > **Groups**, select the group from the sidebar, and open the **Workspaces** tab. Use the **Role** dropdown to assign the appropriate access level for each workspace.
Roles determine the actions a group can perform within a workspace. You can select from built-in roles (e.g., “Workspace viewer”) or create a custom role for more fine-tuned control. For more information, refer to the [documentation for roles](/workspace-management/roles).
The **Access** column provides an overview of the group's permissions for each workspace, showing the actions they can take for each destination. While it gives a helpful summary, it doesn't display every possible grant or cover permissions related to other resource types.
## Transferring organization admin access
As people join, change roles, or leave your company, you may need to transfer **organization admin** privileges. Organization admins control organization-wide settings including SSO/SCIM configuration, billing, user management, and group permissions.
**Best practice:** Keep at least two active organization admins at all times. This prevents lockout situations if one admin leaves unexpectedly.
### Self-serve transfer
If at least one current org admin still has access to Hightouch, the transfer is fully self-serve.
**To add a new organization admin:**
1. If the new admin doesn't have a Hightouch account yet, go to **Organization settings** > **Users** and click **Invite user** to send them an invitation. Check **Organization admins** under "Assign to groups" to grant admin access immediately.
2. Go to **Organization settings** > **Groups** and select the **Organization admins** group.
3. In the **Members** tab, click **+ Add users** and add the new admin.
4. Click **Save changes**.
The new user now has full organization admin access.
**To remove a departing admin:**
1. In the **Organization admins** group, check the box next to the departing admin's name and click **Remove users**.
2. Click **Save changes**.
Hightouch requires at least one organization admin at all times. If you attempt to remove the last remaining admin, the operation will be blocked. Add a replacement admin first, then remove the departing one.
### Transfer via SSO/SCIM
If your organization uses SSO with SCIM provisioning, manage org admin membership through your identity provider instead of directly in Hightouch:
1. In your identity provider (for example, Okta or Entra ID), confirm you have a group mapped to Hightouch's **Organization admins** group.
2. Add the new admin to that IdP group and remove the departing admin.
3. SCIM will automatically sync the membership changes to Hightouch.
This approach keeps your identity provider as the single source of truth for admin access.
### When all admins have left
If no one in your company has organization admin access — for example, because all previous admins have left — you won't be able to make this change self-serve. In that case:
1. **Do not create a new organization.** Your existing organization, workspaces, and configuration are still intact.
2. Have someone from your company from a verified company email address.
3. In the request, include:
- Your **company name** and Hightouch **organization name or workspace URL**
- The **name and email** of the person who should become the new organization admin
- Any relevant context, such as when the previous admins departed
Because organization admins control billing and security settings, Hightouch may require additional verification — such as confirmation from an existing commercial or security contact on file — before assigning a new admin.
---
## Workspaces
**URL:** https://hightouch.com/docs/workspace-management/workspaces
**Description:** Learn how to create, manage, and delete Hightouch workspaces — including what each field on the creation form means, how to switch between workspaces, and common setup mistakes to avoid.
**Section:** Workspace management
*When you first sign up for Hightouch, or when your team needs an additional workspace, you'll use the workspace creation flow. This guide walks through when and how to create a workspace, what each field means, and common mistakes to avoid.*
Before creating a new workspace, make sure you actually need one. For most teams, a single production workspace is sufficient — Hightouch's [governance controls](/workspace-management/governance), including user groups, roles, Spaces, and approval flows, let multiple teams share a workspace safely. See [Do you need another workspace?](#do-you-need-another-workspace) below.
## Who should create workspaces
Workspace creation is restricted to users who have **organization admin** privileges. In most companies, this is the [**IT or platform admin**](/getting-started/it) — typically the person who also owns SSO configuration, billing, and user provisioning. If you're setting up Hightouch for the first time, the person who signs up becomes the first organization admin automatically.
If your organization uses SSO, workspace creation may be further restricted — in that case, contact your IT team or Hightouch administrator.
If you see the message **"SSO users are not allowed to create workspaces"**, your organization has disabled self-serve workspace creation. Reach out to your platform admin to have them create the workspace on your behalf.
## First sign-in
What you see when you first log in to Hightouch depends on whether your company already has an organization set up.
### New company
If no one at your company has used Hightouch before, sign up at [app.hightouch.com](https://app.hightouch.com?from=docs) using Google, Microsoft, or email.
After signing up, you'll be prompted to:
1. **Create an organization** — this represents your company and is the container for billing, SSO, and user management.
2. **Create your first workspace** — using the form described below.
After creating your workspace, the **Get started** checklist guides you through connecting a data source, adding destinations, and inviting teammates.
### Existing company
If your company already has an organization and workspaces in Hightouch, you should **not** create a new organization. Instead:
- Ask an existing admin to **invite you** to the appropriate workspace, or
- Log in through your company's **SSO provider**, which will route you to the correct organization and workspace automatically.
If you were invited to Hightouch but land on the workspace creation screen instead of your team's workspace, **stop and contact the person who invited you**. Creating a new organization or workspace in this situation will result in a duplicate that needs to be cleaned up.
## Workspace creation form
The creation form has four fields: **Organization**, **Workspace name**, **Workspace slug**, and **Region**.
### Organization
The **Organization** dropdown lets you select which organization the new workspace belongs to.
- **New organization** — if you're the first person at your company signing up for Hightouch, select **New organization**. You'll provide your organization name (typically your company's legal or common name, such as "Acme Inc."), and Hightouch will create the organization and make you its first organization admin.
- **Existing organization** — if your company already has an organization, select it from the dropdown. You'll only see organizations where you have permission to create workspaces.
Most companies should have **exactly one** organization. Multiple organizations are only appropriate when you operate truly separate legal or identity domains — for example, a holding company with independent subsidiaries that don't share users, billing, or SSO configuration.
If your company already uses Hightouch, do not select **New organization**. This creates a duplicate organization that will need to be cleaned up. Instead, ask an existing admin to invite you, or log in via SSO.
### Workspace name
The name appears in the workspace switcher and helps your team identify the workspace. Choose something descriptive:
- **Single workspace:** Your company name, or something like ` Production`.
- **Production + sandbox:** ` Production` and ` Sandbox` (or `Playground`).
- **Multiple workspaces:** Use a pattern that makes the distinction clear, such as ` – US` and ` – EU`, or `` and ``.
You can change the workspace name later in **Settings → Workspace**.
### Workspace URL/slug
The slug becomes part of your workspace URL: `app.hightouch.com/`. It must be globally unique and is auto-generated from the workspace name.
Treat the slug as **permanent**. While workspace admins can technically update it later in **Settings → Workspace**, changing the slug breaks all existing bookmarked URLs, saved links, scripts, and integrations that reference the old slug — and there is no automatic redirect. Choose carefully before submitting the form.
### Region and cloud provider
The region determines where Hightouch processes your data. You'll select both a **geographic region** (United States, European Union, Asia Pacific, or Middle East) and a **cloud provider** (AWS, Google Cloud, or Azure). The available cloud providers vary by region.
When choosing a region:
- **Prefer the region closest to your primary data warehouse** and your end users to reduce latency.
- **Consider data residency requirements.** If compliance mandates that data stay within a specific jurisdiction, select a region in that jurisdiction.
- **Check feature availability.** Not all features are available in every region. Refer to the [Regions](/security/regions) documentation for a current compatibility matrix.
Region cannot be changed after workspace creation. If you create a workspace in the wrong region, you'll need to create a new workspace in the correct region and migrate your configuration. if you need assistance.
Some newer regions may be in limited availability. If the region you need isn't visible in the dropdown, to check availability and have our team help provision your workspace.
## After creation
Once your workspace is created, Hightouch presents a **Get started** checklist that walks you through initial setup:
1. **Invite teammates** — add collaborators so they can help configure the workspace.
2. **Set up storage bucket** *(Business tier only)* — bring your own S3, GCS, or Azure Blob Storage bucket for data-at-rest. This step only appears for Business tier workspaces. See [Storage](/security/storage).
3. **Connect to your data warehouse** — set up access to your data source. See [Sources](/sources/overview).
4. **Add destinations** — connect your marketing, sales, or other downstream tools. See [Destinations](/destinations/overview).
The exact steps shown may vary depending on your plan tier and workspace configuration.
## Switching between workspaces
If you have access to multiple workspaces, you can switch between them in two ways:
- **Workspace switcher** — click the workspace name in the top-left corner of the sidebar to open the workspace dropdown, or press **S** then **W** anywhere in the app. You can also open the command bar with **Cmd+K** (Mac) / **Ctrl+K** (Windows) and type "Switch workspace." Search by name or slug and select the workspace you want.
- **Direct URL** — navigate to a URL containing another workspace's slug (for example, `app.hightouch.com/other-workspace/syncs`). Hightouch will automatically switch your active workspace to match.
## Deleting a workspace
There is no self-serve option to delete a workspace. You can find the deletion option in **Settings → Workspace**, but it routes you to Hightouch support.
If you need to remove a workspace — for example, one created by mistake — . Before submitting your request, remove or disconnect all sources, destinations, syncs, and other resources from the workspace. All current organization or workspace admins will need to approve the deletion.
## Do you need another workspace?
Before creating an additional workspace, consider whether [Environments](/workspace-management/environments) or other governance controls would be a better fit.
**Use Environments when:**
- You need dev / staging / production promotion for the same data and teams.
- You want change control and approval flows without hard security isolation.
**Create a separate workspace when:**
- You have strict isolation requirements — regulatory, regional, or business-unit-level — where sources, destinations, and data must not be visible across boundaries.
- You need fully separate tenants under the same commercial relationship, such as independent subsidiaries.
| Consideration | Environments | Separate workspaces |
| :--- | :--- | :--- |
| Shared sources and destinations | Yes | No — each workspace has its own |
| Shared user roster | Yes | No — managed separately per workspace |
| Approval flows | Yes | Yes, but independent per workspace |
| Data isolation | Logical (same workspace) | Physical (nothing shared) |
| Typical use case | Dev → Staging → Prod | US region vs EU region |
For more on structuring your organization, see [Governance best practices](/workspace-management/governance).
## Troubleshooting
### I was invited but see the create-workspace screen
This usually means the invitation didn't link you to the correct organization or workspace. Don't create a new organization — instead, contact the person who invited you and ask them to re-send the invite or check that your account is correctly associated with their organization.
### I accidentally created a duplicate organization
If you signed up directly instead of using an invitation or SSO, you may have created a second organization for your company. to consolidate your organizations.
### I need to change my workspace region
Region cannot be changed self-serve after creation. to plan a migration. Before you reach out, be aware of the following:
- **Brief outage required.** The migration involves a short period of downtime for the affected workspace. Schedule it during low-traffic hours.
- **Tunnels must be recreated.** Existing [SSH tunnels](/security/ssh-tunneling) cannot be migrated between regions. You'll need to set up new tunnels in the target region after the migration.
- **IP allowlists need updating.** If your sources or destinations are behind a firewall that allowlists Hightouch's [IP addresses](/security/networking#ip-addresses), add the new region's IP range before the migration and remove the old range afterward.
- **Sync history and debugger data.** If you use [self-hosted storage](/security/storage#self-hosted-storage) or the [Lightning sync engine](/syncs/lightning-sync-engine), your sync history and CDC state are preserved. If you use Hightouch-managed storage with the standard sync engine, syncs will need a full resync after the migration and pre-migration debugger data may not carry over.
## Related resources
- [Core concepts](/workspace-management/overview)
- [Governance best practices](/workspace-management/governance)
- [Environments](/workspace-management/environments)
- [Regions](/security/regions)
- [Storage](/security/storage)
- [Managing users and groups](/workspace-management/user-management)
---
## Amazon Web Services
**URL:** https://hightouch.com/docs/security/aws
**Description:** Grant Hightouch access to resources within your AWS account.
**Section:** Security and compliance
## Overview
Hightouch's AWS integration powers several features:
- Workspace deployment on AWS (see [Regions](/security/regions#supported-regions))
- [Self-hosted storage via S3](/security/storage#self-hosted-storage)
- [Athena](/sources/amazon-athena) source
- [S3](/sources/s3) source
- [S3](/destinations/s3) destination
- [AWS PrivateLink](/security/aws/privatelink)
## Setup
To give Hightouch access to resources on your AWS account,
go to the [**Cloud providers**](https://app.hightouch.com/settings/cloud-providers?from=docs) tab on the Settings page.
Click **Add cloud provider**, and select **Amazon Web Services**.
Give your credential a **Display name**, then select either **Cross-account role** or **Access key**.
### Cross-account role
Cross-account roles are a mechanism provided by AWS to allow you to grant secure access to Hightouch without requiring that you hand over sensitive secrets like Secret Access Keys.
The Hightouch UI displays an **AWS account ID** and a randomly generated **AWS external ID**, as shown below, which you need to enter into an IAM Role you create in your AWS account. Store these values for future reference.
Neither the Account ID nor the external ID are secrets, so you don't need to worry about securely storing them.
1. From your AWS console, navigate to **Identity and Access Management (IAM)** > **Access Management** > **Roles** and click **Create Role**.
2. Under **Select type of trusted entity**, choose **AWS account**. Then under **An AWS account**, select **Another AWS account**.
3. Enter the **Account ID** you copied from the Hightouch UI, select **Require external ID** and enter the external ID you copied from the Hightouch UI. Click **Next**.
4. Attach permissions to the role, choose a role name, then click **Create role**. The exact permission policies you attach depend on which Hightouch features you intend to use. Consult the documentation [for those services](#overview) for further guidance.
5. Copy the **Role ARN** from AWS IAM and paste it into the **Role ARN** field in Hightouch. Click **Create** to complete the process.
For more information, read [AWS's tutorial](https://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_cross-account-with-roles.html) on delegating access cross-account using IAM Roles.
### Access key
The **Access Key** access type allows you to configure Hightouch to use an IAM user by providing the user's **Access Key ID** and **Secret Access Key**.
If you need help generating these keys, consult the [IAM article](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) on this topic.
Once you have an Access Key ID and Secret Access Key, paste those values into the form and click **Create**.
## Tips and troubleshooting
{/* */}
### CredentialsError: Missing credentials in configuration
{/* */}
If you receive the following error `CredentialsError: Missing credentials in config`, it means Hightouch can't assume the role required by your Athena or S3 sync, or [external storage](/security/storage) configuration.
This error is often due to an issue with the [IAM trust policy](https://aws.amazon.com/blogs/security/how-to-use-trust-policies-with-iam-roles/). This policy defines which other AWS accounts can assume roles in your account. When you set up AWS credentials in Hightouch using a [cross-account role](#cross-account-role), we provide you with Hightouch's Account ID and an additional external ID. You should include Hightouch's Account ID in your role trust policy, and can optionally include the external ID.
To troubleshoot this error, follow these steps:
- Ensure that the role exists.
- Ensure that the role has a trust policy attached.
- Ensure that the role's trust policy includes our Hightouch AWS account ID (`324528010823`) as a principal.
- If the role trust policy includes an external ID check, ensure it matches the external ID generated by Hightouch. See the example below:
If you want to check the external ID but misplaced the external ID that Hightouch generates, you need to [re-create the cloud provider integration in Hightouch](#setup). Hightouch only displays this value once during cloud provider setup.
---
## AWS PrivateLink
**URL:** https://hightouch.com/docs/security/aws/privatelink
**Description:** Securely connect your warehouse to Hightouch using AWS PrivateLink.
**Section:** Security and compliance
Using [AWS PrivateLink](https://aws.amazon.com/privatelink/), you can securely connect your AWS-deployed services to Hightouch.
Network traffic between Hightouch and your services connected by AWS PrivateLink stays on the [AWS backbone network](https://aws.amazon.com/about-aws/global-infrastructure/global_network/) and doesn't travel over the public internet. That means that you can restrict public access to your services, and your data is never exposed to the public internet.
Hightouch supports managed AWS PrivateLink integrations for [Snowflake](/sources/snowflake), [Redshift](/sources/amazon-redshift), [Databricks](/sources/databricks), [Amazon MSK](/events/sources/kafka), and other integrations with [VPC Endpoint Service](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-share-your-services.html).
## Snowflake
Hightouch supports connecting to your Snowflake account via AWS PrivateLink.
### Prerequisites
- Hightouch supports AWS PrivateLink on [Business tier](https://hightouch.com/pricing) accounts.
- Snowflake supports AWS PrivateLink on [Business Critical](https://docs.snowflake.com/en/user-guide/intro-editions.html#business-critical-edition) accounts or higher.
- Your Snowflake account must be located in one of the following AWS regions: `us-east-1`, `us-west-2`, `eu-west-1`, `ap-south-1`, or `ap-southeast-2`
- Your Hightouch workspace must be located in the AWS `us-east-1`, `eu-west-1`, or `ap-south-1` region.
### Getting started
Please to our customer success team to get started with AWS PrivateLink for Snowflake.
### Learn more
Check out these AWS and Snowflake docs to learn more:
- [What is AWS PrivateLink?](https://docs.aws.amazon.com/vpc/latest/privatelink/what-is-privatelink.html)
- [AWS PrivateLink & Snowflake](https://docs.snowflake.com/en/user-guide/admin-security-privatelink.html)
## Databricks
Hightouch can use AWS PrivateLink to connect to your Databricks workspace control plane. This allows Hightouch to securely interact with your Databricks workspace without exposing your data to the public internet.
Hightouch uses Databricks "Front-end PrivateLink", which is distinct from Databricks "Back-end PrivateLink", which is used to connect the Databricks control plane to a compute plane running in your AWS VPCs.
### Prerequisites
- Hightouch supports AWS PrivateLink on [Business tier](https://hightouch.com/pricing) accounts.
- Your Hightouch workspace must be located in the AWS `us-east-1`, `eu-west-1`, or `ap-south-1` region.
- Your Databricks workspace must be located in one of the following AWS regions: `us-east-1`, `us-west-2`, `eu-west-1`, `ap-south-1`, or `ap-southeast-2`
- Your Databricks workspace must have been created with a "Private Access Settings" object. Databricks does not support setting a Private Access Settings object on existing workspaces. If you need to use PrivateLink with an existing workspace, you will need to create a new workspace with the appropriate settings.
### Getting started
Please to our customer success team to get started with AWS PrivateLink for Databricks.
### Learn more
- [Enable AWS PrivateLink](https://docs.databricks.com/aws/en/security/network/front-end/front-end-private-connect)
## Redshift
Hightouch supports connecting to both provisioned Redshift clusters and Redshift Serverless workgroups via PrivateLink with [Redshift-managed VPC Endpoints](https://docs.aws.amazon.com/redshift/latest/mgmt/managing-cluster-cross-vpc.html).
### Prerequisites
- Hightouch supports AWS PrivateLink on [Business tier](https://hightouch.com/pricing) accounts.
- Your Redshift cluster or workgroup must be located in one of the following AWS regions: `us-east-1`, `us-west-2`, `eu-west-1`, `ap-south-1`, or `ap-southeast-2`
- Your Hightouch workspace must be located in the AWS `us-east-1`, `eu-west-1`, or `ap-south-1` region.
**For provisioned clusters:**
- Your Redshift cluster must be of the `RA3` node type.
- Your Redshift cluster must have [Cluster Relocation](https://docs.aws.amazon.com/redshift/latest/mgmt/managing-cluster-recovery.html) or Multi-AZ enabled.
### Getting started
Please to our customer success team to get started with AWS PrivateLink for Redshift.
### Learn more
- [What is AWS PrivateLink?](https://docs.aws.amazon.com/vpc/latest/privatelink/what-is-privatelink.html)
- [Redshift-managed VPC endpoint documentation](https://docs.aws.amazon.com/redshift/latest/mgmt/managing-cluster-cross-vpc.html)
- [Connecting to Amazon Redshift Serverless](https://docs.aws.amazon.com/redshift/latest/mgmt/serverless-connecting.html#serverless-cross-vpc)
## Amazon MSK
Hightouch supports [Multi-VPC private connectivity](https://docs.aws.amazon.com/msk/latest/developerguide/aws-access-mult-vpc.html) for Amazon Managed Streaming for Apache Kafka (MSK). This is a managed solution that handles the networking infrastructure automatically, without requiring you to set up a Network Load Balancer or VPC Endpoint Service.
### Prerequisites
- Hightouch supports AWS PrivateLink on [Business tier](https://hightouch.com/pricing) accounts.
- Your MSK cluster must be running Apache Kafka 2.7.1 or later.
- Your MSK cluster must use `SASL_IAM` or `SASL_SCRAM` authentication.
- Your MSK cluster cannot use the `t3.small` instance type.
- You must [turn on multi-VPC connectivity](https://docs.aws.amazon.com/msk/latest/developerguide/mvpc-cluster-owner-action-turn-on.html) for your cluster.
- You must [enable cross-account access](https://docs.aws.amazon.com/msk/latest/developerguide/mvpc-cluster-owner-action-policy.html) for the Hightouch account `810028259040`.
- Your MSK cluster must be located in one of the following AWS regions: `us-east-1`, `us-west-2`, `eu-west-1`, `ap-south-1`, or `ap-southeast-2`
- Your Hightouch workspace must be located in the AWS `us-east-1`, `eu-west-1`, or `ap-south-1` region.
### Getting started
Please to our customer success team to get started with AWS PrivateLink for Amazon MSK.
### Learn more
- [Amazon MSK multi-VPC private connectivity](https://docs.aws.amazon.com/msk/latest/developerguide/aws-access-mult-vpc.html)
## Other integrations
Hightouch can connect to any source or destination exposed via a [VPC Endpoint Service](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-share-your-services.html). This allows you to expose any service running in your VPC to Hightouch without exposing it to the public internet.
### Prerequisites
- Hightouch supports AWS PrivateLink on [Business tier](https://hightouch.com/pricing) accounts.
- Your integration (source / destination) must be exposed via a [VPC Endpoint Service](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-share-your-services.html). This generally requires a [Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) to sit in front of your service.
- Your VPC Endpoint Service must be located in one of the following AWS regions: `us-east-1`, `us-west-2`, `eu-west-1`, `ap-south-1`, or `ap-southeast-2`
- Your Hightouch workspace must be located in the AWS `us-east-1`, `eu-west-1`, or `ap-south-1` region.
### Getting started
Please to our customer success team to get started with AWS PrivateLink for VPC Endpoint Services.
### Learn more
- [What is AWS PrivateLink?](https://docs.aws.amazon.com/vpc/latest/privatelink/what-is-privatelink.html)
- [What is a Network Load Balancer?](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html)
- [Share your services through AWS PrivateLink](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-share-your-services.html)
---
## Microsoft Azure
**URL:** https://hightouch.com/docs/security/azure
**Description:** Grant Hightouch access to resources within your Azure account.
**Section:** Security and compliance
## Overview
Hightouch's Azure integration powers several features:
- Workspace deployment on Azure (see [Regions](/security/regions#supported-regions))
- [Self-hosted storage via Azure Blob Storage](/security/storage#self-hosted-storage)
- [Azure Blob Storage](/sources/azure-blob) source
- [Azure Blob Storage](/destinations/azure-blob) destination
- [Azure Private Link](/security/azure/private-link)
## Configure Azure
Azure configuration includes creating an app registration and granting access to Blob Storage. Once you've done both of these, you can use your Blob Storage information and Azure credentials to connect Hightouch.
### Create an app registration
1. In the [Azure Portal](https://portal.azure.com/), navigate to **Azure Active Directory**.
2. Locate and click **Manage** > **App registrations** in the left-hand menu.
3. Create a new app registration by selecting **+ New registration**. Give the application a descriptive **Name**, for example, "Hightouch-integration-user," and leave the **Supported account type** as **Accounts in this organizational directory only (Default Directory only - Single tenant)**. You don't need a redirect URI for Hightouch storage integration. Click **Register** when done.
4. Next, you need an app secret for this account. While on the app registration screen, navigate to **Manage** > **Certificates & secrets** in the left-hand menu.
In the **Client secrets** tab, click **New client secret**. Enter a **Description** and **Expires** time frame. Click **Add** to create the secret.
Make sure to select a sufficient expiry time frame. If this credential expires, Hightouch can't
access your Azure Blob Storage, and if your workspace is configured to use Azure for log storage, all syncs will fail.
5. Copy and store the secret **Value**, _not_ the **Secret ID**, after you create it. You won't be able to read the value again later.
6. Navigate to the App **Overview** in the left-hand menu. Copy and save the **Application (client) ID** and **Directory (tenant) ID**. You need both of these and the Client secret value
to configure Hightouch.
### Grant access to Blob Storage
1. In the [Azure Portal](https://portal.azure.com/), navigate to your storage account, and then to the Blob Storage container you want to connect to Hightouch. If you don't have a container, [create one](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-portal#create-a-container).
2. In the configuration for your Blob Storage container, go to **Access Control (IAM)** in the left-hand menu.
3. Select **+Add** then **Add role assignment**.
4. Select the role you want to grant to the Hightouch integration. To use the integration for storage, we suggest
using the **Storage Blob Data Contributor** role.
If you want to assign more granular permissions, refer to the for [storage](/security/storage) documentation
and consult your Active Directory administrator.
Once you've selected the role, click **Next** to assign **Members** to the role.
4. Keep **User, group, or service principal** as the access assignment. Click **+ Select members** and search for and select the Hightouch integration user you created during [app registration](#create-an-app-registration).
6. Click **Review + assign** to confirm. The Hightouch user should now have access to your Azure Blob Storage.
## Configure Hightouch
To configure Hightouch to use the Azure integration user,
go to the [**Cloud providers**](https://app.hightouch.com/settings/cloud-providers?from=docs) tab on the Settings page. Click **Add cloud provider**, and select **Microsoft Azure**.
Give your credential a **Display name**, and enter the **Tenant ID**, **Client ID**, and **Client secret** from your Azure configuration.
## Azure credential usage
Once you've set up your Microsoft Azure credentials in Hightouch, you can select Microsoft Azure when setting up [external storage](/security/storage).
---
## Azure Private Link
**URL:** https://hightouch.com/docs/security/azure/private-link
**Description:** Securely connect to Hightouch using Azure Private Link.
**Section:** Security and compliance
Using [Private Link](https://learn.microsoft.com/en-us/azure/private-link/private-link-overview), you can securely connect your Azure-deployed services to Hightouch.
Network traffic between Hightouch and your services connected by Azure Private Link stays on the Microsoft backbone network and doesn't travel over the public internet. That means that you can restrict public access to your services, and your data is never exposed to the public internet.
Hightouch currently supports Azure Private Link for the following services:
- Azure Databricks
- Azure Blob Storage
- Snowflake (Azure-hosted)
## Azure Databricks
### Prerequisites
- Hightouch supports Azure Private Link on [Business tier](https://hightouch.com/pricing) accounts.
### Getting started
Please to our customer success team to get started with Azure Private Link for Azure Databricks.
## Azure Storage
### Prerequisites
- Hightouch supports Azure Private Link on [Business tier](https://hightouch.com/pricing) accounts.
### Getting started
Please to our customer success team to get started with Azure Private Link for Azure Storage.
## Snowflake (Azure-hosted)
### Prerequisites
- Hightouch supports Azure Private Link on [Business tier](https://hightouch.com/pricing) accounts.
### Getting started
Please to our customer success team to get started with Azure Private Link for Snowflake.
#### Setup steps
1. Run [this command](https://docs.snowflake.com/en/sql-reference/functions/system_get_privatelink_config) in Snowflake to retrieve your PrivateLink config:
```sql
SELECT SYSTEM$GET_PRIVATELINK_CONFIG();
```
2. Once verified, Hightouch will connect to your Snowflake destination privately via Azure Private Link.
For reference, see [Snowflake’s documentation](https://docs.snowflake.com/en/user-guide/privatelink-azure#configure-access-to-snowflake-with-azure-private-link) on Azure Private Link setup.
### Learn more
- [What is Azure Private Link?](https://learn.microsoft.com/en-us/azure/private-link/private-link-overview)
- [Configure private connectivity to Azure Databricks](https://learn.microsoft.com/en-us/azure/databricks/security/network/front-end/front-end-private-connect#private-connectivity-to-azure-databricks-overview)
- [Use private endpoints for Azure Storage](https://learn.microsoft.com/en-us/azure/storage/common/storage-private-endpoints)
---
## Google Cloud Platform
**URL:** https://hightouch.com/docs/security/gcp
**Description:** Grant Hightouch access to resources within your Google Cloud project.
**Section:** Security and compliance
## Overview
Hightouch's Google Cloud integration powers several features:
- Workspace deployment on Google Cloud (see [Regions](/security/regions#supported-regions))
- [Self-hosted storage via GCS](/security/storage#self-hosted-storage)
- [BigQuery](/sources/google-bigquery) source
- [Google Cloud Storage](/sources/gcs) source
- [Google Cloud Storage](/destinations/gcs) destination
- [Google Private Service Connect](/security/gcp/private-service-connect)
- [Workload Identity Federation](/security/gcp/workload-identity-federation)
## Setup
To give Hightouch access to resources on your GCP account,
go to the [**Cloud providers**](https://app.hightouch.com/settings/cloud-providers?from=docs) tab on the Settings page.
Click **Add cloud provider**, and select **Google Cloud Platform**.
Give your credential a **Display name**, then select either **Service account managed by Hightouch** or **Bring your own service account**.
### Managed service account
When using the managed service account access type, Hightouch creates a service account within our secure Google Cloud account, to which you can bind your project's IAM policies. IAM policies grant permissions within your Google Cloud project.
When you click **Create a new service account**, Hightouch generates a new service account in our project. The service account is unique to your workspace and this credential. You can then copy the service account email and click **Create** to confirm the service account creation.
Next, you need to bind IAM roles to this service account by calling [`gcloud projects add-iam-policy-binding`](https://cloud.google.com/sdk/gcloud/reference/projects/add-iam-policy-binding) via the Google Cloud CLI. The specific roles you grant depend on the types of resources you want to grant Hightouch access to, such as [Cloud Storage](/security/storage) or [BigQuery](/sources/google-bigquery). Consult the relevant docs for more information on the specific permissions Hightouch needs.
### Bring your own service account
When you **Bring your own service account** you upload your service account's key, which Hightouch securely stores.
First, you need to [create a service account](https://cloud.google.com/iam/docs/creating-managing-service-accounts), [grant it the relevant roles](https://cloud.google.com/iam/docs/granting-changing-revoking-access), and [generate a key](https://cloud.google.com/iam/docs/creating-managing-service-account-keys). Make sure you create **JSON** key, not P12. Once the service account and key have been created, download the key to your machine and drag and drop it into the file selector.
Hightouch then parses the key, and if it's valid, you see the **Client ID** and **Client Email** fields populated. If those correspond with your intended service account, click **Create**.
Make sure the service account you attached has the specific roles you need.
These permissions depend on the types of resources you want to grant Hightouch access to, such as [Cloud Storage](/security/storage) or [BigQuery](/sources/google-bigquery).
Consult the relevant docs for more information on the specific permissions Hightouch needs.
### Workload Identity Federation
If you need keyless authentication, [Workload Identity Federation](/security/gcp/workload-identity-federation) is available by request as an alternative to uploading a service account key.
## Google Cloud credential usage
Once you've set up your Google Cloud credentials in Hightouch, you can now use them throughout the app, for example for [external storage](/security/storage), the [BigQuery](/sources/google-bigquery) and [GCS](/sources/gcs) sources, and other Google Cloud integrations like [Google Ad Manager](/destinations/gam360).
Regardless of whether you use a managed service account or being your own,
ensure it has required permissions. Consult the relevant docs for more
information on the specific permissions Hightouch needs.
---
## Private Service Connect
**URL:** https://hightouch.com/docs/security/gcp/private-service-connect
**Description:** Securely connect your warehouse to Hightouch using Google Cloud Private Service Connect.
**Section:** Security and compliance
Using [Private Service Connect](https://cloud.google.com/vpc/docs/private-service-connect), you can securely connect your Google Cloud-deployed services to Hightouch.
Network traffic between Hightouch and your services connected by Private Service Connect stays on the Google Cloud network and doesn't travel over the public internet. That means that you can restrict public access to your services, and your data is never exposed to the public internet.
## Cloud SQL
Hightouch supports Google Cloud Private Service Connect for Cloud SQL instances (MySQL, Postgres, SQL Server).
### Prerequisites
- Hightouch supports GCP Private Service Connect on [Business tier](https://hightouch.com/pricing) accounts.
- Your Hightouch workspace must be hosted on **GCP** (any Hightouch GCP region).
- Your Cloud SQL instance must be in the GCP `us-east1`, `us-east4`, or `us-west2` regions. If you're interested in using a different region, please reach out to our customer success team.
- Your Cloud SQL instance must have Private Service Connect enabled.
### Getting started
You will need to enable Private Service Connect on your Cloud SQL instance. You can do this by following the documentation for [MySQL](https://cloud.google.com/sql/docs/mysql/private-service-connect), [PostgreSQL](https://cloud.google.com/sql/docs/postgres/private-service-connect), or [SQL Server](https://cloud.google.com/sql/docs/sqlserver/configure-private-service-connect#create-cloud-sql-instance-psc-enabled). You will also need to authorize the Hightouch Google Cloud Project to create a Private Service Connect endpoint to your Cloud SQL instance.
Please to our customer success team to get started with GCP Private Service Connect for Cloud SQL.
### Learn more
- [What is Private Service Connect?](https://cloud.google.com/vpc/docs/private-service-connect)
- [Configure Private Service Connect - MySQL](https://cloud.google.com/sql/docs/mysql/configure-private-service-connect)
- [Configure Private Service Connect - PostgreSQL](https://cloud.google.com/sql/docs/postgres/configure-private-service-connect)
- [Configure Private Service Connect - SQL Server](https://cloud.google.com/sql/docs/sqlserver/configure-private-service-connect)
---
## Workload Identity Federation
**URL:** https://hightouch.com/docs/security/gcp/workload-identity-federation
**Description:** Authenticate Hightouch to a Google Cloud service account you own using AWS Workload Identity Federation, with no service account key.
**Section:** Security and compliance
Workload Identity Federation is available by request only. It relies on a
legacy authentication path, and for most use cases a [managed or
bring-your-own service account](/security/gcp) is simpler and recommended.{" "}
{" "}
to request access.
Workload Identity Federation (WIF) lets Hightouch authenticate as a Google Cloud service account you own, without sharing a service account JSON key. Hightouch's workers run on AWS, so your Google Cloud Workload Identity Pool trusts Hightouch's AWS identity, which then impersonates your service account to access your data.
## Prerequisites
- Workload Identity Federation must be enabled for your workspace. Use the **Contact Hightouch** link above to request access.
- You need administrator access to your Google Cloud project, specifically the `roles/iam.workloadIdentityPoolAdmin` and `roles/iam.serviceAccountAdmin` roles.
- The [Google Cloud CLI](https://cloud.google.com/sdk/docs/install) (or [Cloud Shell](https://cloud.google.com/shell)) with your target project active.
## Set up Workload Identity Federation
Run each block below in order, in your project's Cloud Shell. The first block sets variables that the rest reuse, so you can paste each one without edits. Make sure the right project is active first with `gcloud config set project `.
**1. Set variables**
```bash
```
**2. Create a Workload Identity Pool**
```bash
gcloud iam workload-identity-pools create "$HT_POOL" \
--project="$HT_PROJECT_ID" \
--location=global \
--display-name="Hightouch WIF"
```
**3. Add an AWS provider that trusts only Hightouch's worker identity**
The `--attribute-condition` is the security boundary: Google issues a token only when the caller is exactly Hightouch's AWS worker identity.
```bash
gcloud iam workload-identity-pools providers create-aws "$HT_PROVIDER" \
--project="$HT_PROJECT_ID" \
--location=global \
--workload-identity-pool="$HT_POOL" \
--account-id="$HT_AWS_ACCOUNT" \
--attribute-mapping="google.subject=assertion.arn,attribute.aws_identity=assertion.arn" \
--attribute-condition="assertion.arn == '$HT_AWS_PRINCIPAL'"
```
**4. Create a dedicated service account for Hightouch**
```bash
gcloud iam service-accounts create "$HT_SA" \
--project="$HT_PROJECT_ID" \
--display-name="Hightouch (WIF)"
```
**5. Allow only Hightouch's identity to impersonate the service account**
```bash
gcloud iam service-accounts add-iam-policy-binding "$HT_SA_EMAIL" \
--project="$HT_PROJECT_ID" \
--role=roles/iam.workloadIdentityUser \
--member="principalSet://iam.googleapis.com/projects/$HT_PROJECT_NUMBER/locations/global/workloadIdentityPools/$HT_POOL/attribute.aws_identity/$HT_AWS_PRINCIPAL"
```
**6. Generate the credential configuration file**
```bash
gcloud iam workload-identity-pools create-cred-config \
"projects/$HT_PROJECT_NUMBER/locations/global/workloadIdentityPools/$HT_POOL/providers/$HT_PROVIDER" \
--service-account="$HT_SA_EMAIL" \
--aws \
--output-file=wif-config.json
```
## Grant the service account access to your data
The setup above lets Hightouch sign in as `ht-wif-sa`, but it doesn't grant access to anything yet. Which roles you need depends on the connector, so check that connector's docs (for example [BigQuery](/sources/google-bigquery)) and grant those roles to `ht-wif-sa`.
Grant them on the resource itself, not just the project. To let Hightouch write to a Cloud Storage bucket, for example, bind `roles/storage.objectAdmin` (or the narrower `roles/storage.objectCreator`) on the bucket:
```bash
gcloud storage buckets add-iam-policy-binding gs://YOUR_BUCKET \
--member="serviceAccount:$HT_SA_EMAIL" \
--role=roles/storage.objectAdmin
```
A project-level grant only covers buckets that project owns, so if your bucket
lives in another project it won't apply. Grant the role on the bucket itself
to be safe. The service account and the bucket don't have to share a project,
and IAM changes can take a minute or two to take effect.
## Upload the configuration to Hightouch
In Hightouch, go to [**Cloud providers**](https://app.hightouch.com/settings/cloud-providers?from=docs), add a **Google Cloud Platform** credential, select **Workload Identity Federation**, and upload the `wif-config.json` file you generated.
The `gcloud ... create-cred-config --aws` output works as-is. There's no need
to hand-edit the file before uploading.
## Troubleshooting
### `storage.objects.create` denied (HTTP 403)
You finished the setup, but a sync fails with something like:
```json
{
"error": {
"code": 403,
"message": "ht-wif-sa@YOUR_PROJECT.iam.gserviceaccount.com does not have storage.objects.create access to the Google Cloud Storage object. Permission 'storage.objects.create' denied on resource '...' (or it may not exist)."
}
}
```
This means federation is already working: the error names your service account (`ht-wif-sa@...`), so Google exchanged Hightouch's AWS identity and impersonated the account successfully. What's missing is that account's access to the resource itself. Work through these in order:
1. **Use the Policy Troubleshooter.** The 403 includes a [Policy Troubleshooter](https://cloud.google.com/iam/docs/troubleshooting-access) link (`console.cloud.google.com/iam-admin/troubleshooter/...`). It names the exact principal, permission, and resource, and the binding you're missing.
2. **Confirm the bucket's project.** For a resource it can't see, GCS returns `403 ... (or it may not exist)` rather than `404`, so a bucket in the wrong project looks just like a permissions problem. Check which project owns it:
```bash
gcloud storage buckets describe gs://YOUR_BUCKET --format="value(name,project_number)"
```
3. **Grant the role on the resource, then wait.** See [Grant the service account access to your data](#grant-the-service-account-access-to-your-data). Bucket and object IAM changes take a minute or two to take effect.
4. **Rule out WIF.** Impersonate the service account directly, without Hightouch involved, and retry:
```bash
echo test > /tmp/wif-test.txt
gcloud storage cp /tmp/wif-test.txt gs://YOUR_BUCKET/wif-test.txt \
--impersonate-service-account=$HT_SA_EMAIL
```
If that fails too, it's purely a GCP IAM problem on the resource, and fixing the grant fixes the sync. If it works but Hightouch still gets a 403, .
## Learn more
- [Workload Identity Federation](https://cloud.google.com/iam/docs/workload-identity-federation)
- [Configuring Workload Identity Federation with AWS](https://cloud.google.com/iam/docs/workload-identity-federation-with-other-clouds)
- [Google Cloud Platform credentials in Hightouch](/security/gcp)
---
## Networking
**URL:** https://hightouch.com/docs/security/networking
**Description:** Allow Hightouch to connect to your warehouse, destinations, and other internal services.
**Section:** Security and compliance
Hightouch can connect directly to services running within your private network, including your data warehouse, self-hosted destinations, and even tools like a [private Git server](/extensions/git-sync) or your own [hosted Looker deployment](/sources/looker).
For added security, considering using an [SSH tunnel](/security/ssh-tunneling) to layer on public key-based authentication and encryption.
## IP addresses
Many private networks need to be configured to allow external connections. You can provide Hightouch access by adding Hightouch's IP addresses to your firewall allowlist.
Ensure you configure your network firewall rules to allow access from _only_ the IP addresses corresponding to the [region](/security/regions) you selected when you created your Hightouch workspace.
### Amazon Web Services
| Region | IP Addresses |
| ---------------------------- | ---------------------------------------------------------------------------------------------- |
| ap-south-1 (Mumbai, India) | `13.126.190.7` `65.1.90.146` |
| eu-west-1 (Ireland) | `34.255.226.212` `52.16.84.199` `52.211.114.138` |
| us-east-1 (N. Virginia, USA) | `54.196.30.169` `52.72.201.213` `18.213.226.96` `3.224.126.197` `3.217.26.199` |
### Google Cloud Platform
| Region | IP Addresses |
| ---------------------------------- | ---------------------------------------------------------- |
| europe-west1 (Belgium) | `35.241.218.183` `35.195.167.179` `104.199.61.185` |
| me-central2 (Dammam, Saudi Arabia) | `34.166.244.44` `34.166.247.32` `34.166.115.20` |
| us-east4 (N. Virginia, USA) | `34.145.155.148` `35.245.150.71` `35.245.58.91` |
### Microsoft Azure
| Region | IP Addresses Ranges |
| --------------------- | ------------------- |
| East US (N. Virginia) | `23.100.16.164/30` |
---
## Security
**URL:** https://hightouch.com/docs/security/overview
**Description:** Learn about Hightouch's security practices and infrastructure.
**Section:** Security and compliance
[**Explore the interactive architecture diagram**](https://architecture.hightouch.com/?from=docs&product=reverse-etl&utm_source=hightouch-docs&utm_medium=referral&utm_campaign=architecture-explorer&utm_content=reverse-etl)
See how data moves through Hightouch — including security controls, private networking, and storage boundaries.
## Overview
At Hightouch, we work with large healthcare, financial, and public enterprises to activate and govern sensitive customer data—across syncs, events, Customer Studio, and AI Decisioning. We are committed to building a warehouse‑native customer data and AI platform designed to meet rigorous enterprise [security standards](https://hightouch.com/platform/security).
This document gives an overview of our security processes and infrastructure architecture.
---
## Key security highlights
We’ve been through thousands of rigorous security reviews. These are the most important, high-level points to understand before diving into the details:
- **Hightouch never stores your customer data at rest***
For standard Reverse ETL, Events, and composable CDP use cases, Hightouch does **not** create a separate, long‑lived copy of your primary customer tables in Hightouch infrastructure. Your source‑of‑truth data stays in your warehouse and other systems of record. Hightouch uses only the **minimal, configurable storage** needed for logs, change‑data‑capture state, and certain AI workloads.
- **Your cloud, your region**
Any Hightouch workspace can be deployed in a [supported region](/security/regions) on AWS, GCP, or Azure.
- **Least-privilege warehouse access**
- Hightouch requires limited `READ` access to your warehouse, following the principle of least privilege. You can tightly scope this access using native warehouse permissions.
- With the default **Basic sync engine**, change-data-capture runs on Hightouch infrastructure and only minimal metadata is stored in your configured bucket.
- If you enable the [**Lightning sync engine**](/syncs/lightning-sync-engine) (recommended for larger or high‑volume syncs), Hightouch also requires `WRITE` access for logging and advanced features. This access is isolated to a configurable Hightouch-managed location in your warehouse and never intersects with your own data.
- **Customer-owned storage by design**
Hightouch uses a cloud storage bucket as a caching layer for high-performance data movement and debugging. We strongly advocate that this is your bucket, in your cloud, where you maintain ownership and data retention configuration. See [Cloud storage](/security/storage) and [Bring your own bucket](#bring-your-own-bucket).
- **Private networking support**
When communicating with your warehouse and your cloud bucket, we support private connectivity across all major clouds ([AWS PrivateLink](/security/aws/privatelink), [Azure Private Link](/security/azure/private-link), [GCP Private Service Connect](/security/gcp/private-service-connect)) to ensure no communication between Hightouch and your data infrastructure is exposed to the public internet.
- **Encryption and regional data residency**
All customer data is encrypted in transit (TLS 1.2+). Data processed by Hightouch remains within the cloud and region where your workspace is deployed. Any outbound API traffic is encrypted.
- **Strong governance and access controls**
Hightouch [Workspace governance](/workspace-management/overview) supports SSO/SCIM, granular [RBAC and custom roles](/workspace-management/roles), label-based controls, approval workflows, and audit logs to ensure users only access the data and features appropriate to their role.
\* *This design applies to all core Hightouch products, including Reverse ETL, Events, and Customer Studio.
Certain features—such as [Real‑Time](/real-time/overview) and AI/ML features—may cache feature data, outcomes, or logs in the storage bucket configured for your workspace to meet latency and observability requirements. These artifacts are single‑tenant, can be stored in **your** cloud bucket, and are never used to train shared or third‑party models. See the [AI Addendum](https://hightouch.com/ai-addendum) and [AI Decisioning data handling](/ai-decisioning/security) for details.*
---
## Compliance
Hightouch maintains **SOC 2 Type 2 compliance** and is **ISO 27001 certified** by **Schellman**, an accredited certification body.
Hightouch supports customer compliance with **HIPAA**, **GDPR**, and **CCPA** requirements.
In addition to these certifications, we follow established security best practices, including:
- Automated vulnerability scanning
- Regular third-party penetration testing — please for the latest report
### SOC 2 Type 2 audit report
If you are an existing Hightouch customer, you can:
or ping us in your dedicated Slack channel to request our SOC 2 audit report.
If you are trialing Hightouch, your point of contact can provide you with the SOC 2 audit report under NDA.
---
## Architecture overview
At a high level, Hightouch follows a simple flow:
1. Connect securely to your warehouse or other [source](/getting-started/concepts#sources).
2. Define what data to pull from your source using [models](/getting-started/concepts#models).
3. Use [syncs](/getting-started/concepts#syncs) to map that data to [destinations](/getting-started/concepts#destinations) and control when it runs.
### Connecting to your data
Hightouch connects directly to your warehouse over TLS by default. For most customers, this direct connection is sufficient.
If your source is deployed within a private network or VPC, Hightouch also supports secure connectivity via:
- [SSH tunneling](/security/ssh-tunneling) or [reverse SSH tunneling](/security/ssh-tunneling#reverse-tunneling)
- Private networking such as [AWS PrivateLink](/security/aws/privatelink), [GCP Private Service Connect](/security/gcp/private-service-connect), or [Azure Private Link](/security/azure/private-link)
See [Networking overview](/security/networking) for architecture patterns and trade-offs.
A direct TLS connection is appropriate for most deployments. Private networking or tunneling is typically required when your warehouse is not publicly accessible.
### Defining data with models
After connecting your source, you create **models**.
Models are SQL definitions that Hightouch executes directly on your warehouse. They define what data to retrieve, but they do **not** store customer data inside Hightouch.
### Syncing to destinations
A **sync** determines:
- Which model to use
- How fields are mapped to a destination
- When and how frequently the sync runs
When a sync initiates, Hightouch:
1. Executes the model’s SQL query directly on your warehouse.
2. Computes incremental changes using our change data capture process (see [Change data capture architecture](#change-data-capture-architecture)).
3. Translates the resulting rows into the appropriate destination APIs.
### Data handling during execution
Customer data flows through Hightouch infrastructure **only during an active sync**.
During this process, data is:
- Encrypted in transit using [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security) 1.2 or higher
- Processed within the same cloud and region where your Hightouch workspace is deployed
- Not stored as primary customer tables within Hightouch infrastructure for standard Reverse ETL and CDP use cases
Limited operational logs and change-data-capture metadata are stored as described in [Data storage](#data-storage).
Hightouch does not expose its compute instances directly to the public internet. Infrastructure is secured according to cloud provider security best practices.
---
## Transit details
Hightouch uses [AWS Elastic Load Balancing](https://aws.amazon.com/elasticloadbalancing/) to distribute incoming traffic across healthy targets and increase application and network availability.
To protect data in transit, Hightouch enforces:
- **TLS 1.2 or later** on all HTTPS load balancers
- Disabling legacy SSL and older TLS versions with known vulnerabilities
- Alignment with the [AWS Well-Architected Framework](https://aws.amazon.com/architecture/well-architected/) principles for:
- Operational excellence
- Security
- Reliability
- Performance efficiency
- Cost optimization
We apply the following best practices for traffic between Hightouch, your data sources, and your destinations:
- **Encryption in transit everywhere** (TLS 1.2+)
- **Authenticated network communications** for all connections
- **Automated detection** of unintended data access
- **Private connectivity options** when required:
- SSH tunneling (standard and reverse)
- [AWS PrivateLink](/security/aws/privatelink)
- [GCP Private Service Connect](/security/gcp/private-service-connect)
- [Azure Private Link](/security/azure/private-link)
All outbound traffic to public SaaS APIs is also encrypted via TLS 1.2 or higher.
---
## Data storage
After sending data downstream, Hightouch persists **only the operational data needed for change‑data‑capture and debugging**. This includes request/response payloads, row‑level status, and diff state, stored in a cloud storage bucket. By default, this bucket is Hightouch‑managed for self‑service plans, but you can [use your own cloud bucket](#bring-your-own-bucket) so these artifacts live entirely in your infrastructure.
---
## Change data capture architecture
To prevent unnecessary API requests and limit the amount of data sent downstream, Hightouch uses [**change data capture (CDC)**](/syncs/cdc) or “diffing.”
For most enterprise and high‑volume workloads, Hightouch recommends the [**Lightning sync engine**](/syncs/lightning-sync-engine) for performance and to keep more state inside your warehouse.
There are two main approaches:
- The [**Lightning sync engine**](/syncs/lightning-sync-engine) computes diffs **inside your warehouse**.
- Hightouch requires `WRITE` access to specific tables in a dedicated internal schema (for example, `hightouch_planner`) to store metadata from previous syncs.
- This in‑warehouse computation typically enables faster syncs at higher volumes and keeps more state **inside your environment**.
- The **Basic sync engine** (also called “local diffing”) computes diffs **in Hightouch infrastructure**.
- It does not require `WRITE` access to your warehouse.
- It may have slightly slower sync speeds at very high volumes but can offload some compute from your warehouse.
- Diff metadata is stored alongside sync logs in your Hightouch‑managed or [BYO bucket](#bring-your-own-bucket).
In both cases, the **authoritative source of customer data remains your warehouse**; Hightouch stores only the minimal metadata needed to compute and apply changes efficiently.
---
## Bring your own bucket
Hightouch supports two main storage patterns for sync logs and diff metadata:
- **[Self‑service plans](/pricing/ss-pricing) (default):**
- Hightouch stores query results, diff metadata, and sync logs in an encrypted, Hightouch‑managed cloud bucket.
- A default short retention period (for example, 30 days) is applied to minimize data at rest while still enabling debugging.
- **[Business and enterprise plans](/pricing/bt-pricing) (recommended):**
- You can [host your own bucket](/security/storage) on **AWS**, **Google Cloud**, or **Azure**.
- Hightouch writes logs and diff metadata **directly into your storage**, so:
- Data stays within your own cloud account and residency regime.
- You control retention, lifecycle policies, and access management.
For many enterprise deployments, the recommended pattern is:
- **Lightning sync engine** for in‑warehouse diff computation, and
- **BYO bucket** for sync logs and any remaining observability data.
This combination ensures both compute and storage remain as close to your own infrastructure and controls as possible.
If you’d like to enable BYO bucket, please
.
---
## PII redaction and access control
Sometimes, you need to sync **personally identifiable information (PII)** to a destination, but you don't want your entire team to have access to it inside Hightouch.
For example:
- You may need to sync emails, phone numbers, or other identifiers to ad platforms for accurate targeting.
- You may want marketers to build [audiences](/customer-studio/overview) that rely on PII under the hood, but without exposing those raw values in the UI.
Hightouch supports a **defense‑in‑depth** approach for PII, combining workspace‑level governance with field‑level controls:
### Workspace governance and RBAC
Use **workspace governance** and **role‑based access control (RBAC)** to control who can see and act on sensitive data:
- Use [roles and custom roles](/workspace-management/roles) to limit who can:
- Edit sources and destinations
- View sync logs and row‑level data
- Access Customer Studio schema and traits
- Use SSO/SCIM and [group‑based access](/workspace-management/user-management) to centralize control over who can access your workspace and its features.
These controls ensure that only the right teams have access to areas of the product where PII may be visible (for example, logs and row‑level previews).
### Data masking and privacy levels in Customer Studio
For additional protection in **Customer Studio**, use [data masking](/customer-studio/data-masking) to control how sensitive columns appear across the platform while still using them for activation:
- When configuring your [schema](/customer-studio/schema#set-privacy-levels-for-columns) in Customer Studio, you can assign a **privacy level** to each column. Available levels include **Blocked**, **Redacted**, **Sync-only**, **Sync-blocked**, and **Approved**.
- For example, columns marked as **Redacted**:
- Can still be used as identifiers or filters behind the scenes
- Will not show raw values in audience previews or insights
- Hightouch also automatically flags potential PII columns for review, so you can classify them before marketers begin building audiences.
Together, RBAC, privacy levels, and audit logs help ensure that only the right users see sensitive data, and even then, only where it’s strictly necessary.
---
## Regions
**URL:** https://hightouch.com/docs/security/regions
**Description:** You can configure Hightouch to run in multiple regions around the world.
**Section:** Security and compliance
## Overview
Your Hightouch region defines where we host the Hightouch _data plane_, which is the component of our system that directly interacts with your [sources](/getting-started/concepts#sources) and [destinations](/getting-started/concepts#destinations).
## Supported regions
Hightouch operates in the following Amazon Web Services, Google Cloud Platform, and Azure regions:
| Cloud | Region | Location |
|-------|--------------|:---------------------|
| AWS | ap-south-1 | Mumbai, India |
| AWS | eu-west-1 | Ireland |
| AWS | us-east-1 | N. Virginia, USA |
| Azure | East US | N. Virginia, USA |
| GCP | europe-west1 | Belgium |
| GCP | me-central2 | Dammam, Saudi Arabia |
| GCP | us-east4 | N. Virginia, USA |
You may prefer to use a particular region for a variety of reasons, including data residency requirements, availability of features like [AWS PrivateLink](/security/aws/privatelink) or [Private Service Connect](/security/gcp/private-service-connect), or use of a self-managed storage bucket in Amazon S3, Google Cloud Storage.
## Setup
When creating a new workspace, you must select your desired region. Once a workspace has been created, its region can't be changed.
### Data in transit
During syncs, your data flows through Hightouch's data plane on its way to your destinations.
During the in-transit phase of a sync, your data remains within your workspace region. However, Hightouch can't verify where your destinations store your data. If your destination's data region is important to you, verify it with the destination directly.
### Data at rest
Hightouch only stores data needed to perform [change data capture](/getting-started/concepts#change-data-capture-cdc) and [debugging](/security/storage#observability-and-debugging). By default, Hightouch stores this data in an encrypted Amazon S3 bucket, Google Cloud Storage bucket, or Azure Storage account, depending on the cloud in which your workspace is created. Your Hightouch bucket, and all access to that bucket, is scoped to your region, ensuring compliance with data residency regulations.
Hightouch customers on the [Business tier](https://hightouch.com/pricing) can set up an [external storage bucket](/security/storage), granting more control over their data at rest. If you use an external storage bucket, Hightouch doesn't validate that the bucket is hosted in the same region as your Hightouch workspace.
---
## SSH Tunneling
**URL:** https://hightouch.com/docs/security/ssh-tunneling
**Description:** Hightouch supports self-service SSH tunneling, allowing you to securely connect to your data warehouse.
**Section:** Security and compliance
## Overview
Tunneling lets Hightouch securely open a connection to a data warehouse in your private network or Virtual Private Cloud (VPC) without exposing it to the internet. SSH tunnels are secure, authenticated, encrypted, and dedicated to your workspace. To learn more about SSH tunneling, check out [this introductory article](https://www.ssh.com/academy/ssh/tunneling).
This feature is supported for the following:
### Sources
- [Azure Synapse](https://hightouch.com/docs/sources/synapse)
- [ClickHouse](https://hightouch.com/docs/sources/clickhouse)
- [Datawarehouse.io](https://hightouch.com/docs/sources/datawarehouseio)
- [Greenplum](https://hightouch.com/docs/sources/greenplum)
- [Metabase](https://hightouch.com/docs/sources/metabase)
- [Mixpanel](https://hightouch.com/docs/sources/mixpanel)
- [MongoDB](https://hightouch.com/docs/sources/mongodb)
- [MySQL](https://hightouch.com/docs/sources/mysql)
- [Oracle DB](https://hightouch.com/docs/sources/oracle)
- [PostgreSQL](https://hightouch.com/docs/sources/postgresql)
- [Redshift](https://hightouch.com/docs/sources/amazon-redshift)
- [SingleStore](https://hightouch.com/docs/sources/single-store)
- [Snowflake](https://hightouch.com/docs/sources/snowflake)
- [Tableau](https://hightouch.com/docs/sources/tableau)
- [Teradata Vantage](https://hightouch.com/docs/sources/teradata-vantage)
- [Trino](https://hightouch.com/docs/sources/trino)
### Event Sources
- [Kafka](https://hightouch.com/docs/events/sources/kafka)
### Destinations
- [AlloyDB](https://hightouch.com/docs/destinations/alloydb)
- [Databricks](https://hightouch.com/docs/destinations/databricks)
- [Elasticsearch](https://hightouch.com/docs/destinations/elasticsearch)
- [MariaDB](https://hightouch.com/docs/destinations/mariadb)
- [MongoDB](https://hightouch.com/docs/destinations/mongodb)
- [MySQL](https://hightouch.com/docs/destinations/mysql)
- [OpenSearch](https://hightouch.com/docs/destinations/opensearch)
- [Oracle DB](https://hightouch.com/docs/destinations/oracle-db)
- [PostgreSQL](https://hightouch.com/docs/destinations/postgres)
- [Redis](https://hightouch.com/docs/destinations/redis)
- [Snowflake](https://hightouch.com/docs/destinations/snowflake)
### Event Destinations
- PostgreSQL
- Redshift
### Standard versus reverse tunneling
Both **Standard** and **Reverse** tunnels open a secure port connection between Hightouch and your data warehouse. However, they differ in implementation, and you may prefer one based on your network specifications.
[Standard tunnels](#standard-tunneling) require you to run `sshd` on a [bastion host](https://aws.amazon.com/quickstart/architecture/linux-bastion/) accessible from the public internet. Our systems open an SSH connection to your bastion, then open a port forwarding connection from your bastion to the private service you specify.
[Reverse tunnels](#reverse-tunneling) let you forward a port by connecting as a client to an SSH server managed by Hightouch. This removes the necessity for a bastion host in your infrastructure but requires you to maintain that connection. That means if your connection goes down for whatever reason, your systems automatically re-open the connection. You can use programs like [`autossh``](https://linux.die.net/man/1/autossh), [Docker container restart policies](https://docs.docker.com/config/containers/start-containers-automatically/), or process supervisors like [supervisord](http://supervisord.org/) to help maintain your connection.
## Standard tunneling
### Requirements
- Allowlist the Hightouch [static IP addresses](/security/networking#ip-addresses) corresponding to your region.
- Allow connections from the bastion host to your warehouse.
- Set up a user on the bastion host named `hightouch`.
### Setup
To get started, navigate to [**Settings** > **Tunnel**](https://app.hightouch.com/settings/tunnels?from=docs).
1. Select **Add Tunnel**.
1. Enter a **Name** for your tunnel.
1. Fill out the **SSH Host** and **SSH Port**.
- These are the connection details for your public-facing bastion server host.
- The port is most likely 22, standard for `sshd`.
1. Fill out the **Service Host** and **Service Port**.
- These are the connection details for the data warehouse you are connecting to Hightouch.
- Think of your bastion server as a [jump host](https://en.wikipedia.org/wiki/Jump_server). Hightouch jumps through it to connect to your warehouse using these details.
1. Select **Create**.
1. Copy or download the generated SSH public key. Add this to the `~/.ssh/authorized_keys` file for the `hightouch` user on your bastion server. You can use [`ssh-copy-id`](https://www.ssh.com/academy/ssh/copy-id) to help with this.
1. The tunnel **Status** turns green when the connection is established. Your tunnel is now ready for use.
### Troubleshooting
If you're having trouble establishing a connection with a standard tunnel, check the following:
- Check that you've allowlisted the [static Hightouch IP addresses](/security/networking#ip-addresses) on your bastion host. See [requirements](#requirements) for more information.
- Check that the `hightouch` user exists on your bastion host and that the Hightouch public key is in their `~/.ssh/authorized_keys` file.
- Check permissions on the `hightouch` user's SSH files.
- `~/.ssh` directory should be `0700`
- Fix: `chmod 0700 /home/hightouch/.ssh`
- `~/.ssh/authorized_keys` file should be `0644`
- Fix: `chmod 0644 /home/hightouch/.ssh/authorized_keys`
- `~/.ssh` directory and `~/.ssh/authorized_keys` file should be owned by the `hightouch` user
- Fix: `chown -R /home/hightouch/.ssh`
- Check that the bastion host can network to your warehouse.
- `nc -z $warehouse_host $warehouse_port`
- If all else fails, .
## Reverse tunneling
### Requirements
- You need a server within your VPC to act as the SSH client.
- The SSH client server must be able to connect to both the public internet and your warehouse.
- Make sure your Hightouch workspace is in a region where tunneling is supported:
- AWS ap-south-1
- AWS eu-west-1
- AWS us-east-1
- GCP europe-west1
- GCP us-east4
### Setup
To get started, navigate to [**Settings** > **Tunnel**](https://app.hightouch.com/settings/tunnels?from=docs).
1. Select **Add reverse tunnel**.
1. Enter a **Name** for your tunnel.
1. Select **Create**.
1. Download the private key to your machine.
{" "}
Hightouch doesn't store your private key. As a result, we can't recover a
lost key. Please make a local copy of this key and store it securely.
1. Copy the example `ssh` command and save it. You'll need to run it later.
- **Example**
```bash
ssh -i path/to/key.pem \
-R 0.0.0.0:$TUNNEL_PORT:$SERVICE_HOST:$SERVICE_PORT \
$TUNNEL_ID@tunnel..hightouch.io -p 22 \
-o ExitOnForwardFailure=yes
```
- This command includes the remote Hightouch `sshd` host and port, and remote forwarding port.
- Set or replace the `$SERVICE_HOST` and `$SERVICE_PORT` variables with your internal warehouse service host and port.
- Set or replace the `$TUNNEL_PORT` and `$TUNNEL_ID` variables with the tunnel port and ID provided by Hightouch upon creating a reverse tunnel
- `hightouch-region` will be the [region](#requirements-1) of your Hightouch workspace (e.g. `aws-us-east-1`).
1. Upload the private key to your SSH client server, store it securely, and ensure its permissions are set to `0400`.
1. From your SSH client server, run the modified `ssh` command.
- Ensure the `-i` flag is pointing to the correct path of the private key.
- You most likely want to wrap this `ssh` command with a process manager to restart in case of failure. Consider [`autossh`](https://linux.die.net/man/1/autossh).
1. The tunnel **Status** turns green when the connection is established. Your tunnel is now ready for use.
**Hightouch Reverse SSH Host Key**
If you prefer, you can check that Hightouch's hostname is correctly set to either `tunnel.aws-us-east-1.hightouch.com`, `tunnel.aws-eu-west-1.hightouch.com`, `tunnel.aws-ap-south-1.hightouch.com`, `tunnel.gcp-us-east4.production.hightouch.com`, or `tunnel.gcp-europe-west1.production.hightouch.com` depending on your workspace region and that the Host Key is correctly included in `~/.ssh/known_hosts` as follows:
```
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXjpH+c24rYGDBoUdHGN3Yazp4SrDU8Z0Oqb0WRaKJz0VpSPS+Kqmc0MioY3yK1aPDh32VSEFUykEHIobV7dsH9uKlR3OllTkxN9pIk3xfJBn1/x4TS2VX1I+XM7KE6xJH/QmBVaVyNzsYzZ+lR/hUdrVC3czV+ReJa5j+YS23atS3x9NznxHSBBOLcq0RpFj65/WZt0kJwbIqhrGYIq5qXR/8d7gzyCj6vg2iNNUszTFQvcFeQ8zeXC63ke4C9oCFHbneX2odvsDY5DQM6WF7H6j91qRhGEq+niQg2zCns1kJq1Gfa7bY3OxIHmQ3HP0R8ExJMsBBqdKoL5uNY8UK717yNuFzBQUV8fgJi1HZC6JR6uDVpLadbXUV3WD0LA0YQ1h24KKGtPRDrnyZxNTdn0R19asz31npyFHKrP95OIz2RoGuqsn9fr3FzO1wfXjdzYp5l6AAy04K6yPNQfTC/+LLk8U1PWbSsMBqlQnHkp/eMaSdTEVMVxqhAFKCz8M=
```
### Troubleshooting
If you're having trouble establishing a connection with a reverse tunnel, check the following:
- Check that your SSH client server is running and can access the public internet.
- Check that you've uploaded the private key and that it's only readable by the user initiating the SSH connection (`chmod 0400`).
- Add the `-v` (verbose) flag to your SSH command to see more detailed errors.
- If all else fails, . Please include any errors in the SSH output.
---
## Storage
**URL:** https://hightouch.com/docs/security/storage
**Description:** Learn more about what data Hightouch stores, whether with Hightouch or your own infrastructure, to power your syncs.
**Section:** Security and compliance
Hightouch needs to store some data to power your syncs and to power features like the [sync debugger](/syncs/debugger). You have full control over where this data is stored.
If you prefer more direct control or have specific security concerns, you can configure Hightouch to store this data in your own Amazon S3, Google Cloud Storage, or Azure Blob Storage account. If you prefer convenience, Hightouch can store this data in our own secure, encrypted infrastructure.
## What data does Hightouch store?
Hightouch stores data at-rest for two purposes:
1. [Change data capture](/getting-started/concepts#change-data-capture-cdc)
2. [In-app observability and debugging](/syncs/debugger)
Depending on your privacy and compliance needs, you can configure Hightouch to store **all** data-at-rest within your Virtual Private Cloud (VPC), or in a secure, encrypted bucket.
- If you're on the [Free, Starter, or Pro tier](https://hightouch.com/pricing), Hightouch hosts and manages a secure encrypted bucket on your behalf. For more information, see the [managed storage section](#managed-storage).
- If you're on the [Business tier](https://hightouch.com/pricing), you can self-host storage in your own infrastructure. For more information, see the [self-hosted storage section](#self-hosted-storage).
### Change data capture
To prevent making excessive API requests and send only necessary updates to your destinations, Hightouch uses a process called **change data capture** (CDC) or **diffing**.
In this process, Hightouch stores query results and execution plans after each sync run. When the next sync run occurs, Hightouch uses these diff files to determine incremental changes that should be sent downstream.
By default, Hightouch stores the diff files in a secure encrypted bucket hosted by Hightouch. [Business tier](https://hightouch.com/pricing) accounts can use the [Lightning sync engine](/syncs/lightning-sync-engine) to compute and stores diffs directly within their data warehouse.
To learn more about CDC, check out the [change data capture docs](/syncs/cdc).
### Observability and debugging
In addition to storing previous query results, Hightouch stores row-level log metadata including successes and failures, operations performed, and API request and response payloads. This data powers the [in-app debugger](/syncs/debugger) and can be stored either in your VPC or Hightouch's encrypted bucket.
## Managed storage
If you're on a [Free, Starter, or Pro plan](https://hightouch.com/pricing) Hightouch stores data-at-rest in a secure, encrypted, Hightouch-managed bucket. For workspaces running in a Hightouch AWS region, this is an Amazon S3 bucket. For workspaces running in a Hightouch Google Cloud region, this is a Google Cloud Storage bucket. No matter your region, all data in Hightouch managed buckets is encrypted at rest. If you require data-at-rest to live entirely in your VPC, see [self-hosted storage](/security/storage/#self-hosted-storage).
### Data retention
Data automatically expires from Hightouch-managed buckets after 30 days. If [change data capture](/syncs/cdc) is done in Hightouch-managed buckets, syncs that have not run in over 30 days will require a **Full Resync** or **Reset CDC** sync since Hightouch depends on diffing files to detect changes in the data model.
## Self-hosted storage
[Business tier](https://hightouch.com/pricing) customers can configure Hightouch to store all customer data-at-rest within their own external storage bucket or blob. Hightouch integrates with these cloud storage providers:
- Amazon S3
- Google Cloud Storage (GCS)
- Microsoft Azure Blob Storage
If you choose to self-host your storage, Hightouch only processes data-in-transit. You can select any supported storage provider to store your data, regardless of your [Hightouch region](/security/regions).
When hosting your own storage, Hightouch places full control over object lifecycle, security, and expiration into your hands.
We don't expire objects automatically or change your object encryption settings.
Ensure that you've configured object expiration, encryption, and access control settings according to your needs.
#### Recommended architecture and security practices
Use the following guidelines to reduce setup issues and simplify operations:
- Scope IAM permissions to the exact bucket/container prefix used by Hightouch. Avoid wildcard access to the entire bucket or storage account.
- Use a dedicated bucket/container, or a dedicated top-level prefix, per environment (for example `prod/`, `staging/`, and `dev/`) to avoid cross-environment access.
- Co-locate storage with your warehouse and destinations when possible to reduce latency and egress costs.
- Prefer private connectivity over public internet paths when available: [AWS PrivateLink](/security/aws/privatelink), [Azure Private Link](/security/azure/private-link), or [Google Private Service Connect](/security/gcp/private-service-connect). If using public endpoints, maintain strict IP allowlists.
- Use encryption at rest with your standard key-management policy (for example SSE-KMS, CMEK, or customer-managed keys) and ensure key policies grant your Hightouch principal the required access.
- Apply lifecycle rules that auto-expire objects according to your retention policy, and enable storage access logging when required for audits.
- After initial setup or credential rotation, run a small validation sync and confirm data is accessible in the [sync debugger](/syncs/debugger) before broad rollout.
Setting up self-hosted storage disrupts the [change data capture](/syncs/cdc) process for active syncs.
To reset it, after you've configured self-hosted storage, you need to [trigger a full resync](/syncs/overview#resync-full-query) or [reset cdc sync](/syncs/overview#reset-cdc) for all existing syncs that previously ran with [Hightouch-managed storage](#managed-storage).
Make sure that all your syncs satisfy the [full resync prerequisites](/syncs/overview#full-resync-prerequisites) before setting up self-hosted storage.
Don't hesitate to if you have any doubts or concerns.
Once you've run a sync after setting up a custom storage bucket, you can't
make further changes to your storage configuration self-serve, including
disabling it. This is because changing your storage configuration is
disruptive to active syncs and CDC state. If you need to change your bucket,
see [Changing your storage bucket](#changing-your-storage-bucket) below.
## Changing your storage bucket
If you haven't yet configured an external bucket, you can set one up self-serve by following the provider-specific instructions [above](#self-hosted-storage). The setup is self-serve, but can be disruptive to active syncs — review the [CDC impact](#self-hosted-storage) and consider reaching out to support for monitoring if you have a large number of syncs.
Once syncs have run with an external bucket configured, changing your storage configuration — for example, moving to a new AWS account, switching regions, or pointing to a different bucket — is a **support-assisted operation**. There is no self-serve way to change your bucket configuration after syncs have run.
to start the process. To help the team plan your migration efficiently, include
the following information in your request:
**Scope:**
- Workspace name(s) and URL(s) affected
- Whether the workspace is production or non-production
**New bucket details:**
- Cloud provider and region (for example, `AWS us-east-1`)
- Bucket or container name (for example, `my-company-hightouch-prod`)
- Optional prefix, if you want Hightouch to write to a specific path within the bucket
**Access credentials:**
- For AWS: the IAM role ARN (and external ID, if required) with the [permissions listed above](#authenticate-hightouch-with-aws)
- For GCS: the service account email and confirmation that it has the required permissions (`storage.objects.list`, `storage.objects.create`, `storage.objects.get`, `storage.objects.delete`, `storage.buckets.get`)
- For Azure: the storage principal and scope
- If you're using prefix-scoped IAM policies, include the exact prefix so the team can confirm compatibility
**Feature usage:**
- Whether you're using the [Lightning sync engine](/syncs/lightning-sync-engine), [AI Decisioning](/ai-decisioning/overview), or other features that store artifacts in the bucket
- Whether you're using **Snowflake-integrated features** such as **Snowflake Warehouse Export** — these rely on Snowflake external volumes and stages that also need to be updated to point at the new bucket
- Any lifecycle policies on the old or new bucket that automatically delete objects — particularly planner-state files, which can trigger unexpected full resyncs if expired
**Maintenance window:**
- A preferred time window (ideally during low traffic) when syncs can be paused and the configuration updated
### What to expect
Hightouch will update your workspace's storage configuration to point to the new bucket and validate connectivity. The impact on your syncs depends on the sync engine:
- **Lightning sync engine:** CDC state is stored primarily in your data warehouse, not in the bucket. Changing buckets generally does not require re-seeding CDC, but historical debugger logs from the old bucket won't carry over to the new one.
- **Standard sync engine:** CDC relies on snapshots and diff files stored in the bucket. After a bucket change, each affected sync will need a one-time [full resync](/syncs/overview#resync-full-query) or [CDC reset](/syncs/overview#reset-cdc) to establish a clean baseline in the new bucket. Plan for higher warehouse and API load during these initial runs.
- **AI Decisioning:** The bucket stores long-lived ML artifacts and interaction logs. Migrating may require copying these artifacts to the new bucket, or AI Decisioning will rebuild state from your warehouse over time. Call this out in your support request so the team can plan accordingly.
- **Snowflake-integrated features (Snowflake Warehouse Export and others):** If you use features that rely on Snowflake external volumes or stages, the Hightouch team will also update the associated storage integrations to point at the new bucket. Make sure the new bucket's cloud and region are compatible with your Snowflake account.
After the configuration change, run a small test sync and verify that new artifacts and logs appear in the new bucket before resuming all workloads.
### Amazon S3
Before getting started, [connect Hightouch with your Amazon Web Services account](/security/aws).
#### Create your S3 bucket
In Amazon S3, [create your bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html). We recommend the name `-hightouch`.
**Make sure to:**
- Block all public access to the bucket.
- Enable Amazon S3 key encryption \(SSE-S3\). If using [SSE-KMS](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html) for encryption, you may need to [update your IAM policies](https://repost.aws/knowledge-center/cross-account-access-denied-error-s3) to grant Hightouch access.
- Prefer bucket policies with IAM condition scoping over ACL-based access controls.
- Disable bucket versioning.
- Configure your [bucket object lifecycle](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html), to enhance security and cut down on costs.
#### Authenticate Hightouch with AWS
Hightouch supports authenticating with AWS using Cross-account roles (via STS AssumeRole), or with an Access Key ID / Secret Access Key that you provide. We strongly encourage you to use Cross-account roles, as it doesn't require Hightouch to hold any of your secrets.
To set up your Hightouch AWS credential, follow our [connection instructions](/security/aws).
Hightouch needs the following IAM actions to store and retrieve items from your bucket:
| Action | Details |
| :--------------------- | :------------------------------------------------------------------------------------ |
| `s3:GetObject` | Grants permission to retrieve objects from Amazon S3 |
| `s3:GetObjectVersion` | Grants permission to read versioned objects when applicable |
| `s3:PutObject` | Grants permission to add an object to a bucket |
| `s3:ListBucket` | Grants permission to list some or all the objects in an Amazon S3 bucket (up to 1000) |
| `s3:GetBucketLocation` | Grants permission to read bucket region metadata |
| `s3:DeleteObject` | Grants permission to delete objects from an Amazon S3 bucket |
You can use the following JSON sample to create a prefix-scoped IAM policy:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ListBucketOnPrefix",
"Effect": "Allow",
"Action": ["s3:ListBucket", "s3:GetBucketLocation"],
"Resource": ["arn:aws:s3:::${bucketName}"],
"Condition": {
"StringLike": {
"s3:prefix": ["${prefix}/*"]
}
}
},
{
"Sid": "ObjectAccessOnPrefix",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:GetObjectVersion",
"s3:PutObject",
"s3:DeleteObject"
],
"Resource": ["arn:aws:s3:::${bucketName}/${prefix}/*"]
}
]
}
```
#### Configure your bucket in Hightouch
1. In Hightouch, on the [**Storage** tab](https://app.hightouch.com/settings/storage?from=docs) of the **Settings** page, select **Amazon S3** as the **Cloud provider**.
2. Select your **AWS region**, enter your **Bucket name**, and select the **AWS credentials** you previously set up.
3. Click **Save**.
Once you save your settings, your new syncs automatically start using your bucket. [Run a few syncs](/syncs/create-your-first-sync) and visit your S3 bucket to check files are saving there.
Don't hesitate to if you have any questions.
#### Common errors
If you receive a `CredentialsError: Missing credentials in configuration` error, refer to the [troubleshooting tips](/security/aws#credentialserror-missing-credentials-in-configuration) in the AWS integration docs.
### Google Cloud Storage
Before getting started, [connect Hightouch with your Google Cloud account](/security/gcp).
#### Create a bucket
In the Google Cloud console, [create a new bucket](https://cloud.google.com/storage/docs/creating-buckets). We recommend the name `-hightouch-bucket`. Copy the bucket name and save it for later.
Configure your [bucket object lifecycle](https://cloud.google.com/storage/docs/lifecycle), to enhance security and cut down on costs.
**Make sure to:**
- Enable [Uniform bucket-level access](https://cloud.google.com/storage/docs/uniform-bucket-level-access) and manage access through IAM policies instead of object ACLs.
- Use CMEK where required by your compliance controls, and ensure your service account has access to the key.
- If you use [VPC Service Controls](https://cloud.google.com/vpc-service-controls/docs/overview), include the bucket and service account in the same perimeter to avoid access denials.
#### Authenticate with Google Cloud
Hightouch supports authenticating with GCP using Hightouch-managed service accounts, or by using a service account that you control.
To set up your Hightouch GCP credential, follow our [connection instructions](/security/gcp).
Hightouch needs the following IAM permissions to store and retrieve items from your bucket:
| Permission | Details |
| :----------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `storage.objects.list` | Grants access to view objects and their metadata, excluding ACLs. Can also list the objects in a bucket. |
| `storage.objects.create` | Grants permission to create, replace, and delete objects; list objects in a bucket; read object metadata when listing (excluding IAM policies); and read bucket metadata, excluding IAM policies. |
| `storage.objects.get` | Grants access to view objects and their metadata, excluding ACLs. Can also list the objects in a bucket. |
| `storage.objects.delete` | Grants permission to overwrite existing objects. |
#### Configure your bucket in Hightouch
1. In Hightouch, on the [**Storage** tab](https://app.hightouch.com/settings/storage?from=docs) of the **Settings** page, select **Google Cloud Storage** as the **Cloud provider**.
2. Enter the **Project ID** and **Bucket name** and select the **Google Cloud credentials** you previously set up.
3. Click **Save**.
Once you save your settings, your new syncs automatically start using your bucket. [Run a few syncs](/syncs/create-your-first-sync) and visit your Google Cloud bucket to check files are saving there.
Don't hesitate to if you have any questions.
### Microsoft Azure Blob Storage
Before getting started, [connect Hightouch with your Azure account](/security/azure).
#### Create a container
In your Azure portal, [create a container](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-portal#create-a-container). We recommend the name `-hightouch-container`. Copy the container name and save it for later.
Configure your [storage lifecycle](https://learn.microsoft.com/en-us/azure/storage/blobs/lifecycle-management-overview), to enhance security and cut down on costs.
#### Authenticate with Azure
Grant the app two roles: the **Storage Blob Data Contributor** role for the storage account (or container), and the **Storage Blob Delegator** role at the storage-account level. The delegator role is required for file downloads and previews (such as Data Agents and AI Decisioning CSV exports), which Hightouch serves through signed links — without it, uploads succeed but downloads fail with `AuthorizationPermissionMismatch`.
Use a Service Principal with RBAC over long-lived SAS tokens when possible, and rotate client secrets or certificates on your normal security cadence.
If you want to create a custom role with more granular permissions, Hightouch needs the following IAM permissions to store and retrieve items from your Blob Storage **Container**:
- `"actions"."Microsoft.Storage/storageAccounts/blobServices/containers/read"`
- `"dataActions"."Microsoft.Storage/storageAccounts/blobServices/containers/blobs/add/action"`
- `"dataActions"."Microsoft.Storage/storageAccounts/blobServices/containers/blobs/write"`
- `"dataActions"."Microsoft.Storage/storageAccounts/blobServices/containers/blobs/delete"`
Also grant the following on the storage **Account** (required for file downloads and previews):
- `"actions"."Microsoft.Storage/storageAccounts/blobServices/generateUserDelegationKey/action"`
If you're using ADLS Gen2 (hierarchical namespace enabled), ensure list operations target stable folder prefixes (for example with trailing `/`) and validate that your processing handles directory entries correctly.
#### Configure your container in Hightouch
1. In Hightouch, on the [**Storage** tab](https://app.hightouch.com/settings/storage?from=docs) of the **Settings** page, select **Microsoft Azure** as the **Cloud provider**.
2. Enter the **Account name** and **Container name** from the Azure portal (see [connecting Hightouch with your Azure account](/security/azure)), and select the **Azure credentials** you previously set up.
3. Click **Save**.
Once you save your settings, your new syncs automatically start using your container. [Run a few syncs](/syncs/create-your-first-sync) and visit your Blob Storage container to check files are saving there.
Don't hesitate to if you have any questions.
#### Common errors
Your Azure storage configuration may fail to save for a few reasons:
- `invalid_client`: Check that you've provided your client secret **value**, not client secret ID. See the [configuration docs](/security/azure#create-an-app-registration) for more information.
- `This request is not authorized to perform this operation`: Check that Hightouch has [adequate access](#authenticate-with-azure) to your storage account. If you've configured your storage account's **Network access** to be **from selected virtual networks and IP addresses**, be sure to add Hightouch's [IP addresses](/security/networking#ip-addresses) to your [network rules](https://learn.microsoft.com/en-us/azure/storage/common/storage-network-security?tabs=azure-portal).
---
## Real-Time Personalization overview
**URL:** https://hightouch.com/docs/real-time/overview
**Description:** Use live session behavior and warehouse context to personalize website and app experiences in less than a second.
**Section:** Hightouch Real-time
| Audience | How you'll use this article |
| --------------- | ---------------------------------------------------------------------------------------------------------------- |
| **Marketing** | Understand what Real-Time Personalization does, when to use it, and how it connects to audiences and activation. |
| **Data teams** | Learn how realtime events, warehouse context, and audience evaluation work together. |
| **Engineering** | Understand event collection, evaluation architecture, and Personalization API integration options. |
_Real-Time Personalization combines live session behavior with customer data from your warehouse so you can tailor website and app experiences as people browse, search, and convert — typically in less than a second._
---
## What you'll learn
- [What Real-Time Personalization does](#overview)
- [Core features: realtime audiences, realtime syncs, and Personalization API](#core-features)
- [How to get started](#get-started)
- [When to use Real-Time Personalization](#when-to-use-real-time-personalization)
- [How Real-Time Personalization connects to other Hightouch products](#how-real-time-personalization-fits-into-hightouch)
---
## Overview
Real-Time Personalization lets you respond to what someone is doing on your website or app while they're still active. Instead of waiting for a scheduled audience refresh, you can update audience membership, change on-site content, or trigger downstream actions based on live session behavior.
It works by combining two types of information:
- **What the person is doing right now** — live events like page views, searches, clicks, and cart actions, streamed from [Hightouch Events](/events/overview) or another supported event source.
- **What you already know about them** — customer attributes from your warehouse, such as loyalty tier, past purchases, or predicted scores.
When an event arrives, Hightouch evaluates whether the person qualifies for a realtime audience based on rules you define — for example, "viewed the pricing page 3 times in the last 10 minutes and is on the free plan." If they qualify, you can deliver a personalized experience immediately through a realtime sync or the Personalization API.
This shifts the question from "who qualified for this audience today?" to "does this person qualify right now?"
---
## Core features
### Realtime audiences
Realtime audiences evaluate membership using live session events combined with warehouse attributes. You build them in [Customer Studio](/customer-studio/overview) using the same audience builder you use for standard audiences, but you add conditions that reference a realtime event model. When you turn on **Realtime evaluation**, Hightouch evaluates the audience continuously as events arrive instead of on a schedule.
For example, you can build a realtime audience that includes people who searched for a product during their current session and have a lifetime value above a certain threshold. When someone meets those conditions, they enter the audience immediately. When they no longer meet them, they exit — including when a time-based condition expires, so users age out of the audience once the qualifying behavior falls outside its lookback window.
When an audience combines realtime events with warehouse-backed conditions, Hightouch evaluates the warehouse side against a cache that refreshes on a schedule you control. You can see the cache's freshness on the audience and trigger an on-demand refresh when needed. See [Refresh warehouse-backed conditions](/real-time/setup#refresh-warehouse-backed-conditions).
Realtime audiences still use your existing parent models and schema — the difference is how and when evaluation happens.
### Realtime syncs
Realtime syncs deliver audience membership changes to supported destinations as they happen. When a person enters or exits a realtime audience, Hightouch sends that update to the destination immediately. You can [choose which membership changes to forward](/real-time/setup#configure-events) — entry events, exit events, or both — depending on what the downstream system needs.
You can use realtime syncs to trigger messages, suppress promotions after a conversion, or update campaign targeting without waiting for a batch sync to run.
### Audience observability
Each realtime audience has an **Overview** tab you can use to verify that events are flowing, the audience is evaluating correctly, and error rates are within acceptable bounds. Check here first when a realtime audience isn't behaving as expected — a spike in errors or a flat entrance rate usually points to an event instrumentation issue or a misconfigured audience condition.
The overview also includes an **Entrance rate** chart (percentage of events that result in audience entrances) and an **Errors** chart, which can help you distinguish between low event volume and a broken evaluation rule.
For details on the audience overview experience — including membership tracking, size history, and related configuration — see [Audience overviews](/customer-studio/audience-overviews).
### Personalization API
The [Personalization API](/real-time/personalization-api) is a low-latency API that serves warehouse-backed customer data to your website, app, or backend services. You define the dataset with a model and sync, and Hightouch maintains a managed cache that your application can query over HTTP.
Use the Personalization API to retrieve attributes like product recommendations, customer scores, or profile data, then combine that with realtime audience membership to decide what experience to show. For example, you might check whether someone is in a high-intent pricing audience, then fetch plan details and a recommended upgrade path to render a targeted offer.
---
## Get started
Setting up Real-Time Personalization usually involves four steps: sending live events, defining realtime events in your schema, building an audience with realtime conditions, and choosing how to activate it.
| Step | What to do | Who typically does this |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------ |
| 1. Connect an event source | Set up [Hightouch Events](/events/overview) or connect a supported streaming source to send live events to Hightouch. | **Engineering team** for SDK instrumentation, or **data or engineering team** for an existing streaming source |
| 2. Define realtime events | In your [Customer Studio schema](/customer-studio/schema), create a realtime event model that tells Hightouch which events to evaluate, which fields to expose, and how the event maps back to a user. | **Data or engineering team** |
| 3. Build a realtime audience | Create an audience that includes conditions referencing your realtime event model, then turn on **Realtime evaluation**. | **Marketing, lifecycle, or data team** |
| 4. Activate | Create a [realtime sync](/real-time/setup#option-1-realtime-syncs) to push membership changes to a destination, or connect the [Personalization API](/real-time/personalization-api) for website or app use cases. | **Marketing ops, lifecycle, or data team** for realtime syncs; **engineering team** for Personalization API integrations |
For the full walkthrough, see [Set up realtime audiences](/real-time/setup).
Realtime audiences do not currently support traits, experiments, or journeys.
See [Constraints and
limitations](/real-time/architecture#constraints-and-limitations) for the full
list.
---
## When to use Real-Time Personalization
Use Real-Time Personalization when you need to respond to user behavior during an active session. For scheduled campaign targeting or batch audience updates, standard audiences are usually simpler.
| Scenario | Recommended approach |
| --------------------------------------------------------- | ----------------------------------------------------- |
| Daily or hourly campaign refresh | Standard batch audiences |
| Event-triggered messaging, alerts, or operational actions | [Event streaming](/events/event-streaming) |
| Personalization during an active session | **Real-Time Personalization** |
| Serving warehouse data to your app or website | [Personalization API](/real-time/personalization-api) |
### Common use cases
- **Suppress promotions after conversion** — Remove someone from a promotional audience within seconds of a purchase, so they stop seeing the offer during the same session.
- **Personalize based on browsing behavior** — Show product recommendations or content based on what someone is searching for or viewing right now.
- **Trigger follow-ups from in-session actions** — Send a push notification or update a downstream system when someone abandons checkout or revisits a pricing page.
- **Combine session context with warehouse data** — Build audiences that mix live behavior with historical attributes for more precise targeting.
---
## How Real-Time Personalization fits into Hightouch
Real-Time Personalization is the activation layer for sub-second digital experiences. It uses live events and warehouse context to evaluate audience membership and deliver personalized experiences while a person is still active.
#### What Real-Time Personalization uses
- **Behavioral signals from [Events](/events/overview)** — Real-Time Personalization uses live events such as page views, searches, clicks, and cart actions to understand what someone is doing right now.
- **Customer context from your [connected source](/sources/overview)** — It combines those live signals with customer data from your warehouse, such as loyalty status, past purchases, preferences, and predicted scores.
- **Audiences from [Customer Studio](/customer-studio/overview)** — You define realtime audiences using the same audience builder and schema you use for standard audiences, with the addition of realtime event conditions.
#### What Real-Time Personalization sends
- **Personalized website and app experiences** — Use realtime audience membership and the Personalization API together to tailor on-site content, offers, and recommendations based on what a person is doing right now and what you already know about them.
- **Audience membership updates to [destinations](/destinations/overview)** — Realtime syncs send audience entry and exit updates to downstream tools for fast activation across channels.
- **Live decision inputs to [AI Decisioning](/ai-decisioning/overview)** — Real-time behavioral signals can help determine which experience or message is most relevant at a given moment.
### Real-Time Personalization vs. Customer Studio
[Customer Studio](/customer-studio/overview) and Real-Time Personalization work together, but they handle different parts of the problem:
- **Customer Studio** helps you define audiences and orchestration logic using your warehouse data. Standard audiences update on a schedule.
- **Real-Time Personalization** evaluates audience membership from live session events in less than a second, so you can act on behavior while someone is still active.
You can use them together: build your audience logic in Customer Studio, then turn on Realtime evaluation to deliver responsive experiences when someone qualifies during their session.
---
## Permissions
Real-Time Personalization does not have its own tab in the custom role builder. Realtime audiences use the standard audience and model permissions from Customer Studio, and realtime audience syncs inherit permissions from the associated source and destination grants. To control access, configure grants in the **Customer Studio**, **Sources**, and **Destinations** tabs. See [Roles](/workspace-management/roles#customer-studio) for details.
---
## Personalization API
**URL:** https://hightouch.com/docs/real-time/personalization-api
**Description:** A fast, flexible API to tailor customer experiences across web, mobile, and email.
**Section:** Hightouch Real-time
This feature is available as an add-on to [Business tier plans](https://hightouch.com/pricing). To get access and learn more about implementation, please .
## Overview
Hightouch Personalization API is a fully managed service that combines the analytical power of the data warehouse with the real-time performance of a low-latency API. You can think of this feature as a cache for your data warehouse.
With the Personalization API, you can serve any data model—customers, products, transactions, recommendations, etc.—to any internal app, web experience, or email marketing platform. Define your dataset with SQL and Hightouch handles the rest.
Example use cases include:
- **Powering personalized marketing campaigns** by enriching marketing touchpoints with dynamic product recommendations
- **Delivering in-app or web personalization** like customized search results, cohort-based pricing and messaging, article recommendations, or nearest store locations
- **Optimizing product experimentation** by delivering up-to-date customer information like audience inclusion/exclusion to decisioning and experimentation systems
---
## Architecture
Under the hood, the Personalization API is backed by a distributed, low-latency database that stores a cache of your model. This cache updates as frequently as you see fit with Hightouch's [scheduling options](/syncs/schedule-sync-ui). Hightouch then serves this cache over HTTP so that it can be queried by your SaaS tools and other services.
Our benchmarks show p99 response times under 30 milliseconds in the same region. The Personalization API can support peak throughput of 1 million requests per second. To obtain the most recent benchmarks for your region, please .
---
## Implementation steps
Using this feature is a three-step process:
1. Enabling the Personalization API in the regions closest to your own infrastructure
2. Creating syncs to define which models should be materialized in the Hightouch-managed cache
3. Using your HTTP endpoints to access data
---
## Getting started
After the Personalization API is enabled for your workspace, navigate to the newly created destination and configure the region from which the API should be served.
Currently, the following AWS regions are available for self-serve deployment:
- `us-east-1`
- `us-west-2`
- `eu-west-1`
- `ap-southeast-2`
For other regions, please to complete setup.
---
## Sync configuration
There are two core concepts in the Personalization API: collections and records.
- **Collections** are conceptually similar to database tables. Each should represent a particular type of data, such as customers or invoices. Collection names must be alphanumeric and will become part of your Personalization API endpoint.
- **Records** are conceptually similar to rows in a database. Each should represent a single item, like a customer profile. Collections consist of one or more records.
You can create multiple syncs to the same collection provided that each sync updates different record fields.
For example, you might have one model representing basic customer demographic data—`age`, `gender`, `location`, etc.—and another model representing account usage data—`last_login_at`, `num_messages_sent`, etc. By creating two syncs to the same `customers` collection, you can create more complex records that contain all attributes necessary for personalization.
Next, specify which column from your model should serve as the primary index for record lookups. This field must uniquely identify each record in the collection and is often the same as your model's primary key.
The Personalization API supports lookups on multiple indices. For example, you might want to retrieve customer profiles using `user_id`, `anonymous_id`, or `email_address`. To enable multiple indices, please .
Use the field mapper to specify which columns from your model should be included in the API response payload. You can rename these fields and use the advanced mapper to apply transformations using the [Liquid template language](https://shopify.github.io/liquid/).
### Delete behavior
You have the following options for how Hightouch should handle rows leaving your query results:
| Behavior | Description |
| ----------------------------------------------------------------------------- | -------------------------------------------------------------------- |
| **Do nothing** | All cached data is preserved indefinitely. This option is *strongly* recommended against. |
| **Clear the fields managed by this sync but do not delete the entire record** | Fields managed by other syncs are preserved. |
| **Delete the entire record** | Requests for deleted records receive `404 Not Found` responses. |
You probably want to delete the entire record if your collection is managed using only one sync. However, if multiple syncs are writing to the same collection, it's best to clear only the fields updated by this sync.
---
## Access data
To start using the API, you must first create an API key for your workspace.
### Create an API key
### Make API requests
Once you have an API key, you can make requests to the Personalization API to retrieve data. The API is accessible at `https://personalization.{region}.hightouch.com`, for example, `https://personalization.us-west-2.hightouch.com`.
#### Authentication
You can specify your API key via a bearer token in an HTTP Authorization header.
```
Authorization: Bearer XXXXX
```
#### Usage
The API has a single endpoint:
| Verb | Path |
| ---- | ------------------------------------|
| GET | /v1/collections/`:collection_name`/records/id/`:index_value` |
Refer to your [sync configuration](#sync-configuration) for your `collection_name`.
A request looks like this:
```bash
curl -X GET \
-H "Authorization: Bearer 55555555-4444-3333-2222-111111111111" \
"https://personalization.us-west-2.hightouch.com/v1/collections/customers/records/id/12345"
```
When the query is successful, the HTTP response is a `200 OK` and returns the record specified by the index value, in this case the customer with `user_id` (the index key) equal to `12345` (the index value). The data in the response payload is entirely flexible and defined by your [sync configuration](#sync-configuration).
For example:
```json
{
"user_id": 12345,
"full_name": "Jane Doe",
"lifetime_value": 1492.18,
"churn_risk": 0.04,
"90_day_summary": {
"num_songs_listened": 813,
"top_genres": [
"house",
"techno",
"ambient"
],
"top_artists": [
"deadmau5",
"Marsh",
"Enamour"
],
},
"recommendations": {
"concerts": [
{
"artist": "Aphex Twin",
"location": "San Francisco, CA",
"event_date": "2023-01-31"
},
{
"artist": "Sultan + Shepard",
"location": "San Francisco, CA",
"event_date": "2023-02-25"
}
],
"upcoming_album_release": {
"title": "Universal Language",
"artist": "Simon Doty",
"label": "Anjunadeep",
"release_date": "2023-04-28"
}
},
"_audiences": {
"top_concert_ticket_purchasers": true,
"attended_concert_last_month": false,
"electronic_music_listeners": true
}
}
```
---
## Tips and troubleshooting
### Unique primary key requirement
The model you select for your sync configuration _must_ have a unique identifier as its [primary key](/models/creating-models#unique-primary-key-requirement).
### Recommended usage
The Personalization API should not be called directly from a webpage, mobile app, or any untrusted client. We recommend proxying API calls through a service that you own, such as your application backend. This allows you to enforce reasonable rate limits and ensure that clients cannot retrieve data belonging to other users. Keep in mind that authentication is performed using a Hightouch API key, which grants access to all records in your Personalization API collection.
### Live debugger
### Sync alerts
### Merging records
When [configuring multiple syncs to update the same collection](#sync-configuration),
there are some common errors and misconfigurations to be aware of.
In general, it is simpler and less error-prone to ensure that different syncs do not attempt to manage
the same fields.
#### Merging records with the same field names
If two syncs both update the _same field_ on the record, it will of course take whatever value is written last.
This may have unexpected results if a record is _removed_ from one sync, but not another, as the behavior
is defined by whatever [deletion behavior](#delete-behavior) is specified for the sync that deletes the record.
For example, the record may be deleted, or if two syncs write to the same field name, the field may be cleared
and thus have no value at all until another sync updates the record.
#### Merging records with nested fields
When merging records, e.g. writing an `address` field to a `customer` record created by another sync, the
merging operates simply by replacing the `address` value with the new one. There is no special logic to
merge property _values_, or nested values. For example, if one sync creates a customer record such as
`{ id: 1, name: "John Doe", address: { country: "US" } }`, and another sync updates it with
`{ id: 1, age: 33, address: { zip: 86023 } }`,
the resulting record will be `{ id: 1, age: 33, name: "John Doe", address: { zip: 86023 } }`: it merges fields
from both syncs (`name`, `age`), but the value of the `address` field, originally `{ country: "US" }`, was
overwritten with the new value `{ zip: 86023 }`. It is not possible in the Personalization API to perform
a deep merge, setting the field to `{ country: "US", zip: 86023 }`. Instead, we recommend setting the
entire `address` field using a single sync.
---
## Set up realtime audiences
**URL:** https://hightouch.com/docs/real-time/setup
**Description:** Create audiences that evaluate live user behavior using realtime events and deliver membership updates instantly.
**Section:** Hightouch Real-time
| | |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers, platform admins |
| **Prerequisites** |
An event source sending events to Hightouch (Hightouch Events, Segment, RudderStack, or Snowplow)
A destination that supports realtime syncing, or a system that will call the Membership API
Realtime sync capability enabled for your workspace
|
_Realtime audiences evaluate user behavior during the current session using streaming events and warehouse context._
---
## What you'll learn
After reading this article, you'll know how to:
- Configure realtime events in your Customer Studio schema
- Build an audience that evaluates live user behavior
- Deliver audience membership through realtime syncs or the Membership API
- Choose the right delivery pattern for your personalization or campaign workflow
---
## Overview
[Realtime audiences](/real-time/overview) evaluate audience membership from live behavioral events during the current session, combining streaming events with warehouse attributes.
This guide walks you through the four steps to create one:
1. Connect an event stream
2. Define a **realtime event** in your schema
3. Build an audience that references that event
4. Configure how audience membership is delivered
---
## Step 1: Connect an event stream
Hightouch must receive realtime events before audiences can be evaluated.
Supported event sources include:
- Hightouch Events
- Snowplow
- RudderStack
- Segment
These events flow through Hightouch’s event pipeline and can be used for realtime audience evaluation once a corresponding **realtime event model** is defined in your schema.
See:
- [Set up Hightouch Events](/events/event-streaming#setup)
- [Migrate to Hightouch Events](/events/migration-guide/overview)
---
## Step 2: Create a realtime event
Realtime events define which incoming events should be available for realtime audience evaluation.
To create one:
1. Go to **Customer Studio → Schema**
2. Locate or create a [**parent model**](/customer-studio/schema#define-the-parent-model) (usually `Users`)
3. Click the **plus (+)** icon next to the parent model
4. Select **Create a related realtime event**
---
### Configure the event source
In the **Select event source** screen:
- Choose the **event source** (for example Segment or Hightouch Events)
- Enter the **event type** (for example `track`, `page`, or `order_completed`)
- Define event properties by adding columns and optional aliases
- Select the **timestamp column**, which determines when the event occurred
Click **Continue**.
---
### Configure the event model
Next, configure how the event relates to your schema.
Provide:
| Field | Description |
| -------------------- | -------------------------------------------------------------------- |
| **Name** | Display name for the event in Customer Studio |
| **Description** | Optional explanation of the event |
| **Timestamp column** | Used for time-based audience filters |
| **Foreign key** | Column linking the event to the parent model (for example `user_id`) |
Click **Create realtime event**.
Once created, the realtime event appears as an event filter option in the
audience builder.
---
## Step 3: Create an audience
Next, create an audience that references the realtime event.
1. Go to **Customer Studio → [Audiences](https://app.hightouch.com/audiences?from=docs)**
2. Create a new audience or open an existing one.
3. Select the **parent model** connected to your realtime event.
Any existing audience can become a realtime audience once it includes a
realtime event condition.
---
## Step 4: Add a realtime event condition
Add a filter that evaluates the realtime event.
1. In the audience builder, click **Add filter**
2. Select **Event** and choose your realtime event model
3. Define the event condition—for example, set it to "performed within the last 5 minutes" to capture recent activity
4. Click **Continue**
5. Finalize audience settings and click **Finish**
Time-based conditions expire automatically. When a user no longer meets a
lookback-window condition—for example, "viewed a page 3 times in the last 4
hours"—Hightouch removes them from the audience even if they generate no
further events. This produces an audience exit, which is sent downstream if
[exit events](#configure-events) are enabled on the sync.
---
## Step 5: Choose a delivery method
Realtime audiences can deliver membership updates through two mechanisms.
| Method | Best for | Description |
| ------------------ | ------------------------------------------- | ------------------------------------------------------------ |
| **Realtime syncs** | Campaign tools, CRMs, advertising platforms | Push audience entry or exit events to supported destinations |
| **Membership API** | Websites, apps, personalization engines | Query audience membership on demand |
### Option 1: Realtime syncs
Realtime syncs send audience membership changes to supported destinations.
When a user enters or exits an audience, Hightouch forwards a membership event to the destination.
Example event:
`User 123 entered audience "High-intent pricing visitors"`
#### Create a realtime sync
1. Open your audience and go to the **Syncs** tab
2. Click **Add Sync**
3. Select a destination that **supports realtime syncing**
Supported destinations include:
- [Braze](/destinations/braze)
- [Iterable](/destinations/iterable)
- [Meta Custom Audiences](/destinations/facebook)
- [Google Ads Customer Match](/destinations/google)
- [Segment](/destinations/segment)
- [Amplitude](/destinations/amplitude)
- [Webhooks](/destinations/http-request)
4. Complete the sync setup.
#### Configure events
After your sync has been created, open its **Configuration** tab to choose which membership changes to forward to the destination.
Depending on the destination, you can configure:
- **Entry events** — sent when a user qualifies for the audience
- **Exit events** — sent when a user leaves the audience
- **Custom event mappings** — additional event or property mappings supported by the destination
Send entry events when downstream systems only need to react when a user qualifies. Add exit events when downstream systems also need to react when a user stops qualifying—for example, to remove a suppression, cancel a journey, or stop targeting. Exit events fire whenever a user leaves the audience, including when a time-based condition [ages out](/real-time/architecture#time-based-membership-expiration-age-outs), not only when their latest event activity stops matching.
### Option 2: Membership API
The **Membership API** allows applications to check audience membership directly.
Your application sends a `user_id` and receives all realtime audiences the user currently qualifies for.
This is commonly used for:
- website personalization
- feature gating
- experimentation platforms
- in-app messaging
**Example request:**
```bash
GET /v1/audiences/{parent_model_id}/{user_id}
Authorization: Bearer
```
**Example response:**
```json
{
"parent_model_id": "1234567",
"user_id": "123xyz",
"audience_ids": ["1212121", "3434343"]
}
```
The Membership API returns realtime audience membership. To retrieve
warehouse-backed traits and profile attributes, use the [Personalization
API](/real-time/personalization-api).
---
## Refresh warehouse-backed conditions
If a realtime audience includes warehouse-backed conditions, Hightouch evaluates those conditions against a cached copy of your warehouse data rather than querying the warehouse for every incoming event. That cache refreshes on a schedule, and you can see its freshness and control when it refreshes from the audience.
When an audience has warehouse-backed conditions, a freshness indicator appears in the audience header next to the source, showing the current state—**Current**, **Stale**, **Refreshing**, or **Pending**. Hovering over it opens a panel with the **Last pull** time and result, the **Next scheduled pull**, and a link to the pull history.
To control refreshes:
- **Run an on-demand pull.** On the **History** tab, select **Run [source] pull** (labeled with your source, for example **Run Snowflake pull**) to refresh the cache immediately—useful when you've changed warehouse data or audience logic and want it reflected sooner than the next scheduled pull.
- **Adjust the refresh cadence.** Open the **Configuration** tab and set the **Pull schedule**. Options are every 4, 6, 8, 12, or 24 hours (the default is every 24 hours).
- **Review pull history.** The **History** tab lists recent pulls with their **Pull Date**, **Status**, **Trigger** (on-demand or scheduled), **Duration**, and **Rows**, so you can confirm refreshes are succeeding.
Scheduled pulls are always full refreshes. On-demand pulls may be partial when the staging table is still compatible.
Running an on-demand pull and changing the pull schedule require permission to
update the audience.
Realtime event conditions are always evaluated live and are not affected by the warehouse refresh cadence.
---
## Example: Suppression use case
**Goal:** Hide a promotional banner after a user completes a purchase.
**Audience filters:**
- Event: `order_completed` in the last 5 minutes
- Trait: `LTV > 300`
- Property: `channel = web`
**Delivery:**
Realtime webhook sync to the personalization system
**Result:**
A user completes a purchase and immediately exits the promotional audience. The banner no longer appears during the same session.
This pattern is commonly used for cart abandonment suppression, real-time
promotions, and session-based personalization.
---
## Constraints
Keep the following limitations in mind when designing realtime audiences:
- Realtime event conditions **must include a lookback window** (maximum **24 hours**)
- Only event **count** conditions are supported today (no arbitrary aggregations like SUM)
- User property conditions from `identify` calls must go through the warehouse and cannot be evaluated in realtime
For the full list of constraints, see [Constraints and limitations](/real-time/architecture#constraints-and-limitations) in the architecture guide.
---
## Best practices
When designing realtime audiences:
- Focus on **recent session behavior** (minutes rather than hours)
- Keep event conditions **specific and narrow**
- Combine live events with **warehouse traits** for richer targeting
- Monitor delivery through **sync logs** or **API responses**
Realtime event conditions must include a lookback window and support windows up to **24 hours**.
---
## What's next
- [Real-Time Personalization overview](/real-time/overview)
- [Realtime audience architecture](/real-time/architecture)
- [Personalization API](/real-time/personalization-api)
---
## Ads
**URL:** https://hightouch.com/docs/campaign-intelligence/ads
**Description:** The Ads view in Hightouch's Campaigns feature helps you understand conversions and campaign data from your connected ad platforms.
**Section:** Campaign Intelligence
## Overview
The ads dashboard is the best way to view performance for ad campaigns across any ad channels. You can report on platform level metrics like ad spend and impressions or see conversions from each campaign with customized attribution models.
## Configuring your view
To configure your view, follow these steps to select the appropriate columns:
### 1. Select columns
- **Asset columns**
Asset columns describe the asset and include attributes such as the created date, author, status, account, channel, and geography. These columns are sourced from the asset model in the schema layer.
- **Asset metrics**
Asset metrics typically include cost, clicks, and impressions. For ads reporting, these metrics are loaded from ad platforms and processed in the warehouse. Note that these metrics are usually reported at the campaign/ad set/ad level and cannot be tied to individuals. Learn more from dbt [docs](https://hub.getdbt.com/fivetran/ad_reporting/latest/). These metrics are created from the ad stats model in the schema layer.
- **Conversion metrics**
Conversion metrics are based on metrics defined in the metrics layer and use the attribution model defined in the attribution layer. Common conversion metrics include the number of transactions, total revenue, unique user transactions, bookings, and product views. These metrics can be derived from any customer event, both online and offline.
### 2. Apply filters
Filters enable you to customize the view to see only the campaigns or ads that are relevant. You can filter on any of the columns defined above. Common filters include filtering by campaign name, campaign dates, or spend amounts. You can apply string-based filters or numeric filters.
### 3. Time range
Campaign Intelligence has four default time ranges: 7 days, 14 days, 30 days, and 90 days.
Note: the time range is not to be confused with the attribution lookback
window. The time range only determines which conversions to report in the UI,
whereas the lookback window in the attribution model determines which
touchpoints should be attributed for a conversion.
### 4. Column sorting
You can customize your view by sorting your columns either alphabetically or numerically.
## Saving your view
You can save a view to revisit later or share across your team. The saved view captures the current filters, sorts, visible columns, and time range for a given table. You can quickly switch between them in the "Views" menu.
{/* */}
Additional documentation is coming soon, please if you're interested in this feature.
{/* */}
---
## Assets
**URL:** https://hightouch.com/docs/campaign-intelligence/assets
**Description:** Assets are the most granular elements of a campaign.
**Section:** Campaign Intelligence
## Overview
**Assets** are the most granular elements of a campaign. For paid channels this might be ads and for lifecycle marketing this might be an email or even a variant.
We support the following **Asset types**:
- Emails
- SMS
- Ads
{/* */}
---
## Charts Overview
**URL:** https://hightouch.com/docs/campaign-intelligence/charts
**Description:** Learn what how to configure, save, and update charts
**Section:** Campaign Intelligence
## Chart types
Charts are the building blocks of Hightouch's Intelligence features. We currently support several chart types:
* [Insights](https://hightouch.com/docs/campaign-intelligence/insights): Analyze audience composition, breakdowns, and performance.
* [Funnels](https://hightouch.com/docs/campaign-intelligence/funnels): Identify drop-off points across a sequence of events.
* [Splits](https://hightouch.com/docs/campaign-intelligence/splits): Measure results of experiments between randomized groups.
## Key elements
While the analysis varies between chart types, the general anatomy stays consisent across them.
- **Parent model\:** The dataset for your analysis, defined in the schema setup step.
- **Chart type\:** The analysis type (Insights, Funnels, Splits).
- **Metrics\:** An aggregation over events. Metrics can have an aggregation type, filters, and an attribution window. Metrics can be on the cohort level (sum of purchases for all users) or on the user level (average purchase amount per user).
- **Segment by\:** The segment of users to include in the analysis. Default is to include the entire parent model. You can select multiple existing audiences or create inline audiences. If using an attribution window, events are only included if the user falls within the attribution window for the audience-metric pair.
- **Group by\:** The dimension to breakdown a metric by. There are several types of dimensions:
- User properties: Attributes, traits, or merge columns of a user. Examples: gender, state, loyalty status
- Event properties: Attributes on the specific event. Examples: brand, campaign, utms.
- Related properties: Attributes on a related model. Examples: item, item property.
- **Time interval\:** Time granularity for metric rollup (daily, weekly, monthly). For example, sessions from distinct users per day = DAU, per week = WAU.
- **Time window\:** The time period for analysis. Events outside this window won't contribute to metrics.
- **Saved charts\:** Drawer showing all saved charts across the workspace.
- **Export as CSV\:** Option to download the chart's datapoints as CSV.
- **Sampling\:** For parent models that have Sampling turned on, enable `Fast queries` to query a sampled percentage of your parent model.
- **Mode\:** The type of data aggregation (`Over time` or `Cumulative`) when viewing time-series charts (Line, Column, and Area).
- **Visualization\:** The type of data visualization (Line, Column, Area, Bar, Table, Heat map, Number).
---
## Copilot
**URL:** https://hightouch.com/docs/campaign-intelligence/copilot
**Description:** Hightouch's Campaign Intelligence
**Section:** Campaign Intelligence
Documentation is coming soon. Please see the video below for a preview of our Copilot features.
---
## Dashboards
**URL:** https://hightouch.com/docs/campaign-intelligence/dashboards
**Description:** Hightouch's dashboards feature provides a snapshot of your audience and performance data.
**Section:** Campaign Intelligence
**Audience:** Marketers and analysts
**Prerequisites:** Saved charts created from your schema models
_Use dashboards to combine saved charts into a single, visual workspace where you can analyze trends, validate campaign logic, and monitor performance._
---
## Overview
Dashboards let you organize saved charts in a flexible grid layout. You can use them to:
- Explore key metrics and user behavior
- Validate audience criteria before launching campaigns
- Monitor engagement, conversions, and lift over time
Dashboards are composed of saved **Charts**. [Each chart](/campaign-intelligence/overview#campaign-intelligence-chart-types) is a data visualization created from your models (users, events, and related models). Charts can be reused across dashboards.
All charts in a dashboard must share the same parent model.
## Who uses it
| Role | Responsibilities |
| ---------------------------------------- | --------------------------------------- |
| **Data, Marketing Ops or MarTech teams** | Add and configure charts and dashboards |
| **Marketers** | Explore and filter dashboards |
## Create a dashboard
### Step 1: Open the Dashboards tab
1. Go to `Intelligence → Dashboards` to see all dashboards in your workspace.
2. Click `Create dashboard` in the top right.
### Step 2: Name your dashboard and choose a parent model
In the modal:
1. Select a parent model (e.g. Users).
2. Enter a dashboard name.
3. Optionally, add a description.
4. Click `Create dashboard` to save and open your new dashboard.
### Step 3: Add charts
Click `Add chart` to choose one of the following options:
- **Saved chart**: Add an existing chart from your library
- [**Insights**](/campaign-intelligence/insights) **chart**: Build a new visualization to measure your marketing efforts
- [**Funnel**](/campaign-intelligence/funnels) **chart**: Track drop-off across a multi-step flow
After adding charts, use drag-and-drop to arrange and resize them in your layout. Additional charts can be added within the same row or a new row, unless the maximum number of 4 charts for the row has been reached.
{" "}
Archived charts will still appear on existing dashboards, but can't be added to
new dashboards.{" "}
## Apply filters
Use the filter bar at the top of the dashboard to apply filters across all charts. You can filter by:
- **Time range**: Default (respects the time filters on the individual charts), 7d, 30d, 60d, 90d, or set a custom range.
- **Properties**: Parent, event, and related model fields like `country`, `device_type`.
- **Audience membership**: One or multiple audiences can be applied to the dashboard.
Filters help you explore performance across specific timeframes, properties, or user segments.
{" "}
Filters applied in View mode are temporary. Switch to Edit mode and click **Save
changes** to make them permanent.{" "}
## Edit and manage dashboards
### Modes and editing
Dashboards support two modes:
| Mode | Description |
| -------- | ------------------------------------------------------------- |
| **Edit** | Add, remove, or resize charts. Update filters and layout. |
| **View** | Explore charts and apply temporary filters. No changes saved. |
To save layout or filter changes, switch to Edit mode and click `Save changes`.
### Manual refresh and data retention
- Dashboard data is cached for up to **30 days**
- Each dashboard displays a `Last refreshed` timestamp
- Click the refresh button to manually reload chart data. New filters are not automatically applied, you must refresh the dashboard to apply them.
### Scheduled refresh
You can also schedule dashboard refreshes so data is automatically loaded the next time you open the dashboard. The following options are available:
- Manual: The dashboard can only be refreshed manually.
- Custom recurrence: The dashboard refreshes at a specific time and schedule (for example, every Monday at 9:00 AM).
### Chart behavior
If a saved chart is updated or removed after being added to a dashboard:
| Saved chart status | Behavior in dashboard |
| ------------------ | ---------------------------------------------------------------------------- |
| **Deleted** | A deleted chart placeholder appears |
| **Updated** | A warning is displayed. Changes appear only after the dashboard is refreshed |
| **Archived** | An archived badge appears until removed or restored |
### Sharing dashboards
Dashboards are visible to anyone in your workspace with access to the parent model and can be shared via link with other workspace members.
### Exporting dashboards
The data displayed in the dashboard charts can be exported. The export generates a ZIP file containing a separate CSV for each individual chart in the dashboard.
### Dashboard performance
- Limit dashboards to **20 charts** for optimal performance
- If performance is a concern, consider leveraging [sampling](/customer-studio/sampling) (fast queries). If enabled, it applies to the entire dashboard, not individual charts. Sampling _could_ impact the accuracy of certain charts.
## Example: Campaigns performance dashboard
**Goal:** Monitor engagement and conversions for Q3 promo campaigns
1. Create a saved chart for **audience size over time**
2. Create a dashboard named **Q3 campaigns monitoring**
3. Add:
- Saved audience size chart
- Build a chart for "Email delivered"
- Build a chart for "Checkouts" in-line
4. Apply a **Last 30 Days** global filter
5. Refresh as needed to monitor performance throughout the campaign
---
## Emails
**URL:** https://hightouch.com/docs/campaign-intelligence/emails
**Description:** The Emails view in Hightouch's Campaigns feature helps you understand conversions and campaign data from your email messages.
**Section:** Campaign Intelligence
## Overview
The emails dashboard is the best way to view performance for all your email messages. You can view engagement metrics like sends, opens, and clicks along with custom conversion metrics such as transactions.
## Configuring your view
To configure your view, follow these steps to select the appropriate columns:
### 1. Select columns
- **Asset columns**
Asset columns describe the asset and include attributes such as the created date, author, status, variant, and campaign type. These columns are sourced from the asset model in the schema layer.
- **Engagement metrics**
Engagement metrics include sent, delivered, opened, clicked, click rate, unique clicks, and unsubscribed. These events are typically generated from channel tools such as Iterable, Braze, SFMC, Klaviyo, and Attentive and loaded into the warehouse. These metrics are pre-defined based on the **Event type** selected.
- **Conversion metrics**
Conversion metrics are based on metrics defined in the metrics layer and use the attribution model defined in the attribution layer. Common conversion metrics include the number of transactions, total revenue, unique user transactions, bookings, and product views. These metrics can be derived from any customer event, both online and offline.
### 2. Apply filters
Filters enable you to customize the view to see only the campaigns or ads that are relevant. You can filter on any of the columns defined above. Common filters include filtering by campaign name, campaign dates, or spend amounts. You can apply string-based filters or numeric filters.
### 3. Time range
Campaign Intelligence has four default time ranges: 7 days, 14 days, 30 days, and 90 days.
Note: the time range is not to be confused with the attribution lookback
window. The time range only determines which conversions to report in the UI,
whereas the lookback window in the attribution model determines which
touchpoints should be attributed for a conversion.
### 4. Column sorting
You can customize your view by sorting your columns either alphabetically or numerically.
## Saving your view
You can save a view to revisit later or share across your team. The saved view captures the current filters, sorts, visible columns, and time range for a given table. You can quickly switch between them in the “Views” menu.
{/* */}
Additional documentation is coming soon, please if you're interested in this feature.
{/* */}
---
## Events
**URL:** https://hightouch.com/docs/campaign-intelligence/events
**Description:** Hightouch's Campaigns connects user events to asset models in a Schema's parent model.
**Section:** Campaign Intelligence
## Overview
**Events** connect a user to an asset model. **Events** require:
1. a timestamp
2. a relationship with a parent model
3. a relationship with an asset model
Apply one of the standard **Event types** to an **Event** to populate the default metrics on the various asset dashboards of the Campaigns page. We will add more **Event types** over time.
{/* */}
---
## Experiments
**URL:** https://hightouch.com/docs/campaign-intelligence/experiments
**Description:** Measure the impact of your marketing campaigns using experiment results from Audience Experiments.
**Section:** Campaign Intelligence
**Experiments** were previously called **Splits**.
| | |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers and analysts who want to compare performance across randomized test and control groups. |
| **Prerequisites** |
Created [Audience Experiments](/customer-studio/experiments) in Customer Studio
Audience synced at least once after experiments were created
Event data (such as purchases or clicks) available for measurement
|
_Experiments help you evaluate how different marketing strategies perform by analyzing outcomes across randomized experiment groups. You can measure lift, compare treatment and holdout results, and understand whether your campaigns drive meaningful changes in user behavior._
---
## What you'll learn
After reading this article, you’ll know how to:
- [Identify how Experiments are created and managed](#how-experiments-are-created)
- [Set up Audience Experiments for measurement](#setup-and-requirements)
- [Navigate to and review all Experiments](#1-view-list-of-experiments)
- [Configure metrics, start dates, and measurement windows](#2-configure-an-experiment)
- [Interpret lift, confidence intervals, and performance trends](#3-interpret-results)
- [Normalize results for clearer comparisons](#4-normalize-results)
---
## Overview
**Experiments** provide a measurement layer for [Audience Experiments](/customer-studio/experiments).
Whenever you create an Audience Experiment in Customer Studio or configure an experiment on an [A/B split node in Journeys](/customer-studio/journeys#ab-split), Hightouch automatically generates a corresponding **Experiment** so you can track performance across randomized groups.
Experiments allow you to:
- Compare holdout vs. treatment outcomes
- Analyze lift and confidence intervals
- Visualize performance over time
- Evaluate strategies continuously
| Feature | Description |
| ---------------------- | ----------------------------------------------------------- |
| **Experiment results** | Displays lift, performance trends, and confidence intervals |
| **Configuration** | Controls metrics, measurement windows, and start dates |
| **Normalization** | Enables per-member or baseline-scaled comparisons |
Experiment measurement charts have been updated. All experiment reporting now
lives in **`Intelligence` → `Experiments`**.
---
## How Experiments are created
Experiments are automatically managed based on Audience Experiments in Customer Studio:
- Creating an **Audience Experiment** automatically creates a corresponding **Experiment**.
- Disabling or deleting an Audience Experiment removes its Experiment.
- Restoring an Audience Experiment restores its Experiment.
This ensures measurement stays aligned with the audiences you're actively using.
---
## Setup and requirements
### 1. Create Audience Experiments
Before measuring experiment results:
1. Create [Audience Experiments](/customer-studio/experiments) in Customer Studio.
2. Ensure the audience has synced at least once _after_ the experiment groups were created.
3. Confirm that users generate measurable events (such as purchases, page views, or clicks).
For every Audience Experiment created in Customer Studio, Hightouch
automatically creates a corresponding Experiment in **`Intelligence` →
`Experiments`**. Deleting or restoring an Audience Experiment removes or
restores the linked Experiment.
---
## Measure results
The **`Experiments`** section of Intelligence helps you compare outcomes between experiment groups and evaluate the impact of your campaigns.
Navigate to: **`Intelligence` → [`Experiments`](https://app.hightouch.com/experiments?from=docs)**
---
### 1. View list of Experiments
The Experiments list shows all Experiments, their statuses, and recent updates.
Statuses include:
- **Draft:** Missing one or both required configuration elements (primary metric or start date).
- **Scheduled:** Fully configured; the start date is in the future.
- **Running:** Fully configured; the start date is today or in the past.
---
### 2. Configure an experiment
Open an experiment and select the **`Configuration`** tab.
From here, you can:
- **Choose a primary metric** (required) and optional secondary metrics (e.g., _Conversions_, _Revenue_).
- [Learn how to create a metric](/campaign-intelligence/metrics).
- _Optional:_ Use the **`Filter by`** dropdown to refine by user properties or events.
- **Set a Start date**
- Determines when measurement begins.
- Does **not** apply retroactively.
- Does _not_ affect sync or activation behavior.
- **Choose a Measurement window**
- Example: _Entry → 30 days after entry_ measures events from the moment the user enters the audience to 30 days later.
---
### 3. Interpret results
The **`Overview`** tab displays experiment outcomes and performance trends.
Key elements:
- **Lift %:** Percentage difference between treatment and holdout group performance.
- **Lift interval bar:**
- **Green:** Significant positive lift
- **Red:** Significant negative lift
- **Gray:** Not statistically significant (interval overlaps 0%)
- **Performance lines:**
- Solid lines show performance over time
- Shaded regions represent 95% confidence intervals
Lift intervals use a Bayesian method, enabling continuous monitoring without
needing to wait for an experiment to complete.
---
### 4. Normalize results
Use the **`Normalization`** dropdown to switch perspectives:
- **Per member (default):** Average performance per user
- **Normalized to baseline group:** Scales performance to compare groups evenly
Hover over the lift card to view raw totals.
---
## Funnels
**URL:** https://hightouch.com/docs/campaign-intelligence/funnels
**Description:** Hightouch's Customer Studio Funnels helps you pinpoint funnel drop-offs and track conversion rates over time.
**Section:** Campaign Intelligence
## Overview
Funnels are an easy way to explore where dropoff occurs across a customer's journey or analyze any sequence of events. This is important to understand which touch points need to be improved to improve overall conversion rate.
### Building a Funnel
To build a funnel analysis in Hightouch, follow these steps:
1. Select Starting Event: Choose the initial event to analyze. This can be any specific or general event.
2. Add Event Properties: Apply filters to events by selecting property names and values.
3. Define Event Sequence: Choose additional events and set the order (exact order, any order, this order).
4. Segment Users: Identify and segment the user base for the analysis by adding properties or importing saved segments.
### Compare Segments
While funnels tell you where your customers are dropping off overall, it doesn't tell you who is dropping off at each point. Get deeper insights by comparing conversion paths against different segments, user properties, or event properties.
### Create an Audience
Once you know who is dropping off where, you are in a great spot to nudge them past this point. You can create an audience by selecting on the chart where you can then sync it to any of our destinations.
---
## Insights
**URL:** https://hightouch.com/docs/campaign-intelligence/insights
**Description:** Hightouch's Intelligence product provides the Insights feature to measure how your audiences are performing.
**Section:** Campaign Intelligence
## Overview
Suppose you've just built a new [audience](/customer-studio/usage) and deployed them to a campaign.
You are probably wondering: What actions are they taking? How has my audience reacted to that new campaign?
The Insights feature enables you to view breakdowns of your members and define campaign success metrics to measure your audience against over time, all within Hightouch. You can track insights on metrics such as:
- Revenue per country
- Number of subscription sign-ups
- Number of inactive members who've recently logged back in
You can then track how these metrics perform over time and compare them across audiences.
Insights also works seamlessly with Hightouch [Experiments](/customer-studio/experiments). You can set up an A/B test within Hightouch, see how each experiment group performs, and determine a winner.
This doc provides step-by-step instructions for creating **Insight** charts.
## Understanding the core concepts
#### Custom metrics (“Saved metrics”)
You can define and save metrics using any [**event**](/customer-studio/schema#add-event-models) in your [data schema](/customer-studio/schema).
Metrics are defined on a [parent-model](/customer-studio/schema#define-the-parent-model) basis, and you can enable them on any audience built off the parent model.
#### Events and related models
Events and related models provide additional characteristics or actions to filter your [parent-model](/customer-studio/schema#define-the-parent-model). They're similar, but the key difference is that events require a timestamp.
Events and related models contain **properties** — if you think of each instance as a row, properties are the columns.
#### Properties
Properties that are defined by your parent model. If you think of each member in your parent model as a row, member properties are the columns.
#### Traits
[Traits](/customer-studio/traits) are custom columns added to your parent model.
## Getting started with Insights
Within the **Charts** product, the **Insights** tab allows you to create these charts. You can also view Insights charts that have already been created and saved.
## Metrics and properties
You can base an insight chart off of several different measurement options, and depending on the type you select, you can select an aggregation type, define a measurement window and add filters.
- **Aggregation type:** Defines how individual instances are combined to form the metric – for example, the total number of rows, unique members, or the sum, count or average of a property.
- **Measurement window:** Specifies the time period in which the selected metric must occur - for example, measuring an event from audience entry to 30 days after audience exit.
- **Filters:** Narrow down the data included. Filters are based on the properties of the models you use – for example location, product type, or price.
### Saved metrics
Saved metrics already include an aggregation, so additional aggregations can't be applied within **Charts**. However, you can change the measurement window and filters.
### Events and event properties
Different aggregations are available depending on whether an event property is included.
### Related model and related model properties
Different aggregations are available depending on whether a [related model](/customer-studio/schema#add-related-models) property is included.
### Properties
You can use the property you select in an aggregation on the parent model. Different aggregations are available depending on whether the member property is included.
Additional filters cannot be applied, but the metric can be **segmented** and **grouped**, as detailed below.
### Traits
Traits have a per-member aggregation built in, so additional aggregations can't be applied within **Charts**.
## Segmenting and grouping
### Segmenting
Segment data by one or more predefined Audiences, or create ad-hoc member segments, to focus the analysis on specific groups of members.
### Grouping
Group your data based on one or more selected properties or Traits (e.g., gender, location, brand loyalty). This allows you to compare the metric across different segments of members.
## Visualizations
### Line charts
Used to view and compare data trends over-time or cumulatively, across a selected time-frame and interval (e.g., daily, weekly). Ideal for tracking changes over time and comparing multiple series.
**Examples:**
**How many marketing emails did I send to this audience weekly over the last 60 days?**
**How many marketing emails did I send to these two audiences weekly over the last 60 days?**
### Column charts
View and compare aggregated values across categories using vertical bars. This can be done over-time or cumulatively, across a selected time-frame and interval (e.g., daily, weekly). Great for visual side-by-side comparisons.
**Example: Across two audiences, how can I compare how many members of each audience have started a checkout process over the last 7 days?**
### Bar charts
View and compare aggregated data across segments and groupings using horizontal bars for a given time frame. Best for top-N comparisons or summaries.
**Example: How is my audience distributed by state?**
### Area charts
View changes in data volume over-time or cumulatively, across a selected time-frame and interval (e.g., daily, weekly). Useful for visualizing contribution trends and growth.
**Example: What is the cumulative number of marketing emails sent to my audience over the last 30 days?**
### Table view
View aggregated data in rows, allowing for detailed comparison across values of a specific property for a given time-frame. Ideal for in-depth row-by-row analysis.
**Example: How many members by state have received marketing emails over the last 60 days?**
### Number chart
Communicate the most important numbers at a glance, typically used on dashboards to display and track prominent metrics or key performance indicators.
**Example: How many total members are in my audience?**
## Saving a chart
You can use the `Save chart` button to save any chart for future use. Once saved, the chart can be accessed from the Charts panel.
## Exporting a chart
You can use the `Export as CSV` button to download a chart’s data points in CSV format. You can export the data only after the chart has finished running.
---
## Metrics
**URL:** https://hightouch.com/docs/campaign-intelligence/metrics
**Description:** Learn how to create custom metrics to use in the Intelligence product.
**Section:** Campaign Intelligence
## Overview
A metric is a quantifiable measure used to track and assess the status of a specific business process. Metrics help in understanding performance and identifying areas for improvement. Key components of a metric are shown below:
### Creating a Metric
- **Event**
An event is a specific action or occurrence that can be measured. Examples include a user clicking a button, a transaction being completed, or a user becoming a subscriber.
- **Aggregation Type**
The aggregation type defines how individual events are combined to form the metric. Our aggregations include sum, count, unique count, and average.
- **Filter**
Filters are used to narrow down the data included in a metric. Filters can be based on various criteria such as location, product type, or price.
### Adding an Attribution Method
While metrics can stand alone, you must apply an attribution method to tie them back to a campaign, enabling you to understand the impact of different marketing efforts on these metrics.
Metrics can have multiple attribution methods. This is useful for gaining a more comprehensive understanding of how different events contribute to conversions. Learn more about attribution [here](/campaign-intelligence/attribution-methods).
---
## Intelligence overview
**URL:** https://hightouch.com/docs/campaign-intelligence/overview
**Description:** Intelligence gives marketers and analysts self-serve analytics inside the same workspace where they build audiences and run campaigns — metrics, charts, dashboards, and experiment measurement, all from your warehouse data.
**Section:** Campaign Intelligence
| Audience | How you'll use this article |
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| **Marketing and analysts** | Understand what Intelligence does and how to use it day-to-day to define metrics, build charts, create dashboards, and measure experiments. |
| **Data teams** | Understand the setup Intelligence requires and how it connects to the rest of the Hightouch platform. |
_Intelligence gives you self-serve analytics inside your activation workflow. Explore audience composition, measure experiment results, and track campaign performance — all from the same warehouse data you use to build and sync audiences._
---
## What you'll learn
- [What Intelligence is and what problems it solves](#overview)
- [How to access Intelligence](#access-intelligence)
- [Core features: metrics, charts, dashboards, and experiments](#core-features)
- [How to get started, by role](#get-started)
- [When to use Intelligence](#when-to-use-intelligence)
- [How Intelligence connects to other Hightouch products](#how-intelligence-fits-into-hightouch)
---
## Overview
Most marketing tools only show analytics for campaigns run on their platform. Business intelligence tools provide broader analysis but sit outside the campaign workflow. Intelligence brings analytics into the same workspace where teams build audiences, measure campaigns, and manage data.
Instead of switching between platforms or waiting for reports, you can define shared KPIs, build charts, measure experiments, and track results directly from your warehouse data. Because Intelligence runs on the same connected source and schema as Customer Studio, your charts, audiences, and syncs use the same source data and schema.
---
## Access Intelligence
1. Go to [**app.hightouch.com/analytics**](https://app.hightouch.com/analytics?from=docs).
2. Select **Charts**, **Metrics**, or **Dashboards** to begin.
---
## Core features
### Define metrics
Use [metrics](/campaign-intelligence/metrics) to standardize how your team measures performance. A metric defines a specific calculation — like conversion rate or revenue per user — that you can reuse across charts and dashboards so every team uses the same definition.
### Build charts
Charts are the building blocks of Intelligence. Choose a chart type based on the question you're trying to answer:
- **[Insights](/campaign-intelligence/insights)** — Analyze audience composition, breakdowns, and performance over time.
- **[Funnels](/campaign-intelligence/funnels)** — Identify where users drop off across a sequence of events.
- **[Experiments](/campaign-intelligence/experiments)** — Compare results between randomized test and control groups.
See [Charts overview](/campaign-intelligence/charts) for details on configuring, saving, and updating charts. After building a chart, save it so you can reference it later or add it to a dashboard.
### Create dashboards
[Dashboards](/campaign-intelligence/dashboards) combine saved charts into a single monitoring view. Use them to track ongoing campaign performance, compare audience segments, or review experiment results over time.
All charts in a dashboard must share the same parent model. Plan your chart
strategy around this constraint when building dashboards.
### Measure experiments
If you're running an A/B test or holdout group, use Experiments to measure whether a treatment worked. Create [audience experiments](/customer-studio/experiments) in Customer Studio to randomly divide an audience into test and control groups, then measure results with [experiment charts](/campaign-intelligence/experiments) in Intelligence.
Experiments require several prerequisites before results appear: audience
snapshots must be enabled, split groups must be created, at least one sync
must run after split creation, and post-snapshot event data must exist for
measurement.
---
## Get started
Intelligence shares the same schema and source configuration as Customer Studio. If your workspace already has a schema configured, marketers can start using Intelligence immediately.
### For data teams and platform admins
Complete these steps if your workspace doesn't already have a schema configured. This setup is typically done once per workspace.
| Step | What to do | Why |
| ---- | ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| 1 | [Connect a source](/sources/overview) | Intelligence queries warehouse data directly — it needs at least one connected source. |
| 2 | [Define your data schema](/customer-studio/schema) | The schema defines the parent model, events, and related models that power metrics and charts. |
| 3 | [Add destinations (optional)](/destinations/overview) | Connect destinations if your team will move from analysis to activation, such as syncing refined audiences after reviewing chart results. |
If your workspace already has a Customer Studio schema configured, skip
directly to the marketer workflow below.
### For marketers and analysts
Once the schema is set up, the core workflow is: define a metric, build a chart, save it, and optionally add it to a dashboard. Start by [creating a metric](/campaign-intelligence/metrics) for a core KPI your team tracks, then build an [Insights chart](/campaign-intelligence/insights) to visualize it. From there, explore [funnels](/campaign-intelligence/funnels) for drop-off analysis, [experiment charts](/campaign-intelligence/experiments) for test measurement, and [dashboards](/campaign-intelligence/dashboards) to organize everything into a monitoring view.
### Additional setup for Campaigns
The core Intelligence workflow — metrics, charts, dashboards, and experiments — covers most analytics use cases. **Campaigns** is a separate surface for channel-level campaign reporting and attribution across Emails, Ads, and SMS. Campaigns requires interaction models configured in your schema. See [Campaign schema](/campaign-intelligence/campaign-schema) for setup instructions.
---
## When to use Intelligence
Intelligence is the right tool when you need to analyze, measure, or visualize data that's already in your warehouse. For activation and orchestration, use Customer Studio or AI Decisioning instead.
| Scenario | Recommended approach |
| -------------------------------------------------------------- | ------------------------------------------------------------------ |
| Define shared KPIs and reuse them across charts and dashboards | **Intelligence** [Metrics](/campaign-intelligence/metrics) |
| Analyze audience composition, breakdowns, or trends over time | **Intelligence** [Insights](/campaign-intelligence/insights) |
| Identify where users drop off in a conversion sequence | **Intelligence** [Funnels](/campaign-intelligence/funnels) |
| Measure A/B test or holdout group results | **Intelligence** [Experiments](/campaign-intelligence/experiments) |
| Track channel-level campaign reporting and attribution | **Intelligence** [Campaigns](/campaign-intelligence/campaigns) |
| Build audiences and sync them to destinations | [Customer Studio](/customer-studio/overview) |
| Automate which message, channel, and timing each user receives | [AI Decisioning](/ai-decisioning/overview) |
---
## How Intelligence fits into Hightouch
Intelligence is the measurement layer of the Hightouch platform. It reads from the same warehouse data that powers audiences and syncs, so your analytics and activation always stay aligned.
#### What Intelligence uses
- **Audiences and traits from [Customer Studio](/customer-studio/overview)** — Intelligence analyzes the audiences you build, using traits and event data already defined in your schema.
- **Experiment groups from [Experiments](/customer-studio/experiments)** — When you create randomized test and control groups in Customer Studio, Intelligence provides the charts to measure results.
- **Warehouse data from your [connected source](/sources/overview)** — Metrics and charts query your warehouse directly, so any data available to Hightouch is available in Intelligence.
#### What Intelligence sends
- **Insights that refine audiences** — After reviewing chart results or funnel drop-offs, marketers can adjust audience logic or create new segments directly in Customer Studio.
- **Dashboards for stakeholders** — Combine charts into shareable dashboards so teams can monitor campaign performance and customer trends in one place.
---
## Permissions
Access to Intelligence is governed by Customer Studio schema and model permissions, plus the analytics entitlement for your workspace. There is no separate Intelligence role or grant tab. See [Roles](/workspace-management/roles#customer-studio) for Customer Studio grants.
---
## Get started with Intelligence
**URL:** https://hightouch.com/docs/campaign-intelligence/quick-start
**Description:** Set up Intelligence and learn the end-to-end workflow for defining metrics, building charts, creating dashboards, and measuring experiments.
**Section:** Campaign Intelligence
| | |
| ----------------- | ----------------------------------------------------- |
| **Audience** | Platform admins, data teams, marketers, and analysts |
| **Prerequisites** | Customer Studio access, at least one connected source |
Intelligence gives marketers and analysts self-serve analytics inside the same workspace where they build audiences and run campaigns. Instead of switching between platforms, you can define shared metrics, build charts, measure experiments, and track results — all directly from your warehouse data.
This guide covers:
| Section | What you'll learn |
| ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
| [**Setup for platform admins and data teams**](#setup-for-platform-admins-and-data-teams) | Connect your data and configure the schema so Intelligence can query users, events, and related models. |
| [**Day-to-day usage for marketers and analysts**](#how-marketers-and-analysts-use-intelligence) | Define metrics, build charts, organize them into dashboards, and measure experiments. |
---
## Setup for platform admins and data teams
Before marketers can explore data in Intelligence, your workspace needs a connected source and a configured schema. This setup is typically done once per workspace.
| Step | What to do | Why |
| ---- | ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| 1 | [Connect a source](/sources/overview) | Intelligence queries warehouse data directly — it needs at least one connected source. |
| 2 | [Define your data schema](/customer-studio/schema) | The schema defines the parent model, events, and related models that power metrics and charts. |
| 3 | [Add destinations (optional)](/destinations/overview) | Connect destinations if your team will move from analysis to activation, such as syncing refined audiences after reviewing chart results. |
Intelligence shares the same schema and source configuration as Customer
Studio. If your workspace already has a schema configured, marketers can start
using Intelligence immediately.
---
## How marketers and analysts use Intelligence
Once the schema is set up, you can use Intelligence to answer questions, track KPIs, and measure experiments without leaving Hightouch.
The core workflow is: define a metric, build a chart, save it, and optionally add it to a custom dashboard. If you're running a test, use Experiments to measure results.
### Define metrics
Use metrics to standardize how your team measures performance. A metric defines a specific calculation — like conversion rate or revenue per user — that you can reuse across charts and dashboards.
| What to do | Why |
| ------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| [Create a metric](/campaign-intelligence/metrics) | Establish consistent KPI definitions so every chart and dashboard uses the same calculation. |
### Build charts
Charts are the building blocks of Intelligence. Choose a chart type based on the question you're trying to answer.
| Chart type | When to use it |
| ------------------------------------------------- | -------------------------------------------------------------------- |
| [Insights](/campaign-intelligence/insights) | Analyze audience composition, breakdowns, and performance over time. |
| [Funnels](/campaign-intelligence/funnels) | Identify where users drop off across a sequence of events. |
| [Experiments](/campaign-intelligence/experiments) | Compare results between randomized test and control groups. |
See [Charts overview](/campaign-intelligence/charts) for details on configuring, saving, and updating charts.
### Save and reuse charts
After building a chart, save it so you can reference it later or add it to a dashboard. Saved charts become reusable building blocks — start with one chart tied to a core KPI or audience question, then build from there.
### Create custom dashboards
Dashboards combine saved charts into a single monitoring view. Use them to track ongoing campaign performance, compare audience segments, or review experiment results over time.
| What to do | Why |
| ------------------------------------------------------- | ------------------------------------------------------------------------ |
| [Create a dashboard](/campaign-intelligence/dashboards) | Organize saved charts into a single view for monitoring and exploration. |
All charts in a dashboard must share the same parent model. Plan your chart
strategy around this constraint when building dashboards.
### Measure experiments
If you're running an A/B test or holdout group, use Experiments to measure whether a treatment worked.
| What to do | Why |
| -------------------------------------------------------------- | -------------------------------------------------------------------- |
| [Create audience experiments](/customer-studio/experiments) | Randomly divide an audience into test and control groups. |
| [Measure with Experiments](/campaign-intelligence/experiments) | Compare performance metrics between split groups to evaluate impact. |
Experiments require several prerequisites before results appear: - Audience
snapshots must be enabled. - Split groups must be created. - At least one sync
must run after split creation. - Post-snapshot event data must exist for
measurement.
---
## When to use Campaigns
The core Intelligence workflow — metrics, charts, dashboards, and experiments — covers most analytics use cases. **Campaigns** is a separate surface for campaign-specific reporting and attribution.
Use Campaigns when you need channel-level campaign reporting with attribution across Emails, Ads, and SMS. For everything else — audience exploration, funnel analysis, experiment measurement, and custom dashboards — use the core Intelligence features described above.
### Additional setup for Campaigns
Campaigns requires a parent model with **interaction models** configured in your schema. A standard parent model with only event and related models is not enough — the Campaigns page will show a "configure a parent model" warning until this is done.
To set up your schema for Campaigns:
| Step | What to do | Why |
| ---- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- |
| 1 | Go to **Customer Studio**, then **Schema** and select your parent model. | Campaigns reads from the same schema as the rest of Customer Studio. |
| 2 | Add an event model as a relationship from the parent model, then set its **Event type** to a specific interaction type (for example, Email Delivered, Sessions, or SMS Sent) rather than leaving it as Generic. | Setting a specific event type converts the event model into an interaction model, which is what Campaigns requires. |
| 3 | (Optional) Relate the interaction model to an **asset model** that represents the campaign content — such as which email, ad, or SMS message was sent. | Asset models enable Campaigns to break down reporting by individual campaign content, not just by channel. |
The Campaigns page shows tabs for Emails, Ads, and SMS. Which tabs appear
depends on which interaction types you've configured in your schema. For
example, configuring an Email Delivered interaction model enables the Emails
tab.
See [Campaigns](/campaign-intelligence/campaigns) and [Campaign schema](/campaign-intelligence/campaign-schema) for details.
---
## Ads Schema
**URL:** https://hightouch.com/docs/campaign-intelligence/schema/ads
**Description:** Schema information and diagrams for Hightouch's Campaign Intelligence for ads.
**Section:** Campaign Intelligence
## Overview
Campaign Intelligence for [ads](/campaign-intelligence/ads) leverages four types of models:
1. **Users** are the [parent model](/customer-studio/schema#define-the-parent-model).
2. **Sessions** are a type of [**Event**](/campaign-intelligence/events) and connect a user to an ad. They necessarily have a timestamp and typically include `utm` parameters.
3. **Ads** are a type of [**Asset**](/campaign-intelligence/assets), and minimally include campaign and platform information.
4. **Ad stats** is a model that indicates engagement with an ad, such as clicks, impressions, and spend. It should fit the format of [this dbt model](https://fivetran.github.io/dbt_ad_reporting/#!/model/model.ad_reporting.ad_reporting__ad_report).
The [relationships](/customer-studio/schema#define-relationships) between these models depend on whether your ad model has information about the ad itself, such as an `ad_name` or `ad_group_id` alongside the ad campaign information or just campaign information.
Below are [entity relationship diagrams](https://en.wikipedia.org/wiki/Entity%E2%80%93relationship_model) (ERDs) that show expected columns and relationships in both situations, including necessary primary (PK) and foreign keys (FK). We note differences between the schemas.
## Schema with only campaign data for ads
The ERD above includes the important features:
- Sessions connect to **Ads** with a campaign identifier based on what's available in the session, in this case through a mapping from `utm_campaign` to `campaign_name`.
- **Ads** also connect to **Ad stats** through a campaign identifier, in this case the `campaign_id`.
- Hightouch generates a primary key in the **Ad stats** model that combines the `date_day` and `campaign_id` columns.
## Schema with ad, ad group, and campaign data
Two notable differences in the schema are evident when there is additional ad and ad group information.
- Sessions now connect to **Ads** through an identifier at the level of an individual ad, in this case with a mapping from `utm_content` to `ad_name`.
- **Ads** also now connect to **Ad stats** through an individual ad level identifier, here the `ad_id`.
- Both **Ads** and **Ad stats** have additional columns for individual ad level information.
---
## Messages Schema
**URL:** https://hightouch.com/docs/campaign-intelligence/schema/messages
**Description:** Schema information and diagrams for Hightouch's Campaign Intelligence for email and SMS messages.
**Section:** Campaign Intelligence
## Overview
Campaign Intelligence for [messages](/campaign-intelligence/emails), such as emails and SMS, leverages three types of models:
1. **Users** are the [parent model](/customer-studio/schema#define-the-parent-model).
2. [**Events**](/campaign-intelligence/events) are events that connect users to an asset, such as email clicks or opens.
3. **Emails** are a type of [asset](/campaign-intelligence/assets).
## Schema
In the entity relationship diagram (ERD) below, we show a schema for messages using emails as an example. The ERD shows expected model columns and their relationships, with necessary primary (PK) and foreign keys (FK).
In the ERD above, Users are connected to different **Events** -- such as email opens and clicks -- through a unique `user_id`. **Events** require a `timestamp`. They can have additional columns based on the **Event type**, such as a `clicked_url` for email click events.
Each **Event** is also linked to an **Asset model**, either email or SMS, by way of a `message_id`. The message can have a variety of columns, including date columns such as `start_at` and `ended_at` to indicate the dates that email asset was in use.
---
## Business tier pricing
**URL:** https://hightouch.com/docs/pricing/bt-pricing
**Description:** Learn about Hightouch's Business tier pricing, including platform fees, consumption-based pricing, and feature add-ons.
**Section:** Pricing
{/* */}
{/* Allow for use of "We" in sensitive topics like pricing and security --> */}
Hightouch's Business tier pricing varies depending on your use case and required features.
It typically includes:
- A **platform fee**, which could include consumption-based pricing based on [active syncs](https://hightouch.com/docs/pricing/bt-pricing#active-syncs).
- Optional **feature add-ons**, such as [Customer Studio](/customer-studio/overview) features or the [Personalization API](/destinations/personalization-api).
This allows us to create a plan that fits your specific feature usage and consumption needs.
For the complete list of features included in Business tier packages, please visit our [pricing page](https://hightouch.com/pricing).
Don't hesitate to to receive a quote or discuss whether the Business tier is a good fit for you.
{/* */}
## SOC 2 report
If you are an existing Hightouch Business Tier customer, you can request a copy of our full SOC 2 report at the [Hightouch Trust Center](https://trust.hightouch.com/).
If you are not yet a Business Tier customer but are considering buying Business Tier, we can provide you the report under NDA.
## Active syncs
An **active sync** is any sync that has run at least once in the **previous calendar month**.
This includes syncs that are later deleted or disabled—if a sync ran successfully during that month, it is still counted as active. At the start of each new calendar month, the active sync count resets.
---
## Self-serve pricing
**URL:** https://hightouch.com/docs/pricing/ss-pricing
**Description:** Learn about our self-serve pricing and how Hightouch calculates usage based on active syncs.
**Section:** Pricing
{/* */}
{/* Allow for use of "We" in sensitive topics like pricing and security --> */}
This page describes Self-serve pricing (Free and Self-serve tiers).
Visit the [pricing page](https://hightouch.com/pricing) for a full feature comparison, or see the
[Business tier pricing](/pricing/bt-pricing) page to learn about Business plans.
## Overview
Hightouch’s self-serve pricing is based on the **number of [active syncs](https://hightouch.com/docs/pricing/ss-pricing#active-syncs)** in your organization.
## Plan limits
- **Free plan**: 2 active syncs per month.
- **Self-serve plan**: 10 active syncs per month.
There are **no limits** on destinations, destination types, user seats, or sync runs.
Free and Self-serve plans are limited to **hourly sync frequency**.
To ensure we can serve all customers sustainably, we impose an operations cap of 100,000,000 operations per month. Most workspaces fall under this limit. Please get in touch with us if you’re concerned about usage.
### Workspaces
[Organizations](/workspace-management/overview#core-concepts) on self-serve plans are limited to **one workspace per org**.
## Active syncs
An **active sync** is any sync that has run at least once in the **previous calendar month**.
This includes syncs that are later deleted or disabled—if a sync ran successfully during that month, it is still counted as active. At the start of each new calendar month, the active sync count resets.
{/* */}
---
## Hightouch MCP
**URL:** https://hightouch.com/docs/ai-integrations/mcp
**Description:** Connect your AI assistant directly to your marketing stack to build audiences, design journeys, generate creatives, and analyze performance in plain language.
**Section:** AI integrations
## Overview
The Hightouch MCP server connects your AI assistant directly to your marketing stack so your team can build audiences, design journeys, generate emails and ads, and analyze performance — all in plain language.
## What you can do
- **All customers:** Understand your customer schema, describe an audience or customer journey in plain English, iterate, and save it. Run marketing data analysis with Hightouch's AI agents.
- **Lifecycle Studio Customers:** Ideate campaign concepts tied to a specific goal. Build and iterate on email and SMS content.
- **Ad Studio Customers:** Generate visual ad creatives from a prompt, refine them with follow-up instructions, and browse existing projects.
## How it works
Connect to Hightouch's MCP from Claude, ChatGPT, or another AI assistant you already use, and build audiences, draft campaigns, or analyze performance through conversation. Your assistant uses Hightouch's MCP as needed to gather information and take action, and you can see results in real time, iterate, and save when you're happy with the outcome.
Example use cases:
- _"Create an audience of customers that added an item to their cart in the last day but haven't completed a transaction."_
- _"Show me which email campaigns drove the most conversions last quarter, and how did they perform compared to the quarter before?"_
- _"Create creatives for our new card promo targeted towards working adults in high-net-worth households."_
- _"Generate concepts for a Father's Day campaign to drive menswear purchases."_
## Why use the Hightouch MCP
**Work in your existing flow and across your stack.** If your team already works in Claude or another AI assistant, Hightouch is now accessible from that same interface. Because your assistant can connect to other tools simultaneously, you can orchestrate entire workflows in one conversation — get insights from Hightouch, draft a report, and send it to your team on Slack without switching tabs.
**Gives your assistant the context it needs.** The MCP server connects you to Hightouch's agents, which have all the relevant skills and context about your business and data to build the right audiences, journeys, and campaigns out of the box, without you having to re-explain your business context every time.
**Built for action.** Your assistant can create and save audiences, design journeys, and generate creatives and campaign concepts, not just describe what's possible.
## FAQ
**Is my data safe?**
Yes. All operations are scoped to your authenticated Hightouch workspace and respect your existing role-based access controls.
**Can the MCP take actions in my workspace, or is it read-only?**
Both. Your assistant can read your schema and explore your data, and also take actions — like creating and saving audiences, designing journeys, and generating creatives.
**Do we charge for MCP access?**
No. Access to the MCP is no additional cost.
**Do I need to be technical to use this?**
No. The MCP is designed for marketers. You interact in plain English, and the assistant handles the underlying logic.
**Which AI client should I use?**
Any AI client that supports the MCP standard can connect to Hightouch — including Claude, ChatGPT, Gemini Enterprise, Microsoft Copilot, and more.
---
## Snowflake CoWork
**URL:** https://hightouch.com/docs/ai-integrations/snowflake-cowork
**Description:** Use Snowflake CoWork to understand your Hightouch models
**Section:** AI integrations
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Data teams, marketers, and admins connecting Snowflake CoWork with Hightouch |
| **Prerequisites** |
A Snowflake account with required privileges (e.g., ACCOUNTADMIN or equivalent role to create Network Rules, Secrets, and Integrations).
A Hightouch API key.
At least one model configured in Hightouch.
|
*Use Snowflake CoWork to explore your Hightouch models and audiences using natural-language questions powered by your warehouse.*
---
## Overview
[**Snowflake CoWork**](https://www.snowflake.com/en/product/snowflake-cowork/) is Snowflake’s Enterprise Intelligence Agent, conversational AI assistant that helps you explore warehouse data and connected systems using natural language.
By integrating Snowflake CoWork with Hightouch, you can ask questions about your Customer Studio models, customer segments, activation campaigns, and audience performance directly from Snowflake.
This guide explains how to let Snowflake CoWork fetch **Hightouch model metadata** (such as model name, description, and query type) using the Hightouch [**Models API**](https://hightouch.com/docs/api-reference#operation/ListModel).
---
## What you'll learn
After reading this article, you’ll know how to:
- Understand how Snowflake CoWork uses Hightouch model metadata
- Configure Snowflake to call the Hightouch Models API
- Test the connection from Snowflake
- Ask questions in Snowflake CoWork about your Hightouch models and audiences
---
## How Snowflake CoWork works with Hightouch
Snowflake CoWork can call external APIs using Snowflake’s **External Access Integrations (EAIs)**.
When you authorize Snowflake to call the Hightouch Models API, Snowflake CoWork can:
- Retrieve a list of your Hightouch models
- Identify which models power visual audiences
- Read model descriptions and query types
- Reference this metadata when answering user questions
Hightouch continues to remain the system of record for model definitions—Snowflake only receives the metadata needed to answer questions.
Snowflake CoWork does *not* modify, create, or run Hightouch models.
It simply enriches your warehouse-level conversations with Hightouch model context.
---
## Example questions you can ask
Once connected, Snowflake CoWork can answer questions such as:
- "What audiences have we built that target churned customers?"
- "Which segments am I sending to my email platform?"
- “Which of my audiences was updated most recently?”
---
## Example setup: Calling the Hightouch Models API from Snowflake CoWork
This example shows how to configure Snowflake CoWork to call the Hightouch Models API.
You can extend this pattern to connect Snowflake CoWork with other Hightouch APIs like our Destinations API for activation insights or Events API for real-time customer behavior, creating a comprehensive conversational interface.
### 1. Use a Snowflake role with integration privileges
```sql
USE ROLE ACCOUNTADMIN;
USE DATABASE DEMO_DB;
USE SCHEMA PUBLIC;
```
---
### 2. Create a network rule
Allows Snowflake to connect to the Hightouch API.
```sql
CREATE OR REPLACE NETWORK RULE hightouch_rule
MODE = egress
TYPE = host_port
VALUE_LIST = ('api.hightouch.com:443');
```
---
### 3. Create a secret with your Hightouch API key
```sql
CREATE OR REPLACE SECRET hightouch_api_key
TYPE = generic_string
SECRET_STRING = '';
```
**Note:** Replace `` with your actual Hightouch API key. Create an API key in your [workspace settings →](https://app.hightouch.com/settings/api-keys?from=docs).
---
### 4. Create the External Access Integration
```sql
CREATE OR REPLACE EXTERNAL ACCESS INTEGRATION hightouch_eai
ALLOWED_NETWORK_RULES = (hightouch_rule)
ALLOWED_AUTHENTICATION_SECRETS = (hightouch_api_key)
ENABLED = TRUE;
```
---
### 5. Create a stored procedure to call the Hightouch Models API
```sql
CREATE OR REPLACE PROCEDURE CALL_HIGHTOUCH_API()
RETURNS STRING
LANGUAGE PYTHON
RUNTIME_VERSION = '3.10'
PACKAGES = ('requests','snowflake-snowpark-python')
EXTERNAL_ACCESS_INTEGRATIONS = (HIGHTOUCH_EAI)
SECRETS = ('cred' = HIGHTOUCH_API_KEY)
HANDLER = 'main'
AS
$$
def main():
"""
Calls the Hightouch models endpoint
"""
api_key = _snowflake.get_generic_secret_string('cred')
url = "https://api.hightouch.com/api/v1/models"
headers = {
"Authorization": f"Bearer {api_key}",
}
try:
resp = requests.get(url, headers=headers, timeout=30)
resp.raise_for_status()
return json.dumps(resp.json())
except Exception as e:
return f"Error: {str(e)}"
$$;
```
---
### 6. Test your connection
```sql
CALL CALL_HIGHTOUCH_API();
```
You should receive a JSON list of your Hightouch models.
If you receive an empty response, verify that the API key’s user has access to Models in Hightouch.
---
## Enable Snowflake CoWork to use the procedure
Create a Snowflake CoWork agent that can call the stored procedure.
```sql
CREATE OR REPLACE AGENT hightouch_agent
WITH PROFILE='{ "display_name": "Hightouch Agent" }'
COMMENT=$$ Enables users to interact with the Hightouch API $$
FROM SPECIFICATION
$$
{
"models": { "orchestration": "auto" },
"orchestration": {},
"instructions": {
"orchestration": "This is a simple agent for general knowledge with access to the Hightouch API"
},
"tools": [
{
"tool_spec": {
"type": "generic",
"name": "hightouch_models",
"description": "{\n \"name\": \"hightouch_models\",\n \"description\": \"Fetch Hightouch models. If the user asks about audiences, they are referring to models with query_type = 'visual'\",\n \"parameters\": { \"type\": \"object\", \"properties\": {}, \"required\": [] },\n \"example_payload\": {},\n \"stored_proc_call\": \"CALL CALL_HIGHTOUCH_API();\"\n }",
"input_schema": { "type": "object", "properties": {}, "required": [] }
}
}
],
"tool_resources": {
"hightouch_models": {
"execution_environment": { "type": "warehouse", "warehouse": "" },
"identifier": "DEMO_DB.PUBLIC.CALL_HIGHTOUCH_API",
"name": "CALL_HIGHTOUCH_API",
"type": "procedure"
}
}
}
$$;
```
---
## Troubleshooting
| Issue | Likely cause | Resolution |
| ----------------------------------------- | -------------------------------------------------------- | -------------------------------------------------------- |
| Empty API response | API key lacks required permissions | Use an admin-level API key or adjust permissions |
| Network rule error | Incorrect host or missing port | Ensure `api.hightouch.com:443` is configured |
| “EAI not found” | Integration wasn’t created in the active database/schema | Recreate using the same database/schema as the procedure |
| Snowflake agent cannot call the procedure | Missing tool spec or stored procedure reference | Recreate the agent using the example spec |
---
## What’s next
- [Explore your Hightouch models](/models/creating-models)
- [Learn more about the Hightouch Models API](/api-reference#operation/ListModel)
- [Review Snowflake’s guide to Snowflake CoWork](https://www.snowflake.com/en/product/snowflake-cowork/)