# Hightouch Documentation — Full Content
> This file contains the full text content of all public Hightouch documentation pages. It is generated automatically on every push to master.
---
## Hightouch Agents
**URL:** https://hightouch.com/docs/getting-started/agents
**Description:** Learn how Hightouch Agents help marketers answer questions, analyze performance, build audiences, and automate routine marketing work using complete data context.
**Section:** Getting started
## Overview
Hightouch Agents is an **AI platform** for marketers. Our agents are specifically designed for marketing workflows and tasks.
They can answer marketing questions, analyze campaign and creative asset performance, build audiences, and automate routine work like weekly reporting.
Agents are powered by complete context from your data warehouse and connected marketing tools, so answers and actions are based on your full customer and campaign data, not a single system.
---
## What you can do with Agents
Agents are designed for the types of questions and tasks marketers handle every day.
| Use case | Example |
|--------|---------|
| **Plan marketing campaigns** | “Design a campaign to sell through my overstock inventory.” |
| **Ask anything about your customers** | “In the last 30 days, what percentage of new account sign-ups opened a new brokerage account?” |
| **Analyze creative performance** | “How are the Facebook ads I launched last week performing based on click-through rate?” |
| **Discover new audiences** | “Give me three high-value audiences that will buy these at 20% off.” |
| **Automate reporting** | “Generate a report at 9AM PT every day listing my top 5 lifecycle campaigns along with send volume and deliverability stats.” |
---
## How Hightouch Agents work
Hightouch Agents are built on two core principles.
### 1. Complete context
Hightouch Agents connect to your data warehouse and marketing channels, ensuring insights are powered by all available information—not just data from a single tool.
Agents can access:
- **Warehouse data:** Products, customer transactions, inventory levels, and more
- **Customer-facing channels:** Advertising platforms, CRM, email, customer service tools, and more
- **Internal brand knowledge:** Brand guidelines, voice and tone, and marketing best practices
### 2. Purpose-built for marketers
Hightouch Agents are trained to handle complex marketing questions with high accuracy on the first try. They understand key marketing concepts such as:
- Creative fatigue and optimization
- Attribution modeling
- Audience segmentation
- Campaign performance analysis
By combining complete context with marketing expertise, Hightouch Agents help marketers make faster, smarter, and more data-driven decisions.
---
## Daily usage (Marketers & business teams)
**URL:** https://hightouch.com/docs/getting-started/business
**Description:** Learn where marketers work in Hightouch and how to get started with Customer Studio or AI Decisioning (AID).
**Section:** Getting started
| | |
| ----------------- | ------------------------------------------------------------------------------------------------------ |
| **Audience** | Marketers, business users |
| **Prerequisites** |
[Workspace setup](/getting-started/it)
[Data setup](/getting-started/data)
|
*Marketers use Hightouch daily to build audiences, launch campaigns, and personalize customer experiences. The two main product areas designed for marketing teams are Customer Studio and AI Decisioning (AID).*
---
## Customer Studio
**Customer Studio** is the no-code workspace for creating, activating, and measuring audiences.
- Build and sync audiences directly to your marketing tools
- Orchestrate cross-channel journeys and campaign triggers
- Measure results with Campaign Intelligence dashboards
→ [**Customer Studio quick start**](/customer-studio/quick-start)
---
## AI Decisioning (AID)
**AI Decisioning (AID)** automates campaign decision-making using machine learning.
- **Agents** determine the best channel, message, and timing for each customer
- **Insights** and **Inspector** provide visibility into AI-driven choices
- **Collections** and **Configuration** make campaign logic reusable and governed
→ [**AI Decisioning overview**](/ai-decisioning/overview)
---
## Extend your campaigns
Expand campaign capabilities with additional Composable CDP products.
- **Hightouch Events**: Trigger campaigns based on customer actions.
- [**Batch events**](/events/overview) run on scheduled data loads for broad campaigns (e.g., daily win-back emails).
- [**Real-time events**](/real-time/real-time) update audiences and trigger campaigns within seconds (e.g., send a push notification after a loyalty redemption).
- [**Identity Resolution**](/identity-resolution/overview): Unify customer profiles across devices and channels for consistent personalization.
- [**Match Booster**](/match-booster/overview): Improve ad platform match rates by enriching records with additional identifiers (such as phone numbers or emails).
Learn how these products fit together in the [**Composable CDP overview →**](/getting-started/cdp)
---
## Composable CDP
**URL:** https://hightouch.com/docs/getting-started/cdp
**Description:** Learn how the Hightouch Composable CDP connects your data warehouse to marketing, analytics, and personalization tools—enabling data teams to stay in control while marketers build and activate campaigns from trusted data.
**Section:** Getting started
## What is the Composable CDP?
The **Composable CDP** in Hightouch connects your data warehouse directly to the tools your teams use.
- **Data teams** define reusable datasets while keeping data governed at the source.
- **Marketers** use those datasets in a visual builder to create audiences, sync to marketing tools, build automated workflows, and measure results.
Unlike traditional CDPs, Hightouch doesn’t copy or store data in a separate system. It runs on top of your existing infrastructure, giving marketers direct access while allowing data teams to stay in control.
The Composable CDP is also **modular**: you can adopt only the products you need.
[**Explore the interactive architecture diagram →**](https://architecture.hightouch.com/?from=docs&product=overview&utm_source=hightouch-docs&utm_medium=referral&utm_campaign=architecture-explorer&utm_content=overview)
See how data flows and where each product fits in your stack.
## Product suite
### Reverse ETL
Send modeled data from your warehouse to hundreds of destinations.
- Define models with SQL, dbt, BI tools, or Hightouch’s visual builder
- Schedule syncs or trigger them when data changes
- Track syncs easily with built-in monitoring and logs
**Example**: Sync churn-risk users to Braze for reactivation campaigns
→ [**Reverse ETL overview**](/getting-started/concepts)
### Customer Studio
Build and explore audiences with a **visual, no-code interface**.
- Define conditions based on traits, events, or relationships
- Preview audience size and membership instantly
- Use trusted data prepared by your data team
- Orchestrate Journeys: automate cross-channel campaigns triggered by events or audience membership
**Example**: Target users who viewed a product but didn’t complete checkout
→ [**Customer Studio overview**](/customer-studio/overview)
### Intelligence
Measure and improve campaigns with built-in charts and dashboards.
- **Test strategies** with audience splits and holdouts
- **Compare performance** across traits, audiences, and treatments
- **Use dashboards** to track results over time and drill into key metrics
**Example**: Compare conversion rates for discount vs. no-discount offers, then monitor performance in a dashboard
→ [**Intelligence overview**](/campaign-intelligence/overview)
### Events
Trigger workflows when customers take action by streaming or batching event data from your apps and websites.
- **Batch events**: Use scheduled data (e.g., purchases, cancellations) to power campaigns like win-back emails.
- **Real-time events**: Stream user actions to enable same-session personalization, such as showing an offer while the user is still browsing.
**Example**
- Send a cart abandonment email when a user leaves checkout
→ [**Events overview**](/events/overview)
→ [**Real-time overview**](/real-time/real-time)
### Identity Resolution
Bring customer data together into a single, unified profile.
- Combine web, app, and offline activity
- Enable consistent personalization and reporting
**Example**
- Merge app activity and ecommerce purchases into one profile
→ [**Identity Resolution overview**](/identity-resolution/overview)
### Match Booster
Improve match rates on ad platforms by enriching your records with additional identifiers (such as phone numbers or emails).
**Examples**
- Add phone numbers to an email list before syncing to Meta
→ [**Match Booster overview**](/match-booster/overview)
---
## Data activation overview
**URL:** https://hightouch.com/docs/getting-started/concepts
**Description:** Learn the foundational concepts in Hightouch—sources, models, syncs, destinations, and change data capture—that power Reverse ETL, the Composable CDP, and AI Decisioning.
**Section:** Getting started
Hightouch is a **data and AI platform for marketing and personalization**. At its core, Hightouch moves data from your warehouse into downstream tools. This process is called **Reverse ETL**.
Reverse ETL runs on three building blocks—**sources**, **models**, and **syncs**—along with the **destinations** where data is activated. These components form the foundation for both [Composable CDP](/getting-started/cdp) and [AI Decisioning](/ai-decisioning/overview).
[**Explore the interactive architecture diagram →**](https://architecture.hightouch.com/?from=docs&product=reverse-etl&utm_source=hightouch-docs&utm_medium=referral&utm_campaign=architecture-explorer&utm_content=reverse-etl)
See how data flows and where Reverse ETL fits in your stack.
---
## Sources
A **source** is any system where your data resides. Common examples include Snowflake, BigQuery, Databricks, or PostgreSQL. Sources can also be files, APIs, or BI tools.
***For marketers**: this is where customer and business data lives, such as purchase history or campaign activity.*
→ [**Sources overview**](/sources/overview)
---
## Models
A [**model**](/models/creating-models) defines what data to query from a source. Models can represent users, events, products, or any other entity.
Data teams typically create models by:
- Writing SQL in the editor
- Selecting from tables or views
- Re-using dbt models or BI queries (Looker, Sigma)
Every model requires a **unique primary key** to identify each row and track changes between syncs.
***For marketers**: think of a model as a reusable dataset like “all active customers,” “abandoned carts,” or “email subscribers.”*
→ [**Models overview**](/models/creating-models)
### Models in Customer Studio
Models also surface in [**Customer Studio**](/customer-studio/overview) through the [**schema**](/customer-studio/schema).
- Data teams expose traits, events, and relationships from underlying models into the schema.
- Marketers then use this schema to build [**audiences**](/customer-studio/usage) and [**journeys**](/customer-studio/journeys) in a no-code UI.
***For marketers**: schema is how you see and use model data in the audience builder without writing SQL.*
→ [**Customer Studio schema setup**](/customer-studio/schema)
---
## Syncs
A [**sync**](/docs/syncs/overview) defines *how data appears in the destination and when*.
Syncs specify:
- **Type**: object, event, audience, etc.
- **Mode**: insert, update, upsert, or archive
- **Mapping**: how source columns map to destination fields
- **Schedule**: interval, cron, or triggered by tools like dbt Cloud or Airflow
Data teams configure syncs; marketers rely on them to keep tools current. Multiple syncs can be created from the same model to support different use cases.
***For marketers**: a sync is how that dataset shows up in tools like Braze, Salesforce, or Meta Ads so you can run campaigns with it.*
→ [**Syncs overview**](/syncs/overview)
---
## Destinations
A **destination** is any system where data is consumed—CRMs, ad platforms, support tools, analytics, or custom APIs.
***For marketers**: these are the tools where you build campaigns, measure results, or engage with customers.*
→ [**Destinations overview**](/docs/destinations/overview)
---
## Change data capture (CDC)
Hightouch optimizes syncs with **change data capture (CDC)**. Instead of sending the full query result on every run, CDC compares the current results to the previous run and only sends:
- New rows
- Changed rows
- Removed rows
***For marketers**: this means your tools only update when something changes, so you’re not wasting budget or flooding campaigns with unnecessary updates.*
By default, CDC runs on Hightouch-managed infrastructure.
For higher performance at scale, data teams can enable the [Lightning Sync Engine](/syncs/lightning-sync-engine) to compute CDC directly in the warehouse.
→ [**Change data capture overview**](/syncs/cdc)
---
## How it fits together
1. Connect a **source** (e.g. Snowflake)
2. Define a **model** (e.g. “active customers in the last 30 days”)
3. Configure one or more **syncs** (e.g. to HubSpot, Braze, or Meta Ads)
4. Data flows into **destinations** where teams act on it
These foundations power Hightouch’s products:
- [Composable CDP](/getting-started/cdp) — modular tools for building audiences, orchestrating journeys, and measuring results
- [AI Decisioning](/ai-decisioning/overview) — AI agents that decide the right message, channel, and timing for each user
---
## Next steps
**Data teams**: Get started with the [**Quick start guide**](/getting-started/data).
- Set up your first [Source](/docs/sources/overview)
- Create a [Model](/models/creating-models)
- Configure a [Sync](/syncs/overview)
- Explore [Destinations](/destinations/overview)
- Learn how these concepts apply in the [Composable CDP](/getting-started/cdp) and [AI Decisioning](/ai-decisioning/overview)
---
## Technical setup (Data teams & engineers)
**URL:** https://hightouch.com/docs/getting-started/data
**Description:** Connect your data warehouse, configure sync infrastructure, and prepare reusable datasets in Hightouch.
**Section:** Getting started
| | |
| ----------------- | ------------------------------------------------------------------------------------------------------ |
| **Audience** | Data engineers, analytics teams |
| **Prerequisites** |
[Platform and workspace setup](/getting-started/it)
|
*Connect your data warehouse, configure sync infrastructure, and prepare datasets so marketers can build and activate audiences directly in Hightouch.*
Before you begin, review the [**Data activation concepts**](/getting-started/concepts), which explain sources, models, syncs, destinations, and Change Data Capture (CDC). This article focuses on the hands-on setup work for data teams.
---
## Setup steps
### 1. Connect your data
Connect your warehouse as the **source of truth** for all activation.
1. Add your primary data source (e.g., Snowflake, Databricks, BigQuery, Redshift) from **Integrations → Sources**.
2. **(Optional)** Connect additional data sources such as databases, APIs, or SaaS tools if they’re part of your activation strategy.
→ [**Sources overview**](/sources/overview)
*Hightouch syncs modeled data from your warehouse to destinations through sources, models, and syncs.*
---
### 2. Configure sync infrastructure
Set up the systems that power reliable, performant syncs between your warehouse and marketing tools.
1. [**Lightning Sync Engine**](/syncs/lightning-sync-engine)
Use [Change Data Capture (CDC)](/syncs/cdc) to detect and sync only changed records. This enables features like Identity Resolution, warehouse sync logs, and audience holdouts.
2. [**Warehouse Sync Logs**](/syncs/warehouse-sync-logs)
Write sync metadata—such as row counts, errors, and rejected records—back to your warehouse for monitoring and visibility.
3. [**Destinations**](/destinations/overview)
Connect downstream tools (e.g., Braze, Salesforce, ad platforms) where marketers will activate audiences and campaigns.
---
### 3. Define your schema
Define the data structure that marketers will use in Customer Studio to build audiences and campaigns.
This is typically a **one-time setup owned by data teams**.
- **Parent model:** The main dataset (for example, your `users` table)
- **Related models:** Supporting entities such as accounts, subscriptions, or products
- **Events:** Timestamped actions (purchases, logins, cancellations)
- **Relationships:** Connections between models via foreign keys
→ [**Define data schema**](/customer-studio/schema)
*Data teams define parent, related, and event models. Marketers use these attributes in Customer Studio to build audiences.*
---
### 4. Security and change management
Security and workspace-level governance are usually managed by IT or platform admins.
As a data team, coordinate with IT to ensure:
- **Environments** are available for staging models and syncs
- **Audit logs** and **approval flows** are enabled per compliance policies
→ [**Workspace setup (IT & admins)**](/getting-started/it)
---
### 5. Extend your setup (optional)
Add additional Composable CDP products to enhance audience accuracy, personalization, and activation.
- [**Identity Resolution**](/identity-resolution/overview): Bring customer data together into unified profiles by merging records across identifiers (email, phone, device ID).
- [**Events**](/events/overview): Stream or batch event data from your apps or warehouse to trigger campaigns when customers take action.
- [**Match Booster**](/match-booster/overview): Improve ad platform match rates by enriching records with additional identifiers (e.g., emails or phone numbers).
Learn how these products fit together in the [**Composable CDP overview →**](/getting-started/cdp)
---
## Workspace setup (IT & platform admins)
**URL:** https://hightouch.com/docs/getting-started/it
**Description:** Set up secure access, governance, and user roles in your Hightouch workspace so data and marketing teams can activate safely.
**Section:** Getting started
| | |
| ----------------- | ------------------------------------------------------------------------------------------------------ |
| **Audience** | IT teams, admins, platform owners |
| **Prerequisites** |
|
*Configure your Hightouch workspace to ensure secure access, clear permissions, and consistent governance across teams.*
---
## Setup checklist
### 1. Learn the building blocks
Before configuring anything, familiarize yourself with how Hightouch is organized — organizations, workspaces, user groups, roles, environments, and Spaces.
→ [**Core concepts**](/workspace-management/overview) — organizations, workspaces, user groups, roles, environments, and Spaces
*Use one workspace per brand or region, with environments for staging and production separation.*
---
### 2. Create your workspace
Set up your organization and first workspace. Choose a region, pick a workspace name and slug, and walk through the onboarding checklist.
→ [**Workspaces**](/workspace-management/workspaces) — create, configure, switch, and delete workspaces
---
### 3. Set up access control
Configure how your team authenticates and what they can access. Set up SSO and SCIM to automate provisioning, map identity provider groups to Hightouch user groups, and assign roles.
→ [**SSO setup**](/workspace-management/sso) — connect your identity provider for centralized authentication
→ [**Manage users and groups**](/workspace-management/user-management) — invite users, assign groups, manage membership
→ [**Roles**](/workspace-management/roles) — pre-built and custom permission sets
→ [**Transfer org admin access**](/workspace-management/user-management#transferring-organization-admin-access) — hand off admin privileges when team members change
*User groups manage access across workspaces, with roles assigned at the group level.*
Start with pre-built roles during onboarding. Move to custom roles once your org structure or compliance needs require it.
---
### 4. Plan your governance
Decide how teams should collaborate within your workspace — who can publish changes, what data they can reach, and what requires approval. Hightouch offers several controls that layer on top of each other.
→ [**Governance best practices**](/workspace-management/governance) — when to use each control and how they layer together
→ [**Environments**](/workspace-management/environments) — dev/staging/prod promotion within a workspace
→ [**Approval flows**](/workspace-management/approval-flows) — require review before changes go live
→ [**Spaces**](/workspace-management/spaces) — scope resource visibility across teams
→ [**Audit logs**](/workspace-management/audit-logs) — track all in-app user actions for 90 days
---
### 5. Secure your infrastructure
Configure storage, networking, and monitoring to meet your security and compliance requirements.
→ [**Security overview**](/security/overview) — encryption, compliance, and hosting details
→ [**Storage**](/security/storage) — configure self-hosted storage for data-at-rest
→ [**Networking**](/security/networking) — tunnels, PrivateLink, IP allowlisting
→ [**Alerting**](/syncs/alerting) — sync failure notifications via Slack, email, PagerDuty, or Datadog
→ [**Sync logs**](/syncs/warehouse-sync-logs) — write sync history to your warehouse
---
### 6. Organize resources (optional)
As your workspace grows, use folders, labels, and filters to keep things navigable.
→ [**Folders, filters, and labels**](/workspace-management/folders-filters)
---
## Welcome
**URL:** https://hightouch.com/docs/getting-started/welcome
**Description:** Hightouch is a data and AI platform for marketing and personalization. Learn how to get started with workspace setup, technical setup, and daily usage.
**Section:** Getting started
## What is Hightouch?
Hightouch is a **data and AI platform for marketing and personalization**.
It helps teams unlock customer data and optimize campaigns in two core product areas:
### Composable CDP
Give marketers access to warehouse data so they can build and activate audiences. Choose the components you need:
- Build audiences in **Customer Studio**
- Deliver them to tools through **Reverse ETL**
- Unify profiles with **Identity Resolution**
- Enrich reach with **Match Booster**
- Trigger campaigns with **Events** (real-time or batch)
- Measure performance with **Campaign Intelligence**
Learn more → [**Composable CDP overview**](/getting-started/cdp)
### AI Decisioning (AID)
Use **AI agents** to decide the best message, channel, and timing for each user, and analyze results with AI-powered insights.
Learn more → [**AID overview**](/ai-decisioning/overview)
***
## Choose your path
| Persona | Quick Start |
|-------------------|------------------------------------------------------------------------------|
| **Platform admins and IT** | [Set up Hightouch for your org](/getting-started/it) → |
| **Data teams and engineers** | [Data activation concepts](/getting-started/concepts) → [Connect data and schema](/getting-started/data) → |
| **Marketers and other business users** | [Build and run marketing campaigns](/getting-started/business) → |
---
## Models overview
**URL:** https://hightouch.com/docs/models/creating-models
**Description:** Create Hightouch models using the SQL editor or the table selector which queries existing tables and views within your source.
**Section:** Models
To learn about defining data models for Audiences, refer to the [Customer Studio documentation](/customer-studio/schema).
Hightouch models define and organize the data you want to query from a [source](/getting-started/concepts).
To create a new model, navigate to the [**Models** page](https://app.hightouch.com/models) and click **Add model**. Next, select one of you the sources you've setup. Then choose a modeling method:
- SQL editor
- Table selector
- dbt model selector
- Looker
- Sigma
Supported modeling methods depend on the source. Refer to source documentation to learn about supported modeling methods.
Refer to the [SQL editor](/models/sql-editor) and [table selector](/models/table-selector) docs for details on how to use those modeling methods.
Refer to the Extensions documentation to learn more about how to connect Hightouch to these platforms and build Hightouch models using existing [dbt models](/extensions/dbt-models), [Looker Looks](/extensions/looker-models), or [Sigma workbooks](/extensions/sigma).
## Unique primary key requirement
Regardless of which method you use to build your models, you must always designate a column as the model's primary key, which is a non-repeating, non-null value that uniquely identifies each row. For example, if you have a table containing customer profiles, you might have a column named `CustomerID` that distinguishes each customer from all the others. (While several customers can share the same first name, location, or even phone number, they cannot share the same `CustomerID`.)
Select your primary key column when prompted during model setup.
If your dataset doesn't include any truly unique columns, you can use the [SQL editor](/models/sql-editor) to either [filter out](/models/sql-editor#filter-out-duplicates) duplicate rows or create a [composite column](/models/sql-editor#composite-primary-keys) to use for your primary key. If you're not using the SQL modeling method or if SQL isn't supported by your source, you need to make changes to your data upstream to ensure it includes a unique column.
Rows with `null` primary keys will be ignored by the sync.
Hightouch's sync engine relies on unique primary keys to detect when rows are added, changed, or removed in a data model. See the [change data capture](/getting-started/concepts#change-data-capture) docs to learn how Hightouch accomplishes this.
As a safeguard to prevent duplicate records from reaching to your destination, **Hightouch will automatically filter out rows with non-unique primary keys**. In the sync logs, they will be marked as “Rejected”, just like any other row-level error. It is safer to reject these rows than to risk syncing wrong or duplicate data.
Make sure to read through the [primary key updates](#updating-a-primary-key) section before making any changes to your primary key.
### Updating a primary key
Hightouch relies on unique primary keys to detect when rows are added, changed, or removed. Therefore, altering the primary key requires Hightouch to reset its [change data capture (CDC)](/getting-started/concepts#change-data-capture) for syncs that depend on your model.
When changing a primary key, you will be asked whether or not a backfill should be performed. Two options are available:
**Reset CDC without backfill**: By electing to reset CDC without a backfill, you are instructing Hightouch to ignore the current state of your model and to track only rows that are added, changed, or removed in the future. The next sync run will be used as an opportunity to capture a snapshot of your model, but no data will be sent to the destination. Thereafter, future sync runs will track new changes only. (This option is recommended for insert-only use cases like conversion events, operational alerts, and other situations where a backfill would create undesired duplicate records. Beware that skipping the backfill may cause your destination to drift out of sync with your data model.)
**Reset CDC with backfill (i.e., [full resync](/syncs/overview#resync-full-query))**: By electing to reset CDC with a backfill, you are instructing Hightouch to perform a full resync of your entire data model. This means that Hightouch will not perform any diffing and will instead process every row as if it's the first time the sync has ever run. All rows will be sent to the destination again. (This option may be preferred for syncs to CRMs and other services that are configured to update existing records instead of creating new ones. Do not use this option if a backfill will create duplicates or trigger downstream actions.)
By default, proceeding with your primary key change will immediately trigger a run for every sync that depends on your model. It is necessary to reset CDC as quickly as possible after the primary key is updated. However, you can opt out of this behavior, in which case the CDC reset will be deferred until the next sync run.
You don't need to manually trigger a full resync if you change the primary key column's [data type](/models/data-types-casting#view-model-data-types). If you change the primary key's data type in the [model configuration](/models/data-types-casting#casting-from-your-model-configuration), your sync will process normally. If you make this change in your source or in the [SQL editor](/models/sql-editor#unique-primary-key-requirement), the entire model query result set is automatically resynced as if you triggered a full resync. As outlined in the [full resync prerequisites](https://hightouch.com/docs/syncs/overview#full-resync-prerequisites) section, this can create duplicates in your destination data. You may wish to reset change data capture without a backfill instead.
## Row and column ordering
Hightouch’s sync engine is optimized for performance and scale. It divides data into batches and processes them in parallel so large syncs complete quickly. This distributed design means rows and columns may be sent to destinations in a different order than they appear in your model.
This is normal behavior and does not affect which records are synced or the values written. If you notice that files or datasets appear in a different order between runs, that variation is expected. Any apparent ordering you observe is incidental and not guaranteed to remain consistent over time.
This applies to all destination types, including spreadsheets and file uploads. Even if you use a custom SQL model with an `ORDER BY` clause, that ordering applies only when previewing the model. The order in which rows and columns are processed during syncs can still vary.
## Data types and casting
When you define a model, Hightouch doesn't change the [data types](/models/data-types-casting#data-types) found in your [source](/getting-started/concepts#sources) unless otherwise specified.
You can view a column's type in a model's **Columns** tab.
To safeguard your syncs from failing, the data types your model returns must align with your [destination's](/getting-started/concepts#destinations) data type expectations.
If your source data types don't match your destination's expectations, you can use [data casting](/models/data-types-casting#casting) while setting up your models.
Hightouch intentionally stringifies your chosen [**primary key**](/getting-started/concepts#unique-primary-key-requirement) column for enhanced performance during [change data capture](/getting-started/concepts#change-data-capture).
If you need to sync the primary key column as a non-string value, use [SQL aliasing](/models/data-types-casting#sql-aliasing) in your model to create a new column specifically for syncing.
## Preview model results
When first defining a model, it's highly recommened to **Preview results** before you continue with setup. Previewing lets you validate the data your model returns before syncing it to downstream destinations.
Previewing a model's query results can also be helpful when troubleshooting issues. To preview an existing model's results, open the model's overview page and click **Edit**.
You can then select to **Preview results**.
If you want to display all model rows, click the gear symbol, then toggle off **Limit preview to 100 rows**.
You can also turn off **Show row count** to improve loading performance for larger data models.
## Column descriptions
Hightouch automatically pulls in column descriptions from supported sources. Column descriptions are supported for
Snowflake, BigQuery , Databricks, and other SQL-based sources that provide the standard `INFORMATION_SCHEMA.COLUMNS`
table, such as PostgreSQL, MySQL, MSSQL, and their cloud-managed variants. You can add descriptions to your models'
columns so that team members (and your future self) can better understand the data. Go to a model's **Columns** tab and
click the a column's **description** to edit it. If you edit a description in Hightouch, your version displays instead
of the comment on the source column.
If you're using the [dbt extension](/extensions/dbt-models) to build models, Hightouch automatically [pulls in columns descriptions](/extensions/dbt-models#model-column-descriptions) from dbt.
### Updating column descriptions from the source
To update column descriptions from your source, you can navigate to any model from that source -->
click the `Refresh columns available in source` button.
Alternatively, you can trigger this refresh by navigating to the table selector and refreshing the table list.
The table selector is available in two places:
1. Activation -> Models -> any model within the source
2. Customer Studio -> Schema -> select a model -> Query -> edit.
Once in the table selector, click the refresh icon to refresh the tables available from that source, this also updates the column descriptions.
## Clone models
After creating a model, you can clone it by opening the horizontal three-dot menu on its overview page and clicking **Clone**.
This opens a new page where you can:
- edit and preview the cloned model's query
- select which syncs to clone, if the model you're cloning has syncs associated with it
- give it a custom name and description
- confirm which model column to use as the [primary key](/getting-started/concepts#unique-primary-key-requirement)
Once you finish creating the cloned model, you can further edit it and configure additional syncs on it, like any other model.
---
## Data types and casting
**URL:** https://hightouch.com/docs/models/data-types-casting
**Description:** Third-party APIs require the data you send to be of a specific data type. Unless otherwise specified, Hightouch takes the data from your source and leaves the data type unchanged to send to your destination. You can choose to cast data into different types using SQL.
**Section:** Models
## Overview
When syncing data to a destination, their underlying APIs require the data you send to be correctly typed. For example, a `Name` field may require a `string` data type, while a `timestamp` field may require a particular `datetime` format. If you try to sync an incorrect type, for example, a `number` type for an identifier, when a `string` type is expected, APIs usually reject the request, and the sync fails.
When you define a [model](/getting-started/concepts#models), Hightouch doesn't change the data types found in your [source](/getting-started/concepts#sources) unless otherwise specified. To safeguard your syncs from failing, the data types your model returns must align with your destination's data type expectations. If your source data types don't match your destination's expectations, you can use [casting](#casting) while setting up your models.
## Data types
Hightouch recognizes the following data types for [model columns](#view-model-data-types):
- Boolean
- Number
- String
- Timestamp
- Date
- Object / Array
And the following for [destination fields](#view-destination-data-types):
- Boolean
- Number
- String
- Timestamp
- Date/time
- Associated field (linked to another table)
- Null value
- Unknown type
The Hightouch UI uses these icons to distinguish various data types for destination fields:
### View model data types
If you don't know the data types of the columns returned from your model, you can view them from the **Columns** tab on the model's overview page:
You can also [change the data types](#casting-from-your-model-configuration) returned from a query here.
Make sure to read through the [primary key
updates](/models/creating-models#updating-a-primary-key) section before making
any changes to your primary key.
Hightouch intentionally stringifies your chosen [**primary
key**](/getting-started/concepts#unique-primary-key-requirement) column for
enhanced performance during [change data
capture](/getting-started/concepts#change-data-capture). If you need to sync
the primary key column as a non-string value, use [SQL
aliasing](#sql-alaising) in your model to create a new column specifically for
syncing.
### View destination data types
When creating a sync configuration, you can view the destination fields' expected data types by looking at their icons when setting up [field mappings](/syncs/mapping-data):
## Casting
For your data to reach its destination without error, its data type must match the data type required by the destination. You can use **[type casting](https://en.wikipedia.org/wiki/Type_conversion)** to change column data types. You have these options:
- Click and point casting from a model's **Columns** tab: suitable for all [modeling methods](/models/creating-models)
- Casting via SQL in your model definition: suitable if you've built your model using the [SQL editor](/models/sql-editor) and prefer to encode casting in your SQL query
- SQL aliasing: recommended if your [primary key](/getting-started/concepts#unique-primary-key-requirement) data type is incompatible with the expected destination data type
### Casting from your model configuration
1. From the [**Models** overview page](https://app.hightouch.com/models), select the model whose columns you want to cast.
2. Select the **Columns** tab.
3. Use the picklist to the right of each column to choose the data type you want for that column.
### Casting in SQL
You can cast values from one data type to another when writing the SQL query that defines a model.
For example, this query casts a column called `total_storage_used_mb` as an integer:
```sql
SELECT *,
CAST (total_storage_used_mb AS int)
FROM public.organizations;
```
For more on SQL casting, see [this guide on SQL type casting](https://learn.microsoft.com/en-us/sql/t-sql/functions/cast-and-convert-transact-sql?view=sql-server-ver16/).
### SQL aliasing
If your model's primary key isn't a string, Hightouch stringifies it for enhanced record-matching performance. If stringification makes your primary key incompatible with the data type your destination is expecting, you can use **SQL aliasing** to create an additional field for mapping.
For example, if you use a field called `id` as your primary key, Hightouch stringifies its value to perform the matching and [diffing operations](/getting-started/concepts#change-data-capture).
Many destinations expect a field called `id` to be a string. Some destinations, however, expect `id` to be an integer. This mismatch causes a data type conflict if you need to map your primary key in addition to using it for matching.
Using SQL aliasing and casting can solve this issue. This example query selects the `customer_id` column twice. The second time, it casts the column as an `int`, and aliases it as a new `id_for_mapping` column.
```sql
SELECT customer_id, CAST(customer_id AS int) AS id_for_mapping
FROM customers;
```
The query results will now include a string `customer_id` column and an integer `id_for_mapping` column. You can then use `customer_id` as the primary key in your model setup and `id_for_mapping` for field mapping in a sync configuration.
---
## Match boosting
**URL:** https://hightouch.com/docs/models/match-boosting
**Description:** Increase match rates for your audiences in advertising platforms.
**Section:** Models
Match Booster improves **match rates** on [various paid advertising platforms](#supported-destinations) by providing additional user identifiers. Learn more in the main [Match Booster documentation](/match-booster/overview).
## Implementation steps
Using Match Booster is a two-step process:
1. Enable and configure match boosting on relevant models.
2. Configure match boosting in sync configurations to [supported destinations](/match-booster/overview#supported-destinations).
### Model configuration
Match boosting begins with your model. By enabling and configuring Match Booster on particular models, you can enrich them with additional identifiers from Hightouch's data partners.
1. To get started, go to your [**Models** overview page](https://app.hightouch.com/models/). If you want to enable matching boosting on an audience, go to its parent model on the **Schema** page.
2. Select the model on which you want to enable Match Booster and open the **Match boosting** tab.
3. Toggle **Enable match boosting** on.
4. Select the identifiers you want Hightouch to use to find additional data. This includes selecting an **identifier type** and mapping it to a **column name** in your model. The data in the columns you select should [have unhashed data](/match-booster/faqs#what-format-should-source-identifiers-use).
5. Click **Save**.
Model enrichment—the process of adding additional identifiers to your model
results—can take up to a few hours to process.
Once you've enabled match boosting on a model, you can configure [match boosting on syncs](/syncs/match-boosting) based on the model.
Check out the main [Match Booster documentation](/match-booster/faqs) page for
FAQs.
---
## Resolve SQL compilation errors
**URL:** https://hightouch.com/docs/models/sql-compilation-errors
**Description:** Troubleshoot SQL compilation errors that appear in model previews, Customer Studio audiences and journeys, or sync runs.
**Section:** Models
SQL compilation errors in Hightouch usually come from your warehouse when Hightouch runs a query to preview a model, compute an audience, evaluate journey logic, or execute a sync.
In most cases, Hightouch is surfacing the warehouse's error message directly.
## Where these errors appear
You may see SQL compilation errors in:
- Model preview and model edits
- Customer Studio audience preview or count calculations
- Customer Studio journeys that evaluate audience/event conditions
- Sync run failures
## Quick checklist
When you hit a SQL compilation error:
1. Copy the full error message and note object names, column names, and line/position details.
2. Run the same SQL in your warehouse using the same role/context as Hightouch.
3. Classify the error pattern below.
4. Fix the issue at the right layer (warehouse object/permissions vs model/filter/mapping config).
5. Re-run model preview, then re-run the audience/journey/sync flow.
## Common error patterns
### Invalid identifier (including `_ht_to_filter`)
Example patterns:
- `SQL compilation error: invalid identifier ...`
- `invalid identifier '"_ht_to_filter"."COLUMN_NAME"'`
What it means:
- A referenced column is missing from the query result Hightouch is filtering or mapping.
Common causes:
- Column renamed or removed in model SQL.
- Audience filter, trait, journey condition, or sync mapping still references old column.
How to fix:
1. Confirm the column still exists in model output.
2. Update filters/conditions/mappings that reference the old name.
3. Preview and re-save the model if metadata is stale.
### Unresolved column with suggestions
Example pattern:
- `[UNRESOLVED_COLUMN.WITH_SUGGESTION] ... cannot be resolved. Did you mean ...`
What it means:
- A referenced column name does not exist in the active query output.
Common causes:
- Typo in a field name.
- Renamed/removed model column (for example after changing from `SELECT *` to an explicit column list).
- Stale audience, trait, journey filter, or suggestion-job configuration.
How to fix:
1. Verify the column exists in the parent model/event model output.
2. Preview and re-save the model to refresh available columns.
3. Update filter logic to use valid column names.
### Ambiguous column name
Example pattern:
- `SQL compilation error: ambiguous column name 'COLUMN_NAME'`
What it means:
- The final query references a column name that exists in multiple joined sources, and the warehouse cannot infer which one you intended.
Common causes:
- Multiple tables expose the same column name and the model output does not disambiguate with aliases.
- Downstream filters/mappings reference a shared name after query wrapping.
How to fix:
1. Alias ambiguous columns in model SQL (for example, `a.faq_url AS faq_url_from_a`, `b.faq_url AS faq_url_from_b`).
2. Update filters, traits, journeys, and sync mappings to reference the intended alias.
3. Re-run preview and dependent workflows.
### Object does not exist or not authorized
Example pattern:
- `Object 'DB.SCHEMA.OBJECT' does not exist or not authorized`
What it means:
- Warehouse object is missing, moved, renamed, or inaccessible to the configured role.
How to fix:
1. Verify object path (database/schema/object) is correct.
2. Confirm the Hightouch role has required `USAGE`/`SELECT` privileges.
3. Update model SQL or restore the object.
### View definition column mismatch
Example pattern:
- `View definition ... declared N column(s), but view query produces M column(s)`
What it means:
- The warehouse view definition is out of sync with the underlying query.
How to fix:
1. Recreate or repair the view in the warehouse.
2. Prefer explicit column lists over `SELECT *` in long-lived views.
3. Re-test the view in warehouse SQL before re-running in Hightouch.
### Invalid data type for predicate / row comparison
Example pattern:
- `Invalid data type ... for predicate [ROW(...)]`
What it means:
- Filter predicates compare incompatible data types.
How to fix:
1. Inspect predicate/filter logic for mixed types.
2. Align types with explicit `CAST(...)` where needed.
3. Re-test in warehouse, then in Hightouch.
### Works in warehouse UI but fails in Hightouch
What to check:
- Role differences between your user and Hightouch connection.
- Different default database/schema/warehouse context.
- Model points to a different object/version than expected.
- SQL succeeds in UI context but fails under Hightouch role with a standard pattern above.
How to fix:
1. Re-run under the same role/context as Hightouch.
2. Use fully qualified names (`DATABASE.SCHEMA.OBJECT`) in SQL.
3. Confirm Hightouch model references the intended object.
## When to contact support
Contact support with:
- Link to model/audience/journey/sync run
- Full SQL compilation error text
- Whether the error reproduces in warehouse SQL with the same role/context
---
## Create models with the SQL editor
**URL:** https://hightouch.com/docs/models/sql-editor
**Description:** Create Hightouch models using the SQL editor.
**Section:** Models
With the SQL editor modeling method, you can write a SQL query to define your model. The SQL editor can execute whatever SQL is native to your warehouse or database.
If you encounter warehouse query failures while previewing or running downstream syncs, see [Resolve SQL compilation errors](/models/sql-compilation-errors).
Your query depends on the data in your model. For example, if you are using the [Sample B2B SaaS source](/sources/sample-data), you can use the following query to select the entire `public.users` table:
```sql
SELECT * FROM public.users
```
Before you can continue, you must click **Preview** to see a preview of the data. By default, Hightouch limits the preview to the first 100 rows. Once you've confirmed that the preview contains the data you expect, click **Continue**.
You can't save a model if your query doesn't return any results. If you need to save a model with such a query because you expect there to be results in the future, see the [save a model without results section](#save-a-model-without-results).
The last step for model setup is to enter a descriptive **Name** and select the column to use as a [**Primary key**](/getting-started/concepts#unique-primary-key-requirement). You can optionally select a [folder](/workspace-management/folders-filters#folders) to move the model to.
Click **Finish** to complete your model setup.
## Unique primary key requirement
All models require a [unique **primary key**](/models/creating-models#unique-primary-key-requirement).
If your dataset doesn't inherently include any truly unique columns, you can use SQL to either:
- [filter out](#filter-out-duplicates) duplicate rows, if you're not concerned about losing data from one or more of the duplicated rows
- create a [composite column](#composite-primary-keys) to use for your primary key
As explained in the [primary key updates](/models/creating-models#updating-a-primary-key) section, if you alter a model's primary key by selecting a different column, you will be prompted to perform a full sync or reset the change data capture for all syncs that depend on that model. Be careful when modifying primary keys. If you keep the same column name but alter the way its values are calculated, some records may be added or deleted in your destination, depending on how your sync is configured.
Hightouch intentionally stringifies your chosen [**primary key**](/getting-started/concepts#unique-primary-key-requirement) column for enhanced performance during [change data capture](/getting-started/concepts#change-data-capture).
If you need to sync the primary key column as a non-string value, use [SQL aliasing](/models/data-types-casting#sql-aliasing) in your model to create a new column specifically for syncing.
### Filter out duplicates
When you write a `SELECT` statement to define your model, you can use a `GROUP BY` clause to retrieve only the unique rows based on the column you want to use for your primary key:
```SQL
SELECT *
FROM users
GROUP BY primary_key_column
```
This query returns only one row for each unique `primary_key_column` value, while ignoring the duplicates.
This query returns one row for each `primary_key_column` value, so if there are multiple rows with the same `primary_key_column` value, only one of them will be returned.
If you want your model's query results to return every row in your dataset, it's better to create a [composite primary key](#composite-primary-keys).
### Composite primary keys
If your dataset doesn't inherently include a truly unique primary key, you can create one. For example, in a CRM, you may have users that are identified by email, but belong to multiple organizations:
{/* */}
| org_id | email | created_at |
|--------|-------------------|------------|
| 3 | alice@example.com | 2023-01-18 |
| 1 | bob@example.com | 2022-05-06 |
| 3 | bob@example.com | 2022-12-13 |
| 2 | carol@example.com | 2023-03-29 |
The user `bob@example.com` belongs to both `org_id:1` and `org_id:3`. Their email address is unique within each organization but non-unique across organizations. Therefore, if you were to select `email` as the primary key, downstream destinations receiving the data would likely reject these rows.
To ensure a sync with this type of data includes all records, you can use a **hash function** for the model's primary key. The hash should combine enough columns in the data to create a unique value, for example, `org_id` and `email`.
```SQL
SELECT org_id,
email,
created_at,
HASH(CONCAT(org_id, '-', email)) AS composite_key
FROM users
```
This query selects the `org_id`, `email`, and `created_at` columns, and creates a `composite_key` column.
The composite_key column contains a hash of the concatenation of `org_id` and `email`, creating a unique value for each row.
| org_id | email |created_at |composite_key|
|--------|-------------------|------------|-------------|
| 3 | alice@example.com | 2023-01-18 | 123456780 |
| 1 | bob@example.com | 2022-05-06 | 987654321 |
| 3 | bob@example.com | 2022-12-13 | 709230458 |
| 2 | carol@example.com | 2023-03-29 | 382910382 |
{/* */}
You would then select the `composite_key` column as the primary key for the model.
Because the composite keys for the `bob@example.com` rows are unique, Hightouch would sync both rows to the destination.
### Save a model without results
You can't save a model if your query doesn't return any results.
If you need to save a model with such a query because you expect there to be results in the future, add the following SQL to the end of your query:
```sql
UNION ALL
select 'ignore', 'ignore', 'ignore'
```
This SQL adds a row of 'ignore' values that don't match against anything in your destination but are always present so you can save the query.
If your column datatype isn't a `string`, you can replace 'ignore' with a `null` value.
The number of 'ignore' or `null` values needs to be the same as the number of columns in your model.
### Exclude rows with null values
Some [destinations](/getting-started/concepts#destinations) reject requests when required fields contain `null` values.
You can prevent syncs from sending `null` values using the [**Don't sync null values** option](/syncs/mapping-data#dont-sync-null-values) in the [advanced mapper](/syncs/mapping-data#advanced-mapper).
If you want to exclude rows with `null` values at a model level, you can include the following SQL in your model defintiion:
```sql
SELECT *
FROM your_table
WHERE column1 IS NOT NULL AND column2 IS NOT NULL AND column3 IS NOT NULL;
```
Be sure to change `column1`, `column2`, etc. to your column names and to include any columns where you believe there could be `null` values.
---
## Create models with the table selector
**URL:** https://hightouch.com/docs/models/table-selector
**Description:** Create Hightouch models using the table selector which queries existing tables and views within your source.
**Section:** Models
With the table selector modeling method, you can build Hightouch models using existing tables in your source without writing any SQL.
The first time you use the table selector you may not see your tables populated. Click the **Refresh** button to the left of the search bar and your tables should appear.
Start by selecting the table or view that you want to use as your model.
Before you can continue, you must click **Preview** to see a preview of the data. By default, Hightouch limits the preview to the first 100 rows. Once you've confirmed that the preview contains the data you expect, click **Continue**.
The last step for model setup is to enter a descriptive **Name** and select the column to use as a [**Primary key**](/getting-started/concepts#unique-primary-key-requirement). You can optionally select a [folder](/workspace-management/folders-filters#folders) to move the model to.
Click **Finish** to complete your model setup.
---
## Athena
**URL:** https://hightouch.com/docs/sources/amazon-athena
**Description:** Amazon Athena is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. Athena is serverless, so there is no infrastructure to manage, and you pay only for the queries that you run.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in Amazon Athena and push it to downstream destinations. Connecting Hightouch to Amazon Athena requires some setup in both platforms. In Amazon, you need to make sure the AWS credentials you will use to connect Hightouch has the correct permissions.
## Required permissions
The AWS user that you use to connect to Athena must have the following permissions:
Read and list permissions for Athena resources and execute permissions for Athena queries:
- `ListWorkGroups`
- `ListDataCatalogs`
- `ListDatabases`
- `ListTableMetadata`
- `StartQueryExecution`
- `GetQueryExecution`
- `GetQueryResults`
- `GetQueryResultsStream`
Read and write permissions for the output location in S3 to both write the results to S3, read them back, and perform cleanup:
- `PutObject`
- `ListBucket`
- `ListBucketMultipartUploads`
- `ListMultipartUploadParts`
- `GetBucketLocation`
- `GetObject`
- `AbortMultipartUpload`
- `DeleteObject`
See the AWS managed policy [`AmazonAthenaFullAccess`](https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AmazonAthenaFullAccess.html)
for reference.
If you are using Lake Formation for [managing permissions](https://docs.aws.amazon.com/lake-formation/latest/dg/managing-permissions.html) you need to [grant database permissions](https://docs.aws.amazon.com/lake-formation/latest/dg/granting-database-permissions.html) to the AWS user that you use to connect to Athena, otherwise no databases load when configuring the source.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **Amazon Athena** and follow the steps below.
### Configure your credentials
You can either select AWS credentials you've previously configured in Hightouch or choose to add **New credentials** directly from this page.
The credentials must be for a user who has permission to access Athena and the S3 output location. Refer to the [required permissions](#required-permissions) section for permission details and to the [AWS credential configuration](/security/aws) docs for credential setup information.
### Configure your Amazon Athena source
Enter the following fields into Hightouch:
- Athena **workgroup** you previously set up in AWS.
- (Optional) **Query output location**—you only need to enter this if your workgroup doesn't have an output location configured. This is an S3 path.
- **Data catalog** name
- **Database** name
- (Advanced) **VPC connection endpoint**-if you wish to connect to Athena through a private VPC endpoint, please contact Hightouch and we can provide the endpoint necessary for this step.
Workgroup and data catalog options populate depending on your AWS credentials. If you don't see the expected values, confirm your AWS credentials and click **Refresh**. Your database options populate depending on your selected data catalog.
### Choose your sync engine
#### Basic versus Lightning engine comparison
#### Lightning engine setup
Hightouch uses two schemas (`hightouch_planner` and `hightouch_audit`) for storing logs of previously synced data. Hightouch must be able to read and write to these schemas, but the specific schema names might vary.
```sql
-- Required for the Lightning engine
CREATE SCHEMA IF NOT EXISTS .hightouch_planner;
-- For sync logs, optional
CREATE SCHEMA IF NOT EXISTS .hightouch_audit;
```
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Athena.
The Athena source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Looker Looks](/extensions/looker-models)
## Data types
Hightouch parses most Athena data types into JavaScript types before sending them to your destination. Hightouch leaves the following Athena data types as strings:
- `BIGINT`
- `ARRAY`
- `MAP`
Read more about Athena data types in the [Athena documentation](https://docs.aws.amazon.com/athena/latest/ug/querying-iceberg-supported-data-types.html).
## Tips and troubleshooting
### Performance considerations
The following Service Quota changes to Amazon Athena are recommended in order to support fast and reliable syncs:
- Active DML queries: 400
- To support concurrent `CREATE TABLE AS` queries which are [DML statements](https://docs.aws.amazon.com/athena/latest/ug/create-table-as.html).
---
## Redshift
**URL:** https://hightouch.com/docs/sources/amazon-redshift
**Description:** Amazon Redshift is a fully managed, petabyte-scale data warehouse service in the cloud. It is built on Amazon Web Services
**Section:** Sources
## Overview
Hightouch lets you pull data stored in your Amazon Redshift data warehouse and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to Redshift for information like your host, port, database name, and credentials.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **Amazon Redshift** and follow the steps below.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your Redshift cluster. The hostname can be found by visiting the [Redshift web console](https://console.aws.amazon.com/), navigating to the **Clusters** panel, and clicking your cluster. Copy the **Endpoint string**, excluding the port and database name.
- **Port**: The port number of your Redshift cluster. The default is 5439, but yours may be different. To confirm, visit the [Redshift web console](https://console.aws.amazon.com/), navigate to the **Clusters** panel, and click your cluster. The port number is shown in the **Properties** tab.
- **Database**: The name of the database in your Redshift cluster. Most clusters have only one database. Visit the [Redshift web console](https://console.aws.amazon.com/), navigate to the **Clusters** panel, and click your cluster. The database name is shown in the **Properties** tab.
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
To set up the Lightning engine, you need to grant Hightouch write access to Redshift. You can do so by running the following SQL snippet.
```sql
CREATE USER hightouch_user WITH PASSWORD '********';
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT CREATE, USAGE ON SCHEMA hightouch_audit TO hightouch_user;
GRANT CREATE, USAGE ON SCHEMA hightouch_planner TO hightouch_user;
```
The snippet creates a dedicated Amazon Redshift user for Hightouch. It also provisions two schemas, `hightouch_planner` and `hightouch_audit`, for storing logs of previously synced data.
### Provide credentials
Enter the following fields into Hightouch:
- **User**: This can be your personal Redshift login or a dedicated user for Hightouch. At minimum, this user must have read access to the data you wish to sync. If using the Lightning sync engine, you must also grant this user additional permissions as described above.
- (Optional) **Password**: The password for the user specified above.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Amazon Redshift.
The Amazon Redshift source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Looker Looks](/extensions/looker-models)
- leveraging existing [Sigma workbooks](/extensions/sigma)
## Workload management
Amazon Redshift offers powerful and flexible settings for [workload management (WLM)](https://docs.aws.amazon.com/redshift/latest/dg/cm-c-implementing-workload-management.html).
Hightouch cannot automatically manage your WLM rules, but does enable the use of WLM to manage resource use by Hightouch by
[setting query groups](https://docs.aws.amazon.com/redshift/latest/dg/cm-c-executing-queries.html#cm-c-executing-queries-assigning-a-query-to-a-query-group).
Hightouch will automatically set the query group to either
- `ht_interactive` for interactive queries which need low latency, such as running [model previews](/models/creating-models#preview-model-results) or schema discovery
- `ht_batch` for batch operations that run in the background, such as [rETL syncs](/syncs/overview)
If no WLM configuration references these query groups, all Hightouch queries will run using Redshift’s default queue behaviour.
## Tips and troubleshooting
### Spectrum nested query error
You may receive a `Spectrum nested query error` if you are using [Amazon Redshift Spectrum](https://docs.aws.amazon.com/redshift/latest/dg/c-getting-started-using-spectrum.html) as your data source.
Hightouch uses a wrapper query around your model query for obtaining the `COUNT` of query results. Spectrum has [limitations around nested data](https://docs.aws.amazon.com/redshift/latest/dg/nested-data-restrictions.html) that this can causes issues with.
To resolve the error, you can try the following:
- Create a [materialized view](https://docs.aws.amazon.com/redshift/latest/dg/materialized-view-overview.html) of the results set you want Hightouch to query.
- Add a commented out `-- ORDER BY` statement to your model definition. Hightouch's Redshift source disables the row counter wrapper query if there's an `ORDER BY` in the query. However, Redshift Spectrum doesn't allow `ORDER BY` statements, hence the need to comment it out. Even commented out, Hightouch disables the wrapper query.
### Error: could not identify an ordering operator for type "unknown"
You may receive a this error if you are using many UNION set operators within your SQL query.
To resolve the error, you can try replacing UNION with UNION ALL.
To learn more about UNION and UNION ALL, check out the article by [The Data School](https://dataschool.com/learn-sql/what-is-the-difference-between-union-and-union-all/#:~:text=The%20main%20differenc[…]ds%2C%20including%20duplicates).
### Error: permission denied for relation 'objectname'
You may receive this error when trying to access newly created objects in the schema and you lack permissions to do so. This error happens when access is granted for only the objects present in a schema when the access was first granted. By default, access isn't automatically granted for objects that are created under the current schema.
To learn about how to grant permissions you can check the AWS documentation [here](https://repost.aws/knowledge-center/redshift-grant-permissions-new-objects).
---
## Azure Blob Storage
**URL:** https://hightouch.com/docs/sources/azure-blob
**Description:** Azure Blob Storage helps you create data lakes for your analytics needs, and provides storage to build powerful cloud-native and mobile apps. Optimize costs with tiered storage for your long-term data, and flexibly scale up for high-performance computing and machine learning workloads.
**Section:** Sources
## Overview
Hightouch lets you pull data from CSV and JSON files stored in Azure Blob Storage and push them to downstream destinations. To get started, you need an Azure Blob Storage account and container.
## Azure credential setup
See the guide for [configuring Azure credentials](/security/azure) to learn how you can set up a service account and use its credentials. The IAM user whose credentials you use must have programmatic access enabled and permission to read from the Blob Storage path you want to use.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **Azure Blob Storage** and follow the steps below.
### Configure your service account
Select the credentials you previously created or click **Create new** to set them up now.
### Configure your source
Enter your Blob Storage **Account name** and **Container name**.
## Test your connection
When setting up Azure Blob Storage as a source for the first time, Hightouch validates your Azure credentials and access to your Blob Storage. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from your Blob Storage. The file must have either a `.csv` or `.json` extension.
### CSV requirements
CSV files must meet the following requirements:
- They must have a header row. The values in the header row are automatically available as column names when you set up a sync.
- They must use comma-separated values; tabs and other delimiters aren't supported.
- They must use double quotes (`"`) for quoted values.
### JSON requirements
JSON files must meet the following requirements:
- The input file must contain an array at the top level.
- Each element in the input file must be an object with the same keys.
The keys of the array elements are automatically available as column names when you set up a sync.
### Model setup
1. In Hightouch, go to the [**Models** overview page](https://app.hightouch.com/models).
1. Click **Add model.**
1. Select the Azure Blob Storage source you previously created.
1. Enter the **relative path** to the CSV or JSON file that you want to sync data from.
1. Preview your model's query results.
1. Click **Continue**.
1. Name your model and select its [**primary key**](/models/creating-models#unique-primary-key-requirement). Hightouch uses the primary key to determine which rows have been added, changed, or removed since the last sync.
## Tips and troubleshooting
---
## ClickHouse
**URL:** https://hightouch.com/docs/sources/clickhouse
**Description:**
**Section:** Sources
Hightouch only supports ClickHouse v20 and newer.
## Overview
Hightouch lets you pull data stored in [ClickHouse](https://clickhouse.com/) and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to ClickHouse for information like your host, port, database name, and credentials.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **ClickHouse** and follow the steps below.
### Choose authentication method
You can select to use either a **password** or **connection string**. A **connection string** is useful if you need to pass additional configuration options. Hightouch uses the [official ClickHouse JDBC driver](https://github.com/ClickHouse/clickhouse-java/tree/main/clickhouse-jdbc).
If you select **connection string**, enter a JDBC connection string and skip to the [connection test section](#test-your-connection) of this page.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your ClickHouse server.
- **Port**: The port number of your ClickHouse server's HTTP API. The default is 8123, but yours may be different.
- **Database**: The name of the database to use when Hightouch executes queries in ClickHouse.
- **Username**: This can be your personal ClickHouse login or a dedicated user for Hightouch.
- (Optional) **Password**: The password for the user specified above.
## Test your connection
When setting up ClickHouse as a source for the first time, Hightouch checks that you have the correct permissions. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from ClickHouse.
The ClickHouse source support writing queries in [the SQL editor](/models/sql-editor) as its modeling method.
## Tips and troubleshooting
### Table selector fails to populate
{/* Unfortunately, this is entirely opaque to the user at the moment; there's no error information in the UI. */}
If the [table selector](/models/table-selector) fails to populate, it may be that the Clickhouse user
configured for the connection lacks necessary privileges. In order for Hightouch to automatically inspect the
schema and populate the table selector, the user configured for the connection must have `SELECT` permissions
on `INFORMATION_SCHEMA.COLUMNS`.
---
## Databricks
**URL:** https://hightouch.com/docs/sources/databricks
**Description:** Databricks is a data science and analytics platform built on top of Apache Spark. Databricks implement the Data Lakehouse concept in a single unified, cloud based platform.
**Section:** Sources
## Overview
Databricks is a popular source that supports our Reverse ETL, Composable CDP, and AI Decisioning features. This article discusses the steps for connecting to Databricks.
You can connect Databricks to Hightouch using
[Databricks Partner Connect](https://hightouch.com/blog/hightouch-databricks-partner-connect)
to bypass the setup steps outlined below. You can learn more about this in
[Databricks' documentation](https://docs.databricks.com/en/partner-connect/index.html).
## Databricks credential setup
Hightouch allows you to authenticate with Databricks using one of two methods:
Either [OAuth](https://docs.databricks.com/en/dev-tools/auth/oauth-u2m.html) or
a [Personal Access Token (PAT)](https://docs.databricks.com/en/dev-tools/auth/pat.html).
We recommend using OAuth for configuring new connections.
### Configuring a service principal with OAuth
1. Create a [service principal](https://docs.databricks.com/en/admin/users-groups/service-principals.html).
1. In your Databricks workspace, go to the user dropdown at the top right and select **Settings**.
1. In the **Identity and access** page, find **Service Principals** and click **Manage**.
1. Choose **Add new**, give your service principal a name, and click **Add**.
1. Take note of the Service Principal name as well as the **Application id** listed on the Service Principal
Configurations page. This is a UUID (for example, `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`). You may
need this later to complete setup.
1. Create an OAuth secret pair.
1. On the page for the Service Principal created above, to go the Secrets tab.
2. Click **Generate secret**. A dialog will pop up with a **Secret** and a **Client ID**.
Take careful note of these values as it is only visible now; you can't look it up later.
1. Grant the necessary catalog permissions to your new Service Principal. The above steps are sufficient
to connect and authenticate, but the Service Principal may not be _authorized_ to execute
queries against your Databricks catalog. Depending on your Databricks workspace configuration,
it may already be configured with the necessary permissions; but if the connection test fails,
you may need to follow these steps.
1. In the Databricks sidebar on the left, click **Catalog** and locate your schema in the tree view.
2. Select the schema and open the **Permissions** page. Click the **Grant** button.
3. In the **Principals** input box, type the **name** of your new Service Principal.
* If you are using the Lightning sync engine (recommended; see below),
select the **Data Editor** preset or manually grant the following permissions:
* `USE SCHEMA`
* `EXECUTE`
* `READ VOLUME`
* `SELECT`
* `MODIFY`
* `CREATE TABLE`
* Make sure to enable the checkbox at the bottom to "Also grant `USE CATALOG`"
* If you are using the Basic sync engine,
select the **Data Reader** preset or manually grant the following permissions:
* `USE SCHEMA`
* `EXECUTE`
* `READ VOLUME`
* `SELECT`
* Make sure to enable the checkbox at the bottom to "Also grant `USE CATALOG`"
1. You may also need to configure _warehouse_ permissions. As with the previous step
concerning _catalog_ permissions, your Databricks workspace configuration may or may not
automatically grant the Service Principal permission to use the compute resources.
You can read more about it in the [Databricks docs](https://docs.databricks.com/en/compute/clusters-manage.html#compute-permissions).
1. In the Databricks sidebar, click **Compute** and find your data warehouse. It may
be located on the **SQL warehouses** page.
1. On the far right column of the row corresponding to the desired warehouse, click
the icon for the kebab menu **⋮**, then **Permissions**.
If you do not see the **Permissions** option, you may lack the privileges
necessary to configure permissions—even if you are a Databricks workspace
admin. If this happens, please contact your Databricks account admin.
1. Enter the name of your Service Principal and ensure that the setting in the
dropdown is set to **Can use**. Click **Add**.
### Alternative: Configuring a Personal Access Token (PAT)
1. In your Databricks Account console, go the [**Workspaces** page](https://accounts.cloud.databricks.com/workspaces).
Select the relevant workspace and then click **Open workspace**.
2. In your workspace, go the **Compute** page and click the relevant cluster. This brings you to its **Configuration** tab.
3. At the bottom of the page, expand the **Advanced Options** toggle and open the **JDBC/ODBC** tab.
4. This tab displays your cluster's **Server Hostname**, **Port**, and **HTTP Path**, which you need to connect to Hightouch.
Keep the tab open, or save these to a secure location.
5. Create a **Personal access token** by following [the Databricks documentation](https://docs.databricks.com/dev-tools/auth.html#pat).
Once you've saved these Databricks credentials, you're ready to set up the connection in Hightouch.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button.
Select **Databricks** and follow the steps below.
### Configure your source
Enter the following required fields into Hightouch:
- **Server hostname**: To find your server hostname, visit the Databricks web console and locate your cluster.
Then, click to reveal Advanced options and navigate to the JDBC/ODBC tab.
- **Port**: The default port number is 443, but yours may be different.
To find your port, visit the Databricks web console and locate your cluster.
Then, click to reveal Advanced options and navigate to the JDBC/ODBC tab.
- **HTTP path**: To find your HTTP path, visit the Databricks web console and locate your cluster.
Then, click to reveal Advanced options and navigate to the JDBC/ODBC tab.
You can choose to route all queries to a single HTTP path, or configure separate HTTP paths optimized for different workloads (recommended):
- **Route all queries to a single HTTP path**: Use this option if you want to use a single HTTP path for all Hightouch operations.
- **Route queries to different HTTP paths (split by workload)** (Recommended): This option allows you to optimize performance by routing different types of queries to different HTTP paths:
- **Batch HTTP path**: For background jobs like sync runs, schema sampling, audience snapshots, and other workloads that are not latency-sensitive.
- **Interactive HTTP path**: For latency-sensitive queries (e.g., audience previews) and heavier workloads that need more compute (e.g., identity resolution).
- (Optional) **Catalog**: You can optionally include a Databricks catalog.
Catalogs are the first level of [Unity Catalog](https://docs.databricks.com/aws/en/data-governance/unity-catalog/)'s three-level namespace.
If Lightning engine is enabled, the 'hightouch_audit' and 'hightouch_planner' [schemas](https://hightouch.com/docs/syncs/lightning-sync-engine#warehouse-schemas)
need to be created in the catalog specified here. If this field is left empty,
these schemas should be created in the [default catalog](https://docs.databricks.com/aws/en/catalogs/default).
Note that data in other catalogs can still be accessed for model creation by specifying
the full name (catalog.schema.table) in the Hightouch SQL interface.
- **Schema**: The initial schema to use for the connection.
- **SQL dialect**: By default, Hightouch assumes that your queries use the Databricks SQL dialect.
You may wish to override this behavior if your queries use legacy ANSI SQL-92 syntax.
Some features are not available when legacy syntax is used.
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse,
which makes its setup more involved than the Basic sync engine.
However, it is more performant and reliable than the Basic engine.
This makes it the ideal choice to guarantee faster syncs, especially with large data models.
It also supports more features, such as
[Warehouse Sync Logs](/syncs/warehouse-sync-logs),
[Match Booster](/match-booster/overview),
and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
To set up the Lightning engine, ensure that your Databricks user or service principal has the appropriate permissions.
You can do so by running the following SQL snippet.
Before running the snippet, make sure to include your own username, which can be found in the top right corner
of the Databricks web console. Alternatively, you can create an API-only service principal by following
[these instructions](https://docs.databricks.com/administration-guide/users-groups/service-principals.html).
{/* Conflicting info: Used to have */}
{/* GRANT OWNERSHIP ON SCHEMA hightouch_audit TO ROLE identifier($ht_default_role); */}
{/* We don't really know which is correct; the below mirrors in-app instructions. */}
```sql
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT ALL PRIVILEGES ON SCHEMA hightouch_audit TO ;
GRANT ALL PRIVILEGES ON SCHEMA hightouch_planner TO ;
```
{/* We've had users confused about permissions around "temporary tables", which they seem */}
{/* to have interpreted as some kind of `CREATE TEMPORARY TABLE` or equivalent. */}
The `hightouch_audit` and `hightouch_planner` schemas exist to keep Hightouch's temporary tables, logs, and other
artifacts separate from your source data. This simplifies governance because you can restrict write access to only
these Hightouch-managed schemas. We recommend using `GRANT ALL PRIVILEGES` for simplicity, but you can also grant the
[required privileges](#configuring-a-service-principal-with-oauth) separately to achieve the same effect.
(“Temporary tables” here refers to tables that Hightouch will automatically clean up once they are no longer
needed; it doesn't necessarily refer to special table types in the warehouse.)
If you are using a Service Principal to connect, you may need to specify the user
above by UUID rather than name. For example, if you have created a Service Principal
named `test_principal` with an **Application Id** of `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`,
you may need to run
```sql
GRANT ALL PRIVILEGES ON SCHEMA hightouch_audit TO `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`;
GRANT ALL PRIVILEGES ON SCHEMA hightouch_planner TO `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`;
```
### Provide credentials
Enter the **Access token** you generated in the [Databricks credential setup](#databricks-credential-setup).
### Unity Catalog, external locations, and managed storage
The Lightning sync engine persists change data capture state in managed tables under `hightouch_planner` and `hightouch_audit`. When Unity Catalog manages these tables, Hightouch can apply extra optimizations during the CDC phase that improve throughput and stability on large models.
Hightouch only supports S3 for self-hosted managed storage today. If your workspace uses a cloud provider other than AWS, contact Support before proceeding.
#### Prerequisites
- You plan to use Lightning with Databricks Unity Catalog.
- You have the [service principal](#configuring-a-service-principal-with-oauth) Hightouch uses to connect, including its Application id for `GRANT` statements.
- You have a metastore or account administrator (or delegate) who can create Unity Catalog objects, specifically:
- To create the [storage credential](https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/manage-storage-credentials): `CREATE STORAGE CREDENTIAL` on the metastore.
- To create the external location: `CREATE EXTERNAL LOCATION` on both the metastore and the storage credential that backs it.
- To create the Lightning schemas: `CREATE MANAGED STORAGE` on the external location plus `USE CATALOG` and `CREATE SCHEMA` on the catalog ([Create schemas](https://docs.databricks.com/aws/en/schemas/create-schema)).
- To run the `GRANT` statements in step 3: `MANAGE` (or ownership) on the external location and storage credential.
- Someone who can attach IAM roles to your bucket path is usually involved ([S3 + Unity Catalog](https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/s3/s3-external-location-cfn)).
#### What to do
1. Pick an S3 path for Hightouch-managed data — typically the same bucket as [self-hosted workspace storage](/security/storage#amazon-s3) (for example `s3://your-company-hightouch/hightouch-databricks/`). Every `MANAGED LOCATION` you set later must sit inside the external location URL you register.
2. Create a storage credential and an external location in Databricks. Follow [CloudFormation quickstart for S3](https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/s3/s3-external-location-cfn), [Create a storage credential](https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/manage-storage-credentials), and [Create an external location](https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/manage-external-locations#create-an-external-location-using-catalog-explorer). Record the external location name and storage credential name.
3. Grant file access to the Hightouch service principal. Replace the names and the Application id (in backticks) below.
```sql
GRANT BROWSE, CREATE EXTERNAL TABLE, READ FILES, WRITE FILES
ON EXTERNAL LOCATION your_hightouch_external_location
TO `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`;
GRANT CREATE EXTERNAL TABLE, READ FILES, WRITE FILES
ON STORAGE CREDENTIAL your_hightouch_storage_credential
TO `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`;
```
Always grant `READ FILES` with `WRITE FILES` — `WRITE FILES` alone returns `PERMISSION_DENIED` ([privilege reference](https://docs.databricks.com/aws/en/data-governance/unity-catalog/manage-privileges/privileges#write-files)).
4. Create the Lightning schemas inside the catalog you configured in the Hightouch source (`my_catalog` below).
If your catalog already has a `MANAGED LOCATION`, omit the `MANAGED LOCATION` clause below — schemas inherit catalog storage automatically ([managed storage hierarchy](https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/managed-storage)).
```sql
CREATE SCHEMA IF NOT EXISTS my_catalog.hightouch_audit
MANAGED LOCATION 's3://your-company-hightouch/hightouch-databricks/audit';
CREATE SCHEMA IF NOT EXISTS my_catalog.hightouch_planner
MANAGED LOCATION 's3://your-company-hightouch/hightouch-databricks/planner';
```
5. Grant catalog and schema privileges to the Hightouch service principal. `USE CATALOG` is required in addition to schema grants; if you didn't already grant it via the [credential UI](#configuring-a-service-principal-with-oauth), include the first statement below.
```sql
GRANT USE CATALOG ON CATALOG my_catalog TO `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`;
GRANT ALL PRIVILEGES ON SCHEMA my_catalog.hightouch_audit TO `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`;
GRANT ALL PRIVILEGES ON SCHEMA my_catalog.hightouch_planner TO `63fc7e90-a8a2-4639-afd8-36ef6bb67cfa`;
```
6. Ensure the service principal has `Can use` on the SQL warehouse Hightouch uses ([warehouse permissions](#configuring-a-service-principal-with-oauth)).
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up
[models](/getting-started/concepts#models) to define which data you want to pull from Databricks.
The Databricks source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Sigma workbooks](/extensions/sigma)
The [SQL editor](/models/sql-editor) allows you to query data from all databases that your Databricks credentials
[have access to](#databricks-credential-setup), unless you [specify a catalog](#configure-your-source).
The [table selector](/models/table-selector) only supports querying from the schema specified in
the [source configuration](#configure-your-source).
## Tips and troubleshooting
### Unity Catalog support
Hightouch integrates with Databricks' [Unity Catalog](https://docs.databricks.com/en/data-governance/unity-catalog/index.html) feature.
During [initial configuration](#configure-your-source), include a **catalog** when you use Unity Catalog.
### Subquery error
Some Databricks [models](/getting-started/concepts#models) return the following error:
{/* */}
`java.lang.IllegalArgumentException: requirement failed: Subquery subquery#485, [id=#937] has not finished {...}.`
{/* */}
This error can occur when the model query contains a subquery, especially when the subquery has an `ORDER BY` clause.
For example:
```sql
SELECT user_query.*
FROM (
SELECT * FROM default.subscriptions_table
ORDER BY last_name
) user_query.*
```
The best way to resolve the error is to rewrite your model query to remove subqueries.
[Common table expressions (CTE)](https://docs.databricks.com/sql/language-manual/sql-ref-syntax-qry-select-cte.html) are supported.
If you require assistance, don't hesitate to .
### Error connecting to the database. Unauthorized/Forbidden: 403
This error can occur when [testing the source connection](#test-your-connection)
or when running a sync that uses a Databricks model.
It is typically caused by an expired Databricks access token.
To solve this, [generate a new token](https://docs.databricks.com/dev-tools/auth.html#pat)
and insert it in the source configuration.
---
## Microsoft Fabric
**URL:** https://hightouch.com/docs/sources/fabric
**Description:** Microsoft Fabric is a comprehensive analytics and data integration solution that includes cloud-based data warehousing as part of its extensive suite of capabilities.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in Microsoft Fabric and push it to downstream destinations. Connecting Hightouch to Microsoft Fabric requires some setup in both platforms. In Microsoft Fabric, you need to create a service principal and grant it access to your Fabric resource.
## Service principal creation
First, we need to create a service principal. These are the credentials Hightouch will use to access your Microsoft Fabric resources.
1. Sign in to the [Microsoft Entra admin center](https://entra.microsoft.com/).
2. On the sidebar, select **Identity** > **Applications** > **App registrations**.
3. Select **New registration**.
4. Pick a name for your application and select **Accounts in this organizational directory only** as the supported account type.
5. Click **Register**.
6. Note your newly created **Application (client) ID** and **Directory (tenant) ID**.
7. Click **Add a certificate or secret**.
From here, you can opt to use either a **Client secret** or a **Certificate**.
### Client secret
1. Select the **Client secrets** tab.
2. Click **New client secret**.
3. Name your client secret and select its expiration date. To maintain access to Fabric, make sure to refresh your client secret prior to its expiration.
4. Note your client secret **Value** now as it cannot be retrieved later. It will be used to connect Hightouch to Microsoft Fabric.
### Certificate
1. Select the **Certificates** tab.
2. Click **Upload certificate** and upload a valid signed certificate.
3. Note your certificate's **Thumbprint**.
For more detailed instructions on service principal creation, you can check out [Microsoft's documentation](https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal).
## Fabric configuration
By default, Microsoft Fabric may not be accessible via service principal. To connect, you'll need to take the following steps.
1. Log in to [Microsoft Fabric](https://app.fabric.microsoft.com/).
2. Click **Synapse Data Engineering**.
3. In the top right of the window, select **Settings** > **Governance and insights** > **Admin portal**.
4. On the sidebar, select **Tenant settings**.
5. Scroll down to **Developer settings** and select **Service principals can use Fabric APIs**. Make sure the toggle is set to **Enabled**.
6. Next go to **OneLake settings** and select **Users can access data stored in OneLake with apps external to Fabric**. This toggle should be **Enabled** as well.
## Attaching your service principal to your Fabric warehouse
1. Log in to [Microsoft Fabric](https://app.fabric.microsoft.com/).
2. Select the workspace that contains your warehouse.
3. Hover over your warehouse and select the three dots menu.
4. Choose **Manage permissions** from the menu.
5. On the next page, select **Add user**.
6. Type in the name of your service principal and check the box for **Read all data using SQL (ReadData)**.
7. Select **Grant**.
## Connection configuration
In Hightouch, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **Microsoft Fabric** and follow the steps below.
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your Microsoft Fabric instance. If using the hostname, make sure to only insert the hostname and not the full URL (remove the leading `http://` and the final `/`).
- **Port**: The port number of your Microsoft Fabric instance. The default is 1433, but yours may be different.
- **Database**: The name of the database in Microsoft Fabric.
Optionally you can enter a **Request timeout duration** and whether to **Trust server certification**.
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
To set up the Lightning engine, run the following commands. Make sure to replace **your_service_principal** with the name of the service principal created earlier.
```sql
IF NOT EXISTS (SELECT 1 FROM sys.schemas WHERE name = 'hightouch_audit')
BEGIN
EXEC('CREATE SCHEMA [hightouch_audit]');
END;
IF NOT EXISTS (SELECT 1 FROM sys.schemas WHERE name = 'hightouch_planner')
BEGIN
EXEC('CREATE SCHEMA [hightouch_planner]');
END;
GRANT CONTROL ON SCHEMA::hightouch_audit TO [your_service_principal];
GRANT CONTROL ON SCHEMA::hightouch_planner TO [your_service_principal];
```
### Provide credentials
Then you'll enter the credentials from the previous service principal creation step
- **Tenant ID** (or Directory ID)
- **Client ID** (or Application ID)
If you chose to create a **Client Secret** for your service principal, you'll enter it here.
If you instead chose to create a **Certificate**, enter in your certificate's **Thumbprint** and the associated **private key**. Make sure that the private key is in PEM format and includes the header and footer.
### Test your connection
When setting up Microsoft Fabric as a source for the first time, Hightouch validates your credentials and access to your database. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, you've completed setting up Microsoft Fabric as a source. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Microsoft Fabric.
The Microsoft Fabric source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
## Tips and troubleshooting
Most issues with the Microsoft Fabric source can be resolved by verifying that your Microsoft Fabric configuration is correct. It must allow external connections via service principal.
---
## Google Cloud Storage
**URL:** https://hightouch.com/docs/sources/gcs
**Description:** Cloud Storage is a managed service for storing unstructured data. Store any amount of data and retrieve it as often as you like.
**Section:** Sources
## Overview
Hightouch lets you pull data from CSV and JSON files stored in Google Cloud Storage (GCS) and push them to downstream destinations. To get started, you need a GCS bucket and Google Cloud credentials.
## Cloud credential setup
See the guide for [configuring GCP credentials](/security/gcp) to learn how you can set up a service account and use its credentials. The IAM user whose credentials you use must have programmatic access enabled and permission to read from the Cloud Storage path you want to use.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select Google Cloud Storage and follow the steps below.
### Configure your service account
Select the credentials you previously created or click **Create new** to set them up now.
### Configure your source
Enter the **Project ID** or your GCS project and the **Bucket name**. The bucket name should be just the name of the bucket, not a URL.
## Test your connection
When setting up GCS as a source for the first time, Hightouch validates your GCP credentials and access to your GCS bucket. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from your GCS bucket. The file in your GCS bucket must have either a `.csv` or `.json` extension.
### CSV requirements
CSV files must meet the following requirements:
- They must have a `.csv` extension.
- They must have a header row. The values in the header row are automatically available as column names when you set up a sync.
- They must use comma-separated values; tabs and other delimiters aren't supported.
- They must use double quotes (`"`) for quoted values.
### JSON requirements
JSON files must meet the following requirements:
- They must have a `.json` extension.
- The input file must contain an array at the top level.
- Each element in the input file must be an object with the same keys.
The keys of the array elements are automatically available as column names when you set up a sync.
### Model setup
1. In Hightouch, go to the [**Models** overview page](https://app.hightouch.com/models).
1. Click **Add model.**
1. Select the GCS source you previously created.
1. Enter the **relative path** to the CSV or JSON file that you want to sync data from, like `path/file.csv`. The path shouldn't contain the bucket name.
1. Preview your model's query results.
1. Click **Continue**.
1. Name your model and select its [**primary key**](/models/creating-models#unique-primary-key-requirement). Hightouch uses the primary key to determine which rows have been added, changed, or removed since the last sync.
## Tips and troubleshooting
---
## BigQuery
**URL:** https://hightouch.com/docs/sources/google-bigquery
**Description:**
**Section:** Sources
## Overview
Hightouch lets you pull data stored in your Google BigQuery warehouse and push it to downstream destinations.
Connecting Hightouch to BigQuery requires some setup in both platforms. It's recommended to set up a service account with the correct permissions in BigQuery before configuring the connection in Hightouch.
## BigQuery credential setup
Setup in BigQuery has three main steps:
1. Enable BigQuery for your Google Cloud project
2. Create a service account
3. Grant the Hightouch service account access to your project
### Create a project and enable BigQuery API
1. Login to the [Google Developers Console](https://console.developers.google.com/).
2. Configure the [Cloud Platform](https://console.cloud.google.com/):
- If you don't have a project already, [create one](https://support.google.com/cloud/answer/6251787?hl=en&ref_topic=6158848).
- Once you have a project, [enable the BigQuery API](https://console.cloud.google.com/flows/enableapi?apiid=bigquery&_ga=2.71379221.724057513.1673650275-1611021579.1664923822&_gac=1.213641504.1673650813.EAIaIQobChMIt9GagtPF_AIVkgB9Ch331QRREAAYASAAEgJfrfD_BwE) for it.
3. Copy your **Project ID** for later use.
4. Find the **location** of your BigQuery dataset or sets. You can find this by querying the `INFORMATION_SCHEMA.SCHEMATA` view or by visiting the [Google Cloud web console](https://console.cloud.google.com/) and clicking on a BigQuery dataset in the Explorer panel. You need both **Project ID** and **Data location** when connecting Hightouch to BigQuery.
Make sure [billing is enabled](https://support.google.com/cloud/answer/6293499#enable-billing) on your project, otherwise Hightouch can't write into the cluster.
### Create a service account
To create a service account, follow the setup instructions in our [Google Cloud Provider (GCP) documentation](/security/gcp).
### Grant access
By default, your GCP service account doesn't have [permission to read data](https://cloud.google.com/bigquery/docs/access-control#permissions-predefined-roles) from BigQuery.
You can set up your service account to have [full access](#grant-full-access) to your project using a [predefined role](https://cloud.google.com/iam/docs/understanding-roles#predefined_roles).
Otherwise, you can create a [custom role](https://cloud.google.com/iam/docs/roles-overview#custom) and provide [limited access](#grant-limited-access) according to a user-specified list of permissions.
#### Grant full access
You can grant full access by assigning the `bigquery.user` and `bigquery.dataViewer` roles to your service account.
You can do this in the [Google Cloud web console](https://cloud.google.com/iam/docs/granting-changing-revoking-access#grant-single-role) or by running these snippets in the [Cloud Shell](https://cloud.google.com/shell/docs/run-gcloud-commands).
Grant permission to **read metadata and list tables**:
```bash
gcloud projects add-iam-policy-binding \
--member serviceAccount: \
--role roles/bigquery.user
```
Grant permission to **read data from tables and views**:
```bash
gcloud projects add-iam-policy-binding \
--member serviceAccount: \
--role roles/bigquery.dataViewer
```
#### Grant limited access
If you don't want to grant full access to your BigQuery service account, you can opt to grant limited access instead.
You can do this by assigning the `bigquery.dataViewer` role only to the specific datasets, tables, or views you want to use in Hightouch.
Since you are assigning the `bigquery.dataViewer` role only to specific resources, you need to assign the `bigquery.user` role **and** grant the `bigquery.tables.get` permission at the project level.
For this, you can create a [custom role](https://cloud.google.com/iam/docs/creating-custom-roles#creating) in the Google Cloud web console based on an [existing predefined role](https://cloud.google.com/bigquery/docs/access-control#bigquery.user) (`bigquery.user`), which you can name `custom.bigquery.user`.
When setting up the custom role, click **Add permissions** to add the `bigquery.tables.get` permission to this custom role.
Then assign this role to your service account at the project level.
You can do this in the [Google Cloud web console](https://cloud.google.com/iam/docs/granting-changing-revoking-access#grant-single-role) or by running this snippet in the [Cloud Shell](https://cloud.google.com/shell/docs/run-gcloud-commands):
```bash
gcloud projects add-iam-policy-binding \
--member serviceAccount: \
--role roles/custom.bigquery.user
```
If this `custom.bigquery.user` role still isn't limited enough, you can try assigning the [`bigquery.jobUser`](https://cloud.google.com/bigquery/docs/access-control#bigquery.jobUser) role **and** granting the `bigquery.dataset.get`, `bigquery.tables.get`, and `bigquery.tables.list` permissions at the project level.
Hightouch needs to be able to list the schemas and tables in your BigQuery project when [creating a model](/models/creating-models).
Therefore, the [source connection test](#test-your-connection) fails if you don't grant the `bigquery.tables.get` permission at the project level.
You can then decide which datasets, tables, or views your GCP service account has access to by [granting access to a resource](https://cloud.google.com/bigquery/docs/control-access-to-resources-iam#grant_access_to_a_resource) in the Google Cloud web console.
For every resource you would like to use in Hightouch, select your BigQuery service account as the **Principal** and the `bigquery.dataViewer` role as the **Role**.
Hightouch lists all tables in your BigQuery project when creating a model using the [table selector](/models/table-selector).
However, Hightouch can only query data from tables assigned the `bigquery.dataViewer` role.
Tables that weren't assigned this role return an error if you attempt to query them.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **BigQuery** and follow the steps below.
### Configure your service account
Select the **GCP credentials** you [previously created](#create-a-service-account) or click **Create new**. To learn more about these credentials, see the [Google Cloud Provider (GCP) documentation](/security/gcp).
### Configure your source
Enter the **Project ID** for the project you [enabled the BigQuery API](#create-a-project-and-enable-bigquery-api) for and the **Dataset location**.
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
When using the [Lightning sync engine](/syncs/lightning-sync-engine) with BigQuery, Hightouch recommends configuring [self-hosted workspace storage](/security/storage#google-cloud-storage) (Business tier) in Google Cloud Storage. Keeping your workspace data in GCS aligns with BigQuery's native export paths and region constraints and helps maximize sync performance, especially for large models.
#### Lightning engine permissions
To enable the Lightning engine, you need to provide your service account additional permissions to create schemas and read/write data.
Run the following snippet to provision the `hightouch_planner` and `hightouch_audit` schemas, which are used for storing logs of previously synced data.
```sql
CREATE SCHEMA IF NOT EXISTS `hightouch_audit`;
CREATE SCHEMA IF NOT EXISTS `hightouch_planner`;
GRANT `roles/bigquery.dataViewer`, `roles/bigquery.dataEditor` ON SCHEMA `hightouch_planner` TO "serviceAccount:";
GRANT `roles/bigquery.dataViewer`, `roles/bigquery.dataEditor` ON SCHEMA `hightouch_audit` TO "serviceAccount:";
```
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from BigQuery.
The BigQuery source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Looker Looks](/extensions/looker-models)
- leveraging existing [Sigma workbooks](/extensions/sigma)
The [SQL editor](/models/sql-editor) allows you to query data from all projects that your GCP service account [has access to](#create-a-service-account).
The [table selector](/models/table-selector) only supports querying from the project specified in the [source configuration](#configure-your-source).
When syncing large amounts of data, it can take a long time for your model to reflect changes made in BigQuery. To speed up BigQuery model updates in the Hightouch UI, you can preview a model and save it. New or updated columns should then be reflected in the Hightouch UI.
## Tips and troubleshooting
### Hashed values are displayed as objects of bytes
When [previewing your model](/models/creating-models#preview-model-results) or when syncing your data to the destination, you might see an object of bytes, such as:
```bash
{"0":123,"1":456,"2":789,"3":012,...}
```
This happens if you use certain [hashing algorithms](https://cloud.google.com/bigquery/docs/reference/standard-sql/hash_functions) in BigQuery to hash your data, such as [SHA256](https://cloud.google.com/bigquery/docs/reference/standard-sql/hash_functions#sha256). To resolve this, open the [SQL editor](/models/sql-editor) and use [TO_HEX](https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#to_hex) or [TO_BASE64](https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#to_base64) to query this model column:
```bash
SELECT TO_HEX(model_column_name) as string_model_column_name, * FROM dataset.table
```
### No matching signature for operator \{...\} for argument types: \{...\}, \{...\}.
An example of the error message is `No matching signature for operator != for argument types: DATE, TIMESTAMP.`
This error means that your [model definition](/getting-started/concepts#models) is using the named operator, for example, `!=` to compare two incompatible types, for example, a `date` to a `timestamp`.
To resolve the issue, ensure your model columns are properly [typed](/models/data-types-casting#casting-from-your-model-configuration) or revise your model query.
### Unable to process number due to [big.js]
An example of the error message is `Unable to process number due to [big.js] Imprecise conversion: [big.js] Imprecise conversion`
This error occurs when one or more values have a very high precision, for example `123.12345678910`. Hightouch attempts to convert the value to a `number` (based on the data type selection of your model). An error occurs if the number can't be precisely represented by a JavaScript number.
To resolve the issue, ensure the values in your model columns have a lower precision. Additionally, if your value doesn't need to be a `number` datatype, casting the value to a `string` datatype can also resolve this issue.
### Dataset was not found in location
You may encounter this error if you customized the Lightning engine cache location in the source configuration. When using a customized location, you must use fully qualified table names (`project_id.dataset.table`) in model queries. This requires using the [SQL editor](/models/sql-editor) to create your model, because all other modeling methods ([table selector](/models/table-selector), dbt, Looker, Sigma) aren't compatible with this setup.
Otherwise, this error can happen if your dataset is located in a region other than `US` (such as `EU`). By default, BigQuery creates new schemas in the US region, so running the [Lightning engine permissions](#lightning-engine-permissions) SQL snippet (shown in the source setup UI) creates them in that region. If your dataset is not in the US region (for example, it might be in `EU`), this causes a region incompatibility between the main dataset and those two schemas, which results in that error message. To fix this, you need to manually check the [schema details](https://cloud.google.com/bigquery/docs/managing-table-schemas) in BigQuery and update their region/location value to be the same as the main dataset.
---
## BigQuery Iceberg
**URL:** https://hightouch.com/docs/sources/google-bigquery-iceberg
**Description:** BigQuery is a fully-managed enterprise data warehouse that helps you manage and analyze your data with built-in features like machine learning, geospatial analysis, and business intelligence.
**Section:** Sources
## Overview
This source is currently in preview. Please for more details.
Hightouch lets you sync data from BigLake tables using the Apache Iceberg REST catalog to downstream destinations.
BigLake Iceberg tables use the [Apache Iceberg](https://iceberg.apache.org/) open table format, with data stored as Parquet files in Google Cloud Storage (GCS). Table metadata is managed by the [BigLake Metastore](https://cloud.google.com/biglake/docs/overview) and accessible via the **Iceberg REST Catalog**, enabling interoperability between BigQuery, Apache Spark, and other Iceberg-compatible engines.
Hightouch discovers your tables and schemas via the BigLake Iceberg REST Catalog API, and uses BigQuery as the query engine to read your data.
If you already have a standard BigQuery source in Hightouch, you still need to create a separate BigQuery Iceberg source. The two source types use different table discovery mechanisms and cannot be combined.
## GCP prerequisites
Before configuring the source in Hightouch, you need to set up several GCP resources. Each step below shows both **Google Cloud Console** and **gcloud CLI** methods—you only need to follow one.
### Step 1: Enable required APIs
BigLake Iceberg requires the following APIs on your GCP project:
- **BigQuery API** (`bigquery.googleapis.com`)
- **BigLake API** (`biglake.googleapis.com`)
#### Console
1. Go to the [APIs & Services dashboard](https://console.cloud.google.com/apis/dashboard).
2. Click **Enable APIs and Services**.
3. Search for **BigQuery API** and click **Enable** (if not already enabled).
4. Repeat for **BigLake API**.
#### gcloud CLI
```bash
gcloud services enable bigquery.googleapis.com \
--project=YOUR_PROJECT_ID
gcloud services enable biglake.googleapis.com \
--project=YOUR_PROJECT_ID
```
### Step 2: Create a Cloud Storage bucket
Iceberg table data (Parquet files and metadata) is stored in a GCS bucket. If your Iceberg tables already exist in a bucket, you can use that bucket. Otherwise, create a new one.
The bucket must **not** use any of the following features, which are incompatible with BigLake Iceberg:
- Hierarchical namespaces
- Object versioning
- Object lock or bucket lock
- Customer-supplied encryption keys (CSEK)
#### Console
1. Go to [Cloud Storage > Buckets](https://console.cloud.google.com/storage/browser).
2. Click **Create**.
3. Enter a globally unique bucket name (e.g., `your-company-iceberg-data`).
4. Choose a **Location type** that matches where your BigQuery jobs run (e.g., `us-central1` or `US` multi-region).
5. For **Default storage class**, select **Standard**.
6. For **Access control**, select **Uniform**.
7. Leave all other settings at their defaults—do **not** enable versioning, retention policies, or object lock.
8. Click **Create**.
#### gcloud CLI
```bash
gcloud storage buckets create gs://YOUR_BUCKET_NAME \
--project=YOUR_PROJECT_ID \
--location=YOUR_REGION \
--uniform-bucket-level-access
```
### Step 3: Create a service account for Hightouch
Hightouch needs a GCP service account with permissions to query BigQuery, read from the Iceberg REST catalog, and access GCS. If you already have a service account from a standard BigQuery source, you can reuse it—just add the additional roles below.
| Role | Purpose |
|---|---|
| **BigQuery User** (`roles/bigquery.user`) | Run queries |
| **BigQuery Data Viewer** (`roles/bigquery.dataViewer`) | Read table data via BigQuery |
| **BigLake Viewer** (`roles/biglake.viewer`) | Read table metadata from the Iceberg REST catalog |
| **Storage Object Viewer** (`roles/storage.objectViewer`) | Read Parquet data files from GCS |
| **Service Usage Consumer** (`roles/serviceusage.serviceUsageConsumer`) | Required for REST catalog API billing attribution |
#### Console
1. Go to [IAM & Admin > Service Accounts](https://console.cloud.google.com/iam-admin/serviceaccounts).
2. Click **Create Service Account**.
3. Enter a name (e.g., `hightouch-bigquery-iceberg`) and click **Create and Continue**.
4. Add the roles listed above and click **Continue**.
5. Click **Done**.
6. Click on the newly created service account, go to the **Keys** tab, and click **Add Key > Create new key > JSON**. Download and save the key file.
#### gcloud CLI
```bash
SA_EMAIL="hightouch-bigquery-iceberg@YOUR_PROJECT_ID.iam.gserviceaccount.com"
gcloud iam service-accounts create hightouch-bigquery-iceberg \
--display-name="Hightouch BigQuery Iceberg" \
--project=YOUR_PROJECT_ID
for role in roles/bigquery.user roles/bigquery.dataViewer roles/biglake.viewer roles/serviceusage.serviceUsageConsumer; do
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
--member="serviceAccount:$SA_EMAIL" \
--role="$role"
done
gcloud storage buckets add-iam-policy-binding gs://YOUR_BUCKET_NAME \
--member="serviceAccount:$SA_EMAIL" \
--role=roles/storage.objectViewer
gcloud iam service-accounts keys create hightouch-key.json \
--iam-account="$SA_EMAIL"
```
### Step 4: Verify your setup
Verify that your bucket is accessible as an Iceberg REST catalog. The BigLake Metastore automatically maps GCS buckets to Iceberg catalogs—no explicit catalog creation is needed.
You can verify by querying the REST catalog config endpoint (requires `gcloud` authentication):
```bash
curl -s -H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "x-goog-user-project: YOUR_PROJECT_ID" \
"https://biglake.googleapis.com/iceberg/v1/restcatalog/v1/config?warehouse=gs://YOUR_BUCKET_NAME"
```
A successful response includes a `prefix` field and a list of supported endpoints:
```json
{
"overrides": {
"prefix": "projects/YOUR_PROJECT_NUMBER/catalogs/YOUR_BUCKET_NAME"
},
"endpoints": ["GET /v1/{prefix}/namespaces", ...]
}
```
If you already have Iceberg tables in your bucket (created by Spark, Flink, or another engine), they will be automatically discoverable through the REST catalog. No migration is needed.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **BigQuery Iceberg** and follow the steps below.
### Configure your service account
Select the **GCP credentials** you [previously created](#step-3-create-a-service-account-for-hightouch) or click **Create new**. To learn more about these credentials, see the [Google Cloud Provider (GCP) documentation](/security/gcp).
### Configure your source
Enter the following required fields into Hightouch:
- **Project ID**: Your GCP project ID.
- **Dataset location**: The geographic location of your data (e.g., `us-central1` or `US`).
- **Iceberg catalog warehouse URI**: The GCS bucket URI that backs your Iceberg REST catalog (e.g., `gs://your-company-iceberg-data`). Hightouch uses this to discover namespaces, tables, and schemas.
### Choose your sync engine
BigQuery Iceberg currently supports the **Basic sync engine** only. The Lightning sync engine will be supported in a future release.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from your Iceberg tables.
The BigQuery Iceberg source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
The [table selector](/models/table-selector) browses tables discovered from your Iceberg REST catalog. The [SQL editor](/models/sql-editor) allows you to write arbitrary BigQuery SQL, including queries that join Iceberg tables with standard BigQuery tables.
## Known limitations
- **Lightning sync engine**: Not yet supported. BigQuery DML (INSERT, UPDATE, DELETE) is not available on REST-catalog-managed Iceberg tables. This is a Google Cloud limitation expected to be resolved in a future BigQuery release.
- **Write operations**: Hightouch reads from Iceberg tables but does not write to them. Data must be loaded into Iceberg tables using an Iceberg-compatible writer (Spark, Flink, PyIceberg, etc.).
## Tips and troubleshooting
### "Not found" errors when querying tables
BigQuery queries against REST catalog tables use four-part naming: `project.catalog.namespace.table`. If you see "Dataset not found" errors, verify that:
- Your GCS bucket name is correct in the **Iceberg catalog warehouse URI** field
- The Hightouch service account has the `biglake.viewer` and `serviceusage.serviceUsageConsumer` roles
- The BigLake API is enabled on your project
### Tables not appearing in the table selector
If your Iceberg tables don't appear in the table selector:
- Verify the tables exist in the REST catalog by querying the namespaces endpoint (see [Step 4](#step-4-verify-your-setup))
- Ensure the tables are in a namespace other than `hightouch_planner` or `hightouch_audit` (these are filtered from discovery)
- Try refreshing the schema in the Hightouch UI
---
## Google Sheets
**URL:** https://hightouch.com/docs/sources/google-sheets
**Description:** Google Sheets is a web-based spreadsheet offered by Google as part of their Drive office services. It makes it easy to create, edit and collaborate with others from any device.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in Google Sheets and push it to downstream destinations. Before connecting your Google Sheets, it's best to make sure they meet formatting requirements.
## Google Sheets formatting requirements
- Your Google Sheet must be a Sheets file, not an `.XLS` or `.XLSX` file. If this is the case, you need to convert it to a Sheets file. To do so, go to [**File > Save as Google Sheets**](https://support.google.com/docs/answer/9331167?hl=en#2.5).
- Your Google Sheet must have column names in the first row of your sheet. In the following image, you can see that the column names are `id`, `created_at`, `name`, `website`, and `description`.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **Google Sheets** and follow the steps below.
### Authorize connection
1. Click **Log in to Google Sheets** and select the desired Google Account.
2. After reading what authorization you're giving to Hightouch, select **Allow**.
3. Select the **Spreadsheet** containg the data you wish to sync from the dropdown.
4. Click **Continue**.
Remember that [`.XLS` or `.XLSX` file types aren't supported](#google-sheets-formatting-requirements) and won't appear in the dropdown.
## Test your connection
When setting up Google Sheets as a source for the first time, Hightouch checks that you have the correct permissions. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Google Sheets.
### Model setup
1. In Hightouch, go to the [**Models** overview page](https://app.hightouch.com/models).
1. Click **Add model.**
1. Select the Google Sheets source you previously created.
1. Select the **Sheet name** of the data you want to sync. Hightouch treats each sheet as a separate table. If you want to sync data from multiple sheets, you need to set up a separate model for each one.
1. Preview your model's query results.
1. Click **Continue**.
1. Name your model and select its [**primary key**](/models/creating-models#unique-primary-key-requirement). Hightouch uses the primary key to determine which rows have been added, changed, or removed since the last sync.
## Tips and troubleshooting
### Internal error encountered
This error may occur when the spreadsheet you're syncing to contains sheets with charts or pivot tables.
Try moving the data you would like to update with Hightouch to a separate spreadsheet.
Then reference this data's range using a formula like [`IMPORTRANGE`](https://support.google.com/docs/answer/3093340?hl=en) in the spreadsheet with your charts and analysis.
This way, the sheet that Hightouch updates won't have the timeout issue, and you can still see the data on the sheet that contains charts or pivot tables.
### Unable to parse range
This error indicates that the spreadsheet (in the source configuration) or the sheet (in the model configuration) have either been renamed or deleted. You can fix this by ensuring that both the source and model are querying data from an existing spreadsheet and sheet.
---
## Greenplum Database
**URL:** https://hightouch.com/docs/sources/greenplum
**Description:** Greenplum reduces data silos by providing you with a single, scale-out environment for converging analytic and operational workloads, like streaming ingestion.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in your Greenplum database and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to your Greenplum instance for information like your host, port, database name, and credentials.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **Greenplum Database** and follow the steps below.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your Greenplum server.
- **Port**: The port number of your Greenplum server. The default port number is 5432, but yours may be different.
- **Database**: This specifies the database to use when Hightouch executes queries in Greenplum.
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
To set up the Lightning engine, you need to grant Hightouch write access to Greenplum. You can do so by running the following SQL snippet.
```sql
CREATE USER hightouch_user WITH PASSWORD '********';
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT CREATE, USAGE ON SCHEMA hightouch_audit TO hightouch_user;
GRANT CREATE, USAGE ON SCHEMA hightouch_planner TO hightouch_user;
```
The snippet creates a dedicated Greenplum user for Hightouch. It also provisions two schemas (`hightouch_planner` and `hightouch_audit`) for storing logs of previously synced data.
### Provide credentials
Enter the following fields into Hightouch:
- **User**: This can be your personal Greenplum login or a dedicated user for Hightouch. At minimum, this user must have read access to the data you wish to sync. If using the Lightning sync engine, you must also grant this user additional permissions as described above.
- **Password**: The password for the user specified above.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Greenplum.
The Greenplum source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Looker Looks](/extensions/looker-models)
- leveraging existing [Sigma workbooks](/extensions/sigma)
## Tips and troubleshooting
---
## Lakebase
**URL:** https://hightouch.com/docs/sources/lakebase
**Description:** Lakebase is a serverless Postgres-compatible database in the Databricks platform.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in your Lakebase data warehouse and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to your Lakebase instance for information like your host, port, database name, and credentials.
## Connection configuration
Hightouch officially supports Lakebase versions 1.0 and above. While some features may still work with earlier versions, compatibility isn't guaranteed, and future updates might introduce changes affecting older versions.
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **Lakebase** and follow the steps below.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your Lakebase server.
- **Port**: The port number of your Lakebase server. The default port number is 5432, but yours may be different.
- **Database**: This specifies the database to use when Hightouch executes queries in Lakebase.
For authentication setup instructions, refer to the [Databricks Lakebase authentication documentation](https://docs.databricks.com/aws/en/oltp/oauth#authenticate-with-postgres-roles-and-passwords).
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
To set up the Lightning engine, you need to grant Hightouch write access to Lakebase. You can do so by running the following SQL snippet.
```sql
CREATE USER hightouch_user WITH PASSWORD '********';
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT CREATE, USAGE ON SCHEMA hightouch_audit TO hightouch_user;
GRANT CREATE, USAGE ON SCHEMA hightouch_planner TO hightouch_user;
```
The snippet creates a dedicated Lakebase user for Hightouch. It also provisions two schemas (`hightouch_planner` and `hightouch_audit`) for storing logs of previously synced data.
### Provide credentials
Enter the following fields into Hightouch:
- **User**: This can be your personal Lakebase login or a dedicated user for Hightouch. At minimum, this user must have read access to the data you wish to sync. If using the Lightning sync engine, you must also grant this user additional permissions as described above.
- **Password**: The password for the user specified above.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Lakebase.
The Lakebase source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Looker Looks](/extensions/looker-models)
- leveraging existing [Sigma workbooks](/extensions/sigma)
## Tips and troubleshooting
---
## Materialize
**URL:** https://hightouch.com/docs/sources/materialize
**Description:** Materialize is a streaming database powered by Timely and Differential Dataflow, purpose-built for low-latency applications. It lets you ask complex questions about your data using SQL, and incrementally maintains the results of these SQL queries as the underlying data changes.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in your Materialize database and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to your Materialize instance for information like your host, port, database name, cluster, and credentials.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **Materialize** and follow the steps below.
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your Materialize server.
- **Port**: The port number of your Materialize server. The default port number is 6875, but yours may be different.
- **Database**: This specifies the database to use when Hightouch executes queries in Materialize.
### Provide credentials
Enter the following fields into Hightouch:
- **User**: This can be your personal Materialize login or a dedicated user for Hightouch. At minimum, this user must have read access to the data you wish to sync.
- **Password**: The password for the user specified above.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Materialize.
The Materialize source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Looker Looks](/extensions/looker-models)
- leveraging existing [Sigma workbooks](/extensions/sigma)
## Tips and troubleshooting
### Canceling statement due to statement timeout
This error occurs when the required execution time of your Materialize query exceeds the timeout limit for the database.
To address the error, increase the timeout by executing the following query:
```json
set statement_timeout = '300 s'; -- 300 seconds, 5 minutes
```
Be sure to adjust the time to be as long as it takes for your query to execute.
---
## Microsoft Excel
**URL:** https://hightouch.com/docs/sources/microsoft-excel
**Description:** Microsoft Excel is the industry leading spreadsheet software program, a powerful data visualization and analysis tool.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in Microsoft Excel and push it to downstream destinations. Before connecting your Microsoft Excel source, it's best to make sure they meet the formatting requirements.
## Microsoft Excel formatting requirements
- Your Microsoft Excel workbook must be a `.xlsx` file. Other extension workbooks, for example, `.xls` or `.csv` aren't supported. [Convert your file](https://answers.microsoft.com/en-us/msoffice/forum/all/converting-xls-to-xlsx/ba8d05f4-d38a-462a-a062-d6083db4b479) if necessary.
- Your Excel worksheet must have column names in the first row. In the following image, you can see that the column names are `Country`, `Product`, `Discount Band`, `Units Sold`, and ` Manufacturing Price` etc.
- To ensure accurate processing, all valid data must be present in the top-left portion of the sheet. If a column name is left empty, the preceding column will be treated as the last column in the file. Similarly, if a row is left empty, the preceding row will be considered the last row in the file, and any subsequent rows will not be processed.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **Microsoft Excel** and follow the steps below.
### Authorize connection
1. Click **Log in to Microsoft Excel** and select the desired Microsoft Account.
2. After reading what authorization you're giving to Hightouch, click **Accept**.
3. Select the **Drive** containg the data you wish to sync from the dropdown.
4. Click **Continue**.
## Test your connection
When setting up Microsoft Excel as a source for the first time, Hightouch checks that you have the correct permissions. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Microsoft Excel.
### Model setup
1. In Hightouch, go to the [**Models** overview page](https://app.hightouch.com/models).
1. Click **Add model.**
1. Select the Microsoft Excel source you previously created.
1. Select the **Workbook** which has the data you want to sync.
1. Next, select a **Sheet** to read from. Hightouch treats each sheet as a separate table. If you want to sync data from multiple sheets, you need to set up a separate model for each one.
1. Preview your model's query results.
1. Click **Continue**.
1. Name your model and select its [**primary key**](/getting-started/concepts#primary-key-requirement). Hightouch uses the primary key to determine which rows have been added, changed, or removed since the last sync.
## Tips and troubleshooting
### Need admin approval
If you see a **Need admin approval** page appear when [accessing Microsoft's OAuth login flow](#connect-to-microsoft-onedrive), your Azure Active Directory user account might not have the necessary permissions to connect applications to your organization.
Ask one of your Azure organization's admins to edit your user account permissions.
You can read more about [Azure user permissions](https://learn.microsoft.com/en-us/azure/active-directory/develop/active-directory-how-applications-are-added#who-has-permission-to-add-applications-to-my-azure-ad-instance) and [configuring user settings](https://learn.microsoft.com/en-us/azure/active-directory/manage-apps/configure-user-consent?pivots=portal#configure-user-consent-settings) in Azure's documentation.
### Files not appearing in model dropdown
Only `.xlsx` files are [supported](#microsoft-excel-formatting-requirements).
Other files won't appear in the dropdown.
[Convert your file type to `.xlsx`](https://answers.microsoft.com/en-us/msoffice/forum/all/converting-xls-to-xlsx/ba8d05f4-d38a-462a-a062-d6083db4b479) to make it available to Hightouch.
---
## MySQL
**URL:** https://hightouch.com/docs/sources/mysql
**Description:** MySQL is an open source relational database management system (RDBMS) with a client-server model.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in MySQL and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to MySQL for information like your hostname, port, and credentials.
Hightouch unofficially supports backwards compatible MySQL forks such as [TiDB by Pingcap](https://en.pingcap.com/).
### MariaDB support
Hightouch supports using MariaDB as a source via the MySQL source. Follow the instructions below, substituting MySQL information and credentials with your MariaDB ones.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **MySQL** and follow the steps below.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your MySQL.
- **Port**: The port number of your MySQL. The default is 3306, but yours may be different.
- **Database**: The name of the database in your MySQL.
### Provide your credentials
Enter the following fields into Hightouch:
- **User**: This can be your personal MySQL login or a dedicated user for Hightouch.
- (Optional) **Password**: The password for the user specified above.
## Test your connection
When setting up MySQL as a source for the first time, Hightouch validates your credentials and access to your database. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, you've completed setting up MySQL as a source. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from MySQL.
The MySQL source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
## Tips and troubleshooting
### Server does not support secure connection
The [connection test](#test-your-connection) returns this error message if you don't enable SSL connections in your MySQL server configuration.
To solve this, make sure to configure MySQL to [use encrypted connections](https://dev.mysql.com/doc/refman/8.0/en/using-encrypted-connections.html).
---
## Oracle DB
**URL:** https://hightouch.com/docs/sources/oracle
**Description:** Power internal tools, in-app experiences, and more
**Section:** Sources
## Overview
Hightouch lets you pull data stored in your Oracle database and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to your Oracle instance for information like your host, port, database name, and credentials.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **Oracle DB** and follow the steps below.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your Oracle server.
- **Port**: The port number of your Oracle server. The default port number is 1521, but yours may be different.
- **Database**: This specifies the database to use when Hightouch executes queries in Oracle DB. This is different than the host, but your host address might contain your database name.
- **Username**: The user that has access through Hightouch to the database and tables you want to pull from. It's best to [create a new user](https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/CREATE-USER.html) specifically for Hightouch access. _Do not use the root user._
- **Password**: The password for the specified user.
### Choose your sync engine
For optimal performance, Hightouch [tracks incremental changes](/getting-started/concepts#change-data-capture)
in your data model—such as added, changed, or removed rows—and only syncs those records.
You can choose between two different sync engines for this work.
The **Basic engine** requires read-only access to Oracle. Hightouch executes a query in your database, reads all query results, and then determines incremental changes using Hightouch's infrastructure. This engine is easier to set up since it requires read—not write—access to Oracle.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Oracle.
The Oracle source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
## Tips and troubleshooting
### Compatible versions
The supported Oracle DB version is `19c`. Other versions aren't officially supported but may work regardless.
### Unsupported data types
Hightouch does not support `INTERVAL` types for Oracle.
---
## Sources overview
**URL:** https://hightouch.com/docs/sources/overview
**Description:** Learn how to connect data sources to Hightouch so you can define models and activate data through syncs.
**Section:** Sources
## Overview
A **source** is any system where your business data lives.
Common examples include cloud data warehouses such as **Snowflake**, **BigQuery**, or **Databricks**. Sources can also be transactional databases, flat files (CSV, SFTP), or BI tools like Looker and Sigma.
Sources are the starting point in Hightouch’s [**Reverse ETL**](https://hightouch.com/docs/getting-started/concepts) pipeline. Once connected, you can query data from them to build [**models**](https://hightouch.com/docs/models/creating-models), which are then delivered to **destinations** through [**syncs**](https://hightouch.com/docs/syncs/overview).
**For marketers**: a source is simply where customer and business data lives, such as purchase history, website events, or CRM records.
---
## Add a source
To create a new source:
1. Go to to **Integrations → [Sources](https://app.hightouch.com/sources)**.
2. Click **Add source**.
You’ll be prompted to choose from supported source types and provide the necessary connection details, such as:
- **Connection method** (e.g. username/password, OAuth, API key)
- **Host and port** (for databases)
- **Authentication details**
- **Permissions** required by Hightouch to query data
The exact configuration steps depend on the source type. See catalog and refer to **integration-specific documentation** for instructions.
---
## Security and connectivity
Hightouch connects to sources securely:
- All data in transit is encrypted with TLS.
- For private networks or VPCs, you can connect via [**SSH tunnel**](https://hightouch.com/docs/security/ssh-tunneling), [**AWS PrivateLink**](https://hightouch.com/docs/security/aws/privatelink), [**Azure Private Link**](https://hightouch.com/docs/security/azure/private-link), or [**Google Cloud Private Service Connect**](https://hightouch.com/docs/security/gcp/private-service-connect).
- By default, no customer data is copied into Hightouch storage—queries are run directly against your source.
See [**Security and compliance overview →**](https://hightouch.com/docs/security/overview) for details.
---
## How sources fit into data activation
- **Data teams**: configure sources and expose trusted tables or views.
- **Marketers**: work with models and schema that are built on top of these sources, without needing to understand the connection details.
**Example:**
- Source: a Snowflake warehouse containing ecommerce data
- Model: SQL query for “users with >3 purchases in last 90 days”
- Sync: send this model to Meta Ads for retargeting
---
## Next steps
- [Add your first source](https://app.hightouch.com/sources)
- Learn how to define [Models](https://hightouch.com/docs/models/creating-models) on top of sources
- Configure [Syncs](https://hightouch.com/docs/syncs/overview) to deliver data into destinations
---
## PlanetScale
**URL:** https://hightouch.com/docs/sources/planetscale
**Description:** PlanetScale is a MySQL-compatible serverless database that brings you scale, performance, and reliability — without sacrificing developer experience.
With PlanetScale, you get the power of horizontal sharding, non-blocking schema changes, and many more powerful database features without the pain of implementing them.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in PlanetScale and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to PlanetScale for information like your hostname, database, and credentials.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **PlanetScale** and follow the steps below.
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your PlanetScale.
- **Port**: The port number of your PlanetScale. The default is 3306, but yours may be different.
- **Database**: The name of the database in your PlanetScale.
### Provide your credentials
Enter the following fields into Hightouch:
- **User**: This can be your personal PlanetScale login or a dedicated user for Hightouch.
- (Optional) **Password**: The password for the user specified above.
## Test your connection
When setting up PlanetScale as a source for the first time, Hightouch validates your credentials and access to your database. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, you've completed setting up PlanetScale as a source. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from PlanetScale.
The PlanetScale source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
## Tips and troubleshooting
---
## PostgreSQL
**URL:** https://hightouch.com/docs/sources/postgresql
**Description:** PostgreSQL, also known as Postgres, is a free and open-source relational database management system emphasizing extensibility and SQL compliance.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in your PostgreSQL database and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to your PostgreSQL instance for information like your host, port, database name, and credentials.
## Connection configuration
Hightouch officially supports PostgreSQL versions 15 and above. While some features may still work with earlier versions, compatibility isn’t guaranteed, and future updates might introduce changes affecting older versions.
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **PostgreSQL** and follow the steps below.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your PostgreSQL server.
- **Port**: The port number of your PostgreSQL server. The default port number is 5432, but yours may be different.
- **Database**: This specifies the database to use when Hightouch executes queries in PostgreSQL.
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
To set up the Lightning engine, you need to grant Hightouch write access to PostgreSQL. You can do so by running the following SQL snippet.
```sql
CREATE USER hightouch_user WITH PASSWORD '********';
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT CREATE, USAGE ON SCHEMA hightouch_audit TO hightouch_user;
GRANT CREATE, USAGE ON SCHEMA hightouch_planner TO hightouch_user;
```
The snippet creates a dedicated PostgreSQL user for Hightouch. It also provisions two schemas (`hightouch_planner` and `hightouch_audit`) for storing logs of previously synced data.
### Provide credentials
Enter the following fields into Hightouch:
- **User**: This can be your personal PostgreSQL login or a dedicated user for Hightouch. At minimum, this user must have read access to the data you wish to sync. If using the Lightning sync engine, you must also grant this user additional permissions as described above.
- **Password**: The password for the user specified above.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from PostgreSQL.
The PostgreSQL source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Looker Looks](/extensions/looker-models)
- leveraging existing [Sigma workbooks](/extensions/sigma)
## Tips and troubleshooting
### Canceling statement due to statement timeout
This error occurs when the required execution time of your PostgreSQL query exceeds the timeout limit for the database.
To address the error, increase the timeout by executing the following query:
```json
set statement_timeout = '300 s'; -- 300 seconds, 5 minutes
```
Be sure to adjust the time to be as long as it takes for your query to execute.
### Terminating connection due to conflict with recovery
This error can happen if the `max_standby_archive_delay` and `max_standby_streaming_delay` values in your PostgreSQL configuration are too low. Try increasing them as suggested in the [AWS Knowledge Center](https://repost.aws/knowledge-center/rds-postgresql-error-conflict-recovery).
### Prepared statement did not exist
This error occurs when your PostgreSQL instance is accessed through a connection pooler running in transaction pooling mode.
To resolve this, adjust your source configuration to use a direct connection string instead of the pooled URL.
---
## S3
**URL:** https://hightouch.com/docs/sources/s3
**Description:** Securely sync CSV or JSON data from any S3 bucket
**Section:** Sources
Need to join, transform, or filter your S3 data? Consider using the [AWS
Athena](/sources/amazon-athena) source to directly query your S3 data using
SQL.
## Overview
Hightouch lets you pull data from CSV and JSON files stored in an Amazon S3 bucket and push them to downstream destinations. To get started, you need an S3 bucket and AWS credentials.
## AWS credential setup
See the guide for [configuring AWS credentials](/security/aws) to learn how you can set up an IAM principal and use its credentials. The IAM principal whose credentials you use must have programmatic access enabled and permission to read from the S3 path you want to use.
Hightouch needs the following IAM actions to retrieve items from your bucket:
| Action | Details |
| :----------------------- | :---------------------------------------------------------------------------------------------- |
| `s3:GetObject` | Grants permission to retrieve objects from Amazon S3 |
| `s3:ListBucket` | Grants permission to list some or all the objects in an Amazon S3 bucket (up to 1000) |
| `sts:GetFederationToken` | (Optional) Grants permission to generate short-lived AWS credentials for faster warehouse reads |
If you use an **access key** credential in Hightouch, we recommend adding
`sts:GetFederationToken`. This lets Hightouch generate short-lived credentials
scoped to the S3 prefix being written, which enables syncs to export data from
your warehouse to S3 (faster than reading rows via streaming). If you don't
grant this permission, Hightouch can still work, but syncs will fall back to
the slower streaming method instead. If you use a **cross-account role**,
`sts:GetFederationToken` isn't required.
You can use the following JSON sample to create your IAM policy:
```yaml
{
"Version": "2012-10-17",
"Statement":
[
{
"Effect": "Allow",
"Action": ["s3:GetObject", "s3:ListBucket"],
"Resource":
["arn:aws:s3:::${bucketName}/*", "arn:aws:s3:::${bucketName}"],
},
],
}
```
If you want to add the optional permission, include the following statement:
```json
{
"Sid": "AllowFederationTokenForS3",
"Effect": "Allow",
"Action": ["sts:GetFederationToken"],
"Resource": "*"
}
```
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **Amazon S3** and follow the steps below.
### Configure your credentials
Select the credentials you previously created or click **Create new** to set them up now.
### Configure your source
Enter your S3 bucket's **Region** and the **Bucket name**. The bucket name should be just the name of the bucket, not a URL.
## Test your connection
When setting up Amazon S3 as a source for the first time, Hightouch validates your AWS credentials and access to your S3 bucket. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from your S3 bucket. The file must have either a `.csv` or `.json` extension.
### CSV requirements
CSV files must meet the following requirements:
- They must have a header row. The values in the header row are automatically available as column names when you set up a sync.
- They must use comma-separated values; tabs and other delimiters aren't supported.
- They must use double quotes (`"`) for quoted values.
### JSON requirements
JSON files must meet the following requirements:
- The input file must contain an array at the top level.
- Each element in the input file must be an object with the same keys.
The keys of the array elements are automatically available as column names when you set up a sync.
### Model setup
Hightouch supports syncing data from a specific file in your S3 bucket as well
as from the last modified file that matches a given prefix.
1. In Hightouch, go to the [**Models** overview page](https://app.hightouch.com/models).
1. Click **Add model.**
1. Select the Amazon S3 source you previously created.
1. Enter the **relative path** to the CSV or JSON file that you want to sync
data from, like `path/prefix/file.csv`. The path shouldn't contain the bucket
name.
1. To sync the last modified file with a given prefix, check the **Use last
modified** box and enter a prefix like `path/prefix` in the path field. See
Amazon's [S3 docs on
prefixes](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-prefixes.html)
for examples of how prefixes match objects. To sync the last modified file in
the entire bucket, check the **Use last modified** box and leave the path field
blank.
1. Preview your model's query results.
1. Click **Continue**.
1. Name your model and select its [**primary key**](/models/creating-models#unique-primary-key-requirement). Hightouch uses the primary key to determine which rows have been added, changed, or removed since the last sync.
## Tips and troubleshooting
---
## Sample datasets
**URL:** https://hightouch.com/docs/sources/sample-data
**Description:** Explore Hightouch using the B2B SaaS and B2C eCommerce sample data sources—no warehouse setup required.
**Section:** Sources
## Overview
Hightouch provides two built-in **Sample datasets** that let you explore the product without connecting your own data warehouse. They're hosted on a read-only database that Hightouch manages for you, so there are no credentials, networking, or permissions to configure.
Sample datasets are ideal for:
- Running your first sync end-to-end before your production source is ready.
- Evaluating Hightouch during a trial or POC.
- Learning how [models](/getting-started/concepts#models), [syncs](/getting-started/concepts#syncs), and [Customer Studio](/customer-studio/overview) work against realistic data.
- Testing a new [destination](/destinations/overview) integration without touching production data.
Sample datasets are real, customer-facing sources—not internal-only demo tooling. They appear in every workspace at the end of the source list, and usage of sample sources is excluded from your workspace's billing and analytics metrics.
Two sample sources are available:
- **B2B SaaS**: models a typical SaaS business with users and organizations, useful for testing CRM, marketing, and product analytics destinations.
- **B2C eCommerce**: contains customer profiles and cart events data for a B2C eCommerce company that retails shoe brands, useful for testing ad platforms, email, loyalty, and retention destinations.
## Add a sample source
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button.
Scroll to the bottom of the source catalog. The **Sample datasets** section contains both **B2B SaaS** and **B2C eCommerce**. Select the one you want to use.
Because the underlying database is managed by Hightouch, there are no credentials to enter and no connection test to run. Hightouch skips directly to the **Finalize source** step.
Give the source a name, review the default permissions, and click **Finish**. The source is ready to use immediately.
## Create a model
Models define what data to pull from your source. You can build [models](/models/creating-models) on top of a sample source using the [SQL editor](/models/sql-editor) or the [visual table selector](/models/table-selector). [dbt](/extensions/dbt-models), [Looker](/extensions/looker-models), and [Sigma](/extensions/sigma) modeling methods aren't supported on sample sources.
### Add a model
Go to the [**Models** overview page](https://app.hightouch.com/models) and click **Add model**.
### Select the sample source
On the **Select source** step, choose your **B2B SaaS** or **B2C eCommerce** sample source. Sample sources are clearly labeled with a **Sample** badge.
### Choose a modeling method
On the **Define model** step, choose either the **SQL query** editor or the **Table selector**.
### Use the pre-populated sample queries
When you pick a sample source in the SQL editor, Hightouch shows a **Sample queries** panel with ready-to-run examples tailored to the dataset. Click a query to populate the editor, then click **Preview results** to run it.
The included sample queries are a good starting point for common use cases—product-led upsell, abandoned-cart retargeting, seasonal campaigns, VIP audiences—and can be edited freely.
For a full reference on writing model queries, see [Create models with the SQL editor](/models/sql-editor) or [Create models with the table selector](/models/table-selector).
### Finalize the model
Once you're happy with the preview, click **Continue** to move to the **Finalize model** step. Give the model a name, select a **Primary key**, and click **Finish**.
Your new model appears on the [**Models** overview page](https://app.hightouch.com/models) and is ready to use in syncs.
## Send data with a sync
With a model in place, you can create a [sync](/syncs/overview) that delivers sample data to any destination in your workspace.
1. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click **Add sync**.
2. On the **Select model** step, pick the model you just built on top of the sample source.
3. Pick a destination. If you don't have one connected yet, [Google Sheets](/destinations/google-sheets) is the fastest way to see results—it's a [lite destination](/pricing/ss-pricing#lite-destinations) and easy to set up.
4. Map fields from your sample model to the destination's schema.
5. Set a schedule and click **Finish**.
For a full walkthrough, see [Create your first sync](/syncs/create-your-first-sync).
## Use in Customer Studio
Sample sources also work with [Customer Studio](/customer-studio/overview), so you can try audiences, journeys, and other activation features without real warehouse data.
### Create a parent model
Go to **Customer Studio → Schema** and switch the source selector at the top of the page to your sample source. If no parent models exist yet, click **Create parent model**.
On the **Select table** step of the **New parent model** flow, the **Sample queries** panel is pre-populated with starter queries. Pick one to auto-fill the SQL editor, then click **Preview results** to check your data before continuing.
On the **Configuration** step, give the parent model a name, pick a **Primary key**, and optionally select primary and secondary labels used when previewing audiences. Click **Create parent model** to finish.
Your new parent model appears in the **Schema** tab with all its columns visible and ready for audience building.
### Build audiences, journeys, and more
Once a parent model exists, you can use it just like any production source to:
- Build [audiences](/customer-studio/audience-overviews)—for example, VIP customers with items still in their cart, or Self-Serve organizations approaching their storage limit.
- Add [related models](/customer-studio/schema#2-add-related-models) (like purchases, subscriptions, or events) to enrich filtering.
- Create [traits](/customer-studio/traits) for reusable calculated fields such as lifetime value or last-seen timestamp.
- Orchestrate lifecycle [journeys](/customer-studio/journeys) across multiple destinations.
- Sync audiences to any connected destination with [Customer Studio syncs](/customer-studio/syncs).
See the [Customer Studio quick start](/customer-studio/quick-start) for an end-to-end walkthrough.
## Sample dataset contents
Each sample database is organized under the `public` schema. Use the **table selector** when creating a model to see the full list of tables and columns—the schemas may expand over time as Hightouch adds more examples.
### B2B SaaS
Models a SaaS business with records for users and their organizations. The dataset includes tables such as `public.users` and `public.organizations`, with attributes like `type`, `segment`, `total_storage_used_mb`, `total_users_invited`, and `documents_uploaded_l30d`. This makes it a good fit for testing:
- product-led growth workflows (for example, surfacing upsell opportunities to Self-Serve accounts nearing a storage limit)
- lifecycle marketing to admins and inactive users
- CRM and marketing-automation syncs to destinations like Salesforce, HubSpot, Marketo, Braze, or Segment
The **Sample queries** panel includes starter queries for *Product-led sales upsell opportunities*, *Lifecycle marketing to inactive admins*, and *New Business Tier product introduction*. A typical starter query—surfacing Self-Serve organizations using more than 50 GB of storage—looks like:
```sql
SELECT *
FROM public.organizations
WHERE type = 'Self-Serve'
AND CAST(total_storage_used_mb AS INT) > 50000
```
### B2C eCommerce
Models a direct-to-consumer retailer selling shoe brands. The dataset includes a `public.customers` table with attributes such as `customer_id`, `first_name`, `last_name`, `email`, `city`, `ltv`, `brand_affinity`, `VIP` tier, and `items_in_cart`, alongside cart-event data. This makes it a good fit for testing:
- audience-building workflows (for example, VIP customers or high-intent shoppers)
- abandoned-cart retargeting to ad platforms like Meta, Google Ads, or TikTok
- email and SMS campaigns through tools like Iterable, Klaviyo, or Attentive
- loyalty and retention use cases
The **Sample queries** panel includes starter queries for *Abandoned cart retargeting ads*, *Seasonal Rebook campaign - Nano X2*, and *Annual VIP celebration event in Florida*. A typical starter query—pulling non-VIP customers with items still in their cart—looks like:
```sql
SELECT *
FROM public.customers
WHERE VIP = 'Normal'
AND CAST(items_in_cart AS int) > 0
```
## Limitations
Sample sources are intentionally simple. A few things to know:
- **Read-only.** You can't write to a sample database, create tables, or modify its schema.
- **Shared across workspaces.** The underlying database is the same for every Hightouch customer. Don't store anything you expect to be private in a sample source—you shouldn't be able to, but it's worth stating plainly.
- **Lightning sync engine isn't supported.** Sample sources always use the Basic sync engine (Hightouch's `LocalDiffPlanner`). Lightning engine features such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution) therefore aren't available on syncs powered by a sample source.
- **Not for production.** Sample datasets are meant for exploration and demos only. When you're ready to run real workloads, [connect your own source](/sources/overview).
If a sync against a sample source reports a `permission denied` error, that usually means the sync was configured to use the Lightning engine. Switch the sync to the Basic engine (or recreate the sync) and the error will clear.
## Next steps
With your sample source connected, you can:
- Follow the [Create your first sync](/syncs/create-your-first-sync) tutorial end-to-end.
- Pick a destination from the [destination catalog](/destinations/overview) and map sample data to it.
- Explore [Customer Studio](/customer-studio/quick-start) to build audiences and journeys on top of sample data.
- When you're ready to move to production data, review the [Sources overview](/sources/overview) and add your own warehouse.
---
## SFTP
**URL:** https://hightouch.com/docs/sources/sftp
**Description:**
**Section:** Sources
## Overview
Hightouch supports syncing data from CSV files stored on a remote server via SFTP. To connect to Hightouch you need your hostname, port, and credentials for an SFTP user. This user needs to have permission to read the file you want to use as a data source.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **SFTP** and follow the steps below.
### Choose authentication method
You can select to use either a **password** or SSH **private key**. If using a **private key**, you can optionally include a **passphrase**.
### Configure your source
- **Host**: The hostname or IP address of your SFTP server.
- **Port**: The port number of your SFTP server. The default port number is 22, but yours may be different.
- **Username**: This can be your personal SFTP login or a dedicated user for Hightouch.
- Either your **password** or **private key**: Your private key must be in OpenSSH format and should include header and footer. For example:
```bash
-----BEGIN OPENSSH PRIVATE KEY-----
-----END OPENSSH PRIVATE KEY-----
```
- (Optional) **Private key passphrase**: If using an encrypted private key, you can specify a passphrase to decrypt it.
FTP configurations are not supported at this time.
## Test your connection
When setting up SFTP as a source for the first time, Hightouch validates your credentials. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull via SFTP. The file must have a `.csv` extension.
### CSV requirements
CSV files must meet the following requirements:
- They must have a header row. The values in the header row are automatically available as column names when you set up a sync.
- They must use comma-separated values; tabs and other delimiters aren't supported.
- They must use double quotes (`"`) for quoted values.
### Model setup
1. In Hightouch, go to the [**Models** overview page](https://app.hightouch.com/models).
1. Click **Add model.**
1. Select the SFTP source you previously created.
1. Enter the **relative path** to the CSV file you want to sync data from.
1. Preview your model's query results.
1. Click **Continue**.
1. Name your model and select its [**primary key**](/models/creating-models#unique-primary-key-requirement). Hightouch uses the primary key to determine which rows have been added, changed, or removed since the last sync.
## Tips and troubleshooting
### Unsupported Unicode escape sequence
This error may occur when previewing a model from an SFTP source if the file is not encoded in UTF-8. Re-export or convert the file to UTF-8 and retry the preview.
---
## SingleStore
**URL:** https://hightouch.com/docs/sources/single-store
**Description:** SingleStore is a modern relational database for cloud and on-premises delivering immediate insights for modern applications and analytical systems.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in SingleStore and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to SingleStore for information like your hostname, database, and credentials.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **SingleStore** and follow the steps below.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your SingleStore.
- **Port**: The port number of your SingleStore. The default is 3306, but yours may be different.
- **Database**: The name of the database in your SingleStore.
- **Verify CA Certificate**: Enable CA Certificate verification for a more secure connection.
When you enable **Verify CA Certificate**, Hightouch will require you to provide the CA certificate from SingleStore.
Get the certificate file from SingleStore by going to **Connect Directly** option of the workspace your database is in and downloading the **TLS/SSL Certificate**
You can then upload the certificate in Hightouch in the **CA Certificate** section.
### Provide your credentials
Enter the following fields into Hightouch:
- **User**: A dedicated user for Hightouch.
- (Optional) **Password**: The password for the user specified above.
## Test your connection
When setting up SingleStore as a source for the first time, Hightouch validates your credentials and access to your database. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, you've completed setting up SingleStore as a source. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from SingleStore.
The SingleStore source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
## Tips and troubleshooting
---
## Snowflake
**URL:** https://hightouch.com/docs/sources/snowflake
**Description:** Hundreds of industry leaders use Hightouch to turn Snowflake into a marketing, sales, success and operational engine
**Section:** Sources
## Overview
Hightouch lets you pull data stored in your Snowflake warehouse and push it to downstream destinations.
You can connect Snowflake to Hightouch using [Snowflake Partner Connect](https://hightouch.com/blog/hightouch-snowflake-partner-connect) to bypass the setup steps outlined below. You can learn more about this in [Snowflake's documentation](https://docs.snowflake.com/en/user-guide/ecosystem-partner-connect).
Connecting Hightouch to Snowflake requires some setup in both platforms. It's recommended to set up a service account with the correct permissions in Snowflake before configuring the connection in Hightouch.
---
## Snowflake credential setup
To allow Hightouch access to Snowflake, it's best to create a user specifically provisioned with access to the required tables and schemas. You can also use a personal Snowflake login for your credentials, as long as it has the correct permissions.
You can use the following SQL template to create a service account with the necessary roles and permissions with some considerations:
- This snippet provides an example of creating a service account; you may need to alter it depending on your Snowflake implementation details.
- RSA key pair authentication setup is detailed in the [Provide credentials](#provide-credentials) section.
- If you want Hightouch to access multiple databases, run the snippet multiple times, changing `ht_database` each time.
- If you want Hightouch to have read-only access, omit the last line of the query.
- The snippet includes lines for creating a new warehouse and database for Hightouch to use; if you already have databases and warehouses you intend to use, omit these lines.
```sql
-- Edit the following variables
set ht_username='HIGHTOUCH_USER';
set ht_default_warehouse='';
set ht_database='';
set ht_default_namespace='';
set ht_default_role='HIGHTOUCH_ROLE';
set ht_comment='Used for Hightouch integrations';
-- Set role for grants
USE ROLE ACCOUNTADMIN;
-- Create a role for Hightouch
CREATE ROLE IF NOT EXISTS identifier($ht_default_role)
COMMENT = $ht_comment;
-- Only if you want to create a new warehouse for Hightouch to use
CREATE WAREHOUSE IF NOT EXISTS identifier($ht_default_warehouse);
-- Create Hightouch's user
CREATE USER IF NOT EXISTS identifier($ht_username)
TYPE=service
rsa_public_key=''
DEFAULT_WAREHOUSE=$ht_default_warehouse
DEFAULT_NAMESPACE=$ht_default_namespace
DEFAULT_ROLE=$ht_default_role
COMMENT=$ht_comment;
-- Grant permissions to the role
GRANT ROLE identifier($ht_default_role) TO ROLE SYSADMIN;
GRANT USAGE ON WAREHOUSE identifier($ht_default_warehouse) TO ROLE identifier($ht_default_role);
GRANT ROLE identifier($ht_default_role) TO USER identifier($ht_username);
-- Only if you want to create a new database for Hightouch to use
CREATE DATABASE IF NOT EXISTS identifier($ht_database);
GRANT USAGE ON DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT USAGE ON ALL SCHEMAS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON ALL TABLES IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON FUTURE TABLES IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON ALL VIEWS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON FUTURE VIEWS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
-- Grant write access for the Lightning Sync Engine (faster syncs) & Sync Logs - Optional
USE identifier($ht_database);
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT OWNERSHIP ON SCHEMA hightouch_audit TO ROLE identifier($ht_default_role);
GRANT OWNERSHIP ON SCHEMA hightouch_planner TO ROLE identifier($ht_default_role);
```
If you opt to use [the Lightning engine](#choose-your-sync-engine), you must grant [additional permissions](#lightning-engine-setup).
Once you've created a Snowflake service account, you're ready to set up the connection in Hightouch.
---
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **Snowflake** and follow the steps below.
### Configure your source
Enter the following required fields into Hightouch:
- **Account identifier**: You can find this identifier at the beginning of your Snowflake URL, for example, (`https://ACCOUNT_IDENTIFIER.snowflakecomputing.com`). Only enter the `ACCOUNT_IDENTIFIER` and not the full URL.
- **Warehouse**: This specifies the warehouse to use when Hightouch executes queries in Snowflake.
You can choose to route all queries to a single warehouse, or configure separate warehouses optimized for different workloads (recommended):
- **Route all queries to a single warehouse**: Use this option if you want to use a single warehouse for all Hightouch operations.
- **Route queries to different warehouses (split by workload)** (Recommended): This option allows you to optimize performance and cost by routing different types of queries to different warehouses:
- **Batch warehouse**: For background jobs like sync runs, schema sampling, audience snapshots, and other workloads that are not latency-sensitive. This warehouse can be optimized for cost efficiency.
- **Interactive warehouse**: For latency-sensitive queries (e.g., audience previews) and heavier workloads that need more compute (e.g., identity resolution). This warehouse should be sized up for best experience.
- **Database**: This specifies the database to use when Hightouch executes queries in Snowflake.
**Account identifier** format may differ based on Snowflake account age.
For example, older Snowflake accounts often have identifiers that look like `ACCOUNT_LOCATOR.CLOUD_REGION_ID.CLOUD`, whereas newer Snowflake accounts have identifiers that look like `ORGNAME-ACCOUNT_NAME`.
For more details, visit [Snowflake's account identifier docs](https://docs.snowflake.com/en/user-guide/admin-account-identifier.html).
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
To set up the Lightning engine, ensure that the service account you previously created has the appropriate permissions.
Specifically, append these commands to the [service account creation](#snowflake-credential-setup):
```sql
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT OWNERSHIP ON SCHEMA hightouch_audit TO ROLE identifier($ht_default_role);
GRANT OWNERSHIP ON SCHEMA hightouch_planner TO ROLE identifier($ht_default_role);
```
### Provide credentials
Enter the following fields into Hightouch:
- **Username**: This can be your personal Snowflake login or a dedicated user for Hightouch. When using the Basic engine, this user must have read access to the data you wish to sync. If using the Lightning engine, you must also grant the permissions described in the preceding snippet. **Note**: Use the "Login name", which may [differ from the Snowflake username](https://docs.snowflake.com/en/user-guide/admin-user-management#using-classic-console).
- **Role** (optional) - Use this field to specify the role Hightouch should use when executing queries in Snowflake. If left blank, Hightouch uses the user's default role.
You have two options for finalizing your credentials:
- RSA key pair (recommended)
- Password
Snowflake is [deprecating password
authentication](https://www.snowflake.com/en/blog/blocking-single-factor-password-authentification/) in favor of RSA key pair authentication or multi-factor authentication. Hightouch will continue to support password authentication for existing customers until they have migrated.
#### RSA authentication
Generate a [private key](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#step-1-generate-the-private-key) and [public key](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#step-2-generate-a-public-key) by running the following commands in your terminal:
```bash
$ openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8
$ openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub
```
**Save** the password you used for the key, you will need to enter it when creating the Snowflake source in Hightouch.
Then in Snowflake, execute an `ALTER USER` command to assign the public key to your Hightouch Snowflake user:
```bash
alter user ht_user set rsa_public_key='MIIBIjANBgkqh...';
```
In the `Authentication Method` of the Snowflake Source settings, drag and drop or upload the private key you just generated into the **Private key file** field. Enter the password for your private key in the box below here, unless you used an unencrypted key.
---
## Test your connection
---
## Manage warehouse compute
Hightouch runs queries in Snowflake to power audiences, syncs, and other workflows. Because Snowflake bills primarily on compute, your warehouse setup, schema design, and job frequency can affect both cost and performance.
This section outlines recommended patterns to help you:
- Keep Hightouch’s Snowflake usage predictable and easy to review
- Scope Hightouch’s access to only the data it needs
- Avoid unnecessary compute for common activation workflows
For a warehouse-agnostic overview, see [**Manage warehouse compute**](/sources/warehouse-compute).
---
### Recommended Snowflake setup
When you connect Snowflake to Hightouch, we recommend that you:
- **Use a dedicated user, role, and warehouse**
So compute usage and data access are easy to track and control.
- **Set practical warehouse defaults**
Start with a modest size, shorten idle time, and add reasonable query time limits.
- **Separate interactive and scheduled work when needed**
Run UI-driven queries and background jobs on different warehouses if you need stronger performance isolation.
- **Design models with query cost in mind**
Keep parent models focused, handle large event tables carefully, and combine tables only when it clearly helps.
- **Review usage regularly**
Use Snowflake’s cost dashboards, resource monitors, and query history to confirm performance and spend.
---
### Use a dedicated user, role, and warehouse
Snowflake organizes access and compute around three objects:
- A **user** (who runs queries)
- A **role** (what they can access)
- A **virtual warehouse** (where queries run)
Creating all three specifically for Hightouch makes both compute and data access easier to manage.
---
#### Scope access with a dedicated role
The role controls which objects Hightouch can read and write.
- Grant read access only to the databases and schemas that back your Hightouch models.
- Grant write access only when required (for example, for Warehouse Sync Logs or Hightouch-managed audit schemas).
- Avoid broad account-level privileges.
If you need additional governance, Snowflake supports:
- Masking policies on sensitive columns
- Row- or column-level security
- Views that expose filtered or redacted data
These controls ensure Hightouch can only access the data you intentionally expose.
---
#### Separate Hightouch from human users
Create a dedicated Snowflake user for Hightouch:
- Set its default role to the Hightouch role.
- Set its default warehouse to the Hightouch warehouse.
- Use strong authentication (for example, RSA key pair authentication).
With a dedicated user and role, you can filter query history to see exactly what Hightouch is running.
---
#### Isolate compute with a dedicated warehouse
The warehouse is the compute engine Snowflake bills.
- Create a virtual warehouse dedicated to Hightouch (for example, `HIGHTOUCH_WH`).
- Attach it to the Snowflake source configuration in Hightouch.
A dedicated warehouse makes it easier to:
- Attribute Snowflake credits to Hightouch
- Adjust warehouse settings without impacting other workloads
---
### Set practical warehouse defaults
Once Hightouch has its own warehouse, a few settings determine how it behaves.
#### Choose a starting size and adjust as needed
Start with a modest size rather than the largest option available. Many teams begin with **Medium** and adjust based on real usage.
Watch for:
- How long syncs and other Hightouch jobs take
- Whether Customer Studio previews remain responsive
- Whether Snowflake reports consistent queuing
If performance is acceptable and the warehouse is mostly idle, try a smaller size.
If you see frequent queuing or slow UI performance, increase the size or simplify models before scaling up.
#### Reduce idle time with auto-suspend
Auto-suspend controls how long the warehouse stays running when idle.
Snowflake’s default (often 10 minutes) is higher than many teams need. Reducing auto-suspend to around **1 minute**, while keeping auto-resume enabled, is common.
This helps the warehouse shut down soon after Hightouch stops issuing queries while still starting automatically when new work begins.
#### Add query time limits as a safeguard
Statement timeouts limit how long a query can run and help prevent unexpectedly long-running workloads.
The default timeout (2 days) is typically higher than necessary. Set a maximum duration that:
- Covers your typical sync runtimes
- Prevents a query from running indefinitely
If queries hit time limits:
- Review Snowflake query history
- Identify the models or filters causing long runtimes
- Simplify the query before increasing the timeout
#### Use multi-cluster only when concurrency is the issue (optional)
**Concurrency** (sometimes called parallelization) means how many queries run at the same time.
Multi-cluster warehouses can add extra clusters during busy periods to handle higher concurrency.
This can help when:
- Many Hightouch jobs run at once
- You see consistent queuing during peak times
- Usage spikes unpredictably
If a single warehouse handles your workload without queuing, multi-cluster may not be necessary.
If you choose to go with a larger warehouse (M/L), you can ignore this suggestion.
---
### Separate interactive and scheduled work
Hightouch issues both interactive and batch queries:
- **Interactive:** audience previews, traits exploration, dashboards, and other UI actions
- **Batch:** sync runs, Identity Resolution jobs, heavy traits, sampling refreshes, and snapshots
If needed, configure Hightouch to run these on different Snowflake warehouses.
#### Common pattern: two warehouses
Many teams use:
- An **interactive warehouse** for fast UI queries
- A **batch warehouse** for scheduled and background jobs
To set this up:
1. Create two warehouses (for example, `HIGHTOUCH_INTERACTIVE_WH` and `HIGHTOUCH_BATCH_WH`).
2. Configure Hightouch to send interactive work to one and batch work to the other.
3. Set size and auto-suspend settings independently.
This keeps the UI responsive during heavy sync windows and gives you clearer cost control.
---
### Design Customer Studio models with query cost in mind
Warehouse configuration determines how much compute is available. [Schema design](/customer-studio/schema) strongly affects how much work each query must do and which data Hightouch touches.
#### Keep parent and related models simple
For Customer Studio:
- The parent model should represent the main entity (for example, customer or account).
- Include stable identifiers and frequently used segmentation fields.
- Include keys needed to join related models.
Related models can represent 1:many relationships such as orders or sessions.
Where possible, do complex joins and calculations ahead of time (for example, in dbt), and have Hightouch read from tables you’ve already built instead of re-running large joins every time.
#### Limit event tables to what activation requires
Event tables can grow very large. To keep queries manageable:
- Limit time windows to what activation actually needs
- Organize tables based on common filters (often date or tenant)
- Create smaller summary tables for common metrics when helpful
[**Traits**](/customer-studio/traits) can perform aggregations directly, which may reduce the need for additional summary tables.
#### Combine tables only when it clearly helps
As a general guideline:
- Keep data in separate tables by default.
- Combine tables only when it clearly improves common Hightouch queries.
- Avoid flattening very large tables unless it solves a specific performance problem.
For large or time-series tables, consider clustering once you’ve observed real query patterns in Snowflake.
---
### Improve audience building performance
If Customer Studio previews feel slow, you can increase warehouse size—but before doing that, consider enabling [**sampling**](/customer-studio/sampling).
- Sampling runs previews against a subset of rows for faster results.
- You can still run full queries for production syncs or reporting.
Sampling often provides a strong balance between responsiveness and Snowflake cost.
---
### Reduce cost
Overspending typically results from multiple factors rather than a single issue.
#### Reduce unnecessary query volume
Running more queries than needed increases cost by:
- Keeping warehouses active longer
- Increasing concurrency, which may require larger warehouses
If you need frequent updates, consider running those workloads on a right-sized warehouse and separating them from interactive work.
#### Right-size warehouses for your workload
Many teams overspend because their warehouse is larger than necessary.
Aim for balance: enough capacity to meet performance expectations without paying for idle time or excess headroom.
---
### Monitor cost and usage
Use Snowflake’s built-in tools to validate Hightouch usage.
#### Review cost dashboards
- Filter by warehouse name to track credit usage
- Compare Hightouch warehouses with other workloads
#### Use budgets and resource monitors
- Set budgets for Hightouch warehouses
- Configure alerts or automatic suspension for unexpected spikes
#### Inspect query history
- Filter by Hightouch user, role, or warehouse
- Identify expensive or slow queries
- Tie them back to specific models or workflows
Regular monitoring ensures Hightouch stays aligned with your performance, cost, and governance expectations.
---
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Snowflake.
The Snowflake source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
- leveraging existing [Looker Looks](/extensions/looker-models)
- leveraging existing [Sigma workbooks](/extensions/sigma)
The [SQL editor](/models/sql-editor) allows you to query data from all databases that your Snowflake service account [has access to](#snowflake-credential-setup).
The [table selector](/models/table-selector) only supports querying from the database specified in the [source configuration](#configure-your-source).
---
## FAQs
### How can I tell how much compute Hightouch is using?
If you’re using a dedicated warehouse for Hightouch, most high-level cost questions can be answered directly in the Snowflake console.
1. Navigate to **Admin > Cost Management**.
2. Open the **Consumption** tab.
3. Filter by your Hightouch warehouse name.
This view shows:
- Credit usage over time
- Breakdown by warehouse
- Query and service-level cost trends
You must be an `ACCOUNTADMIN` or have the necessary permissions to access these views.
If Hightouch uses its own warehouse, this report provides a clear view of total Snowflake spend attributable to Hightouch workloads.
---
### How can I identify specific queries driving cost?
For more detailed analysis, Snowflake exposes usage and query telemetry in the shared `SNOWFLAKE` database.
Two views are especially useful:
#### `SNOWFLAKE.ACCOUNT_USAGE.QUERY_ATTRIBUTION_HISTORY`
This view includes the `CREDITS_ATTRIBUTED_COMPUTE` column, which reports how many credits were consumed by each query.
- It accounts for concurrent queries by allocating weighted compute usage.
- It excludes idle time between query completion and warehouse shutdown.
- It allows you to identify the most expensive individual queries.
#### `SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY`
This related view includes:
- Full SQL query text
- Query duration
- Bytes scanned and written
- Rows produced
- Queuing time
- Spill metrics
Together, these views help you determine:
- Which queries consume the most credits
- Whether queuing is contributing to performance issues
- Whether large scans or joins are driving cost
The easiest way to access recent query history for a specific warehouse is via the table function `SNOWFLAKE.INFORMATION_SCHEMA.QUERY_HISTORY_BY_WAREHOUSE`, which allows filtering by `WAREHOUSE_NAME`.
#### Example: Identify expensive Hightouch queries in the last day
The following example joins query attribution and warehouse-scoped history to show the most expensive queries executed on a specific Hightouch warehouse in the last 24 hours:
```sql
SELECT
QAH.*,
QHBW.QUERY_TEXT,
QHBW.QUEUED_PROVISIONING_TIME,
QHBW.ROWS_WRITTEN_TO_RESULT
FROM
SNOWFLAKE.ACCOUNT_USAGE.QUERY_ATTRIBUTION_HISTORY AS QAH
JOIN (
SELECT
*
FROM
TABLE(
SNOWFLAKE.INFORMATION_SCHEMA.QUERY_HISTORY_BY_WAREHOUSE(
WAREHOUSE_NAME => 'HIGHTOUCH_SYNCS',
RESULT_LIMIT => 10000
)
)
) AS QHBW
ON QHBW.QUERY_ID = QAH.QUERY_ID
WHERE
QAH.START_TIME > CURRENT_DATE - INTERVAL '1 DAY'
ORDER BY
QAH.CREDITS_ATTRIBUTED_COMPUTE DESC;
```
You can modify:
- `WAREHOUSE_NAME` to match your Hightouch warehouse
- The time window in the `WHERE` clause
- The `RESULT_LIMIT` parameter
This query helps you:
- Rank queries by compute consumption
- Inspect SQL text to identify inefficient joins or scans
- Confirm that configuration changes (warehouse size, routing, schema adjustments) are having the intended effect
---
## Tips and troubleshooting
### Connection timeout
When initially [testing your connection](#test-your-connection), you may receive a connection timeout error.
Once a connection is established, subsequent API requests should happen more quickly, so it's best to retry the tests if they first fail.
You can do this by clicking **Test again**.
### Network error: Could not reach Snowflake
You may receive this error if the input for [Account identifier](#configure-your-source) is invalid.
Instead of using the complete Snowflake URL, for example, `https://ACCOUNT_IDENTIFIER.snowflakecomputing.com`,
ensure that you're only using the `ACCOUNT_IDENTIFIER` part of the URL, for example, `companyname-xu12345` or `companyname-xu12345.us-east-1`.
---
## Snowflake Iceberg
**URL:** https://hightouch.com/docs/sources/snowflake-iceberg
**Description:** Hundreds of industry leaders use Hightouch to turn Snowflake into a marketing, sales, success and operational engine
**Section:** Sources
## Overview
Hightouch lets you pull data stored in Iceberg Tables through your Snowflake warehouse and push it to downstream destinations.
You can connect Snowflake to Hightouch using [Snowflake Partner Connect](https://hightouch.com/blog/hightouch-snowflake-partner-connect) to bypass the setup steps outlined below. You can learn more about this in [Snowflake's documentation](https://docs.snowflake.com/en/user-guide/ecosystem-partner-connect).
Connecting Hightouch to Snowflake requires some setup in both platforms. It's recommended to set up a service account with the correct permissions in Snowflake before configuring the connection in Hightouch.
## Snowflake credential setup
To allow Hightouch access to Snowflake, it's best to create a user specifically provisioned with access to the required tables and schemas. You can also use a personal Snowflake login for your credentials, as long as it has the correct permissions.
You can use the following SQL template to create a service account with the necessary roles and permissions with some considerations:
- This snippet provides an example of creating a service account; you may need to alter it depending on your Snowflake implementation details.
- RSA key pair authentication setup is detailed in the [Provide credentials](#provide-credentials) section.
- If you want Hightouch to access multiple databases, run the snippet multiple times, changing `ht_database` each time.
- If you want Hightouch to have read-only access, omit the last line of the query.
- The snippet includes lines for creating a new warehouse and database for Hightouch to use; if you already have databases and warehouses you intend to use, omit these lines.
- Iceberg tables have limited support for timestamps with timezones. We recommend setting the default timezone of the Hightouch user to `UTC` to maintain consistent behavior.
```sql
-- Edit the following variables
set ht_username='HIGHTOUCH_USER';
set ht_default_warehouse='';
set ht_database='';
set ht_default_namespace='';
set ht_default_external_volume=''
set ht_default_role='HIGHTOUCH_ROLE';
set ht_comment='Used for Hightouch integrations';
-- Set role for grants
USE ROLE ACCOUNTADMIN;
-- Create a role for Hightouch
CREATE ROLE IF NOT EXISTS identifier($ht_default_role)
COMMENT = $ht_comment;
-- Only if you want to create a new warehouse for Hightouch to use
CREATE WAREHOUSE IF NOT EXISTS identifier($ht_default_warehouse);
-- Create Hightouch's user
CREATE USER IF NOT EXISTS identifier($ht_username)
TYPE=service
rsa_public_key=''
DEFAULT_WAREHOUSE=$ht_default_warehouse
DEFAULT_NAMESPACE=$ht_default_namespace
DEFAULT_ROLE=$ht_default_role
DEFAULT_TIMEZONE = 'UTC'
COMMENT=$ht_comment;
-- Grant permissions to the role
GRANT ROLE identifier($ht_default_role) TO ROLE SYSADMIN;
GRANT USAGE ON WAREHOUSE identifier($ht_default_warehouse) TO ROLE identifier($ht_default_role);
GRANT USAGE ON EXTERNAL VOLUME identifier($ht_default_external_volume) TO ROLE identifier($ht_default_role);
GRANT ROLE identifier($ht_default_role) TO USER identifier($ht_username);
-- Only if you want to create a new database for Hightouch to use
CREATE DATABASE IF NOT EXISTS identifier($ht_database);
GRANT USAGE ON DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT USAGE ON ALL SCHEMAS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON ALL TABLES IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON FUTURE TABLES IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON ALL VIEWS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON FUTURE VIEWS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
-- Grant write access for the Lightning Sync Engine (faster syncs) & Sync Logs - Recommended
USE identifier($ht_database);
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT OWNERSHIP ON SCHEMA hightouch_audit TO ROLE identifier($ht_default_role);
GRANT OWNERSHIP ON SCHEMA hightouch_planner TO ROLE identifier($ht_default_role);
```
If you opt to use [the Lightning engine](#choose-your-sync-engine), you must grant [additional permissions](#lightning-engine-setup).
Once you've created a Snowflake service account, you're ready to set up the connection in Hightouch.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **Snowflake Iceberg** and follow the steps below.
### Configure your source
Enter the following required fields into Hightouch:
- **Account identifier**: You can find this identifier at the beginning of your Snowflake URL, for example, (`https://ACCOUNT_IDENTIFIER.snowflakecomputing.com`). Only enter the `ACCOUNT_IDENTIFIER` and not the full URL.
- **Warehouse**: This specifies the warehouse to use when Hightouch executes queries in Snowflake.
- **Database**: This specifies the database to use when Hightouch executes queries in Snowflake.
- **External volume**: This specifies the integration point between Snowflake and your cloud provider's storage.
- **Base location**: This specifies the prefix within the External Volume where Iceberg tables will be created.
**Account identifier** format may differ based on Snowflake account age.
For example, older Snowflake accounts often have identifiers that look like `ACCOUNT_LOCATOR.CLOUD_REGION_ID.CLOUD`, whereas newer Snowflake accounts have identifiers that look like `ORGNAME-ACCOUNT_NAME`.
For more details, visit [Snowflake's account identifier docs](https://docs.snowflake.com/en/user-guide/admin-account-identifier.html).
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
To set up the Lightning engine, ensure that the service account you previously created has the appropriate permissions.
Specifically, append these commands to the [service account creation](#snowflake-credential-setup):
```sql
CREATE SCHEMA IF NOT EXISTS hightouch_audit;
CREATE SCHEMA IF NOT EXISTS hightouch_planner;
GRANT OWNERSHIP ON SCHEMA hightouch_audit TO ROLE identifier($ht_default_role);
GRANT OWNERSHIP ON SCHEMA hightouch_planner TO ROLE identifier($ht_default_role);
```
### Provide credentials
Enter the following fields into Hightouch:
- **Username**: This can be your personal Snowflake login or a dedicated user for Hightouch. When using the Basic engine, this user must have read access to the data you wish to sync. If using the Lightning engine, you must also grant the permissions described in the preceding snippet. **Note**: Use the "Login name", which may [differ from the Snowflake username](https://docs.snowflake.com/en/user-guide/admin-user-management#using-classic-console).
- **Role** (optional) - Use this field to specify the role Hightouch should use when executing queries in Snowflake. If left blank, Hightouch uses the user's default role.
You have two options for finalizing your credentials:
- RSA key pair (recommended)
- Password
Snowflake is [deprecating password
authentication](https://www.snowflake.com/en/blog/blocking-single-factor-password-authentification/) in favor of RSA key pair authentication or multi-factor authentication. Hightouch will continue to support password authentication for existing customers until they have migrated.
#### RSA authentication
Generate a [private key](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#step-1-generate-the-private-key) and [public key](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#step-2-generate-a-public-key) by running the following commands in your terminal:
```bash
$ openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8
$ openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub
```
Then in Snowflake, execute an `ALTER USER` command to assign the public key to your Hightouch Snowflake user:
```bash
alter user ht_user set rsa_public_key='MIIBIjANBgkqh...';
```
In the `Authentication Method` of the Snowflake Source settings, drag and drop or upload the private key you just generated into the **Private key file** field.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Snowflake.
The Snowflake Iceberg source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
The [SQL editor](/models/sql-editor) allows you to query data from all databases that your Snowflake service account [has access to](#snowflake-credential-setup).
The [table selector](/models/table-selector) only supports querying from the database specified in the [source configuration](#configure-your-source).
## Tips and troubleshooting
### Connection timeout
When initially [testing your connection](#test-your-connection), you may receive a connection timeout error.
Once a connection is established, subsequent API requests should happen more quickly, so it's best to retry the tests if they first fail.
You can do this by clicking **Test again**.
### Network error: Could not reach Snowflake
You may receive this error if the input for [Account identifier](#configure-your-source) is invalid.
Instead of using the complete Snowflake URL, for example, `https://ACCOUNT_IDENTIFIER.snowflakecomputing.com`,
ensure that you're only using the `ACCOUNT_IDENTIFIER` part of the URL, for example, `companyname-xu12345` or `companyname-xu12345.us-east-1`.
---
## SQL Server
**URL:** https://hightouch.com/docs/sources/sqlserver
**Description:** SQL Server is a relational database management system, or RDBMS, developed and marketed by Microsoft.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in Microsoft SQL Server and push it to downstream destinations. Connecting Hightouch to Microsoft SQL Server may require some setup in both platforms. If using an Azure provisioned Microsoft SQL Server, you need to configure your firewall settings.
## Azure firewall setup
*This section only applies when connecting to SQL Server instances hosted in Azure.*
By default, Azure protects your SQL Server from any external IP address from connecting to your SQL Server. Follow these steps to add Hightouch's IP addresses to the firewall:
1. Log into your [Azure Console Dashboard](https://portal.azure.com/#home).
2. Click on **SQL databases** under **Services**.
3. Select the SQL database you want to update.
4. Select **Set server firewall** .
5. Under **Public network access**, select **Selected networks**.
6. Scroll to **Firewall rules** and click **Add a firewall rule**.
7. Allowlist the [Hightouch IP addresses](https://hightouch.com/docs/security/networking#ip-addresses) for your region.
8. Click the **Save** button on the bottom left.
## Connection configuration
In Hightouch, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **Microsoft SQL Server** and follow the steps below.
### Choose authentication method
When creating a SQL Server source, you can either enter the Host, Port, Database, User, and Password by selecting **Password** or you can provide the full URI **Connection string**.
#### Connection string
Enter your connection string in this format: `Server=server;Database=database;User Id=username;Password=password;Encrypt=boolean`.
Once you've entered your connection string and clicked **Continue**, you can skip to the [Test your Connection](#test-your-connection) section of this doc.
### Choose connection type
### Configure your source
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your Microsoft SQL Server. If using the hostname, make sure to only insert the hostname and not the full URL (remove the leading `http://` and the final `/`).
- **Port**: The port number of your Microsoft SQL Server. The default is 1433, but yours may be different.
- **Database**: The name of the database in your Microsoft SQL Server.
- **User**: This can be your personal Microsoft SQL Server login or a dedicated user for Hightouch.
- (Optional) **Password**: The password for the user specified above.
Optionally you can enter a **Request timeout duration** and whether to **Trust server certification**.
## Test your connection
When setting up Microsoft SQL Server as a source for the first time, Hightouch validates your credentials and access to your database. Once the test passes, click **Continue** to finish setup.
If you see the following error message, you need to [update your firewall settings](#azure-firewall-setup) in your Azure Console.
```json
Cannot open server 'hightouch-test' request by login.
Client with IP address '54.196.30.169' is not allowed to access the server.
To enable access, use the Windows Azure Management Portal or run sp_set_firewall_rule
on the master database to create a firewall rule for this IP address or address range.
It may up take up to five minutes for this change to take effect.
```
## Next steps
Once your source configuration has passed the necessary validation, you've completed setting up Microsoft SQL Server as a source. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Microsoft SQL Server.
The Microsoft SQL Server source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
## Tips and troubleshooting
Most issues with the Microsoft SQL Server source can be resolved by verifying the firewall settings according to the setup instructions above.
If you see an error message similar to the following
```json
Login failed for user ''
```
or
```json
login failed for user '\'
```
verify that the username and password work locally. You may also need to adjust your Microsoft SQL Service authentication mode. See [Microsoft's documentation](https://learn.microsoft.com/en-us/sql/relational-databases/errors-events/mssqlserver-18456-database-engine-error?view=sql-server-ver16#login-failed-for-user-username-or-login-failed-for-user-domainusername) for more information about login errors and authentication modes.
---
## Azure Synapse
**URL:** https://hightouch.com/docs/sources/synapse
**Description:** Hightouch can turn Azure Synapse into a marketing, sales, success and operational engine.
**Section:** Sources
## Overview
Hightouch lets you pull data stored in Azure Synapse and push it to downstream destinations. Connecting Hightouch to Azure Synapse requires some setup in both platforms. In Azure Synapse, you need to configure your firewall settings.
## Azure firewall setup
By default, Azure protects your Synapse data from any external IP address from connecting to your Synapse workspace. Follow these steps to add Hightouch's IP addresses to the firewall.
1. Log into your [Azure Console Dashboard](https://portal.azure.com/#home).
2. Click on your Synapse workspace under **Resources**.
3. In the left sidebar, select **Security** > **Networking** .
4. Allowlist the [Hightouch IP addresses](https://hightouch.com/docs/security/networking#ip-addresses) for your region.
5. Click the **Save** button in upper left corner.
## Connection configuration
In Hightouch, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **Azure Synapse** and follow the steps below.
### Choose authentication method
When creating an Azure synapse source, you have four options to authenticate:
- **Password**: Use a username and password
- **Connection String**: Provide the full URI Connection String
- **Microsoft Service Principal with Client Secret**: Use a Microsoft Service Principal created using Microsoft Entra ID with a Client Secret
- **Microsoft Service Principal with Certificate**: Use a Microsoft Service Principal created using Microsoft Entra ID with a signed Certificate
### Choose connection type
### Configure your source
If connecting via **Password** or **Microsoft Service Principal**, you must first enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your Azure Synapse.
- **Port**: The port number of your Azure Synapse. The default is 1433, but yours may be different.
- **Database**: The name of the database in your Azure Synapse.
Optionally you can enter a **Request timeout duration** and whether to **Trust server certification**.
### Password
To connect using a username and password, enter the following fields into Hightouch:
- **Username**: This can be your personal Azure Synapse login or a dedicated user for Hightouch.
- **Password**: The password for the user specified above.
### Microsoft Service Principal
To connect using a Microsoft Service Principal, use the following instructions to create a new Service Principal:
1. Sign in to the [Microsoft Entra admin center](https://entra.microsoft.com/).
2. On the sidebar, select **Identity** > **Applications** > **App registrations**.
3. Select **New registration**.
4. Pick a name for your application and select **Accounts in this organizational directory only** as the supported account type.
5. Click **Register**.
6. Note your newly created **Application (client) ID** and **Directory (tenant) ID**.
7. Click **Add a certificate or secret**.
From here, you can opt to use either a **Client secret** or a **Certificate**.
#### Client secret
1. Select the **Client secrets** tab.
2. Click **New client secret**.
3. Name your client secret and select its expiration date. To maintain access to Azure Synapse, make sure to refresh your client secret prior to its expiration.
4. Note your client secret **Value** now as it cannot be retrieved later. It will be used to connect Hightouch to Azure Synapse.
#### Certificate
1. Select the **Certificates** tab.
2. Click **Upload certificate** and upload a valid signed certificate.
3. Note your certificate's **Thumbprint**.
For more detailed instructions on service principal creation, you can check out [Microsoft's documentation](https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal).
#### Synapse configuration
Next, we need to grant the service principal access to the Azure Synapse instance:
1. Navigate to your instance with the [Azure Portal](https://portal.azure.com/#home)
2. On the sidebar, select **Access control (IAM)**
3. Click **Add** > **Add role assignments**
4. Select the role you would like to assign to your service principal (Hightouch will need at least the **Reader** role)
5. On the next screen, select the service principal that was created above and apply the changes.
For more details, please take a look at [Microsoft's documentation](https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-portal).
#### Credentials
Finally, you can enter the following fields into Hightouch that were created during the above instructions:
- **Tenant ID** (or Directory ID)
- **Client ID** (or Application ID)
If you chose to create a **Client Secret** for your service principal, you'll enter it here.
If you instead chose to create a **Certificate**, enter in your certificate's **Thumbprint** and the associated **private key**. Make sure that the private key is in PEM format and includes the header and footer.
### Connection String
If connecting via **Connection String**, simply enter your connection string in this format: `Server=server;Database=database;User Id=username;Password=password;Encrypt=boolean`.
### Choose your sync engine
#### Basic versus Lightning engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
#### Lightning engine setup
Please note that Lightning engine is only supported for Azure Synapse instances that use a [dedicated SQL pool](https://learn.microsoft.com/en-us/azure/synapse-analytics/sql-data-warehouse/sql-data-warehouse-overview-what-is). Serverless Azure Synapse does not have the capabilities to support the Lightning engine.
To set up the Lightning engine, run the following commands. Make sure to replace **your_service_principal_or_user** with the name of the service principal or user used to connect to your Synapse instance.
```sql
IF NOT EXISTS (SELECT 1 FROM sys.schemas WHERE name = 'hightouch_audit')
BEGIN
EXEC('CREATE SCHEMA [hightouch_audit]');
END;
IF NOT EXISTS (SELECT 1 FROM sys.schemas WHERE name = 'hightouch_planner')
BEGIN
EXEC('CREATE SCHEMA [hightouch_planner]');
END;
GRANT CONTROL ON SCHEMA::hightouch_audit TO [your_service_principal_or_user];
GRANT CONTROL ON SCHEMA::hightouch_planner TO [your_service_principal_or_user];
```
## Test your connection
When setting up Azure Synapse as a source for the first time, Hightouch validates your credentials and access to your database. Once the test passes, click **Continue** to finish setup.
If you see the following error message, you need to [update your firewall settings](#azure-firewall-setup) in your Azure Console.
```
Cannot open server 'hightouch-test' request by login. Client with IP address '54.196.30.169' is not allowed to access the server. To enable access, use the Windows Azure Management Portal or run sp_set_firewall_rule on the master database to create a firewall rule for this IP address or address range. It may up take up to five minutes for this change to take effect.
```
## Next steps
Once your source configuration has passed the necessary validation, you've completed setting up Azure Synapse as a source. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Azure Synapse.
The Azure Synapse source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
## Tips and troubleshooting
---
## Teradata Vantage
**URL:** https://hightouch.com/docs/sources/teradata-vantage
**Description:** Teradata Corporation is a leading data and analytics company that specializes in providing enterprise-level data warehousing, data management, and advanced analytics solutions
**Section:** Sources
## Overview
Hightouch lets you pull data from Teradata Vantage and push it to downstream destinations. Hightouch connects to Teradata using [Open Database Connectivity](https://en.wikipedia.org/wiki/Open_Database_Connectivity) (ODBC).
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **Teradata Vantage** and follow the steps below.
### Choose connection type
### Configure your source
- **Host**: The hostname or IP address of your Teradata instance.
- **Username**: This can be your personal Teradata Vantage login or a dedicated user for Hightouch.
- **Password**: The password for the specified user.
## Test your connection
When setting up Teradata Vantage as a source for the first time, Hightouch validates your credentials. Once the test passes, click **Continue** to finish setup.
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models)
to define which data you want to pull via Teradata Vantage.
The Teradata Vantage source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
## Tips and troubleshooting
---
## Trino
**URL:** https://hightouch.com/docs/sources/trino
**Description:** Trino is an open-source distributed SQL query engine designed to query large data sets distributed over one or more heterogeneous data sources.
**Section:** Sources
## Getting started
Hightouch lets you pull data stored in [TrinoDB](https://trino.io/) (previously PrestoSQL) and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to your Trino instance for information like your host, port, database name, and credentials.
Hightouch can also connect to managed Trino services like Starburst.
## Connection configuration
To get started, go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button. Select **Trino** and follow the steps below.
### Choose connection type
### Configure your source
Enter the following fields into Hightouch:
- **Host**: The hostname or IP address of your Trino coordinator server.
- **Port**: The port number of your Trino coordinator server.
- (Optional) **Default catalog**: This specifies the catalog that will be used when Hightouch executes queries in Trino.
- **Engine**: Select whether to use the Trino or Presto engine to execute queries.
- **Connector**: Select the connector that will be used to access the data source used by the sync engine. We support the following connectors:
- Hive
- Iceberg
- PostgreSQL
- (Optional) **Disable encryption**: Encryption is enabled by default. Check this option if your Trino instance doesn't support encrypted connections.
### Choose your sync engine
#### Basic versus Lightning engine comparison
#### Lightning engine setup
To set up the Lightning engine, you need to grant Hightouch write access to your Trino connector. The steps to do so depend on the connector. These snippets serve as a reference and should be run on your Trino server.
**Hive & Iceberg**
```sql
CREATE SCHEMA IF NOT EXISTS .hightouch_audit;
CREATE SCHEMA IF NOT EXISTS .hightouch_planner;
```
**PostgreSQL**
```sql
CREATE USER hightouch_user WITH PASSWORD '********';
CREATE SCHEMA IF NOT EXISTS .hightouch_audit;
CREATE SCHEMA IF NOT EXISTS .hightouch_planner;
GRANT CREATE, USAGE ON SCHEMA .hightouch_audit TO hightouch_user;
GRANT CREATE, USAGE ON SCHEMA .hightouch_planner TO hightouch_user;
```
The snippet provisions two schemas (`hightouch_planner` and `hightouch_audit`) for storing logs of previously synced data and, in some cases, creates a dedicated SQL user. Hightouch must be able to read and write to these schemas, but the specific username and schema names might vary.
### Provide credentials
Enter the following fields into Hightouch:
- **User**: This can be your personal Trino login or a dedicated user for Hightouch. At minimum, this user must have read access to the data you wish to sync. If using the Lightning sync engine, you must also grant this user additional permissions as described above.
- **Password**: The password for the user specified above.
## Test your connection
## Next steps
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up [models](/getting-started/concepts#models) to define which data you want to pull from Trino.
The Trino source supports these modeling methods:
- writing a query in [the SQL editor](/models/sql-editor)
- using the visual [table selector](/models/table-selector)
- leveraging existing [dbt models](/extensions/dbt-models)
## Data types
Hightouch supports all built-in [Trino data types](https://trino.io/docs/current/language/types.html). Custom data types provided by plugins are represented as strings.
## Tips and troubleshooting
---
## Manage warehouse compute
**URL:** https://hightouch.com/docs/sources/warehouse-compute
**Description:** Learn how warehouse compute works with Hightouch and how to optimize cost, performance, and governance through warehouse configuration, schema design, and monitoring strategies.
**Section:** Sources
## Overview
When you connect Hightouch to your warehouse as a source, Hightouch runs directly in that warehouse. When you build audiences, run syncs, calculate traits, or execute journeys in Hightouch, your warehouse runs SQL queries to power those actions.
Because of this, Hightouch contributes to your **warehouse compute** usage.
Many customers find that Hightouch accounts for less than 1% of their annual warehouse compute costs. This guide explains how to plan for, monitor, and optimize that usage intentionally.
You'll learn:
- What typically increases warehouse cost
- How your data structure affects performance
- What you can change in your warehouse
- What you can change in Hightouch
---
## How warehouse billing works
Most warehouses charge separately for:
- **Compute** – the processing power used to run queries
- **Storage** – how much data is stored
- **Platform services** – management and metadata features
Hightouch primarily affects **compute**.
Each time your warehouse evaluates an audience, calculates a trait, runs a sync, or processes a journey step, compute is consumed.
---
## What usually increases warehouse compute
Warehouse cost typically goes up for one of five reasons:
1. Queries run more often
2. Queries do more work
3. More queries run at the same time
4. The warehouse is not the right size for the workload
5. The warehouse stays on longer than necessary
### 1. Queries run more often
More frequent queries increase total compute usage. Common drivers include:
- [Syncs](/syncs/overview) scheduled more frequently
- [Triggered journeys](/customer-studio/journeys) running continuously
- [Audiences](/customer-studio/usage) recalculating often
If a use case doesn’t require real-time updates, consider lowering the schedule. Many background jobs work just as well daily or weekly.
### 2. Queries do more work
Queries that perform large joins, heavy aggregations, or full-table scans consume more compute.
Queries become more expensive when they:
- Join large tables repeatedly
- Scan long time ranges of event data
- Perform heavy calculations each time they run
To reduce compute, move complex joins or calculations earlier in your data pipeline (for example, in dbt) and have Hightouch read from a simpler, materialized table. This avoids repeating the same heavy work every time a sync, audience, or trait runs.
### 3. More queries run at the same time
**Concurrency** (sometimes called parallelization) refers to how many queries run at the same time.
When many syncs, journeys, or recalculations start at once, your warehouse must process them simultaneously.
Higher concurrency can:
- Increase short-term cost spikes
- Slow down interactive work
Adjust concurrency based on what your warehouse can handle and how quickly jobs need to finish. If jobs are not time-sensitive, spreading them out can reduce peak usage.
### 4. The warehouse is not the right size for the workload
Warehouse size (sometimes called a compute engine) determines how much compute is available at once.
- Larger warehouses process more data per unit time but cost more while running.
- Smaller warehouses cost less while running but may introduce wait times during spikes.
Start with a modest warehouse size and scale up only if jobs regularly take too long. Many customers begin with a **Medium** warehouse.
Some warehouses can automatically add capacity during busy periods (for example, multi-cluster or serverless) and scale down when demand drops.
### 5. The warehouse stays on while idle
Warehouses that remain running when no queries are executing still incur compute cost.
Idle behavior depends on:
- **Auto-suspend** (how quickly the warehouse turns off when it’s idle)
- How jobs are **scheduled** throughout the day
- Whether background jobs run **continuously** or in **batches**
Many environments default to ~10 minutes of inactivity. Lowering auto-suspend to around **1 minute**—while keeping **auto-resume enabled**—is often one of the simplest ways to reduce unnecessary compute usage.
---
## How Hightouch uses your warehouse
Understanding how Hightouch workloads behave helps explain why certain cost drivers appear in practice.
Hightouch workloads fall into a few broad categories. Understanding them helps you decide what needs to be fast and what can run in the background.
### Interactive workloads
Interactive workloads are user-driven and show up directly in the platform UI:
- Audience previews in [**Audiences**](/customer-studio/usage)
- Trait previews in [**Traits**](/customer-studio/traits)
- Insights and dashboards in [**Intelligence**](/campaign-intelligence/overview)
- Some AI-assisted workflows
These workloads prioritize quick load times because users expect fast responses.
### Batch workloads
Batch workloads run on a schedule or in the background:
- Sync runs to downstream tools
- Identity Resolution jobs
- Large trait recalculations
- Audience snapshots and backfills
These workloads prioritize processing large volumes of data reliably and can often run on a smaller or lower-priority warehouse.
### Background jobs
Some jobs run periodically but are not time-sensitive:
- Match Booster enrichment refreshes
- Cached column suggestion refreshes
- Non-urgent audience updates
Because these jobs tolerate delay, they are good candidates for:
- Lower-frequency schedules
- Smaller, dedicated warehouses
- Running during off-peak windows
---
## Strategies to reduce warehouse compute
There are three main levers that affect how much compute Hightouch consumes:
1. [**Warehouse configuration**](#1-warehouse-configuration)
2. [**Data modeling and table design**](#2-data-modeling-and-table-design)
3. [**Hightouch configuration**](#3-hightouch-configuration)
Optimizing across all three provides predictable performance and cost control.
---
## 1. Warehouse configuration
### Use a dedicated warehouse and user for Hightouch
Create:
1. A **dedicated warehouse** for Hightouch
- Allows you to clearly see what Hightouch costs and prevents it from competing with other workloads.
2. A **dedicated user** (or service account) and **role** with access only to the data Hightouch needs
- Makes it easier to control access and audit the queries Hightouch runs.
Starting here provides the clearest cost visibility and strongest governance controls.
### Choose the right warehouse size
Start with a modest size (often **Medium**) and scale based on runtime and queueing patterns.
Increase size if jobs regularly exceed acceptable runtimes.
Decrease size if the warehouse is mostly idle.
### Use serverless or multi-cluster options when appropriate
If your warehouse supports dynamic scaling (for example, serverless or multi-cluster configurations), consider enabling it.
These options:
- Add capacity automatically during busy periods
- Scale down when demand drops
- Prevent you from permanently running a larger warehouse
This is especially helpful if you have steady daily usage with occasional spikes.
### Reduce idle time
Lower **auto-suspend** (around **1 minute** is common) and keep **auto-resume** enabled to prevent idle compute waste.
### Control how many queries run at once
Adjust how many queries your warehouse can run at the same time.
- Increasing concurrency allows more work to finish sooner but may increase peak usage.
- Decreasing concurrency smooths load but can extend total runtimes.
Adjust this setting based on whether cost spikes or missed deadlines are the bigger concern.
**For data teams**
Review warehouse query history for:
- Time spent queued before execution
- Peak credit usage during sync windows
- Missed downstream deadlines
Compare before and after making concurrency adjustments.
### Set query time limits
Configure maximum query runtimes to prevent a single query from running indefinitely.
- Set reasonable time limits to guard against runaway compute usage.
- If queries frequently hit the limit, optimize the underlying model before increasing the threshold.
### Use budgets and resource monitors
Most warehouses provide built-in tools to:
- Track credit usage
- Set spending thresholds
- Alert on anomalies
Using these with a dedicated Hightouch warehouse makes unusual usage easier to detect and investigate.
### Route different workloads to different warehouses
If your warehouse supports multiple compute engines, route:
- **Interactive workloads** (e.g., audience previews, dashboards, AI-powered workflows) to a warehouse tuned for low-latency (fast UI queries)
- **Batch workloads** (e.g., syncs, IDR, heavy traits) to a warehouse tuned for throughput (large scheduled jobs)
This keeps the UI responsive during heavy batch windows and separates cost management by workload type.
----
## 2. Data modeling and table design
These strategies reduce how much work your warehouse performs each time Hightouch runs a query.
### Turn complex queries into tables
If a model performs heavy joins or aggregations every time it runs, your warehouse must repeat that work for every sync, audience, or trait.
This can quietly increase compute usage over time.
To reduce repeated work:
- Run those joins and calculations earlier in your data pipeline (for example, in dbt) and have Hightouch read from a simpler, materialized table or view that serves as a presentation layer.
- Simplify the model Hightouch reads from.
If a simple `SELECT * FROM your_model LIMIT 1` query runs slowly, the model is likely doing too much work at read time and should be refactored.
**For data teams**
Check warehouse query history for:
- Repeated large joins across the same tables
- High bytes scanned for simple filters
- Long runtimes even when limiting rows
If the same heavy logic appears across many Hightouch queries, it likely belongs upstream rather than inside the model Hightouch reads.
### Structure your tables intentionally
Hightouch frequently runs queries that filter, join, and aggregate data to compute audiences, traits, and journey steps. Efficient table structure reduces repeated work and lowers compute usage.
A practical default is:
- **Keep core warehouse layers normalized by default** (separate entities into separate tables and follow good modeling practices).
- **Expose a flattened, presentation-layer table or view for Hightouch** when it clearly removes slow, repeated joins for the way you actually query the data (especially for Customer Studio parent models).
- **Avoid denormalizing very large fact tables unnecessarily**, as this can increase how much data the warehouse must read; instead, denormalize surgically where it meaningfully reduces query cost for common patterns.
For time-series and large tables, consider adding **clustering keys** (how your warehouse organizes data to make common filters faster) once you understand real query patterns.
---
## 3. Hightouch configuration
These strategies involve controls inside Hightouch that affect how often and how intensively queries run.
### Simplify the models that power your Customer Studio schema
Customer Studio uses **parent**, **related**, and **event** models to evaluate audiences and traits. Your [schema design](/customer-studio/schema) directly affects how much work your warehouse must perform each time a query runs.
If these models contain complex logic, repeated joins, or large amounts of unnecessary data, every audience preview, sync, and journey step becomes more expensive.
As a general rule, use:
- **Parent models** for users or accounts
- **Related models** for supporting records (like orders or subscriptions)
- **Event models** for behavioral activity
Keeping these entities separate by default makes queries more predictable and reduces repeated work.
**For data teams**
Review models that power Customer Studio and look for:
- Long chains of joins inside a single model
- Heavy aggregations calculated at read time
- Queries that remain slow even when limiting rows
If a model is doing significant transformation work each time it’s queried, consider moving that logic earlier in your data pipeline so Hightouch reads from a simpler structure.
### Limit large event tables to what activation requires
**Event tables** can grow quickly. To keep them efficient:
- Limit time windows to what activation actually requires
- Use pre-aggregated tables for common metrics when helpful
- Organize tables based on the columns most often used in filters (for example, date or tenant)
For large event models, partition event tables by a timestamp column and always include a partition-aligned time filter in audiences and traits to avoid full-table scans.
Traits can perform aggregations directly for lighter or audience-specific metrics. For heavy or widely reused metrics, pre-aggregated tables or materialized views are usually more efficient than re-aggregating on every trait or audience run.
**For data teams**
Review whether:
- Audiences scan long historical windows
- Event volume has increased significantly
- Date filters still result in large scans
If so, narrow time windows or reorganize the table to better support common filters.
### Schedule background jobs at the lowest effective frequency
Many workloads do not require high-frequency updates.
- Reduce sync frequencies for enrichment updates, cached column suggestions (often daily or weekly is sufficient), and non-time-sensitive audiences
- Choose schedules that match when stakeholders actually use the data
See [**Schedule syncs in the UI**](/syncs/schedule-sync-ui) to learn how to set up sync schedules.
### Use sampling and fast queries for exploration
Audience and trait exploration does not always require exact counts.
- **Sampling** maintains a smaller, regularly refreshed copy of your parent model.
- **Fast queries** instruct Hightouch to use that sample for previews instead of scanning the full dataset.
This significantly reduces the amount of data scanned during exploration and provides much faster previews in exchange for approximate (rather than exact) counts.
Fast queries and sampling affect **previews only**. They do not change final audience membership or sync volume.
#### Enable fast queries in audiences
Use fast queries when:
- Brainstorming logic
- Validating filters
- Comparing audience shape
When you need exact numbers (for example, right before go-live, for small or highly targeted audiences, or for compliance and reporting) turn fast queries off to run against the full dataset.
To enable fast queries:
1. Go to **Customer Studio → Audiences**.
2. Open an audience.
3. Enable the **Fast queries** toggle in the audience header, if available.
#### Enable sampling in models
Sampling is especially helpful when:
- Your parent model contains millions or billions of rows
- Audience previews feel slow during filter changes
- Data teams want to reduce compute usage from iterative exploration
See [**Sampling**](/customer-studio/sampling/) for detailed setup instructions and configuration guidance.
### Control how many jobs Hightouch runs at once
Work with your Hightouch team to modify the number of concurrent workers that are available when triggering syncs within your Hightouch workspace.
Hightouch can run many jobs at the same time. This setting controls how many Hightouch syncs and background jobs are initiated in parallel. It is separate from warehouse-level concurrency.
- **Higher concurrency** reduces how long jobs take overall but increases short-term compute usage.
- **Lower concurrency** spreads work out over time and reduces peak load.
In general, keep Hightouch workspace concurrency at or below your warehouse’s effective concurrency limits (and any queue timeouts) so sync queries aren’t killed while waiting for a warehouse slot.
Match this setting to your warehouse limits and job timing requirements.
----
## If warehouse costs increase unexpectedly
Start with these checks:
1. Did sync or journey frequency change?
2. Did event volume grow?
3. Did new audiences or traits launch?
4. Did concurrency settings change?
5. Did auto-suspend settings change?
6. Did a data model change introduce heavier joins or scans?
Most cost increases can be traced to one of these changes.
---
## Alerting
**URL:** https://hightouch.com/docs/syncs/alerting
**Description:** You can set Hightouch alerts to monitor your syncs, including successes and errors, through SMS, Slack, email, and PagerDuty.
**Section:** Syncs
We recently released an upgraded alerting system. If you’d like access to the
new features, please{" "}
## Overview
Hightouch alerting helps you monitor sync health at a glance. Each sync is evaluated against a set of alert triggers that check for issues like failed runs, rejected rows, slow performance, or unexpected data drops.
Rather than showing only the result of the last run, each sync now displays a
**rollup** of its current alerting state. If no triggers are alerting, the
sync shows “No alerts.” If one or more are in a CRITICAL or WARNING state,
you’ll see the number of active alerts.
This gives you a more accurate picture of whether the sync is healthy, degraded, or needs attention—regardless of whether it ran recently.
Each alert is tracked at the **trigger level**. You can fine-tune thresholds, mute specific conditions, or change recipient settings either in bulk (per destination) or individually (per sync).
## Alerts vs. notifications
In Hightouch, an **alert** is a signal that a sync has entered a CRITICAL or WARNING state. A **notification** is what you receive (via email, Slack, SMS, or PagerDuty) when someone is subscribed to alerts at that level.
- All syncs have **fatal error** and **rejected row** triggers enabled by default. These cannot be disabled, but their thresholds can be customized.
- You’ll only receive a **notification** if a recipient is configured to receive alerts at that level.
- Recipients can be set at the **destination level** or overridden per sync.
- Each recipient can be configured to receive only **CRITICAL** alerts, or **CRITICAL + WARNING**.
### When notifications are sent
You receive **one notification per trigger status change**, not per run.
For example:
- A sync triggers a rejected row alert → you receive one WARNING notification
- It keeps rejecting rows in future runs → no new alerts
- It rejects many more rows in a future run, exceeding the CRITICAL threshold → you receive one CRITICAL notification
- It recovers → you receive a recovery alert
- Later, it fatally fails → you receive one CRITICAL notification
This avoids noise while still ensuring you’re informed when something breaks or escalates.
The costs for SMS alerting are covered by Hightouch.
---
## Configuration
Every sync starts with default alerting enabled:
- **Fatal error alerts**: CRITICAL, triggered by syncs that can’t run due to infrastructure or config issues.
- **Rejected row alerts**: WARNING, triggered by row-level sync failures.
These default triggers are always active but can be tuned with custom thresholds.
### How to manage alert configuration
You can manage alerting behavior at two levels:
#### 1. At the destination level (bulk configuration)
1. Navigate to the **Destinations** page.
2. Select a destination and go to the **Alerting** tab.
3. Set default recipients and trigger thresholds for all syncs using this destination.
#### 2. At the individual sync level (override)
1. Navigate to the **Syncs** page and open a sync.
2. Go to the **Alerting** tab.
3. Adjust trigger thresholds and notification recipients for that sync.
Changes here override any destination-level settings for this sync.
When monitors are changed or enabled/disabled, the changes will take effect
the next time the trigger thresholds are evaluated; after the next sync run,
or approximately every five minutes for time-based monitors.
## Trigger types
Each sync supports multiple trigger types that determine when it should be considered alerting. These conditions monitor for issues like failed runs, data quality problems, and unexpected changes in sync behavior.
All syncs have **fatal error** and **rejected row** triggers enabled by
default. These cannot be disabled, but their thresholds can be customized.
Other triggers are optional and configurable at the destination or sync level.
---
### Fatal errors
Detects infrastructure- or configuration-level sync failures that prevent a run from completing.
- **Default behavior**: Alert if one or more sync runs fail to finish
- **Custom options**: Configure a higher failure streak (e.g., alert only after 3 consecutive failures)
- **Evaluation timing**: Evaluated immediately after each sync run ends, regardless of whether the run completed successfully
- **Metric source**: Sync run status (completed, failed)
- **Logic**: Evaluates the number of consecutive failed runs and compares to the configured threshold. A sync run is considered "fatally failed" if it could not complete due to issues like:
- Unreachable source or destination
- Invalid credentials
- Broken SQL or API query logic
- Misconfigured transformation or mapping
---
### Rejected rows
Identifies sync runs that contain rejected records due to schema mismatches, data validation failures, or destination-level constraints.
- **Default behavior**: Alert if at least 1 row is rejected
- **Custom options**: Alert only if rejections exceed a configured absolute count or relative percentage
- **Evaluation timing**: Evaluated immediately after each sync run ends, regardless of whether the run completed successfully
- **Metric source**: Sum of all rejected rows reported by the sync engine across all operation types (i.e., added, changed, or removed)
- **Logic**: Compares rejected row count (or percentage of attempted rows) against user-defined thresholds. You can choose to measure:
- Absolute count of rejected rows OR
- Percentage of **queried** rows that were rejected
---
### Model size
Monitors the number of rows returned by a sync’s source model query, which may indicate issues with upstream filters, joins, or warehouse state.
- **Default behavior**: Alert if the model returns fewer than 1 row at sync runtime
- **Custom options**: Configure a different minimum row count (e.g., alert if fewer than 1000 rows are returned)
- **Evaluation timing**: Evaluated immediately after each sync run ends, regardless of whether the run was successful
- **Metric source**: Raw row count returned by the model at sync run time
- **Logic**: Compares returned row count against the configured lower bound. If the count is below threshold, a warning or critical status is triggered depending on severity configuration.
---
### Sync duration
Tracks sync performance by measuring wall-clock runtime from start to finish.
- **Default behavior**: Alert if runtime exceeds 60 minutes
- **Custom options**: Set a duration threshold (in hours) between 1 and 24
- **Evaluation timing**: Evaluated every 5 minutes; triggers as soon as duration exceeds threshold, even if the run is not complete
- **Metric source**: Calculated as the time from when the sync was enqueued to the time it finished, or to now if the sync is still running
- **Logic**: Evaluates total time spent across all sync phases— queued, querying, syncing, and writing logs— and compares against the configured duration threshold.
---
### Successful operations (Throughput)
Detects stalled or idle syncs that are no longer pushing meaningful data to their destination.
- **Default behavior**: Alert if fewer than 1 successful operation occurred in the last 7 days
- **Custom options**: Customize the operation threshold and time window (e.g., alert if fewer than 50 ops in 24 hours)
- **Evaluation timing**: Evaluated every 5 minutes; triggers as soon as duration exceeds threshold, even if the run is not complete
- **Metric source**: Aggregated sum of successfully added, changed, and removed rows across all sync runs within the time window
- **Logic**: Compares the total number of successful operations in the time window against the configured threshold.
---
## Muting triggers
You can mute individual alert triggers for a sync to temporarily suppress notifications.
- Muting prevents notifications, but the sync's alert status still updates in the UI
- You can mute for a specific duration (e.g., 1 hour, 1 day) or indefinitely
- Muting is helpful during planned maintenance, known issues, or when you need quiet hours
To mute a trigger, click the "Mute..." link at the top of the alert on the sync page, or:
1. Go to the sync’s **Alerting** tab
2. Click the volume button on the trigger
3. Select a mute duration from the options
Muted triggers still appear in rollups and dashboards—but without triggering notifications to your team.
---
## Array expansion
**URL:** https://hightouch.com/docs/syncs/array-expansion
**Description:** Expand array columns into individual sync rows so each array element is synced as a separate record to your destination.
**Section:** Syncs
## Overview
**Array expansion** lets you take a column that contains an array of values and expand each element into its own row during a sync. This is useful when your source data stores multiple identifiers or values in a single array column, but your destination expects one record per value.
For example, if a user has multiple device IDs stored in an array column, array expansion creates a separate sync row for each device ID, allowing you to sync each one individually to your destination.
Array expansion is currently supported with the **lightning sync engine** on **Postgres**, **Snowflake**, **BigQuery**, and **Databricks** sources.
## How it works
When you map an array-type column as the identifier in your sync configuration, Hightouch automatically enables array expansion. Each element of the array becomes a separate row, and a generated column is used as the identifier value sent to the destination.
### Example
Consider this source table with an array column `device_ids`:
| user_id | name | device_ids |
|---------|---------|---------------------------|
| 1 | Alice | `["ios_a1", "android_b2"]`|
| 2 | Bob | `["ios_c3"]` |
| 3 | Charlie | `[]` |
| 4 | Dana | `["web_d4", "web_d4"]` |
When you map `device_ids` as the identifier for your sync, Hightouch expands each row so that every array element becomes its own record:
| \_ht\_device\_ids\_element | user_id | name |
|---------------------------|---------|---------|
| `ios_a1` | 1 | Alice |
| `android_b2` | 1 | Alice |
| `ios_c3` | 2 | Bob |
| `web_d4` | 4 | Dana |
The generated column `_ht_device_ids_element` contains the individual element value and is used as the identifier sent to the destination.
### Empty arrays
Rows where the array column is empty (`[]`) or `NULL` produce **zero output rows**. In the example above, Charlie's row is omitted entirely because `device_ids` is an empty array.
### Duplicate elements
If an array contains duplicate values (like Dana's two `"web_d4"` entries), the duplicates produce a **primary key collision**. The sync completes with a "Report" status and the duplicate rows are flagged as rejected. Only one row per unique element value is synced.
## Configuration
Array expansion is configured automatically when you map an array-type column as the identifier in the **"How should rows be matched?"** section of your sync configuration.
When an array column is selected:
1. Hightouch sets the `arrayExpansionSourceColumn` in your sync config
2. The identifier mapping is automatically translated to use the generated element column (`_ht_{column}_element`)
3. An info banner confirms that array expansion is active
To disable array expansion, change the identifier mapping to a non-array column.
## Supported column types
Array expansion works with columns of the following types:
- **Native arrays**: Postgres `text[]`, Snowflake `ARRAY`, BigQuery `ARRAY`, Databricks `ARRAY`
- **JSON arrays**: Postgres `jsonb`, Snowflake `VARIANT`, BigQuery `JSON`
- **JSON string arrays**: `VARCHAR` columns containing JSON-encoded arrays like `'["a","b"]'`
Hightouch automatically detects the column type and applies the appropriate expansion strategy.
## Limitations
- Only **one array column** can be expanded per sync
- Array expansion requires the **lightning sync engine** (in-warehouse planner)
- Only supported on **Postgres**, **Snowflake**, **BigQuery**, and **Databricks** sources
- Duplicate values within an array cause row-level rejections
- Array expansion is not available on syncs that use [Match Booster](/syncs/match-booster), as Match Booster applies its own identity resolution logic that is incompatible with row expansion
---
## Sync array identifiers to audience destinations
**URL:** https://hightouch.com/docs/syncs/array-identifier-fan-out
**Description:** Automatically expand array-valued identifiers into individual records for audience syncs — maximize match rates across Google Ads, Meta, TikTok, and more.
**Section:** Syncs
## Overview
Audience syncs in Hightouch automatically **fan out** array-valued identifier fields into separate records — one per array element. This allows you to map a column containing multiple identifiers (such as an array of email addresses or mobile advertising IDs) and have each value synced individually to the destination.
This behavior is specific to **audience destination syncs** (for example, Google Ads Customer Match or Meta Custom Audiences). It requires no configuration — it happens automatically at sync time whenever a mapped identifier field contains an array. For enhanced identity resolution and match rates beyond what your source data provides, see [Match Booster](/syncs/match-booster).
This is distinct from [array expansion](/syncs/array-expansion), which is a separate feature that explodes array columns at the model query level for non-audience sync types. Audience identifier fan-out is automatic and applies only to identifier fields in audience syncs.
## How it works
If your model contains a row with an array-valued identifier field:
| user_id | mobile_ids |
|---------|------------|
| 123 | `["GAID_1", "GAID_2", "GAID_3"]` |
Hightouch sends **three separate records** to the destination:
| user_id | mobile_ids |
|---------|------------|
| 123 | `GAID_1` |
| 123 | `GAID_2` |
| 123 | `GAID_3` |
Each array element is synced individually so the destination can match on every identifier. Scalar (non-array) values are unaffected and synced as-is.
## Supported destinations
Fan-out works automatically with all audience destinations that support identifier-based matching, including:
- Google Ads (Customer Match)
- Meta (Custom Audiences)
- Snapchat (Customer List)
- TikTok (Audience)
- Google Display & Video 360
- Bing/Microsoft Ads (Customer Match)
- Reddit Ads
- The Trade Desk
- Pinterest Ads
- LinkedIn (Matched Audiences)
- Twitter/X
- Amazon Ads
- Criteo
- Yahoo Ads
- Taboola
This list is not exhaustive — any audience destination in Hightouch that accepts identifier fields supports fan-out.
## Requirements
- The source column must be a **true array type**, not a JSON string. For example, Snowflake `ARRAY`, BigQuery `ARRAY`, or Postgres `text[]` / `jsonb` arrays all work.
- A string like `'["id1","id2"]'` stored as `VARCHAR` will **not** be fanned out. If your data is stored this way, use a SQL transformation (e.g., `PARSE_JSON()` in Snowflake) to convert it to a native array before syncing.
Fan-out increases the number of API requests sent to the destination. If your arrays are large, this may affect sync duration.
## Linked identifier fields
When multiple identifier fields are mapped (for example, first name and last name), Hightouch preserves the association between values at the same array index. For independent fields like email and mobile ID, all combinations are sent to maximize match rates.
## Related features
- [**Match Booster**](/syncs/match-booster) — Enrich your audience data with additional identifiers to improve match rates across audience destinations.
---
## Change data capture (CDC)
**URL:** https://hightouch.com/docs/syncs/cdc
**Description:** Learn how Hightouch uses change data capture (CDC) to detect row-level changes in your models and send only incremental updates to destinations.
**Section:** Syncs
## Overview
Hightouch optimizes syncs by sending only the rows that have changed since the last run.
This process is called [**change data capture (CDC)**](https://en.wikipedia.org/wiki/Change_data_capture), or [**diffing**](https://en.wikipedia.org/wiki/Diff).
Without CDC, every sync would resend the entire dataset, causing unnecessary API calls, slower syncs, and wasted downstream processing.
---
## How CDC works
Whenever a new sync is triggered, Hightouch compares the previous sync run to the current set of query results. To do this, Hightouch keeps a record of the data sent in the last sync. This record is the **diff file**.
CDC only considers mapped model columns when creating and updating the diff
file. The only exception is custom destinations, such as the [HTTP Request](/destinations/http-request) destination, which consider all columns.
Steps in the CDC process:
1. Hightouch queries the source using the defined model.
2. Hightouch compares the current results to the results from the previous sync.
3. Hightouch identifies differences by primary key:
- New rows
- Updated rows
- Removed rows
4. Hightouch syncs only the changes to the destination, including any failed rows from the previous run.
5. A new diff file is created for the next comparison.
---
## Requirements
CDC relies on a **unique primary key** for every row in a model.
If primary keys are missing or duplicated, Hightouch can’t reliably track changes.
See [Primary key requirements →](https://hightouch.com/docs/models/creating-models#unique-primary-key-requirement).
---
## CDC by sync mode
CDC behavior depends on the sync mode:
- **Insert mode**: only syncs rows whose primary key wasn’t present in the previous run.
- **Update / Upsert modes**: syncs rows where values have changed.
- **All and Archive modes**: do not perform CDC; every row is sent every run.
See [Sync types and modes →](/syncs/types-and-modes).
---
## Difference-based CDC
Hightouch uses a [method](https://www.fivetran.com/blog/change-data-capture-what-it-is-and-how-to-use-it) called **difference-based CDC** because it compares the full before/after query results.
This is required when syncing from warehouses, since they cannot produce CDC logs for arbitrary SQL queries or dbt models.
In contrast, OLTP databases (like Postgres or MySQL) log incremental changes natively, which ETL tools often use when sending data into warehouses.
Because Hightouch does the reverse (sending data *from* warehouses), log-based CDC isn’t possible. This is a key distinction between ETL and reverse ETL.
---
## When CDC occurs
CDC happens **after Hightouch receives the query results from the source and before data is written to the destination.**
If you see the sync status **Querying** in the UI, Hightouch is in one of these states:
- Waiting for query results
- Saving the diff file
- Performing CDC computation
---
## Where CDC is computed
By default, CDC is performed on Hightouch-managed infrastructure.
For supported sources, you can enable the [**Lightning Sync Engine**](/syncs/lightning-sync-engine) to compute CDC directly in your warehouse.
- **Basic engine**: CDC runs on Hightouch infra.
- **Lightning engine**: CDC runs in your warehouse, enabling faster syncs at higher volumes.
---
## Where CDC data is stored
Hightouch stores previous query results (diff files) to compute changes.
- By default, diff files are stored in an encrypted Hightouch-managed bucket.
- Some plans allow you to [bring your own bucket →](https://hightouch.com/docs/security/storage#self-hosted-storage) to store diffs in your own infrastructure.
Diff files are retained for 30 days in Hightouch-managed storage.
If a sync doesn’t run for more than 30 days, you’ll need to [reset CDC](/syncs/overview#reset-cdc) or run a [full resync](/syncs/overview#resync-full-query).
---
## Resetting or resyncing CDC
Sometimes you may need to reset how CDC tracks changes. Common cases:
- **Primary key changes**: Hightouch prompts you to reset CDC when you change a model’s primary key.
- **Resync full query**: forces Hightouch to resend all rows, then resume CDC tracking.
- **Reset CDC**: ignores existing state and starts fresh on the next run (no data is sent during the reset run).
See [Resync and reset docs →](/syncs/overview#resync-full-query).
---
## FAQ
**What happens if I change my model configuration?**
- Only mapped columns affect CDC tracking, except for [HTTP Request destinations](/destinations/http-request), which consider all columns.
- If you change a column’s primary key, Hightouch requires a CDC reset.
- For other model changes, see [Model configuration docs](https://hightouch.com/docs/models/creating-models).
As explained in the [primary key
updates](/models/creating-models#updating-a-primary-key) section, if you alter
a model's primary key by selecting a different column, you will be prompted to
reset the change data capture for all syncs that depend on that model. Be
careful when modifying primary keys. If you keep the same column name but
alter the way its values are calculated, some records may be added or deleted
in your destination, depending on how your sync is configured.
Learn more about changes to your model configuration in the [model column
changes](/syncs/mapping-data#model-column-changes) section.
**Does Hightouch keep historical CDC files?**
No. Only the most recent diff file is stored and compared against.
**What happens if my sync fails?**
CDC re-attempts failed rows on the next run.
**What happens if I stop syncing for more than 30 days?**
Diff files in managed storage expire after 30 days. You’ll need to reset CDC or perform a full resync.
**What happens if I change sync mappings?**
Changing mappings may cause a full reprocess. See [Field mapping updates](/syncs/mapping-data#field-mapping-updates).
---
## Related articles
- [Data activation overview →](/getting-started/concepts)
- [Lightning Sync Engine →](/syncs/lightning-sync-engine)
- [Storage →](/security/storage)
---
## Create your first sync
**URL:** https://hightouch.com/docs/syncs/create-your-first-sync
**Description:** Learn the steps to creating a Hightouch sync and follow along to run your first one.
**Section:** Syncs
Running your first sync is satisfying—you can see your source data appear in the tools and apps your organization lives in. To get there, you need to set a few things up first.
It's best to follow this setup sequence when creating your first sync:
1. [**Connect a source**](#connect-a-source)
2. [**Set up a model**](#set-up-a-model)
3. [**Connect a destination**](#connect-a-destination)
4. [**Configure a sync**](#configure-a-sync)
You can learn the specifics of each step by clicking on the section link. Or check out the [core concepts](/getting-started/concepts) to review Hightouch's fundamentals.
## Connect a source
A [source](/getting-started/concepts#sources) is where your organization's business data lives. To add a new source to your Hightouch workspace, follow these steps:
1. Go to the [**Sources** overview page](https://app.hightouch.com/sources) and click the **Add source** button.
2. Select the source you want to add.
If you want to test things out without connecting a production source, you
can use the B2B SaaS or B2C eCommerce [**Sample datasets**](/sources/sample-data).
They're available at the end of the source list and require no credentials
or setup.
3. Enter the relevant credentials. These differ depending on the source. Refer to the relevant source documentation for further details.
4. Click **Continue**. Before continuing, Hightouch validates that you have the necessary access and permissions. If the test fails, you need to confirm and re-enter your credentials.
5. Give your source a **Source name**. This name is how your source appears in the Hightouch workspace. It helps to include details about the sources' environment, data contents, and owners, particularly if you plan on connecting multiple instances of the same source, for example, a development and prod version.
6. Click **Finish**.
Once you've finished, your source appears on the [**Sources** overview page](https://app.hightouch.com/sources) and you can use it set up models.
## Set up a model
Models define what data to pull from your source. To set up a new model, follow these steps:
1. Go to the [**Models** overview page](https://app.hightouch.com/models) and click **Add model**.
2. Select a source you've already connected.
3. Choose a [modeling method](/models/creating-models): SQL editor, table selector, dbt model, or Looker Look selector. If you're using one of the [Sample dataset](/sources/sample-data) sources, the SQL editor comes with pre-populated SQL queries. You can also use the table selector with the Sample datasets.
4. Before continuing, you must **Preview** your model to ensure it's querying the data you're interested in. By default, the preview displays the first 100 records. Once you've validated your data, click **Continue**.
5. Give your model a **Model name**. It's best to include details about the model's contents and business purpose so you can easily identify it. Including more information is beneficial if you plan on using your model for multiple syncs or having many team members in your workspace.
6. Select a **Primary key**. A primary key should be a column with unique identifiers. A column with unique values is essential for Hightouch to [sync data efficiently](/getting-started/concepts#change-data-capture). Click **Finish**.
Once you've finished, your model appears on the [**Models** overview page](https://app.hightouch.com/models) and you can use it to set up syncs.
To set up a sync, you need to connect a destination.
## Connect a destination
[Destinations](/getting-started/concepts#destinations) are the tools and services you want to receive data from your source. You need to connect your destination to Hightouch before you can set up the sync configuration to the destination.
To set up a new destination, follow these steps:
1. Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button.
2. Select the destination you want to add and click **Continue**.
If you want to test things out without connecting a production destination,
you can use the [Google Sheets](/destinations/google-sheets) destination.
Google Sheets is a [lite destination](/pricing/ss-pricing#lite-destinations)
and straightfoward to set up.
3. Authorize connecting to your destination or enter the relevant credentials and inputs. These differ depending on the destination. Refer to the relevant destination documentation for further details.
4. Click **Continue**. Before continuing, Hightouch validates that you have the necessary access and permissions. If the test fails, you need to confirm and re-enter your credentials.
5. Give your destination a **Destination name**. This name is how your destination appears in the Hightouch workspace. It helps to include details about the destination's business purposes and owners, particularly if you plan on connecting multiple instances of the destination, for example, a development and prod version.
6. Click **Finish**.
Once you've finished, your destination appears on the [**Destinations** overview page](https://app.hightouch.com/destinations) and you can use it configure syncs.
## Configure a sync
[Syncs](/getting-started/concepts#syncs) declare how your source data should appear in your destination. To configure one, you need to have set up a source, model, and destination first.
1. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button.
2. Select the relevant model connected to the source from which you want to pull data. If you need help determining which model to choose, you can select **View query** to see the model's definition.
3. Select the destination you want to send data to.
4. Configure how you want to sync data to your destination. Configuration inputs differ depending on the destination. Refer to the relevant destination documentation for further details. Click **Continue**.
5. Set a schedule for the sync. You can learn more about scheduling options in the [sync overview docs](/syncs/overview#schedule).
6. Click **Finish**.
Once you've finished configuring a sync, you land on that sync's overview page. This page shows previous sync runs and lets you edit its **Configuration**, [**Schedule**](/syncs/overview#schedule), and set up [**Alerts**](/syncs/alerting). It's also home to the [**live debugger**](https://hightouch.com/docs/syncs/debugger) which helps understand previous runs down to the API request and response level. To learn more about the sync overview page, check out the [sync documentation](/syncs/overview).
To run the sync manually, click **Run**. Check in your destination to see the fruits of your labor.
---
## Data Extraction
**URL:** https://hightouch.com/docs/syncs/data-extraction
**Description:** Write back the data from your ad platforms to your warehouse
**Section:** Syncs
## Overview
Hightouch's data extraction feature allows you to write back the data from your
ad platforms to your data system. With this data, you can conduct dynamic queries
and generate insightful reports for stakeholders.
## Setup
To get started, you will need:
1. A source with [lightning engine](/syncs/lightning-sync-engine) so Hightouch has write-back privileges
2. A destination to any of the [supported](#supported-destinations) ad platforms
If your destination supports this feature, you'll see a `Data Extraction` tab on the destination's page.
In this tab, you can enable this feature and select which data system to write to (one of your configured sources).
Currently, Hightouch only supports the `Audience size report` so by turning this feature on,
Hightouch will write this report back to your data system.
## Audience size report
The audience size report extracts a daily historical log of the estimated size of your audiences in each ad platform, including metadata about the connected Hightouch destination and sync (when applicable).
### How it works
For each enabled destination:
- Hightouch schedules a daily job to fetch and write the data from the ad platform to your data system
- This job pulls _all_ audiences from the active ad account(s) authorized by the user
- Each job is a snapshot of the audiences at that point in time, so no previous data is updated or deleted
- The data is then written back to your data system under the `hightouch_audit` schema.
- This data lives in its own designated table: `{destination-type}_{destination-id}_audiences`
Hightouch also creates a view that combines all the different destination tables into one, allowing you
to query the data holistically.
### Data structure
You can find all of the data from this feature in the `hightouch_audit` schema.
There will be one table per enabled Hightouch destination (`{destination-type}_{destination-id}_audiences`).
Each destination table's schema closely mirrors the ad platform's API's schema. Hightouch tries to
preserve the data in a form as close to the original API response as possible so you can adapt and
transform the data according to your specific needs.
Hightouch combines the table into a view, `external_audiences_metadata_v1`, with a standardized schema of:
- ad_account_id
- audience_id
- audience_name
- audience_size
- ht_fetched_at
- ht_destination
- ht_destination_id
- ht_sync_id (will be `null` if there are no HT syncs for this audience)
To ensure consistency and backwards compatibility, our combined view is versioned.
Any updates or changes to the view schema will result in a new version (e.g., v2, v3, etc.).
Previous versions of the view will remain accessible as long as it existed before a new updated version.
## Supported destinations
- [Amazon Ads DSP](/destinations/amazon-ads)
- [Meta Custom Audiences](/destinations/facebook)
- [Google Ads](/destinations/google)
- [Microsoft Bing Ads Audiences](/destinations/bingads)
- [Snapchat](/destinations/snapchat)
- [TikTok](/destinations/tiktok)
---
## Live debugger
**URL:** https://hightouch.com/docs/syncs/debugger
**Section:** Syncs
## Overview
Hightouch provides a live debugger to provide visibility into each of your sync executions or "runs." The debugger shows all requests and responses Hightouch made to your destination during a sync run. You can access the debugger from the **Runs** tab of any [sync's overview page](/syncs/overview#individual-sync-overview-page).
Hightouch stores row-level sync run data for seven days. If you need older logs, consider using [Warehouse Sync logs](/syncs/warehouse-sync-logs).
### View run details
You can see a table of recent sync runs and their statuses and [other details](/syncs/overview#run-details) from the **Runs** tab of any sync's overview page.
Each row in the table is one sync run.
The table shows:
- the run's [**status**](#sync-run-status): either **Querying** or **Processing** if the run is still processing, **Completed** if the run finished without errors, **Completed with errors** if the run included some rejected rows, **Aborted due to fatal error** if the run did not complete, or **Cancelled by user** if the run was cancelled
- the time the run **started** and its **duration**
- the number of **rows queried**—this is the number of rows in the [model](/getting-started/concepts#models)'s query results
- the number of **operations**, or updates, sent to the destination
- the **results**—the number resulting operations, split into successful and rejected operations
You can click a row to open a run's detail page and access the debugger.
The run detail page shows the number of successful and rejected rows broken down into added, changed, or removed rows.
- **Added** rows are new rows in the model's query results as compared to the previous run.
- **Changed** rows were in the previous run's query results but now have updated values.
- **Removed** rows are those that are no longer in the query results as compared to the previous run.
If the rows appear under the **Successful** tab, it means the run successfully made the changes in your destination.
If the rows appear under the **Rejected** tab, it means that though the model's query results may have changed, the run couldn't propogate the changes to your destination.
The numbers for added, changed, and removed rows reflect your sync configuration.
For example, if you configure a sync using [**insert** mode](/syncs/types-and-modes#sync-modes), Hightouch only shows **added** rows, even if some rows were removed from your model's query results.
If you configure a sync to the [HTTP request destination](/destinations/http-request) and only configure **Rows changed** [request triggers](/destinations/http-request#choose-request-triggers), you'll only see **changed** rows, even if some rows were added to or removed from your model's query results.
#### Sync run status
The preceding example sync run has a **healthy** status, meaning the sync ran as expected without any rejected rows.
The example sync run below has a **Aborted due to fatal error** status, meaning it didn't execute as intended.
This final example run has a **Completed with errors** status, meaning the destination rejected at least one row.
### View error messages
To view an error message for an entire run that failed, click **See error message** from the **Runs** overview tab.
Hightouch displays error messages in the format third-party APIs send them, including standard HTTP status codes and any trace IDs, if the third party provides them.
To see a specific row's error message, click **View error** on the row in the run's detail page table:
Hightouch displays error messages provided by the third-party API related to individual rows.
If the run fails during query execution with warehouse SQL errors, see [Resolve SQL compilation errors](/models/sql-compilation-errors).
### Filter runs
To narrow down the runs to inspect, you can use the **Filter** option to hide runs with either zero operations or zero successful operations.
### Search rows
Use the search input above the run details table to find a row with a primary key that matches the search value. This search only searches through the column you selected as the model's [primary key](/getting-started/concepts#unique-primary-key-requirement).
### Export rows and row errors
You can also export the data from all rows or just row errors as a CSV or JSON file.
Note: exporting requires workspace admin permissions.
You can then inspect the file to see details of rows the sync run added, changed, or deleted:
Or, download and then inspect just the row errors:
Use your CSV editor's search feature to locate rows or specific error messages more quickly.
## Use the live debugger
To take a closer look at the individual HTTP requests and responses that comprise a sync run, click any row to open the live debugger.
The live debugger shows the request and response JSON payloads of which the row was part and the other rows that were part of the batch.
In this example, the POST request to update this batch failed because of a 400 error. The response displays the third-party error message saying there was an `undefined` object in the request:
Hightouch displays the request's URL for diagnostic purposes. An `undefined` sub-URL means that the row ID number needed for this POST request failed to populate the URL. The next step would be to inspect the query results returned from the model.
### Payload inspection
To expand a payload for closer inspection, click the expand icon in the top right corner of the payload viewer.
Once expanded, you can scroll through the payload or click in the payload body and use Ctrl/Command-F to open a search window. Use the search window to find row IDs, email addresses, or any other payload data you need to inspect or trace.
To copy the payload, click the copy button in the top right corner of the payload viewer; the browser saves the payload to your clipboard.
For large, nested JSON payloads, it's helpful to paste the JSON into a viewer like [CodeBeautify](https://codebeautify.org/jsonviewer) to view the payload's structure and values. Click **Tree viewer** to inspect the JSON as collapsible properties:
## Hightouch debugging tips
### Filter runs
When you have a large number of runs to inspect, use the [run filter](#filter-runs) to narrow down your view to only the runs you want to inspect.
### Run timestamps
Use the run timestamps to isolate the runs you need to inspect based on your observation of when an error occurred. Run timestamps appear in your local timezone under the header **Started**.
### Search the live debugger
Use the live debugger to search payloads for missing rows or to see how Hightouch sent the data to your destination. Expand the payload viewer and use command+F to search for entities you need to inspect.
### Search error messages
Hightouch returns error messages directly from third-party APIs as they occur. Therefore, you will likely find more information and solutions by copying and pasting the error message into your favorite search engine or [Stack Overflow](https://stackoverflow.com/).
Some destinations also have active developer communities where you can find explanations for common error messages:
- [Salesforce Developer Community](https://developer.salesforce.com/forums#!/feedtype=RECENT&criteria=ALLQUESTIONS&)
- [HubSpot Developer Community](https://community.hubspot.com/t5/HubSpot-Developers/ct-p/developers)
- [NetSuite Support Community](https://community.oracle.com/netsuite)
### Understand HTTP status codes
Hightouch displays any HTTP status codes your destination includes in the HTTP response. These universal status codes can help you understand what may have gone wrong with your sync. If you see an HTTP status code in your Hightouch error message, it comes from your destination, not from Hightouch.
HTTP status codes are grouped into five categories:
- **1xx** Informational Response: The destination request was received and understood. Request processing continues.
- **2xx** Success: The request was successfully received, understood, and accepted.
- **3xx** Redirection: The client must take further action to complete the request.
- **4xx** Client Errors: The client caused an error. The request contains the wrong syntax or can't be fulfilled.
- **5xx** Server Errors: The server encountered an error and failed to fulfill the request.
See [Mozilla's documentation](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status) for further information on HTTP status codes.
#### Common status codes in Hightouch
The most common status codes Hightouch syncs receive are:
- 200 success: This is good news and means your destination successfully received and accepted the request from Hightouch.
- 400 status: A 400 error often means there is invalid data. Check that your data conforms to the destination's formatting expectations by referring to the destination's API documentation.
- 401 status: A 401 error often means there is a problem with the credentials you've used to set up your destination. Check that your credentials have the correct permissions to access the destination.
500 series (500, 502, 504, etc.) errors are "server errors," meaning something has gone wrong within the destination you are trying to sync to. Visit the status page of the destination to see if there are any reported outages, for example, [Slack System Status](https://status.slack.com/) or [HubSpot Status](https://status.hubspot.com/).
If a sync fails because of a destination server error, Hightouch [retries](/syncs/retries) the sync.
### Data types and casting
Third-party APIs require the data you send to be of a specific data type. Unless otherwise specified, Hightouch takes the data from your source and leaves the data type unchanged to send to your destination. See the [data types and casting docs](/models/data-types-casting) for tips on resolving data type incompatibilities.
---
## Lightning sync engine
**URL:** https://hightouch.com/docs/syncs/lightning-sync-engine
**Description:** Use the Lightning sync engine to speed up sync performance by storing change data capture in your data warehouse rather than on Hightouch managed infrastructure.
**Section:** Syncs
{/* */}
{/* Allow usage of "Warehouse planning" just this once */}
The Lightning sync engine was previously known as warehouse planning. Both terms describe the same feature.
{/* */}
## Overview
Hightouch identifies the incremental changes in your data model so that syncs send only the necessary updates to your destinations. For every sync, Hightouch computes what rows to send by pulling all rows within your data model and comparing them with the last sync. This process is called **change data capture** (CDC) or **diffing.** You can find more details in the [core concepts](/getting-started/concepts#change-data-capture).
By default, the change data capture computation and file storage happens on Hightouch managed infrastructure. This can be slow for large models that take a long time to query.
With the **Lightning sync engine**, Hightouch computes change data capture **directly in your warehouse**. It stores previously synced data into a special schema managed by Hightouch. Customers with extensive datasets requiring high performance syncs typically use the Lightning sync engine.
Hightouch recommends using the Lightning sync engine for [supported sources](#supported-sources) when syncing more than 100 thousand rows of data.
### Engine comparison
The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).
| Criteria | Basic sync engine | Lightning sync engine |
| ---------------------------------------------------- | ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Performance | Slower | Quicker (up to 100 times faster) |
| Ideal for large data models (over 100 thousand rows) | No | Yes |
| Reliability | Normal | High |
| Resilience to sync interruptions | Normal | High |
| Extra features | None | [Warehouse Sync Logs](/syncs/warehouse-sync-logs), [Match Booster](/match-booster/overview), [Identity Resolution](https://hightouch.com/platform/identity-resolution)|
| Ease of setup | Simpler | More involved |
| Location of change data capture | Hightouch infrastructure | Warehouse schemas managed by Hightouch |
| Required warehouse permissions | Read-only | Read and write |
| Ability to switch | You can move to the Lightning engine at any time | You can't move to the Basic engine once Lightning is configured |
## Supported sources
The Lightning sync engine works with the following sources:
- [Amazon Athena](/sources/amazon-athena)
- [Amazon Redshift](/sources/amazon-redshift)
- [Azure Synapse](/sources/synapse)
- [Databricks](/sources/databricks)
- [Google BigQuery](/sources/google-bigquery)
- [Greenplum Database](/sources/greenplum)
- [Lakebase](/sources/lakebase)
- [Microsoft Fabric](/sources/fabric)
- [PostgreSQL](/sources/postgresql)
- [Snowflake](/sources/snowflake)
- [Snowflake Iceberg](/sources/snowflake-iceberg)
- [Trino](/sources/trino)
## Enable the Lightning sync engine
When first setting up one of the [supported sources](#supported-sources), Hightouch prompts you to **Choose your sync engine**.
You can set up the Lightning sync engine at this point or continue with the Basic sync engine and [upgrade to the Lightning engine](#migrate-from-basic-sync-engine) later. Once you've configured the Lightning engine, you can't switch back to the Basic engine.
In addition to enabling the Lightning sync engine in the Hightouch UI, you need to:
1. Grant Hightouch write permission to your data warehouse.
2. Create the warehouse schemas for Hightouch to use.
Refer to the source-specific in-app instructions on how to complete these steps.
### Warehouse schemas
Hightouch requires you to create two schemas:
- `hightouch_planner`: Hightouch uses this schema to power the Lightning sync engine.
- `hightouch_audit`: Hightouch uses this schema to power [Warehouse Sync Logs](/syncs/warehouse-sync-logs).
Whenever a sync that uses the Lightning sync engine runs, Hightouch creates two new tables in the `hightouch_planner` schema: one to log the model's query results (the `_plan` table) and one to log rows rejected during the sync (the `_rejections` table).
Hightouch uses these tables to perform [change data capture](#overview) and plan which rows to sync.
Hightouch only keeps the two most recent pairs of tables to compute the diff for the current sync.
In other words, Hightouch [doesn't maintain historical records](/getting-started/concepts#does-hightouch-keep-historical-records-of-change-data-capture) of these tables, and any given sync never has more than the two most recent pairs of tables.
Because these tables and their names change with every sync run, Hightouch requires permission to write to the entire `hightouch_planner` schema rather than specific tables.
You can learn more about the `hightouch_audit` schema on the [Warehouse Sync Logs](/syncs/warehouse-sync-logs#schema) page.
#### Table removal
Removing tables from the `hightouch_planner` and `hightouch_audit` schemas isn't recommended:
- `hightouch_planner`: these tables are required for [change data capture](#overview) to work correctly; if you delete these tables, you need to trigger a [full resync](/syncs/overview#resync-full-query) to fix your syncs.
- `hightouch_audit`: removing tables from the `hightouch_audit` schema would mean deleting the [Warehouse Sync Logs](/syncs/warehouse-sync-logs) related to a sync's prior runs, which are useful for historical analysis.
If your [source](#supported-sources) is located in the European Union, you might want to avoid having PII older than 30 days in your data warehouse, in accordance with the [GDPR](https://en.wikipedia.org/wiki/General_Data_Protection_Regulation).
In this case, we recommend running **all** syncs in your workspace at least once every 15 days.
This ensures that Hightouch refreshes the tables in the `hightouch_planner` schema and the **Snapshot** table in the [`hightouch_audit` schema](/syncs/warehouse-sync-logs#schema), removing all older PII before it reaches the 30-day threshold.
Historical data is never automatically cleared out from the **Changelog** table in the `hightouch_audit` schema.
If required, you need to manually remove older PII from this table.
The **Sync runs** table in the `hightouch_audit` schema doesn't contain PII.
## Migrate from Basic sync engine
You can migrate to the Lightning sync engine by going to the **Configuration** tab of any previously setup [supported source](#supported-sources).
1. From the source's **Configuration** tab, select **Lightning sync engine** under **Choose your sync engine** .
2. Follow the source-specific in-app instructions to grant Hightouch write permissions to your warehouse and create the schemas for Hightouch to use.
3. Click **Save changes**.
4. Test your connection to ensure setup is correct.
If you enable the Lightning sync engine on an existing source, Hightouch migrates any syncs on the Basic engine to the Lightning sync engine on the next run. Any previously synced data won't be unnecessarily resynced since the existing [change data capture](#overview) data is copied over to your warehouse schemas. In most cases you don't need to trigger [full resyncs](/syncs/overview#resync-full-query) or [reset change data capture (CDC)](/syncs/overview#reset-cdc), unless you [encounter errors](#migration-tips).
The first run after migrating to the Lightning sync engine can take longer than usual, as it uses a special migration sync engine to create the necessary [schemas](#warehouse-schemas) in your warehouse.
### Migration tips
Follow these tips when performing the migration:
- The Lightning sync engine requires your [primary key to be unique](#primary-key-is-not-unique) for every model using the source. If you're unsure if your primary key is unique, check before migrating.
- If you have a large number of syncs to migrate, please
so that we can help you migrate incrementally.
- If you hit any errors, you can often fix them with a [full resync](/syncs/overview#resync-full-query). Ensure your sync meets the [full resync prerequisites](/syncs/overview#full-resync-prerequisites) before doing so. Otherwise, you can [reset change data capture (CDC)](/syncs/overview#reset-cdc).
## Common errors
### Primary key is not unique
The Lightning sync engine requires that every row in your model has a unique value for its [primary key](/getting-started/concepts#unique-primary-key-requirement). This is because the primary key is used to uniquely identify each row, and for example, whether it failed previously.
Models with duplicate primary keys can be deduplicated in SQL with the `ROW_NUMBER()` function.
For example:
```sql
WITH your_data_model AS (
// copy/paste your model here
)
WITH your_data_model_with_rank AS (
SELECT *, ROW_NUMBER() OVER (PARTITION BY your_primary_key) as rank FROM your_data_model
)
SELECT * FROM your_data_model_with_rank WHERE rank=1
```
The preceding example de-duplicates your model by _partitioning_ on the primary key. It chooses an arbitrary record from your duplicates. You can use `ORDER BY` or `WHERE` to filter and select which data from the duplicates you want to keep.
---
## Field mapping
**URL:** https://hightouch.com/docs/syncs/mapping-data
**Description:** Mapping is the establishment of a one-to-one relationship between a column in your source and a property or field in a destination.
**Section:** Syncs
## Overview
One of the key aspects of sync configuration is **field mapping**. Field mapping defines which columns from your model query results should appear in your destination and how. In other words, it's a way of relating model columns to destination fields or properties.
**"Columns" vs. "fields" vs. "properties"**: Though some may use these terms
interchangably,
Hightouch uses the term "columns" to refer to a set of vertical data values in
a source or model and "fields" or "properties" to refer to them in a destination.
Depending on your destination, you can map your data with either the **basic mapper** or the **advanced mapper**. The basic mapper maps columns to fields in a one-to-one way. The advanced mapper gives the additional options of:
- mapping with **static** or constant values
- mapping with **variables**
- mapping with [Liquid](https://shopify.dev/api/liquid) **templates**
- **inline mapping**, which lets you create nested objects and arrays from model columns
### Usage
You can start with either the source or destination input fields when using either basic or advanced mapper.
The Hightouch UI offers this flexibility to enable you to map data the way that makes the most sense for you.
For example, you may approach mapping with an idea of all model columns you want to send to your destination.
To do so, click **Add mapping** and select each model column in the left-hand inputs.
Then select the appropriate destination fields to map them to in the corresponding right-hand dropdowns.
Conversely, you may know all the destination fields you need to sync to.
Select each destination field from the dropdowns on the right and map the appropriate model columns on the left.
You can also select to **Suggest mappings**.
This option proposes relevant destination fields for model columns.
For example, a model column named **email_address** would receive a suggested destination field like **Email**.
This autocompletion works by searching for matching and near-matching column and field names.
Before saving your configuration, you should confirm that the autocompleted suggestions have made the correct selections.
## Basic mapper
With the basic mapper, you tell Hightouch which columns in your source to feed into which fields in your destination. For example, you can use the basic mapper to relate model columns to destination fields like these:
| Column in your source | Field in your destination |
| --------------------- | ------------------------- |
| `email_address` | `Email` |
| `first_name` | `FirstName` |
| `last_name` | `LastName` |
| `account_id` | `AccountId` |
## Advanced mapper
For destinations with the **advanced mapper**, you can create mappings that include:
- a column value
- a static value
- a sync-based variable value
- values using the Liquid templating language
- nested objects and arrays using the [**inline mapper**](#inline-mapping)
You can open the advanced mapper by clicking the source field you want to map. If you click a source field and the advanced mapper doesn't appear, it isn't supported. Please if you have a use case requiring the advanced mapper on a destination that doesn't support it.
### Column values
Mapping column values is similar to the [basic mapper's](#basic-mapper) functionality. You can search for and select the model column name you want to map to a destination field.
#### Don't sync null values
By default, Hightouch syncs any `null` values found in your model query results to your destination. You can select to sync nothing instead of null values by enabling **Don't sync null values**.
### Static values
With static mapping, you can specify a constant value to use for each field value. For example, suppose you want to map the string value `EMEA` for every record's `region` field value in a destination. You can do this with static mapping:
1. Open the advanced mapper by clicking on a model column.
1. Select **Static value** and the value's data type: **String**, **Number**, **Boolean**, or **Null**.
1. Enter the desired value in the field provided.
1. Click **Apply**.
1. In the field input to the right of the static value, select the destination field name to which you want to map the static value.
### Variable values
With variable mapping, you can send sync metadata to a destination field. The available metadata variables are:
- `model-id` (string)
- `model-name` (string)
- `current-timestamp` (date/time)
- `sync-id` (string)
- `sync-run-id` (string)
- `operation` (string)
For example, you may want to send a sync's timestamp to an `updated_at` field in a destination.
1. Open the advanced mapper by clicking on a model column.
1. Select **Variable value** and the desired metadata, for example, **current-timestamp**.
1. Click **Apply**.
1. In the field input to the right of the variable value, select the destination field name to which you want to map the variable value.
With this example mapping, Hightouch sends ISO time/date value (YYYY-MM-DD) into every record's `updated_at` field:
### Template mapping
The template mapper is convenient when your destination requires data in a specific format, and you want to avoid baking this specificity into your model.
For example:
- [Webflow](/destinations/webflow) requires slugs not to have any special characters. You can use the template mapper's `regex_replace` function to remove them.
- You want to validate email addresses before sending them to Mailchimp or another email service provider.
- You want to enable lead routing in your destination using Liquid conditionals without touching your central dbt model.
To create nested objects or arrays, you need to use the [inline
mapper](#inline-mapping).
The template mapper uses the [Liquid templating language](https://shopify.dev/api/liquid) to create complex mappings.
1. Open the advanced mapper by clicking on a model column.
1. Select **Template**, then select a **Variable** and **Function**. The function's code automatically appears in the editor window.
1. Click **Apply**.
1. In the field input to the right of the templated value, select the destination field name to which you want to map the templated value.
In the example screenshot, the template uses the function `{{ row['last_name'] | includes: "Stark" }}` to check checks if a row's `last_name` column includes the string `"Stark"`. If it does, the row is processed. If it doesn't, the row is rejected.
Beyond using row values, accessible by `{{ row['column_name'] }}`, you can also access any of the [variable values](#variable-values) present in the advanced mapper.
These variable values are available on the `context` object.
For example, you could you use `{{ row['id'] | concat: context['sync_run_id'] }}` to concatenate a row's ID with the current sync run's ID.
The template mapper supports all Liquid native features and also includes [Liquid-inspired](#liquid-inspired-functions-in-the-advanced-mapper) functions.
To learn more about the Liquid templating language, skip to the [Liquid reference](#liquid-reference) section in this doc or check out the [Liquid reference documentation](https://shopify.dev/api/liquid).
### Inline mapping
The inline mapper is only available for certain destinations and is currently in beta. Please {" "}if you'd like to be considered for early access.
When syncing data to a marketing automation or enterprise resource planning (ERP) tool, you may need to nest related entities. For example, you may have nested items in an invoice or nested entities for a user: associated devices, shipping address, pets, etc. The inline mappers allows you to create these nested objects and arrays from your model query results _without altering your model definition_.
Each destination has its own formatting expectations for these nested entities. For example, in Stripe, an `invoice` object supports an `items` field that expects an [array of items](/destinations/netsuite-rest#arrays).
The inline mapper lets you model your data _once_ and format it during sync configuration rather creating a separate model for each destination requiring nested objects or arrays.
#### Create an object
The object inline mapper allows you to construct objects and map them to destination fields usings columns from your model. This section walks through constructing an object for the [shipping](https://stripe.com/docs/api/customers/create#create_customer-shipping-address) field in Stripe's customer object.
1. In the destination field input on the right, select the field name to which you want to map an object, for example, **Shipping Address**.
2. The model column input on the left autopopulates to **Object** mode if the selected field is of type `object`. If the type of the field is not provided, open the advanced mapper by clicking on the source input field and select **Create an object**.
3. Select the relevant destination fields you want to map values to and map the nested fields as needed.
Given the previous example [row](#create-an-object) and the mappings shown in the screenshot above, Hightouch would sync the following payload:
```json
{
"email": "doe@test.com",
"shipping": {
"address": {
"city": "San Francisco",
"state": "California",
"postal_code": "94105"
},
"name": "John Doe",
"phone": "123-456-7890"
}
}
```
#### Create an array
The array inline mapper allows you to construct an array of objects and map them to destination fields using a model column that contains an array of objects. This section walks through constructing a [line items](https://stripe.com/docs/api/quotes/object#quote_object-line_items) array for Stripe's quote object. You would use the same steps for any other object arrays you want to create.
#### Required data format
To use the inline mapper to create an array of objects, the query results from your [model](/models/creating-models) need to have a certain format. Specifically, they should appear as an array of objects, like the example **items** field in the entity relationship diagram (ERD) above.
```json
[
{
"item_id": "542",
"quantity": 50
},
{
"item_id": "631",
"quantity": 20
}
]
```
If you don't already have a data field formatted like this, you can define your model using a SQL query that constructs an array of objects from other fields.
For example, imagine you have a `line_items` data table where each row represents an invoice item. The columns are:
- `invoice_id` for the ID of the invoice the item belongs to
- `customer_id` for the ID of the customer who purchased the item
- `amount` for the cost of item
- `description` for a short description of the item
For example:
| invoice_id | customer_id | amount | description |
| ---------- | ----------- | ------ | ---------------------- |
| in_1 | cust_abcd | 55 | Waffle Maker |
| in_1 | cust_abcd | 7 | Dog Chew Toy |
| in_2 | cust_efgh | 12 | HDMI Cable |
| in_3 | cust_abcd | 45 | Humidifier Filters |
| in_3 | cust_abcd | 109 | Electric Kettle |
| in_3 | cust_abcd | 15 | Wall Calendar |
| in_4 | cust_ijkl | 25 | Mediterranean Cookbook |
| in_4 | cust_ijkl | 10 | AAA Batteries |
You want to sync an array of items for each invoice for each customer. For example, for `customer_abcd` from the preceding table, they would have two separate invoice objects, each with their own `items` array, one for the invoice with ID `in_1` and one for the `in_3` invoice:
```json
[
{
"invoice_id": "in_1"
"customer_id": "cust_abcd",
"items": [
{
"description": "Waffle Maker",
"amount": 55,
},
{
"description": "Dog Chew Toy",
"amount": 7,
}
]
},
{
"invoice_id": "in_3"
"customer_id": "cust_abcd",
"items": [
{
"description": "Humidifier Filters",
"amount": 45,
},
{
"description": "Electric Kettle",
"amount": 109,
},
{
"description": "Wall Calendar",
"amount": 15,
}
]
}
]
```
To make these fields available in the inline mapper to construct an `items` array, you would need to include SQL like this in your model definition:
```SQL
select
inv.customer_id,
inv.invoice_id,
array_agg(object_construct(it.*)) as items
from
line_items inv
join items it on it.invoice_id = invoice._id
group by
1,
2
```
This query selects the `customer_id` and `invoice_id` as well as all properties—in this case just description and amount—on each item and then aggregates them into an array of objects. The `group by` statement groups them based on the `customer_id` and `invoice_id` so that each aggregated array is for one invoice for one customer. Without the inline mapper, you would need to write more complex SQL manually constructing each item attribute as part of the item object.
#### Map fields
Once your model query results have an array of objects to pull data from, you can use the inline mapper to configure as many syncs as you need to sync these items.
1. In the **Field Mapping** section of your sync configuration, in the destination field input on the right, select the field name to which you want to map an array, for example, **Line items**.
2. The model column input on the left autopopulates to **Array** mode if the selected field is of type `array`. If the type of the field is not provided, open the advanced mapper by clicking on the source input field and select **Create an array**.
3. You will be prompted to pick a column that contains an array of objects. Select your desired column from the list of compatible columns. If no columns appear, ensure your model query results conform to the [required data format](#required-data-format).
4. Select the relevant destination fields you want to map values to and map the nested fields as needed.
The columns available under the array inline mapper are properties of the previously selected column.
Hightouch uses the first 100 rows from your query to extract the available properties at each index of the array. To add additional properties that may be missing, type into the input field and select the custom option or press **Enter**.
Hightouch only processes properties if the mapped object property at the index of the array exists.
For example, if `[0].price` exists and `[1].price` doesn't exist, `items[0].price` will be mapped and `items[1].price` won't be mapped. Hightouch syncs `null` values if you [don't choose to ignore them](#dont-sync-null-values).
For each row in your query, the array created using the inline mapper matches the length of the array in your column.
Given the previous example [row](#create-an-array) and the mappings shown in the screenshot above, Hightouch would sync the following payload:
```json
{
"line_items": [
{
"price_data": {
"currency": "USD",
"product": "542"
},
"quantity": 50
},
{
"price_data": {
"currency": "USD",
"product": "631"
},
"quantity": 20
}
],
"customer": 231,
"description": "Two items"
}
```
#### Nesting depth
You may need to sync arrays or objects with more deeply nested objects. For example, you may need to sync a `Shipping Address` object with this format:
```json
{
"name": "Jane Doe",
"address": {
"address_line_1": "123 High Street",
"address_line_2": "",
"city": "San Francisco",
"state": "California",
"postal_code": "94105",
"country": "USA"
},
"phone_number": "4151234567"
}
```
The "address" key expects a further nested object. Hightouch supports nesting up to two levels deep. You can do this by selecting **Create an object** or **Create an array** while already using the inline mapper.
### Liquid reference
This reference goes over the following:
- Basic [liquid formatting](#liquid-template-format) rules in the advanced mapper
- Available [liquid variables](#liquid-variables-in-the-advanced-mapper) and [functions](#liquid-inspired-functions-in-the-advanced-mapper)
- [Extended use cases](#extended-liquid-use-cases)
For more detailed information, refer to the [Liquid reference documentation](https://shopify.dev/api/liquid).
#### Liquid template format
Variables follow this format:
```
{{ row['company'] }}
```
The double handlebars enclose a Liquid snippet that generates an output. The column name, `company` in the preceding example, should be in single quotes inside the square brackets.
Hightouch also provides these alternative syntaxes:
```
{{ row.company }}
```
```
{{ company }}
```
These shorter alternatives can be useful when templating messages, such as when syncing to [Slack](/destinations/slack#templating-messages), [Mattermost](/destinations/mattermost#templating-messages), [Microsoft Teams](/destinations/microsoft-teams#templating-messages), or to the [SMTP Email](/destinations/email#templating-emails) destination.
Make sure to use the `{{ row['first name'] }}` syntax if your model column name contains spaces.
Functions follow this format:
```
{{ row['last_name'] | includes : 'Stark' }}
```
A single pipe (`|`) comes after the variable and before the name of the function and any necessary parameters. See the [Liquid Reference](https://shopify.dev/api/liquid) or [Liquid Cheatsheet](https://www.shopify.com/partners/shopify-cheat-sheet?shpxid=7bd156ec-6E04-4D53-42BC-9C614D0E5E0E) for a complete list of available functions.
#### Liquid variables in the advanced mapper
You can use metadata variables in your liquid templates, such as a model's name, all model column names, and the timestamp of a sync's last run.
```
$
$ Model Name
$ Timestamp
```
You can view all available variables in the **Variables** column in the template editor.
#### Liquid-inspired functions in the advanced mapper
Hightouch provides these non-native functions for you to use out of the box:
| Function | Syntax | Description |
| ---------------- | ------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `base64_decode` | `row['email']\|b64_decode` | Base64 decode input |
| `base64_encode` | `row['email']\|b64_encode` | Base64 encode input |
| `cast` | `row['phone']\|cast:'string'` | Cast input to specified type. Allowable types are 'string', 'number', 'boolean' |
| `includes` | `row['last_name']\|includes:'Stark'` | Check input for a substring. Row will be rejected if substring isn't found |
| `json_construct` | `\| json_construct: 'city', row['city'], 'state', row['state_province']` | Construct JSON object from key/value arguments |
| `MD5` | `row['email']\| md5` | Hash input using MD5 algorithm |
| `null_if_empty` | `row['email']\| null_if_empty` | Replace empty strings with `null` |
| `parse` | `row['first_name']\|parse` | Parse a JSON-formatted string to a JSON object |
| `push` | `row['first_name']\|push : ''` | Append an item to an array |
| `regex_replace` | `row['last_name']\|regex_replace: '<[a-zA-Z]>', ''` | Search and replace substrings of input using [RegEx](https://regexr.com/) |
| `regex_test` | `row['last_name']\|regex_test: '<[a-zA-Z]>'` | Check input for a [RegEx](https://regexr.com/) match. Row will be rejected if no match is found using [RegEx](https://regexr.com/) |
| `sha256` | `row['email']\|sha256` | Hash input using SHA-256 |
| `sha512` | `row['email']\|sha512` | Hash input using SHA-512 |
| `to_date` | `row['dob']\|to_date` | Parse input and return a [Date](https://shopify.dev/api/liquid/objects/date) object ([ISO 8601](https://www.iso.org/iso-8601-date-and-time-format.html) format) |
| `to_unix` | `row['date_sent']\|to_unix` | Convert input to [Unix](https://www.unixtimestamp.com/index.php) time |
| `validate_email` | `row['email']\|validate_email` | Reject emails that aren't [RFC 2822](http://haacked.com/archive/2007/08/21/i-knew-how-to-validate-an-email-address-until-i.aspx/) compliant email addresses |
#### Liquid function chaining
You can use more than one Liquid function in a template by [chaining](https://en.wikipedia.org/wiki/Method_chaining) them. For example, you may want to put a string variable into camel case before hashing it. To chain functions, separate them with a single pipe (`|`):
Examples:
```
{{ row['full_name'] | camelcase | sha256 }}
```
```
{{ row['email'] | downcase | validate_email | sha256 }}
```
### Extended Liquid use cases
See Shopify's [Liquid docs](https://shopify.github.io/liquid/basics/introduction/), [Liquid Cheatsheet](https://www.shopify.com/partners/shopify-cheat-sheet), or [Liquid Reference](https://shopify.dev/api/liquid) for a complete list of available functions.
In this section, you can learn how to use Liquid to:
- set up [conditional outputs](#conditional-output)
- output a `null` value for [empty values](#conditional-output-for-empty-values)
- set a [default value](#default-value)
- [escape special characters](#json-function) in JSON mappings
You can also use Liquid to work with different [data types](/models/data-types-casting#data-types):
- **Numbers**: use [math functions](#math-functions) (such as `abs` and `round`) or format [currency amounts](#currency-amounts)
- **Booleans**: output a [conditional boolean value](#conditional-boolean-output)
- **Strings**: use [string functions](#string-functions) (such as `capitalize`, `strip`, and `append`) or [isolate parts of a string](#isolate-parts-of-a-string)
- **Dates**: use the [date function](#date-function)
- **Arrays**: [construct a static array](#construct-a-static-array), [sync an empty array](#sync-an-empty-array), or [convert an array to a string](#convert-an-array-to-a-string)
- **Objects**: [convert an object to a string](#json-function) or [isolate object properties](#isolate-object-properties)
#### Conditional output
You can use Liquid conditionals to specify output based on input. For example, this Liquid snippet:
```
{%- if row['discount_percent'] >= 50 -%}
Over half off!
{%- elsif row['discount_percent'] == 50 -%}
50% off!
{%- elsif row['discount_percent'] > 0 -%}
Discounted!
{%- else -%}
null
{%- endif -%}
```
Outputs the following result in a destination column:
#### Conditional output for empty values
Conditionals are also helpful to guard against missing values.
For example, you could use the following Liquid snippet to send a `null` value for empty values.
This is helpful if you need to map a column that exists for some rows but not others:
```
{%- if row['column_name'] !=nil -%}
{{ row['column_name'] }}
{%- else -%}
{{ null }}
{%- endif -%}
```
#### Conditional boolean output
You could also use the [`assign`](https://shopify.dev/docs/api/liquid/tags#assign) function to output a conditional boolean value:
```
{%- if row['column_name'] == "value" -%}
{%- assign variable = true -%}
{%- else -%}
{%- assign variable = false -%}
{%- endif -%}
{{ variable }}
```
#### Default value
You can also use Liquid to set a [default](https://shopify.github.io/liquid/filters/default/) value, in case an input value is `null`, `false`, or empty. For example, this Liquid snippet sets the `revenue` field as `0` if the value is missing or `null` in a model's row:
```
{{ row['revenue'] | default: 0 }}
```
Be sure to change `revenue` to the column name you want to set default values for.
#### Math functions
See [Liquid Math Functions](https://shopify.dev/api/liquid/filters/math-filters) for a complete list of math functions.
In Hightouch, the absolute value and rounding functions are some of the most commonly used:
- Use the `abs` function to send an absolute value:
```
{{ row['value'] | abs }}
```
- Use the `round` function to round a value to the nearest integer or a specified number of decimals:
```
{{ row['value'] | round: 2 }}
```
#### Currency amounts
Hightouch doesn't support [`money`](https://shopify.dev/docs/api/liquid/filters/money-filters) functions.
If you would like to format currency amounts, such as prices, you can set this up manually by using the following template:
```
{%- assign decimals = row['price'] | split: "" | reverse | slice: 0,2 | reverse | join: "" -%}
{%- assign units = row['price'] | split: "" | reverse | slice: 3,3 | reverse | join: "" -%}
{%- assign thousands = row['price'] | split: "" | reverse | slice: 6,3 | reverse | join: "" -%}
{%- assign millions = row['price'] | split: "" | reverse | slice: 9,3 | reverse | join: "" -%}
{%- if millions != "" -%}
${{millions}},{{thousands}},{{units}}.{{decimals}}
{%- elsif thousands != "" -%}
${{thousands}},{{units}}.{{decimals}}
{%- else -%}
${{units}}.{{decimals}}
{%- endif -%}
```
To explain this expression:
- [`split`](https://shopify.github.io/liquid/filters/split/): converts the value in the `row['price']` model column (either `number` or `string`) to an array
- [`reverse`](https://shopify.github.io/liquid/filters/reverse/): reverses the order of items in the array
- [`slice`](https://shopify.github.io/liquid/filters/slice/): isolates the items in groups of threes
- [`reverse`](https://shopify.github.io/liquid/filters/reverse/): reverses the order of items in the groups
- [`join`](https://shopify.github.io/liquid/filters/join/): transforms the single groups of items back into strings
The second part is a [conditional](#conditional-output) that displays the currency amount in the correct format.
If your model column contains currency amounts without decimal places, make sure to remove the `decimals` variable from the Liquid expression and update the `slice` functions accordingly.
#### String functions
See [Liquid String Functions](https://shopify.dev/api/liquid/filters/string-filters) for a complete list of string functions.
In Hightouch, the `capitalize`, `strip`, and `append` functions are some of the most commonly used:
- Use the `capitalize` function to capitalize the first letter of a string:
```
{{ row['title'] | capitalize }}
```
- Use the `strip` function to remove all whitespace from the left and right sides of a string:
```
{{ row['title'] | strip }}
```
- Use the `append` function to add a file extension to a string:
```
{{ row['file_name'] | append: '.png' }}
```
{/* */}
#### Isolate parts of a string
You can use the Liquid [`split` function](https://shopify.github.io/liquid/filters/split/) to isolate a certain part of a string.
For example, suppose you have a model column that returns URLs, including query parameters.
An example value may look like this: `www.example.com/?gclid-123`.
You may want to send the base URL (`www.example.com/`) as one field and one of the query parameters as another field.
Query parameters often include click IDs, such as the `gclid` or `fbclid` that you should include on events you send to [Google Ads](/destinations/google#syncing-conversion-events) or [Meta Conversions](/destinations/meta-conversions#syncing-events) (or [Meta Offline Conversions](/destinations/meta-offline#syncing-events)) respectively.
For the example value `www.example.com/?gclid-123`, this Liquid snippet would return the base URL, `www.example.com/`:
```
{{ row['column_name'] | split: "?gclid-" | first }}
```
And this Liquid snippet would return the `gclid` value, `123`:
```
{{ row['column_name'] | split: "?gclid-" | last }}
```
{/* */}
#### Date function
If your destination expects dates in a different format than your model returns them,
you can use the Liquid [`date` function](https://shopify.github.io/liquid/filters/date/) to reformat them.
For example, this Liquid snippet ensures that dates from the column are always in the `DD/MM/YYYY` format:
```
{{ row['column_name'] | date: '%d/%m/%Y' }}
```
And this snippet always returns dates in the `MM/DD/YYYY` format:
```
{{ row['column_name'] | date: '%m/%d/%Y' }}
```
You can also include other parameters to include time and timezone data, by adding `%H:%M:%S` and `%z`:
```
{{ row['column_name'] | date: '%s' | date: '%Y-%m-%dT%H:%M:%S%z' }}
```
You can use Liquid to change a timestamp's time and date values. For example, you may want to send a date one day earlier than the original. This snippet subtracts 24 hours (represented as 86,400 seconds) from the original date in the column:
```
{{ row['column_name'] | date: '%s' | minus: 86400 | date: '%Y-%m-%dT%H:%M:%S' }}
```
To add time to a date, replace `minus` with `plus`.
For more examples of how to transform dates and times, refer to this [documentation](https://support.sweethawk.com/hc/en-us/articles/4412143470221-Liquid-markup-for-date-and-time#index_3).
#### Construct a static array
You can sync a static array value to your destination by using this Liquid snippet:
```
{%- assign value = '[' | append: '"value1","value2"' | append: ']' -%}
{{ value | parse }}
```
You can replace `'"value1","value2"'` with the values you wish to insert in your static array.
#### Sync an empty array
You can sync an empty array to your destination by using this Liquid snippet:
```
{{ '[]' | parse }}
```
If you decide to use this with [Liquid conditionals](#conditional-output) or other complex syntax, make sure to [use a hyphen](https://shopify.github.io/liquid/basics/whitespace/) in your tag syntax to strip whitespace.
#### Convert an array to a string
You can convert an array to a string by using the [`join`](https://shopify.github.io/liquid/filters/join/) function:
```
{{ row['items'] | join: "" }}
```
#### JSON function
You can convert an object to a JSON-formatted string by using this Liquid snippet:
```
{{ row['invoice'] | json }}
```
You can also use this function to escape special characters when creating a JSON payload.
For example, if you attempt to map this value, `This is the "value" in the column` in the `special_characters` column, it will fail:
```
"column_mapping" : "{{ row.special_characters }}"
```
To successfully sync this value, select the `json` function and use the example format below:
```
"column_mapping" : {{ row.special_characters | json }}
```
#### Isolate object properties
If your model columns contain `object` type values, you may want to sync only one or some of the object's properties to your destination.
You can do this in the template mapper using the [dot notation](https://en.wikipedia.org/wiki/Property_(programming)#Dot_notation).
Specifically, you can write a Liquid expression containing the model column's name and the property's name: `{{ row['model_column'].property }}`
You can use dot notation for [field mapping](#usage) and [record matching](/syncs/record-matching), but only for destinations that support Liquid templating.
Furthermore, dot notation doesn't work when [testing a row](/syncs/overview#test-a-row).
To test your sync, trigger a sync run and take a look at the [live debugger](/syncs/debugger#use-the-live-debugger).
For example, suppose you had a model column named `invoice` that contains objects with this structure:
```json
{
"invoice_number": "INV-001",
"issue_date": "2023-06-08",
"due_date": "2023-06-30",
"customer": {
"name": "John Doe",
"email": "johndoe@example.com",
"address": "123 Main St, City, State, ZIP"
},
"items": [
{
"description": "Product A",
"quantity": 2,
"unit_price": 10.99,
"total_price": 21.98
},
{
"description": "Product B",
"quantity": 1,
"unit_price": 15.99,
"total_price": 15.99
}
],
"subtotal": 37.97,
"tax": 3.60,
"total": 41.57,
"currency": "GBP"
}
```
You may only want to map the `total` column to a `price` field in your destination.
You could do this by including the following Liquid in the template mapper:
```
{{ row['invoice'].total }}
```
You can also use this to map nested properties, for example, `customer.name`:
```
{{ row['invoice'].customer.name }}
```
If you want to sync multiple object properties individually, you can use these types of expressions for multiple mappings in your sync configuration.
Make sure you've set the model column's type as `Object / Array` in the [model's **Columns** tab](/models/data-types-casting#casting-from-your-model-configuration).
Otherwise, Hightouch syncs an `undefined` value to your destination.
Dot notation only works with model columns that contain single objects. For arrays of objects, you can use the [inline mapper](/syncs/mapping-data#inline-mapping).
## Field mapping updates
If you edit the field mappings in your sync configuration, the sync engine automatically reprocesses the entire [model query result set](/models/creating-models#preview-model-results) during the next sync run. Depending on the selected [sync mode](/syncs/types-and-modes#sync-modes), Hightouch syncs the model rows detected as **added** or as **changed** by [change data capture](/getting-started/concepts#change-data-capture) to your destination.
If the sync uses either **update** or **upsert** mode, Hightouch resyncs previously synced rows (with the updated mappings) as "changed" rows. If the sync uses **insert** mode, Hightouch doesn't reprocess previously synced rows. Instead, it only syncs model rows that are detected as **added** after the field mapping change was saved in your sync configuration.
### Model column changes
Most changes to your [model columns](/models/creating-models) aren't automatically detected by sync configurations.
Hightouch handles the changes differently depending on whether columns are added, renamed, or removed from your model definition, or if you change a column's data type.
If you add a new column to your model, Hightouch doesn't automatically add it as a field mapping to your sync configurations. This is because Hightouch doesn't know which columns you would like to sync to the destination, and which destination fields they should be mapped to. Follow the [field mapping](#usage) instructions to include new columns in your sync configuration.
You need to [preview and save](/models/creating-models#preview-model-results) your model to have new columns appear in the Hightouch UI.
If you rename a column in your model, all syncs that had that column mapped in their configuration fail with an `Unknown field name` error.
The error message also contains the original name of the renamed column. To resolve this, remove or update the relevant field mappings in your sync configuration.
Changes to model column names aren't detected for [**All** / **Mirror**](/syncs/types-and-modes#sync-modes) type syncs.
If you remove a column from your model, Hightouch doesn't automatically remove it from the field mappings in the sync configuration. Make sure to remove the mapping manually.
If you change a model column's [data type](/models/data-types-casting), the sync engine considers this update as a [field mapping update](#field-mapping-updates) and reprocesses all model rows during the next sync run.
As explained in the [primary key updates](/models/creating-models#updating-a-primary-key) section, if you update a model's primary key by selecting a different column, you need to trigger a [full resync](/syncs/overview#resync-full-query) or reset change data capture (CDC) for all syncs that use that model. Otherwise, [change data capture](/getting-started/concepts#change-data-capture) can't process your model data correctly, which can make your syncs fail.
### Destination field changes
If you rename a field in your destination, you need to update the mapping to this field in your sync configuration. Select the new field name from the dropdown menu and save your changes.
---
## Match boosting
**URL:** https://hightouch.com/docs/syncs/match-boosting
**Description:** Increase match rates for your audiences in advertising platforms.
**Section:** Syncs
Match Booster onboards your 1st party data and improves **match rates** on [various paid advertising platforms](/match-booster/overview#supported-destinations) by providing additional user identifiers. Learn more in the main [Match Booster documentation](/match-booster/overview).
## Implementation steps
Using Match Booster is a two-step process:
1. Enrichment: configure Match Booster to enrich your models in transit
2. Sync configuration: select which enriched identifiers to send into [supported destinations](/match-booster/overview#supported-destinations).
### Sync configuration
Once you've enabled [match boosting on a model](/match-booster/implementation#model-configuration), you can configure match boosting on syncs based on the model.
These include [audience syncs](/match-booster/implementation#sync-configuration) in Customer Studio.
The following instructions show you how to implement match boosting on existing syncs, but they also apply when configuring syncs for the first time.
You can only configure Match Booster on syncs to [supported
destinations](/match-booster/overview#supported-destinations).
1. Go to your [**Syncs** overview](https://app.hightouch.com/syncs) page and select the sync on which you want to configure **Match Booster**. Alternatively, create a new sync.
2. In the **Configuration** tab, ensure your [sync type](/syncs/types-and-modes#sync-types) is set to custom audience, customer match list, or segment. Match Booster only applies to [customer segment sync types](/match-booster/implementation#sync-configuration)—not events or other objects.
3. Locate the field mapping section with the text **Match boosting enabled**.
4. Select the **boosted column names** you want to map to fields in your destination.
5. **Save** your sync configuration.
From now on, Hightouch will sync any available additional identifiers to your destination.
Model enrichment can take up to few hours to process.
Therefore, if you enabled match boosting in your [model configuration](/match-booster/implementation#model-configuration) shortly before updating or creating your sync configuration,
it may take a few hours for you to see [increased match rates](/match-booster/implementation#view-match-rates).
Additionally, ad platforms have different practices on how frequently they calculate and update match rates, so you may continue see variation for a day or two after updating your configuration.
Check out the main [Match Booster documentation](/match-booster/faqs) page for
FAQs.
---
## Syncs
**URL:** https://hightouch.com/docs/syncs/overview
**Description:** An overview of your syncs along with their status, which model they use, the destination they write to, the schedule, last run, and when they were created.
**Section:** Syncs
**Syncs** declare how you want query results from a model to appear in your destination.
To learn more about syncs on a conceptual level, see the [core concepts page](/getting-started/concepts#syncs). Refer to destination docs for specific configuration details.
To learn more about the information the Hightouch UI displays on your syncs and what actions you can take from the UI, continue to the following sections.
## Syncs overview page
Hightouch displays all syncs, both active and inactive on the [**Syncs** overview](https://app.hightouch.com/syncs) page.
For each sync, the overview shows:
- The sync's [**status**](#sync-status)
- The sync's associated [**model**](/getting-started/concepts#models)
- The sync's associated [**destination**](/getting-started/concepts#destinations)
- The date and time, in your timezone, of the sync's **last run**
- The date the sync was **created at**, and who created the sync
You can view more in-depth information for any sync by clicking on it. This takes you to that individual sync's [overview page](#sync-overview).
### Sync status
Your sync's status is based on the latest information available on a run:
- **Healthy**: a recent run completed
- **Pending**: no runs have been scheduled yet
- **Disabled**: the sync has been turned off
- **Querying**: the sync is querying its associated model
- **Preparing**: the sync is preparing data to send to its associated destination
- **Queued**: the sync is scheduled to begin writing to its destination
- **In Progress**: the sync is writing to its destination
- **Incomplete**: the sync had a temporary interruption and will resume in a new run
- **Aborted/Failed**: there was an error in the sync progress—check the errors on the sync to see why the sync failed
## Individual sync overview page
You can view an individual sync's information by clicking on it on the on the general [**Syncs** overview](https://app.hightouch.com/syncs) page.
A particular sync's overview page shows general information, including:
- The sync's associated model and destination
- Whether the sync is currently **enabled** or **disabled**
- The option to manually **Run** the sync
You can also choose to move the sync to a [folder](/workspace-management/folders-filters#folders), edit its [labels](/workspace-management/lbac#what-are-labels), [resync the full query results](#resync-full-query) of a model, [reset change data capture](#reset-cdc), or delete a sync from here.
The overview page also shows more specific information including:
- **Sync ID**: a numerical unique identifier for the sync that is necessary when setting up an [Airflow](/extensions/airflow) operator, or using the [Dagster](https://github.com/hightouchio/dagster-hightouch), [Prefect](https://github.com/PrefectHQ/prefect-hightouch), or [Mage](https://docs.mage.ai/integrations/hightouch) extensions
- **Sync schedule type**: manual, interval, custom, cron, dbt Cloud-, or Fivetran-triggered
- **Sync mode**: this only appears if there are multiple sync mode options, see the [sync mode docs](/syncs/types-and-modes#sync-modes) for more information
- **Match rate**: this only appears if the sync is to a paid advertising destination, see the [match rate docs](/customer-studio/syncs#view-match-rates) docs for more information
- **Any associated labels**: see [label based access-controls](/workspace-management/lbac) for more information
Finally, you can click the tabs on the overview page to view and edit settings:
- **Runs**: You can see the status, time, and [more details](#run-details) from the most recent runs. Selecting a particular run brings you to the [live debugger](https://hightouch.com/docs/syncs/debugger).
- **Configuration**: You can view and edit the sync's [configuration](#configuration).
- **Schedule**: You view and edit the sync's run [schedule](#schedule).
- **Alerts**: You view and edit the sync's [alerts](/syncs/alerting).
- **Sync logs**: You can view and edit where your [sync's historical logs](/syncs/warehouse-sync-logs) are stored.
- **Activity**: You can view recent changes to the sync configuration and other settings
### Resync full query
By default, Hightouch keeps track of changes to your data between syncs and [only sends necessary updates](/getting-started/concepts#change-data-capture).
Therefore, an initial sync run [often shows](/syncs/debugger#view-run-details) that many records were **Added**. Subsequent runs often show fewer records **Added**, and instead show that many records have **Changed** or **Removed**.
If you wish to reset the sync's tracking to its initial state—as if it were syncing for the first time again—you can select **Resync full query** from the menu on the sync's main page.
Before you do, make sure that your sync meets [the prerequisites](#full-resync-prerequisites) to safely do so.
#### Full resync prerequisites
Full resyncs process all rows that currently exist in the model as **Added** rows. Check the following before triggering a full resync:
| Criteria | Explanation |
| ----------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Sync mode](/syncs/types-and-modes#sync-modes) |
Syncs set to [**update** or **upsert** mode](/syncs/types-and-modes#sync-modes) are safe to fully resync because if a row was already synced, Hightouch updates the same record.
[**Insert** mode](/syncs/types-and-modes#sync-modes) syncs, [event syncs](/syncs/types-and-modes#event-syncs), and [webhook destinations](/destinations/http-request) aren't recommended for full resyncs because reprocessing rows can create duplicates.
|
| [Delete behavior](#delete-behavior) |
If your destination allows for deleting records, and you selected to **Do nothing** when records leave the model query result set, a full resync will have no impact.
If you selected **Clear fields** or **Delete destination record**, this setting gets ignored during a full resync. This is because full resyncs only process rows that currently exist in the model and run the sync as if it were the initial run. This means that there will be no [diffing](/getting-started/concepts#change-data-capture) or comparison to the prior run to identify what has been removed. Therefore, all **Removed** rows (in comparison to the previous sync run) are ignored even if delete behavior is enabled.
|
Resyncing the full query could create duplicates or other issues in your destination data if the prerequisites above haven't been met.
### Clear and fill
If the destination supports removing all records in a segment or audience, this option will become available. This resync mode is similar to a [full resync](#resync-full-query) mentioned above with an additional preprocessing step.
Before processing all rows that are currently in the model, the segment/audience selected in the sync configuration will be cleared of all users/identifiers.
### Reset CDC
If you want to reset [change data capture](/getting-started/concepts#change-data-capture) without resyncing records, you can select the "Reset CDC" option instead.
This instructs Hightouch to ignore the current state of your model and to track only rows that are added, changed, or removed in the future. The next sync run will be used as an opportunity to capture a snapshot of your model, but no data will be sent to the destination. Thereafter, future sync runs will track new changes only.
The "Reset CDC" option is recommended for insert-only use cases like conversion events, operational alerts, and other situations where a backfill would create undesired duplicate records. Beware that skipping the backfill may cause your destination to drift out of sync with your data model.
### Run details
When viewing a specific sync, you can inspect detailed run-level information. A run is a particular invocation of a sync, either triggered manually or scheduled.
Each run provides:
- The particular run's **status**
- The time the run **started** and its duration in seconds
- The number of **rows queried**
- The total number of **operations**, including rows added, changed, removed
- The **results**—the total number of successful operations in green and the number of rejected operations in red
If there were any errors, you can click the **View Error** button to view details related to the run.
### Configuration
The configuration section exposes destination-specific configuration options. While sync configurations vary depending on the destination, they generally include:
- **Type**: What data type to sync to in your destination; for example, objects, events, subscription lists, etc. To learn more, see sync [types and modes](/syncs/types-and-modes).
- **Mode**: What update mode to use; for example, upsert, insert, update, etc. To learn more, see sync [types and modes](/syncs/types-and-modes).
- **Record matching** If updating or upserting, how rows in the source should be matched to records in the destination. To learn more, refer to the [record matching docs](/syncs/record-matching).
- **Field mapping**: What columns from your source should be synced to the destination, and what destination fields they should appear as. To learn more, refer to the [field mapping docs](/syncs/mapping-data).
- **Batching**: How data should be batched in API requests.
- **Delete behavior**: What to do when a record leaves your source—in other words, when your model no longer returns a record that was returned in the previous sync run.
#### Delete behavior
For some sync types and destinations, you may want to delete records if they no longer appear in your model's query results.
For example, you may be syncing customer records to your CRM based on a [model](/getting-started/concepts#models) that queries all active users from your data warehouse. If some users become inactive—in other words, they no longer appear in your model's query results—you may want to delete them or some information about them from your CRM.
Depending on the destination and sync mode, these are common options for delete behavior:
| Behavior | Description |
| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Do nothing** | Preserve the record in the destination. This option is useful for when you want your destination to continually **accrue** records from your source. |
| **Clear fields** | Keep the record, but clear the mapped fields. This option useful to retain records in your destination, but clear the fields that are mapped in your sync. |
| **Delete destination record** | Delete the record in the destination. This option is useful for when you want your destination to exactly **mirror** your source by dropping records that have dropped out of your model. |
Hightouch only acts on records as they leave your model's query results, and _after_ you apply deletion settings. Hightouch doesn't have access to historically removed records, so syncs can't delete records or clear fields that have already been removed from your source. Hightouch only deletes records that left your source in comparison to the immediately previous run. A [full resync](#resync-full-query) doesn't delete or clear historically removed records.
Regardless of the option you choose for delete behavior, a [sync run's detail page](/syncs/debugger#view-run-details) always displays removed rows.
Check out the particular destination's documentation for its delete behavior configuration details.
#### Configuration actions
When working with a sync configuration, you can also choose to [test a row](#test-a-row) or edit the configuration [as a JSON](#edit-as-json) object.
#### Test a row
You can test a sync by sending just one row from your model to your destination. This lets you verify that the sync is working as intended before initiating a run of your entire sync.
From your sync's configuration page, click **Test** and select a row to sync.
Before testing a row, you can edit the columns’ data types and values by hovering over them.
Click the pencil icon, make your edit, and then click the check mark to save any changes.
Editing the model column used for [record matching](/syncs/record-matching#model-column-selection) is not supported since Hightouch [intentionally strigifies](/models/data-types-casting) it for enhanced performance.
The row testing window only displays certain [data types](/models/data-types-casting#data-types), such as **string** and **number**.
If a model contains a column with a different type, such as **date**, the UI displays the type as **string**.
This only concerns the UI—Hightouch doesn't convert the data type to string when syncing data.
To confirm a column's data type, refer to its type on the [model configuration](/models/data-types-casting#view-model-data-types) page.
You can then decide to sync the row as an [added, changed, or removed row](/syncs/debugger#view-run-details) and observe the API requests and responses.
The UI might take a few seconds to update before displaying the results.
#### Edit as JSON
You can edit your sync's configurations as JSON code to quickly make multiple changes or copy and paste one sync's configurations to another.
For example:
```JSON
{
"mode": "upsert",
"type": "object",
"fromId": "track_visitor_id",
"object": "user",
"mappings": [
{
"to": "uuid-western",
"from": "uuid"
},
{
"to": "Trialer Status",
"from": "trialer_status"
}
],
"customMappings": [
{
"to": "unique_notification_status",
"from": "notify_status"
}
]
}
```
### Schedule
Part of sync configuration is deciding how frequently the sync should run. Besides triggering syncs manually, you can schedule syncs with these options:
- **Interval**: You can schedule your sync to run on a set interval, for example once every day.
- **Custom recurrence**: You can schedule your sync to run a specific date and time, for example every Monday at 9 AM.
- **Cron expression**: You can schedule your sync using a [cron expression](https://en.wikipedia.org/wiki/Cron).
- **dbt Cloud**: You can trigger syncs automatically via [dbt Cloud](/extensions/dbt-cloud).
- **Fivetran**: You can trigger syncs automatically via [Fivetran](/extensions/fivetran).
Though not listed as schedule options, you can also trigger syncs using [Airflow](/extensions/airflow), [Dagster](https://github.com/hightouchio/dagster-hightouch), [Prefect](https://docs-v1.prefect.io/api/latest/tasks/hightouch.html), [Mage](https://docs.mage.ai/integrations/hightouch), or the [REST API](/api-reference). To learn more about your scheduling options, refer to the [sync scheduling page](/syncs/schedule-sync-ui).
If you have dependencies between syncs, for example you require one sync to complete before another begins, sync [Sequences](/syncs/schedule-sync-with-sequences) can help you manage scheduling.
If you're planning around Daylight Saving Time changes, review the [DST guidance](/syncs/schedule-sync-ui#daylight-saving-time-dst) on the sync scheduling page.
### Alerts
You can configure [alerting](/syncs/alerting) on a per-sync basis from this section.
### Sync logs
You can configure [Warehouse Sync Logs](/syncs/warehouse-sync-logs) from this section. This feature writes information on the results of your syncs back into your warehouse so that you can perform more complex analysis.
---
## Record matching
**URL:** https://hightouch.com/docs/syncs/record-matching
**Description:** Learn how Hightouch matches records from your model rows to your destination records.
**Section:** Syncs
If you're using [update, upsert, or archive mode](/syncs/types-and-modes) to sync data, one of the first steps of your sync configuration is defining how Hightouch should match rows from your model to existing destination records.
To do this, Hightouch asks you to select the model column and destination field to match on.
**"Columns" vs. "fields" vs. "properties"**: Though some may use these terms
interchangably,
Hightouch uses the term "columns" to refer to a set of vertical data values in
a source or model and "fields" or "properties" to refer to them in a destination.
## Destination field selection
It's often best to start by selecting the destination field to match on because some destinations have requirements or recommendations on the field to use.
For example, some destinations require matching on PII fields like email or phone number. Hightouch only displays permitted fields in the dropdown when selecting a destination field to match records on.
For example, [Attentive](/destinations/attentive#matching-users) allows you to match users on several fields, including email, phone number, and various IDs.
If there is a recommended field, Hightouch displays **(recommended)** next to the field name. For example, [Salesforce](/destinations/salesforce) recommends using an external ID field like `External_ID__c` for matching.
Refer to the record matching section of each destination doc for more information.
### External IDs
Some destinations, for example, [Meta Conversions API](https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/external-id/) and [Salesforce](/destinations/salesforce), use the concept of **external IDs**. External IDs refer to unique identifiers from another system "external" to the destination platform. This allows the platform to maintain its own primary unique identifiers while enabling associations with data from other platforms.
Each platform may have different specifications for fields used as external IDs. For example, [Salesforce](https://help.salesforce.com/s/articleView?id=000385174&type=1) requires `External ID` type fields to be the explicit External ID type in Salesforce, not just an external ID field by practice. Ensure that any destination field you select for record matching complies to the platform's specifications.
## Model column selection
When selecting which model column to use, you need to consider more than the destination requirements.
Since the column you select is used to identify which records to update, it's imperative to **choose a column with unique values**. For that reason, the model column you choose is often the same as the [primary key](/getting-started/concepts#unique-primary-key-requirement) column you select for your model. Though the model's primary key can be used for record matching, this isn't mandatory.
For example, you may have `email` and `user_id` fields in your model. Which you choose for record matching depends on:
- Which fields in your destination are available and expected for matching
- Which field is guaranteed to be unique
If both fields are accepted by the destination for matching, you should choose the one which you are more confident is strictly unique.
---
## Retry strategy
**URL:** https://hightouch.com/docs/syncs/retries
**Section:** Syncs
Whenever a sync runs, Hightouch tracks all records that the destination rejects using their [primary key](/getting-started/concepts#unique-primary-key-requirement). Hightouch then automatically retries syncing these records two ways:
1. **Within an Active Sync:** If Hightouch receives any record-level errors during an active sync, it automatically retries those records at an exponential backoff until it meets a destination-dependent failure threshold. Hightouch's retry process is similar to the one [described by AWS](https://docs.aws.amazon.com/general/latest/gr/api-retries.html). Some destinations support [split retries](#split-retries) as part of the active sync retry strategy.
2. **On Subsequent Syncs:** Hightouch keeps an index of all failed records and automatically retries them on every subsequent sync until they succeed.
## Example retry scenario
Imagine you are trying to sync 10,000 records to Salesforce in a single sync, and five of the records are locked within Salesforce at the beginning of the sync.
- During that sync run, imagine two of those five records become available for update. Because Hightouch performs retries during an active sync, these two will update successfully during the sync run.
- On the subsequent sync run, Hightouch automatically retries the remaining three records. Assuming they're unlocked by that point, they will successfully update on the subsequent sync. Hightouch continues to retry the failed records until they are successfully synced.
## Split retries
Some destinations count all records in a batch request as rejected if the request contains a single invalid record.
For some of these, Hightouch provides you the option of enabling **split retries**.
If you enable split retries, Hightouch splits the rejected batch in half and retries each sub-batch separately.
If either of those sub-batches fail, Hightouch splits _it_ in half and retries each sub-batch separately.
Hightouch repeats this process until the destination has marked each record in the batch as successful or rejected.
The split retry strategy allows you to pinpoint which records are getting rejected with which errors.
It also reduces the number of valid records that get retried.
Split retries are part of the [active sync retry strategy](/syncs/retries).
Hightouch retries records that are still rejected at the end of the split retry process in the subsequent sync.
To see if a destination supports split retries, refer to its docs page.
Enabling split retries may lead to longer sync times.
## FAQ
### If part of my sync failed, does Hightouch retry every record in the sync?
Hightouch generally only retries failed records.
For example, if you try to sync 1,000 records to Salesforce, but only 600 pass, Hightouch retries only the 400 that failed.
The exception to this is when a destination marks an entire batch as failed, even if only some of the records are invalid.
In that case, Hightouch retries the entire failed batch.
Enabling [split retries](#split-retries) can help to reduce the number of valid records that get retried.
### How can I be alerted to failing records or syncs?
Hightouch retries failed records until the underlying issue is addressed.
To become aware of failed records or syncs and fix them, it's best to set up [alerting](/syncs/alerting).
### How can I avoid Hightouch retrying invalid rows?
If one or more of your records have malformed data that won't be accepted by the destination, Hightouch retries them and they will continue to be rejected.
If you don't want to sync these records to the destination, you can [edit your model's definition](/models/sql-editor) to filter out these rows.
The SQL query you use to filter out invalid rows will depend on your destination's definition of accepted values.
### Why is Hightouch retrying a removed row?
If rows are removed from your model query result and you've configured [delete behavior](/syncs/overview#delete-behavior) for your sync, Hightouch sends requests to your destination to clear or delete removed records. Hightouch either clears or deletes the removed records from the destination based on the delete behavior setting you configured.
If a sync is unable to clear or delete certain destination records, Hightouch retries the request during the following sync run. This can happen because the destination rejects the request or if the destination's API is inaccessible.
If you don't want the sync to retry removed rows in future sync runs, you can choose one of the following options:
1. Set delete behavior to **Do nothing**. Keep in mind that this affects all rows removed from the model's query results, not just the ones being retried.
2. Temporarily set delete behavior to **Do nothing**, run the sync [manually](/syncs/schedule-sync-ui#manual-runs) once,
and then reconfigure the sync to the delete behavior setting you would like. This clears out the removed rows from the [diff file created by CDC](/getting-started/concepts#change-data-capture).
3. [Reset change data capture (CDC)](/syncs/overview#reset-cdc) without a backfill.
Please keep in mind that:
- with option 2), during the run where delete behavior is set to **Do nothing**, no model rows that CDC marks as "removed" are synced to the destination
- with option 3), any changes to your model (added, changed, or removed rows) between the previous run and the Reset CDC run aren't synced to the destination
To minimize the risk of model data not being synced to the destination, we recommend triggering a regular sync run right before following through with options 2) or 3).
---
## Schedule syncs in the UI
**URL:** https://hightouch.com/docs/syncs/schedule-sync-ui
**Section:** Syncs
## Overview
Syncs declare how you want query results from a model to appear in your destination, including *how often* you want the data to be updated. This last step of sync configuration is called a sync's **schedule**, and you can configure it from the Hightouch UI.
From the [**Syncs** overview](https://app.hightouch.com/syncs) page, select the sync you want to schedule and open the **Schedule** tab. You can then choose the sync's **Schedule type**.
### Schedule types
- **Manual**: Run your sync manually or via the [Airflow](/extensions/airflow), [Dagster](https://github.com/hightouchio/dagster-hightouch), [Prefect](https://docs-v1.prefect.io/api/latest/tasks/hightouch.html), [Mage](https://docs.mage.ai/integrations/hightouch) extensions, or the [REST API](/api-reference). You must also select this schedule type for any syncs you plan to use [in a sequence](/syncs/schedule-sync-with-sequences).
- **Interval**: Run your sync on a set interval, for example, every 12 hours.
- **Custom recurrence**: Use a visual interface to set what days and times your sync should run, for example, every Monday at 8:30 AM.
- **Cron Expression**: Use a cron expression to decide when your sync should run. This option is helpful if you need highly customized schedules, for example, every minute from 9 AM - 5 PM and every hour outside of business hours.
- **dbt Cloud**: Schedule your sync to run upon the completion of a dbt job.
- **Fivetran**: Schedule your sync to run upon the completion of a Fivetran job.
Times you configure in the UI and the respective sync schedules are in your local timezone. Cron expressions are evaluated in UTC.
### Manual runs
A sync with a **Manual** schedule type only runs when you tell it to run. Click the **Run** button in the top right corner of your sync's configuration page to kick off a manual run.
You also need to select a **Manual** schedule if you plan to:
- use the sync [in a sequence](/syncs/schedule-sync-with-sequences)
- trigger the sync with the [Airflow](/extensions/airflow), [Dagster](https://github.com/hightouchio/dagster-hightouch), [Prefect](https://docs-v1.prefect.io/api/latest/tasks/hightouch.html), or [Mage](https://docs.mage.ai/integrations/hightouch) extensions
- trigger the sync with the [REST API](/api-reference)
### Interval schedules
Use the **Interval** schedule type to configure a sync to run every *n* **minutes**, **hours**, **days**, or **weeks**.
You can choose when to apply the schedule—effective immediately or from a specific date and time—and how long the schedule should remain effective—indefinitely or until a particular date and time.
Interval scheduling starts the next sync run at approximately the specified interval after the last run. The sync start time can drift due to various reasons including [manual runs](#manual-runs) and background processes.
If you require precise sync runs start times, use [custom recurrence scheduling](#custom-recurrence-schedules) for a more stable sync schedule.
### Custom recurrence schedules
Use the **Custom recurrence** option to schedule your sync on specific days and times. The time you select in the UI is your local timezone, and your sync runs at that time in your local timezone.
You can add multiple recurrences by selecting **Add recurrence** and configuring the day and time.
The preceding screenshot shows a sync schedule that runs at midnight on weekdays and on 8 AM weekends.
You can choose when to apply the schedule—effective immediately or from a specific date and time—and how long the schedule should remain effective—indefinitely or until a particular date and time.
### Cron expression schedules
You can schedule your sync with a [cron](https://en.wikipedia.org/wiki/Cron) for even greater specificity. Your cron expression should contain five—not six—fields, since the smallest time denomination Hightouch supports is minutes not seconds. Refer to the example expressions below:
| Cron expression | Description |
| :-------------- | :-------------------------------------------------- |
| * * * * * | Every minute |
| 0 12 * * * | Every day at noon |
| 0 12 1 * * | Every month on the first day of the month at noon |
| 0 12 L * * | Every month on the last day of the month at noon |
| 0 12 * 12 1,2 | Every Monday and Tuesday at noon in December only |
Consult a [cron expression generator](https://crontab.guru/#*_*_*_*) to help build an expression that meets your needs.
Cron expressions are evaluated in UTC.
Hightouch doesn't support the following non-standard characters for Cron expressions: `W`, `LW`, `L-2`, `L-3`, `#`.
### dbt Cloud
To schedule syncs via dbt Cloud, dbt requires you to have a paid plan.
Refer to our documentation on [scheduling a sync with dbt](/extensions/dbt-cloud) for setup instructions.
### Fivetran
Refer to our documentation on [scheduling a sync with Fivetran](/extensions/fivetran) for setup instructions.
## Tips and troubleshooting
### Daylight Saving Time (DST)
Daylight Saving Time can affect schedules differently depending on the schedule type:
- **Cron expression** schedules are evaluated in UTC, so the UTC runtime stays fixed while local wall-clock time can shift by one hour when DST starts or ends.
- **Custom recurrence** schedules run at the time shown in your local timezone in the UI.
For DST-sensitive schedules, Hightouch recommends these approaches:
- If possible, define your SLA in UTC and keep the schedule fixed.
- If a one-hour local-time shift is acceptable, keep your current schedule and monitor around DST transitions.
- If you need a strict local wall-clock time (for example, "7:00 AM local" year-round), use an external timezone-aware scheduler and trigger the sync with the [REST API](/api-reference). Common options include [Airflow](/extensions/airflow), [Dagster](https://github.com/hightouchio/dagster-hightouch), [Prefect](https://docs-v1.prefect.io/api/latest/tasks/hightouch.html), cloud schedulers, or your own application scheduler.
### Disabled syncs continue to run
If you [disable](/syncs/overview#individual-sync-overview-page) a sync but it still has recent runs, check to see if it's part of a [sequence](/syncs/schedule-sync-with-sequences). Sync schedules and sync sequences run independently of each other, so a sync that is disabled still runs if it is part of a sequence. Hightouch recommends setting any syncs you plan to use in a sequence to the [manual schedule type](#manual-runs).
Disabled syncs can also run if they're [triggered manually](#manual-runs) or by an external service, such as [Airflow](/extensions/airflow).
---
## Schedule syncs with Sequences
**URL:** https://hightouch.com/docs/syncs/schedule-sync-with-sequences
**Section:** Syncs
## Overview
Sync **Sequences** allow you to execute a series of related syncs in a predefined order. Sequences are helpful when you have dependencies between syncs. For example, you may have two syncs you need to run in order:
1. Sync A creates a set of new contacts in your CRM
2. Sync B adds the new contacts to their associated accounts
Without Sequences, you would have to schedule the two syncs individually to give the first enough time to finish before the second starts. It would be challenging to guarantee the correct behavior since sometimes sync duration can vary.
Sequences make orchestration a first-class feature in Hightouch so you can control how and when to sync data to your destinations. You can schedule sequences to run on various [schedules](/syncs/schedule-sync-ui#schedule-types), such as [triggered by dbt Cloud runs](/extensions/dbt-cloud) or [cron schedules](/syncs/schedule-sync-ui#cron-expression-schedules), just as you would with syncs. You can also [initiate sequences manually](#sequence-management).
## Sequence example
Suppose you manage your staff members, their permissions, customer information, and staff customer assignments using Hightouch. To do so, you use four syncs that need to run in a specific order:
1. The table of staff members needs to be updated first.
2. Then the staff permissions can be updated.
3. Separately, a table of customer prospects needs to be updated.
4. The final sync updates staff member assignments to customer prospects.
To represent this sequence of syncs, you can create a sequence that executes these four syncs in order with the following error handling:
- The staff member sync terminates the sequence if it fatally errors or has too many rejected rows.
- The second and third syncs only terminate the sequence if they fatally error.
- The final staff member assignment sync doesn't check for errors, so the sequence completes successfully even if this sync fails.
## Sequence setup
Before you create a sequence, ensure that all syncs you plan to use in it have their schedule type set to [**Manual**](/syncs/schedule-sync-ui#schedule-types). You can configure a sync to have a set schedule and use it in a sequence simultaneously, but we recommend against it unless you have a particular reason to do so.
Then, to create a new sequence, go to the [**Sequences** page](https://app.hightouch.com/sequences) and click the **Add a sequence**.
### Add syncs to the sequence
Click **Add syncs** to select syncs for the new sequence. You can include as many syncs as you want and add more later as needed.
### Configuring sync order and error handling
Once you've added syncs to the sequence, you can re-order them using the up and down arrows.
You can configure each sync's error-handling behavior using the **Terminate** drop-down menu.
### Set your sequence schedule
To finalize your sequence, give it a name and define its schedule type. You have all the same options for sequence scheduling as for [sync scheduling](/syncs/schedule-sync-ui#schedule-types).
Once you save your new sequence, it's visible on the main [**Sequences** page](https://app.hightouch.com/sequences).
## Sequence management
You can click a sequence from the [**Sequences** page](https://app.hightouch.com/sequences) to go to its details page. There you can:
1. Disable or enable it
2. Run it manually
3. See the sequence's run history
4. Or update its configuration or schedule
Clicking on a particular run displays its details in a pop-up, including the execution time and final state of each sync run belonging to the sequence run.
---
## Sync types and modes
**URL:** https://hightouch.com/docs/syncs/types-and-modes
**Description:** Learn more about different types of syncs and update modes, inluding upsert, update, insert, and more.
**Section:** Syncs
When [configuring a sync](/syncs/overview#configuration) there are two key settings that determine the basic behavior of your sync:
- Sync type
- Sync mode
The options for both vary depending on the destination. Refer to the specific destination documentation for more information about its supported sync types and mdoes.
## Sync types
The **sync type** refers to _what_ you are syncing in a destination. These are the most common types:
- **Objects**: such as customers, accounts, organizations, catalog items, and the objects' associated attributes or metadata
- **Events**, **Actions**, and **Activities**: such as conversion events
- **Segments**, **Audiences**, and **Lists**, : such as subscription or campaign lists, specifically user membership in these lists
When syncing to CRM type destinations, it's commonly to object type records. When syncing to advertising or marketing platforms, it's commonly to objects, events, or lists.
For some destinations, you can also [manage journeys](/destinations/sfmc#managing-user-journeys), [trigger campaigns](/destinations/marketo#campaign-trigger), or [merge anonymous and known users](/destinations/braze#merge-anonymous-and-known-users) by syncing data. Check out a destination's dedicated docs to learn its supported sync types.
## Sync modes
Depending on your sync type, you may have to select your **sync mode**. For example, if you're updating contacts, you might have the option to **Insert**, **Update**, or **Upsert** them.
Also called update mode, this configuration describes _how_ you want to update records in your destination based on your [model's](/models/creating-models) query results. For example, if your model queries a **Users table** and rows in that table change, your sync mode determines how those changes should translate to your destination.
Depending on the destination and sync type, the supported sync modes could include:
- **Update**: only update existing records in your destination
- **Insert**: only insert new records without touching existing records
- **Upsert**: update existing records and insert new ones
- **Add**: add records, usually user records in a segment, list, or journey
- **Remove**: remove records, usually user records in a segment, list, or journey
- **Archive**: archive all records that appear in your model's query results (only for Mailchimp, Intercom, and Salesforce destinations)
- **All**: overwrites all existing records in your destination with the results of your model query; sometimes called **Mirror** mode
- **Snapshot**: creates a new file on each sync run with the full results set of your model query
- **Diff**: creates three new files on each sync run—one for rows added, one for rows changed, and another for rows removed since the last sync
Some sync types such as triggering campaigns, merging users, or sending messages don't require you to select a mode since the sync applies an action rather than updates records. Other sync types, such as [events](#event-syncs), have only one sync mode option.
In **All** and **Snapshot** sync modes, the order of rows and columns in exported files may differ from how they appear in your model. This is normal behavior. [Learn more about row and column ordering →](/models/creating-models#row-and-column-ordering)
### Sync modes and change data capture
[Change data capture (CDC)](/getting-started/concepts#change-data-capture) is a process that captures changes made to your source data and that records those changes. Specifically, CDC tracks rows that were added to, changed, or removed from your model's query results since the last sync run.
Your sync mode determines how Hightouch acts on observed changes:
- In **update** mode, Hightouch only acts on changes to existing rows.
- In **insert** mode, Hightouch only syncs new rows—those whose [primary key](/getting-started/concepts#unique-primary-key-requirement) wasn't present in the previous sync run.
- In **upsert** mode, Hightouch acts on existing _and_ new rows rows.
- **Archive** and **all** modes don't perform change data capture and don't store requests in the [live debugger](/syncs/debugger).
Sync mode is separate from [delete behavior](/syncs/overview#delete-behavior), which determines how to treat removed rows—those that leave your model's query results.
## Sync type and mode combinations
Certain destination types tend to support certain sync modes.
|Destination type|Example destinations|Commonly supported sync modes |
|--|--|--|
|CRMs and other object-based destinations |Salesforce, HubSpot, Braze, Iterable | Upsert, Update, Insert |
|Advertising| Google Ads, Meta Conversions API, TikTok, Amazon Ads | Insert for events, Update or Upsert for customer segments, lists, and audiences |
|File storage| Google Drive, Amazon S3, Azure Blob,|Insert, All, Diff|
Check out Hightouch's [integration catalog](https://hightouch.com/integrations) for a complete list of destinations and categories.
### Event syncs
Event syncs are **insert** only, meaning that you can't update previously synced events or activities.
Hightouch follows this convention because events tend to be part of [fact tables](https://allthingssql.com/fact-table-vs-dimension-table/) rather than dimension tables.
Fact tables store transactions as rows that shouldn't change.
Recall that in [insert mode](#sync-modes-and-change-data-capture), Hightouch syncs rows whose [primary key](/getting-started/concepts#unique-primary-key-requirement) wasn't present in the previous sync run.
If the event sync's associated model has a primary key that appears in multiple events—for example, `user_id`—syncs will omit some events.
To ensure syncs include all events, select a truly unique column, for example a unique `eventID`.
If your events data doesn't contain an inherently unique column, define a [composite primary key](/models/creating-models#composite-primary-keys) during model creation.
---
## Warehouse Sync Logs
**URL:** https://hightouch.com/docs/syncs/warehouse-sync-logs
**Section:** Syncs
The **Warehouse Sync Logs** feature writes sync metadata back into your data warehouse. It makes per-row information from the [live debugger](/syncs/debugger) available in your warehouse so that you can perform complex analysis, rather than just inspect syncs row-by-row.
When you enable Warehouse Sync Logs, Hightouch logs a corresponding row for every row processed during the sync. This includes the row's **status** and any errors from processing the row. You can then explore these logs using SQL or BI tools you use on top of your warehouse.
For example, you can:
- **Categorize** all the errors in your sync using [regular expressions](https://docs.snowflake.com/en/sql-reference/functions-regexp.html) and find unexpected errors.
- **Filter** out previously failed rows from your [model](/getting-started/concepts#models) using a `JOIN`.
- **Aggregate** the sync history to see what rows are changing the most. Flapping rows can be a sign of data integrity issues.
- **Visualize** row changes in your models over time. For example, you may be interested in seeing how targeted users in an ad campaign changed over the campaign duration.
Refer to the [example queries](/syncs/warehouse-sync-logs#example-queries) section for concrete examples.
## Schema
Hightouch writes sync logs into three tables within the `hightouch_audit` schema:
- **Changelog**: This table contains a row for every operation performed by Hightouch. It includes the result of the operation and any error messages from syncing.
- **Snapshot**: This table contains each row's latest status in your model. The information is similar to the **Changelog** table, but since it contains the latest status, it's easier to query for some use cases.
- **Sync runs**: This table contains a log of all the sync runs. You can `JOIN` the changelog and snapshot tables to this table for more information on when the sync occurred and how it was configured.
Information across all syncs is written into these same three tables—you can differentiate which rows were part of which sync using the `sync_id` column.
See the [detailed schema](/syncs/warehouse-sync-logs#detailed-schema) section for a detailed description of the available columns.
You can learn more about PII in these tables on the [Lightning sync
engine](/syncs/lightning-sync-engine#table-removal) page.
## Setup
Enabling Warehouse Syncs logs requires you to enable [the Lightning sync engine](/syncs/lightning-sync-engine) first. Hightouch supports using Warehouse Sync Logs with the following sources:
- [Amazon Redshift](/sources/amazon-redshift)
- [Databricks](/sources/databricks)
- [Google BigQuery](/sources/google-bigquery)
- [Greenplum Database](/sources/greenplum)
- [PostgreSQL](/sources/postgresql)
- [Snowflake](/sources/snowflake)
### Required permissions
The user you used to connect your source to Hightouch must be able to write into the `hightouch_audit` schema. You shouldn't require any additional permissions once you've set up [the Lightning sync engine](/syncs/lightning-sync-engine#granting-required-warehouse-permissions).
## Enable Warehouse Sync Logs for a source
To enable logging for **all syncs** that use a source:
1. Go to `Integrations` → `Sources`.
2. Select your source.
3. Open the `Sync logs` tab.
4. Select the log tables you want Hightouch to write to your warehouse:
- **Sync snapshots**
- **Changelog**
- **Sync runs**
- **Audience tables** (optional)
5. Save.
Enabling a table here allows Hightouch to write logs for **any sync** that uses this source.
**Be mindful of storage usage**. Sync logs can generate a lot of data depending on how many tables you enable and how often your syncs run. If you only need logs for a few workflows, you can enable them at the **sync level** instead of the source level to reduce storage. However, enabling logs at the **source level** ensures logs are always available when debugging issues, which can save considerable time if a sync fails unexpectedly.
### Enable Warehouse Sync Logs for a sync
Use sync-level logging when you only need logs for specific workflows or want to limit storage usage. Use source-level logging when you want logging enabled consistently across all syncs powered by the source.
Warehouse Sync Logs are off for all syncs by default. To enable them on a particular sync:
1. Ensure the [Lightning sync engine](/syncs/lightning-sync-engine#enable-the-lightning-sync-engine) is enabled for the source.
2. Go to the **Sync Logs** tab in the sync's overview page.
3. Enable your desired [tables](/syncs/warehouse-sync-logs#schema): **Snapshot**, **Changelog**, and/or **Sync runs**.
## Example queries
The following example queries are written for Snowflake, but you could create similar queries for other sources. Check out Hightouch's [dbt package](https://github.com/hightouchio/dbt-hightouch) for more use cases.
### Get the most common sync error
This SQL groups and counts rows by `failure_reason`, enabling you to find the most common sync error.
```
select
failure_reason,
count(*) as c
from hightouch_audit.sync_snapshot
where failure_reason is not null
group by failure_reason
order by c desc
```
### Track when users entered and exited a model
This SQL tracks when users enter and exit a model. It's particularly useful when used with [Customer Studio audiences](/customer-studio/overview) and visualized in a BI tool.
```
with details as (
select
model_name,
row_id,
op_type as type,
started_at as timestamp,
lag(op_type) over(partition by model_name, row_id order by started_at) as lag_type
from hightouch_audit.sync_changelog c
join hightouch_audit.sync_runs r on c.sync_id = r.sync_id
where op_type != 'changed'
order by model_name, row_id
)
select
row_id as user_id,
model_name as audience,
type,
timestamp
from details
where (lag_type != type or lag_type is null)
order by model_name, row_id, timestamp
```
### Get the current rows in all models
This SQL finds all current (most recently synced rows that didn't fail) across all models. It's particularly useful to find all members for audiences created in [Customer Studio](https://hightouch.com/platform/customer-studio).
```
with model_names as (
select distinct
sync_id,
model_name
from hightouch_audit.sync_runs
)
select
model_name,
row_id as user_id
from hightouch_audit.sync_snapshot s
join model_names r on s.sync_id = r.sync_id
where s.status != 'failed'
qualify row_number() over (partition by user_id, model_name order by null) = 1
order by user_id
```
## Detailed schema
Hightouch writes to the `sync_changelog`,`sync_snapshot`, and `sync_runs` tables after each sync.
If you've enabled **audience snapshotting**, you'll also find a `hightouch_planner.audience_membership` table in your warehouse.
Refer to the [audience snapshot docs](/customer-studio/audience-snapshots) for its detailed schema.
### Changelog table
This `hightouch_audit.sync_changelog` table is log of all changes across all sync runs. If the same row is synced in multiple sync runs, it has multiple entries in this table.
| COLUMN | DESCRIPTION |
| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `sync_id` | The ID of the sync |
| `sync_run_id` | The ID of the sync run |
| `op_type` | Whether the row was added, changed, or removed relative to the last run. This is computed by Hightouch when planning the sync run |
| `row_id` | The value of the row's primary key as defined from the model |
| `status` | Whether the row was successfully synced into destination. They value may be: succeeded - the row was successfully synced, failed - Hightouch attempted to sync the row, but it failed to sync, or aborted - Hightouch planned to sync the row, but didn't attempt to sync. This may happen if the sync may have been cancelled, or the sync encountered a fatal error that terminated the run early. |
| `failure_reason` | If the status is failed, this field contain a string describing why the row failed to sync. |
| `fields` | A JSON object of the raw data from the model that is synced into the destination. Note that this is the raw data from the warehouse, not the payload that Hightouch sent to the destination. This column has limitations in Redshift (see FAQ for details, below). |
| `split_group` | (Optional) The [split group](/customer-studio/splits) name. If no syncs are using splits, this column isn't created. |
### Snapshot table
This `hightouch_audit.sync_snapshot` table stores the current status of each row in the most recent sync run, even if the row wasn't synced in the most recent run.
After each run, the old statuses for the sync are dropped and replaced with updated statuses.
| COLUMN | DESCRIPTION |
| --------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `sync_id` | The ID of the sync |
| `op_type` | Whether the row was added, changed, or unchanged relative to the last run |
| `row_id` | The value of the row's primary key as defined from the model |
| `status` | The status of the row. See the sync_changelog.status description for a list of possible statuses |
| `failure_reason` | If the status is failed, this contains a string describing why the row failed to sync |
| `fields` | The fields from the model for this row. See the sync_changelog.fields description for more information |
| `split_group` | (Optional) The [split group](/customer-studio/splits) name. If no syncs are using Audience Splits, this column isn't created. |
### Sync runs table
This `hightouch_audit.sync_runs` table stores general metadata information about each sync run. You can join the `sync_changelog` and `sync_snapshot` tables using the `sync_id` column.
| COLUMN | DESCRIPTION |
| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| `sync_id` | The ID of the sync |
| `sync_run_id` | The ID of the sync run |
| `primary_key` | The primary key column of your sync as defined on the model attached to the sync. |
| `destination` | The destination type, for example, Salesforce or Braze |
| `model_name` | The name of the model attached to the sync |
| `model_id` | The ID of the model attached to the sync |
| `status` | The status of the sync run. This will be either succeeded or failed. In general, the per-row results of the sync are a better indication of status. |
| `error` | The sync-level error if the sync terminated early |
| `started_at` | When the sync run started |
| `finished_at` | When the sync run finished |
| `num_planned_add` | The number of planned adds. |
| `num_planned_change` | The number of planned changes. |
| `num_planned_remove` | The number of planned removes. |
| `num_attempted_add` | The number of planned adds that were actually attempted. |
| `num_attempted_change` | The number of planned changes that were actually attempted. |
| `num_attempted_remove` | The number of planned removes that were actually attempted. |
| `num_succeeded_add` | The number of planned adds that were successfully synced to the destination |
| `num_succeeded_change` | The number of planned changes that were successfully synced to the destination |
| `num_succeeded_remove` | The number of planned removes that were successfully synced to the destination |
| `num_failed_add` | The number of planned adds that were attempted, but failed to get synced into destination |
| `num_failed_change` | The number of planned changes that were attempted, but failed to get synced into destination |
| `num_failed_remove` | The number of planned removes that were attempted, but failed to get synced into destination |
## FAQ
### What's the performance impact of enabling Warehouse Sync Logs?
The performance impact of enabling Warehouse Sync Logs is low since it reuses data already present from the Lightning sync engine. Hightouch only writes the rows after Hightouch syncs to the destination, meaning there is no effect on destination throughput.
Pruning entries in the history tables is safe. Doing so doesn't affect future syncs, though the deleted rows won't be rewritten into the history tables.
### What are the limitations in Redshift?
{/* */}
Because [Redshift does not support strings longer than 65,535 bytes](https://docs.aws.amazon.com/redshift/latest/dg/r_Character_types.html#r_Character_types-storage-and-ranges),
it is not possible to store JSON representations of arbitrary length in the warehouse sync logs. Hightouch attempts
to store model data in the `fields` column of `sync_changelog` and `sync_snapshot`, but if the model contains either
long strings or a large number of fields, some fields may be truncated.
Truncated values in `sync_changelog.fields` and `sync_snapshot.fields` do _not_ mean that values were truncated
when Hightouch synced data to the destination; it's purely a limitation of the warehouse logs.
{/* */}
---
## Acoustic
**URL:** https://hightouch.com/docs/destinations/acoustic
**Description:** Market better with all of your up-to-date customer data in Acoustic
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
---------------------|-----------------------------------------------------------|----------------------|------------------------------------------------------------------------------------------------------
Any data set | Sync data from any source to an Acoustic relational table or database | Upsert | [Relational table docs](https://help.goacoustic.com/hc/en-us/articles/360043347953-Relational-tables), [Database docs](https://help.goacoustic.com/hc/en-us/articles/360042858454-Import-a-database)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Acoustic
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Acoustic** and click **Continue**. You can then authenticate Hightouch to **Acoustic** by entering the following required fields into Hightouch:
- **Pod**
- **Client Token**
- **Client Secret**
- **Refresh Token**
You can find your Pod, for example, `6`, in your [Acoustic Account Settings](https://cloud.goacoustic.com/settings).
If you know the region but are unsure of your pod, you can use the table below.
| Campaign URL | Pod |
| ------------ | --- |
|api-campaign-us-1.goacoustic.com| 1|
|api-campaign-us-2.goacoustic.com| 2|
|api-campaign-us-3.goacoustic.com| 3|
|api-campaign-us-4.goacoustic.com| 4|
|api-campaign-us-5.goacoustic.com| 5|
|api-campaign-us-6.goacoustic.com| 6|
|api-campaign-eu-1.goacoustic.com| 7|
|api-campaign-ap-2.goacoustic.com| 8|
|api-campaign-us-6.goacoustic.com| 9|
|api-campaign-ap-1.goacoustic.com| A|
|api-campaign-ap-3.goacoustic.com| B|
|api-campaign-pilot.goacoustic.com| PILOT|
You can generate the other credentials in the **Application Account Access** section of your [Acoustic Organization Settings](https://cloud.goacoustic.com/campaign-automation/Settings/Administration/Organization_settings).
See [these instructions](https://developer.goacoustic.com/acoustic-campaign/reference/getting-started-with-oauth#step-one-defining-your-api-application) for more details.
## Sync configuration
Once you've set up your Acoustic destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Acoustic destination you want to sync to.
### Syncing relational tables or databases
Hightouch requires an existing [database](https://help.goacoustic.com/hc/en-us/articles/360042858454-Import-a-database) or [relational table](https://help.goacoustic.com/hc/en-us/articles/360043347953-Relational-tables) in Acoustic to sync to.
The only supported [sync mode](/syncs/types-and-modes#sync-modes) is **Upsert**.
In Upsert mode, Hightouch inserts new records and keeps [mapped attributes](#field-mapping) up-to-date on existing records in Acoustic.
#### Record matching
You can match rows from your model to rows in Acoustic on any column in your model and any [key](https://help.goacoustic.com/hc/en-us/articles/360042859634-Unique-identifiers) fields in Acoustic.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync model columns via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to any existing columns in your Acoustic database or relational table.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|-----------------------------------------------------------------
**Do nothing** | Keep the record in Acoustic with all its synced fields
**Delete** | Delete the synced records from Acoustic
If you enable deletion, Hightouch provides all columns to Acoustic in the deletion API request.
This enables Hightouch to uniquely identify a single record, rather than a single key column, which could match multiple records.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## ActiveCampaign
**URL:** https://hightouch.com/docs/destinations/active-campaign
**Description:** Enrich customer experiences by using ActiveCampaign's automation platform with rich customer and product data
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
-------------|-------------|---------------------
**Contacts** | Sync data from any source to Contacts in ActiveCampaign | Upsert
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to ActiveCampaign
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **ActiveCampaign** and click **Continue**. You can then authenticate Hightouch to **ActiveCampaign** by entering the following required fields into Hightouch:
- **API URL**
- **API Key**
You can find them in ActiveCampain under **Settings** > **Developer** under **API Access**.
You can find more information about these credentials in the [ActiveCampaign docs](https://help.activecampaign.com/hc/en-us/articles/207317590-Getting-started-with-the-API#what-is-an-api--0-1).
## Sync configuration
Once you've set up your ActiveCampaign destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the ActiveCampaign destination you want to sync to.
### Syncing contacts
This integration only supports the **Upsert** mode. In this mode, Hightouch inserts new contacts into ActiveCampaign and keeps all user attributes up-to-date.
#### Record matching
To match rows from your model to records in ActiveCampaign, you need to select the model column that contains values that match the **ActiveCampaign Email** field.
#### Field mapping
You can map data from any of your model columns to native and custom fields in ActiveCampaign. Ensure the data type of your model column matches the data type of the field you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Adestra
**URL:** https://hightouch.com/docs/destinations/adestra
**Description:** Sync contacts and manage list subscriptions in Adestra (MessageFocus) to power your email marketing campaigns.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes |
| ----------------- | ------------------------------------------------------------ | -------------------- |
| **Contact lists** | Sync data from any source to contact lists in Adestra | Add, Remove |
| **Contacts** | Sync records to contacts in Adestra | Upsert, Update |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Adestra
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Adestra** and click **Continue**. You can then authenticate Hightouch to **Adestra**.
Enter the following fields into Hightouch:
- **API URL**: The base URL of your Adestra API instance.
- **API token**: Your Adestra API token for authenticating requests.
- **Core table ID**: The ID of the core table you want to sync data to.
## Sync configuration
Once you've set up your Adestra destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Adestra destination you want to sync to.
### Syncing contacts
Sync records to contacts in Adestra.
#### Record matching
To match rows from your model to contacts in Adestra, you need to select a model column and corresponding Adestra field. You can match on any of the following Adestra fields:
- **Contact ID**
- **Email**
#### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default and custom contact fields in Adestra.
Ensure your model's columns have the same data types as the fields you want to sync to.
### Syncing contact lists
Sync data from any source to contact lists in Adestra.
#### Record matching
To match rows from your model to contacts in Adestra, you need to select a model column and corresponding Adestra field. You can match on any of the following Adestra fields:
- **Contact ID**
- **Email**
#### Field mapping
Hightouch lets you sync contact list fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default contact list fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Adform
**URL:** https://hightouch.com/docs/destinations/adform
**Description:** Adform empowers your marketing team to analyze conversion events and develop real-time feedback on the effectiveness of your campaigns.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API reference |
| -------------------- | ---------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------- |
| **Conversion Event** | Sync conversion event data to Adform | Insert | [Conversions docs](https://www.adformhelp.com/hc/en-us/articles/9740579489041-Use-Server-Side-Tracking) |
| **Segment** | Sync users from your model to Adform DMP | Add | [Audience base (DMP) docs](https://api.adform.com/help/references/audience-base) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Adform
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations), select **Add destination**, select **Adform**, and then select **Continue**.
Adform supports two sync types with different credentials. Enter the fields for the sync types you plan to use:
- For conversion syncs:
- **Tracking Setup ID**
- **Tracking Domain**
- For segment syncs:
- **Client ID**
- **Client Secret**
You can enter one set of credentials or both if you plan to run both sync types from the same destination.
For segment syncs, ask Adform Support to grant your OAuth client these scopes:
- `https://api.adform.com/scope/dmp.segments`
- `https://api.adform.com/scope/dmp.categories`
- `https://api.adform.com/scope/dmp.authenticated`
- `https://api.adform.com/scope/dmp.dataproviders.readonly`
## Sync configuration
Once you've set up your Adform destination and have a [model](/getting-started/concepts#models) to pull data from, go to the [**Syncs** overview page](https://app.hightouch.com/syncs), select **Add sync**, and then select the relevant model and Adform destination.
For each sync, select **Conversion Event** or **Segment** for **Sync type**.
### Syncing conversion events
Sync conversion event data to Adform.
#### Record matching
You can match rows from your model to conversions in Adform on any column in your model and any field in Adform. Ensure the data types of the model column and Adform field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
### Syncing audience segments
Sync users from your model to Adform DMP segments.
You can sync to an existing segment or create a new one on the first sync.
Adform segment syncs are add-only.
If records leave your model, Hightouch doesn't remove them from the segment.
#### Select a data provider
Select the Adform data provider that owns the segments you want to sync.
Hightouch fetches the available data providers using the destination's **Client ID** and **Client Secret**.
#### Select an existing segment or create a new one
You can sync to an existing segment or create a new segment in Hightouch.
If you create a new segment, you also need to configure these fields:
- **Category**: Every Adform segment must belong to one category. You can select an existing category or create a new one on the first sync.
- **Data consumer ID**: Optional. Enter an agency ID or publisher ID if another Adform entity should have access to the segment.
- **TTL (days)**: Optional. Controls how long a user remains in the segment after their last upload. The default is `30`.
- **Frequency**: Optional. Controls the minimum number of occurrences required for a user to qualify for the segment. The default is `1`.
#### Select a user identity type
Select the Adform identity type for the user IDs you upload.
Depending on your Adform setup, supported identifier types can include:
- First-party IDs
- Mobile advertising IDs (MAIDs)
- Adform cookies
- ID5 identifiers
After you select an identity type, map the column from your model that contains that identifier to **User ID**.
#### Batch size
You can optionally set a batch size for segment uploads.
The default is `10000`, and Adform limits each upload file to 100 MB.
## Tips and troubleshooting
### Common errors
#### Adform DMP token request failed with invalid_scope
This error means the OAuth client configured on your Adform destination is missing one or more required DMP scopes.
Ask Adform Support to grant these scopes to your OAuth client:
- `https://api.adform.com/scope/dmp.segments`
- `https://api.adform.com/scope/dmp.categories`
- `https://api.adform.com/scope/dmp.authenticated`
- `https://api.adform.com/scope/dmp.dataproviders.readonly`
After Adform updates the client, test the destination connection again.
### Live debugger
### Sync alerts
---
## Adikteev
**URL:** https://hightouch.com/docs/destinations/adikteev
**Description:** Upload mobile advertising identifiers to Adikteev
**Section:** Destinations
## Overview
With Hightouch and the Adikteev API, you can send data from your warehouse to audiences in Adikteev.
Deliver relevant ads by utilizing data from various sources within your data warehouse.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------ | ----------------------------------------------------------- | -------------------- |
| **Audience** | Create audience lists and keep them up-to-date in Adikteev. | All (file upload) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Adikteev
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Adikteev** and click **Continue**. You can then authenticate Hightouch to Adikteev via username and password.
Enter your Adikteev credentials and hit **Test Connection** to verify credentials.
If successful, click **Continue**. To finish connecting, give your Adikteev destination a descriptive name.
## Sync configuration
Once you've connected your Adikteev destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Adikteev destination you want to sync to.
### Syncing custom audiences
Hightouch lets you create and maintain custom audiences via the [Custom Audiences API](https://help.adikteev.com/hc/en-us/articles/360003819660-Custom-Audiences-API).
#### User identifiers
To identify which users to add or update in an audience, select the model column that represents the user's device ID.
Adikteev allows two different device ID types for iOS devices: IDFA and IDFV.
Be sure to select the correct option in the ID type dropdown.
#### Custom Audiences API mode
Adikteev lets you upload an entire audience through their [Custom Audiences API](https://help.adikteev.com/hc/en-us/articles/360003819660-Custom-Audiences-API#SendingCustomAudiencesToAdikteev-Step3-Uploading/UpdatingadeviceIDlistforasegment). Hightouch uploads the entire model query results for each sync run.
These results effectively replaces the previous upload.
#### Custom audience list
Hightouch lets you use an existing audience in Adikteev or create a new one. Note that if you create a new audience, you can't
change the name once your sync has run. Instead, you have to create a new sync.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Adjust
**URL:** https://hightouch.com/docs/destinations/adjust
**Description:** Streamline your app growth with Adjust’s mobile marketing platform.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
------------|--------------------------------------------------------------------------------|---------------------
**S2S Events** | Sync records as events to Adjust. This is often in the form of a track call. | Insert
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Adjust
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination**
button. Select **Adjust** and click **Continue**. You can then authenticate Hightouch to **Adjust** by entering the event token and app token:
If you do not have an event or app token, use the following steps from [Adjust](https://help.adjust.com/en/article/server-to-server-s2s-security#s2s-authentication) to set up S2S authentication and generate them:
1. Under **AppView**, select **All apps**.
2. Select your app.
3. Select the **Protection** tab.
4. Under the **S2S Security** section, select **Open S2S Security**.
5. Select **Add token**.
6. Enter a token name, and select **Add token**.
You must also activate S2S Authentication with the following steps in [Adjust](https://help.adjust.com/en/article/server-to-server-s2s-security#activate-s2s):
1. Under **AppView**, select **All apps**.
2. Select your app.
3. Select the **Protection** tab.
4. Under the **S2S Security** section, select **Open S2S Security**.
5. Ensure you have at least one token configured.
6. Switch the toggle **ON**. A warning message appears to inform you that only requests sent with configured tokens will work.
7. Select **Turn on** to proceed.
Once you've entered the event and app, click **Continue**. Give your destination a descriptive name and **Finish** the destination connection setup.
## Sync configuration
Once you've set up your Adjust destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and
the Adjust destination you want to sync to.
#### Record matching
Adjust requires all S2S events to have at least one device identifier. The device identifiers accepted for S2S events are as follows:
- Raw IDFA (IOS only) (`idfa`)
- Raw Google advertising ID (`gps_adid`)
- Raw Amazon Fire advertising ID (`fire_adid`)
- Open Advertising ID (OAID, Huawei only) (`oaid`)
- Raw IDFV (`idfv`)
- Android ID (`android_id`)
#### Field mapping
Adjust recommends including the following parameters with events:
- Adjust ID (`adid`)
- IP address (`ip_address`)
- Timestamp (`created_at` or `created_at_unix`)
- User agent (`user_agent`)
If you do not include the event timestamp, Adjust will use the time the event was received.
Hightouch accepts a normal date time format for the timestamp, and
automatically converts it to the format required by Adjust.
Adjust does not accept events more than 58 days old.
#### Custom Data
Hightouch allows mapping fields as `callback_params` or `partner_params` to send with Adjust events.
**Callback parameters** are available in your raw data exports and **partner parameters** are shared with your network partners.
In Hightouch, type in the name of the field you would like included in these parameters, and choose a column to sync to it.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Adobe Analytics
**URL:** https://hightouch.com/docs/destinations/adobe-analytics
**Description:** Sync your analytics data from your warehouse to Adobe Analytics
**Section:** Destinations
## Overview
Adobe Analytics is a web analytics service that provides tools to understand customer behavior, measure marketing campaigns, and optimize the user experience. With Hightouch, you can send data from your data warehouse to Adobe Analytics to enrich your analytics with valuable customer data from various sources. This helps you build a more complete view of the customer journey and improve your marketing and product strategies.
This destination uses the [Adobe Analytics Batch Data Insertion (BDIA) API](https://developer.adobe.com/analytics-apis/docs/2.0/guides/endpoints/bulk-data-insertion/), which allows you to upload large amounts of data in batches.
## Supported syncing
| Type | Description | Supported Sync Modes |
| ---------- | --------------------------------------------------------- | -------------------- |
| **Events** | Sync event data to Adobe Analytics for detailed analysis. | Insert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Adobe Analytics
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Adobe Analytics** and click **Continue**. You can then authenticate Hightouch to **Adobe Analytics** by entering the following required fields:
- **API Key (Client ID)**
- **Client Secret**
These credentials are not your Adobe Analytics login details. They are obtained by creating an OAuth app in the Adobe Developer Console.
For more information on how to create an OAuth app and obtain these credentials, see [Adobe's documentation](https://developer.adobe.com/developer-console/docs/guides/authentication/ServerToServerAuthentication/implementation#setting-up-the-oauth-server-to-server-credential)
## Sync configuration
Once you've set up your **Adobe Analytics** destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Adobe Analytics destination you want to sync to.
This destination allows you to sync event data to Adobe Analytics. Each row in your model should represent a single event or interaction.
### Visitor identification
You can match rows from your model to visitors in Adobe Analytics using one of the following identifiers. Ensure the data types of the model column and the Adobe Analytics field you select match.
- **Visitor ID**: A unique identifier for the visitor.
- **Marketing Cloud Visitor ID**: The Adobe Marketing Cloud visitor identifier.
- **IP Address**: The visitor's IP address.
- **Customer ID**: Your internal customer identifier.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Adobe Analytics automatically generates an identifier for
every new record synced, so there is no need to match an existing record if
you are only sending new events. However, providing a stable visitor
identifier is crucial for tying events to the correct user profiles.
### Field mapping
Hightouch lets you sync event fields via [field mapping](/syncs/mapping-data). You can map data from any of your model columns to standard and custom Adobe Analytics fields. The following fields are required:
- **Report Suite ID**: Your Adobe Analytics report suite identifier.
- **Timestamp**: The time of the interaction, in Unix timestamp or ISO-8601 format.
- **User Agent**: The device's user agent string.
For a full list of available fields, refer to the [Adobe Analytics BDIA API documentation](https://developer.adobe.com/analytics-apis/docs/2.0/guides/endpoints/bulk-data-insertion/file-format/#required-columns).
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Adobe Campaign Classic
**URL:** https://hightouch.com/docs/destinations/adobe-campaign-classic
**Description:** Adobe Campaign pulls together cross-channel customer data into a single view and then puts it to work to create personalized cross-channel campaigns that meet customers where they’re at.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|----------------|--------------------------------------------------------|------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Recipients** | Sync data from any source to Adobe Campaign recipients | Upsert, Update, Insert | [Recipients docs](https://experienceleague.adobe.com/en/docs/campaign-classic/using/configuring-campaign-classic/api/data-oriented-apis#write---writecollection--xtk-session-) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Adobe Campaign Classic
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Adobe Campaign Classic** and click **Continue**. You can then authenticate Hightouch to **Adobe Campaign Classic**.
Enter the following fields into Hightouch:
- **Server URL**: Enter your server URL.
- **Client ID**: Enter your client ID.
- **Client secret**: Enter your client secret.
## Sync configuration
Once you've set up your Adobe Campaign Classic destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Adobe Campaign Classic destination you want to sync to.
### Syncing recipients
Sync data from any source to Adobe Campaign recipients.
#### Record matching
You can match rows from your model to recipients in Adobe Campaign Classic on any column in your model and any field in Adobe Campaign Classic. Ensure the data types of the model column and Adobe Campaign Classic field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Adobe Campaign Classic automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync recipient fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to any recipient attribute.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
|----------------|-------------------------------------------------------------------------------|
| **Do nothing** | Keep the recipient in Adobe Campaign Classic with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the recipient in Adobe Campaign Classic |
| **Delete** | Delete the synced recipients from Adobe Campaign Classic |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Adobe Experience Platform
**URL:** https://hightouch.com/docs/destinations/adobe-experience-platform
**Description:** Deliver personalized experiences at scale requires a centralized and connected data foundation with the Adobe Experience Platform.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| -------------------------------------------- | -------------------------------------------- | -------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| **Datasets** | Sync data from any source to dataset records | Upsert, Insert | [Datasets docs](https://developer.adobe.com/experience-platform-apis/references/batch-ingestion/#operation/createBatch) |
| **Adobe Journey Optimizer campaign trigger** | Trigger campaigns in Adobe Journey Optimizer | -- | [Campaign trigger docs](https://experienceleague.adobe.com/en/docs/journey-optimizer/using/campaigns/api-triggered-campaigns/api-triggered-campaigns) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Adobe Experience Platform
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Adobe Experience Platform** and click **Continue**. You can then authenticate Hightouch to **Adobe Experience Platform**.
Enter the following fields into Hightouch:
- **Organization ID**
- **Sandbox name**
- **Client ID**
- **Client secret**
For more information on how to obtain these values, refer to [Adobe's docs](https://experienceleague.adobe.com/en/docs/experience-platform/landing/platform-apis/api-authentication).
## Sync configuration
Once you've set up your Adobe Experience Platform destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Adobe Experience Platform destination you want to sync to.
### Syncing datasets
Sync data from any source to dataset records. Upsert mode is only available if your dataset has unified profile and upsert enabled. Refer to [Adobe's docs](https://experienceleague.adobe.com/en/docs/experience-platform/catalog/datasets/enable-upsert) to enable these features.
#### Record matching
You can match rows from your model to datasets in Adobe Experience Platform on any column in your model and any field in Adobe Experience Platform. Ensure the data types of the model column and Adobe Experience Platform field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Adobe Experience Platform automatically generates an
identifier for every new record synced, so there is no need to match an
existing record.
#### Field mapping
Hightouch lets you sync dataset fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the fields defined in the schema of your dataset.
Ensure your model's columns have the same data types as the fields you want to sync to.
### Trigger campaigns via Adobe Journey Optimizer
Trigger push, sms, or emails to your Adobe Journey Optimizer campaigns.
#### Campaign ID
Enter the ID of the campaign you want to trigger. Each sync can trigger sends to a single campaign.
#### Field mapping
You must specify which AEP profile namespace (e.g. `email`) you want to use to map your recipients in AEP and map the
`userId` to your source column to identify that namespace. You may also include other user attributes in the `profile` field.
You can add dynamic content to your campaign through the `context` field.
The `context` field is an object of property-value pairs (e.g. `{ "subject": "Subject line" }`).
To use dynamic content in your campaign properly, you need to set up the
template to reference the context object in Adobe Journey Optimizer. Refer to
this
[tutorial](https://experienceleague.adobe.com/en/docs/journey-optimizer-learn/tutorials/create-campaigns/api-triggered-campaigns)
on how to do so.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Adobe Target
**URL:** https://hightouch.com/docs/destinations/adobe-target
**Description:** Sync customer segments to Adobe Target
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|--------------|----------------------------------------------|----------------------|-----------------------------------------------------------------------|
| **Profiles** | Create and update visitor profile attributes | Upsert | [Profiles](https://experienceleague.adobe.com/en/docs/target-dev/developer/api/profile-apis/profile-bulk-api) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Getting started
To generate the required authentication credentials to connect Adobe Target to Hightouch, follow these instructions:
1. Go to your [Adobe admin console](https://developer.adobe.com/console/home).
2. Select **Create new project** > **Add API** > **Adobe Target** and click **Next**.
3. Select the **OAuth Server-to-Server** authentication method.
4. Once the authentication method has been created, select the credentials.
5. Click **Retrieve client secret**.
## Connect to Adobe Target
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button.
Select **Adobe Target** and click **Continue**.
You can then authenticate Hightouch to **Adobe Target** by entering the following required fields into Hightouch:
- **Client code**: Your client code can be found in the [Adobe Experience Cloud](https://experience.adobe.com/) under **Target** > **Administration** > **Implementation**.
- **Tenant name**: Tenant name can be found in the URL when you log into Experience Cloud (`https://experience.adobe.com/#/@/target/setup/implementation`).
- **Client ID**: This is created along with the client secret from the steps above
- **Client secret**
Make sure to access the **Profile API** slide in your Adobe Target settings and enable the **Require Authentication** toggle.
Then click **Generate New Profile Authentication Token**.
You don't need to use the profile authentication token when setting up an Adobe Target destination in Hightouch, but generating a new token is required by Adobe.
You can find more details about this in [Adobe Target's documentation](https://experienceleague.adobe.com/docs/target-dev/developer/implementation/methods/profile-api-settings.html).
## Sync configuration
Once you've set up your Adobe Target destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Adobe Target destination you want to sync to.
### Syncing profiles
Syncing to Adobe Target's [Bulk Profile API](https://experienceleague.adobe.com/en/docs/target-dev/developer/api/profile-apis/profile-bulk-api) lets you manage visitor profile attributes with data typically stored in cookies or third-party apps.
#### Syncing to audiences
Audiences in Adobe Target are condition-based, so syncing profiles to audiences is handled by setting a `__hightouch_audience_{model name}` attribute to match on. You can only choose to create a new audience, and the condition of the audience will be set to include all user profiles whose `__hightouch_audience_{model name}` attribute is `true`.
#### Record matching
To match rows from your model to profiles in Adobe Target, you need to select a model column and corresponding Adobe Target field. You can match on any of the following Adobe Target fields:
- thirdPartyId (recommended)
- pcID
#### Field mapping
You can map data from any of your model columns to fields in Adobe Target. Ensure the data type of your model column matches the data type of the field you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the profile in Target |
| **Clear** | Keep the profile in Target and clear fields managed by Hightouch |
## Tips and troubleshooting
#### Non-unique IDs
Profiles can be matched from your warehouse to your Adobe Target destination using the pcID or a third party ID. If you selected a model column that isn't unique, the incorrect profile may be updated.
### Common errors
#### Failed to authenticate to Adobe Target: 400 - Invalid request: authentication is disabled for current client
As explained in the [Connect to Adobe Target](#connect-to-adobe-target) section, make sure to access the **Profile API** slide in your Adobe Target settings and enable the **Require Authentication** toggle.
Then click **Generate New Profile Authentication Token**.
You don't need to use the profile authentication token when setting up an Adobe Target destination in Hightouch, but generating a new token is required by Adobe.
You can find more details about this in [Adobe Target's documentation](https://experienceleague.adobe.com/docs/target-dev/developer/implementation/methods/profile-api-settings.html).
### Live debugger
### Sync alerts
---
## Agillic
**URL:** https://hightouch.com/docs/destinations/agillic
**Description:** Agillic is the trusted Nordic marketing automation platform, enabling personalisation that performs with scalability, operational efficiency, and full GDPR compliance.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
----------------|------------------------------------------------------------------|----------------------|---------------------------------------------------------------------------------------------
**Recipients** | Sync data from any source to Agillic recipients | Upsert | [Recipients docs](https://developers.agillic.com/getting-started-with-apis/recipients-api/)
**One-to-Many** | Sync data from any source to Agillic one-to-many tables | Upsert | [One-to-Many docs](https://developers.agillic.com/getting-started-with-apis/recipients-api/)
**Events** | Sync data from any source to achieve recipient events in Agillic | Insert | [Events docs](https://developers.agillic.com/getting-started-with-apis/recipients-api/)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Agillic
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Agillic** and click **Continue**. You can then authenticate Hightouch to **Agillic**.
Enter the following fields into Hightouch:
- **Client ID**
- **Client secret**
## Sync configuration
Once you've set up your Agillic destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Agillic destination you want to sync to.
### Syncing recipients
Sync data from any source to Agillic recipients.
#### Record matching
To match rows from your model to recipients in Agillic, you need to select the model column that contains values that match any unique recipient field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync recipient fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default recipient fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|---------------------------------------------------------
**Do nothing** | Keep the recipient in Agillic with all its synced fields
**Delete** | Delete the synced recipients from Agillic
### Syncing one-to-many
Sync data from any source to Agillic one-to-many tables.
#### Record matching
To match rows from your model to one-to-many records in Agillic, you need to select a model column that contains a unique recipient field
and a unique field for the one-to-many table you are syncing to.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync one-to-many fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default one-to-many fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
### Syncing events
Sync data from any source to achieve recipient events in Agillic.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Airship
**URL:** https://hightouch.com/docs/destinations/airship
**Description:** Build better campaigns on Airship with up-to-date customer data from your data warehouse
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ---------------------- | ------------------------------------------------------------- | -------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| **Named users** | Sync tags and attributes to named users in Airship | Upsert, Update | [Named users docs](https://docs.airship.com/api/ua/?openapi=http#operation-api-named_users-named_user_id-post) |
| **Subscription lists** | Subscribe and unsubscribe named users from subscription lists | Add, Remove | [Subscription lists docs](https://docs.airship.com/api/ua/?openapi=http#operation-api-named_users-scoped-named_user_id-post) |
| **Push** | Send push notifications and rich messages across channels | Insert | [Push endpoint docs](https://docs.airship.com/developer/rest-api/ua/operations/push/#sendpush) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Airship
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Airship** and click **Continue**.
You can then authenticate Hightouch to Airship using either **basic authentication** or **bearer authentication**.
For both types, you need to enter your Airship **Cloud Site**. See Airship's [docs](https://docs.airship.com/api/ua/?openapi=http#servers) for more info.
Then, for basic authentication, enter these fields:
- **App Key**: This is an Airship-generated string identifying your application. You can find it in your Airship dashboard.
- **Master Secret**: Airship-generated string used for server-to-server API access. You can find it in your Airship dashboard.
Follow the instructions in this [article](https://support.airship.com/hc/en-us/articles/213491663-How-do-I-find-my-App-Key-App-Secret-or-Master-Secret-#:~:text=Open%20the%20app%20in%20the,right%20side%20of%20the%20screen.)
to retrieve your **App Key** and **Master Secret**
For bearer authentication:
- **Access Token**: This is an access token used to grant control permissions to the Airship API.
Follow the instructions in this [article](https://docs.airship.com/guides/messaging/user-guide/project/bearer-tokens/) to retrieve your **Access token**.
## Sync configuration
Once you've set up your Airship destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Airship destination you want to sync to.
### Syncing named users
Sync tags and attributes to named users in Airship. Begin by selecting **Object** as your sync type.
#### Record matching
To match rows from your model to named users in Airship, you need to select the model column that contains values that match the **Named User ID** field.
#### Field mapping
You can sync columns from your model to Airship attributes and tags.
Hightouch provides a dropdown for [default attributes](https://docs.airship.com/guides/messaging/user-guide/audience/segmentation/attributes/reference/#default-attributes) and [predefined attributes](https://docs.airship.com/guides/messaging/user-guide/audience/segmentation/attributes/reference/#predefined-attributes).
If you want to map custom attributes, you have to manually enter the [attribute ID](https://docs.airship.com/guides/messaging/user-guide/audience/segmentation/attributes/about/#attribute-components-and-variations), _not attribute name_ in the mapper.
Although [predefined
attributes](https://docs.airship.com/guides/messaging/user-guide/audience/segmentation/attributes/reference/#predefined-attributes)
are avaliable in the dropdown by default, you must first define them in Airship
for them to be valid. Airship skips invalid attributes by default.
Hightouch allows you to sync values to existing tag groups in Airship. Enter the **Group key** in the right side of the tags mappings section to map values to an existing tag group.
See Airship's [docs](https://docs.airship.com/guides/messaging/user-guide/audience/segmentation/tags/#creating-custom-tag-groups) for more information on tag groups.
{/* */}
You can map `string` or `array` values to tags.
If you provide a string, Hightouch automatically transforms it into an `array` using commas as the delimiters for different elements.
For example, Hightouch transforms the string `"dogs, cats, parrots"` into this array: `["dogs, "cats", "parrots"]`.
{/* */}
### Syncing subscription lists
Syncs can subscribe and unsubscribe named users from subscription lists.
#### Record matching
To match rows from your model to named users in Airship, you need to select the model column that contains values that match the **Named User ID** field.
#### Field mapping
You can map data from any of your model columns to fields in Airship. Airship requires the **Scopes** field, so you must map it to complete your configuration. Refer to Airship's [scopes docs](https://docs.airship.com/api/ua/?openapi=http#schemas-scopes) to see valid values.
Ensure the data types of your model columns match the data types of the fields you want to sync to.
### Sending push notifications
Send push notifications and rich messages to your users across multiple channels including iOS, Android, email, and SMS.
#### Field mapping
Configure your push notifications by mapping model columns to the following fields:
- **Audience**: Specify recipients using channels, tags, groups, or other audience selectors. This field is required.
- **Notification**: The message content to be sent. This field is required.
- **Device Types**: Target platform(s) for the notification (e.g., iOS, Android, email). This field is required.
- **Expiry** (optional): Expiration time for time-sensitive messages to eliminate the need for manual removal.
For detailed information about payload structure and additional options, refer to Airship's [Push API documentation](https://docs.airship.com/developer/rest-api/ua/operations/push/#sendpush).
## Tip and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Airtable
**URL:** https://hightouch.com/docs/destinations/airtable
**Description:** Sync your database or data warehouse to your Airtable bases for DataOps, CRM, and much more
**Section:** Destinations
## Supported syncing
|Sync Type |Description |Supported Sync Modes |
|--------------|----------------------------------------------------------------------------------------------|---------------------|
|Objects | Insert new objects without updating others or push new objects and update fields that change |Insert, Upsert |
|Object fields | Update particular fields on existing objects without adding new objects |Update |
## Connect to Airtable
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Airtable** and click **Continue**. You can then authenticate Hightouch to Airtable using an Airtable personal access token.
### Create Airtable personal access token
Navigate to your [Airtable Personal access tokens page](https://airtable.com/create/tokens) and click **Create token**.
Give your token a descriptive name, for example, "Hightouch Integration Key," and grant your token the following scopes:
- `data.records:read`
- `data.records:write`
- `schema.bases:read`
- `schema.bases:write`
Grant your token access to the bases and workspaces you need Hightouch to access, and then copy your token to use it in Hightouch.
## Sync configuration
Once you've set up your Airtable destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Airtable destination you want to sync to.
Choose the **Airtable base** and **table** you want to sync to using the two drop-down lists.
Then select the [mode](/syncs/types-and-modes) for how to alter Airtable objects:
- **Insert**: pushes new objects to Airtable. It doesn't update any objects.
- **Upsert**: pushes new objects to Airtable and updates fields that change in your warehouse.
- **Update**: updates particular fields on existing objects in Airtable. It doesn't add new objects.
### Record matching
If you are using **Upsert** or **Update** mode, you must select a record matching field.
You can match records from your source to your Airtable workspace by the table's primary key or any other unique fields in the table.
You can choose the field you would like to use from the dropdown.
You should match the record using a unique identifier with a `string` type, otherwise the sync may not update the intended records properly.
### Field mapping
You can sync columns from your source to any of your Airtable properties. The Airtable API performs best-effort
automatic data conversion from string values. You can choose the field names from the drop-down lists.
When syncing data to [linked](https://support.airtable.com/docs/linking-records-in-airtable) or [lookup](https://support.airtable.com/docs/lookup-field-overview) Airtable fields, make sure to set your model column to [`string`](/models/data-types-casting#data-types) type. Otherwise, the values synced to these fields won't appear in Airtable.
## Tips and troubleshooting
### Common errors
#### Duplicated records
Duplicated records in Airtable may occur if a data type mismatch between the [model column](/syncs/record-matching#model-column-selection) and the Airtable field selected for [record matching](#record-matching) occurs.
Hightouch [automatically stringifies](/models/data-types-casting#view-model-data-types) the model column you select for your [primary key](/getting-started/concepts#unique-primary-key-requirement). If you choose your model's primary key for record matching, you need to select a `string` type Airtable field for record matching as well. If you select a `number` type Airtable field, duplicate records could occur because, for example, `"1234"` doesn't match `1234`.
If you want to use the primary key column for record matching, but need to match it to a `number` type Airtable field, you can use [SQL aliasing](/models/data-types-casting#sql-aliasing) to create a new column. The new column is a "copy" of the primary key column, but because Hightouch won't automatically stringify it, you can cast it to [`number` type](/models/data-types-casting#data-types) and use it for record matching.
### Live debugger
### Sync alerts
---
## Algolia
**URL:** https://hightouch.com/docs/destinations/algolia
**Description:** Build, power, and optimize digital experiences
**Section:** Destinations
## Overview
You want to provide relevant search and discovery results and doing so heavily relies on data in your Algolia account to be accurate and fresh. By automatically indexing records and events from your data warehouse into Algolia, you no longer need to worry about data consistency because data is no longer flowing in from multiple sources in different formats, and you can focus on building world-class customer experiences.
## Supported syncing
Type | Description | Supported Sync Modes
---------------------|----------------------------------------------------------|-----------------------
**Index records** | Sync data from any source to indices in Algolia | Upsert, Update, Insert
**Insight events** | Sync data from any source to insight events in Algolia | Insert
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Algolia
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination**
button. Select **Algolia** and click **Continue**. You can then authenticate Hightouch to **Algolia** with an API key.
Enter the following required fields into Hightouch:
- **Application ID**
- **API Key**
Follow [these instructions](https://www.algolia.com/doc/guides/security/api-keys/#creating-and-managing-api-keys) to create your API key and provide at least these [permissions](https://www.algolia.com/doc/guides/security/api-keys/#rights-and-restrictions):
Permissions | Required?
---------------------|------------
**listIndexes** | YES
**addObject** | YES
**deleteObject** | If you choose to [delete records](#delete-behavior)
## Sync configuration
Once you've set up your Algolia destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and
the Algolia destination you want to sync to.
### Indices
An [Algolia index](https://support.algolia.com/hc/en-us/articles/4406981910289-What-is-an-index-) is the place where the data used by a search engine is stored. You can use Hightouch to update individual records in an index.
#### Record matching
You must match rows in your model with objects in Algolia on the [**Object ID** field](https://www.algolia.com/doc/guides/sending-and-managing-data/send-and-update-your-data/#using-unique-object-identifiers).
In **Insert** mode, Algolia automatically generates a **Object ID** for every new record synced, so there is no need to match an existing record.
#### Field mapping
Records in Algolia index don't need to follow a schema so you can sync any custom fields to them.
Visit Algolia's docs for more information about [Algolia records](https://www.algolia.com/doc/guides/sending-and-managing-data/prepare-your-data/#algolia-records).
#### Delete behavior
**Clear** is only available for update mode. **Delete** is only available for upsert mode. All sync modes use the **Do nothing** behavior by default.
Behavior | Description
-------------- | -------------------------------------------------------------------------------------------------
**Do nothing** | Keep the customer record in Algolia with all its synced fields
**Clear** | Unset all the mapped fields from Algolia
**Delete** | Delete the synced record from your Algolia index. Your [API key needs the **deleteObject** ACL](#connect-to-algolia) to enable this
### Insight events
Insights events lets you capture click, conversion, and view events to help you understand how your users interact with your digital experience.
#### Field mapping
When syncing insight events to Algolia, you must provide the following fields:
- **Event Name**: Name of the event. Typically the action performed.
- **Event Type**: Type of the event. The allowed values are `"click"`, `"conversion"`, or `"view"`.
- **Index**: Name of the targeted index.
- **User Token**: A user identifier. Depending if the user is logged-in or not, several strategies can be used from a `sessionId` to another technical identifier. You should always send pseudonymous or anonymous userTokens.
YOu can also provide these optional fields:
- **Timestamp**: Time of the event expressed in milliseconds since the [Unix epoch](https://en.wikipedia.org/wiki/Unix_time).
- **Query ID**: Algolia `queryID`. This is required when an event is tied to a search.
- **Object IDs**: An array of index `objectIDs`. Limited to 20 objects. An event can't have both `objectIDs` and filters at the same time.
- **Filters**: An array of filters. Limited to 10 filters. An event can't have both `objectIDs` and filters at the same time.
- **Positions**: Position of the click in the list of Algolia search results. This field is required if a `queryID` is provided. One position must be provided for each `objectID`.
When an event is tied to an Algolia search, it must also provide a queryID. If that event is a click, their absolute positions should also be passed. Consult [Algolia's documentation](https://www.algolia.com/doc/api-reference/api-methods/send-events/#parameters) for more information on event fields.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## AlloyDB
**URL:** https://hightouch.com/docs/destinations/alloydb
**Description:** AlloyDB combines the best of Google with one of the most popular open-source database engines, PostgreSQL, for superior performance, scale, and availability
**Section:** Destinations
## Setup
### Networking
AlloyDB supports network connectivity through private, internal IP addresses configured for private services access.
To connect to AlloyDB from Hightouch, configure an SSH connection tunnel to go through a bastion host in your VPC.
Read about more options here: [https://cloud.google.com/alloydb/docs/connection-overview](https://cloud.google.com/alloydb/docs/connection-overview)
### SSH tunneling
Hightouch can connect directly to AlloyDB over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your AlloyDB instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
### Required permissions
The user credentials must have one of the following IAM roles in the Cloud project you are using. If you don't have any of these roles, contact your Organization Administrator to request access.
- `roles/alloydb.admin` (the AlloyDB Admin predefined IAM role)
- `roles/owner` (the Owner basic IAM role)
- `roles/editor` (the Editor basic IAM role)
To create private services access configuration, you also must have these IAM permissions:
- `compute.networks.list`
- `compute.addresses.create`
- `compute.addresses.list`
- `servicenetworking.services.addPeering`
To ensure your credentials are correct, click **Test connection**. This confirms
if Hightouch is able to properly connect to your database by running a simple `SELECT` query.
## Syncing
Hightouch supports **Upsert** mode, with the option to delete removed rows, and **Insert** mode
using the `COPY FROM` statement.
### Record matching
Hightouch requires a unique identifier in the table you are syncing to in order to add, remove, and update rows.
The main identifier column in your sync configuration must match on a `UNIQUE` or `PRIMARY KEY` column within your database.
You should see those columns as available options under the records matching section. If there are no fields in the dropdown,
you need to add a unique column type to your table.
### Column types
Hightouch works out of the box with all standard column types, including:
- `BIGINT`
- `INT`
- `TEXT`
- `VARCHAR`
- `TIMESTAMPTZ`
- `BOOLEAN`
- `DECIMAL`
If you see type errors, it may be because your SQL query is producing the wrong format.
Hightouch doesn't support arrays of `USER-DEFINED` types that are JSON
objects. We do support arrays of regular JSON objects (JSON or JSONB).
right now.
### Batch size
You can tune the number of rows that Hightouch upserts or inserts per query based on your needs and database threshold.
The default is 1k rows per batch.
If you want to improve the sync's speed performance, you can increase
the batch size to a higher number like 10k. However, keep in mind that AlloyDB will fail the entire batch
of rows if it detects any erroneous row. If you suspect that you will have many bad rows, you should
not have a high batch size. Also make sure you account for your database's capacity when increasing the
batch size to avoid any locks.
### Insert mode
Hightouch supports insert mode through the `COPY FROM STDIN` AlloyDB statement. This will load the
queried rows into your table. However, AlloyDB will throw an error and reject the entire batch if any row
already exists or the row contains a primary key or unique value that already exists.
The [`COPY FROM`](https://www.postgresql.org/docs/current/sql-copy.html) statement should be faster than the regular statement we use for Upsert mode so we recommend
selecting Insert mode if you are only inserting rows and want to sync your data faster.
The `COPY FROM STDIN` statement should support the same column types as Upsert
mode, but the array of JSONB type (`jsonb[]`) column. Instead, you will need
to use a JSONB column type, which supports an array format.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Amazon Ads DSP and AMC
**URL:** https://hightouch.com/docs/destinations/amazon-ads
**Description:** Ensure that you never serve another irrelevant advertisement with Amazon Ads
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ------------- | ---------------------------------------------------------------------------------------------- | -------------------- | -------------------------------------------------------------------------------------------- |
| **Events** | Sync data from any source to Amazon Ads DSP as offline conversion events | Insert | [Events docs](https://advertising.amazon.com/API/docs/en-us/dsp-conversion-builder) |
| **Audiences** | Sync first-party data and third-party data from any source to Amazon Ads DSP and AMC audiences | Add | [Audiences docs](https://advertising.amazon.com/API/docs/en-us/dsp-audiences/#tag/Audiences) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Amazon Ads
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Amazon Ads** and click **Continue**. You can then authenticate Hightouch to **Amazon Ads**.
For the **Authentication method**, select **Log in to Amazon Ads** and log into your Amazon Ads account. Once successful, you will be redirected back to Hightouch to enter an advertiser ID and a descriptive name for your destination to complete setup.
## Sync configuration
Once you've set up your Amazon Ads destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Amazon Ads destination you want to sync to.
### Syncing events
Sync data from any source to Amazon Ads as offline conversion events.
#### Create a conversion definition in Amazon Ads
Before syncing events, you need to create a conversion definition in Amazon Ads.
See the [Amazon Ads API documentation](https://advertising.amazon.com/API/docs/en-us/dsp-conversion-builder) for more information.
You can create a conversion definition using the Amazon Ads UI or API.
To create a conversion definition using the Amazon Ads API, use the endpoint documented [here](https://advertising.amazon.com/API/docs/en-us/dsp-conversion-builder#tag/Amazon-Conversion-Definitions/operation/dspAmazonCreateConversionDefinitions).
To create a conversion definition using the Amazon Ads UI, follow these steps:
1. [Sign in](https://advertising.amazon.com/en-us/sign-in) to Amazon Ads.
1. Navigate to **Amazon DSP > Campaign manager**.
1. Navigate to **Advertisers** and select an advertiser.
1. Navigate to the **Events manager** for that advertiser.
1. Go to the **Conversions** tab and click the `Add conversion` button.
After creating a conversion definition, you can select it in the sync configuration.
#### Record matching
For events, you can use any of the following fields as identifiers in Amazon Ads:
- **Email**
- **Phone**
- **City**
- **State**
- **Address**
- **Postal**
- **MAID**
- **First name**
- **Last name**
Please see the [Amazon Ads API documentation](https://advertising.amazon.com/API/docs/en-us/dsp-conversion-builder) for more information on the required fields for event syncing.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
### Syncing audiences
Sync data from any source to Amazon Ads audiences.
#### Audience type
You can sync to first-party data or third-party data based on the type of audience.
- **First-party data audience**: Sync hashed user data to audiences
- **Third-party data audience**: Sync external identifiers such as MAIDs and cookie IDs
#### Record matching
To match rows from your model to audience in Amazon Ads, you need to select a model column and corresponding Amazon Ads field. For third-party data audiences, you can match on any of the following fields:
- **Cookie ID**
- **Mobile Advertising ID**
- **External ID**
For first-party data audiences, you can match on any of the following fields:
- **Email**
- **Phone**
- **City**
- **State**
- **First name**
- **Last name**
- **Address**
- **External ID**
#### Handling PII and hashing
By default, Hightouch automatically hashes first-party user data fields before sending requests to Amazon Ads.
You can turn off this behavior in the sync configuration. When off, the data from the model should be appropriately normalized and hashed according to Amazon Ads' hashing requirements.
## Tips and troubleshooting
### Matched users count
Below only applies to the audiences sync type.
### Common errors
#### Invalid conversion definition ID
If you receive an error that the conversion definition ID is invalid, ensure that you have created a conversion definition in Amazon Ads and selected it in the sync configuration.
If you are using a column for the conversion definition ID, ensure that the column values match the numeric ID of the conversion definition in Amazon Ads, rather than its name.
### Live debugger
### Sync alerts
---
## Amazon EventBridge
**URL:** https://hightouch.com/docs/destinations/amazon-eventbridge
**Description:** Amazon EventBridge is a serverless, fully managed, and scalable event bus that enables integrations between AWS services, Software as a services (SaaS), and your applications.
**Section:** Destinations
## Overview
Hightouch integrates directly with Amazon EventBridge to support high-throughput, distributed, or asynchronous workloads, letting you build a custom connector to your internal systems.
This destination was designed to be as flexible as possible. You can publish into different buses, have custom sources attributes and detail types for each message trigger allowing you to enrich your events.
## Getting started
### Connect to your AWS account
When setting up the Amazon EventBridge destination for the first time, you need to enter your **AWS Credentials** to give Hightouch access to your AWS account. Hightouch needs permission to send events to Amazon EventBridge on your behalf.
Hightouch believes in the principle of least privilege. We ask for no more permissions than necessary. For the Amazon EventBridge destination, Hightouch requires `events:PutEvents` and `events:ListEventBuses`.
When configuring your credentials in Hightouch, you have two options:
- Setting up a cross-account role (recommended)
- Providing an access key
#### Set up cross-account role
Cross-account roles are the most secure method for granting Hightouch access to Amazon EventBridge in your AWS account.
You need the **Account ID** and **External ID** in the Hightouch UI to set up a cross-account role.
1. In the **AWS Console**, navigate to **IAM** → **Roles**.
1. Click **Create Role**.
1. Under **Trusted entity type**, choose **AWS account**.
1. Select **Another AWS account** and also click **Require external ID**.
1. Copy/paste the **Account ID** from Hightouch into the **Account ID** field in AWS.
1. Copy/paste the **External ID** from Hightouch into the **External ID** field in AWS.
1. On the **Add permissions** screen, proceed to attach permissions to the role using any policy that includes at least `events:PutEvents` and `events:ListEventBuses`.
1. Copy/paste the **Role ARN** from AWS into Hightouch. Click **Create** to finish setting up your cross-account role.
#### Provide access key
If you don't want to create a cross-account role, you can create a regular IAM user and share a **Access Key ID** and **Secret Access Key** with Hightouch.
Be sure to attach the user to a permission policy that include `events:PutEvents` and `events:ListEventBuses` for the specific resource you want to use via a Hightouch sync.
An example policy looks like this:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "events:PutEvents",
"Resource": ["YOUR_EVENTBRIDGE_BUS_ARN_1", "YOUR_EVENTBRIDGE_BUS_ARN_2", "YOUR_EVENTBRIDGE_BUS_ARN_3"]
},
{
"Effect": "Allow",
"Action": "events:ListEventBuses",
"Resource": "*"
}
]
}
```
### Select your AWS region
In this step, you specify which region has the resources you want to sync to.
## Syncing data
Once you've authenticated your AWS account in Hightouch and selected an AWS Region, you've completed setup for an Amazon EventBridge destination in your Hightouch workspace. The next step is to configure a sync that send events whenever rows are added, changed, or removed in your model.
### Configure your events trigger
Hightouch monitors your data model for added, changed, and removed rows. In this step, you specify which of these events should trigger message publishing.
### Choose your event bus
In this step, you choose to send the events to a specific event bus. Hightouch allows you to sync to existing buses in your AWS EventBridge.
## Customize your source and detail type attributes
In this step, you tell Hightouch what value to put for the `source` and `detail-type` attribute in EventBridge for the event. More information can be found here on what these fields mean can be found in the [AWS docs](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-events.html).
#### Use JSON editor
With the JSON editor, you can compose any JSON object using the [Liquid template language](https://shopify.github.io/liquid/). This is particularly useful for complex message data bodies containing nested objects and arrays, which can sometimes be difficult to model entirely in SQL.
Suppose your data model looks like this:
| full_name | age | email_address | phone_number |
| --------- | --- | ------------------ | ------------ |
| John Doe | 30 | `john@example.com` | +14158675309 |
And you want your message data like this:
```json
{
"name": "John Doe",
"age": 30,
"contact_info": [
{
"type": "email",
"value": "john@example.com"
},
{
"type": "phone",
"value": "+14158675309"
}
]
}
```
Your Liquid template should look like this:
```txt
{
"name": "{{row.full_name}}",
"age": {{row.age}},
"contact_info": [
{
"type": "email",
"value": "{{row.email_address}}"
},
{
"type": "phone",
"value": "{{row.phone_number}}"
}
]
}
```
This makes it so you can reference any column using the syntax `{{row.column_name}}`. You can also use advanced Liquid features to incorporate control flow and loops into your dynamic message data.
When injecting strings into your JSON request body, be sure to surround the
Liquid tag in double quotes.
#### Use one column from model
If you're already storing JSON data in your source, or if you have the ability to construct a JSON object using SQL, you can select one column in your model that already contains the full message data.
This setting is commonly used when syncing web events that have already been collected and stored as JSON objects in your database.
#### Use multiple columns from model
For the simplest use cases, Hightouch can construct a JSON object with key/value pairs based on multiple columns in your model.
Suppose your model looks like this:
| email | first_name | last_name |
| ----------------------- | ---------- | --------- |
| `alice.doe@example.com` | Alice | Doe |
| `bob.doe@example.com` | Bob | Doe |
| `carol.doe@example.com` | Carol | Doe |
The field mapping in the preceding screenshot would generate the following message data for the first row:
```json
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "alice.doe@example.com"
}
```
You can use the field mapper to rename fields. For example, `first_name` can
be mapped to `customer_first_name`.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows may require performing a backfill of all rows during the initial sync. For other use cases, you might only want to send messages in response to future data changes.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Amazon Kinesis
**URL:** https://hightouch.com/docs/destinations/amazon-kinesis
**Description:** Ingest and process your data via a data stream
**Section:** Destinations
## Overview
Hightouch integrates directly with Amazon Kinesis to support high-throughput, distributed, or asynchronous workloads, letting you build a custom connector to your internal systems.
## Getting started
### Connect to your AWS account
When setting up the Amazon Kinesis destination for the first time, you need to enter your **AWS Credentials** to give Hightouch access to your AWS account. Hightouch needs permission to send streaming data to Amazon Kinesis on your behalf.
Hightouch believes in the principle of least privilege. We ask for no more permissions than necessary. For the Amazon Kinesis destination, Hightouch requires `kinesis:PutRecords` and `kinesis:ListStreams`.
When configuring your credentials in Hightouch, you have two options:
- Setting up a cross-account role (recommended)
- Providing an access key
#### Set up cross-account role
Cross-account roles are the most secure method for granting Hightouch access to Amazon Kinesis in your AWS account.
You need the **Account ID** and **External ID** in the Hightouch UI to set up a cross-account role.
1. In the **AWS Console**, navigate to **IAM** → **Roles**.
1. Click **Create Role**.
1. Under **Trusted entity type**, choose **AWS account**.
1. Select **Another AWS account** and also click **Require external ID**.
1. Copy/paste the **Account ID** from Hightouch into the **Account ID** field in AWS.
1. Copy/paste the **External ID** from Hightouch into the **External ID** field in AWS.
1. On the **Add permissions** screen, proceed to attach permissions to the role using any policy that includes at least `kinesis:PutRecords` and `kinesis:ListStreams`. Then create the role.
1. Copy/paste the **Role ARN** from AWS into Hightouch. Click **Create** to finish setting up your cross-account role.
#### Provide access key
If you don't want to create a cross-account role, you can create a regular IAM user and share a **Access Key ID** and **Secret Access Key** with Hightouch.
Be sure to attach the user to a permission policy that includes `kinesis:PutRecords` and `kinesis:ListStreams` for the specific resource you want to use via a Hightouch sync.
An example policy looks like this:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["kinesis:PutRecords", "kinesis:ListStreams"],
"Resource": [
"YOUR_KINESIS_ARN_1",
"YOUR_KINESIS_ARN_2",
"YOUR_KINESIS_ARN_3"
]
}
]
}
```
### Select your AWS region
In this step, you specify which region has the resources you want to sync to.
## Syncing data
Once you've authenticated your AWS account in Hightouch and selected an AWS Region, you've completed setup for a Amazon Kinesis destination in your Hightouch workspace. The next step is to configure a sync that send messages whenever rows are added, changed, or removed in your model.
### Configure your events trigger
Hightouch monitors your data model for added, changed, and removed rows. In this step, you specify which of these events should trigger message publishing.
### Choose your data stream
In this step, you choose which data stream to send messages to. Hightouch allows you to sync to existing data streams that are already in your Amazon Kinesis.
### Customize your streaming data
In this step, you tell Hightouch how to build the JSON streaming data object using data from your model.
#### Configure partition key
This field is sent along with the streaming data and determines which shard in the stream the data record is assigned to. Amazon Kinesis Data Streams uses the partition key as input to a hash function that maps the partition key and associated data to a specific shard. Specifically, an MD5 hash function is used to map partition keys to 128-bit integer values and to map associated data records to shards. As a result of this hashing mechanism, all data records with the same partition key map to the same shard within the stream.
This destination offers three methods of composing a JSON object:
- [Using a JSON editor](#use-json-editor)
- [Using one column from a model](#use-one-column-from-model)
- [Using multiple columns from a model](#use-multiple-columns-from-model)
#### Use JSON editor
With the JSON editor, you can compose any JSON object using the [Liquid template language](https://shopify.github.io/liquid/). This is particularly useful for complex streaming data bodies containing nested objects and arrays, which can sometimes be difficult to model entirely in SQL.
Suppose your data model looks like this:
| full_name | age | email_address | phone_number |
| --------- | --- | ------------------ | ------------ |
| John Doe | 30 | `john@example.com` | +14158675309 |
And you want your streaming data like this:
```json
{
"name": "John Doe",
"age": 30,
"contact_info": [
{
"type": "email",
"value": "john@example.com"
},
{
"type": "phone",
"value": "+14158675309"
}
]
}
```
Your Liquid template should look like this:
```txt
{
"name": "{{row.full_name}}",
"age": {{row.age}},
"contact_info": [
{
"type": "email",
"value": "{{row.email_address}}"
},
{
"type": "phone",
"value": "{{row.phone_number}}"
}
]
}
```
This makes it so you can reference any column using the syntax `{{row.column_name}}`. You can also use advanced Liquid features to incorporate control flow and loops into your dynamic streaming data.
When injecting strings into your JSON body, be sure to surround the Liquid tag
in double quotes.
#### Use one column from model
If you're already storing JSON data in your source, or if you have the ability to construct a JSON object using SQL, you can select one column in your model that already contains the full streaming data.
This setting is commonly used when syncing web events that have already been collected and stored as JSON objects in your database.
#### Use multiple columns from model
For the simplest use cases, Hightouch can construct a JSON object with key/value pairs based on multiple columns in your model.
Suppose your model looks like this:
| email | first_name | last_name |
| ----------------------- | ---------- | --------- |
| `alice.doe@example.com` | Alice | Doe |
| `bob.doe@example.com` | Bob | Doe |
| `carol.doe@example.com` | Carol | Doe |
The field mapping in the screenshot above would generate the following streaming data for the first row:
```json
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "alice.doe@example.com"
}
```
You can use the field mapper to rename fields. For example, `first_name` can
be mapped to `customer_first_name`.
### Configure optional message properties
Along with your row data in JSON format, you can also optionally include an explicit hash key to explicitly determine the shard the data record is assigned to.
##### `ExplicitHashKey`
This is a string field that should be a hash value and will override the partition key hash.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows may require performing a backfill of all rows during the initial sync. For other use cases, you might only want to send messages in response to future data changes.
## Tips and troubleshooting
### Common errors
#### ProvisionedThroughputExceededException: Rate exceeded for shard.
This error refers to Amazon Kinesis’ [quotas and limits](https://docs.aws.amazon.com/streams/latest/dev/service-sizes-and-limits.html). Manual intervention isn't required since our sync engine [retries rejected rows](/syncs/retries) during the following sync run.
### Live debugger
### Sync alerts
---
## Amazon SQS
**URL:** https://hightouch.com/docs/destinations/amazon-sqs
**Description:** Connect to your internal microservices via a message queue
**Section:** Destinations
## Overview
Hightouch integrates directly with Amazon SQS to support high-throughput, distributed, or asynchronous workloads, letting you build a custom connector to your internal systems.
This destination was designed to be as flexible as possible. You can publish into different queues including standard and FIFO queues for each message trigger and define custom ordering keys and metadata fields to enrich your messages.
## Getting started
### Connect to your AWS account
When setting up the Amazon SQS destination for the first time, you need to enter your **AWS Credentials** to give Hightouch access to your AWS account. Hightouch needs permission to send messages to Amazon SQS on your behalf.
Hightouch believes in the principle of least privilege. We ask for no more permissions than necessary. For the Amazon SQS destination, Hightouch requires `sqs:SendMessage` and `sqs:ListQueues`.
When configuring your credentials in Hightouch, you have two options:
- Setting up a cross-account role (recommended)
- Providing an access key
#### Set up cross-account role
Cross-account roles are the most secure method for granting Hightouch access to Amazon SQS in your AWS account.
You need the **Account ID** and **External ID** in the Hightouch UI to set up a cross-account role.
1. In the **AWS Console**, navigate to **IAM** → **Roles**.
1. Click **Create Role**.
1. Under **Trusted entity type**, choose **AWS account**.
1. Select **Another AWS account** and also click **Require external ID**.
1. Copy/paste the **Account ID** from Hightouch into the **Account ID** field in AWS.
1. Copy/paste the **External ID** from Hightouch into the **External ID** field in AWS.
1. On the **Add permissions** screen, proceed to attach permissions to the role using any policy that includes at least `sqs:SendMessage` and `sqs:ListQueues`. Then create the role.
1. Copy/paste the **Role ARN** from AWS into Hightouch. Click **Create** to finish setting up your cross-account role.
#### Provide access key
If you don't want to create a cross-account role, you can create a regular IAM user and share a **Access Key ID** and **Secret Access Key** with Hightouch.
Be sure to attach the user to a permission policy that includes `sqs:SendMessage` and `sqs:ListQueues` for the specific resource you want to use via a Hightouch sync.
An example policy looks like this:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["sqs:SendMessage", "sqs:ListQueues"],
"Resource": ["YOUR_SQS_ARN_1", "YOUR_SQS_ARN_2", "YOUR_SQS_ARN_3"]
}
]
}
```
### Select your AWS region
In this step, you specify which region has the resources you want to sync to.
## Syncing data
Once you've authenticated your AWS account in Hightouch and selected an AWS Region, you've completed setup for a Amazon SQS destination in your Hightouch workspace. The next step is to configure a sync that send messages whenever rows are added, changed, or removed in your model.
### Configure your events trigger
Hightouch monitors your data model for added, changed, and removed rows. In this step, you specify which of these events should trigger message publishing.
### Choose your queue
In this step, you choose to send the messages to either Standard or FIFO queues. Hightouch allows you to sync to existing queues that are already in your Amazon SQS.
### Customize your message
In this step, you tell Hightouch which column in your model contains the unique message ID and how to build the JSON message data object using data from your model.
#### Configure unique message ID
This unique ID field is sent along with the message data and is an identifier for a message in this batch used to communicate the result.
#### Configure message group ID (for FIFO queues only)
If you are syncing to a FIFO queue, you need to provide a message group ID to specify that a message belongs to a specific message group. Messages that belong to the same message group are processed in a FIFO manner however, messages in different message groups might be processed out of order.
You can toggle between a column in your model or a static value to set your message group ID.
This destination offers three methods of composing a JSON object:
- [Using a JSON editor](#use-json-editor)
- [Using one column from a model](#use-one-column-from-model)
- [Using multiple columns from a model](#use-multiple-columns-from-model)
#### Use JSON editor
With the JSON editor, you can compose any JSON object using the [Liquid template language](https://shopify.github.io/liquid/). This is particularly useful for complex message data bodies containing nested objects and arrays, which can sometimes be difficult to model entirely in SQL.
Suppose your data model looks like this:
| full_name | age | email_address | phone_number |
| --------- | --- | ------------------ | ------------ |
| John Doe | 30 | `john@example.com` | +14158675309 |
And you want your message data like this:
```json
{
"name": "John Doe",
"age": 30,
"contact_info": [
{
"type": "email",
"value": "john@example.com"
},
{
"type": "phone",
"value": "+14158675309"
}
]
}
```
Your Liquid template should look like this:
```txt
{
"name": "{{row.full_name}}",
"age": {{row.age}},
"contact_info": [
{
"type": "email",
"value": "{{row.email_address}}"
},
{
"type": "phone",
"value": "{{row.phone_number}}"
}
]
}
```
This makes it so you can reference any column using the syntax `{{row.column_name}}`. You can also use advanced Liquid features to incorporate control flow and loops into your dynamic message data.
When injecting strings into your JSON request body, be sure to surround the
Liquid tag in double quotes.
#### Use one column from model
If you're already storing JSON data in your source, or if you have the ability to construct a JSON object using SQL, you can select one column in your model that already contains the full message data.
This setting is commonly used when syncing web events that have already been collected and stored as JSON objects in your database.
#### Use multiple columns from model
For the simplest use cases, Hightouch can construct a JSON object with key/value pairs based on multiple columns in your model.
Suppose your model looks like this:
| email | first_name | last_name |
| ----------------------- | ---------- | --------- |
| `alice.doe@example.com` | Alice | Doe |
| `bob.doe@example.com` | Bob | Doe |
| `carol.doe@example.com` | Carol | Doe |
The field mapping in the preceding screenshot would generate the following message data for the first row:
```json
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "alice.doe@example.com"
}
```
You can use the field mapper to rename fields. For example, `first_name` can
be mapped to `customer_first_name`.
### Configure optional message properties
Along with your row data in JSON format, you can optionally include ordering keys to configure the order your queue receives your message and metadata fields as headers.
##### `DelaySeconds` (for standard queues only)
A number field that postpone the delivery of new messages to consumers for a number of seconds. Any messages that you send to the queue will remain invisible to consumers for the duration of the delay period. The maximum length of time you can delay for is 15 minutes.
##### `MessageDeduplicationId` (for FIFO queues only)
This is a string field. If a message with a particular Message Deduplication ID is sent successfully, subsequent messages with the same Message Deduplication ID are accepted successfully but aren't delivered. If you enabled Content-based deduplication when configuring your FIFO queue then you may optionally embed a deduplication message ID for your message. If you turned Content-based deduplication off, then this section is required for the messages to be published successfully.
##### `MessageAttributes`
This is an object containing key/value pairs of custom mapping fields. Values can be text or byte strings.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows may require performing a backfill of all rows during the initial sync. For other use cases, you might only want to send messages in response to future data changes.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Amobee
**URL:** https://hightouch.com/docs/destinations/amobee
**Description:** Transfer data from your data warehouse to Amobee via sFTP file drop.
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|Any data set|Sync data from a source to a tab-delimited Amobee file|Insert only|
This destination ignores the concept of additions, changes, removals. Syncs always send the full results of your model. For example, if your model has 1,000 records, your Amobee data file will be 1,000 records.
## Getting started
### Setup on Amobee
The initial setup process involves getting assigned a `market ID` from Amobee. If you don't know this ID, reach out to your Amobee representative.
## Setup on Hightouch
To get started with the Amobee destination on Hightouch, you must provide the market ID.
## Syncing data
In addition to the data file, Hightouch sends configuration and taxonomy files during the sync. Amobee's system picks up the files hourly.
Amobee also emails a success or failure notification to the address you provide in the sync configuration.
### Configuration file
To create the configuration file, enter the following information into the Hightouch during sync setup:
- Advertiser ID
- Email address to receive success notifications
- Name of the data file you want to send to Amobee
- Classification symbol
- Data file unique identifier ID type
- Data file start date
Hightouch uses this information to construct the configuration file included in the sync.
### Taxonomy file
You also need to upload a taxonomy file as part of the sync configuration form. An Amobee taxonomy file is an Excel document that helps categorize data into a meaningful format.
The accepted file types include: `.xls` and `.xlsx`. For reference, you can check out this [taxonomy file example](https://bit.ly/3UzkGP5).
### Data file format and encoding
The data file must be delimited, with fields and categories separated. Categories need to be
separated using a comma (`,`).
### Columns to sync
For this destination, you can export all columns represented in your model.
If you need to remap the fields you're exporting, you can do so manually.
In that case, the sync only exports fields you've mapped. This example exports `ID` and `CategoryID` and ignores all other fields in the model.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Amplitude
**URL:** https://hightouch.com/docs/destinations/amplitude
**Description:** Empower your product teams with richer data from your warehouse to perform deeper analysis and better understand user behavior
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-------------|--------------------------------------------------------------------------------|----------------------|---------------
**Objects** | Sync records to objects such as users or organizations in Amplitude | Upsert | [Objects docs](https://amplitude.com/docs/analytics/apis/identify-api)
**Events** | Sync records as events to Amplitude. This is often in the form of a track call | Insert | [Events docs](https://amplitude.com/docs/analytics/apis/batch-event-upload-api)
**Cohorts** | Sync records to cohorts in Amplitude | Add, Remove |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Amplitude
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination**
button. Select **Amplitude** and click **Continue**. You can then authenticate Hightouch to **Amplitude** by entering these fields into Hightouch:
- **API Key**: You can find your API key by following [these instructions in Amplitude's docs](https://www.docs.developers.amplitude.com/analytics/find-api-credentials/).
- **Region**: This is either a **Standard server** or **EU residency server**.
Once you've entered these fields, click **Continue**. Give your destination a descriptive name and **Finish** the destination connection setup.
## Sync configuration
Once you've set up your Amplitude destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and
the Amplitude destination you want to sync to.
### Objects
Hightouch can sync data to either **[User](https://help.amplitude.com/hc/en-us/articles/115002380567-User-properties-and-event-properties)** and **[Group](https://developers.amplitude.com/docs/group-identify-api)** objects. A dropdown is available to select between **Users** and **Groups** when configuring an Amplitude sync.
This integration only supports the **Upsert** mode. In this mode, Hightouch inserts new users or groups into Amplitude and keeps all user attributes up-to-date.
#### Record matching
Hightouch matches rows from your model to Amplitude users by **Amplitude User ID** or **Amplitude Device ID**. Select the model column that contains these values if you are syncing user objects. See [Amplitude's docs](https://help.amplitude.com/hc/en-us/articles/206404628-Step-2-Identifying-your-users) for more on identifying users.
Hightouch matches rows from your model to Amplitude groups by **Group Value**. Select the model column that contains these values if you are syncing groups.
#### Field mapping
You can sync columns from your source to Amplitude's default and custom fields. User properties only apply on the next event **after** the property has been changed, and don't apply retrospectively. For more information, check out [Amplitude's FAQ](https://help.amplitude.com/hc/en-us/articles/115002380567-User-Properties-Event-Properties#applying-user-properties-to-events).
#### Delete behavior
The delete behavior you select dictates what to do when a row no longer appears in your model's query results. You have the following options for Amplitude objects:
Behavior | Description
---------------|-----------------------------------------------------------------
**Do nothing** | Keep the customer or group record in Amplitude with all its synced fields
**Clear** | Remove all the mapped fields, but keep the record in Amplitude
### Events
Hightouch supports sending events of a given name, for example, `Signed Up`. Enter this name and choose the model column containing the either user ID or device ID. You can also select the model column containing the event timestamp. If the timesamp field is empty, Hightouch uses the time the event arrives at the server.
Hightouch accepts a normal date time format for the timestamp, and
automatically converts it to the format required by Amplitude.
#### Field mapping
Hightouch allows you to pass data to the [default properties](https://developers.amplitude.com/docs/http-api-v2#keys-for-the-event-argument) and the custom event properties of an Amplitude event.
### Cohorts
Hightouch can sync data to Behavioral Cohorts in Amplitude.
#### Record matching
Hightouch matches rows from your model to Amplitude users by **User ID** or **Amplitude ID**. Select the model column that contains these values if you are syncing to cohorts. See [Amplitude's docs](https://amplitude.com/docs/apis/analytics/behavioral-cohorts) for more on behavioral cohorts.
#### Creating new Cohorts
Hightouch can **Create a new cohort** when syncing to Amplitude cohorts. Hightouch will create a single placeholder user when creating a new cohort, and then sync the mapped identifiers to your newly created cohort.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## NCR Advanced Marketing Solution
**URL:** https://hightouch.com/docs/destinations/ams
**Description:** Manage targeted promotions and loyalty programs for all the stores in your chain, from a central point.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes |
| ------------------ | ---------------------------------------------------------------------- | -------------------- |
| **Customer group** | Sync customer groups by external ID to NCR Advanced Marketing Solution | Insert, All |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to NCR Advanced Marketing Solution
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **NCR Advanced Marketing Solution** (AMS) and click **Continue**. You can then authenticate Hightouch to **NCR Advanced Marketing Solution** by entering the following fields into Hightouch:
- **Host**: The hostname or IP address of your NCR AMS instance
- **Username**: This can be your personal NCR AMS login or a dedicated user for Hightouch
- (Optional) **Password**: The password for the specified user
- (Optional) **Private Key**: An SSH private key
- (Optional) **Private Key Passphrase**: The passphrase associated with the private key
## Sync configuration
Once you've set up your NCR AMS destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the NCR AMS destination you want to sync to.
### Syncing customer groups
Hightouch syncs customer groups to NCR AMS as `.cgi` files.
Hightouch automatically batches your query results into up to two million rows per file.
Each `.cgi` file contains `ID`,`Name`, and `BulkData` XML elements.
The `BulkData` element contains the external IDs of the customer groups you are syncing.
Refer to the [example `.cgi` file](#example-file) for details.
#### File path
The file path defines where Hightouch should send the `.cgi` files.
The file naming convention is `<>_<>_<>.cgi`.
Hightouch automatically adds the file number so you only need to provide the `<>_<>.cgi`.
For example, suppose you input `TestGroup_AMSID_73237_UPC_53395004106.cgi`, the first two million rows of data would appear in a `.cgi` file with the name `TestGroup_AMSID_73237_UPC_53395004106_01`.
The second two million rows would appear in a file with the name `TestGroup_AMSID_73237_UPC_53395004106_02`, etc.
#### Batch ID
Hightouch uses your input in the **Batch ID** for the `Name` and `ID` elements in the `.cgi` file and automatically suffixes a `_<>`.
For example, suppose you input `37293_US` as the batch ID, the first two million rows would use `37293_US_01` for both `Name` and `ID` elements.
The second two million rows would use `37293_US_02` for `Name` and `ID` elements, etc.
#### External ID column
Use this field to specify which column from your model contains the external IDs to sync.
In the `.cgi` file, these values have the suffix `,0\n` (comma, zero digit, and new line) and are sent as the `BulkData` element.
#### Example file
Hightouch syncs `.cgi` files with contents like this:
```
37293_US_0137293_US_01
405247281386905506,0
213550969606979806,0
497647364788236008,0
914660244431276315,0
637947282525555405,0
525462241744844516,0
494763443801427416,0
augment
```
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Anaplan
**URL:** https://hightouch.com/docs/destinations/anaplan
**Description:** Optimize enterprise performance using fresh data from your data warehouse
**Section:** Destinations
## Overview
With the Anaplan destination, Hightouch can automate file uploads and trigger import actions in Anaplan.
This flexible integration allows you to import a variety of data from your warehouse, such as purchase orders, customer data, and more.
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| ----------- | -------------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------ |
| **Records** | Sync data from any source to Anaplan records | Insert, All | [Import action docs](https://help.anaplan.com/5ef6eaab-72d9-43de-91b3-3f4dcc4711e2-Perform-an-import-action) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Setup
Select which auth type to use oAuth or basic auth (username and password).
### OAuth
#### Setup Anaplan OAuth client
Hightouch supports Anaplan OAuth clients that have the type `Authorization code grant` with both non-rotatable and rotatable refresh tokens.
- Non-rotatable: The refresh token used for getting access tokens do not expire for as long as you allow the connection.
- Rotatable: The refresh token used for getting access tokens changes every time a new access token is retrieved.
Rotatable tokens require setting a `Refresh token lifespan` in Anaplan. This
setting is the total time in seconds before the connection will need to be
re-authenticated in Hightouch.
**Ex:** Setting `Refresh token lifespan` token to 604800 (7 days) will rotate the
refresh token every time an access token expires but will require you to re-authenticate
your account at the end of that time. Until you re-authenticate all syncs will
continue to fail. [More detail on refresh tokens](https://help.anaplan.com/rotate-client-secret-357f0086-5752-435b-a5bd-5eb6797ffc3f)
Use Non-rotatable refresh tokens for syncs to minimize chances of sync
failures.
#### Connecting with Hightouch
Select the region you want to connect to and then enter your client ID and client secret.
Your client ID and secret are provided by Anaplan when you setup your OAuth client.
Continue through the auth flow to connect your account.
### Basic auth
Select the region you want to connect to and enter your username and password.
_You may need to create an IP whitelist for Hightouch servers to allow authentication_
- See how to create a whitelist on Anaplan [here](https://help.anaplan.com/enable-ip-allow-list-9fcc01f4-fcb9-4384-8568-4ebb1bdfe9f9).
- See Highotuch Server IP addresses [here](https://hightouch.com/docs/security/networking#ip-addresses).
## Creating a import data source in Anaplan
You must create an existing data source for Hightouch to upload data into. Navigate to your model -> Model settings -> Actions -> Import Data Source.
Create a new "Import Data Source" by uploading a placeholder CSV.
The Hightouch integration will overwrite this CSV with data from your warehouse based on the mappings in your sync configuration.
## Creating an import in Anaplan
You can optionally create imports based on the data source, and trigger them after the upload is complete in Hightouch. Navigate to a list and click the "Import" button.
After selecting the placeholder data source, you can create an import that can then be selected in Hightouch and viewed in the imports page.
## Creating a sync in Hightouch
After setting up the data source and imports in the Anaplan destination, you can use Hightouch to automate sending data to the data sources and triggering
the import by creating a sync.
### Selecting the model and import data source
Select the model and data source that you created (or an existing one). Please keep in mind that Hightouch will overwrite the import data source. You can choose between two modes:
- Insert: The data source is overwritten with only the rows that have been added since the last run.
- All: The data source is overwritten with the full results of the query in this run.
### Selecting the triggered imports
You can select one or more imports to trigger after the data source upload is complete. By default, Hightouch triggers these imports in parallel. Hightouch will wait until
the imports are complete before completing the sync.
### Mapping your columns
You can map the columns in your Hightouch model to columns in your Anaplan data source.
The Anaplan column names you map to in Hightouch must exactly match the column
names configured your Anaplan data source. The columns must also be in the
same order as they appear in your data source. Incorrect column names or order
mismatch will cause your sync to error.
---
## Antavo
**URL:** https://hightouch.com/docs/destinations/antavo
**Description:** Attract customers, scope loyalty, and build your dream customer loyalty program with Antavo.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-------------------|----------------------------------------------------|----------------------|----------------------------------------------------------------------------------
**Events** | Sync data from any source to Antavo events | | [Events docs](https://developers.antavo.com/reference/about-events-api)
**Customer Lists** | Sync data from any source to Antavo customer lists | Add, Remove | [Customer Lists docs](https://developers.antavo.com/reference/about-entities-api)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Antavo
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Antavo** and click **Continue**. You can then authenticate Hightouch to **Antavo**.
Enter the following fields into Hightouch:
- **API key**
- **API secret**
- **Region**
## Prerequisite steps in Antavo
There are prerequisite steps in Antavo to sync Events. Custom event creation is required before syncing to Antavo.
For Customer Lists, you have the option to create a customer list in Antavo directly and use the existing list in Hightouch, or create a new customer list.
If you run into any issues, you can reach out to the Antavo CSM team.
## Sync configuration
Once you've set up your Antavo destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Antavo destination you want to sync to.
### Syncing events
Sync data from any source to Antavo events. You can either:
- sync one [event type](https://developers.antavo.com/docs/api-events) for the entire batch by specifying an `Event type`, and map the corresponding event's attributes or
- sync multiple event types but pass in the `data` objet directly in your mappings.
#### Record matching
You can match rows from your model to events in Antavo on any column in your model and any field in Antavo. Ensure the data types of the model column and Antavo field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
### Syncing customer lists
Sync data from any source to Antavo customer lists.
#### Record matching
To match rows from your model to customer lists in Antavo, you need to select the model column that contains values that match the **ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync customer list fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default customer list fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Apache Kafka
**URL:** https://hightouch.com/docs/destinations/apache-kafka
**Description:** Connect to your internal microservices via a message queue
**Section:** Destinations
## Overview
Hightouch integrates directly with Apache Kafka to support high-throughput, distributed, or asynchronous workloads, letting you build a custom connector to your internal systems.
This destination was designed to be as flexible as possible. Some of its capabilities include:
- connecting to multiple brokers
- authenticating with Simple Authentication and Security Layer (SASL)
- using your own certificate authority
- publishing different topics for each message trigger
- defining custom ordering and partition keys
Hightouch supports all managed Kafka services (Amazon MSK, Confluent Cloud, etc.) and can also connect to self-hosted instances.
## Getting started
### Connect to your Kafka server
For Hightouch to connect to your Kafka server, you need to enter:
- a client ID
- your broker information
- authentication method used in your Kafka server configuration
#### Client ID
The [**Client ID**](https://kafka.apache.org/documentation/#design_quotasgroups) is a logical identifier of a client application—in this case it's the Hightouch Apache Kafka destination. It's used to distinguish each running application of your Kafka server. You can choose to name this anything you want that fits your use case.
#### Brokers
For Hightouch to sync data to the right Kafka Brokers, you need to provide the **host** and **port** number in the format `{host}:{port}`. If you want to configure the destination to connect to multiple brokers, you can input the details separated by a comma. For example, you could enter `{host1}:{port1},{host2}:{port2}`.
#### Authentication
Hightouch can connect to your Kafka server either with SASL or directly without authentication. For security purposes, it's best to configure your Kafka server to require SASL authentication when syncing production data, and to only omit authentication requirements for testing purposes.
When configuring your SASL mechanism in Hightouch, you have four options:
- PLAIN
- SCRAM SHA256
- SCRAM SHA512
- AWS IAM
All four provide the option to include your own self-signed certificate authority.
#### PLAIN/SCRAM
For **Username** and **Password**, enter your username and password configured in your Kafka server. If you are using a managed Kafka service, your details can usually be found in your environment's settings or as an API key and secret.
#### AWS IAM
If you want to authenticate via AWS IAM, we assume your Kafka server is configured to use AWS IAM as an authentication method, that is [STACK's Kafka AWS IAM LoginModule](https://github.com/STACK-Fintech/kafka-auth-aws-iam) or a compatible alternative is installed on all target brokers.
- **Authorization Identity** must be the `aws:userid` of the AWS IAM identity. Typically, you can retrieve this value using the `aws iam get-user` or `aws iam get-role` commands of the AWS CLI toolkit. The `aws:userid` is usually listed as the `UserId` or `RoleId` property of the response.
- You can find your **Access Key ID**, **Secret Access Key**, and **Session Token** in your AWS account. For more information on AWS IAM credentials and authentication, refer to the official [AWS docs](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html).
## Syncing data
Once you've connected your Kafka server to Hightouch, you've completed setup for a Apache Kafka destination in your Hightouch workspace. The next step is to configure a sync that send messages whenever rows are added, changed, or removed in your model.
### Configure your events trigger
Hightouch monitors your data model for added, changed, and removed rows. In this step, you specify which of these events should trigger message publishing.
### Choose your topic
In this step, you choose which topics to publish the messages to. Hightouch allows you to sync to existing topics that are already in your Kafka cluster.
Suppose you want to sync to multiple existing topics but don't want to create a new sync for every topic. As long as your model has a column associated to topic names in your Kafka cluster, Hightouch can sync to multiple Apache Kafka topics in just one sync.
To enable this feature, toggle **USE COLUMN**, and select a column in your model containing the topic name rows.
When syncing to multiple topics, if a topic name in the selected column of
your model doesn't exist in the Kafka cluster, then the entire batch of
messages will fail to sync.
### Customize your message
In this step, you tell Hightouch how to build the JSON message data object using data from your model.
This destination offers three methods of composing a JSON object:
- [Using a JSON editor](#use-json-editor)
- [Using one column from a model](#use-one-column-from-model)
- [Using multiple columns from a model](#use-multiple-columns-from-model)
#### Use JSON editor
With the JSON editor, you can compose any JSON object using the [Liquid template language](https://shopify.github.io/liquid/). This is useful for complex message data bodies containing nested objects and arrays, which can sometimes be difficult to model entirely in SQL.
Suppose your data model looks like this:
| full_name | age | email_address | phone_number |
| --------- | --- | ------------------ | ------------ |
| John Doe | 30 | `john@example.com` | +14158675309 |
And you want your message data like this:
```json
{
"name": "John Doe",
"age": 30,
"contact_info": [
{
"type": "email",
"value": "john@example.com"
},
{
"type": "phone",
"value": "+14158675309"
}
]
}
```
Your Liquid template should look like this:
```txt
{
"name": "{{row.full_name}}",
"age": {{row.age}},
"contact_info": [
{
"type": "email",
"value": "{{row.email_address}}"
},
{
"type": "phone",
"value": "{{row.phone_number}}"
}
]
}
```
This makes it so you can reference any column using the syntax `{{row.column_name}}`. You can also use advanced Liquid features to incorporate control flow and loops into your dynamic message data.
When injecting strings into your JSON object, be sure to surround the Liquid
tag in double quotes.
#### Use one column from model
If you're already storing JSON data in your source, or if you have the ability to construct a JSON object using SQL, you can select one column in your model that already contains the full message data.
This setting is commonly used when syncing web events that have already been collected and stored as JSON objects in your database.
#### Use multiple columns from model
For the simplest use cases, Hightouch can construct a JSON object with key/value pairs based on multiple columns in your model.
Suppose your model looks like this:
| email | first_name | last_name |
| ----------------------- | ---------- | --------- |
| `alice.doe@example.com` | Alice | Doe |
| `bob.doe@example.com` | Bob | Doe |
| `carol.doe@example.com` | Carol | Doe |
The field mapping in the preceding screenshot would generate the following message data for the first row:
```json
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "alice.doe@example.com"
}
```
You can use the field mapper to rename fields. For example, `first_name` can
be mapped to `customer_first_name`.
### Configure optional message properties
Along with your row data in JSON format, you can optionally include ordering keys to configure the order your Kafka cluster receives message and metadata fields as headers.
##### `partition`
A number field that determines which partition to send the message to. This field takes precedence over the `key` field. That is if you provided `partition` and `key`, the message will be sent to the partition stated in the `partition` field and not the `key` field. Hightouch automatically tries to cast the value to a number. If we can't cast the value to a number then it is sent as `null`.
##### `key`
If no `partition` column is selected but a `key` of string type is selected, then Kafka chooses a partition to send the message to based on a murmur2 hash of the key. For example, if you use an `orderId` as the key, you can ensure that all messages regarding that order will be processed in order.
If no `partition` or `key` is included, then the message will be sent to a partition in a round-robin fashion.
##### `headers`
This is an object containing key/value pairs of custom mapping fields.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows may require performing a backfill of all rows during the initial sync. For other use cases, you might only want to send messages in response to future data changes.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Apollo.io
**URL:** https://hightouch.com/docs/destinations/apollo
**Description:** Apollo’s comprehensive solution provides the most accurate B2B contact data, built-in engagement tools, and intelligence features that help companies find and convert quality prospects.
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes | API Reference|
|--|--|--|--|
|**Accounts**|Sync data from any source to an account |Upsert, Update, Insert|[Account docs](https://apolloio.github.io/apollo-api-docs/?shell#accounts-api)|
|**Contacts**|Sync data from any source to a contact|Upsert, Update, Insert|[Contact docs](https://apolloio.github.io/apollo-api-docs/?shell#contacts-api)|
|**Sequences**|Sync data from any source to update membership in Apollo sequences|Insert, Remove|[Sequence docs](https://apolloio.github.io/apollo-api-docs/?shell#sequences-api)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Apollo.io
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Apollo.io** and click **Continue**.
You can then authenticate Hightouch to Apollo.io with by entering an Apollo.io **API Key**.
Follow [these instructions](https://knowledge.apollo.io/hc/en-us/articles/4415734629773-Configure-Access-to-the-Apollo-REST-API) to create one.
Because Apollo.io API key scopes are based on endpoints and not permissions, you must create a master key to use the Hightouch integration.
## Sync configuration
Once you've set up your Apollo.io destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Apollo.io destination you want to sync to.
### Syncing accounts
Account objects represent companies in Apollo.io and can be used as leads or opportunities.
#### Record matching
To match source rows to contacts in Apollo.io, you need to select a source column and corresponding Apollo.io field. You can match on any of the following Apollo.io fields:
- Name
- ID
#### Field mapping
You can sync columns from your source to Apollo.io default and custom fields. For custom picklist fields, you can send display values, and Hightouch remaps the value to Apollo.io's internal ID.
### Syncing contacts
Contact objects are people in Apollo.io and are used as the point-of-contact for leads and opportunities.
#### Record matching
To match source rows to contacts in Apollo.io, you need to select a source column and corresponding Apollo.io field. You can match on any of the following Apollo.io fields:
- Email (recommended)
- ID
- Name
- Title
- Company
#### Field mapping
You can sync columns from your source to Apollo.io default and custom fields. For custom picklist fields, you can send display values, and Hightouch remaps the value to Apollo.io's internal ID.
### Syncing sequences
Sequences are outreach campaigns with any number of sequential contact points and tasks that you can customize to personally engage your target audiences at scale.
#### Record matching
To match rows from your model to records in Apollo.io, you need to select the model column that contains values that match the **ID** Apollo.io field.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Appcues
**URL:** https://hightouch.com/docs/destinations/appcues
**Description:** Push user behavior data from your warehouse into Appcues
**Section:** Destinations
## Setup
After selecting Appcues from our destinations catalog, you will be required to enter your Appcues _Account ID_ (found in _Appcues_ > _Settings_ > [_Account_](https://studio.appcues.com/settings/account)) and _API Key / API Secret_ (found in _Appcues_ > _Settings_ > [_API Keys_](https://studio.appcues.com/settings/keys)).
Hightouch supports syncing both `Objects` and `Events` to Appcues.
## Syncing objects
Hightouch supports syncing to the following Appcues objects:
- `Users`
### Sync modes
Hightouch supports the following sync modes:
- **Upsert** - pushes new `User` objects to Appcues and updates properties that change in your warehouse.
### Record matching
`User` records can be matched from your source to your Appcues workspace by User ID (`user_id`), which is a unique identifier
in Appcues for each `User`.
### Field mapping
You may also sync columns from your source to Appcues's `User` properties. Appcues doesn't provide information on properties set for `User` objects,
so you will need to type the exact name of the property you would like to map to in Appcues. These property names must be spelled correctly and are case-sensitive.
## Sending events to Appcues
The Appcues integration also allows you to send events to Appcues with custom attributes.
The three required elements of any Appcues event are the _Event Name_ (which can be a single string used to name all events, or pulled from your warehouse per event), a _Timestamp_ (in ISO 1806 format, or a Unix timestamp in **milliseconds**), and a **User ID** (the unique ID of the user corresponding to the event).
You can also map properties that will become event `attributes` in Appcues.
---
## AppLovin
**URL:** https://hightouch.com/docs/destinations/applovin
**Description:** AppLovin is a mobile technology company that provides app developers with tools to grow, monetize, and analyze their apps through user acquisition, in-app bidding, and ad mediation. Its platform uses machine learning to optimize ad delivery and maximize revenue across a global network.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-----------------|----------------------------------------------------|----------------------|-------------------------------------------------------------------------------------------
**Suppressions** | Sync data from any source to AppLovin suppressions | Insert | [Suppressions docs](https://developers.applovin.com/en/app-discovery/api/suppression-api/)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to AppLovin
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **AppLovin** and click **Continue**. You can then authenticate Hightouch to **AppLovin**.
Enter the following fields into Hightouch:
- **API key**: API key (or Management key) for authentication. Find it in the [Keys tab](https://www.applovin.com/analytics#keys) of your account page. If the Management Key is not visible, [open a ticket](https://support.applovin.com/hc/en-us/requests/new) with AppLovin and request one be generated for you.
## Sync configuration
Once you've set up your AppLovin destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the AppLovin destination you want to sync to.
### Syncing suppressions
Sync data from any source to AppLovin suppressions.
#### Package Name
Specify the package name you would like to use with the AppLovin suppression API. The package name can be a bundle ID (iOS) or a general package name (Android) of your app.
#### Record matching
You can match rows from your model to suppressions in AppLovin on any column in your model. Ensure the data types of the model column and the AppLovin Device UUID field match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
The AppLovin suppressions API only accepts UUID format device IDs. Ensure your device IDs are the correct format.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
------------------------|------------------------------------------------------------
**Do nothing** | Keep the suppression in AppLovin
**Remove from list** | Remove the suppression from AppLovin
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## AppsFlyer
**URL:** https://hightouch.com/docs/destinations/appsflyer
**Description:** Give your team visibility into campaign performance, scale app installs, and maximize customer LTV with a new standard of privacy-enhancing measurement and deep linking solutions.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
|-----------------------------|-------------------------------------------------------------|----------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|
| **Server-to-server events** | Sync mobile events that occur outside the app to AppsFlyer | Insert | [Server-to-server events docs](https://dev.appsflyer.com/hc/reference/post_s2s_inappevent) |
| **Audiences** | Import predefined audience segments as lists into AppsFlyer | Add, Remove | [Import audience docs](https://support.appsflyer.com/hc/en-us/articles/360013291857-Audiences-import-an-audience#use-the-import-audience-api) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to AppsFlyer
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **AppsFlyer** and click **Continue**. You can then authenticate Hightouch to AppsFlyer by entering the following fields depending on your use case:
To sync S2S events:
- **AppsFlyer V2 S2S key** to use AppsFlyer's `api3` events endpoint. Follow [these instructions in AppsFlyer's docs](https://support.appsflyer.com/hc/en-us/articles/360004562377-Managing-AppsFlyer-tokens) to create a S2S key.
- **Dev key** *(legacy)* to use AppsFlyer's `api2` events endpoint. As of December 2023, AppsFlyer has deprecated `api2`. We recommend updating your configuration to use the S2S key for best performance. If you choose to use a dev key, follow [these instructions in AppsFlyer's docs](https://support.appsflyer.com/hc/en-us/articles/211719806-App-settings-#sdk-authentication-dev-key)
For more information about upgrading from `api2` to `api3`, see AppsFlyer's announcement [here](https://support.appsflyer.com/hc/en-us/articles/20509378973457-Bulletin-Upgrading-the-AppsFlyer-S2S-API).
To sync to audiences:
- **AppsFlyer V2 API key**: You can create a V2 API key by following [these instructions](https://support.appsflyer.com/hc/en-us/articles/360004562377-Managing-API-and-Server-to-server-S2S-tokens#manage-your-tokens).
You only need to provide credentials for the sync type you intend to use. For example, if you would only like to sync S2S events, you do not need to provide an API key but are required to provide a S2S key.
## Sync configuration
Once you've set up your AppsFlyer destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the AppsFlyer destination you want to sync to.
### Syncing server-to-server events
You can use server-to-server events to send mobile events that occur outside the app from your servers to AppsFlyer.
Appsflyer's API requires the following event parameters:
- `app_id`
- `eventName`
- `eventValue`
The sync configuration form ensures all these are set and provides some additional options.
#### Appsflyer app ID
The app ID is the identifier used in the AppsFlyer dashboard. Insert it exactly as it appears on the dashboard. You can either provide a static value or select to **use a column** from your model. Hightouch syncs the static or column value as the [`app_id` path parameter](https://dev.appsflyer.com/hc/reference/post_s2s_inappevent#path-params) the API requires.
#### Event name
Providing an event name is required to send an event to the Appsflyer API. You can either provide a static value or select to **use a column** from your model. Hightouch syncs the static or column value as the [`eventName` body parameter](https://dev.appsflyer.com/hc/reference/post_s2s_inappevent#path-params) the API requires.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Appsflyer uses the time the event arrives at the server.
#### Field mapping
Hightouch lets you map model columns to event metadata. Some of the metadata fields you can map to include:
- AppsFlyer ID
- IDFV
- IP
- Operating system
The AppsFlyer ID is a required field, so ensure you include that in your mapping.
#### Event value mapping
Each event _must_ contain a JSON `eventValue`, for example:
```json
{
"af_revenue": "6",
"af_currency": "USD",
"af_content_type": "wallets",
"af_content_id": "15854",
"your_custom_attribute" :"456.123"
}
```
Hightouch lets you map these fields to create the JSON object the `eventValue` field expects.
#### Custom mappings
You can also map additional custom data that Hightouch sends to the AppsFlyer platform as JSON using the `custom_data` field.
### Syncing audiences
This sync type lets you update which users belong to an audience.
You must use an existing audience in AppsFlyer.
#### Record matching
To identify which users to add or update in an audience, select model columns and the corresponding AppsFlyer fields. You can match on the following fields:
- App ID
- IDFV
- IDFA
- GAID
- OAID
- IMEI
- CUID
- Email
- Phone
- Braze ID
- Amplitude ID
You must include the App ID for all audience types.
Depending on whether you're syncing an Android or iOS audience, you need to include at least one additional identifier.
For Android audiences, you must also include at least one of the following:
- GAID
- OAID
- IMEI
- CUID
- Braze ID
- Amplitude ID
For iOS audiences, you must also include at least one of the following:
- IDFV
- CUID
- Braze ID
- Amplitude ID
See [AppsFlyer's docs](https://support.appsflyer.com/hc/en-us/articles/360013291857#identifier-hash) on identifiers for more information.
#### Select an existing audience or create a new one
You can create a new audience or use an existing one. When creating a new audience, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model. To use an existing audience, you need to enter the audience's `import_key`.
To get an audience's `import_key` follow these instructions:
1. In the AppsFlyer left-side panel, go to **Integration > Audiences**.
2. Click on the name of the audience you want to update.
3. Select the **Import** tab.
4. In the **Import using API** section, choose **Add new devices/users**
5. Copy the `import_key` in the **API request body**.
For more information about the **Import Audience API**, refer to [AppsFlyer's docs](https://support.appsflyer.com/hc/en-us/articles/360013291857-Audiences-import-an-audience?query=import%20key).
#### Handling PII and hashing
By default, Hightouch automatically hashes fields that require it before sending them to Appsflyer.
You can disable this behavior in the sync configuration. If disabled, the data from the model should be appropriately normalized and hashed according to [Appsflyers's hashing requirements](https://support.appsflyer.com/hc/en-us/articles/360017807278-Audiences-additional-identifiers).
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when an audience member no longer appears in your model's query results. You have the following options:
| Behavior | Description |
|----------------|-----------------------------------|
| **Do nothing** | Keep the user in the audience |
| **Delete** | Remove the user from the audience |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Asana
**URL:** https://hightouch.com/docs/destinations/asana
**Description:** Keep your team's projects and tasks up-to-date with fresh customer data
**Section:** Destinations
## Overview
With the Asana destination, Hightouch can upsert or update tasks, or update projects.
For the task object, the destination supports:
- **Upsert**: Upsert mode creates new tasks and updates existing tasks based on an external ID.
- **Update**: Update mode updates existing or newly created tasks based on the Asana GID. The update mode can also update tasks created with Hightouch using the external ID.
For the projects object, the destination supports:
- **Update**: Update mode updates existing projects based on the Asana GID.
## Setup
Connect Asana via OAuth by granting permission to the Hightouch Asana Connect app.
## Tasks
### Upsert
Hightouch can upsert tasks based on an external ID in Asana. This means it will update the task in Asana if it was previously created by Hightouch, otherwise it will create a new task. The external ID is unique to Hightouch thus in this mode it will only update tasks created via Hightouch.
| Name | Description |
| :--------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Hightouch to Asana Merge Rule | The column to use as the external ID in Asana. This should be a unique identifier and will most likely be some form of ID. |
| Hightouch to Asana Field Mappings | A list of columns that will be pushed to tasks in Asana. Each time a sync occurs in Hightouch these columns will be updated. For array properties such as projects or followers, [refer to notes on array properties.](#array-properties) |
| Hightouch to Asana Custom Field Mappings | A list of custom fields that will be pushed to tasks in Asana. [Refer the notes on custom fields in Asana.](#custom-fields) |
### Update
Hightouch can update tasks based on the Asana GID. Hightouch can also update tasks based on an external ID if the task was created using Hightouch.
If upserting or updating via an external ID, Hightouch can only update tasks
created with such an external ID through Hightouch, due to the way Asana
[external IDs are limited on a per application
basis](https://developers.asana.com/docs/custom-external-data).
| Name | Description |
| :--------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Hightouch to Asana Merge Rule | The column that matches either the Asana GID or external ID in Asana. |
| Hightouch to Asana Field Mappings | A list of columns that will be pushed to tasks in Asana. Each time a sync occurs in Hightouch these columns will be updated. For array properties such as projects or followers, [refer to notes on array properties.](#array-properties) |
| Hightouch to Asana Custom Field Mappings | A list of custom fields that will be pushed to tasks in Asana. [Refer the notes on custom fields in Asana.](#custom-fields) |
## Projects
Hightouch can update projects on the Asana GID. Unfortunately, Hightouch can't upsert projects or update based on external ID because Asana doesn't yet support external metadata for objects other than projects.
| Name | Description |
| :--------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Hightouch to Asana Merge Rule | The column that matches the Asana Project GID. |
| Hightouch to Asana Field Mappings | A list of columns that will be pushed projects in Asana. Each time a sync occurs in Hightouch these columns will be updated. For array properties such as followers, [refer to notes on array properties.](#array-properties) |
| Hightouch to Asana Custom Field Mappings | A list of custom fields that will be pushed to projects in Asana. [Refer the notes on custom fields in Asana.](#custom-fields) |
## Array properties
Array properties can be passed to Asana by returning the data as either a comma delimited string or an array in your SQL query. Examples of array properties are:
- Tasks: projects, followers, tags
- Projects: followers
In the SQL query, a column `taskFollowers`containing a string such as `'1199948419683571,1185085322202311'` can be mapped to the followers field of tasks in Asana.
Using the example of followers, keep in mind that every type Hightouch runs a sync, it will check the followers assigned to the task in Asana, and:
- Add all followers that are present in `taskFollowers` and not present in Asana
- Remove all followers that are present in Asana and not present in `taskFollowers`.
## Subtasks
A `subtask` is an Asana task object that is associated with a `parent` task object.
To create a `subtask`, select the column from your source that references `parent` tasks, then map it to the `parent` field in
the Mappings section.
In your source, you can identify the `parent` task either by the external ID or the Asana GID. The external ID or the Asana GID should
be a string. Not passing any value in for the `parent` field will remove an existing parent-subtask association - the task will simply be considered a normal `task` instead of a `subtask`.
## Custom fields
Custom fields are complex in Asana because they can be associated with both tasks and projects, and they can be restricted to only tasks in certain projects. To start, Hightouch supports custom fields that are available on a workspace level, which can be done by adding the field to the workspace's field library in Asana.
Secondly, custom fields may cause errors if the field is not enabled in the project you are inserting the task into.
## Delete behavior
For the task object, you can choose what Hightouch's behavior is when rows leave your query results. The default is doing nothing.
In **Upsert** mode, you have the following options:
| Behavior | Description |
| ----------------------------- | ------------------------------------------ |
| **Do nothing** | Keep the task in Asana |
| **Mark As Complete** | Keep the task but mark as complete |
| **Delete Destination Record** | Delete the task from Asana completely |
In **Update** mode, you have the following options:
| Behavior | Description |
| -------------------- | ----------------------------------------- |
| **Do nothing** | Keep the task in Asana |
| **Mark As Complete** | Keep the task but mark as complete |
---
## Attentive
**URL:** https://hightouch.com/docs/destinations/attentive
**Description:** Attentive empowers the most innovative brands to create meaningful interactions through personalized text messaging
**Section:** Destinations
## Overview
This destination lets you push data from various sources into Attentive to maintain an up-to-date subscriber list for mobile messaging campaigns. Add attributes to subscribers in Attentive to build better segments for
personalized experiences and create user-associated events to trigger targeted workflows.
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| --------------------- | -------------------------------------------------- | -------------------- | ---------------------------------------------------------------------------------------- |
| **Identity** | Associate user IDs with other identifiers | Update | [Identity](https://docs.attentive.com/openapi/reference/tag/Identity/) |
| **Custom Events** | Send any event-based data associated with a user | Insert | [Custom Events](https://docs.attentive.com/openapi/reference/tag/Custom-Events/) |
| **Custom Attributes** | Add customizable data or characteristics to a user | Upsert | [Custom Attributes](https://docs.attentive.com/openapi/reference/tag/Custom-Attributes/) |
| **Subscribers** | Subscribe and unsubscribe users from subscriptions | Update | [Subscribers](https://docs.attentive.com/openapi/reference/tag/Subscribers/) |
## Getting started
To use the Attentive destination in Hightouch, you will need to generate an API key within Attentive.
To generate the API key, follow these [steps](https://docs.attentive.com/pages/authentication/).
Ensure that the generated key has the correct scopes for the endpoints you wish to use:
- **Identity:** `identity:write`
- **Custom Events:** `events:write`
- **Custom Attributes:** `attributes:write`
## Syncing data
### Identity
Syncing to Attentive identities lets you associate a client user identifier or custom identifier with other identifiers. A client user or custom identifier must be accompanied by at least one of the following identifiers:
`phone`, `email`, `Shopify ID`, `Klaviyo ID`, `client user ID`, or `custom identifier`.
Refer to the [Attentive Identity API docs](https://docs.attentive.com/openapi/reference/tag/Identity/) to learn more.
#### Matching users
You can match rows in your model with users in Attentive on the following fields:
- Email
- Phone number
- Client User ID
- Shopify ID
- Klaviyo ID
#### Field mapping
You can map other identifiers to associate with the primary identifier, including external IDs not natively supported by Attentive.
### Custom events
Syncing custom events lets you send user actions to use in the Attentive Segment Builder and the Journey Builder.
This data can't contain any sensitive or special categories of information as defined in applicable data
protection and privacy laws, including the California Consumer Privacy Act (CCPA) and California Privacy Rights Act (CPRA).
Refer to the [Attentive Custom Events API docs](https://docs.attentive.com/openapi/reference/tag/Custom-Events/) to learn more.
#### Associating users
You can use the following fields to map associated events with users:
- Email
- Phone number
- Client User ID
- Shopify ID
- Klaviyo ID
- Custom Identifiers
#### Event types
The event type can be a static value or a field from the model to send multiple different event types.
The event type name is case-sensitive. For example, "Order shipped" and "Order Shipped" are different event types.
#### Event universal unique identifier
Hightouch allows you to send an event universal unique identifier (UUID) for each event. Though optional, using a UUID is [recommended by Attentive](https://docs.attentive.com/openapi/reference/tag/Custom-Events/#tag/Custom-Events/operation/postCustomEvents).
#### Timestamps
Hightouch allows you to send a timestamp of when a custom event occurred in ISO 8601 format.
If the timestamp is older than 12 hours, relevant Journeys aren't triggered in Attentive.
If an event doesn't include a timestamp, the time of the API request is used.
#### Event properties
You can use properties to send metadata associated with the event.
Object keys are expected to be strings and can't contain any of the following special characters:
- Double quote (`"`)
- Curly braces (`{ }` )
- Square brackets (`[ ]`)
- Backslash (`\`)
- Vertical bar (`|`)
### Custom attributes
Syncing custom attributes lets you send attribute-based user data to Attentive. You can create as many custom attributes as needed. Note that you can create net-new properties with this API, however,
it can't be used to create new values for an existing UI-created property name. Whenever you create a property name on the Attentive platform, you must also define all possible property values there. Refer to the [Attentive Attributes API docs](https://docs.attentive.com/openapi/reference/tag/Custom-Attributes/) to learn more.
#### Associating users
You can associate custom attributes with users on the following fields:
- Email
- Phone number
- Client User ID
- Shopify ID
- Klaviyo ID
- Custom Identifiers
#### User attributes
User attributes are any metadata associated with a user. Attribute object keys should be strings, and object values can be any type. Both object keys and object values are case-sensitive. "Favorite color" and "Favorite Color" are considered different custom attributes.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Attio
**URL:** https://hightouch.com/docs/destinations/attio
**Description:** Push enriched data from your warehouse into Attio
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ------------------ | ------------------------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------ |
| **Companies** | Sync data from any source to companies in Attio | Upsert | [Companies docs](https://developers.attio.com/reference/put_v2-objects-companies-records) |
| **Custom objects** | Sync data from any source to custom objects in Attio | Upsert | [Custom objects docs](https://attio.com/developers/rest/record#assert-a-record) |
| **Deals** | Sync data from any source to deal objects in Attio | Upsert | [Deal docs](https://developers.attio.com/reference/put_v2-objects-deals-records) |
| **People** | Sync data from any source to people in Attio | Upsert | [People docs](https://developers.attio.com/reference/put_v2-objects-people-records) |
| **Users** | Sync data from any source to user objects in Attio | Upsert | [User docs](https://developers.attio.com/reference/put_v2-objects-users-records) |
| **Workspaces** | Sync data from any source to workspace objects in Attio | Upsert | [Workspace docs](https://developers.attio.com/reference/put_v2-objects-workspaces-records) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Attio
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Attio** and click **Continue**.
You will be required to enter your Attio API key. You can locate this API key within your Attio account by navigating to Workspace Settings > Developers:
## Sync configuration
Hightouch supports syncing to the following Attio objects:
- `Companies`
- `Custom objects`
- `Deals`
- `People`
- `Users`
- `Workspaces`
### Sync modes
Hightouch supports the following sync modes:
- **Upsert** - pushes new objects to Attio and updates fields that change in your warehouse.
### Record matching
Depending on the object you are syncing, records can be matched from your source to your Attio workspace by `domains` for companies, `email_addresses` for people, or other read-only fields for custom objects.
### Field mapping
You may also sync columns from your source to Attio's object properties.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Auth0
**URL:** https://hightouch.com/docs/destinations/auth0
**Description:** Sync user lists to Auth0 and automate and manage all of your passwords, grants, and permissions in a single platform
**Section:** Destinations
## Setup
After selecting Auth0 from our Destinations catalog, you will be prompted to enter
your _Auth0 domain_, _Client ID_, and _Client secret_ from your **registered
machine-to-machine (M2M) application for Hightouch**. You should find this by navigating
to your Auth0 dashboard -> Applications dropdown -> Applications -> select corresponding
Hightouch M2M. This particular M2M will give Hightouch permission to connect with your
Management API and update your users.
### Registering your M2M for Hightouch
If you do not have a registered M2M for Hightouch, refer to
[this Auth0 doc](https://auth0.com/docs/get-started/create-apps/machine-to-machine-apps) on
how to get started. You will then need to select the following configuration:
- Select the **Auth0 Management API**
- Check the **update:users** permission.
After you authorize, you should go to the M2M's settings to obtain the required keys.
Keep in mind you might have to go back and add to the permissions for
potential future features.
## Syncing rows
Hightouch supports syncing to the following Auth0 objects:
- `Users`
User objects in Auth0 will use the external ID as its own ID. Therefore, the user's source ID
(that is, external ID) must be present to properly match the user.
### Sync modes
This integration only supports **Update** mode.
In this mode, the destination will update the mapped attributes and metadata to the user objects.
It will not add new users.
### Record matching
Records can be matched from your source to Auth0 workspace by the external user's ID only.
### Field mapping
You can sync columns from your source to Auth0's user attributes, including the app and user's metadata.
You can view this [list for the supported Auth0 user attributes](https://auth0.com/docs/users/user-profiles/user-profile-structure).
The app and user metadata holds custom data so can take any shape or form.
---
## Awin
**URL:** https://hightouch.com/docs/destinations/awin
**Description:** Empower advertisers and publishers with Awin's affiliate marketing platform to grow their businesses online.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-----------|------------------------------------------|----------------------|-----------------------------------------------------------------
**Orders** | Sync data from any source to Awin orders | Insert | [Orders docs](https://developer.awin.com/apidocs/conversion-api)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Required Permissions
Your AWIN account should have `Full Access` or `Admin` level [permissions](https://success.awin.com/s/article/how-can-i-manage-user-permissions?language=en_US) to use the Conversion API to manage orders. If you don't have either of these permission levels, contact your Organization Administrator to request access.
## Connect to Awin
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Awin** and click **Continue**. You can then authenticate Hightouch to **Awin**.
Enter the following fields into Hightouch:
- **OAuth 2.0 Token**: Find your OAuth 2.0 Token under API Credentials in the User dropdown.
- **Advertiser Id**: Find your Advertiser Id on the Overview page for your Profile.
## Sync configuration
Once you've set up your Awin destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Awin destination you want to sync to.
### Syncing orders
Sync data from any source to Awin orders.
#### Field mapping
Hightouch lets you sync order properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## AWS Lambda
**URL:** https://hightouch.com/docs/destinations/aws-lambda
**Description:** Build your own custom destination using a serverless function
**Section:** Destinations
## Overview
This destination invokes an [AWS Lambda](https://aws.amazon.com/lambda/) function whenever data changes in your model. It makes it possible to build your own custom integrations that execute code written in Python, Java, Go, PowerShell, Node.js, C#, or Ruby.
This destination was designed to be as flexible as possible. You can exercise granular control over function triggers, batching, rate limits, concurrency limits, and even error handling. Together, these features let you integrate Hightouch with any internal tool or third-party API.
Under the hood, the AWS Lambda destination uses Hightouch's powerful sync engine, so you continue to benefit from the security and observability features available in our native SaaS destinations.
Example use cases include:
- Syncing data to web APIs not yet natively supported
- Enriching data using external sources like Clearbit and ZoomInfo
- Transforming and filtering data using code instead of SQL
## Getting started
### Connect to your AWS account
When setting up the AWS Lambda destination for the first time, you need to enter your **AWS Credentials** to give Hightouch access to your AWS account. Hightouch needs permission to invoke Lambda functions on your behalf.
Hightouch believes in the principle of least privilege. We ask for no more permissions than absolutely necessary. For the AWS Lambda destination, Hightouch requires `lambda:InvokeFunction` for the specific function or functions you want to invoke.
When configuring your credentials in Hightouch, you have two options:
- Setting up a cross-account role (recommended)
- Providing an access key
#### Set up cross-account role
Cross-account roles are the most secure method for granting Hightouch access to specific Lambda functions in your AWS account.
You need the **Account ID** and **External ID** in the Hightouch UI to set up a cross-account role.
1. In the **AWS Console**, navigate to **IAM** → **Roles**.
1. Click **Create Role**.
1. Under **Trusted entity type**, choose **AWS account**.
1. Select **Another AWS account** and also click **Require external ID**.
1. Copy/paste the **Account ID** from Hightouch into the **Account ID** field in AWS.
1. Copy/paste the **External ID** from Hightouch into the **External ID** field in AWS.
1. On the **Add permissions** screen, proceed to attach permissions to the role using any policy that includes at least `lambda:InvokeFunction`. Then create the role.
1. Copy/paste the **Role ARN** from AWS into Hightouch. Click **Create** to finish setting up your cross-account role.
#### Provide access key
If you don't want to create a cross-account role, you can create a regular IAM user and share a **Access Key ID** and **Secret Access Key** with Hightouch.
Be sure to attach the user to a permission policy that includes `lambda:InvokeFunction` for the specific function or functions you want to invoke via a Hightouch sync.
An example policy looks like this:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["lambda:InvokeFunction"],
"Resource": [
"YOUR_LAMBDA_ARN_1",
"YOUR_LAMBDA_ARN_2",
"YOUR_LAMBDA_ARN_3"
]
}
]
}
```
## Syncing data
Once you've authenticated your AWS account in Hightouch, you've completed setup for a Lambda destination in your Hightouch workspace. The next step is to configure a sync that invokes your function whenever rows are added, changed, or removed in your model.
### Specify your Lambda function
To get started, you need the ARN of the Lambda function you want to invoke. You can find the ARN in the AWS web console.
When providing an ARN, you can use either a qualified ARN or an unqualified ARN. A qualified ARN has a version suffix, for example, `arn:aws:lambda:aws-region:acct-id:function:helloworld:42`. Unqualified ARNs don't have a version suffix, for example, `arn:aws:lambda:aws-region:acct-id:function:helloworld`.
Enter your ARN into your sync configuration in Hightouch.
### Choose your function triggers
Hightouch monitors your data model for added, changed, and/or removed rows. In this step, you specify which of these events should invoke your function.
Suppose you want to use AWS Lambda to send a confirmation email whenever a customer places a new order or updates an existing order. Your model might look something like `SELECT * FROM orders`. The Lambda function should be triggered whenever rows are added—for example, when customers place new orders—or whenever rows change—when orders are updated. Therefore, you would want to enable the **Rows added** and **Rows changed** triggers. When invoking your function, Hightouch passes along metadata about *why* the function was invoked.
All function invocations are synchronous, meaning that Hightouch invokes your Lambda function and waits for its response.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows, such as hydrating a CRM for the first time, may require performing a backfill of all rows during the initial sync. For other use cases, such as sending confirmation emails, you might only want to invoke your function in response to future data changes.
### Configure batch size
By default, Hightouch separately invokes your Lambda function for each added, changed, or removed row.
Certain high-throughput use cases may require batching together multiple rows in the payload. Hightouch supports batches of up to 1000 rows per function invocation.
Batching isn't recommended unless absolutely necessary for performance reasons. Enabling this feature requires your Lambda function to be either idempotent or significantly more fault tolerant in the event of a partial batch failure.
### Configure rate and concurrency limits
In this step, you declare whether function invocations should be throttled.
To determine the ideal limits, you should consider whether your function interacts with any downstream services that have limits of their own.
Most modern web APIs enforce rate limits, which set a maximum allowed number of requests per second, minute, or hour. Occasionally, APIs may also have concurrency limits, which set a maximum allowed number of requests that can be processed simultaneously. Rate limits and concurrency limits both affect overall sync speed.
### Configure error handling
Lambda functions can fail for many reasons. Hightouch's retry logic for this destination doesn't discriminate between invocation errors, for example, missing permissions, runtime errors, for example, due to syntax issues, or function errors, for example, due to uncaught exceptions. Hightouch retries all errors eventually.
In this step, you decide whether errors should be retried immediately or during the next sync run. If you choose to retry immediately, you can specify how many retries should be attempted during the sync run. If all these retries fail, Hightouch retires the request during subsequent sync runs until it succeeds.
If an error is gracefully caught in your function, you can flag it as a rejected row and provide a custom error message using the response format specified in the [error handling section](#configure-error-handling).
You can also include retry logic inside of your Lambda function. After exhausting all retry attempts during a function invocation, respond with an error so that Hightouch knows to reinvoke the function later.
### Invocation payload schema
The payload, also known as the "event document," contains your row data in JSON format, along with additional metadata. When writing your Lambda function, you can assume that all function invocations include a payload following this schema:
```json
{
"operation": XXXXX,
"primary_key_column": XXXXX,
"rows": [
{
XXXXXX
}
],
"metadata": {
"api_version": XXXXX,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
#### Invocation payload properties
##### `operation`
The row operation, also known as the trigger, explains *why* your function was invoked.
Its possible values are these strings: `"add"`, `"change"`, and `"remove"`.
##### `primary_key_column`
This string refers to the column name of the primary key for your Hightouch model.
##### `rows`
This is an array of objects, with each object containing row data from your Hightouch model.
##### `metadata`
This is an object containing three key/value pairs:
- `api_version` (integer): currently set to `1`, this identifier only changes if the payload schema is modified in the future
- `sync_id` (integer): this is the unique identifier for the Hightouch sync configuration associated with the function invocation
- `sync_run_id` (integer): this is the unique identifier for the specific sync run associated with the function invocation
#### Example invocation payload
Suppose a new row is added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `john.doe@example.com` | John | Doe |
The payload for the function invocation would look like this:
```json
{
"operation": "add",
"primary_key_column": "customer_id",
"rows": [
{
"customer_id": "928713",
"email": "john.doe@example.com",
"first_name": "John",
"last_name": "Doe"
}
],
"metadata": {
"api_version": 1,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
Note that `rows` is an **array** containing a single element.
#### Example invocation payload for batching
Suppose three new rows are added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `alice.doe@example.com` | Alice | Doe |
| 283743 | `bob.doe@example.com` | Bob | Doe |
| 162352 | `carol.doe@example.com` | Carol | Doe |
The payload for the function invocation would look like this:
```json
{
"operation": "add",
"primary_key_column": "customer_id",
"rows": [
{
"customer_id": "928713",
"email": "alice.doe@example.com",
"first_name": "Alice",
"last_name": "Doe"
},
{
"customer_id": "283743",
"email": "bob.doe@example.com",
"first_name": "Bob",
"last_name": "Doe"
},
{
"customer_id": "162352",
"email": "carol.doe@example.com",
"first_name": "Carol",
"last_name": "Doe"
}
],
"metadata": {
"api_version": 1,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
When batching is enabled, rows sharing a common operation type—`add`, `change`, `remove`: are batched together. Function invocations always represent exactly one operation type.
### Response payload schema
If your Lambda function fails to process any rows, it can respond with an array of "rejected rows" that encountered errors. Each rejected row should be identified by its primary key/value and may be associated with an optional error message. Hightouch retries these rows.
If there is an error, the response payload looks like this:
```json
{
"errors": [
{
"primary_key_value": XXXXX,
"reason": XXXXX
}
]
}
```
#### Response payload properties
Errors in the `errors` array have these properties:
##### `primary_key_value`
This identifies the rejected row that needs to be retried.
##### `reason` (optional)
This string may represent an error message or any other information that would be helpful for debugging and monitoring sync health.
#### Example success response payload
If all rows are processed successfully, your function responds with an empty payload.
```json
{}
```
#### Example response payload for rejected rows
Suppose three new rows are added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `alice.doe@example.com` | Alice | Doe |
| 283743 | `bob.doe@example.com` | Bob | Doe |
| 162352 | `carol.doe@example.com` | Carol | Doe |
While processing these rows, your Lambda function encounters two errors:
- Alice's email can't be found in the downstream service
- Bob's email already exists in the downstream service
To surface these errors in Hightouch, your function should respond like this:
```json
{
"errors": [
{
"primary_key_value": "928713",
"reason": "Email not found"
},
{
"primary_key_value": "283743",
"reason": "Email already exists"
}
]
}
```
The primary key values (`928713` and `283743`) refer to `customer_id` values. The `customer_id` column was designated the `primary_key_column` in the function invocation payload.
#### Example response payload for other errors
If your function experiences an invocation error, for example, exceeding AWS Lambda rate limits, runtime error, for example, due to syntax issues, or function error, for example, due to uncaught exceptions, all rows in the batch are automatically marked as rejected rows.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Azure Blob Storage
**URL:** https://hightouch.com/docs/destinations/azure-blob
**Description:** Transfer data at scale from your warehouse to Azure Blob Storage
**Section:** Destinations
## Overview
With Hightouch, you can export all the data in your warehouse to JSON, CSV, or Parquet files in any of your Azure Blob Storage containers. Whether it's for internal processes, analytics, or product data, our Azure Blob Storage integration allows you to securely and easily transfer data at web-scale.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------ | ---------------------------------------------------------------------------- | -------------------- |
| Any data set | Sync data from a source to Azure Blob Storage as JSON, CSV, or Parquet files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Diff mode creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Azure Blob Storage
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Azure Blob Storage** and click **Continue**. You can then authenticate Hightouch to **Azure Blob Storage** using Azure credentials. You can select credentials you previously created or click **New credentials** to set them up.
See the guide for [configuring Azure credentials](/security/azure) to learn how you can set up a service account and use its credentials. The IAM user whose credentials you use must have programmatic access enabled and permission to write to the Azure Blob Storage container you want to use.
Next, enter your Blob Storage **Account name** and **Container name** to complete setup.
You can find these in the [Azure Portal](https://portal.azure.com/).
Go to **Storage accounts** and open the storage account you want to use.
The account name is in the top left above **Storage account**.
You can see your container names once you open **Containers** in the left side menu.
If you want to create a new container for Hightouch to sync to, you need [create it](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-portal#create-a-container) before you can proceed.
## Sync configuration
Once you've set up your Azure Blob Storage destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Azure Blob Storage destination you want to sync to.
### Select file format
Hightouch supports syncing JSON, CSV, and Parquet files to Azure Blob Storage.
### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it. To view different versions of the same file in Azure Blob Storage, you can right-click the file and select **View versions**.
If you are using an audience and would like to include the audience name, you will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter `-86400` (`24 hours * 60 minutes * 60 seconds`). If you want the filename's data to be one hour after the sync takes place, you would enter `3600` (`60 minutes * 60 seconds`).
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `first_name`, `email`, and `last_name` fields. Hightouch exports these fields to new fields in the file and ignores all other columns from your results.
## Tips and troubleshooting
### Common errors
### Sync alerts
---
## Azure Functions
**URL:** https://hightouch.com/docs/destinations/azure-functions
**Description:** Build your own custom destination using a serverless function
**Section:** Destinations
## Overview
This destination invokes a [Microsoft Azure](https://azure.microsoft.com/en-us/products/functions/) function whenever data changes in your model. It makes it possible to build your own custom integrations that execute code written in any language that Azure functions [supports](https://learn.microsoft.com/en-us/azure/azure-functions/supported-languages).
This destination was designed to be as flexible as possible. You can exercise granular control over function triggers, batching, rate limits, concurrency limits, and even error handling. Together, these features let you integrate Hightouch with any internal tool or third-party API.
Under the hood, the Azure functions destination gives you Hightouch's powerful sync engine, so you continue to benefit from the security and observability features available in our native SaaS destinations.
Example use cases include:
- Syncing data to web APIs not yet natively supported
- Enriching data using external sources like Clearbit and ZoomInfo
- Transforming and filtering data using code instead of SQL
## Getting started
### Connect to your Azure functions
When setting up the Azure functions destination for the first time, you need to enter your function's [URL](https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-http-webhook-trigger?tabs=in-process%2Cfunctionsv2&pivots=programming-language-javascript#customize-the-http-endpoint) and it's [access key](https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-http-webhook-trigger?tabs=in-process%2Cfunctionsv2&pivots=programming-language-javascript#authorization-keys) for authorizing invocations.
## Syncing data
Once you've connected to your Azure function in your Hightouch workspace, the next step is to configure a sync that invokes your function whenever rows are added, changed, or removed in your model.
### Choose your function triggers
Hightouch monitors your data model for added, changed, and/or removed rows. In this step, you specify which of these events should invoke your function.
Suppose you want to use an Azure function to send a confirmation email whenever a customer places a new order or updates an existing order. Your model might look something like `SELECT * FROM orders`. The Azure function should be triggered whenever rows are added—for example, when customers place new orders—or whenever rows change—when orders are updated. Therefore, you would want to enable the **Rows added** and **Rows changed** triggers. When invoking your function, Hightouch passes along metadata about *why* the function was invoked.
All function invocations are synchronous, meaning that Hightouch invokes your Azure function and waits for its response.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows, such as hydrating a CRM for the first time, may require performing a backfill of all rows during the initial sync. For other use cases, such as sending confirmation emails, you might only want to invoke your function in response to future data changes.
### Configure batch size
By default, Hightouch separately invokes your Azure function for each added, changed, or removed row.
Certain high-throughput use cases may require batching together multiple rows in the payload. Hightouch supports batches of up to 1000 rows per function invocation.
Batching isn't recommended unless absolutely necessary for performance reasons. Enabling this feature requires your Azure function to be either idempotent or significantly more fault tolerant in the event of a partial batch failure.
### Configure rate and concurrency limits
In this step, you declare whether function invocations should be throttled.
To determine the ideal limits, you should consider whether your function interacts with any downstream services that have limits of their own.
Most modern web APIs enforce rate limits, which set a maximum allowed number of requests per second, minute, or hour. Occasionally, APIs may also have concurrency limits, which set a maximum allowed number of requests that can be processed simultaneously. Rate limits and concurrency limits both affect overall sync speed.
### Configure error handling
Azure functions can fail for many reasons. Hightouch's retry logic for this destination doesn't discriminate between invocation errors, for example, missing permissions, runtime errors, for example, due to syntax issues, or function errors, for example, due to uncaught exceptions. Hightouch retries all errors eventually.
In this step, you decide whether errors should be retried immediately or during the next sync run. If you choose to retry immediately, you can specify how many retries should be attempted during the sync run. If all these retries fail, Hightouch retires the request during subsequent sync runs until it succeeds.
If an error is gracefully caught in your function, you can flag it as a rejected row and provide a custom error message using the response format specified in the [error handling section](#configure-error-handling).
You can also include retry logic inside of your Azure function. After exhausting all retry attempts during a function invocation, respond with an error so that Hightouch knows to reinvoke the function later.
### Invocation payload schema
The payload, also known as the "event document," contains your row data in JSON format, along with additional metadata. When writing your Azure function, you can assume that all function invocations include a payload following this schema:
```json
{
"operation": XXXXX,
"primary_key_column": XXXXX,
"rows": [
{
XXXXXX
}
],
"metadata": {
"api_version": XXXXX,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
#### Invocation payload properties
##### `operation`
The row operation, also known as the trigger, explains *why* your function was invoked.
Its possible values are these strings: `"add"`, `"change"`, and `"remove"`.
##### `primary_key_column`
This string refers to the column name of the primary key for your Hightouch model.
##### `rows`
This is an array of objects, with each object containing row data from your Hightouch model.
##### `metadata`
This is an object containing three key/value pairs:
- `api_version` (integer): currently set to `1`, this identifier only changes if the payload schema is modified in the future
- `sync_id` (integer): this is the unique identifier for the Hightouch sync configuration associated with the function invocation
- `sync_run_id` (integer): this is the unique identifier for the specific sync run associated with the function invocation
#### Example invocation payload
Suppose a new row is added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `john.doe@example.com` | John | Doe |
The payload for the function invocation would look like this:
```json
{
"operation": "add",
"primary_key_column": "customer_id",
"rows": [
{
"customer_id": "928713",
"email": "john.doe@example.com",
"first_name": "John",
"last_name": "Doe"
}
],
"metadata": {
"api_version": 1,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
Note that `rows` is an **array** containing a single element.
#### Example invocation payload for batching
Suppose three new rows are added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `alice.doe@example.com` | Alice | Doe |
| 283743 | `bob.doe@example.com` | Bob | Doe |
| 162352 | `carol.doe@example.com` | Carol | Doe |
The payload for the function invocation would look like this:
```json
{
"operation": "add",
"primary_key_column": "customer_id",
"rows": [
{
"customer_id": "928713",
"email": "alice.doe@example.com",
"first_name": "Alice",
"last_name": "Doe"
},
{
"customer_id": "283743",
"email": "bob.doe@example.com",
"first_name": "Bob",
"last_name": "Doe"
},
{
"customer_id": "162352",
"email": "carol.doe@example.com",
"first_name": "Carol",
"last_name": "Doe"
}
],
"metadata": {
"api_version": 1,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
When batching is enabled, rows sharing a common operation type—`add`, `change`, `remove`—are batched together. Function invocations always represent exactly one operation type.
### Response payload schema
If your Azure function fails to process any rows, it can respond with an array of "rejected rows" that encountered errors. Each rejected row should be identified by its primary key/value and may be associated with an optional error message. Hightouch retries these rows.
If there is an error, the response payload looks like this:
```json
{
"errors": [
{
"primary_key_value": XXXXX,
"reason": XXXXX
}
]
}
```
#### Response payload properties
Errors in the `errors` array have these properties:
##### `primary_key_value`
This identifies the rejected row that needs to be retried.
##### `reason` (optional)
This string may represent an error message or any other information that would be helpful for debugging and monitoring sync health.
#### Example success response payload
If all rows are processed successfully, your function responds with an empty payload.
```json
{}
```
#### Example response payload for rejected rows
Suppose three new rows are added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `alice.doe@example.com` | Alice | Doe |
| 283743 | `bob.doe@example.com` | Bob | Doe |
| 162352 | `carol.doe@example.com` | Carol | Doe |
While processing these rows, your Azure function encounters two errors:
- Alice's email can't be found in the downstream service
- Bob's email already exists in the downstream service
To surface these errors in Hightouch, your function should respond like this:
```json
{
"errors": [
{
"primary_key_value": "928713",
"reason": "Email not found"
},
{
"primary_key_value": "283743",
"reason": "Email already exists"
}
]
}
```
The primary key values (`928713` and `283743`) refer to `customer_id` values. The `customer_id` column was designated the `primary_key_column` in the function invocation payload.
#### Example response payload for other errors
If your function experiences an invocation error, for example, exceeding Azure function rate limits, runtime error, for example, due to syntax issues, or function error, for example due to uncaught exceptions, all rows in the batch are automatically marked as rejected rows.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Batch
**URL:** https://hightouch.com/docs/destinations/batch
**Description:** Build smarter CRM campaigns on Batch with real-time customer data from your Data Warehouse
**Section:** Destinations
## Overview
This integration lets you send server-to-server events, attributes, and custom audiences to Batch without writing a custom pipeline.
## Supported syncing
| Type | Description | Supported Sync Modes |
| ------------------- | --------------------------------------------- | -------------------- |
| **Attributes** | Sync data from any source to your Batch users | Upsert |
| **Events** | Sync records as events to Batch | Insert |
| **Custom Audience** | Sync users to a custom audience in Batch | Add, Remove |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Batch
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Batch** and click **Continue**. You can then authenticate Hightouch to **Batch**.
To get your API and project keys, navigate to `Settings` and copy your REST API key and Project key into Hightouch.
If you plan on using custom audience syncing, you also need to provide your **SDK Key**. You can find this in your Batch settings page.
## Sync configuration
Once you've set up your **Batch** destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Batch destination you want to sync to.
### Events
Any records added to [your source](/getting-started/concepts) are treated as new events and are sent to Batch when your sync runs.
`Event name` and`Custom ID` fields are the minimum required fields to make the API call to Batch. For richer analytics, we recommend that you provide all relevant fields in your use case.
The `Custom ID` field is used to deduplicate events sent to Batch.
If you don't map the `timestamp` field, it will default to the time Hightouch processes the event.
### Attributes
In this mode each record in [your source](/getting-started/concepts) is treated as a user and sent to Batch when your sync runs.
#### Record matching
You can match rows from your model to users in Batch on any column in your model and any field in Batch. Ensure the data types of the model column and Batch field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync user fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to any user attribute. For a reference of natively supported user fields see [Batch's docs](https://doc.batch.com/api/profile/update/#native-attributes).
If you send data for a custom field, Hightouch adds the field and automatically detects its type.
### Custom Audiences
In this mode, you can add or remove users from a custom audience in Batch.
#### Record matching
You can match rows from your model to users in Batch on any column in your model and any field in Batch. Ensure the data types of the model column and Batch field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Audience Selection
You must select the custom audience you want to sync users to. You can select an existing audience from the dropdown or create a new one.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Bazaarvoice
**URL:** https://hightouch.com/docs/destinations/bazaarvoice
**Description:** Drive sales with the most trusted network of authentic product reviews.
**Section:** Destinations
## Overview
This destination lets you send customer transaction data from your data warehouse to Bazaarvoice. This enables you to automatically submit transaction data for review and rating collection, helping you gather authentic customer feedback and display reviews on your products.
## Supported syncing
|Sync Type|Description|Supported Sync Modes|
|--|--|--|
|**Objects**|Sync transaction data from any source to Bazaarvoice transactions|Insert|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Bazaarvoice
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Bazaarvoice** and click **Continue**.
You need to provide the following credentials:
- **Client ID**: Your Bazaarvoice OAuth Client ID. You can obtain this from the Bazaarvoice Developer Portal or from your Bazaarvoice representative.
- **Client Secret**: Your Bazaarvoice OAuth Client Secret. You can obtain this from the Bazaarvoice Developer Portal or from your Bazaarvoice representative.
- **API Environment**: Select whether to use the Production or Staging environment.
### Create API credentials
1. Log in to the [Bazaarvoice Developer Portal](https://developer.bazaarvoice.com/).
2. Navigate to your application settings or contact your Bazaarvoice representative to obtain API credentials.
3. Create or access your OAuth Client ID and Client Secret credentials.
4. Copy your **Client ID** and **Client Secret**.
5. Ensure your credentials are activated. If you see a "Developer Inactive" error, contact Bazaarvoice support to activate your credentials.
## Sync configuration
Once you've set up your Bazaarvoice destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Bazaarvoice destination you want to sync to.
### Objects
Hightouch supports syncing to Bazaarvoice transactions with the following sync mode:
|Sync Type|Supported Sync Modes|
|--|--|
|**Transactions**|Insert|
#### Configuration
Enter the following field into Hightouch:
- **Client Name**: Your Bazaarvoice client instance name (e.g., `mycompany-en_us`, `mycompany-fr_fr`). This identifies which Bazaarvoice deployment zone to send transactions to.
#### Field mapping
Hightouch lets you sync transaction fields via [field mapping](/syncs/mapping-data). You can map data from any of your model columns to the Bazaarvoice transaction fields.
Bazaarvoice requires the following fields, so you must map them to complete your configuration:
**Required fields:**
- **Transaction ID**: A unique identifier for the transaction
- **Email Address**: The customer's email address
- **User ID**: A unique identifier for the user
- **Customer Name** (nickname): The customer's display name
- **Locale**: The locale code (e.g., `en_US`, `fr_FR`)
- **Transaction Date**: The date and time of the transaction
- **Deployment Zone**: The Bazaarvoice deployment zone identifier
- **Products**: An array of product objects. Each product must include:
- `id`: Product identifier (required)
- `name`: Product name (required)
- `imageUrl`: Product image URL (required)
- `money`: Object with `amount` (number) and `currency` (string)
- `incentivizedReview`: Optional string field
**Optional fields:**
- **Transaction Channel**: The channel through which the transaction occurred
Use the [inline mapper](/syncs/mapping-data#inline-mapping) to structure the products array. The products field requires an array of objects, so you'll need to use the inline mapper to transform your source data into the required format.
#### Batch processing
Bazaarvoice processes transactions in batches of up to 200 transactions per request. Hightouch automatically batches your transactions to optimize API usage and ensure efficient data transfer.
#### Error handling
The Bazaarvoice API returns a 207 Multi-Status response, which means some transactions in a batch may succeed while others fail. Hightouch automatically handles these partial failures and reports which specific transactions failed, allowing you to troubleshoot individual issues without affecting successful transactions.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Beeswax
**URL:** https://hightouch.com/docs/destinations/beeswax
**Description:** Combine best-in-breed solutions from across the programmatic ecosystem with your proprietary data, algorithms, and strategies into a seamless whole
**Section:** Destinations
## Setup
After selecting Beeswax from our destinations catalog, you will be required to enter your _Beeswax Subdomain_,
_Email_, and _Password_ from your Beeswax account. Your _Beeswax Subdomain_ is the text before `.beeswax.com` in
the URL, _Username_ and _Password_ are the credentials you use to login to Beeswax.
Optionally you can also add a salt that is provided by Beeswax that will be used when hashing data used for [HEM](#hem) identifiers.
## Syncing segments
Hightouch supports automatically creating a new segment for your sync. You can optionally assign a custom name to your segment.
If no name is provided, Hightouch will default to using your model name.
Here, the model name `User Segment 1` will be used as the segment name:
Additionally, instead of creating a new segment, Hightouch can also sync users to existing segments that are already populated in your Beeswax account.
### Sync modes
This integration supports Segment mode. New records will be inserted into a Beeswax segment. Records that are removed from the model will be removed from the
segment.
### Select Beeswax sync options
Hightouch allows you to choose which Beeswax account and data center you wish to sync data to. This can be selected on the sync configuration page.
### Record mappings
Records can be matched from your source to your segment by `Beeswax ID`, `Customer ID`, `HEM`, `IDFA`, `IDFA_MD5`, `IDFA_SHA1`, `AD_ID`, `AD_ID_MD5`, `AD_ID_SHA1`, or `IP_ADDRESS`.
#### HEM
When using HEM as the matching type, the mapped value will automatically be hashed based on the [docs](https://hub.freewheel.tv/display/BW/Hashed+Email+Addresses+in+FreeWheel+Identity+Network).
You can turn this off using the checkbox if your IDs are already hashed.
---
## BigCommerce
**URL:** https://hightouch.com/docs/destinations/bigcommerce
**Description:** Seamlessly create unique storefronts for each facet of your business, all managed through one BigCommerce dashboard.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes |
| ------------- | ----------------------------------------------------------- | -------------------- |
| **Orders** | Create and update orders, including updating their statuses | Update, Insert |
| **Customers** | Create and update customers with data from your warehouse | Upsert, Update, Insert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to BigCommerce
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination**
button. Select **BigCommerce** and click **Continue**. You can then authenticate Hightouch to **BigCommerce** by entering the following required fields into Hightouch:
- **Access token**: This is the access token for your BigCommerce API account. It should have read/write access to orders and customers.
- **Store hash**: This is the store hash for your BigCommerce store. You can find it in your control panel URL after `store-` or in your API path after `stores/`..
You can find more information on how to generate an access token [in BigCommerce's docs](https://developer.bigcommerce.com/api-docs/getting-started/authentication/rest-api-authentication#obtaining-store-api-credentials).
## Sync configuration
Once you've set up your BigCommerce destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and
the BigCommerce destination you want to sync to.
### Syncing orders
Create and update orders, including updating their statuses.
#### Record matching
To match rows from your model to records in BigCommerce, you need to select the model column that contains values that match the BigCommerce **ID** field.
In **Insert** mode, BigCommerce automatically generates an identifier for
every new record synced, so there's no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to native and custom fields in BigCommerce. Ensure the data type of your model column matches the data type of the field you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ------------------------------------------------------------------ |
| **Do nothing** | Keep the customer record in BigCommerce with all its synced fields |
| **Clear** | Unset all the mapped fields from BigCommerce |
| **Delete** | Delete the synced record from your BigCommerce order |
### Syncing customers
Create and update customers with data from your warehouse.
#### Record matching
To match rows from your model to profiles in BigCommerce, you need to select a model column and corresponding BigCommerce field. You can match on any of the following BigCommerce fields:
- **ID**
- **Email**
In **Insert** mode, BigCommerce automatically generates an identifier for
every new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to native and custom fields in BigCommerce. Ensure the data type of your model column matches the data type of the field you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ------------------------------------------------------------------ |
| **Do nothing** | Keep the customer record in BigCommerce with all its synced fields |
| **Clear** | Unset all the mapped fields from BigCommerce |
| **Delete** | Delete the synced record from your BigCommerce customer |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Microsoft Ads (Bing Ads)
**URL:** https://hightouch.com/docs/destinations/bingads
**Description:** Ensure that you never serve another irrelevant advertisement with Bing Ads
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API reference |
| ------------- | -------------------------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------- |
| **Events** | Sync offline conversion events with data from any source | Insert | [Offline conversion reference](https://learn.microsoft.com/en-us/advertising/bulk-service/offline-conversion) |
| **Audiences** | Create new or use an existing audience to sync users | Add, Remove | [Customer list item reference](https://learn.microsoft.com/en-us/advertising/bulk-service/customer-list-item) |
## Connect to Bing Ads
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Bing Ads** and click **Continue**. You can then authenticate Hightouch to Bing Ads using OAuth.
Click **Log in to Microsoft Bing Ads** , log in with your Microsoft Ads account, and click **Allow**.
Hightouch requires access to manage your ads account and see your email address.
Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Bing Ads destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Bing Ads destination you want to sync to.
Whether you're syncing events or audiences, begin your configuration by selecting the Microsoft Ads account you want to sync data to.
### Syncing events
When syncing offline conversion events, Hightouch treats any records added to [your source](/getting-started/concepts) as new events and sends them to Bing Ads when your sync runs.
#### Conversion goal name
Select the [conversion goal name](https://help.ads.microsoft.com/#apex/ads/en/56689/2) to associate with the offline conversions.
Your options populate depending on the Microsoft Ads account you selected at the beginning of the sync configuration.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server. Hightouch syncs the timestamp as the `ConversionTime` parameter the Microsoft Advertising API requires.
The timestamp field must also:
- be in Coordinated Universal Time (UTC)
- be later than the recorded **Microsoft Click ID**
- be within the conversion goal window
- be within the last 90 days
For more information, see [Microsoft Bing Ads' official documentation](https://learn.microsoft.com/en-us/advertising/bulk-service/offline-conversion).
#### Field mapping
Hightouch lets you sync event fields via field mapping.
You can map data from any of your model columns to [supported event fields](https://learn.microsoft.com/en-us/advertising/bulk-service/offline-conversion?view=bingads-13).
The Microsoft Ads API requires the **Microsoft Click ID** field, so you must map it to complete your configuration.
Ensure your model columns data types match the data types of the fields you want to sync to.
### Syncing audiences
You can use Hightouch to insert and update audience members in a the Bing Ads customer match list audience.
Users removed from the model get removed from the audience.
To sync to a customer match list audience, you need to first create a customer
list and accept terms and conditions in the Microsoft Advertising UI.
#### Select an existing audience or create a new one
You can create a new audience or use an existing one. To use an existing audience, select the desired audience from the dropdown. When creating a new audience, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model.
By default, the audience will be shareable across all accounts, but you can choose to limit access to the audience to only the account creating the audience.
#### User identifiers
Bing Ads requires you to identify which users to add or update in an audience on a valid **Email**.
Select the model column that contains user emails.
Bing Ads doesn't support mapping any other fields.
For more information, see [Microsoft Bing Ads' official documentation](https://learn.microsoft.com/en-us/advertising/bulk-service/customer-list-item?view=bingads-13#subtype).
#### Handling PII
By default, Hightouch automatically detects if your **Email** field requires hashing and hashes the value before sending requests to Bing Ads if needed.
## Tips and troubleshooting
### Matched users count
Below only applies to the audiences sync type.
### Common errors
{/* */}
#### 8003: CustomerListTermsAndConditionsNotAccepted
To sync to a customer match list audience, you need to first create a customer list and accept terms and conditions in the Microsoft Advertising UI.
For more information, read the [Bing Ads documentation](https://learn.microsoft.com/en-us/advertising/bulk-service/customer-list-item?view=bingads-13).
#### Error uploading audience XXX, bulk operation failed: - (4835) - InvalidAudienceId in path
Check the Bing Ads UI to make sure the audience from your sync configuration exists. If the audience no longer exists, you need to create a new sync.
For additional errors, read the Bing Ads [error codes documentation](https://learn.microsoft.com/en-us/advertising/guides/operation-error-codes?view=bingads-13).
{/* */}
### Live debugger
### Sync alerts
---
## Bird
**URL:** https://hightouch.com/docs/destinations/bird
**Description:** The complete platform for enterprise brands to acquire, convert, and retain customers across every channel—powered by AI that actually understands your business.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-------------|--------------------------------------------|----------------------|------------------------------------------------------------------------------------------------
**Contacts** | Sync data from any source to Bird contacts | Upsert, Update | [Contacts docs](https://docs.bird.com/api/contacts-api/api-reference/manage-workspace-contacts)
**Segments** | Sync data from any source to Bird segments | Add | [Segments docs](https://docs.bird.com/api/contacts-api/api-reference/manage-contact-lists)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Bird
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Bird** and click **Continue**. You can then authenticate Hightouch to **Bird**.
Enter the following fields into Hightouch:
- **Workspace ID**
- **Access key**
## Sync configuration
Once you've set up your Bird destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Bird destination you want to sync to.
### Syncing contacts
Sync data from any source to Bird contacts.
#### Record matching
You can match rows from your model to contacts in Bird on any column in your model and any contact identifier in Bird. By default, Bird supports `contactId` and `emailaddress` as identifiers, and you can add new identifier types as well. Hightouch supports matching by `contactId` and `emailaddress`, and if you have an existing custom contact identifier, you can set it in the id mapping section.
Ensure the data types of the model column and Bird field you select match. Refer to the Bird [contact identifier docs](https://docs.bird.com/api/contacts-api/api-reference/manage-contact-identifiers) and the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom contact fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
Contact attributes are mapped in the second section.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|----------------------------------------------------------
**Do nothing** | Keep the contact in Bird with all its synced fields
**Clear** | Clear all the mapped fields, but keep the contact in Bird
**Delete** | Delete the synced contacts from Bird
### Syncing segments
Sync data from any source to Bird segments.
#### Record matching
To match rows from your model to segments in Bird, you need to select the model column that contains values that match the **Contact ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync segment fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default segment fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Split retries
Bird counts all segments in a batch as rejected if the request contains a single invalid segment. To pinpoint which segments are getting rejected with which errors and reduce the number of valid segments that get retried, you can enable [split retries](/syncs/retries#split-retries).
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Bloomreach
**URL:** https://hightouch.com/docs/destinations/bloomreach
**Description:** Bloomreach helps companies personalize customer experiences by targeting the right customers with the right experience at the right time.
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|**Events**|Sync data from any source to any events|Insert|
|**Customers**|Sync data from any source to the customer objects|Upsert|
|**Catalogs**|Sync data from any source to catalog items|Upsert|
## Getting started
To use the Bloomreach destination in Hightouch, you need to provide a Bloomreach API Key, API Secret, API URL, and Project Token. Follow these steps to retrieve your credentials:
1. In Bloomreach, click the wheel icon in the top right corner. Select **Project Settings** from the dropdown.
2. Click on **Access management** in the side navigation. Select **API** from the options.
3. Your **Project token** and **API Base URL** are here. You can either use an existing **API Key ID** and **API Secret**, or you can click **+ Add key** to create another one.
## Syncing data
### Events
When syncing events, Hightouch treats any records added to [your source](/getting-started/concepts) as new events and sends them to Bloomreach when your sync runs.
You need to select the type of event and one or more hard and/or soft customer IDs to make the API call to Bloomreach.
The most common customer IDs are `registered`.
#### Field mapping
One or more of the event properties, also known as attributes, are required. The event properties show up as custom mappings and attach custom data relevant to your event.
Custom mappings are included as the [`properties`](https://documentation.bloomreach.com/engagement/reference/add-event-2#body-parameters)
parameter of the API request payload.
For more information about event properties, refer to [Bloomreach's documentation](https://documentation.bloomreach.com/engagement/reference/add-event-2).
### Customers
You can also sync to the **Customers** object. This integration supports **Upsert** mode, meaning you can insert new customers and update existing ones in Bloomreach.
#### Record matching
You can match rows in your model with customers in Bloomreach on one or more of the following hard and/or soft customer IDs:
- Cookie
- Email
- Registered
ID mappings are included as the [`customer_ids`](https://documentation.bloomreach.com/engagement/reference/update-customer-properties-2)
parameter of the API request payload.
#### Field mapping
You need to include at least customer property you wish to update. The customer properties show up as custom mappings and attach custom data relevant to your customer.
Custom mappings are included as the [`properties`](https://documentation.bloomreach.com/engagement/reference/update-customer-properties-2)
parameter of the API request payload.
For more information about customer properties, refer to [Bloomreach's documentation](https://documentation.bloomreach.com/engagement/reference/update-customer-properties-2).
### Catalogs
You can sync data to Bloomreach **Catalogs** to manage product or item data. This integration supports **Upsert** mode, allowing you to insert new catalog items and update existing ones.
#### Catalog selection
You can either create a new catalog or sync to an existing catalog in Bloomreach. When creating a new catalog, Hightouch automatically creates the catalog with fields based on your mappings.
#### Record matching
You must match rows in your model with catalog items using the **Item ID** field. This uniquely identifies each item in the catalog.
#### Field mapping
Map columns from your source to catalog item properties. These properties contain the data for each item in your catalog, such as product name, price, or description.
For more information about catalogs, refer to [Bloomreach's documentation](https://documentation.bloomreach.com/engagement/reference/catalog-api-overview-2).
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Bluecore
**URL:** https://hightouch.com/docs/destinations/bluecore
**Description:** Empower e-commerce brands to personalize customer communications with Bluecore, the retail marketing platform.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
------------------------|-----------------------------------------|---------------------
**Batch File Delivery** | Upload batch files to Bluecore via SFTP | Insert, All, Diff
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Bluecore
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Bluecore** and click **Continue**. You can then authenticate Hightouch to **Bluecore**.
Enter the following fields into Hightouch:
- **Host**
- **Port**
- **Username**
- **Password**
- (Optional) **Private Key**
- (Optional) **Private Key Passphrase**
## Sync configuration
Once you've set up your Bluecore destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Bluecore destination you want to sync to.
### Syncing via batch file delivery
Upload batch files to Bluecore via SFTP
#### Field mapping
Hightouch lets you sync batch file delivery fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default batch file delivery fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Box
**URL:** https://hightouch.com/docs/destinations/box
**Description:** Transfer data at scale to Box
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
| ------------ | --------------------------------------------------- | -------------------- |
| Any data set | Sync data from a source to Box as JSON or CSV files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Diff mode creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Box
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Box** and click **Continue**. You can then authenticate Hightouch to Box via OAuth.
After logging into your Box account, Hightouch prompts you to grant access to your Box account:
After authorizing, you've successfully connected Hightouch to Box.
## Sync configuration
Once you've set up your Box destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Box destination you want to sync to.
### Select file format
Hightouch supports syncing JSON and CSV files to Box.
### Select folder
Next, enter the folder ID to where the file will be synced to. You can find the folder ID by visiting the folder in the web application and copying the ID from the URL. For example, for the URL https://app.box.com/folders/123, the folder ID is 123.
### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it. To view different versions of the same file, you can right-click the file in Box. Click **Properties** and then **Version History**.
If you are using an audience and would like to include the audience name, you will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter `-86400` (`24 hours * 60 minutes * 60 seconds`). If you want the filename's data to be one hour after the sync takes place, you would enter `3600` (`60 minutes * 60 seconds`).
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `company`, `email`, `name`, and `primaryKey` fields. Hightouch exports these fields to new fields in the file and ignores all other columns from your results.
### CSV options
If you're syncing to a CSV file, you have additional configuration options:
- **Delimiter**: Your options are comma (`,`), semicolon (`;`), pipe (`|`), tilde (`~`), and tab
- (Optional) Whether to **include a CSV header row** in the exported files
- (Optional) Whether to **include a [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark)** in the exported files, the BOM is ``
### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
## Tips and troubleshooting
### Common errors
### Sync alerts
---
## Branch.io
**URL:** https://hightouch.com/docs/destinations/branchio
**Description:** Event tracking helps you understand how a user moves through your app and what actions they are taking along the way. Commerce, Content, and User Lifecycle events can all be tracked this way.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
|-----------------------------|-------------------------------------------------------------|----------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|
| **Events** | Sync data from any source to Branch.io events | Insert | [GitHub docs](https://github.com/BranchMetrics/branch-deep-linking-public-api?tab=readme-ov-file#logging-custom-events) and [Event docs](https://help.branch.io/developers-hub/reference/logstandardevents) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Branch.io
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Branch.io** and click **Continue**. You can then authenticate Hightouch to **Branch.io**.
Enter the following fields into Hightouch:
- **App ID**
- **Branch Key**
- **Branch Secret**
All of these fields can be found by managing the your [Branch.io profile settings](https://help.branch.io/using-branch/docs/profile-settings). The Branch Key and Branch Secret are considered sensitive information.
## Sync configuration
Once you've set up your Branch.io destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Branch.io destination you want to sync to.
### Syncing events
Sync data from any source to Branch.io events.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
At least one of the following fields must be mapped so that events can be uniquely identified with a user:
- Developer Identity
- Browser Fingerprint ID
- IDFA
- IDFV
- AAID
- Android ID
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Braze
**URL:** https://hightouch.com/docs/destinations/braze
**Description:** Build better campaigns on Braze with up-to-date customer data from your data warehouse
**Section:** Destinations
## Overview
Braze is a customer engagement platform that supports real-time and batch messaging across email, SMS, mobile, and web. Use Hightouch to sync user data, events, subscription status, and catalog information so Braze can run accurate, data-driven messaging and lifecycle campaigns.
---
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| ----------------------- | ---------------------------------------------------------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| [**Event**](#sync-events) | Send track events and track purchase type events to Braze | Insert | [Events](https://www.braze.com/docs/api/objects_filters/event_object/) |
| [**Object**](#sync-users) | Sync records to Braze objects like `User` or `Organization`. | Upsert, Update | [Users API](https://www.braze.com/docs/api/endpoints/user_data/) |
| [**Subscription status**](#sync-subscription-status) | Sync SMS or email subscription groups. | Upsert | [Subscription Groups API](https://www.braze.com/docs/api/endpoints/subscription_groups/) |
| [**User identity**](#merge-anonymous-and-known-users) | Merge alias-only users with an identified profile and remove the alias-only profile | Insert | [User identification](https://www.braze.com/docs/api/endpoints/user_data) |
| [**Catalog items**](#sync-catalog-items) | Sync catalog items such as products or offers. | Upsert, Update | [Catalog items](https://www.braze.com/docs/api/endpoints/catalogs) |
| [**Campaign triggers**](#trigger-campaigns) | Trigger campaigns with context from your records. | Insert | [Campaign triggers](https://www.braze.com/docs/api/endpoints/messaging/send_messages/post_send_triggered_campaigns/) |
| [**Canvas triggers**](#trigger-canvases) | Trigger Canvases with context from your records. | Insert | [Canvas triggers](https://www.braze.com/docs/api/endpoints/messaging/send_messages/post_send_triggered_canvases/) |
### How Braze uses the data you sync
Different sync types enable different workflows in Braze:
- **User attributes** update profiles and improve segmentation.
- **Events** can trigger campaigns and Canvas entries or provide context for message logic.
- **Subscription status** controls channel eligibility and opt-in choices.
- **Catalog items** populate dynamic content blocks in messages and templates.
- **User identity data** helps consolidate user profiles across devices.
- **Trigger properties** add context when activating Campaign or Canvas triggers.
---
## Connect to Braze
Connecting Braze to Hightouch requires an **API key** and **region**.
### API key
Generate or locate a REST API key in your Braze dashboard under **Settings > APIs and Identifiers > API Keys**.
#### Required permissions
The API key must include permissions for each sync type you plan to use.
| Sync Type | Required permissions |
| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Event** | `users.track`, `custom_attributes.get`, `users.export.ids` (only required if using the **Retry** [user existence behavior](#sync-events)) |
| **Users** | `users.track`, `users.export.ids`, `users.delete` (optional if you want to rely on Hightouch for user deletions), `users.alias.update` (if you are updating existing users), `custom_attributes.get` |
| **Subscription status** | `subscription.status.set` |
| **User identification** | `users.identify` |
| **Catalog items** | `catalogs.add_items`, `catalogs.update_items`, `catalogs.get_item`, `catalogs.create`, `catalogs.get`, `catalogs.delete_items`, `catalogs.replace_items` |
| **Campaign triggers** | `campaigns.trigger.send`, `campaigns.list`, `campaigns.details` |
| **Canvas triggers** | `canvas.trigger.send`, `canvas.list`, `canvas.details` |
### Region
To find your Braze region, check the domain in your Braze dashboard URL and match it to the table below.
| URL | Region |
| -------------------------------- | ------------------ |
| `https://dashboard-01.braze.com` | `iad-01.braze.com` |
| `https://dashboard-02.braze.com` | `iad-02.braze.com` |
| `https://dashboard-03.braze.com` | `iad-03.braze.com` |
| `https://dashboard-04.braze.com` | `iad-04.braze.com` |
| `https://dashboard-05.braze.com` | `iad-05.braze.com` |
| `https://dashboard-06.braze.com` | `iad-06.braze.com` |
| `https://dashboard-07.braze.com` | `iad-07.braze.com` |
| `https://dashboard-08.braze.com` | `iad-08.braze.com` |
| `https://dashboard-01.braze.eu` | `fra-01.braze.eu` |
| `https://dashboard-02.braze.eu` | `fra-02.braze.eu` |
Refer to the [Braze endpoint
documentation](https://www.braze.com/docs/api/basics/?redirected=true#api-definitions)
to view additional regions. To derive the region, remove the prefix
"`https://rest`" from the respective value in the **REST ENDPOINT** column.
---
## Sync configuration
Once you've set up your Braze destination and have a [model](/getting-started/concepts#models) to pull data from, you can configure syncs to begin sending data:
1. Navigate to the [**Syncs** overview page](https://app.hightouch.com/syncs).
2. Click **Add sync**.
3. Select the relevant model and the Braze destination you want to sync to.
---
### Sync events
Hightouch supports syncing events to Braze and assigning them to a user. You can send two event types to the Braze destination:
- **Track Event:** Sends events of a particular name when a record is added
- **Track Purchase:** Track when a customer makes a purchase
When syncing events, you can configure what happens when the associated user doesn't exist in Braze yet. This is controlled by the **user existence behavior** setting:
- **Create user**: Braze automatically creates a new user profile when the event is received.
- **Skip event**: Braze won't create a new profile or track the event.
- **Retry until user exists**: Hightouch polls Braze to check whether the user exists before sending the event. If the user hasn't been created after 2 minutes, the event is rejected as retriable and retried later.
#### Record matching
Hightouch can match events to users on different Braze fields. If user existence behavior is set to **Create user**, you can only match on **Braze External ID** or **User Alias Object**. If set to **Skip event** or **Retry until user exists**, you can also match on **Braze ID**.
For `track event` type events you can also send the event name and event timestamp. `Track purchases` don't require an event name.
Hightouch accepts a standard date time format for the timestamp and
automatically converts it to the format Braze requires.
#### Field mapping
Hightouch lets you pass data to the event properties of a Braze event.
#### Event property configuration
Hightouch lets you send all your custom properties as a single object column or as multiple columns from your model.
When using a single column, make sure that the model column's [data
type](/models/data-types-casting#data-types) is **Object / Array**.
---
### Sync users
Hightouch supports syncing rows from your model to Braze users. This integration supports both **Upsert** and **Update** modes. In Upsert mode, Hightouch inserts new users into Braze, with all attributes kept up-to-date. In Update mode, Hightouch updates attributes on users that already exist in Braze.
Hightouch automatically normalizes some user data fields to meet Braze's specifications.
| Field name | Hightouch normalization |
| ------------- | ------------------------------------------------ |
| Date of birth | Convert from timestamps to year-month-date |
| Email | Convert to all lowercase letters |
| Phone numbers | Remove leading plus signs (`+`) and white spaces |
#### Record matching
In Upsert mode, you can match records from your source to your Braze workspace using the Braze External User ID. In Update mode, you can also match records based on Braze ID or a user alias object.
If you match records with a column **other** than the primary key of your
model, changes in the model column used for record matching will not be
reflected in Braze. This is to ensure duplicate records are not created in
Braze.
When syncing a large amount of rows, make sure to match the records using
Braze External User ID instead of Braze ID. Using the Braze ID requires more
requests which often results in slower sync speeds. See the [slow initial sync
section](#slow-initial-sync) for more details.
#### Field mapping
You can sync columns from your source to Braze's default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
You can manage your custom fields in **Manage App Group** > **Custom Attributes** in Braze.
You can also map your model columns to **user aliases** in Braze.
This configuration only appears if you select **Braze External ID** for [record matching](/destinations/braze#record-matching).
#### Nested objects and object arrays
Braze supports user attributes containing JSON objects or [object arrays](https://www.braze.com/docs/user_guide/data_and_analytics/custom_data/custom_attributes/array_of_objects/). These objects and object arrays are commonly used to support "entities" related to each user—for example, devices, medications, pets, etc.—especially when marketing communication needs to include metadata about these related entities.
Hightouch is a great tool to sync to these types of attributes because you can leverage your data warehouse to manage these associations. To do so, you must build the full JSON object into a single column within your Hightouch data model.
Modeling nested attributes into JSON can be done directly in your warehouse with something like [dbt](https://www.getdbt.com/), or you can use SQL functions appropriate for your data warehouse and build these objects within the [SQL Editor](/models/sql-editor) in Hightouch.
#### Nested object array example
Let's say you want an array of a customer's pets to be an attribute named `pets` in Braze. You would need a data model with a column named `pets` and would then build the full JSON pets object array to populate that column. This array might look something like this:
```json
[
{
"id": 0,
"type": "dog",
"breed": "Mutt",
"name": "Rae"
},
{
"id": 1,
"type": "cat",
"breed": "Tabby",
"name": "Garfield"
}
]
```
This full object array would need to be the value of a column in your Hightouch data model:
Then you would map this column to the `pets` custom attribute in Braze in your Hightouch sync configuration:
After scheduling your sync, you'd have relevant pet data on each user in Braze.
---
### Sync subscription status
Hightouch supports syncing users to both Email and SMS subscription groups.
Braze has a designated set of fields to manage subscription group opt-ins independent of user attributes.
Hightouch supports syncing to existing subscription groups within Braze. You can enter a static subscription group ID or select a column from your model containing the subscription group ID.
Read [Braze's
docs](https://www.braze.com/docs/user_guide/message_building_by_channel/email/managing_user_subscriptions/)
for information on creating a subscription group.
#### Record matching
You can match records from your source to your Braze subscription groups by Braze External User ID, Phone Number, or Email.
#### Field mapping
You can map any source fields to user fields in Braze, for example name, email, etc.
#### Delete behavior
When a user is removed from a Subscription status sync, Hightouch makes an API request to set the user's [`subscription_state`](https://www.braze.com/docs/api/endpoints/subscription_groups/post_update_user_subscription_group_status_v2/#request-body) to `unsubscribed`.
---
### Merge anonymous and known users
Hightouch supports merging alias-only "anonymous" users with "known" Braze users with an external ID.
It's common to have “soft-signup” workflows where all you collect from a user is an email or phone without forcing the user to create an account. This workflow results in users where all you may have is PII. When they create an account, you need to use this endpoint to “identify” these users with a new login ID.
For example, your workflow might look like this:
1. A user visits your website or app. The user doesn't have a Braze ID to start with—they only enter an email address, phone number, or some other PII. You create a user in you customer database, and an alias-only user profile in Braze via the Braze SDK. Hightouch isn't involved at this point.
2. You can use Hightouch to send user attributes or user-related events to Braze via Hightouch by [matching on a user alias object](#record-matching), since there isn't an external ID yet. You must use Update mode.
3. Once you have a unique user ID in your customer database, you can use Hightouch to call Braze's [Identify endpoint](https://www.braze.com/docs/api/endpoints/user_data/post_user_identify/) to "identify" the alias-only user using your unique user ID as the external ID in Braze. This merges the alias-only user in Braze with a known Braze user. If an external ID doesn't exist when using the Identify endpoint, Braze creates the external ID and the alias-only user is considered “identified.”
4. All subsequent updates to the user should be done in Hightouch using the external user ID, not the user alias.
#### Record matching
You can match and merge records using Braze user external IDs.
#### Field mapping
Hightouch lets you map your Braze aliases to known users using the alias label and value.
If you're adding a JSON object directly to your model, each [`user_alias`](https://www.braze.com/docs/api/objects_filters/user_alias_object) object should consist of `alias_label` and `alias_name` as top-level attributes, rather than nested in another object.
```javascript
# CORRECT FORMAT
"user_aliases": [
{
"alias_label": "label",
"alias_name": "name_1"
},
...
]
# INCORRECT FORMAT
"user_aliases": [
{
"user_alias" : {
"alias_name" : "label",
"alias_label" : "name_1"
}
},
,
...
]
```
---
### Sync catalog items
Hightouch supports syncing rows from your models to your existing catalog through the [Braze Catalogs API](https://www.braze.com/docs/api/endpoints/catalogs).
Braze's Catalog endpoint is in early access. Contact your Braze account
manager if you are interested in participating.
You can choose to create a new [Braze catalog](https://www.braze.com/docs/user_guide/personalization_and_dynamic_content/catalogs/catalog/) the first time the
sync runs or use an existing catalog. If creating a new catalog, you can name the catalog from Hightouch. If you leave this field blank, the name defaults to the
model name.
With the **Create a new catalog** option enabled, Hightouch only creates a
catalog the first time the sync runs. Hightouch doesn't modify the catalog
name after the first run. If you want to change the name or fields, you have
to create a new catalog using a new sync.
Syncing to catalog items allows either **Upsert** and **Update** modes. In Upsert mode, Hightouch inserts new catalog items into the Braze. In Update mode, Hightouch updates fields on catalog items that already exist in Braze.
#### Record matching
In Upsert and Update modes, you can match records from your source to your Braze workspace using the catalog item ID.
#### Field mapping
You can map data from your source to any existing field in your catalog.
If automatically creating a catalog, you can't add new fields or change data
types after the initial sync.
#### Testing
If you are using an existing catalog, you can test your sync by syncing a single row. You can verify that a row was successfully synced by going to your Catalog and finding the synced row by its catalog item ID. If you are automatically creating a catalog, it's not possible to test by row.
---
### Trigger campaigns
Hightouch lets you use records from your models to trigger campaigns through the [Braze Campaign API](https://www.braze.com/docs/api/endpoints/messaging/send_messages/post_send_triggered_campaigns/).
Once configured, your Hightouch sync triggers your campaign once rows appear in your model. Hightouch triggers your campaign for every row in your model the first time you run the sync, whenever you do a full resync, and for every new row added to your query results in subsequent syncs.
#### Prerequisites
Before you can set up a Hightouch sync to trigger a campaign, you need set up an [API-triggered](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns/delivery_types/api_triggered_delivery/) campaign in Braze. See [how to create an email campaign in Braze](https://www.braze.com/docs/user_guide/message_building_by_channel/email/creating_an_email_campaign/) for more information.
Hightouch can only trigger campaigns that are
[API-triggered](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns/delivery_types/api_triggered_delivery/).
These are different from [API
campaigns](https://www.braze.com/docs/api/api_campaigns/).
#### Record matching
This destination lets you trigger a campaign for your Braze users based on their external user ID.
This destination doesn't check for duplicates. Hightouch recommends that you
set [delivery
controls](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns/delivery_types/reeligibility/#canvas)
in Braze to prevent duplicated campaign messages in case your source has
duplicates.
#### Field mapping
This destination allows you to customize your messages via [API trigger properties](https://www.braze.com/docs/api/objects_filters/trigger_properties_object). Hightouch automatically lists trigger properties you can map values with. See [using the templated content included with an API request](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns/delivery_types/api_triggered_delivery/#setting-up-an-api-triggered-campaign) for more information.
#### Send to existing only
This destination allows you to create or update users when you trigger a campaign. To create new users, just uncheck the setting in the sync and map the user properties.
Note: Mapping user properties when using this setting is *required*
#### Testing
Hightouch recommends that you test with your email before running an actual sync for your users.
You can test by following these steps:
1. Create a Google Sheet with a valid `external_id` and your email. You will use this to sync yourself as user in Braze. See [Braze's user ID convention](https://www.braze.com/docs/developer_guide/platform_integration_guides/web/analytics/setting_user_ids/#suggested-user-id-naming-convention) if you are unfamiliar with `external_id`.
2. Use the Google Sheet as a source to [add yourself to Braze as a user](/destinations/braze#syncing-objects).
You can use the same sync to remove yourself from Braze once your testing
is complete. Set the sync configuration's delete mode to "Delete
destination record associated with source record," remove yourself from the
source sheet and rerun the sync.
3. Build your campaign in Braze. See Braze's guide on [how to build a campaign](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns) for more information. When setting up your campaign schedule, make sure to select [API-triggered Delivery](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns/delivery_types/api_triggered_delivery/).
4. Set up your campaign trigger sync and run it on Hightouch.
5. Verify that you are able to receive your campaign message. If you need to make corrections and thus want to re-run the sync, click **Resync full query** next to the **Run** button. Also note that your [re-eligibility settings](https://www.braze.com/docs/user_guide/engagement_tools/campaigns/building_campaigns/delivery_types/reeligibility/#canvas) in Braze campaign must be loosely set for you to re-trigger the sync and actually receive the email.
---
### Trigger Canvases
Hightouch lets you use records from your models to trigger Canvases through the [Braze Canvas API](https://www.braze.com/docs/api/endpoints/messaging/send_messages/post_send_triggered_canvases/).
Once configured, your Hightouch sync triggers your Canvases once rows appear in your model. Hightouch triggers your Canvases for every row in your model the first time you run the sync, whenever you do a full resync, and for every new row added to your query results in subsequent syncs.
#### Prerequisites
Before you can set up a Hightouch sync to trigger a Canvas, you need set up an [API-triggered](https://www.braze.com/docs/user_guide/engagement_tools/canvas/create_a_canvas/create_a_canvas/#step-2b-set-your-canvas-entry-schedule) Canvas in Braze. See [Braze Ideas & Strategies](https://www.braze.com/docs/user_guide/engagement_tools/canvas/ideas_and_strategies) for more information and best practices.
Hightouch can only trigger Canvases that are
[API-triggered](https://www.braze.com/docs/user_guide/engagement_tools/canvas/create_a_canvas/create_a_canvas/#step-2b-set-your-canvas-entry-schedule).
Only non-archived, active canvases are available in sync configuration.
#### Record matching
This destination lets you trigger a Canvas for your Braze users based on their external user ID.
#### Field mapping
This destination allows you to customize your messages via [Canvas entry properties](https://www.braze.com/docs/api/objects_filters/canvas_entry_properties_object). Hightouch automatically lists Canvas entry properties you can map values to.
See [Braze's docs](https://www.braze.com/docs/user_guide/engagement_tools/canvas/create_a_canvas/canvas_entry_properties_event_properties) on using the templated content included with an API request for more information.
#### Send to existing only
This destination allows you to create or update users when you trigger a Canvas. To create new users, just uncheck the setting in the sync and map the user properties.
Note: Mapping user properties when using this setting is *required*
---
## Tips and troubleshooting
### Syncing and data point consumption
Hightouch [only syncs data that needs updating](/getting-started/concepts#change-data-capture) to minimize the number of API requests made to your destinations.
Braze offers different pricing models, and how Hightouch syncs impact your usage depends on which model you're using.
To optimize for speed instead of data point updates, you can disable data point optimization in your Hightouch sync configuration.
#### Data points based pricing
In this model, pricing is based on data points, which count the number of updates made by you (and Hightouch). Each update contributes to your total data point usage, so efficient syncing can help reduce consumption.
#### Monthly Active Users (MAU) based pricing (CY24–25 SKU)
This pricing model emphasizes data flexibility for customers. There is no concept of "data points" in this SKU. However, overall data consumption is still constrained by product limits such as API rate limits and other usage thresholds, as detailed in Braze's documentation.
#### How data points are consumed
These are some key points to understand how Hightouch syncs affect your Braze data point consumption.
When syncing arrays without nested objects, you consume one Braze [data point](https://www.braze.com/docs/user_guide/data_and_analytics/data_points/) per API call:
- Creating an array consumes one data point.
- Updating arrays consumes one data point per value updated.
When syncing [arrays of objects](https://www.braze.com/docs/user_guide/data_and_analytics/custom_data/custom_attributes/array_of_objects/), you may consume multiple data points with one API call:
- Creating an array of objects consumes one data point _per attribute added_ within each object. For example, creating two `pet` objects with four attributes each (`id`, `type`, `color`, and `size`) consumes eight data points.
- Updating an array of objects consumes one data point _per attribute changed_.
- Removals consume one data point _per removal criteria_. For example, removing objects where `id = 4` removes one object and costs one data point. Removing objects where `type = dog` may remove many objects but still only consumes one data point.
When syncing records that don't exist in Braze, Hightouch updates **all** mapped attributes with the values from your query results. This means you consume one data point per attribute per created record.
When syncing new records in the model that already exist in Braze, Hightouch first gets the user attributes from Braze, calculates which columns actually changed, and then only sends the changes for those columns. Each updated attribute equals one data point consumed.
When syncing records that have been previously synced to Braze, Hightouch compares the new record's columns to the previously sent record's columns, calculates which columns actually changed, and then only sends the changes for those columns. Each updated attribute equals one data point consumed.
Hightouch sync strategy aims for the lowest Braze data point consumption
possible. Because we make a `GET` request to first see what user attributes
haven't changed since the last sync, sync speed is slower than other methods
which overwrite all data in Braze and consume more data points.
---
### Required permissions
If your syncs are failing, check that the REST API key you use to [connect Braze to Hightouch](#api-key) has the [correct permissions](#required-permissions) for your sync type or types.
### Slow initial sync
If you're using a Braze ID to [match user records](#record-matching), your initial sync may be slow. When Hightouch updates Braze with new records or fields, we query users in Braze for [change data capture](/getting-started/concepts#change-data-capture).
Braze allows batch searches on external IDs, but only allows querying for one user at a time if using Braze IDs. That means Hightouch needs to make a separate request for each user if using Braze IDs.
It's best to match records on external ID rather than Braze ID. This speeds up the initial sync and limits the number of requests to make to Braze.
### Multiple requests per user
When updating an attribute's value in a nested object to `null`, Braze's API doesn't let you combine this change with other non`-null` updates. Therefore, when updating values to `null`, Hightouch sends the full nested object with the `_merge_objects` parameter set to `false`.
Setting `_merge_objects` to `true` deep merges your update with the existing object data. See the Braze [API documentation](https://www.braze.com/docs/user_guide/data_and_analytics/custom_data/custom_attributes/nested_custom_attribute_support/#api-request-body) for more details.
This means the same user may require two update actions:
- Updates to nested objects with non-null values with `_merge_objects:true`
- Updates setting one or more attributes in nested objects to `null` with `_merge_objects:false`
---
## Common errors
#### 400 – The value provided for the phone number field is not a valid phone number
Braze marks some phone numbers as invalid if they are deactivated, rejected by providers, on an [opt-out list](https://www.braze.com/docs/user_guide/message_building_by_channel/sms/keywords/optin_optout), or already associated with another user. See Braze’s guidance on [importing phone numbers](https://www.braze.com/docs/user_guide/message_building_by_channel/sms/phone_numbers/user_phone_numbers#importing-phone-numbers).
**Fixes:**
- Validate phone number formatting in your source.
- Remove numbers that are known to be deactivated or opted out.
- Check for conflicting assignments of the same number to multiple users.
#### Row with ID \{...\} failed with error: alias objects must be a hash
This error indicates that the alias object is not in the expected format with both `alias_label` and `alias_name`, for example:
```
{
"alias_label": "label",
"alias_name": "name_1"
}
```
See Braze’s [User Alias Object](https://www.braze.com/docs/api/objects_filters/user_alias_object) documentation for the required structure.
A common cause is mapping a string column (for example, a JSON string) to the User Alias Object field instead of a JSON object.
**Fixes:**
- Ensure the model column mapped to the User Alias Object is of type **Object**, not string.
- Build the alias object in the correct shape before syncing.
#### Batch too large or payload exceeds Braze limits
Braze returns errors when event payloads, user attributes, or catalog items exceed size limits.
**Symptoms:** `Max input length exceeded`, `Request body too large`
**Fixes:**
- Reduce the number of mapped fields.
- Remove large nested objects or unnecessary attributes.
- Split very large catalogs or audiences across multiple syncs.
#### Invalid or missing user identifiers
Braze cannot update or target users unless identifiers are valid and non-null.
**Symptoms:** `Missing matching row`, dropped rows, partial updates
**Fixes:**
- Prefer **external_id** as the primary identifier when possible.
- Ensure identifier columns are populated and stable in your model.
- For alias-based syncs, include both `alias_name` and `alias_label`.
#### Event attribute type errors
Braze enforces strict types for event and purchase properties.
**Symptoms:** `TypeError`, `Invalid event attribute`
**Fixes:**
- Convert timestamps to ISO 8601 or Unix milliseconds.
- Ensure numeric fields (such as price or quantity) are not strings.
- Avoid nested objects or arrays in event attributes unless Braze explicitly supports them.
#### Catalog field type or schema mismatches
Catalog syncs fail when field types change or when new attributes do not match the existing catalog schema.
**Symptoms:** `Field type mismatch`, partial item failures
**Fixes:**
- Keep catalog field names and types consistent across syncs.
- When schema changes are required, consider creating a new catalog (Braze catalog deletions are permanent).
#### Subscription group ID or permission errors
Subscription syncs fail when the subscription group ID is invalid or its API key lacks required scopes.
**Symptoms:** `Invalid subscription group ID`, subscription state not updating
**Fixes:**
- Use the subscription group’s **GUID** (not the display name).
- Ensure the API key includes `subscription.status.set` or an equivalent permission.
#### Campaign or Canvas trigger failures
Triggers do not fire when user identifiers or required trigger properties are missing.
**Symptoms:** Trigger fails silently or returns a `400` error
**Fixes:**
- Confirm record matching uses a valid Braze identifier (for example, `external_id`).
- Verify that all required trigger properties exist in the model and are mapped in the sync configuration.
---
## Live debugger
---
## Sync alerts
---
## Braze Cohorts
**URL:** https://hightouch.com/docs/destinations/braze-cohorts
**Description:** Build better campaigns on Braze with up-to-date customer data from your data warehouse
**Section:** Destinations
This destination is specific to the [Braze Cohort Import](https://www.braze.com/docs/partners/data_and_analytics/reverse_etl/hightouch/hightouch_cohort_import) feature, which lets partners like Hightouch manage user cohorts within Braze automatically. The Braze Cohorts feature is an excellent alternative to storing audience subscriptions into user-level attributes in Braze since it can reduce your Braze [data point](https://www.braze.com/docs/user_guide/onboarding_with_braze/data_points/) consumption when segmenting audiences.
### Get Braze data import key
In Braze, navigate to **Technology Partners** and select **Hightouch**. Here, you will find your REST Endpoint and generate your Braze data import key. Once generated, you can create a new key or invalidate an existing one.
### Add Braze Cohorts as a destination in Hightouch
Navigate to the **Destination** page in your Hightouch workspace, search for **Braze Cohorts**, and click **Continue**. From there, take your REST endpoint and data import key and click **Continue**.
### Sync a model or audience into Braze Cohorts
In Hightouch, using your created [model](/syncs/create-your-first-sync#set-up-a-model) or [audience](/customer-studio/usage), create a new sync. Next, select the Braze Cohorts destination you created in the previous step. Lastly, in the Braze Cohorts destination configuration, select the identifier you want to match against and decide whether or not you want Hightouch to create a new Braze Cohort or update an existing one.
#### Sync behavior
Braze Cohorts supports an **Add, Remove** sync pattern:
- **Add**: Adds users present in your model or audience to the selected Braze Cohort.
- **Remove**: Removes users who are no longer present in your model or audience from the Braze Cohort. When Hightouch detects via [CDC](/syncs/cdc) that a user has dropped out of the model or audience, it sends a remove operation for that user so they are taken out of the Braze Cohort on the next sync run. This only removes them from the cohort--it does **not** delete the underlying Braze user profile.
Hightouch handles these changes automatically on each sync run so your Braze Cohort always reflects the current membership of your model or audience.
### Create and use a Braze Segment from the Hightouch custom audience
In Braze, navigate to **Segments**, create a new segment, and select **Hightouch Cohorts** as your filter. From here, you can choose which Hightouch cohort you wish to include. Once created, you can select your Hightouch cohort segment as an audience filter when creating a campaign or Canvas.
To use your Hightouch segment, create a Braze campaign or Canvas and select the segment as your target audience.
---
## Brevo
**URL:** https://hightouch.com/docs/destinations/brevo
**Description:** Streamline your marketing efforts by syncing data into Brevo
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-------------|---------------------------------------------|------------------------|----------------------------------------------------------------------
**Contacts** | Sync data from any source to Brevo contacts | Upsert, Update, Insert | [Contacts docs](https://developers.brevo.com/reference/createcontact)
**Contact Lists** | Sync data from any source to a Brevo contact list | Upsert | [Contact lists docs](https://developers.brevo.com/reference/createlist-1)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Brevo
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Brevo** and click **Continue**. You can then authenticate Hightouch to **Brevo**.
Enter the following required fields into Hightouch:
- **API key**:Enter a valid API key.
## Sync configuration
Once you've set up your Brevo destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Brevo destination you want to sync to.
### Syncing contacts
Sync data from any source to Brevo contacts.
### Syncing contact lists
#### List selection
Select which list in your Brevo account you'd like to upsert contacts to. If you would like to create a new list, enter the name of the list, and choose the folder to create the list in.
### Record matching
For **upsert** mode, you can match contacts by email.
For **update** mode, you can match by email, sms, or the contact's ID. Ensure the data types of the model column and Brevo field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
Brevo requires each email, SMS, and contact ID in your list to be unique. Brevo rejects updates that contain information that conflicts with existing contacts.
In **Insert** mode, Brevo automatically generates an identifier for every new record synced, so there is no need to match an existing record.
### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default contact fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|-----------------------------------------------------------
**Do nothing** | Keep the contact in Brevo with all its synced fields
**Clear** | Clear all the mapped fields, but keep the contact in Brevo
**Delete** | Delete the synced contacts from Brevo
For Brevo Contact Lists, when records leave your query result, they are removed from your audience by default. If you would not like to delete any records from your audience, you can choose to do nothing when records leave your query result.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Cadent
**URL:** https://hightouch.com/docs/destinations/cadent
**Description:** Activate your data as Conversion events in Cadent
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes |
----------------|------------------------------------|----------------------|
**Conversions** | Sync conversion event data to Cadent | Insert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Cadent
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Cadent** and click **Continue**. You can then authenticate Hightouch to **Cadent** by entering your API key.
If you do not have an API key, contact Cadent support to set one up.
## Sync configuration
Once you've set up your Cadent destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Cadent destination you want to sync to.
### Syncing conversions
Sync event conversions data to Cadent.
#### Dataset selection
Enter the dataset ID you would like to sync events to. This ID can be found on the Dataset Details page in Cadent.
#### Field mapping
Each Cadent dataset is set up with a match key. Include this match key in your mappings to successfully sync events to Cadent.
Provide the event name, and optionally provide the event timestamp. If no timestamp is provided, Cadent will use the time the event was received.
Hightouch lets you sync event properties and user properties via field mappings. Select the column in your model you'd like to map to each field in Cadent you'd like to sync.
The following are valid values for the event conversion type in Cadent:
- `email_clickthrough`
- `site_clickthrough`
- `online_purchase`
- `app_install`
- `in_app_purchase`
- `physical_store_visit`
- `offline_purchase`
- `phone_call`
- `chat_event`
- `system_generated`
#### Field hashing
The following fields are required to be hashed, and hashed by default:
- Email (`em`)
- First name (`fn`)
- Last name (`ln`)
- Date of birth (`db`)
- Gender (`ge`)
- City (`ct`)
- State (`st`)
- Zip code (`zp`)
- Country (`country`)
- External ID (`external-id`)
If the data in your model is already hashed, you can disable field hashing in your sync.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Calixa
**URL:** https://hightouch.com/docs/destinations/calixa
**Description:** Accelerate product-led sales with data directly from your data warehouse
**Section:** Destinations
## Setup
To set up Calixa, you will need your Calixa **Account ID** and **Auth Token**
These can be found in Calixa by clicking the dropdown menu on the Calixa console home page and select **API Keys**.
## Supported types
Hightouch supports syncing to the following Calixa types:
- **Events**
- **Objects**
## Syncing objects
### Object types
Hightouch supports syncing to the following Calixa objects:
- **Users**
- **Accounts**
### Event types
Hightouch supports syncing to the following Calixa events:
- **Custom User Events**
- **Custom Account Events**
- **Published Metric Events**
### Object sync modes
- **Upsert** - Upsert mode is supported for Users and Accounts. In upsert mode, Hightouch adds new rows and keeps existing rows up to date.
### Record matching
Records can be matched from your source to your Calixa workspace by an external ID.
### Field mapping
You can sync columns from your source to Calixa's default and custom fields.
---
## Campaign Monitor
**URL:** https://hightouch.com/docs/destinations/campaign-monitor
**Description:** Run better email campaigns on Campaign Monitor with up-to-date customer data from your data warehouse
**Section:** Destinations
## Setup
To find your API keys, navigate to your Profile (top right of the page) > Account Settings > API Key and create a private API key.
## List subscriptions
Hightouch supports automatically creating a new List for your sync. You can optionally assign a custom name to your list.
If no name is provided, Hightouch will default to using your model name.
Here, the model name `Subscription List 1` will be used as the list name:
Additionally, instead of creating a new list, Hightouch can also sync subscribers to existing lists that are already populated in your Campaign Monitor account.
### Sync modes
This integration supports Segment mode. New records will be inserted into the Campaign Monitor
List. Records that are removed from the model will be removed from the
segment.
### Record mappings
Records can be matched from your source to your Campaign Monitor Subscriptions by email.
### Field mappings
You can sync columns from your source to Campaign Monitor's default fields.
Note: You must provide a mapping for the "Consent To Track" field. More details can be found in the [Campaign
Monitor API docs.](https://www.campaignmonitor.com/api/v3-3/subscribers/#importing-many-subscribers)
### Custom mappings
You can also sync custom fields to Campaign Monitor's Subscription Lists. If you are syncing custom fields of
data type `Multiple options - select many`, ensure your column data is an array.
Note: Ensure the Campaign Monitor List has custom fields already defined in your account before syncing custom fields.
---
## Campaigner
**URL:** https://hightouch.com/docs/destinations/campaigner
**Description:** Enrich Campaigner with data from your warehouse to build robust email marketing campaigns to drive higher ROI and grow your audience.
**Section:** Destinations
## Overview
Providing relevant and timely campaigns relies on fresh and accurate data in your Campaigner account. By automatically syncing customer data from your data warehouse into Campaigner, you don't need to worry about data consistency or staleness. Instead, you can focus on building world-class customer experiences.
## Getting started
### Connect to Campaigner
**Create an API user and retrieve the API Token from Campaigner.**
1. Log in to Campaigner.
2. Visit [Campaigner's Users Settings Page](https://secure.campaigner.com/CSB/Account/AccountUsers.aspx).
3. Click on **Create New User**.
4. Select **API user** from the **User Role** dropdown select, and fill out the information required in the **User Profile** tab.
5. Click on **Create API Key**.
6. Add the [Hightouch IP Addresses](/security/networking#ip-addresses) to the IP Allowlist and click **Save**.
7. Click on the newly created user and head to the **API Keys** tab and copy the API key.
#### Add Campaigner as a destination in Hightouch
1. Log in to Hightouch or create a free Hightouch account
2. Click on **Destinations** in the left sidebar
3. Click on **Add destination** in the top right corner
4. Select **Campaigner** in the destination catalog and click **Continue** to proceed
5. Paste your **API Key** and click **Continue** to proceed
6. Give your destination a name and unique slug (for example, Campaigner Production” and `Campaigner-production`)
7. Click **Finish** to create your Campaigner destination.
### Connect to your data source
Hightouch sits on top of your data warehouse and can read from Snowflake, Redshift, BigQuery, Databricks, Postgres, MySQL, and many other data sources. You can even fetch data from Google Sheets and Airtable. Hightouch can help activate your data wherever it resides.
We've written dedicated guides for each supported data source. Take your pick:
- Connect to [Airtable](/sources/airtable)
- Connect to [Amazon Athena](/sources/amazon-athena)
- Connect to [Amazon Redshift](/sources/amazon-redshift)
- Connect to [ClickHouse](/sources/clickhouse)
- Connect to [Databricks](/sources/databricks)
- Connect to [Elasticsearch](/sources/elasticsearch)
- Connect to [Firebolt](/sources/firebolt)
- Connect to [Google BigQuery](/sources/google-bigquery)
- Connect to [Google Sheets](/sources/google-sheets)
- Connect to [Looker](/sources/looker)
- Connect to [Metabase](/sources/metabase)
- Connect to [MySQL](/sources/mysql)
- Connect to [Palantir Foundry](/sources/foundry)
- Connect to [PostgreSQL](/sources/postgresql)
- Connect to [Rockset](/sources/rockset)
- Connect to [SFTP](/sources/sftp)
- Connect to [Snowflake](/sources/snowflake)
- Connect to [SQL Server](/sources/sqlserver)
- Connect to [Trino](/sources/trino)
If you're missing the credentials necessary to connect to your data source,
try inviting your teammates to Hightouch. You can have unlimited collaborators
in your workspace.
#### Connect to a sample database instead
Waiting for credentials to your data warehouse? No problem. You can connect to one of our [sample datasets](/sources/sample-data) to get started right away. For the rest of this tutorial, we'll use the **B2B SaaS** sample source, which contains a `users` table with names, emails, locations, and other user attributes. This sample source is available when you log in for the first time—no setup required.
Next, we'll write a SQL query to define which users should be synced to our Campaigner customers resource. You can follow the same pattern to sync to other support Campaigner resources,
### Create a model
In Hightouch, a model represents a query that filters or transforms the records in your data source. Models are used to determine exactly which records to sync between sources and destinations.
When you connect to a data warehouse, your models will probably be written in SQL. You can also import models from tools like dbt and Looker.
In this example, we'll keep it simple. Suppose you want to sync all your customers to Campaigner. To accomplish this, you'll want to create a model that looks like this:
```sql
SELECT * FROM users;
```
1. Click on **Models** in the left sidebar.
2. Click on **Add model** in the top right corner.
3. Select **Demo Database** as your source and click **Continue** to proceed.
4. Select **SQL Editor** as your modeling method.
5. Paste the SQL query from above into the editor on the left side of the page.
6. Click **Preview** to execute the query.
7. Click **Continue** to proceed.
8. Give your model a name and unique slug (for example, "All Users" and `all-users`).
9. Select **ID** as the primary key for your model.
10. Click **Finish** to create your model.
We just created a model using a SQL query that retrieves all our current users. Next, we'll configure our sync to add these users to Campaigner.
### Create a sync
1. Click on **Syncs** in the left sidebar.
2. Click on **Add sync** in the top right corner.
3. Select **All Users** as your model and click **Continue** to proceed.
4. Select **Campaigner** as your destination and click **Continue** to proceed.
## Sync configuration
### Syncing subscribers object
#### Object selection
Hightouch supports syncing to the following Campaigner resources:
- **Subscribers**. Also referred to as **Contacts** on Campaigner.
#### Sync modes
This integration supports **Upsert** and **Update**.
- In **Upsert** mode, records that aren't found in Campaigner given the provided identifier will be inserted. And existing records will be updated.
- In **Update** mode, only records that are found in Campaigner given the provided identifier will be updated.
#### Record matching
- Records can be matched from your source to your Campaigner workspace by `Email`.
#### Field mapping
Hightouch allows you to sync columns from your source to the supported Campaigner fields. You can map fields in Campaigner that aren't in the table below using [custom field mapping](#custom-field-mapping).
| Mapping | Type | Description |
| :------------------- | :-----: | --------------------------------------------------------------------------------------------------------------------------------------------: |
| `Source ID` | Number | The ID of a valid Source this subscriber should be associated with. |
| `Publications` | Array | An array of Publication IDs to be associated with the subscriber. |
| `Lists` | Array | An array of List IDs to be associated with the subscriber. |
| `Orders` | Array | An array of [order objects](https://media.campaigner.com/apidocs/edapi/#OrderAdd). |
| `Force` | Boolean | If the Force parameter is **true**, "Removed" or "Bounced" subscriber statuses will be ignored and set back to "Active" |
| `Force Publications` | Boolean | Set this property to **true** if you want to force this subscriber back into a publication they might have previously requested removal from. |
#### Custom field mapping
Hightouch allows you to set custom attributes. Any mappings from this section will be synced as custom fields. Campaigner expects custom properties to follow a `PascalCase` format for both _new_ and _existing_ custom fields. For example, if `First Name` exists as a field in Campaigner, enter `FirstName` in our mapper. Use the same format for custom fields you want Hightouch to create in Campaigner.
#### Delete records
When the Hightouch sync engine recognizes that a record has been removed from your source, you can either:
In **Upsert** mode:
- Do Nothing.
- Clear fields that are being synced in the destination.
- Delete record in the destination.
In **Update** mode:
- Do Nothing.
- Clear fields that are being synced in the destination.
Clearing fields in Campaigner through Hightouch will not clear existing
Publications and Lists from the Subscriber's record. However, It will reset
the Source ID, and return the boolean mappings:`Force Publications` and
`Force` to their default if they exist in your mappings.
### Syncing mailing lists
- Create a mailing list and/or add subscribers with the **Mailing Lists** integration. Note that contacts must already exist in your Campaigner account to be synced to a mailing list.
#### Record matching
- Records can be matched from your source to your Campaigner workspace by `Email`.
#### Create a new mailing list
- To create a new mailing list, select `Create a new segment`.
- You can provide a name for this mailing list, or Hightouch will use the name of the model used to create the sync.
#### Use an existing mailing list
- To use an existing mailing list, select `Use existing segment`.
- You can select any of your **active** Campaigner mailing lists from the drop-down select menu.
- If you'd like to sync to segments dynamically, click `Use Column` and pick a column from your model that contains the Campaigner `Mailing List ID`.
### Syncing suppression lists
- Add subscribers to suppression lists with **Suppression Lists** integration.
#### Record matching
- Records can be matched from your source to your Campaigner workspace by `Email`.
#### Use an existing suppression list
- Campaigner doesn't provide the ability to create suppression lists programmatically.
- You can select any of your **active** Campaigner suppression lists from the drop-down select menu.
## Tips and troubleshooting
### Common errors
#### Error: 101 Email address has been screened out
You may see this error on records that were rejected during an otherwise successful sync run. This is frequently due to improper formatting (`@com.com`) or other common email validation checks.
### Live debugger
### Sync alerts
---
## Cendyn
**URL:** https://hightouch.com/docs/destinations/cendyn
**Description:** Capture demand and drive direct bookings by managing your digital ads and metasearch campaigns from one single platform.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
-------------------|--------------------------------------------------|---------------------
**Profiles** | Sync data from any source to Cendyn profiles | Upsert
**Reservations** | Sync data from any source to Cendyn reservations | Upsert
**Transactions** | Sync data from any source to Cendyn transactions | Upsert
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Cendyn
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Cendyn** and click **Continue**. You can then authenticate Hightouch to **Cendyn**.
Enter the following fields into Hightouch:
- **Base URL**
- **Property ID**
- **Subscription ID**
## Sync configuration
Once you've set up your Cendyn destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Cendyn destination you want to sync to.
### Syncing objects
Sync data from any source to Cendyn profiles, reservations, or transactions.
#### Record matching
To match rows from your model to objects in Cendyn, you need to select the model column that contains values that match IDs in the Cendyn objects' **Unique IDs** array.
The value entered as the `Destination field` will be used as the unique ID type. After entering the type, you will be prompted to enter the context of the unique ID.
In the image below, the unique ID's type is `PMS` and its context is `true`.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync object fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default fields for each object.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Chameleon
**URL:** https://hightouch.com/docs/destinations/chameleon
**Description:** Boost product adoption and reduce churn
**Section:** Destinations
## Overview
Power your Chameleon workspace with up-to-date user data straight from your data warehouse. You can sync enriched user data and events from your warehouse, specifically to augment product experiences. Moreover, you can ensure that the data in Chameleon is always up-to-date and in the expected format for your Product and Growth teams.
## Supported syncing
Type | Description | Supported Sync Modes
---------------------|--------------------------------------------------|-----------------------
**Users** | Sync data from any source to any user object | Upsert
**Companies** | Sync data from any source to any company object | Upsert
**Events** | Sync data from any source to any event | Upsert
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Chameleon
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination**
button. Select **Chameleon** and click **Continue**. You can then authenticate Hightouch to **Chameleon**.
### Authenticate with credentials
Enter the following required fields into Hightouch:
- **Chameleon API Key**.
## Sync configuration
Once you've set up your Chameleon destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and
the Chameleon destination you want to sync to.
### Users
Sync data from any source into your Chameleon users.
#### Record matching
You can match rows in your model with users in Chameleon on the following fields:
- User uid
#### Field mapping
You can sync columns from your source to Chameleon custom fields.
### Companies
Sync data from any source into your Chameleon companies.
#### Record matching
You can match rows in your model with users in Chameleon on the following fields:
- Company Uid
#### Field mapping
You can sync columns from your source to Chameleon custom fields.
### Events
Insert events into Chameleon by user id.
#### Field mapping
When sending events to Chameleon you can only specify the event name and user id.
## Tips and troubleshooting
### Common errors
### Live debugger
Hightouch provides complete visibility into the API calls made during each of your sync runs. We recommend reading our
[article on debugging tips and tricks](/syncs/debugger) to learn more.
### Sync alerts
---
## Chargebee
**URL:** https://hightouch.com/docs/destinations/chargebee
**Description:** Enrich Chargebee data with customer data from your warehouse to drive revenue analysis
**Section:** Destinations
## Overview
To get the most out of a tools like Chargebee, you'll need to update customers and custom fields.
Chargebee has a tedious process of having to update each customer individually. Hightouch solves this problem by letting you define your customers and it's custom fields on top of your data warehouse or production database. This ensures that your Chargebee will always remain in sync with your entire data stack.
### Why subscription billing should be based on top of the warehouse
- You can sync analytics as custom fields to your Chargebee customers. This allows you to prioritize a subset of certain customers more easily.
- You no longer have to check and compare your data between Chargebee and your business tools.
## Supported sync methods
Hightouch supports syncing data to a Chargebee customer.
### Sync customers to Chargebee
#### Matching source to destination records
When matching records, it's preferred to match it through Chargebee's customer ID. However, if your source has no access to the ID, it may match with other fields, such as email, last name, etc.
If there are multiple rows with same matching value, such as two customers
having the same email, it will only select the first customer returned from
Chargebee.
#### Modes
- **Upsert**: This mode will update existing customers or insert the customer if they do not already exist.
- **Update**: This mode will just update existing customers.
#### Custom fields
You are able to update Chargebee's custom fields through our destination custom field mapping. When referencing the Chargebee custom field within Hightouch, **use the API name** of the custom field
#### Rate limits
Something to keep in mind, as you process all your rows of data is that Chargebee limits the amount of times we can update the rows within their product per minute. This is completely dependent on which tier plan you have with Chargebee and if you're using live/test site.
[More Information On Chargebee's Rate Limit](https://support.chargebee.com/support/solutions/articles/250853-js-api-rate-limit-chargebee)
## Setup
### Connect to Chargebee
#### Creating API token
1. Log in to Chargebee
2. Click on **Settings**
3. Click on **Configure Chargebee** tab
4. Click on the **API keys** section
5. Click on **Add API key**
6. Select the **Full-Access Key** type, then **All** permissions, then give a name
7. Click **Create Key** and copy the newly created token
#### Adding destination in HT
1. Log in to Hightouch or create a free Hightouch account
2. Click on **Destinations** in the left sidebar
3. Click on **Add destination** in the top right corner
4. Select **Chargebee** in the destination catalog and click **Continue** to proceed
5. Type in your site name into the **Site** field (Hint: It's the subdomain when you're on the Chargebee app)
6. Paste your API token into the **API Access Token** field and click **Continue** to proceed
7. Give your destination a name and unique slug (for example, "Chargebee Production” and `chargebee-production`)
8. Click **Finish** to create your Chargebee destination.
### Connect to your data source
Hightouch sits on top of your data warehouse and can read from Snowflake, Redshift, BigQuery, Databricks, Postgres, MySQL, and many other data sources. You can even fetch data from Google Sheets and Airtable. Hightouch can help activate your data wherever it resides.
We've written dedicated guides for each supported data source. Take your pick:
- Connect to [Airtable](/sources/airtable)
- Connect to [Amazon Athena](/sources/amazon-athena)
- Connect to [Amazon Redshift](/sources/amazon-redshift)
- Connect to [ClickHouse](/sources/clickhouse)
- Connect to [Databricks](/sources/databricks)
- Connect to [Elasticsearch](/sources/elasticsearch)
- Connect to [Firebolt](/sources/firebolt)
- Connect to [Google BigQuery](/sources/google-bigquery)
- Connect to [Google Sheets](/sources/google-sheets)
- Connect to [Looker](/sources/looker)
- Connect to [Metabase](/sources/metabase)
- Connect to [MySQL](/sources/mysql)
- Connect to [Palantir Foundry](/sources/foundry)
- Connect to [PostgreSQL](/sources/postgresql)
- Connect to [Rockset](/sources/rockset)
- Connect to [SFTP](/sources/sftp)
- Connect to [Snowflake](/sources/snowflake)
- Connect to [SQL Server](/sources/sqlserver)
- Connect to [Trino](/sources/trino)
If you're missing the credentials necessary to connect to your data source,
try inviting your teammates to Hightouch. You can have unlimited collaborators
in your workspace.
#### Connect to a sample database instead
Waiting for credentials to your data warehouse? No problem. You can connect to one of our [sample datasets](/sources/sample-data) to get started right away. For the rest of this tutorial, we'll use the **B2B SaaS** sample source, which contains a `users` table with names, emails, locations, and other user attributes. This sample source is available when you log in for the first time—no setup required.
### Create a model
### Create a sync
## Tips and troubleshooting
### Common errors
#### Don't see your data in Chargebee?
Make sure you've configured your sync to use the correct Chargebee environment. You might've accidentally pointed to a different environment, such as using test site instead of live site. If that doesn't solve your issue, . We're happy to help.
### Live debugger
### Sync alerts
---
## ChartMogul
**URL:** https://hightouch.com/docs/destinations/chartmogul
**Description:** Empower your product teams with richer data from your warehouse to perform deeper analysis on subscription revenue and better understand business performance.
**Section:** Destinations
## Setup
To find your API keys, navigate to your profile and create a private API key.
## Syncing
Hightouch supports syncing to the following ChartMogul Objects:
- `Customers`
- `Invoices`
- `Transactions`
### Sync modes
This integration supports **Upsert**, **Update**, and **Insert** modes for `Customers` objects and **Insert** mode for `Invoices` and `Transactions` objects.
- **Upsert**: Upsert mode pushes new objects to ChartMogul and updates fields that change in your warehouse.
- **Update**: Update mode updates particular fields on existing objects in ChartMogul.
- **Insert**: Insert mode pushes new objects to ChartMogul, and doesn't update the objects as they change in your warehouse.
## Record matching
### Customers
In **Upsert** mode, records can be matched from your source to ChartMogul by `External ID` or `Email`.
In **Update** mode, records can be matched from your source to ChartMogul by `External ID` or ChartMogul's `Customer UUID`.
ChartMogul doesn't consider `Email` as a unique identifier, so upserting
records will apply to all matching emails.
## Field mapping
### Customers
You can sync columns from your source to ChartMogul's default and custom attributes. When syncing with **Upsert** or **Insert** mode, `Data Source UUID`, `Name`, and `External ID`
(depending on if `External ID` was selected in the Record Matching section) are required fields in order for the sync to run successfully. When syncing with **Update** mode, `Name`
is a required field in order for the sync to run successfully. To know more about `Customer` fields, refer to the [ChartMogul docs](https://dev.chartmogul.com/reference/customers).
Additionally when syncing custom attributes with **Upsert** or **Insert** mode, Hightouch automatically transforms your data to match the required
format that ChartMogul expects for the `custom` field. For example, say you want to enrich a customer profile by inserting `age` as a custom attribute, where the object is:
```JSON
{
"attributes": {
"custom": {
"age": 25,
},
}
}
```
Hightouch will automatically detect the `type` of the record value and transform the object for you to the expected `custom` field format:
```JSON
{
"attributes": {
"custom": [
{
"type": Integer,
"key": "age",
"value": 25
},
]
}
}
```
### Invoices
You can sync columns from your source to ChartMogul's default and custom attributes. `Customer UUID`, `External ID`, `Date`, `Currency`, `Line Items`, and
`Customer External ID` are required fields in order for the sync to run successfully. To know more about `Invoice` fields, refer to the [ChartMogul docs](https://dev.chartmogul.com/reference/import-customers-invoices).
### Transactions
You can sync columns from your source to ChartMogul's default and custom attributes. `Invoice UUID`, `Date`, `Type`, and `Result` are required fields in order
for the sync to run successfully. To know more about `Transaction` fields, refer to the [ChartMogul docs](https://dev.chartmogul.com/reference/import-invoice-transaction).
## Delete mode
### Customers
You can choose what Hightouch's behavior is when records leave the query result set. The default is doing nothing, but you can also set Hightouch to delete the `Customer` record.
ChartMogul doesn't allow custom attributes to be cleared, so Hightouch will
sync `"null"` to the field instead.
## Tips and troubleshooting
### Common errors
{/* */}
#### write ECONNRESET
{/* */}
You may receive the `write ECONNRESET` error if the model columns used for [record matching](#record-matching) or [mapping](#field-mapping) don't have an associated [data type](/models/data-types-casting#data-types) in Hightouch.
To resolve the error, ensure that each model column has the data type ChartMogul expects.
Refer to the [data types and casting docs](/models/data-types-casting) for more information.
### Live debugger
### Sync alerts
---
## ChurnZero
**URL:** https://hightouch.com/docs/destinations/churnzero
**Description:** Fight customer churn with fresh customer data in ChurnZero
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference |
----------------|-----------------------------------------------------------------------------------|----------------------| ------------- |
**Events** | Sync records as events to Churnzero. This is often in the form of a track call | Insert | [Event schema](https://app.churnzero.net/developers#schemas-event) |
**Enrichment** | Sync records to a Churnzero custom table | All | [Batch import API](https://support.churnzero.com/hc/en-us/articles/360004339852-Batch-Imports-via-CSV) |
**Objects** | Sync records to objects such as accounts and contacts in Churnzero | Upsert | [Account schema](https://app.churnzero.net/developers#schemas-account) and [contact schema](https://app.churnzero.net/developers#schemas-contact)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to ChurnZero
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **ChurnZero** and click **Continue**. You can then authenticate Hightouch to **ChurnZero** by entering a ChurnZero **App Key**.
To find your app key, go to **Admin > Application Keys > New App Key** in ChurnZero.
Create a new key with a descriptive name, for example, "Hightouch Integration".
Though optional, it's best to enter your **App URL** into Hightouch.
You can find it below the Application Keys in ChurnZero.
Do not include the `/i` in the URL you enter into Hightouch.
If you don't provide your App URL, Hightouch defaults to `https://analytics.churnzero.net/`.
## Sync configuration
Once you've set up your ChurnZero destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the ChurnZero destination you want to sync to.
### Syncing events
ChurnZero's events API requires the following event parameters:
- `eventDate`: The date/time the event occurred. ISO 8601 formatted string.
- `AccountId`: The ID of the Account associated with the event
- `ContactId`: The ID of the Contact associated with the event
The sync configuration form ensures all these are set and provides some additional options.
#### Event name
Providing an event name is best practice while event tracking. You can either provide a static value or select to **use a column** from your model. Hightouch syncs the static or column value as the `eventName` parameter.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server. Hightouch syncs the timestamp as the `eventTime` parameter the events API requires.
If you select a column, it should be in [ISO 8601](https://www.iso.org/iso-8601-date-and-time-format.html) format.
#### Field mapping
Hightouch lets you sync default and custom event properties via [field mapping](/syncs/mapping-data).
You must include the **Account External ID** and **Contact External ID** of the account and contact associated with the event.
### Syncing enrichments
Hightouch lets you sync data to custom tables through ChurnZero's [batch import API](https://support.churnzero.com/hc/en-us/articles/360004339852-Batch-Imports-via-CSV).
To use this API, you must first create the custom table within ChurnZero.
You can create custom tables by going to the admin tab in ChurnZero and selecting **Custom Fields**.
To locate the custom table to update, Hightouch requires these two fields in your sync configuration:
- **Custom table ID**: You can find the table ID by going into **Admin/Imports** and selecting the custom table name from the dropdown there.
- **Account ID**: This is the user account the custom table is associated with. You can find it by clicking on your name at the top of the ChurnZero dashboard and selecting **My Account**. Your ID is at the end of the URL.
#### Record matching
You can match rows from your model to enrichment in ChurnZero on any column in your model and any field in ChurnZero.
Ensure the data types of the model column and ChurnZero field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync custom table columns via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the columns you've already created in your custom table.
Ensure your model columns' data types match the data types of the fields you want to sync to.
Hightouch can only sync data to fields you've already created in your custom table in ChurnZero on the **Custom Fields** page.
### Syncing objects
Hightouch supports syncing to the following ChurnZero objects:
- **Contacts**
- **Accounts**
#### Record matching
To match rows from your model to contacts in ChurnZero, you need to select the model column that contains values that match the **Contact External ID** field.
To match rows from your model to accounts in ChurnZero, you need to select the model column that contains values that match the **Account External ID** field.
You can find these identifiers in the top left of your contact or account page.
External identifiers are not the same as the ID in the URL of a contact or account page.
#### Field mapping
Hightouch lets you sync account and contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
You can find the custom fields for your ChurnZero workspace in **Admin > Custom Fields**.
Go to either the contact or account table to see the custom fields for that object type.
When syncing custom fields, use the **CZ API Name** as the destination field name in the Hightouch mapper.
ChurnZero requires that you map at least one default field.
If you use primarily custom fields, make sure to also map at least one default field.
Visit the ChurnZero's [API documentation](https://app.churnzero.net/developers#schemas) to see objects' default fields.
When syncing contacts, you must map a field to the contact's **Account Exernal ID**.
Please make sure that the contact's associated account exists in ChurnZero before attempting to sync contact information.
## Tips and troubleshooting
### Common errors
#### 404 - "null"
You may receive a `404 - "null"` error because there is a trailing `/` at the end of the App URL within the Destination configurations. Remove the trailing `/` from the App URL to clear the error.
### Live debugger
ChurnZero's API doesn't return detailed error messages.
For example, it doesn't state whether an account ID or a custom field you are attempting to sync to exists.
Therefore, Hightouch is unable to display this type of information in the debugger.
### Sync alerts
---
## Epsilon Retail Media (CitrusAd)
**URL:** https://hightouch.com/docs/destinations/citrusad
**Description:** Reach customers across the open web and drive increased sales both on-site and in-store.
**Section:** Destinations
## Overview
Now deliver even more relevant ads both onsite and across the web by combining data from various sources within your data warehouse to build custom audiences for CitrusAd. By keeping your custom audiences updated automatically, you'll never show an irrelevant ad to anyone.
## Supported syncing
| Type | Description | Supported Sync Modes | API documenation |
| ------------- | ----------------------------------------------- | ---------------------- | ---------------- |
| **Customers** | Sync customer data from any source to CitrusAd customers | Upsert, Update, Insert | [Create or update a customer](https://developers.citrusad.com/integration/reference/create-or-update-a-customer)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to CitrusAd
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **CitrusAd** and click **Continue**. You can then authenticate Hightouch to CitrusAd by entering the following required fields into Hightouch:
- **Base URL**: The base URL for staging or production environment. Visit [CitrusAd's docs](https://developers.citrusad.com/integration/reference/api-overview#base-urls) for information on retrieving your base URL.
- **API key**: The API key for authenticating the API requests. Visit [CitrusAd's docs](https://developers.citrusad.com/integration/reference/api-overview#basic-authentication) for information on retrieving your API key.
## Sync configuration
Once you've set up your CitrusAd destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and
the CitrusAd destination you want to sync to.
### Customers
Customer data is required to generate personalized ads with CitrusAd. See CitrusAd's [docs](https://developers.citrusad.com/integration/reference/customer-data-1) for more details.
#### Record matching
You must match rows in your model with customers in CitrusAd on the **Customer ID** field.
In **Insert** mode, CitrusAd automatically generates a customer ID for every new record synced, so there is no need to match an existing record.
#### Field mapping
You can sync any model columns to CitrusAd's default customer fields including `targetingData`.
When creating new customer records in CitrusAd via insert or upsert mode, CitrusAd sets a default value to all fields without a value, whether you've mapped them or not.
You can find CitrusAd's default values in the table below:
| Field name | Default value |
| ----------------- | ----------------|
| **Year of Birth** | `0` |
| **Gender** | `"Other"` |
| **Postcode** | `""` |
| **Suburb** | `""` |
The **Gender** field accepts the following values:
`"Male"`,`"Female"`,`"Other"` case insensitive. If you send any other values, CitrusAd changes them to `"Other"`.
When updating existing customer data via update or upsert mode, Hightouch pulls the existing customer data from CitrusAd and merges it with your updates.
## Tips and troubleshooting
#### Delete behavior
**Clear** is only available for update mode. All sync modes use the **Do nothing** behavior by default.
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the customer record in CitrusAd with all its synced fields |
| **Clear** | Unset all the mapped fields from CitrusAd |
### Common errors
### Live debugger
### Sync alerts
---
## CleverTap
**URL:** https://hightouch.com/docs/destinations/clevertap
**Description:** Build better campaigns on CleverTap with up-to-date customer data from your data warehouse
**Section:** Destinations
## Setup
Navigate to Setting > Project in CleverTap to view credentials.
From the Project tab, copy `Project Id` as `Account Id` and `Passcode` into Hightouch. Use `CleverTap Region` for `Localization`.
## Supported types
Hightouch supports syncing to the following CleverTap types:
- **Objects**
- **Events**
- **Segments**
## Syncing objects
Hightouch supports syncing to the following CleverTap objects:
- **Profile** Hightouch allows you to sync user profiles from a source into CleverTap.
### Sync modes
- **Upsert** - In upsert mode, Hightouchs adds new rows and keeps existing rows up to date.
### Record matching
For each user profile, a user identifier is required. This is the key that CleverTap uses to find the user whose profile needs to be updated. You have to set a value for one of these parameters to identify the user: Identity, CleverTap Global Object ID, Meta ID or Google Plus ID. If this identity is not found a new user profile will be created.
You may also provide additional identifiers through Field Mapping. Profiles
are automatically unified by CleverTap.
### Field mapping
You can sync columns from your source to CleverTap.
CleverTap **highly recommends** you provide common fields like `Name, Email, Phone, etc.`. You can see additional details in the [CleverTap docs](https://developer.clevertap.com/docs/concepts-user-profiles#manually-updating-predefined-user-profile-properties).
Phone numbers must use E164 formatting. CleverTap will reject rows that aren't in this format.
Additional fields that fits your unique use case can be mapped through custom mappings.
CleverTap automatically generates schema for custom fields you provide. You
can see existing schemas by navigating to Setting > Schema > User
Properties in CleverTap.
## Syncing events
When syncing events, Hightouch treats any records added to [your source](/getting-started/concepts) as new events and sends them to CleverTap when your sync runs.
### Event name
You can provide a fixed event name or use a value from your source as event name by toggling the switch button.
New event names are automatically added to CleverTap schema.
### Record matching
For each event, a user identifier is required. This is the key that CleverTap uses to find the user whose profile needs to be updated. You have to set a value for one of these parameters to identify the user: Identity, CleverTap Global Object ID, Meta ID or Google Plus ID. If this identity is not found a new user profile will be created.
### Field mapping
You can sync any column from your source to into any event field in CleverTap.
## Syncing segments
When syncing segments or custom lists, CleverTap requires the users in the segment to have already been synced to CleverTap. You can use [sequences](/syncs/schedule-sync-with-sequences) to ensure that users are synced before syncing segments.
To start syncing, choose a field to identify users (Identity, CleverTap Global Object ID, Meta ID, or Google Plus ID), name the segment, and assign an admin's email for ownership.
CleverTap automatically generates schema for custom fields you provide. You
can see generated and existing schemas by navigating to Setting > Schema
> Events in CleverTap.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## ClickUp
**URL:** https://hightouch.com/docs/destinations/clickup
**Description:** A productivity team app for storing all essentials such as tasks, documents, goals, chat, and more.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
----------|---------------------------------------------------------|------------------------|-----------------------------------------------------------------------------
**Tasks** | Sync data from any source to ClickUp tasks and subtasks | Upsert, Update, Insert | [Tasks docs](https://clickup.com/api/clickupreference/operation/CreateTask/)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to ClickUp
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **ClickUp** and click **Continue**. You can then authenticate Hightouch to ClickUp using OAuth.
Select **Log in to ClickUp** and log into your ClickUp account.
You then need to select the ClickUp workspace or workspaces you want to connect to.
Once done, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your ClickUp destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the ClickUp destination you want to sync to.
### Syncing tasks
In ClickUp, a task is a specific action or activity that needs to be completed to achieve a particular goal or objective.
Hightouch supports syncing tasks and subtasks.
To get started, select the ClickUp **workspace**, **space**, **folder**, and **list** for your tasks.
You can then choose to **insert** new tasks, **update** existing ones, or do both in **upsert** mode.
If you want to sync subtasks, you need to map the **Parent** field. Refer to the [references section](#references) to learn more.
Hightouch doesn't update [closed or archived](https://help.clickup.com/hc/en-us/articles/6309452618647-Create-and-manage-custom-statuses#h_3e1a44262c) ClickUp tasks.
If your model contains data related to a closed or archived task, the sync creates a new task with the provided data.
#### Record matching
You can match rows from your model to tasks in ClickUp on identifer fields in ClickUp.
The identifer fields you can use depend on your sync mode:
- In **Upsert** mode, you can match on [custom fields](https://help.clickup.com/hc/en-us/articles/6303536766231-Custom-Fields-overview), which are displayed as **id** in the dropdown menu.
- In **Update** mode, you can match on any of the following ClickUp fields:
- [Task ID](https://help.clickup.com/hc/en-us/articles/6309890818071-Task-IDs)
- [Custom task ID](https://help.clickup.com/hc/en-us/articles/6304361756951-Custom-Task-IDs-ClickApp-)
- [Custom fields](https://help.clickup.com/hc/en-us/articles/6303536766231-Custom-Fields-overview)
- In **Insert** mode, ClickUp automatically generates an identifier for every new record synced, so there is no need to match an existing record.
To avoid unexpected duplicates or other potential issues, make sure you select a ClickUp field with unique values. Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
When creating a new task, ClickUp requires a name for it.
Therefore, in upsert and insert modes, you must map the **Name** field to complete your configuration.
You can also map data from any of your model columns to default fields in ClickUp.
The following default fields have some special guidelines:
- **Status**: The values you sync must be a valid statuses for your task. Status is case insensitive; for example, `in progress` is the same as `IN PROGRESS`. See [ClickUp's custom status docs](https://docs.clickup.com/en/articles/831184-create-and-manage-custom-statuses) for customization information.
- **Priority**: This is a number that corresponds to the priorities available in the ClickUp UI. **1** is the must urgent priority while **4** is the least urgent.
- **Time Estimate**: This field must be in milliseconds.
#### Arrays
You can sync array fields to ClickUp by selecting model columns that contain either comma-delimited strings or arrays.
If you sync a comma-delimited string, for example `"tag1, tag2, tag3"`, Hightouch transforms it into the array (`["tag1", "tag2", "tag3"]`) ClickUp expects.
Hightouch overwrites array values rather than aggregating them.
For example, suppose you sync the following array `["tag1", "tag2"]` to the tags field of a particular task.
Then, a later sync sends another array `["tag1", "tag3"]` to the same task's tags.
The tasks tags will be `["tag1", "tag3"]`, not `["tag1", "tag2", "tag3"]` after the later sync runs.
#### References
You can relate tasks to other ClickUp objects while field mapping.
To do so, select a ClickUp [reference](https://help.clickup.com/hc/en-us/articles/6309152582935-Create-and-view-Reference-Relationships) field.
Hightouch denotes these with a chainlink icon.
Begin by selecting one of these fields, for example **Links to**.
The Links To field lets you include a task ID to create a linked dependency for the task you're syncing.
Then, select a field on the object you're referencing to match on, for example, a task's ID, and the corresponding column from your model.
Hightouch uses this column to look up the related object's ID and create the relationship with the task you're syncing.
Generally, it's best to match on ID fields, but Hightouch also supports matching on meaningful properties like **email** for users.
A common reference field is **Parent**.
Mapping to the **Parent** field lets you sync subtasks.
The preceding screenshot shows associating a parent task to a subtask based on the parent's task ID.
The parent cannot be a subtask itself and it must be part of the ClickUp list you selected in the sync configuration.
Hightouch syncs only one value to reference fields. This includes reference fields like **assignees** that allow multiple values.
#### Custom field mapping
Hightouch supports syncing data to these [custom field types](https://docs.clickup.com/en/articles/2310503-custom-fields-overview):
|Custom field type|Expected data type or description|Example|
|--|--|--|
|**Checkbox**|Boolean value|`true` or `false`|
|**Date**|Millisecond [Unix](https://en.wikipedia.org/wiki/Unix_time) timestamp; See the [date formatting](#formatting-dates) tip. |`1682452269219`|
|**Dropdown**|A series of options in a menu|-|
|**Email**|Valid email address|`lana@clickup.com`|
|**Labels**|A flexible list of options (invalid labels will be omitted)|-|
|**Money**|A numerical value|`9.99`|
|**People**|People or teams, see (#references)|`Sam`|
|**Phone**|A valid phone number with country and area code|`"+1 123 456 7890"`|
|**Rating**|A number value from `1` through `5`|`5`|
|**Tasks**|Tasks linked together without relationships|`Task A`|
|**Text**|Single-line text|`"This is a single-line text"`|
|**Text Area**|Multi-line text|`"This is a multi-line text..."`|
|**Website**|A valid URL|`https://clickup.com`|
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options for upsert mode:
Behavior | Description
---------------|------------------------------------------------------------
**Do nothing** | Keep the task in ClickUp with all its synced fields
**Clear** | Clear all the mapped fields, but keep the task in ClickUp
**Delete** | Delete the synced tasks from ClickUp
Update mode has these options:
Behavior | Description
---------------|------------------------------------------------------------
**Do nothing** | Keep the task in ClickUp with all its synced fields
Insert mode doesn't support any delete behavior.
## Tips and troubleshooting
### Formatting dates
ClickUp expects millisecond [Unix](https://en.wikipedia.org/wiki/Unix_time) timestamps for all date fields.
A millisecond timestamp is numerical value with 13 digits, for example, `1620170285617`, as compared to a traditional [Unix timestamp](https://en.wikipedia.org/wiki/Unix_time) which has ten.
### Common errors
### Live debugger
### Sync alerts
---
## ClientSuccess
**URL:** https://hightouch.com/docs/destinations/client-success
**Description:** Improve product adoption and reduce churn by enriching ClientSuccess with rich, up-to-date customer data from your data warehouse
**Section:** Destinations
## Overview
This integration allows you to sync Clients, Contacts and Subscriptions from your warehouse to ClientSuccess.
## Setup
To enable syncing with [ClientSuccess](https://www.clientsuccess.com/) first you need a username and a password for
a user to perform the API calls on your behalf.
Then copy these values, and create a new Destination in Hightouch, selecting Client Success as the destination type. Lastly, paste
the values into the configuration form. Save the destination and it's ready to start syncing data.
## Syncing
When creating a sync, first you need to select the type of object you want your data to be transformed into. Following that, you
could define the ID column and the rest of the mappings.
If you have custom-defined fields in your account, they will appear as mappings, with the appropriate data type.
---
## Close
**URL:** https://hightouch.com/docs/destinations/closeio
**Description:** Close more deals by keeping your prospect data up-to-date and in sync with data in your warehouse
**Section:** Destinations
## Overview
Your leads and prospects expect you to have the necessary context before engaging with them. You can accomplish this by syncing lead and prospect data from your data warehouse to Close. You no longer need to worry about executives chasing the wrong leads or engaging with warm ones without proper context.
## Supported syncing
|Sync Type|Description|Supported Sync Modes | API Reference|
|--|--|--|--|
|**Lead**|Sync data from any source to Leads|Upsert, Update| [Lead docs](https://developer.close.com/resources/leads/)|
|**Contact**|Sync data from any source to Contacts|Upsert, Update|[Contact docs](https://developer.close.com/resources/contacts/)|
|**Opportunity**|Sync data from any source to Opportunities|Upsert, Update|[Opportunity docs](https://developer.close.com/resources/opportunities/)|
|**Note Activity**|Sync data from any source to Note activities|Insert, Update| [Note docs](https://developer.close.com/resources/activities/note/)|
|**Custom Activity**|Sync data from any source to custom activities|Upsert, Update|[Custom Activities docs](https://developer.close.com/resources/custom-activities/)|
When inserting Note activities, Hightouch creates and attaches new Note activities to the Lead object you specify in your sync configuration.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Close
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Close** and click **Continue**. You can then authenticate Hightouch to Close with a Close API key.
You can generate a Close API Key in your Close account under [**Settings > API Keys**](https://app.close.com/settings/api/).
For more information, refer to [Close's docs](https://help.close.com/docs/api-keys).
## Sync configuration
Once you've set up your Close destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Close destination you want to sync to.
### Record matching
You can match rows from your model to Close records on any unique property.
You must match Contacts by either email or the contact's ID.
### Field mapping
Use field mapping to sync any model columns to Close's default and custom fields.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## CockroachDB
**URL:** https://hightouch.com/docs/destinations/cockroachdb
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Unique identifiers
Hightouch requires a unique identifier in the destination Cockroach table to add, remove, and update rows. The column must be either a `UNIQUE` or `PRIMARY KEY` column. At least one Hightouch field must map to a unique column.
If you're unsure whether your mapping includes a compatible column, just try it. Hightouch will error if you need to change your mappings.
## Column types
Hightouch works out of the box with all standard column types, including:
- `BIGINT`
- `INT`
- `TEXT`
- `VARCHAR`
- `TIMESTAMPTZ`
- `BOOLEAN`
- `DECIMAL`
If you see type errors, it may be because your SQL query is producing the wrong format. Message us on Intercom if you need help.
## Required permissions
Your Cockroach credentials must be able to:
- `SELECT`, `INSERT`, `UPDATE` and `DELETE` rows from the destination table.
- Perform the following queries:
- `SHOW SCHEMAS`
- `SHOW TABLES FROM `
- `SHOW CONSTRAINTS FROM .
`
- `SHOW COLUMNS FROM .
`
---
## CJ Affiliate
**URL:** https://hightouch.com/docs/destinations/commission-junction
**Description:** CJ Affiliate's technology powers a partnership ecosystem where over 167K publishers and brands engage billions of consumers worldwide.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|------------|--------------------------------------------------|------------------------|------------------------------------------------------------------------------------|
| **Orders** | Sync data from any source to CJ Affiliate orders | Upsert, Update, Insert | [Orders docs](https://developers.cj.com/docs/advertiser-api-tracking/api-overview) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to CJ Affiliate
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **CJ Affiliate** and click **Continue**. You can then authenticate Hightouch to **CJ Affiliate**.
Enter the following fields into Hightouch:
- **Advertiser ID**: Enter your advertiser ID.
- **Access token**: Enter your personal access token.
## Sync configuration
Once you've set up your CJ Affiliate destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the CJ Affiliate destination you want to sync to.
### Syncing orders
Sync data from any source to CJ Affiliate orders.
#### Record matching
To match rows from your model to orders in CJ Affiliate, you need to select the model column that contains values that match the **Order ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, CJ Affiliate automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync order fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom order fields.
CJ Affiliate requires the following fields, so you must map them to complete your configuration:
- **Order ID**
- **Enterprise ID**
- **Action tracker ID**
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Common Room
**URL:** https://hightouch.com/docs/destinations/common-room
**Description:** Get a unified view and automate using every signal across social, community, product, CRM, and more. Initiate conversations, grow adoption, and accelerate pipeline faster than ever before.
**Section:** Destinations
## Supported syncing
| Sync Type | Supported Sync Modes |
| --------------------------------------- | -------------------- |
| **Community Members Information** | Upsert |
| **Community Organizations Information** | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Getting started
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Common Room** and click **Continue**. Then paste in your customer user Common Room URL.
## Sync configuration
Once you've set up your Common Room destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Common Room destination you want to sync to.
#### Field mapping
You can sync columns from your source to Common Room default and custom fields.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Contentstack
**URL:** https://hightouch.com/docs/destinations/contentstack
**Description:** Contentstack is a modern, headless CMS built for teams that want to ship content faster everywhere. Plug in any front-end and use real-time collaboration and workflow automation to keep content fresh and consistent across all platforms.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
----------|-------------------------------------------------|----------------------|----------------------------------------------------------------
**Users** | Sync data from any source to Contentstack users | Update | [Users docs](https://docs.lytics.com/reference/put_attributes-table-key-value)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Contentstack
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Contentstack** and click **Continue**. You can then authenticate Hightouch to **Contentstack** using a Data Activation Layer (DAL) access token.
### Create a Data Activation Layer access token
1) From your Contentstack account, create a new Data Activation Layer (DAL).
2) Access your DAL through **Data and Insights**, and under **Account** and **Access Tokens**, create a new access token. For token roles, select **Data Manager** under **Custom Roles** and generate your token.
If you set an expiration for your access token, ensure that your access token has not expired if you notice syncs failing due to authorization issues.
## Sync configuration
Once you've set up your Contentstack destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Contentstack destination you want to sync to.
### Syncing users
Sync data from any source to Contentstack users.
#### Record matching
To match rows from your model to users in Contentstack, you need to select a model column and corresponding Contentstack field. You can match on any of the following Contentstack fields:
- **External ID**
- **Email**
- **Hashed email**
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync user attributes via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom attributes that are defined in Contentstack.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Adding users to audiences
Contentstack allows you to pass a list of audience IDs to **External Audiences** for users to add them to audiences. The user will be added to every audience that is included in the list passed in. The unique IDs of audiences can be found under **Personalize** > Select a personalize project > **Audiences**.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Cordial
**URL:** https://hightouch.com/docs/destinations/cordial
**Description:** Sync data to Cordial through Contacts API
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
| ---------- | -----------------------------------------------------| ---------------------- | ------------------------------------------------------------------------------------------------------- |
| **Contacts** | Sync data from any source to Cordial as contacts | Upsert, Update, Insert | [Contacts](https://support.cordial.com/hc/en-us/articles/203885958-Contacts-API)|
| **Supplements** | Sync data from any source to Cordial's supplemental data sets | Upsert, Update, Insert | [Supplements](https://support.cordial.com/hc/en-us/articles/204570687-Supplements-API)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Cordial
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Cordial** and click **Continue**.
Enter your API key from your Cordial account.
If you don't have an API key, follow the [instructions on **Cordial** to create API keys](https://support.cordial.com/hc/en-us/articles/115005365087). Ensure the API key has write access for the object you are syncing.
Based on the sync object, ensure your API key has access to the following API endpoints:
| Object | Required API endpoints |
|--------------|------------------------------------|
| **Contacts** |
`GET /accountcontactattributes`
`GET/POST/PUT /contacts`
|
| **Supplements** |
`GET /supplements`
`GET/POST/PUT /supplements//records`
|
Next, select which Cordial domain your account uses. This will either be `admin.cordial.io` or `usw2.admin.cordial.io`. You can determine which one your account uses by navigating to your Cordial dashboard and checking the website URL.
## Sync configuration
Once you've set up your Cordial destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Cordial destination you want to sync to.
Select which object you'd like to sync.
### Sync mode
- **Upsert**: pushes new objects to Cordial and updates mapped fields that change in your source.
- **Update**: updates mapped fields on existing objects in Cordial. New objects are not added.
- **Insert**: pushes new objects to Cordial. Existing objects are not updated.
### Field mapping
#### Contacts
Cordial has the following [contact attribute types](https://support.cordial.com/hc/en-us/articles/204570347-Accountcontactattributes):
- string
- number
- date (ISO 8601 standard)
- geo
- array
Hightouch supports syncing with all of these types. You may use Hightouch's object inline mapping to construct Cordial's object values from your source.
You **must** create a contact attribute on Cordial before syncing to that field. Follow the [instructions on **Cordial**](https://support.cordial.com/hc/en-us/articles/115005528368-Contact-attributes#addAttributes) either via UI or API to create contact attributes.
#### Supplements
Start syncing supplements by [creating a supplement collection on Cordial](https://support.cordial.com/hc/en-us/articles/115005528468-Supplements#ui). Indexes on supplement collections are **required** mapping fields on Hightouch.
If a field you'd like to map to is not available in the mapping dropdown, you can enter the field name manually:
### Record matching
#### Contacts
In **Upsert** and **Update** mode, records must be matched by either Cordial ID or Email. Records without unique IDs will fail to sync.
In **Insert** mode, records must have an **email field** to successfully be added to Cordial. Records without an email identifier will fail to sync.
#### Supplements
Since Cordial does not create ID fields for supplements, **id** is required in **Insert** mode. In **Upsert** and **Update** mode, **id** is also required to match records. Records without unique IDs will fail to sync.
Cordial [does not allow](https://support.cordial.com/hc/en-us/articles/204570687-Supplements-API#postSuppRecords) 24 character hex strings as ID values for supplements.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Courier
**URL:** https://hightouch.com/docs/destinations/courier
**Description:** Power product notifications with your warehouse data in Courier
**Section:** Destinations
## Supported sync methods
Hightouch can sync data to Courier Profiles and List Subscriptions
### Courier profiles
#### Sync modes
Hightouch supports Upsert and Update sync modes:
- In the Upsert mode, new rows will be inserted into Courier and Profile attributes will be kept up-to-date within Courier
- In the Update mode, Profile attributes will be kept up-to-date within Courier.
#### Record matching
Database records can be matched from your source to your Courier workspace by a unique property. Profiles can be matched by a `Recipient ID`.
#### Field mapping
You can sync columns from your source to Courier's Profile default and custom fields.
#### Delete mode
You can choose what Hightouch's behavior is when records leave the query result set. The default is doing nothing, but you can also set Hightouch to delete or clear the Courier profile on record exit.
### Creating a new list
In your sync configuration, you can tell Hightouch to create a new Courier list for you.
You can optionally assign a custom name to your list. If no name is provided, Hightouch will default to using your model name.
### Syncing to existing lists
Instead of creating a new list, Hightouch can also sync recipients to existing lists that are already populated in your Courier workspace.
Suppose you want to sync to multiple existing lists but do not want to create a new sync for every list. As long as your model has a `list_id` column associated to a row, Hightouch can sync to multiple Courier lists in just one sync.
## Getting started
If you're new to Hightouch, be sure to [sign up for a free account](https://app.hightouch.com/signup). Your first destination is free, and Courier is a great way to get started.
Before diving into this tutorial, we also recommend reading [this article on our core concepts](/getting-started/concepts). It's a quick overview of how Hightouch works.
### Connect to Courier
#### Create an API token
1. Log in to Courier
2. Visit your **Settings** page
3. Click on the **API Keys** tab
4. If you already have an API key set up, copy the **Published** token, otherwise click **Generate New API Key**
#### Add Courier as a destination in Hightouch
1. Log in to Hightouch or create a free Hightouch account
2. Click on **Destinations** in the left sidebar
3. Click on **Add destination** in the top right corner
4. Select **Courier** in the destination catalog and click **Continue** to proceed
5. Paste your API token into the **API Key** field and click **Continue** to proceed
6. Give your destination a name and unique slug (for example, “Courier Production” and `courier-production`)
7. Click **Finish** to create your Courier destination.
Now that we've linked Hightouch to your Courier account, it's time to connect to build profiles and list subscriptions using the data that resides in your data warehouse.
### Connect to your data source
Hightouch sits on top of your data warehouse and can read from Snowflake, Redshift, BigQuery, Databricks, Postgres, MySQL, and many other data sources. You can even fetch data from Google Sheets and Airtable. Hightouch can help activate your data wherever it resides.
We've written dedicated guides for each supported data source. Take your pick:
- Connect to [Airtable](/sources/airtable)
- Connect to [Amazon Athena](/sources/amazon-athena)
- Connect to [Amazon Redshift](/sources/amazon-redshift)
- Connect to [ClickHouse](/sources/clickhouse)
- Connect to [Databricks](/sources/databricks)
- Connect to [Elasticsearch](/sources/elasticsearch)
- Connect to [Firebolt](/sources/firebolt)
- Connect to [Google BigQuery](/sources/google-bigquery)
- Connect to [Google Sheets](/sources/google-sheets)
- Connect to [Looker](/sources/looker)
- Connect to [Metabase](/sources/metabase)
- Connect to [MySQL](/sources/mysql)
- Connect to [Palantir Foundry](/sources/foundry)
- Connect to [PostgreSQL](/sources/postgresql)
- Connect to [Rockset](/sources/rockset)
- Connect to [SFTP](/sources/sftp)
- Connect to [Snowflake](/sources/snowflake)
- Connect to [SQL Server](/sources/sqlserver)
- Connect to [Trino](/sources/trino)
If you're missing the credentials necessary to connect to your data source,
try inviting your teammates to Hightouch. You can have unlimited collaborators
in your workspace.
#### Connect to a sample database instead
Waiting for credentials to your data warehouse? No problem. You can connect to one of our [sample datasets](/sources/sample-data) to get started right away. For the rest of this tutorial, we'll use the **B2B SaaS** sample source, which contains a `users` table with names, emails, locations, and other user attributes. This sample source is available when you log in for the first time—no setup required.
Next, we'll write a SQL query to define which profiles and list subscriptions should be synced to Courier.
### Create a model
In Hightouch, a model represents a query that filters or transforms the records in your data source. Models are used to determine exactly which records to sync between sources and destinations.
When you connect to a data warehouse, your models will probably be written in SQL. You can also import models from tools like dbt and Looker.
In this example, we'll keep it simple. Suppose you want to use Courier to send notifications only for your customers located in the city of Nashville. To accomplish this, you'll want to create a model that looks like this:
```sql
SELECT * FROM users WHERE location = 'Nashville';
```
1. Click on **Models** in the left sidebar
2. Click on **Add model** in the top right corner
3. Select **Demo Database** as your source and click **Continue** to proceed
4. Select **SQL Editor** as your modeling method
5. Paste the SQL query from above into the editor on the left side of the page
6. Click **Preview** to execute the query
7. Click **Continue** to proceed
8. Give your model a name and unique slug (for example, "Users in Nashville" and `users-in-nashville`)
9. Select **ID** as the primary key for your model
10. Click **Finish** to create your model.
We just created a model using a SQL query that filters our `users` table down to only those located in Nashville. Next, we'll configure our sync to add these users to Courier.
### Create a sync to Courier Profiles
1. Click on **Syncs** in the left sidebar
2. Click on **Add sync** in the top right corner
3. Select **Users in Nashville** as your model and click **Continue** to proceed
4. Select **Courier Production** as your destination and click **Continue** to proceed
5. Select **Profile** as the object you wish to update in Courier
6. Select which update mode you want to use
7. Identify which field from your model represents the identifier known to Courier
8. Identify which fields from your model you want to sync to a Courier Profile
9. Select how you want to handle records leaving your source
10. Give a name to your segment (for example, "My beta test")
11. Click **Continue** to proceed to the last step
12. Select **Interval** to tell your sync to run on a set interval
13. Configure your sync schedule to run every **5 minutes**
14. Click **Finish** to create your sync.
Your Courier sync configuration would look something like this.
And your sync schedule configuration would resemble this.
### Create a sync to Courier Lists
1. Click on **Syncs** in the left sidebar
2. Click on **Add sync** in the top right corner
3. Select **Users in Nashville** as your model and click **Continue** to proceed
4. Select **Courier Production** as your destination and click **Continue** to proceed
5. Identify which field from your model represents the identifier known to Courier
6. Select Create a new segment
7. Give a name to your segment (for example, "My beta test")
8. Click Continue to proceed to the last step
9. Select Interval to tell your sync to run on a set interval
10. Configure your sync schedule to run every 5 minutes
11. Click Finish to create your sync.
Your Courier sync configuration would look something like this.
And your sync schedule configuration would resemble this.
## Tips and troubleshooting
### Common errors
#### Don't see your data in Courier?
Make sure you've configured your destination to use the correct Courier environment API key. There may be quite a few so you might've accidentally copied the wrong one. If that doesn't solve your issue, . We're happy to help.
### Live debugger
### Sync alerts
---
## Criteo
**URL:** https://hightouch.com/docs/destinations/criteo
**Description:** Reach the people who will care most about you, whether they’re just discovering you, or have engaged with you before.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| ------------- | --------------------------------------------------- | -------------------- | --------------------------------------------------------------------------------------- |
| **Audiences** | Create audiences and update contact list membership | Add, Remove | [Audiences reference](https://developers.criteo.com/marketing-solutions/docs/audiences) |
## Connect to Criteo
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Criteo** and click **Continue**. You can then authenticate Hightouch to Criteo using OAuth.
Click **Log in to Criteo**, log in with your Criteo account, and click **Approve**.
Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Criteo destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Criteo destination you want to sync to.
Whether you're syncing events or audiences, begin your configuration by selecting the advertiser ownign the audience you want to sync to.
### Syncing audiences
Syncing to Criteo's [Audience API](https://developers.criteo.com/marketing-solutions/docs/audiences) lets you manage
your contact lists by creating, deleting, and updating audience data used for advertising campaigns.
#### Select an existing audience or create a new one
You can create a new audience or use an existing one. When creating a new audience, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model. To use an existing audience, select the desired audience from the dropdown.
#### User identifiers
To identify which users to add or update in an audience, select model columns and the corresponding Criteo fields.
You can match on any of the following Criteo fields:
- Email
- Mobile ad identifier
- LiveRamp identity link
- Criteo GUM cookie identifier
When using the Criteo GUM cookie identifier, you must also enter your
advertiser's GUM Caller ID. To get the GUM Caller ID, reach out to your Criteo
account team.
#### Delete behavior
You can choose how to handle user records in Criteo when the corresponding rows are deleted in your source.
| Behavior | Description |
| -------------------- | -------------------------------------------- |
| **Do nothing** | Keep the contact in your Criteo audience |
| **Remove from list** | Remove the contact from your Criteo audience |
## Tips and troubleshooting
### Matched users count
### Common errors
### Live debugger
### Sync alerts
---
## CrowdTwist
**URL:** https://hightouch.com/docs/destinations/crowdtwist
**Description:** Oracle CrowdTwist is a SaaS based omni-channel loyalty and analytics platform that is designed to allow marketers to acquire, engage and retain customers
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
| ---------- | -----------------------------------------------------| ---------------------- | ------------------------------------------------------------------------------------------------------- |
| **Users** | Sync data from any source to CrowdTwist as users | Update | [User Update](https://docs.oracle.com/en/cloud/saas/marketing/crowdtwist-develop/Developers/UserUpdate.html)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to CrowdTwist
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **CrowdTwist** and click **Continue**.
Enter your API key. If you do not have a CrowdTwist API key, follow the instructions [here on CrowdTwist](https://docs.oracle.com/en/cloud/saas/marketing/crowdtwist-develop/Developers/GeneratingAPIKeys.htm) to generate a Client API key.
## Sync configuration
Once you've set up your CrowdTwist destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the CrowdTwist destination you want to sync to.
### Record matching
CrowdTwist uses the following ID types to identify users:
- Email address
- Meta user ID
- X user ID
- CrowdTwist ID
- Third party ID
- Username
- Mobile phone number
Select which ID you'd like to use to identify users in your model.
### Field mapping
Sync any column in your model to map to any of the following user attributes in CrowdTwist:
- First name
- Last name
- Date of birth
- Email address
- Email is verified
- Is active
- Remove points
- Language preference
- Postal code
- Username
- Password
- Third party ID
- City ID
- City name
- Country code
- Meta access token
- Meta user ID
- Instagram access token
- Instagram username
- Instagram user ID
- Foursquare access token
- Foursquare user ID
- Extra data
- Gender ID
- Middle name
- Mobile phone number
- Receive email updates
- Custom data
- Tier override
- Password generated
Reference [CrowdTwist's documentation](https://docs.oracle.com/en/cloud/saas/marketing/crowdtwist-develop/Developers/UserUpdate.html) for more detail on the types and requirements for each attribute to reduce the likelihood of rejected rows in your sync.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Embedded Destination
**URL:** https://hightouch.com/docs/destinations/custom
**Description:** Build a custom connector that looks and feels just like a native destination
**Section:** Destinations
If you need to connect to a destination Hightouch doesn't support yet, we recommend using the [HTTP Request destination](/destinations/http-request). Please also —we're always gauging interest in new destinations.
## Overview
The Embedded Destination framework allows Hightouch users to develop their own custom data connector and use it as an extension of Hightouch's existing library.
If you are looking to send data to a private API or a SaaS tool that Hightouch doesn't support, you can use Hightouch to send data to your own connector in a specific format, while continuing to use all of our alerting, reliability, and error handling systems.
To implement the destination, Hightouch will need users to provide information on what objects, modes, and fields are available in the destination, as well as how data is synced.
## Example destinations
Our team has produced some [example destinations](https://github.com/hightouchio/custom-destinations) for reference.
- Slack: Sends messages formatted with Liquid to a Slack channel as rows are added to the query results
- Mailchimp: Adds and updates contacts in a Mailchimp audience list
## Setup
Embedded destinations in Hightouch are implemented by providing information of two steps:
- Sync configuration: Hightouch needs information on what objects, modes, and fields are available, as well as custom forms
- Sync execution: Hightouch sends data that needs to be synced to a method on your connector, which should format and send the data to the destination and return a success or error response
### Sync configuration
Hightouch uses the following methods to render the configuration form.
These methods can be optionally implemented and used in any combination:
- `list_objects`: Retrieves a list of objects available in the destination (for example, users, organizations, etc.)
- `list_modes`: Retrieves a list of modes available in your implementation (for example, upsert, update, mirror, etc.)
- `list_fields`: Retrieves a list of object fields, including identifiers and custom fields
- `list_options`: Retrieves a JSON schema of custom configuration fields, which will be rendered into a form based on [react-jsonschema-form](https://rjsf-team.github.io/react-jsonschema-form)
- `validate`: This method is called when a user attempts to save the configuration form, and is used to display errors that aren't handled automatically by default or by JSON schema
#### Example 1: Objects, modes, fields, and options
When all the methods are implemented and all the concepts are used in conjunction, there is a hierarchy where objects must be selected first before modes are shown, and modes must be selected before fields and options are shown.
This is because the mode options shown are conditional on the object selected, for example, "upsert" and "update" can be enabled for the "user" object, but only "update" is enabled for the "organizations" object.
This will render a form similar to the example above.
#### Example 2: Options only
When only `list_options` is implemented, Hightouch will render a form based on the JSON schema. Note that all formats such as dropdowns, arrays, text areas are available. In this case, only the `list_options` method is implemented.
The other methods such as `list_objects` aren't implemented, and therefore selecting an object is not required.
### Sync execution
The Embedded Destination acts as a transformation and parsing layer between Hightouch and any API. Hightouch calls the following methods when a sync is run, either by schedule or manual run.
When a sync is run, Hightouch first calls for information on sync behavior, then sends sync requests of each operation (add, change, remove) with the data and configuration in batches of a specified size:
- `behavior`: Retrieves behavior settings such as batch size, which operations to send, etc. for the sync run
- `add`: This method is called with a batch of raw data from rows that have been added since the last sync run, as well as configuration details
- `change`: This method is called with a batch of raw data from rows that have changed since the last sync run, as well as configuration details
- `remove`: This method is called with a batch of raw data from rows that have been removed since the last sync run, as well as configuration details
Each of the `add`, `change`, `remove` methods should return a response with any errors encountered when sending the data to the destination API.
## API reference
Implement the following methods in a JSON-RPC server available via a public URL.
The server can be implemented with packages such as:
- [Jayson](https://github.com/pavlov99/json-rpc): A JSON-RPC server for node.js
- [json-rpc](https://github.com/pavlov99/json-rpc): A JSON-RPC implementation written on Python
### `list_objects`
This method should return the objects that are available to sync to in the destination API.
#### Request body
`list_objects` has no request body since it's the highest level method.
#### Response sample
```json
{
"objects": [
{ "label": "User", "id": "user", "description": "A user." },
{
"label": "Organization",
"id": "organization",
"description": "An organization."
}
]
}
```
#### Properties of an `object`
| Name | Description |
| ---------------------- | ------------------------------------------------------------------ |
| `id` | ID of object option will be referenced in the configuration |
| `label` | Label of object option shown in the dropdown |
| `description` (optional) | Description of object option (not shown in the dropdown currently) |
### `list_modes`
This method should return the sync modes that are available for a specific object (or no object).
#### Request body
| Name | Description |
| ------ | ------------------------------------------------------------------------------------------------------------------------------ |
| `object` | ID of object selected in the configuration (can be undefined if `list_object` not implemented or returned an empty object array) |
#### Response sample
```json
{
"modes": [
{
"id": "upsert",
"label": "Upsert",
"description": "Pushes new records and updates records that change in your source."
}
]
}
```
#### Properties of a `mode`
| Name | Description |
| ---------------------- | --------------------------------------------------------- |
| `id` | ID of mode option will be referenced in the configuration |
| `label` | Label of mode option shown in the radio selector |
| `description` (optional) | Description of mode option shown in the radio selector |
### `list_fields`
This method should return the mapping fields that are available for a specific object and mode (or no object and mode).
#### Request body
| Name | Description |
| ------ | ------------------------------------------------------------------------------------------------------------------------------ |
| `object` | ID of object selected in the configuration (can be undefined if `list_object` not implemented or returned an empty object array) |
| `mode` | ID of mode selected in the configuration (can be undefined if `list_mode` not implemented or returned an empty mode array) |
#### Response sample
```json
{
"fields": [
{
"id": "email",
"label": "Email Address",
"type": "STRING",
"identifier": true
},
{
"id": "first_name",
"label": "First Name",
"type": "STRING",
"required": true
},
{
"id": "last_name",
"label": "Last Name",
"type": "STRING",
"required": true
}
]
}
```
#### Properties of a `field`
| Name | Description |
| ---------------------- | ------------------------------------------------------------------------- |
| `id` | ID of the field option will be referenced in the configuration |
| `label` | Label of field option shown in the dropdown |
| `description` (optional) | Description of field option (not shown in the dropdown currently) |
| `required` (optional) | Whether the field is required to be mapped |
| `type` (optional) | The type of the field (check the field types section for accepted values) |
#### Field types
Hightouch supports the following field types. Hightouch will not automatically cast the value to this type, nor will it validate whether the data in the row is of that type. it's generally just for suggestion and display purposes.
```jsx
STRING;
ENUM;
NUMBER;
BOOLEAN;
DATE;
DATETIME;
EMAIL;
OBJECT;
ARRAY;
REFERENCE;
UNKNOWN;
```
### `list_options`
This method should return the `schema` and `uiSchema` of the form that you would like to render. Hightouch uses [React JSON Schema Form](https://react-jsonschema-form.readthedocs.io/en/latest/) to build a form out of a semi-standard JSON Schema.
If your schema at the highest level is an object, we will spread the properties into the configuration, how
| Name | Description |
| ------ | ------------------------------------------------------------------------------------------------------------------------------ |
| `object` | ID of object selected in the configuration (can be undefined if `list_object` not implemented or returned an empty object array) |
| `mode` | ID of mode selected in the configuration (can be undefined if `list_mode` not implemented or returned an empty mode array) |
#### Response sample
```json
{
"options": {
"schema": {
"type": "object",
"properties": {
"channel": {
"title": "Channel",
"description": "Channel that messages are sent to.",
"type": "string",
"enum": [1, 2, 3],
"enumNames": ["#general", "#engineering", "#marketing"]
},
"message": {
"label": "Message",
"description": "Template for messages in Liquid format.",
"type": "string"
}
}
},
"uiSchema": {
"message": {
"ui:widget": "textarea"
}
}
}
```
#### Properties of `options`
| Name | Description |
| -------- | --------------------------------------------------------------------------------------------- |
| `schema` | This property defines which fields will appear in the custom configuration form. |
| `uiSchema` | This property defines the appearance of fields, such as converting text inputs to text areas. |
### `validate`
This method receives a JSON of the configuration and should return an array of errors. If there is no error, it should return an empty array.
#### Request body
| Name | Description |
| ------------- | ------------------------------------------------------------------------------------- |
| configuration | The sync configuration as a JSON, including properties such as object, mode, mappings |
#### Request sample
```json
{
"configuration": {
"object": "user",
"mode": "upsert",
"identifierMapping": [{ "from": "user_email", "to": "email" }],
"mappings": [
{ "from": "user_fn", "to": "first_name" },
{ "from": "user_ln", "to": "last_name" }
],
"customMappings": [{ "from": "user_ltv", "to": "ltv" }],
// the following properties come from custom configuration from list_options
"archiveOnRemove": true
}
}
```
#### Response sample
```json
{
"errors": ["If first name is mapped, last name must also be mapped."]
}
```
### `behavior`
This method should return the operations that Hightouch should send, as well as the batch size Hightouch should send at.
#### Request body
`behavior` has no request body.
#### Response sample
```json
{
"batchSize": 100,
"operations": {
"add": true,
"change": true,
"remove": false
},
"skipInitialRun": false
}
```
### `add` `change` `remove`
This method should send the batch of data forward to the destination API and return an array of rejected rows if any errors occurred.
#### Request body
| Name | Description |
| ------------- | ------------------------------------------------------------------------------------- |
| `idColumn` | The column name of the ID selected as the primary key |
| `configuration` | The sync configuration as a JSON, including properties such as `object`, `mode`, `mappings` |
| `rows` | An array of rows from the query results |
#### Request sample
```json
{
"idColumn": "user_email",
"configuration": {
"object": "user",
"mode": "upsert",
"identifierMapping": [{ "from": "user_email", "to": "email" }],
"mappings": [
{ "from": "user_fn", "to": "first_name" },
{ "from": "user_ln", "to": "last_name" }
],
"customMappings": [{ "from": "user_ltv", "to": "ltv" }],
// the following properties come from custom configuration from list_options
"archiveOnRemove": true
},
"rows": [
{
"user_email": "alice@hightouch.com",
"user_fn": "Alice",
"user_ln": "Doe",
"user_ltv": 0
},
{
"user_email": "bob@hightouch.com",
"user_fn": "Bob",
"user_ln": "Doe",
"user_ltv": 10
}
]
}
```
#### Response sample
```json
{
"rejectedRows": [
{ "id": "alice@hightouch.com", "reason": "Couldn't insert user" }
]
}
```
#### Properties of a `rejectedRow`
| Name | Description |
| ------ | ---------------------------------------- |
| `id` | The ID of the row referenced by `idColumn` |
| `reason` | Reason for failure |
---
## Customer.io
**URL:** https://hightouch.com/docs/destinations/customerio
**Description:** Deliver exceptional engagement campaigns using the freshest customer data from your warehouse
**Section:** Destinations
## Overview
You want to provide relevant, timely interactions to your customers. Doing so means that data in your Customer.io account needs to be as accurate and fresh as possible. By syncing customer data directly from your data warehouse into Customer.io, you no longer need to worry about data consistency.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
|---------------|-------------------------------------------------------|----------------------|
| **Customers** | Sync data from any source to customer objects | Upsert, Update |
| **Events** | Sync data from any source as customer events | Insert |
| **Segments** | Sync data from any source to new or existing segments | Insert |
| **Objects** | Sync data from any source to custom objects | Upsert, Update |
## Getting started
To give Hightouch access to Customer.io, you need to enter:
- tracking site ID
- tracking API key
- app API key
- the region of your Customer.io account
Follow the instructions in the [following sections](#retrieve-tracking-site-id-and-api-key) to retrieve your site ID and API keys.
If you restrict access to your Customer.io workspaces by IP address, you need to add the [Hightouch IP Addresses](/security/networking#ip-addresses) to your **IP Address Allowlist**.
If you see "**Any IP address is currently able to access the App API for all workspaces.**" in your Customer.io configuration, it means you can skip this step.
### Retrieve tracking site ID and API key
1. Navigate to your [Account Settings](https://fly.customer.io/settings) and [Manage API Credentials](https://fly.customer.io/settings/api_credentials?keyType=tracking) page.
2. Under **Tracking API Keys**, copy a pre-created **Site ID** and **API Key**, or create a new one to use for Hightouch.
### Retrieve app API key
1. Navigate to your [Account Settings](https://fly.customer.io/settings) and to the App API Keys tab under the [Manage API Credentials](https://fly.customer.io/settings/api_credentials?keyType=app) page.
2. Create an **App API Key** and assign it to the appropriate workspace, for example "Hightouch."
## Syncing data
### Customers
Sync data from any source to [customer objects](https://www.customer.io/docs/api/#tag/Track-Customers).
#### Record matching
You can match rows in your model with users in Customer.io on the following fields:
- Customer.io External ID
- Email
- CIO ID
#### Field mapping
You can sync columns from your source to Customer.io customer attributes. If the customer attribute doesn't exist, Hightouch creates it.
#### Delete behavior
In **Upsert** mode, you have the following options for how Hightouch should handle rows leaving your query results:
| Behavior | Description |
| ----------------------------- | ------------------------------------------------------ |
| **Do nothing** | Keep the customer in Customer.io |
| **Clear Fields** | Clear the mapped fields but keep the customer record |
| **Delete Destination Record** | Delete the customer record from Customer.io completely |
In **Update** mode, you have the following options for how Hightouch should handle rows leaving your query results:
| Behavior | Description |
| ---------------- | ---------------------------------------------------- |
| **Do nothing** | Keep the customer in Customer.io |
| **Clear Fields** | Clear the mapped fields but keep the customer record |
### Objects
Sync data from any source to [custom objects](https://docs.customer.io/api/track/#section/Identify-object).
#### Record matching
You can match rows in your model with objects in Customer.io on the `Object ID` field
#### Field mapping
You can sync columns from your source to Customer.io object attributes. If the object attribute doesn't exist, Hightouch creates it.
#### Relationships
You can create relationships from each object synced to a customer in Customer.io. You can create relationships to the following fields:
- Customer.io External ID
- Email
- CIO ID
#### Delete behavior
You have the following options for how Hightouch should handle rows leaving your query results:
| Behavior | Description |
|-------------------------------|------------------------------------------------------|
| **Do nothing** | Keep the object in Customer.io |
| **Delete destination record** | Delete the object record from Customer.io completely |
#### Record matching
You can match rows in your model with users in Customer.io on the following fields:
- Customer.io External ID
- Email
- CIO ID
### Events
You can sync the following [customer event](https://www.customer.io/docs/api/#operation/track) types:
- **Track event**
- **Track page views**
- **Track mobile screen views**
#### Event name
Provide an event name. This is how you can reference the event in Customer.io campaigns or segments.
#### Event ID (optional)
Event ID is an identifier Customer.io uses to de-duplicate events. If provided and an event has the same value as previously synced event, Hightouch doesn't process the duplicate event.
#### User ID
A User ID is a unique value representing a user. You may identify a user by ID, email address, or Customer.io ID.
If you wish to sync the events as [_anonymous_](https://www.customer.io/docs/api/#operation/trackAnonymous), you can skip providing a User ID and provide an Anonymous ID instead.
#### Event timestamp
Provide a timestamp of when the event occurred. If this field is empty, Hightouch uses the time the event arrives at the server.
Any `DateTime` value provided from your source is converted to Unix timestamp as expected by Customer.io.
#### Field mapping
You can sync columns from your source to Customer.io customer attributes. If the customer attribute doesn't exist, Hightouch creates it.
### Segments
Sync data from any source to [new or existing segments](https://www.customer.io/docs/api/#tag/Track-Segments).
#### User identifiers
You can match rows in your model with users in Customer.io on the following fields:
- Customer.io External ID
- Email
- CIO ID
#### Create a segment
- To create a new mailing list, select `Create a new segment`.
- You can provide a name for this segment, or Hightouch use the name of the model used to create the sync.
#### Use an existing segment
- To use an existing mailing list, select `Use existing segment`.
- You can select any of your existing Customer.io segments from the drop-down select menu.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Delighted
**URL:** https://hightouch.com/docs/destinations/delighted
**Description:** Create or update your customers' information while sending surveys with rich segmentation data directly from your warehouse.
**Section:** Destinations
## Overview
The Delighted integration allows you to sync customer data and send surveys from your warehouse based on models that you define.
### Why sending surveys should be based on the warehouse
- You can do more complex segmentation by leveraging all the data in your warehouse, including event data and third-party data. Your warehouse is the only place where all of this data coexists.
- You can keep your Delighted segments in sync with the rest of your business tools. For example, you might want to send a survey to users who've been invited to a beta program. You also might want to sync these attributes to a CRM like Salesforce or HubSpot. Hightouch integrates with SaaS tools used for email marketing, customer support, sales, and more. Define your audience once and activate your data everywhere.
- Segments can be defined using the tools that your teams are most comfortable with. You can filter and transform data from your warehouse using SQL or our [visual query builder](/customer-studio/usage#setup)—whichever you prefer. (You can even sync customers to or send surveys through Delighted segments based on work you've already done in dbt or Looker!)
## Getting started
### Connect to Delighted
**Retrieve your API Token from Delighted.**
1. Log in to Delighted.
2. Visit [Delighted's API Introduction Page](https://app.delighted.com/docs/api).
3. Copy your **API key**.
#### Add Delighted as a destination in Hightouch
1. Log in to Hightouch or create a free Hightouch account
2. Click on **Destinations** in the left sidebar
3. Click on **Add destination** in the top right corner
4. Select **Delighted** in the destination catalog and click **Continue** to proceed
5. Paste your **API key** into the **API key** field and click **Continue** to proceed
6. Give your destination a name and unique slug (for example, Delighted Production” and `Delighted-production`)
7. Click **Finish** to create your Delighted destination
### Connect to your data source
Hightouch sits on top of your data warehouse and can read from Snowflake, Redshift, BigQuery, Databricks, Postgres, MySQL, and many other data sources. You can even fetch data from Google Sheets and Airtable. Hightouch can help activate your data wherever it resides.
We've written dedicated guides for each supported data source. Take your pick:
- Connect to [Airtable](/sources/airtable)
- Connect to [Amazon Athena](/sources/amazon-athena)
- Connect to [Amazon Redshift](/sources/amazon-redshift)
- Connect to [ClickHouse](/sources/clickhouse)
- Connect to [Databricks](/sources/databricks)
- Connect to [Elasticsearch](/sources/elasticsearch)
- Connect to [Firebolt](/sources/firebolt)
- Connect to [Google BigQuery](/sources/google-bigquery)
- Connect to [Google Sheets](/sources/google-sheets)
- Connect to [Looker](/sources/looker)
- Connect to [Metabase](/sources/metabase)
- Connect to [MySQL](/sources/mysql)
- Connect to [Palantir Foundry](/sources/foundry)
- Connect to [PostgreSQL](/sources/postgresql)
- Connect to [Rockset](/sources/rockset)
- Connect to [SFTP](/sources/sftp)
- Connect to [Snowflake](/sources/snowflake)
- Connect to [SQL Server](/sources/sqlserver)
- Connect to [Trino](/sources/trino)
If you're missing the credentials necessary to connect to your data source,
try inviting your teammates to Hightouch. You can have unlimited collaborators
in your workspace.
#### Connect to a sample database instead
Waiting for credentials to your data warehouse? No problem. You can connect to one of our [sample datasets](/sources/sample-data) to get started right away. For the rest of this tutorial, we'll use the **B2B SaaS** sample source, which contains a `users` table with names, emails, locations, and other user attributes. This sample source is available when you log in for the first time—no setup required.
Next, we'll write a SQL query to define which users should be synced to our Delighted customers resource. You can follow the same pattern to sync to other support Delighted resources,
### Create a model
In Hightouch, a model represents a query that filters or transforms the records in your data source. Models are used to determine exactly which records to sync between sources and destinations.
When you connect to a data warehouse, your models will probably be written in SQL. You can also import models from tools like dbt and Looker.
In this example, we'll keep it simple. Suppose you want to sync all your customers to Delighted. To accomplish this, you'll want to create a model that looks like this:
```sql
SELECT * FROM users;
```
1. Click on **Models** in the left sidebar.
2. Click on **Add model** in the top right corner.
3. Select **Demo Database** as your source and click **Continue** to proceed.
4. Select **SQL Editor** as your modeling method.
5. Paste the SQL query from above into the editor on the left side of the page.
6. Click **Preview** to execute the query.
7. Click **Continue** to proceed.
8. Give your model a name and unique slug (for example, "All Users" and `all-users`).
9. Select **ID** as the primary key for your model.
10. Click **Finish** to create your model.
We just created a model using a SQL query that retrieves all our current users. Next, we'll configure our sync to add these users to Delighted.
### Create a sync
1. Click on **Syncs** in the left sidebar.
2. Click on **Add sync** in the top right corner.
3. Select **All Users** as your model and click **Continue** to proceed.
4. Select **Delighted** as your destination and click **Continue** to proceed.
5. Give your destination a name and unique slug (for example, "Delighted All Customers Sync" and `Delighted-all-customers-sync`) and click **Finish** to proceed.
6. Select **People** from the object dropdown.
7. Select **Upsert** from the mode radio options.
8. Select `email` and we will map it to `Email` from the external ID mapping field.
9. Select `first_name` and we will map it to `Name` in the fields that we'd like to sync to Delighted.
10. Select **Yes** in the Sending Survey section.
11. Select **Do Nothing** in the Delete section.
12. Click **Continue** to proceed to the last step.
13. Select **Interval** to tell your sync to run on a set interval.
14. Configure your sync schedule to run every **5 minutes**.
15. Click **Finish** to create your sync.
And We're done. Your Delighted sync configuration is complete.
## Sync configuration
### Object selection
Hightouch supports syncing to the following Delighted resources:
- **People**
### Sync modes
This integration supports **Upsert**.
In **Upsert** mode, records that aren't found in Delighted given the provided identifier will be inserted. And existing records will be updated.
### Record matching
Records can be matched from your source to your Delighted workspace by `Email`.
### Field mapping
Hightouch allows you to sync columns from your source to the supported Delighted resources:
| Mapping | Type | Description |
| :----------------------- | :----: | -------------------------------------------------------------------------------------------------------: |
| `Phone Number` | String | Phone number of customer. Format must be E.164 (for example, +17132746524). |
| `Survey Channel` | String | Defaults to `email`. Specify `sms` and provide a `Phone Number` to have the survey request sent via SMS. |
| `Name` | String | Name of person. |
| `Delay To Send Email` | Number | Delay in `seconds` before sending the survey. |
| `Update Email To` | String | Field to update the person's email. |
| `Update Phone Number To` | String | Field to update the person's phone number. |
### Custom field mapping
Hightouch allows you to set custom attributes. Any mappings from this section will be synced to the custom properties of each associated survey sent. You can add as many properties as you need.
Properties are a powerful way to analyze survey results on Delighted. To learn more about it, visit [Properties Overview](https://help.delighted.com/article/575-properties-overview).
### Sending surveys
When configuring your Delighted sync form, there are a couple of choices provided to you when it comes to sending surveys. You can either:
- **Not** send surveys to all synced records.
- **Send surveys** to all synced records. Records that are created will be sent a survey. Records that are updated and are out of Delighted's survey throttling interval will be sent a survey.
- **Send surveys** based on each record **dynamically** by providing a `Boolean` column. To use this feature, toggle **Use Column**, and select that respective column from the dropdown.
### Delete records
When the Hightouch sync engine recognizes that a record has been removed from your source, you can either:
- Do Nothing.
- Delete the record from Delighted.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Digital Turbine
**URL:** https://hightouch.com/docs/destinations/digital-turbine
**Description:** Empower advertisers to grow the bottom line by targeting your first-party audiences with in-app ad experiences across mobile devices.
**Section:** Destinations
## Overview
Empower advertisers to grow the bottom line by targeting your first-party audiences
with in-app ad experiences across mobile devices.
Deliver more relevant ads by combining data from various sources within your data
warehouse to target and suppress your first-party audiences with in-app ad
experiences across mobile devices. By keeping your custom audiences updated
automatically, never show an ad to someone who shouldn't see it.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------- | ---------------------------------- | -------------------- |
| **Audiences** | Create and update custom audiences | All |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Digital Turbine
This integration requires prior approval from the Digital Turbine team and the
Hightouch team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your Digital Turbine rep who will verify approval. Once we have confirmed,
we will unlock the integration for you to connect.
To connect, input the `Advertiser Name` provided by your Digital Turbine rep.
### Syncing custom audiences
You can input the name of the audience you plan on delivering to Digital Turbine.
#### User identifiers
To identify which users to add or update in an audience, select model columns
and the corresponding Digital Turbine fields. You can match on any of the following user
fields:
- MAID
#### Delete behavior
The only supported sync mode is `all` mode, meaning that each sync run will refresh
the entire audience. Users removed from your audience will automatically be dropped
with each refresh - only present users will be in Digital Turbine's audience.
## Tips and troubleshooting
### Refresh rate
Digital Turbine has a max refresh rate of daily. You should not have your Digital
Turbine syncs running more than daily.
### Matched device count
Match rates are not available to preview in the Hightouch platform. Contact your
Digital Turbine rep for insights into match rates and audience reach.
Keep in mind that audience counts reflect the number of devices match, not the
number of users matched. Also keep in mind that matching occurs at the household
level, and that a single household may own multiple devices.
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Sync alerts
---
## Google Display & Video 360
**URL:** https://hightouch.com/docs/destinations/display-video
**Description:** Sync customer segments to Display & Video 360
**Section:** Destinations
## Overview
With the Google Display & Video 360 integration, Hightouch can sync first and third party audiences directly from your warehouse.
This allows you to:
1. Create ad target groups for your own users, using email, physical address or mobile device ID.
2. Create target groups from users shared with a partner.
## Setup via OAuth
Select the **Google Display & Video 360** destination option. Hightouch will prompt you to authorize our app with your Google account with the following a
scope to create and update audiences on your DV360 accounts.
After authorizing, you will have successfully connected Hightouch to DV360.
## Syncing data
Hightouch allows to create and update first and third party audiences as customer match user lists. These lists
support adding members using the mobile device ID, or an array of email, phone numbers, and physical addresses.
When creating the audience, make sure the source of the data is properly selected, between First or Third party sources.
### User consent
Starting in March 2024, Display & Video 360 introduced the [consent object](https://developers.google.com/display-video/api/guides/audiences/upload-customer-match?sjid=10202302840984618943-NA#set_user_consent) for uploading consent with customer match data. The consent object specifies two distinct types of consent and users must grant both types of consent to use their data for customer match lists. Reference the [DV360 docs](https://support.google.com/displayvideo/answer/14673239?hl=en) for more information.
If you don't set the consent status or if the status is denied, the sync results in an error.
### PII data and hashing
Hightouch will automatically hash all the PII data before it's sent to Google. If the data is already
hashed, this option can be disabled. Reference the [DV360 docs](https://developers.google.com/display-video/api/reference/rest/v1/firstAndThirdPartyAudiences#ContactInfo) for more information.
Google's Personalized Ads Policy, Customer Match Policy and Platforms program
policies include requirements for advertisers to show sufficiently established
Google Ads or Display & Video 360 account history before accessing certain
features. To enable uploading data to DV360, please fill [this
form](https://support.google.com/displayvideo/answer/9176010) before you
create the sync.
### User identifiers
Mobile device IDs and other user identifiers can not be mixed. Also, if providing the user's physical address, the zip code,
country code, first and last name must be provided.
Phone numbers must be in [E.164](https://en.wikipedia.org/wiki/E.164) format.
### Common errors
#### Can not use an UNKNOWN or UNSPECIFIED as enum value
Make sure to set the [consent status](#user-consent) because Google rejects any users where the consent status is `Unspecified` or `Denied`.
### Live debugger
### Sync alerts
---
## Drift
**URL:** https://hightouch.com/docs/destinations/drift
**Description:** Engage with your leads at the right time with the right context via Drift
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-------------|---------------------------------------------|------------------------|--------------------------------------------------------------
**Accounts** | Sync data from any source to Drift accounts | Upsert, Update, Insert | [Accounts docs](https://devdocs.drift.com/docs/account-model)
**Contacts** | Sync data from any source to Drift contacts | Upsert, Update, Insert | [Contacts docs](https://devdocs.drift.com/docs/contact-model)
**Users** | Sync data from any source to Drift users | Update | [Users docs](https://devdocs.drift.com/docs/user-model)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Drift
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Drift** and click **Continue**. You can then authenticate Hightouch to **Drift** using OAuth.
For the **Authentication method**, select **Log in to Drift** and log into your Drift account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Drift destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Drift destination you want to sync to.
### Syncing accounts
Accounts can be used for ABM (account based marketing) in Drift, and can be tied to the display conditions of particular playbooks-enabling account specific marketing strategies.
#### Record matching
To match rows from your model to accounts in Drift, you need to select a model column and corresponding Drift field. You can match on any of the following Drift fields:
- **Domain**
- **Account ID**
In **Insert** mode, Drift automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync account fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom account fields.
If you send data for a custom field that doesn't exist, Hightouch adds the field.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available on **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
Behavior | Description
---------------|-----------------------------------------------------------
**Do nothing** | Keep the account in Drift with all its synced fields
**Clear** | Clear all the mapped fields, but keep the account in Drift
**Delete** | Delete the synced accounts from Drift
### Syncing contacts
Contacts are your site visitors who interact with Drift. Contacts may provide you with their email address, or their information may have been updated in the past 90 days. Contacts belong to an account, and may have conversations, composed of messages, with your users.
#### Record matching
You can match rows from your model to contacts in Drift on any column in your model and any field in Drift. Ensure the data types of the model column and Drift field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Drift automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom contact fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available on **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
Behavior | Description
---------------|-----------------------------------------------------------
**Do nothing** | Keep the contact in Drift with all its synced fields
**Clear** | Clear all the mapped fields, but keep the contact in Drift
**Delete** | Delete the synced contacts from Drift
### Syncing users
Drift users are internal agents to your organization or account-while site visitors or contacts are those that are interacting from your website.
Hightouch syncs can only update existing users.
#### Record matching
You can match rows from your model to users in Drift on any column in your model and any field in Drift. Ensure the data types of the model column and Drift field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync user fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default user fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|--------------------------------------------------------
**Do nothing** | Keep the user in Drift with all its synced fields
**Clear** | Clear all the mapped fields, but keep the user in Drift
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Dropbox
**URL:** https://hightouch.com/docs/destinations/dropbox
**Description:** Easy to use, reliable, private, and secure. It’s no wonder Dropbox is the choice for storing and sharing your most important files.
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
| ------------ | ------------------------------------------------------- | -------------------- |
| Any data set | Sync data from a source to Dropbox as JSON or CSV files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Diff mode creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Dropbox
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Dropbox** and click **Continue**. You can then authenticate Hightouch to Dropbox via OAuth.
After logging into your Dropbox account, Hightouch prompts you to grant access to your Dropbox account:
After authorizing, you will have successfully connected Hightouch to Dropbox.
## Sync configuration
Once you've set up your Dropbox destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Dropbox destination you want to sync to.
### Select file format
Hightouch supports syncing JSON and CSV files to Dropbox.
### Select folder
Next, select the folder in Dropbox to which you want to sync your files.
### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it. To view different versions of the same file, you can right-click the file in Dropbox. Click **Activity** and then **Version history**.
If you are using an audience and would like to include the audience name, you will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter `-86400` (`24 hours * 60 minutes * 60 seconds`). If you want the filename's data to be one hour after the sync takes place, you would enter `3600` (`60 minutes * 60 seconds`).
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `company`, `email`, `name`, and `primaryKey` fields. Hightouch exports these fields to new fields in the file and ignores all other columns from your results.
### CSV options
If you're syncing to a CSV file, you have additional configuration options:
- **Delimiter**: Your options are comma (`,`), semicolon (`;`), pipe (`|`), tilde (`~`), and tab
- (Optional) Whether to **include a CSV header row** in the exported files
- (Optional) Whether to **include a [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark)** in the exported files, the BOM is ``
### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
## Tips and troubleshooting
### Common errors
### Sync alerts
---
## Dynamic Yield
**URL:** https://hightouch.com/docs/destinations/dynamic-yield
**Description:** Use Dynamic Yield to update product data.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ------------------ | ------------------------------------------------------------ | -------------------- | ----------------------------------------------------------------- |
| **Product Feed** | Sync data from any source to product feed in Dynamic Yield | Upsert | [Product Feed docs](https://dy.dev/reference/update-product-feed) |
| **User Data Feed** | Sync data from any source to user data feed in Dynamic Yield | Upsert | [User Data Feed docs](https://dy.dev/reference/user-data-api) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Dynamic Yield
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Dynamic Yield** and click **Continue**.
You’ll need to provide a Dynamic Yield API key.
### Create an API key in Dynamic Yield
1. In your Dynamic Yield account, navigate to **Settings › API Keys** and click **New Key**.
2. Fill out the form:
- **Name**: Enter a descriptive name, such as `Hightouch Sync Key`.
- **Notes**: (Optional) Add details for your team.
- **Source**: Select `Server-side` (required for Hightouch).
3. Under **ACL (Access Control List)**, check all the following boxes:
- [**Experience API**](https://dy.dev/docs/experience-api-basics) – Grants Hightouch the ability to update the product catalog and other assets Dynamic Yield uses to run campaigns. This is what lets marketers surface the right products or offers in recommendation modules, banners, and emails.
- [**User Data Profile**](https://dy.dev/docs/intro-to-user-data-enrichment) – Allows Hightouch to sync customer attributes (like loyalty tier, geography, or purchase history). Marketers use this profile data to build audiences and target campaigns.
- [**User Data Events**](https://dy.dev/docs/intro-to-user-data-enrichment) – Enables Hightouch to send customer activity signals (like product views, sign-ups, or cart actions) in real time. Marketers use these events for triggering same-session personalization and measuring engagement.
4. Click **Save** and copy the generated API key.
### Connect the API key in Hightouch
5. In Hightouch, paste the API key in the **Connect destination** step.
6. Select your data center (**United States** or **Europe**). See the [Experience API Basics guide](https://dy.dev/docs/experience-api-basics#data-centers) for details on region-specific endpoints.
## Sync configuration
Hightouch supports syncing to the following Dynamic Yield objects:
- [`Product Feed`](#product-feed)
- [`User Data Feed`](#user-data-feed)
To sync to Product Feed or User Data Feed, your Dynamic Yield API key must have **all three ACL permissions** enabled (`Experience API`, `User Data Profile`, and `User Data Events`).
### Sync modes
Hightouch supports the following sync modes:
- **Upsert** - pushes new products to Dynamic Yield Product feed or User Data API and updates fields that change in your warehouse.
### Product Feed
#### Feed ID
You can find the feed's ID by navigating to Assets > Data Feeds and looking for the ID column in the table of available feeds.
#### Record matching
Dynamic Yield uses product `SKU` as the product ID for matching records in the feed.
#### Field mapping
There are several required properties when syncing products to the feed. You can add additional fields in the custom mapping section.
#### Deleting
You may also opt to remove products from your feed when they are removed from your source.
You can do that by selecting `Delete the Dynamic Yield record associated with source`
### User Data Feed
You can find the feed's ID by navigating to Assets > Data Feeds and looking for the Unique ID column in the table of available feeds.
#### Record matching
Select the ID for the record from the source. You will also need to enter or select the type of ID used, this would be defined in Dynamic Yield when you setup the feed, i.e., hashed email.
#### Field mapping
To map fields for the user object, copy the feed schema from the Dynamic Yield admin into Hightouch. The fields will automatically be generated for mapping.
#### Deleting
You may also opt to remove products from your feed when they are removed from your source.
You can do that by selecting `Delete the Dynamic Yield record associated with source`
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## DynamoDB
**URL:** https://hightouch.com/docs/destinations/dynamodb
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Overview
This destination combines the analytical power of your data warehouse to power your high-performance applications at any scale.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| --------- | ------------------------------------------------------------------------------------------------------------------------------------------ | -------------------- |
| **Items** | Sync rows from any source as [items](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/WorkingWithItems.html) into DynamoDB | Upsert |
## Getting started
To connect to your DynamoDB instance, you need to provide Hightouch access to your AWS account. You can do this via access key or cross-account role.
For either method, you must give permission for the following actions:
```
"dynamodb:BatchGetItem",
"dynamodb:BatchWriteItem",
"dynamodb:PutItem",
"dynamodb:DescribeTable",
"dynamodb:DeleteItem",
"dynamodb:UpdateItem",
```
### Authenticating with a cross-account role
Hightouch recommends using a cross-account role for cleaner access. See our [AWS security page](/security/aws#cross-account-role) to learn more about how to use the cross-account role.
### Authenticating with an access key
You can give Hightouch access to your AWS account with an access key. Copy your Access key ID and Secret access key into Hightouch. See the [AWS docs](https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html) for best practices.
## Syncing data
Once you've authenticated DynamoDB in Hightouch, you need to set up your sync configuration.
### Table selection
In the sync configuration page, select a DynamoDB table to sync to. If your table doesn't exist yet, create it in AWS.
### Record matching
match rows in your model with records in DynamoDB via DynamoDB's primary key. See AWS's guide on [how to choose a primary key](https://aws.amazon.com/blogs/database/choosing-the-right-dynamodb-partition-key/) to learn more.
### Field mapping
Hightouch infers field types from the fields values of rows.
Hightouch sends the `timestamp` field to DynamoDB as an ISO string. If you
want to format it differently, format the value in your model and set the type
to `String`.
#### Delete behavior
When rows are deleted from your source, you can `do nothing`, `clear`, or `delete` them in DynamoDB.
| Type | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the synced record in your DynamoDB |
| **Clear** | Remove the synced fields of the synced record from your DynamoDB |
| **Delete** | Delete the synced record from your DynamoDB |
Deletions are destructive and irreversible. Be sure this is the behavior you
want when selecting `clear` or `delete`.
### Testing a sync
Before running your sync, Hightouch recommends that you test your sync configuration by syncing a single row. You can verify that a row was successfully synced by going to your DynamoDB instance table and finding the synced row by its primary key.
## Tips and troubleshooting
### Rate limits
DynamoDB has quotas that may limit your syncs. If that happens, Hightouch [retries failed rows](https://hightouch.com/docs/syncs/retries). For more information, refer to DynamoDB's [throughput quotas](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ServiceQuotas.html#default-limits-throughput-capacity-modes).
### Empty field mappings
DynamoDB is a key/value NoSQL database. That means the database doesn't store metadata on its columns and column names and you won't see any suggested fields during field mapping in Hightouch.
### Common errors
### Using the Hightouch debugger
### Setting up sync alerts
Hightouch can alert you of sync issues via Slack, PagerDuty, SMS, or email. For details, please visit our [article on alerting](/syncs/alerting).
---
## Dynata
**URL:** https://hightouch.com/docs/destinations/dynata
**Description:** Empower your brand, marketing, and advertising teams to make data-driven decisions with Dynata’s data and insight solutions.
**Section:** Destinations
## Overview
Sync your first-party audiences with Dynata data and insights solutions to
better measure, evaluate, and optimize advertising effectiveness in critical
areas of your media to see how a campaign influences consumers at all levels of
the purchase funnel.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------- | ---------------------------------- | -------------------- |
| **Audiences** | Create and update custom audiences | All |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Dynata
This integration requires prior approval from the Dynata team and the Hightouch
team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your Dynata rep who will verify approval. Once we have confirmed, we will
unlock the integration for you to connect.
To connect, input the following provided by your Dynata rep:
- Access key
- Secret key
- Bucket name
### Syncing custom audiences
You can input the name of the audience you plan on delivering to Dynata.
#### User identifiers
To identify which users to add or update in an audience, select model columns
and the corresponding Dynata fields. You can match on any of the following user
fields:
- Unique ID (required for de-duping)
- Email (SHA256)
- Phone (SHA256)
- MAID
- IP address
- First name
- Last name
- Address1
- Address2
- City
- State
- Postal code
- Zip4
- Country
#### Delete behavior
The only supported sync mode is `all` mode, meaning that each sync run will refresh
the entire audience. Users removed from your audience will automatically be dropped
with each refresh - only present users will be in Dynata's audience.
## Tips and troubleshooting
### Refresh rate
Dynata has a max refresh rate of monthly. You should not have your Dynata syncs
running more than every 4 weeks.
### Matched users count
Match rates are not available to preview in the Hightouch platform. Contact your
Dynata rep for insights into match rates and audience reach.
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Sync alerts
---
## Eesii
**URL:** https://hightouch.com/docs/destinations/eesii
**Description:** Activate your data as campaign records in Eesii
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes |
----------------|------------------------------------|----------------------|
**Campaign records** | Sync data as campaign records to Eesii | Insert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Eesii
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Eesii** and click **Continue**.
You can then authenticate Hightouch to **Eesii** by entering your API key.
To find your API key, navigate to the detail page of one of your campaigns. From there, click "Generate Webhook" and you will see your API key displayed.
## Sync configuration
Once you've set up your Eesii destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Eesii destination you want to sync to.
### Syncing campaign records
#### Campaign and layout selection
Begin setting up your sync by selecting which campaign you'd like to sync records to.
Select which layout you'd like to use to sync records. If you selected a **variant** campaign, you will have multiple layouts to choose from. If you selected a **regular** campaign, you will only have one.
#### Field mapping
Once you have selected a layout, you will see the attributes of your layout as possible mapping options.
Select which columns in your model you'd like to map to record fields in Eesii.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Elasticsearch
**URL:** https://hightouch.com/docs/destinations/elasticsearch
**Description:** Elasticsearch is a distributed, free and open search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured.
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes | API Reference|
|--|--|--|--|
|**Index documents**|Sync data from any source to an Elasticsearch index|Upsert, Update|[Index API](https://www.elastic.co/guide/en/elasticsearch/reference/8.7/docs-index_.html)|
## Connect to Elasticsearch
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Elasticsearch** and click **Continue**. You can then authenticate Hightouch to Elasticsearch either with an Elasticsearch **Cloud ID** and **API key** or via optional [basic authentication](https://www.elastic.co/guide/en/elasticsearch/reference/current/http-clients.html).
### Authenticate with an Elasticsearch Cloud ID and API key
You can find your [Cloud ID](https://www.elastic.co/guide/en/cloud/current/ec-cloud-id.html) in your Elasticsearch deployment. To generate an API key, follow these steps in the [Elasticsearch docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html).
### Authenticate with basic authentication
Enter the following fields into Hightouch:
- **Protocol**: Select the protocol used for connecting to Elasticsearch. This is either `HTTP` or `HTTPS`.
- **Host**: The hostname or IP address of your Elasticsearch server.
- **Port**: The port number of your Elasticsearch server's HTTP API. The default is 9200, but yours may be different.
- **Username**: This can be your personal Elasticsearch login or a dedicated user for Hightouch.
- **Password**: The password for the user specified above.
## Sync configuration
Once you've set up your Elasticsearch destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Elasticsearch destination you want to sync to.
### Syncing to an index
An Elasticsearch [index](https://www.elastic.co/blog/what-is-an-elasticsearch-index) is the place where the data used by a search engine is stored. You can use Hightouch to update individual [documents](https://www.elastic.co/guide/en/elasticsearch/reference/current/documents-indices.html#:~:text=Elasticsearch%20is%20a%20distributed%20document,been%20serialized%20as%20JSON%20documents.) in an index.
#### Record matching
To match rows from your model to documents in Elasticsearch, you need to select a model column and corresponding Elasticsearch field. You can match on any Elasticsearch field.
#### Field mapping
You can sync data to both existing fields and fields that don't yet exist in Elasticsearch. If you send data to a field that doesn't exist yet, Hightouch adds the field and automatically detects its type.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ------------------------------------------------------------------ |
| **Do nothing** | Keep the document in Elasticsearch |
| **Clear** | Keep the document in Elasticsearch, but set mapped fields to empty |
| **Delete** | Remove the document from Elasticsearch |
## Tips and troubleshooting
**Connect to Elasticsearch on AWS**
To connect to an AWS hosted Elasticsearch, connect to Elastic Cloud where the instance has a Cloud ID, then generate an API key. If you can't connect via this method, use the basic authentication method to connect with username and password.
There may be compatibility issues across different versions of Elasticsearch. If you receive an error when you test the connection, please for support for your specific version.
If you are running Elasticsearch via Amazon OpenSearch Service, connect directly via the [OpenSearch destination](/destinations/opensearch)
### Common errors
### Live debugger
### Sync alerts
---
## Eloqua
**URL:** https://hightouch.com/docs/destinations/eloqua
**Description:** Enhance Eloqua marketing automations for mobile, email, video and search results by bringing enriched customer data into Eloqua from your data warehouse.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- |
| **Contacts** | Sync data from any source to [Eloqua contacts](https://docs.oracle.com/en/cloud/saas/marketing/eloqua-user/Help/Contacts/Contacts.htm) | Upsert, Update |
| **Accounts** | Sync data from any source to [Eloqua accounts](https://docs.oracle.com/en/cloud/saas/marketing/eloqua-user/Help/Accounts/Accounts.htm) | Upsert, Update |
| **Custom Objects** | Sync data from any source to any [Eloqua custom object](https://docs.oracle.com/en/cloud/saas/marketing/eloqua-user/Help/CustomObjects/CustomObjects.htm) | Upsert, Update |
| **Contact Lists** | Sync data from any source to [Eloqua contact lists](https://docs.oracle.com/en/cloud/saas/marketing/eloqua-user/Help/SharedLists/SharedContactLists.htm) | Add |
| **Email Trigger** | Sync data from any source to [trigger](https://docs.oracle.com/en/cloud/saas/marketing/eloqua-develop/Developers/RESTAPI/2.0/email-deployment/email-deployment-API.htm) emails in Eloqua | -- |
## Connect to Eloqua
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Eloqua**.
Then **Authorize connection** by clicking **Log in to Eloqua**. Clicking the button leads you through your Eloqua login and asks for Hightouch to have authorization to your Eloqua instance.
Once you've finished, Hightouch displays an **Authorization successful** message.
To finish connecting, give your Eloqua destination a descriptive name.
## Sync configuration
Once you've connected your Eloqua destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Eloqua destination you want to sync to.
### Contacts
Hightouch supports creating new contacts and updating existing ones.
#### Record matching
You can match rows in your model with contacts in Eloqua on any contact field. We recommend matching on a unique field, such as **Email address**.
#### Field mapping
You can sync any columns from your source to Eloqua contact fields.
### Accounts
Hightouch supports creating new accounts and updating existing ones.
#### Record matching
You can match rows in your model with accounts in Eloqua on any account field. We recommend matching on a unique field, such as **Eloqua Company ID** or a unique custom field.
#### Field mapping
You can sync any columns from your source to Eloqua account fields.
### Custom objects
Hightouch supports creating new instances of custom objects and updating existing ones.
Custom objects and their fields must already be defined on your Eloqua UI
before Hightouch can sync to them.
#### Record matching
You can match rows in your model with custom objects in Eloqua on any custom object field.
#### Field mapping
You can sync any columns from your source to Eloqua any existing field in your custom object.
### Contact lists
Hightouch supports syncing data to Eloqua contact lists.
You can create a new contact list or use an existing one. When creating a new list, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model.
#### Record matching
You can match rows in your model with contacts in Eloqua on any contact field. We recommend matching on a unique field, such as **Email address**.
#### Field mapping
You can sync any columns from your source to Eloqua contact fields.
### Email trigger
Hightouch supports triggering emails to contacts in Eloqua.
#### Email
For a given trigger sync, you need to select an email asset to sync to. The options come from the email assets in your Eloqua environment.
Optionally, you can also enter a specific user ID to specify the sender of each email. By default, the email will be sent by the user
the destination is connected to.
#### Contact matching
You can match rows in your model with contacts in Eloqua on by email or Eloqua's contact ID. We recommend matching on the **contact ID** because matching on email
will require a lookup for each record since Eloqua only allows triggering emails by contact IDs. Performing the lookups will impact the performance of your sync.
## Tips and troubleshooting
### Live debugger
### Sync alerts
---
## SMTP Email
**URL:** https://hightouch.com/docs/destinations/email
**Description:** Sync your warehouse with Email notifications and push marketing campaigns and product updates to customers in real time
**Section:** Destinations
## Overview
With the SMTP Email destination, Hightouch can serve as a notifier for various changes. This is a very flexible integration, but here are a few example use cases:
1. Notify your customer success teams when product usage for a key account suddenly drops
2. Notify an account executive when product usage surpasses a certain threshold
3. Send a daily summary of the number of sign ups, compared to the same day last month
4. Send a daily summary on the adoption of a core feature
## Sending emails \(added / changed / removed\)
Hightouch will not send any emails on the initial run to prevent a flood of
emails to your channel. Emails will start after the this initial sync.
Hightouch supports sending emails based on rows that have been added, changed, or removed since the last sync run. Each row is sent as an individual email and can be sent to different channels and templated with Liquid.
Email formatting is best when the following conditions are met:
1. You want to write a query, and have Hightouch detect the added and removed rows
2. You want a custom email sent per each added and removed rows
### Recipient selection
Hightouch supports both sending emails to specific static recipients, or an email each row. Enable the "Use column" toggle to dynamically set the target email addresses.
Hightouch also supports CC and BCC so that multiple recipients can be included on the email chain.
The recipient, CC, and BCC sections all support a single email or a comma-separated list of emails
### Added, changed, and removed emails
Hightouch allows you to template the content you would like to send in your email.
- When a row is added in your query results, Hightouch will send the row added content.
- When a row is changed in your query results, Hightouch will send the row changed content.
- When a row is deleted in your query results, Hightouch will send the row deleted content.
This content can be [templated](#liquid-templating).
## Sending emails \(all results\)
The batch message mode is similar to the normal message mode, except it sends all rows in the query results rather than only added, changed, and removed rows since the last sync.
This mode also allows you to specify how many rows are sent per email, which can range from 1 to 20. With 1 row per email, it would be similar to the message mode, whereas 20 rows per email would be similar to a tabular representation.
This format is good for the following scenarios.
1. You want to write a query, and have Hightouch send an email containing **all** rows
2. You don't want Hightouch to detect added, changed, and removed rows, but instead want to send all rows on each query run
### Recipient selection
The batch message only supports sending your emails to a static email or list of emails.
Hightouch also supports CC and BCC so that multiple recipients can be included on the email chain.
The recipient, CC, and BCC sections all support a single email or a comma-separated list of emails
### Subject and body content
For each email, Hightouch allows setting the subject and row content. Only the row content is required. The row content is repeated per row in the email and has access to the row values via templating.
This content can be [templated](#liquid-templating).
## Templating emails
### Liquid templating
For both the message and batch message modes, [Liquid templating ](https://shopify.github.io/liquid/)is supported. The most important implication of this is that columns from the added or removed row can be accessed as part of the message body. Here is an example message making use of Liquid to insert a column into the message.
```text
User with name {{ row["name"] }} and email {{ row["email"] }} has signed up
```
## Table
Table formatting sends all of your query results formatted as a text table. This mode is recommended only for small sets of data.
This format is good for the following scenarios.
1. You want to write a query, and have Hightouch send an email containing **all** rows, formatted as a table
2. You don't want Hightouch to detect added and removed rows, but instead want to send all rows on each query run
### Recipient selection
The batch message only supports sending your emails to a static email or list of emails.
Hightouch also supports CC and BCC so that multiple recipients can be included on the email chain.
The recipient, CC, and BCC sections all support a single email or a comma-separated list of emails
## CSV
The CSV option allows you to send a CSV file or a zipped CSV file of all records in the query results for each sync attached to a single email. You can customize the subject and body of the email, as well as the name of the CSV file attachment.
This format is useful for reporting purposes, such as sending weekly or monthly reports.
If sending a zipped CSV file, make sure that your email service has the appropriate permissions to send .zip files to recipients. This may require unblocking IP addresses that your service uses.
---
## Emarsys
**URL:** https://hightouch.com/docs/destinations/emarsys
**Description:** Emarsys platform helps marketers deliver personalized, 1:1 experiences across multiple channels and drive business outcomes.
**Section:** Destinations
## Setup
You will need your username, usually in the format of `account_name00X` where `X` is a digit, and the API secret.
To get the API secret, you can create a new API user for Hightouch under **Admin** > **Security Settings**.
Make sure to copy and save the secret key here since the entire key will no
longer display after you close the confirmation dialog box.
You will need to enable the necessary endpoints for this API key. The endpoints
Hightouch requires depend on how you configure your syncs.
#### Always required
- [`field.get`](https://dev.emarsys.com/docs/core-api-reference/a0l7f9tviiuiv-list-available-fields) — Used during the connection test and to load available contact fields for mapping. Without this endpoint enabled, the connection test will fail with a 403 error.
#### Contact syncs
- [`contact.create`](https://dev.emarsys.com/docs/core-api-reference/g617t4kfs4y69-create-contact) — Required for upsert sync mode (creates new contacts).
- [`contact.update`](https://dev.emarsys.com/docs/core-api-reference/f8ljhut3ac2i1-update-contacts) — Required for upsert and update sync modes.
- [`contact.delete`](https://dev.emarsys.com/docs/core-api-reference/szmq945esac90-delete-contacts) — Only required if you plan to remove contacts when data leave your warehouse.
#### Contact list syncs
- [`contactlist.list`](https://dev.emarsys.com/docs/core-api-reference/paz2wputeebqx-list-contact-lists) — Required to load available contact lists for selection.
- [`contactlist.contact.add`](https://dev.emarsys.com/docs/core-api-reference/1m0m70hy3tuov-add-contacts-to-a-contact-list) — Required to add contacts to a list.
- [`contactlist.contact.delete`](https://dev.emarsys.com/docs/emarsys-api/b3A6MjQ4OTk4MzI-remove-contacts-from-a-contact-list) — Only required if you plan to remove contacts from a list.
- [`contactlist.create`](https://dev.emarsys.com/docs/emarsys-api/b3A6MjQ4OTk4MjI-create-a-contact-list) — Only required if you plan to create new contact lists from Hightouch.
The following screenshot shows an example of a working permissions configuration in the Emarsys UI.
It is also safe to enable all **contacts**, **fields**, and **contact lists** endpoints.
API users and permissions are described in the [Emarsys docs](https://dev.emarsys.com/docs/core-api-reference/ef41493bd7812-endpoint-permission-settings).
## Syncing
Hightouch supports syncing **contacts** and **contact lists**.
### Sync modes
Here are the possible modes for the Emarsys contacts:
- **Upsert**: pushes new contacts to Emarsys and updates existing contacts with the fields that change in your warehouse.
- **Update**: updates particular fields on existing contacts in Emarsys. It doesn't add new objects.
This integration also supports **Segment** mode. New users will be inserted into the list.
Users that are removed from the model will be removed from the list.
### Record matching
Records can be matched from your source to Emarsys by _any_ default and custom fields. However, it's important to make sure
that the field you are using is unique to ensure the best results. For example, Emarsys will throw an error if it detects duplicate
values in an attempt to update or remove a batch of contacts.
We recommend using `email` or creating a custom unique field, like an `external_id` (make sure to set it as _unique_).
### Fields mapping
You can also map to any updatable fields in Emarsys, including your custom fields.
However, there are some things to keep in mind:
- The Emarsys API uses the field's ID when making requests so if you need to use the JSON editor, you will not see any named fields.
Instead, you will only see the field ids. You can use this [table](https://dev.emarsys.com/docs/emarsys-api/ZG9jOjI0ODk5NzY3-contact-system-fields) to cross reference the field ids.
You can also view your settings to reference your custom fields.
- Any fields with a date type are formatted: _YYYY-MM-DD_. You can have your date values be a string in this format (for example, "2020-11-30").
- You must use the Emarsys accepted values for any choice fields (that is, `gender`, `title`, etc.).
## Debugging
Similarly to the fields mapping in the sync configuration, the request will only show the fields' ids unless it's Emarsys' internal `id` and `uid`.
Please refer to this [table](https://dev.emarsys.com/docs/emarsys-api/ZG9jOjI0ODk5NzY3-contact-system-fields) and your custom fields settings to cross reference
the ID to the field's name.
The Emarsys API lets us upsert/update/remove contacts in batches per API call. Most cases, the API will let us know which specific rows fail, instead of failing
the entire batch. However, the error message may not be very descriptive but it will reference their [internal error code](https://dev.emarsys.com/docs/emarsys-api/ZG9jOjI0ODk5NzY4-http-200-errors).
---
## EnjoyHQ
**URL:** https://hightouch.com/docs/destinations/enjoy-hq
**Description:** Sync data from your warehouse to EnjoyHQ to consolidate all customer feedback and research data into a single platform, providing you with better insights for informed decision-making and enhancing customer experiences
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
--------------|---------------------------------------------------|----------------------|---------------------------------------------------------------------------
**Documents** | Sync data from any source to the Documents object | Upsert | [Documents docs](https://developers.enjoyhq.com/reference/document-import)
**Users** | Sync data from any source to the Users object | Upsert | [Users docs](https://developers.enjoyhq.com/reference/user-import)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to EnjoyHQ
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **EnjoyHQ** and click **Continue**. You can then authenticate Hightouch to **EnjoyHQ** by entering an EnjoyHQ **API key**.
To retrieve an API key in EnjoyHQ, go to [Manage integrations](https://app.enjoyhq.com/account/integrations) and select the API card in the **Other** section.
You can then use an existing API key or generate a new one by clicking on the **Add New API Key** button.
## Sync configuration
Once you've set up your EnjoyHQ destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the EnjoyHQ destination you want to sync to.
### Syncing documents
Create and update documents.
#### Record matching
To match rows from your model to documents in EnjoyHQ, you need to select the model column that contains values that match the [**Foreign ID** field](https://developers.enjoyhq.com/reference/document-import#updating-documents).
#### Field mapping
You can map data from any of your model columns to fields in EnjoyHQ. EnjoyHQ requires the following fields, so you must map them in order to complete your configuration:
- **Content**
- **Source**
Ensure the data types of your model columns match the data types of the fields you want to sync to.
### Syncing users
Create and update users.
#### Record matching
To match rows from your model to users in EnjoyHQ, you need to select the model column that contains values that match the **Email** field.
#### Field mapping
You can sync columns from your model to EnjoyHQ default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|------------------------------------------------------
**Do nothing** | Keep the record in EnjoyHQ with all its synced fields
**Delete** | Delete the synced record from your EnjoyHQ user
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Expedia Group Advertising
**URL:** https://hightouch.com/docs/destinations/expedia
**Description:** Get a full picture of your Expedia ad performance with attribution reports that track their impact on direct bookings—both online and offline.
**Section:** Destinations
## Expedia Data Clean Rooms
Expedia uses AWS Clean Room technology to build secure, tailored data collaborations that let partners share insights without exposing sensitive information.
## Supported syncing
Type | Description | Supported Sync Modes
--------------------|------------------------------------------------------------------------------------------------------------|---------------------
**Conversion Data** | Sync data from any source to AWS Data Clean Room to use with Expedia Group Media Solutions conversion data | Mirror
Mirror mode means that the current table will be cleared and replaced with all the rows in the new query result, every time the sync runs.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Prerequisites
To get started, you need:
- to contact Expedia Group to set up an AWS Data Clean Room collaboration with them. This will involve sharing an AWS account ID with Expedia Group.
- AWS [credentials configured in Hightouch](/security/aws) with programmatic access enabled and permission to write to the S3 path, use Glue databases and tables, and associate tables to data clean rooms.
- the following AWS resources **(these AWS resources must reside in `us-east-1`, consistent with Expedia's Data Clean Room)**:
- an [S3 bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html) to store your synced data in. **Make sure that this bucket is only used for your data that you are syncing to use with Expedia**.
- an S3 bucket for Data Clean Room analysis results. **This must be a different bucket than the above bucket used for storing synced data**.
- your AWS Data Clean Room collaboration membership ID with Expedia Group. This can be found in your AWS Data Clean Room collaboration with Expedia under Details > Membership Details > Membership ID after creating a collaboration with Expedia.
Ensure that the configured user has the following IAM permission policy:
```yaml
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:*",
"s3-object-lambda:*"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"sts:AssumeRole",
"iam:PassRole"
],
"Resource": "arn:aws:iam::${accountId}:role/${IAMRoleName}"
}
]
}
```
And a separate IAM Role with the following permission policy:
```yaml
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "CloudWatchLogsAccess",
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "*"
},
{
"Sid": "GlueCatalogAccess",
"Effect": "Allow",
"Action": [
"glue:CreateDatabase",
"glue:GetDatabase",
"glue:CreateTable",
"glue:DeleteTable",
"glue:GetTables",
"glue:GetTable"
],
"Resource": [
"arn:aws:glue:us-east-1:${accountId}:catalog",
"arn:aws:glue:us-east-1:${accountId}:database/ht_expedia__glue_database_sync_*",
"arn:aws:glue:us-east-1:${accountId}:table/ht_expedia__glue_database_sync_*/*"
]
},
{
"Sid": "S3AccessForGlue",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetObject",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::{bucketName}",
"arn:aws:s3:::${bucketName}/*"
]
},
{
"Sid": "CleanRoomsAccess",
"Effect": "Allow",
"Action": [
"cleanrooms:ListConfiguredTables",
"cleanrooms:CreateConfiguredTable",
"cleanrooms:GetConfiguredTable",
"cleanrooms:ListConfiguredTableAssociations",
"cleanrooms:CreateConfiguredTableAssociation",
"cleanrooms:GetConfiguredTableAssociation",
"cleanrooms:GetConfiguredTableAnalysisRule",
"cleanrooms:CreateConfiguredTableAnalysisRule",
"cleanrooms:GetConfiguredTableAssociationAnalysisRule",
"cleanrooms:CreateConfiguredTableAssociationAnalysisRule"
],
"Resource": "*"
},
{
"Sid": "AllowPassRole",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "arn:aws:iam::${accountId}:role/${IAMRoleName}"
}
]
}
```
and Trust policy:
```yaml
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowUserAssumeRole",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${accountId}:user/${IAMUserName}"
},
"Action": "sts:AssumeRole"
},
{
"Sid": "AllowGlueAndCleanRoomsService",
"Effect": "Allow",
"Principal": {
"Service": [
"cleanrooms.amazonaws.com",
"glue.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}
```
## Connect to AWS DCR + Expedia Group Media Solutions
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Expedia Group Media Solutions** and click **Continue**. You can then authenticate Hightouch to **Expedia Group Media Solutions**.
Enter the following fields into Hightouch:
- **Bucket Name**
- **AWS Glue and Data Clean Room IAM Role Arn**: For the custom role outlined above, the Role ARN can be found under the custom role's Summary > Arn
- **Data Clean Room Collaboration Membership ID**
## Sync configuration
Once you've set up your Expedia Group Media Solutions destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Expedia Group Media Solutions destination you want to sync to.
### Syncing conversion data
Sync data from any source to Expedia Group Media Solutions conversion data for analysis in AWS Data Clean Room.
Your data will be:
1) Sent to the specified S3 bucket with the path `ht_expedia__dcr/upload.csv`. **Do not put any other files in the folder.**
2) Converted to an S3 table with the name `ht_expedia__glue_table_{sync_id}` inside the S3 database with the name `ht_expedia__glue_database_{sync_id}`. **Do not append any other data to the table**.
3) Associated to the entered Data Clean Room collaboration.
#### Field mapping
Hightouch allows you to sync columns from your source to Expedia's AWS Data Clean Room with this schema:
| Mapping | Type | Required |
| :---------------------------| :-----: | :------: |
| `Event ID` | string | No |
| `Timestamp (ISO8601)` | string | **Yes** |
| `Transaction value (USD)` | string | No |
| `Transaction category` | string | No |
| `Email` | string | No |
| `Email (SHA256 Hex)` | string | No |
| `Phone number` | string | No |
| `Phone number (SHA256 Hex)` | string | No |
Expedia recommends to share both email and phone numbers (either hashed or unhashed) to increase the match rate of the collaboration.
#### Adding your table to the collaboration
Hightouch handles associating a table with your data to the Expedia Data Clean Room collaboration. Additional analysis rules should be added to the Hightouch table that was added to the Expedia Data Clean Room collaboration to use your data with Expedia's.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Firestore
**URL:** https://hightouch.com/docs/destinations/firestore
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Supported syncing
| Supported Sync Modes | Description |
| -------------------- | -------------------------------------------------------------- |
| Upsert | Push new records and update records that change in your source |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Firestore
You can decide whether to set up Firestore with [Firebase](#firebase) or with [Google Cloud Platform](#google-cloud-platform).
The following sections outline the necessary setup for either service.
### Firebase
To find the credentials, navigate to [Firestore](https://console.firebase.google.com) and select your project.
In the left-hand sidebar, click the wheel icon next to **Project Overview** and select **Project settings** from the menu.
Once on the Project setting screen, select the **Service accounts** tab. Make sure you are on the **Firebase Admin SDK**.
You will need to click the **Generate new private key** button to download a JSON file of the credentials.
In Hightouch, paste your **private key** including the header and footer in the following format:
```bash
-----BEGIN PRIVATE KEY-----
-----END PRIVATE KEY-----\n
```
Make sure to include the final `\n` character as well, as that is still part of the private key.
### Google Cloud Platform
To find the credentials, navigate to [Google Cloud Platform](https://console.cloud.google.com/firestore/welcome). In the header, next to Google Cloud Platform, click the **Select a project** dropdown and select a project.
If you do not have Firestore set up within this project, you will be taken to the **Get started** screen. Click the **Select Native Mode** button.
Select a location from the dropdown, and then click the **Create Database** button.
Once you are redirected to the Firestore dashboard screen, click the **Security Rules** in the sidebar. Click on the **Enable Firebase** button, and confirm by clicking the **Enable** link within the modal.
Click on the hamburger menu in the top left corner. Select **APIs & Services** from the sidebar, and then select **Credentials** from the dropdown menu.
Once on the Credentials screen, scroll to the **Service Accounts** table. Click on the edit icon in the row with the **Name** as **App Engine default service account**.
Click on the **Keys** tab, and then click the **Add Key** button.
Select the **Create new key** option. Keep the **Key type** as **JSON**, and click the **Create** button in the modal. This will download a JSON with the needed credentials.
In Hightouch, paste your **private key** including the header and footer in the following format:
```bash
-----BEGIN PRIVATE KEY-----
-----END PRIVATE KEY-----\n
```
Make sure to include the final `\n` character as well, as that is still part of the private key.
{/* */}
### Custom database name
Firestore defaults to the `(default)` database. If your setup requires a custom database name enter it in the **Database Name** field in Hightouch during setup.
## Sync configuration
### Sync modes
Hightouch supports the Upsert sync mode.
During the Upsert mode, new rows will be inserted into Firestore and existing rows will be kept up-to-date within Firestore.
### Record matching
Records can be matched from your source to Firestore by any unique property.
### Columns to sync
For Firestore, we give you the ability to send all columns as they are represented in your model.
You can also choose to manually map fields. Only the fields that you map will be exported. All other columns from your results are ignored.
### Delete mode
The default is to do nothing, but you can also set Hightouch to delete the record when it leaves the query result.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## FreeWheel
**URL:** https://hightouch.com/docs/destinations/free-wheel
**Description:** FreeWheel gives advertisers direct connections to streaming video inventory from the world's leading publishers.
**Section:** Destinations
## Overview
FreeWheel is a Comcast-owned advertising technology platform for CTV/OTT video advertising and monetization. Hightouch's FreeWheel destination allows you to upload audience segment files to FreeWheel's infrastructure via S3.
## Supported syncing
Type | Description | Supported Sync Modes
------------------------|----------------------------------------------------------|---------------------
**S3 file upload** | Upload audience segment files to FreeWheel via S3 | All, Insert, Diff
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to FreeWheel
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **FreeWheel** and click **Continue**.
Enter the following fields into Hightouch:
- **AWS Credentials**: Select or create [AWS credentials](../security/aws)
- **Bucket name**: The FreeWheel bucket name you would like to sync to
## Sync configuration
Once you've set up your FreeWheel destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button. Then, select the relevant model and the FreeWheel destination you want to sync to.
### Sync settings
Configure the following settings for your sync:
- **Sync mode**: Choose from All, Insert, or Diff modes
- **File path**: The S3 object key (file path) where the file will be uploaded, i.e. `aud-sgmt-01.csv`
- **Timestamp offset**: Optional timezone offset for timestamp formatting
### Field mapping
Map columns from your model to the fields you want to include in the uploaded CSV file. Ensure your model's columns have the same data types as the fields you want to sync to.
### File format
FreeWheel syncs automatically use the following CSV settings:
| Setting | Value |
|---------|-------|
| File type | CSV |
| Delimiter | Comma (`,`) |
| Include header | Yes |
| Batching | Enabled |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Freshdesk
**URL:** https://hightouch.com/docs/destinations/freshdesk
**Description:** Deliver better support by making contacts, companies, and tickets available in Freshdesk.
**Section:** Destinations
## Setup
Navigate to your Profile Settings and select the API Key on the right most column.
Copy the API key and fill in the fields in Hightouch destination setup:
- **API Key:** token from Freshdesk
- **Freshdesk Subdomain:** subdomain for your Freshdesk support site, https://hightouch.freshdesk.com
## Syncing
This integration support syncing the following objects:
- **Contacts**
- **Companies**
- **Tickets**
### Sync modes
This integration supports both Update, Upsert, and Insert mode.
In Update mode, users and organizations can be matched based on the Freshdesk ID or the external ID and updated by Hightouch with the latest sync data.
In Upsert mode, users and organizations can be matched based on the Freshdesk ID or the external ID and updated or inserted by Hightouch with the latest sync data.
In Insert mode, any new tickets will be inserted by Hightouch with the latest sync data.
### Field mapping
You can sync columns from your source to Freshdesk's default fields, as well as custom fields.
---
## Freshsales
**URL:** https://hightouch.com/docs/destinations/freshsales
**Description:** Transform your selling experience with Freshsales
**Section:** Destinations
## Setup
In order for Hightouch to sync data to Freshsales, you need to give Hightouch an API key and subdomain so that Hightouch can update your records.
You can retrieve your API key by navigating to Settings > API Settings:
For subdomain, check the domain that you are logged in, it should have the pattern of `https://yoursubdomain.myfreshworks.com`. The subdomain used to configure the destination is `yoursubdomain`.
## Syncing
Hightouch supports syncing the following Freshsales objects:
- `Contacts`
- `Accounts`
### Sync modes
- **Upsert** - pushes new objects to Freshsales _and_ updates fields that change in your warehouse.
- **Update**: only updates particular fields on existing objects in Freshsales. It does _not_ add new objects.
### Record matching
Records can be matched from your source to your Freshsales's account's Name or contact's email.
### Field mapping
You can sync columns from your source to Freshsales's Accounts or Contacts properties.
---
## Front
**URL:** https://hightouch.com/docs/destinations/front
**Description:** Empower your teams to deliver better support and experiences by making customer data available in the tools they live in
**Section:** Destinations
## Setup
To get an API Token, in the Front app, navigate to Settings > Plugins & API > API and click the New Token button.
## Modes
### Upsert contacts
In this mode, new contacts will be inserted into Front and existing contacts will be updated.
**User Identification**
To determine whether a contact should be created or updated, you will need to provide a column mapping that can be associated with a unique [Handle](https://dev.frontapp.com/reference/contacts) in Front.
A user can have multiple handles but we will only use one to determine the existence of a contact in Front. Please provide this mapping in this section of the sync configuration:
Please ensure that the column you are mapping to is unique and non-null. Also
keep in mind that making changes to this mapping later on can result in
unexpected behavior.
### Update contacts
In this mode, existing contacts will be updated in Front but new contacts will not be inserted.
---
## FullStory
**URL:** https://hightouch.com/docs/destinations/fullstory
**Description:** Sync your user properties into FullStory on your schedule, not only when users are interacting with your properties. This allows you to integrate these platforms and gain a more complete view of your customer experience.
**Section:** Destinations
## Overview
You've recently built a great digital experience but need to gather information to improve upon the current iteration. How can you know what works and what doesn't? With Hightouch's FullStory integration, you can sync your most recent user properties data to better analyze who is interacting with your product and how. This allows you to make product improvements driven by meaningful data and optimized to enhance retention rates.
## Supported syncing
| Type | Description | Supported Sync Modes |
| --------- | ----------------------------------------------- | -------------------- |
| **Users** | Sync data from any source to Users in FullStory | Update |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to FullStory
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **FullStory** and click **Continue**. You can then authenticate Hightouch to **FullStory** with your FullStory **API key**.
You can find your API key within your FullStory account by clicking the dropdown button on the top left of the page, and opening **Settings** > **API Keys** under the **Integrations** section.
For more information on FullStory API keys, refer to [FullStory's documentation](https://help.fullstory.com/hc/en-us/articles/360052021773-Managing-API-Keys).
## Sync configuration
Once you've set up your FullStory destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and
the FullStory destination you want to sync to.
### Users
Sync data from any source to Users in FullStory.
#### Record matching
Hightouch matches rows from your model to FullStory users by **User ID**. Select the model column that contains these values.
#### Field mapping
You can sync columns from your source to the following FullStory user fields:
- **Display Name**
- **Email**
#### Custom mapping
You can sync to existing FullStory custom fields or create them via field mapping. When entering new custom fields, ensure the field is named using the pattern `ident_type`,
where `ident` is a sequence of alphanumeric characters `A-Z`, `a-z`, or `0-9` and `type` is a short suffix that denotes the field type. Custom field names must start with an alphabetic character (`A-Z` or `a-z`).
Example acceptable field names includes `nickname_str`, `PasswordResets_int`, and `amtAbandonedInCart_real`.
Refer to the FullStory [docs](https://help.fullstory.com/hc/en-us/articles/4446290296599-Setting-custom-API-properties) for more details.
#### Delete behavior
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the customer record in FullStory with all its synced fields |
| **Clear** | Unset some mapped fields from FullStory |
Hightouch can only clear fields of type `_str`, `_real`, and `_int`. Hightouch sets the
cleared result of `_str` field type to an empty string (`""`) and the
cleared result of `_real` and `_int` to `0`.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Gainsight
**URL:** https://hightouch.com/docs/destinations/gainsight
**Description:** Improve product adoption and reduce churn by enriching Gainsight with rich, up-to-date customer data from your data warehouse
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes | API reference|
|--|--|--|--|
|**Company**|Sync data from any source to Company Objects|Upsert, Update|[Company endpoint](https://support.gainsight.com/Gainsight_NXT/API_and_Developer_Docs/Company_and_Relationship_API/Company_API_Documentation)|
|**Person**|Sync data from any source to Person Objects|Upsert|[Person endpoint](https://support.gainsight.com/Gainsight_NXT/API_and_Developer_Docs/Person_API/Person_API_Documentation)|
|**Custom Objects**|Sync data from any source to Custom Objects|Upsert, Update|[Custom Object endpoint](https://support.gainsight.com/Gainsight_NXT/API_and_Developer_Docs/Custom_Object_API/Gainsight_Custom_Object_API_Documentation)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Gainsight
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Gainsight** and click **Continue**. You can then authenticate Hightouch to Gainsight by entering the following fields into Hightouch:
- **Subdomain**: You can create domains and subdomains by following [these instructions in Gainsight's docs](https://support.gainsight.com/Gainsight_NXT/06Surveys/02Admin_Guides/Set_up_a_Gainsight_Domain).
- **API Key**: You can generate an API key by following [these instructions in Gainsight's docs](https://support.gainsight.com/Gainsight_NXT/Connectors/Connectors/API_Integrations/Generate_API_Access_Key#Generate_Access_Key).
## Sync configuration
Once you've set up your Gainsight destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Gainsight destination you want to sync to.
### Syncing companies
The [Company object](https://support.gainsight.com/Gainsight_NXT/API_and_Developer_Docs/Company_and_Relationship_API/Company_API_Documentation) is one of the core objects in Gainsight and is used to store information about your customers, including individual customer records.
#### Record matching
To match rows from your model to companies in Gainsight, you need to select a model column and corresponding Gainsight field. You can match on any [filterable, unique](https://support.gainsight.com/Gainsight_NXT/API_and_Developer_Docs/Data_Management_APIs/Data_Management_APIs#Get_Describe_OMD) field in Gainsight.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync company fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to any non read-only company fields.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
|Behavior| Description|
|--|--|
|**Do nothing**| Keep the company in Gainsight|
|**Clear** | Remove all the mapped fields, but keep the record in Gainsight|
|**Delete**| Delete the synced company record from Gainsight|
### Syncing people
A Gainsight [Person](https://support.gainsight.com/Gainsight_NXT/API_and_Developer_Docs/Person_API/Person_API_Documentation) is a standard object that stores unique records of people in the real world.
#### Record matching
To match rows from your model to people in Gainsight, you need to select the model column that contains values that match the **Email** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync people fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to any non read-only people fields.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
|Behavior| Description|
|--|--|
|**Do nothing**| Keep the person in Gainsight|
|**Clear** | Remove all the mapped fields, but keep the record in Gainsight|
|**Delete**| Delete the synced person record from Gainsight|
### Syncing custom objects
Hightouch lets you update [Low Volume Custom Objects](https://support.gainsight.com/SFDC_Edition/Data_Management/Managing_Data_In_Gainsight/Low_Volume_Custom_Objects) in Gainsight.
You can use these objects to store low volume data that needs to be edited frequently.
Hightouch diplays Low Volume custom objects in your Gainsight instance in a picklist at the beginning of your sync configuration.
To create custom objects, refer to [Gainsight's docs](https://support.gainsight.com/Gainsight_NXT/02Data_Management/Objects/Gainsight_Objects#Create_Low_Volume_Custom_Objects).
Gainsight doesn't support creating or updating High Volume Custom Objects via [their API](https://support.gainsight.com/Gainsight_NXT/API_and_Developer_Docs/Custom_Object_API/Gainsight_Custom_Object_API_Documentation).
As a workaround, you can sync your data to [S3](/destinations/s3) and use Gainsight's [S3 Connector](https://support.gainsight.com/SFDC_Edition/Connectors/Connectors/File-Based_Integration/Gainsight_S3_Connector) to import your data into a High Volume Custom Object.
#### Record matching
To match rows from your model to custom objects in Gainsight, you need to select a model column and corresponding Gainsight field. You can match on any [filterable, unique](https://support.gainsight.com/Gainsight_NXT/API_and_Developer_Docs/Data_Management_APIs/Data_Management_APIs#Get_Describe_OMD) field in Gainsight.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync custom object fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to any non read-only custom object fields.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
|Behavior| Description|
|--|--|
|**Do nothing**| Keep the custom object in Gainsight|
|**Clear** | Remove all the mapped fields, but keep the record in Gainsight|
|**Delete**| Delete the synced custom object from Gainsight|
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Gainsight PX
**URL:** https://hightouch.com/docs/destinations/gainsightpx
**Description:** Powerful data analytics and product engagement platform
**Section:** Destinations
## Setup
Gainsight PX requires an API key. To find the API key, navigate to the [Gainsight PX Dashboard Settings](https://app.aptrinsic.com/settings/api-keys).
In the administrative settings, select the REST API page under the An API Key can be created on this page.
If an API key has never been created, you can create one here by clicking the New API Key button.
## Syncing users and accounts
Hightouch supports syncing to the following Gainsight PX resources:
- Users
- Accounts
### Sync modes
This integration supports Upsert mode.
In the Upsert mode, new users or accounts will be inserted into Gainsight PX and all attributes will be kept up-to-date within Gainsight PX.
### Record matching
User records can be matched from your source to your Gainsight PX workspace by your Gainsight PX User Email in the Upsert mode.
Account records can be matched from your source to your Gainsight PX workspace by your Gainsight PX Account ID in the Upsert mode.
### Tag keys
Gainsight PX requires a product to sync data to. These products can be accessed by finding the tag key of the product.
Navigating to [Gainsight PX Dashboard Products Page](https://app.aptrinsic.com/settings/products)
All your products can be found here.
The tag key can be copied and entered into the Hightouch sync configuration.
### Field mapping
You can sync columns from your source to Gainsight's default and custom fields. Hightouch automatically detects existing custom fields from your Gainsight PX account.
You can add custom attributes for users and accounts in your Gainsight PX account.
### Delete mode
You can choose what Hightouch's behavior is when records leave the query result set. The default is doing nothing, but you can also set Hightouch to delete the user or account data on record exit.
- `DELETE` : Remove a user or account record and delete the information.
---
## Google Ad Manager 360
**URL:** https://hightouch.com/docs/destinations/gam360
**Description:** Optimize ad delivery with targeted in-sync audiences
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
| ------------ | ---------------------------------------------------------------------- | -------------------- |
| Any data set | Sync data from a source to an Ad Manager cloud storage bucket as a CSV file | Insert, All |
The Google Ad Manager 360 (GAM 360) destination allows you to batch upload identifiers to audience segments in bulk by sending your desired segments to an Ad Manager cloud storage bucket. The segments appear as [a CSV file](#example-exported-file), which you can then use in GAM 360.
The integration supports these sync modes:
- **Insert**: Insert mode creates a CSV file with the rows that were added since the last sync.
- **All**: All mode creates a CSV file with all the rows in the query results, every time the sync runs.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Prerequisites
Before you can use Hightouch to batch upload your identifiers to GAM 360, ensure you've completed these steps, taken from [Google Ad Manager Help](https://support.google.com/admanager/answer/4349785/):
1. Locate and submit your network code. Find your network code in Ad Manager under **Admin > Global settings > Network code**.
2. Ensure that you've created an active first-party audience segment with which you want to associate the identifiers. If you don't want this segment to collect users using your own inventory, select "Publisher Managed" as the Population method.
Expect to wait up for 24 hours from the time of segment creation to the time when you can perform batch upload.
3. [Create a Google Group](https://support.google.com/groups/answer/2464926) that contains all Google Accounts that will have access to upload and view files.
Once your account manager has completed your setup, you can create the file of identifiers you want to upload.
## Connect to Google
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Google Ad Manager 360** and click **Continue**. You can then authenticate Hightouch to Google Ad Manager 360 by entering your:
- **GCP Credentials**: if you need to create a cloud credential, consult [our documentation](/security/gcp)
- **GCP Project ID**: you can follow [these steps](https://support.google.com/googleapi/answer/7014113?hl=en) to find it
- **GCS Bucket Name**: this be should just be the name of the bucket, not a URL
Once you've done this, ensure you run the `gcloud` command to grant Hightouch's service account the [Storage Object Admin](https://cloud.google.com/storage/docs/access-control/iam-roles#standard-roles) role. To run this command, you need to install the [gcloud CLI tool](https://cloud.google.com/sdk/docs/install). Otherwise, if you use [Google Groups](https://groups.google.com/), you can also [assign this role to a group member](https://support.google.com/a/answer/167094) by using the Admin console or Google Groups.
## Sync configuration
Once you've set up your Google Ad Manager 360 destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Google Ad Manager 360 destination you want to sync to.
### List of identifiers
The GAM 360 destination supports several different identifier types.
In the first mapping section, select the column corresponding to the list of identifiers you want to batch upload on the left. On the right, select the type of identifiers that are being uploaded to each segment.
You can find the full list of identifiers in [Google's docs](https://support.google.com/admanager/answer/4349785?hl=en&sjid=15048174647173915590-NA).
### Audience segment identifiers
In the next section, you need to denote the audience segment IDs with which the identifiers from the previous section should be associated. You can either specify a column from your model to associate each identifier with a different segment, or a static value to associate all identifiers with the same segment. Use the toggle on the right of the field to flip between these two options.
### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter '-86400' (`24 hours * 60 minutes * 60 seconds`).
### Empty file results
In insert mode, you can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
## Example exported file
_Example file content for encrypted cookie IDs:_
```
cookie_encrypted,list_id
ScpJKu-yV8je93qkd32MOA,3153490
w2gsrUcwxF-OiJTRmQswQA,3153490
```
_Example file content for raw AdIDs:_
```
cookie_adid,list_id
38400000-8cf0-11bd-b23e-10b96e40000d,3153490
38400001-8cf1-11be-b23f-10b96e40000e,3153490
```
## Tips and troubleshooting
### Common errors
### Sync alerts
---
## Google Cloud Storage
**URL:** https://hightouch.com/docs/destinations/gcs
**Description:** Transfer data at scale from your warehouse to Google Cloud Storage
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
| ------------ | ---------------------------------------------------------------------- | -------------------- |
| Any data set | Sync data from a source to Google Drive as JSON, CSV, or Parquet files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Diff mode creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Google
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Google Cloud Storage** and click **Continue**. You can then authenticate Hightouch to Google Cloud Storage by entering your:
- **GCP Credentials**: if you need to create a cloud credential, consult [our documentation](/security/gcp)
- **GCP Project ID**: you can follow [these steps](https://support.google.com/googleapi/answer/7014113?hl=en) to find it
- **GCS Bucket Name**: this be should just be the name of the bucket, not a URL
## Sync configuration
Once you've set up your Google Cloud Storage destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Google Cloud Storage destination you want to sync to.
### Select file format
Hightouch supports syncing JSON, CSV, Parquet, and XML files to Google Cloud Storage.
### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it.
If you are using an audience and would like to include the audience name, you will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter '-86400' (`24 hours * 60 minutes * 60 seconds`).
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `company`, `email`, `name`, and `price` fields. Hightouch exports these fields to new fields in the file and ignores all other columns from your results.
### CSV options
If you're syncing to a CSV file, you have additional configuration options:
- **Delimiter**: Your options are comma (`,`), semicolon (`;`), pipe (`|`), tilde (`~`), and tab
- (Optional) Whether to **include a CSV header row** in the exported files
- (Optional) Whether to **include a [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark)** in the exported files, the BOM is ``
### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
## Tips and troubleshooting
### Common errors
### Sync alerts
---
## Genesys
**URL:** https://hightouch.com/docs/destinations/genesys
**Description:** Connect conversations across channels and deliver seamless customer experience with Genesys Cloud.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|-------------------|--------------------------------------------------------|------------------------|----------------------------------------------------------------------------------------------------------|
| **External contacts** | Sync data from any source to Genesys external contacts | Upsert, Update, Insert | [Contacts docs](https://developer.genesys.cloud/commdigital/externalcontacts/externalcontacts-apis) |
| **Organizations** | Sync data from any source to Genesys organizations | Upsert, Update, Insert | [Organizations docs](https://developer.genesys.cloud/commdigital/externalcontacts/externalcontacts-apis) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Genesys
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Genesys** and click **Continue**. You can then authenticate Hightouch to **Genesys**.
To create your Genesys OAuth client credentials, follow this [Genesys guide](https://developer.genesys.cloud/authorization/platform-auth/guides/oauth/module-1-client-credentials#:~:text=Create%20Client%20Credentials,Client%20Credentials%20as%20Grant%20Type.).
Enter the following fields into Hightouch:
- **Region** (Required): Find which region to use in the [Genesys API docs](https://developer.genesys.cloud/platform/api/)
- **Client ID** (Required)
- **Client secret** (Required)
## Sync configuration
Once you've set up your Genesys destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Genesys destination you want to sync to.
### Syncing contacts
Sync data from any source to Genesys external contacts
#### Record matching
You can match rows from your model to contacts in Genesys on any column in your model and any field in Genesys. Ensure the data types of the model column and Genesys field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Genesys automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom contact fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
|----------------|--------------------------------------------------------------|
| **Do nothing** | Keep the contact in Genesys with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the contact in Genesys |
| **Delete** | Delete the synced contacts from Genesys |
### Syncing organizations
Sync data from any source to Genesys organizations
#### Record matching
You can match rows from your model to organizations in Genesys on any column in your model and any field in Genesys. Ensure the data types of the model column and Genesys field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Genesys automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync organization fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom organization fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
|----------------|-------------------------------------------------------------------|
| **Do nothing** | Keep the organization in Genesys with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the organization in Genesys |
| **Delete** | Delete the synced organizations from Genesys |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Gladly
**URL:** https://hightouch.com/docs/destinations/gladly
**Description:** Streamline multi-channel customer communication
**Section:** Destinations
## Overview
Gladly is a customer service platform that enables businesses to communicate with their customers across various channels, such as phone, email, chat, social media, and more. Syncing data to Gladly means integrating your customer data with the Gladly platform so that your customer service team can access and use this data to provide better and more personalized support to your customers.
## Supported syncing
|Sync Type|Description|Supported Sync Modes |API Reference |
|--|--|--|--|
|**Customers**|Sync data from any source to Gladly customers|Insert, Upsert, Update| [Customer docs](https://developer.gladly.com/rest/#tag/Customers)|
|**Tasks**|Sync data from any source to any Gladly tasks|Insert, Update|[Task docs](https://developer.gladly.com/rest/#tag/Tasks)|
## Connect to Gladly
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Gladly**. You can then authenticate Hightouch to Gladly by entering:
- A Gladly API Token
- Email on your Gladly account
- Gladly domain
To retrieve a Gladly **API Token**, open the hamburger menu in the top left corner of Gladly. Select the **Settings** option. Under the **App Developer Tools** section, click the **API Tokens** option. You can use an existing token or click the **Create Token** button to generate a new one.
We recommend having the following roles on the account: `Administrator`, `Agent`, `API User`, and `Developer`.
To check the roles, you select **Users** under the **People** section. Hover over the row with the **Email** you provided to Hightouch. Click on the icon with three dots that appear and select **Edit**.
To get the **Email** on your Gladly account, click the user icon in the top right corner. Select the **Profile** option. You should see the account's associated **Email** there.
The **Domain** is the Gladly domain you are logged into. It should be similiar to `https://{organization}.gladly.com`. You only need to provide this portion: `{organization}.gladly.com`.
## Sync configuration
Once you've connected your Gladly destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Gladly destination you want to sync to.
### Customers
Hightouch supports inserting, upserting and updating customers.
#### Record matching
You can match rows in your model with customers in Gladly on **Email**, **External Customer ID**, or **Phone Number**.
#### Field and custom mappings
You can sync any columns from your source to both default properties or custom attributes.
Make sure that if you are creating a new custom attribute, please inform [Gladly Customer Support](https://help.gladly.com/docs/contact-gladly-support) before or after running the sync in order for this attribute to be seen/used on Gladly. If the custom attribute already exists, then you are all set.
### Tasks
Hightouch supports creating new tasks and updating existing ones.
#### Record matching
You can match rows in your model with tasks in Gladly on **Task ID**.
#### Field mapping
You can sync any columns from your source to default properties in Gladly.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Gong
**URL:** https://hightouch.com/docs/destinations/gong
**Section:** Destinations
## Setup
To get started with Gong, create an integration with Gong, and use your Access Token and Secret to create a new destination within Hightouch.
The Gong API page can be found here: \([https://app.gong.io/company/api](https://app.gong.io/company/api)\).
## Select CRM integration
Hightouch allows you to choose which CRM integration you wish to sync data to. This can be selected on the sync configuration page.
## Syncing CRM data
### Upload CRM data
The Gong integration allows users to upsert the following object types:
- CRM Users
- Accounts
- Deals
- Contacts
- Leads
### Field mapping
You can sync columns from your source to Gong's fields.
---
## Google Ads
**URL:** https://hightouch.com/docs/destinations/google
**Description:** Empower your marketing team to run highly granular retargeting and lookalike campaigns on Google
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| **Customer Match User List** | Creates a customer list for the Customer Match feature of Google Ads. Segments are automatically created in Google Ads based on the name of the model | Add, Update, Remove | [Customer Match User List](https://developers.google.com/google-ads/api/docs/remarketing/audience-types/customer-match) |
| **Click Conversions** | Sends click conversion tracking information to Google Ads for each record in the query results | Insert | [Click Conversions](https://developers.google.com/google-ads/api/docs/conversions/upload-clicks) |
| **Click Conversion Adjustments** | Sends click conversion adjustments to Google Ads for each record in the query results | Insert | [Click Conversion Adjustments](https://developers.google.com/google-ads/api/docs/conversions/upload-adjustments) |
| **Call Conversions** | Sends call conversion tracking information to Google Ads for each record in the query results | Insert | [Call Conversions](https://developers.google.com/google-ads/api/docs/conversions/upload-calls) |
| **Offline Store Conversions** | Sends offline store sales conversion tracking information to Google Ads for each record in the query results | Insert | [Offline Store Conversions](https://developers.google.com/google-ads/api/docs/conversions/upload-store-sales-transactions) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
API version in use: [v16](https://developers.google.com/google-ads/api/reference/rpc/v16/overview)
## Connect to Google Ads
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button.
Select **Google Ads** and click **Continue**.
Hightouch supports three authentication methods for Google Ads:
| Method | Description |
| ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Data Manager API OAuth (Recommended)** | OAuth-based authentication that uses Google's [Data Manager API](https://developers.google.com/data-manager/api). Recommended for all new destinations. |
| **OAuth (Legacy)** | OAuth-based authentication that uses the Google Ads API directly. Existing destinations configured with this method continue to work without changes to their sync configurations. |
| **Data Manager API (Existing Partner Link)** | A non-OAuth method for customers who can't complete the OAuth flow (for example, due to two-factor authentication restrictions). Requires a partner link that already exists in your Google Ads account. |
### Data Manager API OAuth (Recommended)
Select **Data Manager API OAuth (Recommended)**, click **Log in to Google Ads**, log in with your Google Account, and click **Allow**.
You should then see a success notification in the Hightouch UI.
### OAuth (Legacy)
Select **OAuth (Legacy)**, click **Log in to Google Ads**, log in with your Google Account, and click **Allow**.
You should then see a success notification in the Hightouch UI.
If you already have a Google Ads destination configured with this method, you can re-authorize it using the **Data Manager API OAuth (Recommended)** method. No changes are required to existing sync configurations.
### Data Manager API (Existing Partner Link)
Use this method if you can't complete the OAuth flow (for example, due to two-factor authentication restrictions on your Google account) and you already have a Hightouch partner link set up in your Google Ads account.
Before connecting, ensure that the partner link between Hightouch and your Google Ads account has been created through the Google Ads UI.
Select **Data Manager API (Existing Partner Link)** and enter your Google Ads account ID. You can find this 10-digit value at the top-right corner of your Google Ads dashboard. Remove any dashes or spaces.
Because this method doesn't authenticate as a specific Google user, Hightouch can't query your Google Ads account on your behalf. As a result, existing user lists and ad accounts don't auto-populate in dropdowns during sync configuration—you'll need to provide these values manually.
## Sync configuration
Once you've set up your Google Ads destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data.
Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin.
Then, select the relevant model and the Google Ads destination you want to sync to.
### Select account
To begin, select the account you want to sync data to.
If an account appears at the top level, it means you have direct access to that ad account or [manager account](https://support.google.com/google-ads/answer/7459399).
Hightouch accesses nested accounts via the root account that you have direct access to.
In the preceding example screenshot, the authenticated account has direct access to the **Hightouch** (5138398925) ad account and the **Hightouch Manager Account** (5277198417).
The authenticated account also has access to the manager account's three nested accounts, including the Hightouch (5138398925) ad account.
Selecting either account at the top level syncs directly to those accounts without logging in via another manager account.
Selecting a nested account requires Hightouch to login via the root manager account.
### Syncing customer match user lists
Google's [Customer Match](https://support.google.com/google-ads/answer/10550383) tool lets you use the online and offline data your customers have shared with you to deliver targeted ads.
Hightouch lets you create and maintain customer match lists via the Google Ads API.
It can take 6 to 48 hours for a list to be populated with members once Hightouch has sent the data to Google.
You can find more information about this on the API reference page linked in the [supported syncing](#supported-syncing) section.
#### User list type
To begin, select the [upload key type](https://developers.google.com/google-ads/api/reference/rpc/v13/CustomerMatchUploadKeyTypeEnum.CustomerMatchUploadKeyType) for the customer match list.
It can be one of these values:
- **CONTACT_INFO**
- **MOBILE_ADVERTISING_ID**
- **CRM_ID**
Choose the type that aligns with the data fields you are sending.
For example, if you are sending email or address data, select **CONTACT_INFO**.
#### Select an existing list or create a new one
You can create a new [Customer match user list](https://support.google.com/google-ads/answer/6276125) or use an existing one.
When creating a new list, you can optionally enter a name.
Otherwise, Hightouch defaults to the name of the associated model.
To use an existing list, select the desired list from the dropdown.
If you choose to create a new list, you can't update the
[account](#select-account) to which you're syncing data after the initial sync
run. If you need to change the account, configure a new sync.
Optionally, you can also set the **membership lifespan** for the list.
The value must be between 0 and 540. If you do not set this value, the default value is 540.
Starting April 7th, 2025, the maximum membership duration for a Google Ads customer list is **540 days**.
From this date, existing lists configured to have no membership expiration set or membership expiration greater than 540 days will be updated to use the maximum membership duration of 540 days, and a membership duration of 540 days will be retroactively applied to all those existing members that have a longer duration.
See [Google's announcement](https://ads-developers.googleblog.com/2025/02/update-to-customer-match-membership.html) for more information.
#### User identifiers
To identify which users to add or update in an audience, select model columns and the corresponding Google Ads fields.
You must map at least one identifier.
#### Handling PII and hashing
You can enable PII hashing if you wish for Hightouch to detect whether your data requires normalization and hashing based on the chosen record mapping.
For example, if you select **Email** and the model column with the email data is unhashed, Hightouch hashes the value for you.
By default, Hightouch automatically detects if the following fields require hashing:
- **Email**
- **Phone**
- **First Name**
- **Last Name**
#### Cleaning your data
Optionally, you can decide if Hightouch should clean your phone numbers and remove `null` or incorrectly formatted values from your data.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results.
You have the following options:
| Behavior | Description |
| ----------------- | -------------------------------------------------------- |
| **Do nothing** | Keep the record in the audience |
| **Remove record** | Remove the Google Ads record from the specified audience |
#### EU user consent policy
If you are uploading data for users in the European Economic Area ([EEA]()),
you need to ensure you are passing the required consent signals to [Google](https://www.google.com/about/company/user-consent-policy/?sjid=2196795394224792925-NA).
For each sync, you can specify the consent status for these types:
- `ad_user_data`: consent for sending user data to Google for advertising purposes
- `ad_personalization`: consent for ad personalization
Check out Google's support [doc](https://support.google.com/google-ads/answer/14310715?hl=en) to learn more.
### Syncing conversion events
Hightouch supports syncing the following conversion events:
- [Click conversions](#syncing-click-conversions) and [Offline click conversions](#offline-click-conversions)
- [Click conversion adjustments](#syncing-click-conversion-adjustments)
- [Call conversions](#syncing-call-conversions)
- [Offline store conversions](#syncing-offline-store-conversions)
When syncing conversion events, Hightouch treats any records added to [your source](/getting-started/concepts) as new events and sends them to Google Ads when your sync runs.
#### Custom conversion variables
You can [map](/syncs/mapping-data) custom variables when syncing conversion events to Google Ads.
First, you need to set up [custom conversion variables](https://support.google.com/google-ads/answer/9962082) in Google. Custom conversion variables are only available for the following:
- Conversion tracking using [tags on your website](https://support.google.com/google-ads/answer/6331314), if you're syncing [click conversions](#syncing-click-conversions) or [call conversions](#syncing-call-conversions)
- [Offline conversion import](https://support.google.com/google-ads/answer/2998031), if you're syncing [offline click conversions](#syncing-offline-click-conversions)
You can read more about this in [Google Ads Help](https://support.google.com/google-ads/answer/9964350) and also in [Google Ads' API documentation](https://developers.google.com/google-ads/api/docs/conversions/conversion-custom-variables).
#### Currency codes
Google requires all conversion events to include a currency code in a separate column in addition to the amount.
This applies to [click conversions](#syncing-click-conversions), [click conversion adjustments](#syncing-click-conversion-adjustments), and [offline store conversions](#syncing-offline-store-conversions).
Consult Google's chart of [supported currency codes](https://developers.google.com/adsense/management/appendix/currencies) to ensure your data meets their specifications.
#### Enhanced conversions
You can sync both types of [**Enhanced conversions**](https://support.google.com/google-ads/answer/9888656) to Google Ads using Hightouch:
1. [**Enhanced conversions for leads**](https://support.google.com/google-ads/answer/9888656#leads): set up an [Offline click conversion](#offline-click-conversions) in Google Ads; then select **Click Conversions** as the [sync type](#supported-syncing) and [set up your field mappings](#user-identifiers-1) accordingly
2. [**Enhanced conversions for web**](https://support.google.com/google-ads/answer/13261987): select [Click Conversion Adjustments](#syncing-click-conversion-adjustments) as the [sync type](#supported-syncing) and **Enhancement** as the [adjustment type](#select-conversion-action-and-adjustment-type)
#### EU user consent policy
If you are uploading data for users in the European Economic Area ([EEA]()),
you need to ensure you are passing the required consent signals to [Google](https://www.google.com/about/company/user-consent-policy/?sjid=2196795394224792925-NA).
For [click conversions](#syncing-click-conversions), [call conversions](#syncing-call-conversions) and [offline store conversions](#syncing-offline-store-conversions), you can specify the Google consent status for advertising purposes, by mapping these fields in the sync configuration:
- `Ad user data consent` (for the `ad_user_data` property)
- `Ad personalization consent` (for the `ad_personalization` property)
As per [Google's API documentation](https://developers.google.com/google-ads/api/reference/rpc/v17/Consent), these fields only accept the following values: `GRANTED`, `DENIED`, `UNSPECIFIED`, `UNKNOWN`.
Check out Google's support [doc](https://support.google.com/google-ads/answer/14310715?hl=en) to learn more.
### Syncing click conversions
Google Ads allows you to [import conversions from ad clicks](https://support.google.com/google-ads/answer/12216424) to see how your online ads are performing.
Before setting up your sync, you need to [set up a conversion action](https://support.google.com/google-ads/answer/12216226) in Google Ads.
After clicking **New conversion action**, remember to select **Website**.
It can take over 24 hours for click conversions to be processed by Google.
You can find more information about this and other requirements on the API reference page linked in the [supported syncing](#supported-syncing) section.
#### Offline click conversions
**Offline click conversions** are not [offline store
conversions](#syncing-offline-store-conversions).
You can sync [offline click conversion imports](https://support.google.com/google-ads/answer/2998031) which allow you to measure what happens in the offline world after an ad results in a click or call to your business.
For these events, you need to [set up offline conversion imports](https://support.google.com/google-ads/answer/7012522).
After clicking **New conversion action**, remember to select **Import**.
Then select **CRMs, files, or other data sources** and **Track conversions from clicks** and continue the offline conversion import setup. You can safely skip setting up a data source.
Check out our [blog
post](https://hightouch.com/blog/google-offline-conversions) and
[playbook](https://hightouch.com/playbooks/automate-offline-conversion-uploads-to-maximize-roas)
to learn how to sync **offline click conversions** to Google Ads.
#### Select conversion action
Select the **conversion action** you previously set up in your Google Ads account.
The dropdown menu displays the conversion action IDs present in your Google Ads account.
#### User identifiers
Select which columns to sync from your source to Google Ads' fields. When syncing click conversions, at least one identifier is required:
- **GCLID**
- **GBRAID**
- **WBRAID**
- **User hashed email(s)**
- **User hashed phone(s)**
Furthermore, **Conversion time** is a mandatory field.
You can send [enhancements as click conversions](https://support.google.com/google-ads/answer/9888656#leads) by mapping the **User hashed email(s)** and **User hashed phone(s)** fields instead of **GCLID**.
The [model columns](/models/data-types-casting) mapped to these fields can contain either single values or arrays.
Read through Google Ads' [API documentation](https://developers.google.com/google-ads/api/docs/conversions/upload-identifiers) for more details.
If you opt to send user emails and/or phone numbers instead of **GCLID**, keep in mind that each click conversion can only have up to five user identifiers in total.
For example, a click conversion can have three emails and two phone numbers, or any combination that adds up to five.
These enhancements are also referred to as **Enhanced conversions for leads**.
They are not **Enhanced conversions for web**, which are described in the
[Syncing click conversion
adjustments](#select-conversion-action-and-adjustment-type) section.
Avoid storing your **GCLID**, **GBRAID**, and **WBRAID** data in the same model column since this can cause issues when syncing your data.
[Google Ads' limitations](https://developers.google.com/google-ads/api/docs/conversions/upload-clicks#id_parameters) determine which values can and cannot be synced together.
If this data is in the same model column and you try mapping this column to multiple Google Ads fields (such as **GCLID** and **WBRAID**), you might receive a `VALUE_MUST_BE_UNSET` or `GBRAID_WBRAID_BOTH_SET` error.
Check out the [common errors](#common-errors) section to learn more about these errors.
If a model row contains a `null` value in a model column mapped to the
**GCLID**, **GBRAID**, or **WBRAID** fields, Hightouch ignores that value when
syncing the row Google Ads. This process helps reduce
[`VALUE_MUST_BE_UNSET`](#value_must_be_unset) or
[`GBRAID_WBRAID_BOTH_SET`](#gbraid_wbraid_both_set) errors.
#### EU user consent policy
Learn more about the EU consent policy [in this section](#eu-user-consent-policy-1).
### Syncing click conversion adjustments
After sending a click conversion to Google Ads, you can [adjust the conversion](https://support.google.com/google-ads/answer/7686447) later by sending a click conversion adjustment.
Wait at least 24 hours after a click conversion is reported to Google Ads before adjusting it to avoid a `CONVERSION_NOT_FOUND` or `TOO_RECENT_CONVERSION` error.
You can find more information about this on the API reference page linked in the [supported syncing](#supported-syncing) section.
#### Select conversion action and adjustment type
Select the **conversion action** you previously set up in your Google Ads account.
The dropdown menu displays the conversion action IDs present in your Google Ads account.
You also need to select the adjustment type. You can choose from:
- [Retraction](https://support.google.com/google-ads/answer/7686447)
- [Restatement](https://support.google.com/google-ads/answer/7686447)
- [Enhancement](https://support.google.com/google-ads/answer/13261987)
Enhancements are also referred to as **Enhanced conversions for web**. They
are not **Enhanced conversions for leads**, which are described in the
[Syncing click conversions](#user-identifiers-1) section.
If you select the **Enhancement** adjustment type, you need to select if this is a first party or third party data upload.
#### Field mapping
Select which columns to sync from your source to Google Ads' fields:
- **GCLID**
- **Conversion time**
- **Adjustment time**
- **Order ID**
If you select the **Retraction** adjustment type, you can only map those fields.
If you select the **Restatement** adjustment type, you can map those four fields plus:
- **Restatement value**, which is mandatory
- **Restatement currency**
If you select the **Enhancement** adjustment type, you can map those four fields plus:
- **Enhancement User Agent**
- various hashed fields: **Email**, **Phone**, **First Name**, **Last Name**, **Street Address**
- **Country**
- **Postal Code**
- **State**
- **City**
#### Handling PII and hashing
You can find more information about this in the [PII and hashing](#handling-pii-and-hashing) section.
### Syncing call conversions
Google Ads allows you to import phone call conversions to keep track of how your calls impact your ad campaigns.
After setting up [phone call conversions](https://support.google.com/google-ads/answer/6275629), you can use Hightouch to sync your data to Google Ads.
You can find more information about this on the API reference page linked in the [supported syncing](#supported-syncing) section.
#### Select conversion action
Select the **conversion action** you previously set up in your Google Ads account.
The dropdown menu displays the conversion action IDs present in your Google Ads account.
#### Field mapping
Select which columns to sync from your source to Google Ads' fields.
**Conversion time**, **Caller ID**, and **Call start time** are mandatory fields.
#### EU user consent policy
Learn more about the EU consent policy [in this section](#eu-user-consent-policy-1).
### Syncing offline store conversions
**Offline store conversions** are not [offline click
conversions](#offline-click-conversions).
Google Ads offers enhanced store sales measurement and reporting data to better optimize your online ads.
After setting up [offline store sales conversions](https://support.google.com/google-ads/answer/9995886), you can use Hightouch to sync your data to Google Ads.
You can find more information about this on the API reference page linked in the [supported syncing](#supported-syncing) section.
#### Select conversion action
Select the **conversion action** you previously set up in your Google Ads account.
The dropdown menu displays the conversion action IDs present in your Google Ads account.
#### Optional settings
You can decide if you would like to include the following information:
- **custom key**, to segment store sales conversions
- **loyalty fraction**, which is a decimal value that sets the percentage of overall sales that can be associated with a customer
- **transaction upload fraction**, which is a decimal value that sets the percentage of overall sales with customer information that are already uploaded to Google Ads
#### First or third party data
You need to select if this is a first party or third party data upload.
#### Field mapping
Select which columns to sync from your source to Google Ads' fields.
**Transaction time** and **Transaction amount (micros)** are mandatory fields.
Any transaction amount here should be in the unit **micros**. For example,
$200 = 200000000.
#### EU user consent policy
Learn more about the EU consent policy [in this section](#eu-user-consent-policy-1).
#### Handling PII and hashing
You can find more information about this in the [PII and hashing](#handling-pii-and-hashing) section.
#### Cleaning your data
You can find more information about this in the [Cleaning your data](#cleaning-your-data) section.
## Tips and troubleshooting
### Matched users count
Below only applies to the customer match user list sync type.
Google Ads contain a few different matched users count in their platform.
Hightouch refers to the [display network
size](https://support.google.com/google-ads/answer/117120?ctx=glossary&sjid=7733029462129080931-NA)
and the [search network
size](https://support.google.com/google-ads/answer/90956?ctx=glossary&sjid=7733029462129080931-NA).
If the display network size is available, that is used in our app, else the
search network size is used. Google Ads also have a concept of a match rate
but this is different than what Hightouch is calculating, which is the % of
targetable users from your model in this GoogleAds audience. Per Google, ”Your
Customer Match match rate is the percentage of your uploaded customer list
that could be connected to Google users, so you can see how much of your list
is usable. Keep in mind that your match rate is not an indicator of how your
lists will perform, but rather an indicator of whether you're uploading the
right data in the right way”
([reference](https://support.google.com/google-ads/answer/10534785?hl=en)).
### Common errors
#### At least one click identifier ('gclid', 'gbraid' or 'wbraid') is required
Make sure that you mapped at least one identifier when syncing [Click conversions](#field-mapping).
#### CONVERSION_PRECEDES_EVENT
As described in [Google Ads' documentation](https://developers.google.com/google-ads/api/reference/rpc/v13/ConversionUploadErrorEnum.ConversionUploadError#conversion_precedes_event), this error occurs if the imported event has a `conversion_date_time` that precedes the click.
To avoid this error, make sure that the timestamps in your source data and your Google Ads account have the same timezone configured.
#### EXPIRED_EVENT
As explained in [Google Ads' documentation](https://developers.google.com/google-ads/api/reference/rpc/v13/ConversionUploadErrorEnum.ConversionUploadError#expired_event), this error happens when a [click conversion](#syncing-click-conversions) can't be accepted since its click occurred before this conversion's [click-through window](https://support.google.com/google-ads/answer/3123169).
#### GBRAID_WBRAID_BOTH_SET
This error happens if you sync data both to the **GBRAID** and **WBRAID** fields.
You can read more about this in [Google Ads' API documentation](https://developers.google.com/google-ads/api/docs/conversions/upload-clicks).
#### INCOMPATIBLE_UPLOAD_KEY_TYPE
When syncing Customer Match user lists, make sure to select the correct [user list type](#user-list-type). If you selected the wrong type when setting up your sync, create a new sync with the correct type.
#### INVALID_CONVERSION_ACTION
When syncing Click conversions, ensure that you've chosen the correct conversion type within the Google Ads UI.
For normal [Click conversions](#syncing-click-conversions), choose the **Website** type. For [Offline click conversions](#offline-click-conversions), choose the **Import** type.
Furthermore, make sure that:
- your data satisfies the requirements outlined in [this section](https://developers.google.com/google-ads/api/docs/conversions/upload-clicks#upload_clickconversion) of Google Ads' API documentation
- you selected the correct [Google Ads account](#select-account) in your sync configuration
Make sure to not confuse [Offline click
conversions](#offline-click-conversions) with [Offline store
conversions](#syncing-offline-store-conversions).
#### INVALID_CUSTOMER_ID
You may receive a `400 - {"error":{"code":400,"message":"Request contains an invalid argument."..."message":"Invalid customer ID...}` error message because you are attempting to sync to a Google account that is different from the initial sync.
If there has been a successful initial sync with sync configurations of `Customer Match User List` and `Create a new user list`, changing the Google account will result in sync failure on the next run. You can resolve the error by either:
- Selecting the original Google account that was used on the initial sync
- Creating a new sync and selecting the new Google Account
#### INVALID_GRANT
The token for the user that connected the Google Ads destination has expired or revoked. Please re-auth within the destination settings.
#### No error message provided
This error can happen if you [rename a model column](/syncs/mapping-data#model-column-changes) after adding it as a field mapping in your sync configuration.
To resolve this, update your field mappings to contain the updated model column names.
#### REQUIRED_FIELD_MISSING
[Customer Match User Lists](#record-matching) require at least one personal identifier. Check the required fields relevant for the match list type you've chosen.
#### UNAUTHORIZED_CUSTOMER
This can happen when you try syncing a [click conversion](#syncing-click-conversions) to Google Ads that is associated with a different ad account. If you wish to sync click conversions from different ad accounts without needing to use separate syncs, you can set up a manager account and enable [cross-account conversion tracking](https://support.google.com/google-ads/answer/3030657).
#### VALUE_MUST_BE_UNSET
This error happens if you:
- sync data to the **GCLID** field and also to the **GBRAID** or **WBRAID** fields
- sync data to [custom conversion variables](#custom-conversion-variables) and also to the **GBRAID** or **WBRAID** fields
You can read more about this in [Google Ads' API documentation](https://developers.google.com/google-ads/api/docs/conversions/upload-clicks).
### Live debugger
### Sync alerts
---
## Google ADH (BigQuery)
**URL:** https://hightouch.com/docs/destinations/google-adh-bigquery
**Description:** Google Ads Data Hub enables customized analysis that aligns with your specific business objectives, while respecting user privacy and upholding Google’s high standards of data security.
**Section:** Destinations
## Overview
Hightouch lets you send data from your warehouse into Google BigQuery to join it with Google's Ads Data Hub.
Connecting Hightouch to BigQuery requires some setup in both platforms. It's recommended to set up a service account with the correct permissions in BigQuery before configuring the connection in Hightouch.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------ | ---------------------------------------------- | -------------------- |
| Any data set | Sync data from any source to a Big Query table | Insert |
## BigQuery credential setup
Setup in BigQuery has three main steps:
1. Enable BigQuery for your Google Cloud project
2. Create a service account
3. Grant the Hightouch service account access to your project
### Create a project and enable BigQuery API
1. Login to the [Google Developers Console](https://console.developers.google.com/).
2. Configure the [Cloud Platform](https://console.cloud.google.com/):
- If you don't have a project already, [create one](https://support.google.com/cloud/answer/6251787?hl=en&ref_topic=6158848).
- Once you have a project, [enable the BigQuery API](https://console.cloud.google.com/flows/enableapi?apiid=bigquery&_ga=2.71379221.724057513.1673650275-1611021579.1664923822&_gac=1.213641504.1673650813.EAIaIQobChMIt9GagtPF_AIVkgB9Ch331QRREAAYASAAEgJfrfD_BwE) for it.
3. Copy your **Project ID** for later use.
4. Find the **location** of your BigQuery dataset or sets. You can find this by querying the `INFORMATION_SCHEMA.SCHEMATA` view or by visiting the [Google Cloud web console](https://console.cloud.google.com/) and clicking on a BigQuery dataset in the Explorer panel. You need both **Project ID** and **Data location** when connecting Hightouch to BigQuery.
Make sure [billing is
enabled](https://support.google.com/cloud/answer/6293499#enable-billing) on
your project, otherwise Hightouch can't write into the cluster.
### Create a service account
To create a service account, follow the setup instructions in our [Google Cloud Platform (GCP) documentation](/security/gcp).
### Grant access
Your GCP service account will need [permission](https://cloud.google.com/bigquery/docs/access-control#permissions-predefined-roles) to read and write data to BigQuery.
You can set up your service account to have [full access](#grant-full-access) to your project using a [predefined role](https://cloud.google.com/iam/docs/understanding-roles#predefined_roles).
Otherwise, you can create a [custom role](https://cloud.google.com/iam/docs/roles-overview#custom) and provide [limited access](#grant-limited-access) according to a user-specified list of permissions.
#### Grant full access
You can grant full access by assigning the `bigquery.user`, `bigquery.dataViewer`, and `bigquery.dataEditor` roles to your service account.
You can do this in the [Google Cloud web console](https://cloud.google.com/iam/docs/granting-changing-revoking-access#grant-single-role) or by running these snippets in the [Cloud Shell](https://cloud.google.com/shell/docs/run-gcloud-commands).
Grant permission to **read metadata and list tables**:
```bash
gcloud projects add-iam-policy-binding \
--member serviceAccount: \
--role roles/bigquery.user
```
Grant permission to **read data from tables and views**:
```bash
gcloud projects add-iam-policy-binding \
--member serviceAccount: \
--role roles/bigquery.dataViewer
```
Grant permission to **write data to tables and views**:
```bash
gcloud projects add-iam-policy-binding \
--member serviceAccount: \
--role roles/bigquery.dataEditor
```
#### Grant limited access
If you don't want to grant full access to your BigQuery service account, you can opt to grant limited access instead.
You can do this by assigning the `bigquery.dataViewer` and `bigquery.dataEditor` roles only to the specific datasets, tables, or views you want to use in Hightouch.
Since you are assigning these roles only to specific resources, you need to assign the `bigquery.user` role **and** grant the `bigquery.tables.get` permission at the project level.
For this, you can create a [custom role](https://cloud.google.com/iam/docs/creating-custom-roles#creating) in the Google Cloud web console based on an [existing predefined role](https://cloud.google.com/bigquery/docs/access-control#bigquery.user) (`bigquery.user`), which you can name `custom.bigquery.user`.
When setting up the custom role, click **Add permissions** to add the `bigquery.tables.get` permission to this custom role.
Then assign this role to your service account at the project level.
You can do this in the [Google Cloud web console](https://cloud.google.com/iam/docs/granting-changing-revoking-access#grant-single-role) or by running this snippet in the [Cloud Shell](https://cloud.google.com/shell/docs/run-gcloud-commands):
```bash
gcloud projects add-iam-policy-binding \
--member serviceAccount: \
--role roles/custom.bigquery.user
```
If this `custom.bigquery.user` role still isn't limited enough, you can try assigning the [`bigquery.jobUser`](https://cloud.google.com/bigquery/docs/access-control#bigquery.jobUser) role **and** granting the `bigquery.dataset.get`, `bigquery.tables.get`, and `bigquery.tables.list` permissions at the project level.
Hightouch needs to be able to list the schemas and tables in your BigQuery
project when setting up your sync configuration.
## Connect to Google ADH via BigQuery
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Google ADH (Big Query)** and click **Continue**.
### Configure your service account
Select the **GCP credentials** you [previously created](#create-a-service-account) or click **Create new**. To learn more about these credentials, see the [Google Cloud Provider (GCP) documentation](/security/gcp).
### Configure your destination
Enter the **Project ID** for the project you [enabled the BigQuery API](#create-a-project-and-enable-bigquery-api) for and the **Dataset location**.
## Sync configuration
Once you've set up your Google ADH (BigQuery) destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Snowflake destination you want to sync to.
### Sync mode
Hightouch supports **Insert** mode with the option to send boosted data with [Match Booster](../match-booster/overview).
### Syncing with match booster
You can choose to send boosted emails and/or phones to BigQuery. Hightouch will clean and hash the data accordingly.
## Tips and troubleshooting
### Common errors
### Compatible versions
The supported Oracle DB version is `19c`. Other versions aren't officially supported but may work regardless.
### Live debugger
### Sync alerts
---
## Google Analytics
**URL:** https://hightouch.com/docs/destinations/google-analytics
**Description:** Send events directly to Google Analytics from your data warehouse
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-----------|-----------------------------------|----------------------|------------------------------------------------------------------------------------------------------------------------
**Events** | Sync events to GA4 via the Measurement Protocol API | Insert | [Measurement Protocol Reference](https://developers.google.com/analytics/devguides/collection/protocol/ga4/reference?client_type=firebase)
**Anonymous Audiences** | Sync events and user properties to GA4 to create anonymous audiences | Upsert | [Audiences reference](https://support.google.com/analytics/answer/12799087?hl=en)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Google Analytics
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Google Analytics** and click **Continue**. You can then authenticate Hightouch to **Google Analytics** by entering a Google Analytics **API Secret**.
To generate an API secret, navigate to **Admin > Data Streams > choose your stream > Measurement Protocol > Create**. If you don't have any existing data streams you need to [create one](https://support.google.com/analytics/answer/9304153?sjid=1280583774983537157-NA#stream) first.
If you are syncing anonymous audiences, you must additionally authenticate via OAuth to grant Hightouch the `analytics.edit` scope. This is needed to create new audiences in GA4.
## Sync configuration
Once you've set up your Google Analytics destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Google Analytics destination you want to sync to.
### Syncing events
Hightouch supports sending following Google Analytics event types:
- [**Firebase App Events**](https://developers.google.com/analytics/devguides/collection/protocol/ga4/reference?client_type=firebase)
- [**Gtag.js Events**](https://developers.google.com/analytics/devguides/collection/protocol/ga4/reference?client_type=gtag#payload)
Firebase app events require an associated **Firebase app ID** (`firebase_app_id`) and **app instance ID** (`app_instance_id`). You can find the `firebase_app_id` in the Firebase console in **Project Settings > General > Your Apps > App ID**. You can retrieve the `app_instance_id` through the [Firebase SDK](https://firebase.google.com/docs/reference/android/com/google/firebase/analytics/FirebaseAnalytics#public-taskstring-getappinstanceid).
If you are having issues syncing your Firebase app events, check out [Google's troubleshooting docs](https://developers.google.com/analytics/devguides/collection/protocol/ga4/troubleshooting?client_type=firebase).
The `app_instance_id` isn't the same as your `firebase_app_id`. The `firebase_app_id` identifies your app uniquely, whereas `app_instance_id` identifies a single installation of the app uniquely.
Gtag.js events require an associated **measurement ID** (`measurement_id`) and **client ID** (`client_id`). You can find the `measurement_id` in the Google Analytics UI in **Admin > Data Streams > choose your stream > Measurement ID**. You can retrieve the `client_id` using [these examples](https://developers.google.com/tag-platform/gtagjs/reference#get) from Google.
If you are having issues syncing your Gtag.js events, check out [Google's troubleshooting docs](https://developers.google.com/analytics/devguides/collection/protocol/ga4/troubleshooting?client_type=gtag).
All users of your website have the same `measurement_id`, but the `client_id` value is different for every user of your app.
Refer to Google Analytics [required parameters](https://developers.google.com/analytics/devguides/collection/protocol/ga4/sending-events?client_type=firebase#required_parameters) for more information.
#### Measurement ID
Your **measurement ID** in Google Analytics is an identifier for a web data stream. It typically starts with `UA`, `MO`, or `G` followed by 10-20 characters, for example `UA-12345678-9`. You can find your measurement ID by following the **Find your Measurement ID** link in [Google's docs](https://support.google.com/analytics/answer/12270356?hl=en#:~:text=A%20Measurement%20ID%20is%20an,same%20as%20your%20destination%20ID).
You can also use the steps outlined in the preceding section.
#### Event name
Google Analtyics requries an event name for all events. You can either provide a static value or select to **use a column** from your model data. Hightouch syncs the static or column value as the `events[].name` parameter Google Analytics requires. See [Google's events reference](https://developers.google.com/analytics/devguides/collection/protocol/ga4/reference/events) for all event name options.
#### Event timestamp
Optionally, you can decide to select a model column that contains your event timestamps. Timestamps should be in [ISO 8601](https://www.iso.org/iso-8601-date-and-time-format.html) format or [Unix timestamps](https://en.wikipedia.org/wiki/Unix_time) in milliseconds, which are automatically converted to the format required by Google Analytics.
If this is selection is empty, Google Analytics creates an event timestamp by using the time the event arrives at their server.
#### Field mapping
Field mapping is where you select which event parameters you want to send to Google Analytics. You can map data from any of your model columns to event parameters in Google Analytics. Ensure your model column data type matches the data type of the parameter you want to sync to.
Different event types require different parameters. For example, `purchase` events require the following parameters:
- `currency` (string)
- `value` (number)
- `transaction_id` (string)
- `items` (array with additional [required parameters](https://developers.google.com/analytics/devguides/collection/protocol/ga4/reference/events#purchase_item))
You can use Hightouch's [advanced mapper](https://hightouch.com/docs/syncs/mapping-data#advanced-mapper) to map [column](/syncs/mapping-data#column-values), [variable](/syncs/mapping-data#variable-values), and [static](/syncs/mapping-data#static-values) values, for example, "USD" for `currency`.
See [Google's events reference](https://developers.google.com/analytics/devguides/collection/protocol/ga4/reference/events) for the suggested parameters for each event.
### Syncing audiences
Create anonymous audiences based on user properties in GA4.
Hightouch creates anonymous audiences in GA4 by first creating a new [custom dimension](https://support.google.com/analytics/answer/14240153?hl=en) based on the name of your audience. The custom dimension will be in the format `HT_`.
Once the dimension is created, Hightouch creates an audience using the new custom dimension as a filter. Hightouch will send events with the custom dimension value set to `1` in user properties, and filter based on that value. Anonymous users with the custom dimension property set will be included in your anonymous audience.
When a record is removed from your audience, the custom audience value is set to `-1` and therefore removed from your audience.
#### Audience population
There may be a lag between syncing events and user properties to GA4 and seeing audiences being populated.
GA4 recommends sending an event with user properties to see your audience populated with the least lag time.
## Tips and troubleshooting
### Validating events
According to the [Google Analytics documentation](https://developers.google.com/analytics/devguides/collection/protocol/ga4/validating-events?client_type=gtag), the Google Analytics Measurement Protocol for Google Analytics 4 does not return HTTP error codes, even if an event is malformed or missing required parameters. To ensure your events are valid, you should test them against the Measurement Protocol Validation Server before deploying them to production.
You can run syncs using the Measurement Protocol Validation Server, if selected in your Hightouch sync config. Alternatively, before running your sync in Hightouch, use this [tool from Google Analytics](https://ga-dev-tools.google/ga4/event-builder/) to validate events and make sure you're not missing any parameters.
If you don't validate the event and are missing parameters in the [field mapping](#field-mapping) section, the sync will look healthy in Hightouch because the response from GA4 won't contain errors, but you won't see events in GA4.
### Common errors
### Live debugger
### Sync alerts
---
## Google Campaign Manager 360
**URL:** https://hightouch.com/docs/destinations/google-campaign-manager
**Description:** Empower your marketing team to dynamically and intelligently serve and view of all of their ad campaigns across all channels
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|----------------------------|------------------------------------------------------------|----------------------|----------------------------------------------------------------------------------------------------------------------|
| **Conversions** | Sync records from your model as new conversion events | Insert | [Conversions docs](https://developers.google.com/doubleclick-advertisers/rest/v4/conversions/batchinsert) |
| **Conversion Adjustments** | Sync records from your model to existing conversion events | Insert | [Conversion Adjustments docs](https://developers.google.com/doubleclick-advertisers/rest/v4/conversions/batchupdate) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Google Campaign Manager 360
You need to have access to at least one [user profile](https://support.google.com/campaignmanager/answer/6098287?hl=en&sjid=13478745075182123470-NA) and one [advertiser](https://support.google.com/campaignmanager/answer/2829356?hl=en&sjid=13478745075182123470-NA) in Google Marketing Platform to sync data.
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Google Campaign Manager 360** and click **Continue**. You can then authenticate Hightouch to **Google Campaign Manager 360** via OAuth.
For the **Authentication method**, select **Log in to Google Campaign Manager 360** and log into your Google Campaign Manager 360 account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Google Campaign Manager 360 destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Google Campaign Manager 360 destination you want to sync to.
### Syncing conversion events
You can choose to sync new conversion events or update existing ones via the **Conversion Adjustments** sync type.
Then, select the Google Campaign Manager profile to sync conversion data to.
#### Floodlight activities and configuration
Google Campaign Manager requires each conversion to be associated with a [Floodlight activity](https://support.google.com/campaignmanager/answer/3027419?hl=en&sjid=13478745075182123470-NA) and [configuration](https://support.google.com/campaignmanager/answer/2893922).
Hightouch provides a dropdown to select the available activities and configurations based on your Google Campaign Manager profile.
You can either select a static value for all events or select to **use a column** from your model.
#### User advertising identifiers
You can opt to include encrypted user advertising IDs in your conversion event payload.
Hightouch obtains encrypted user advertising IDs from the [%m match macro](https://developers.google.com/doubleclick-advertisers/guides/conversions_overview#using_the_m_match_macro) or [Data Transfer](https://developers.google.com/doubleclick-advertisers/guides/conversions_overview#using_data_transfer).
If you are including, `dclid`, `gclid`, `matchId`, or a mobile device ID in your [field mapping](#field-mapping), you don't need to enable this.
#### Field mapping
Field mapping is where you select which customer information parameters or other event parameters you want to send to the Conversions API.
You can map data from any of your model columns to default and custom Google fields.
The Conversions API requires you to include at least one of the user identifier parameters below in your event payload, so be sure to map one here.
The identifiers options are:
- Mobile device ID
- Google Click ID
- Match ID
- Display Click ID
- Encrypted user ID
- Encrypted user ID candidates
#### Enhanced conversions mappings
The CM360 offline conversion API supports [enhanced conversions](https://developers.google.com/doubleclick-advertisers/guides/conversions_ec) with additional user identifiers. To send enhanced conversions, you can map data from any of your model columns to the following user identifiers:
- Hashed email
- Hashed phone number
- Hashed first name
- Hashed last name
- Hashed street address
- City
- State
- Country code
- Postal code
You should provide **only one** of these user identifiers for matching.
For more information, please check the [Google Campaign Manager 360 documentation](https://developers.google.com/doubleclick-advertisers/rest/v3.5/Conversion).
## Tips and troubleshooting
### Common errors
When uploading offline conversions CM360 enforces a 28 day window, therefore, any conversion timestamp older than 28 days will be rejected with errors like "The timestamp is too old". If you receive this error then you should check to ensure that the timestamp of your conversions is at most 28 days old from the time of submission.
This information, along with a few other FAQs, can be found on the CM360 Conversions FAQ page [here](https://developers.google.com/doubleclick-advertisers/guides/conversions_faq).
### Live debugger
### Sync alerts
---
## Google Cloud Functions
**URL:** https://hightouch.com/docs/destinations/google-cloud-functions
**Description:** Build your own custom destination using a serverless function
**Section:** Destinations
## Overview
This destination invokes a [Google Cloud Function](https://cloud.google.com/functions) whenever data changes in your model. It makes it possible to build your own custom integrations that execute code written in Javascrip, Java, Python, Go, Ruby and PHP. See [how to write HTTP function in Google Cloud Functions](https://cloud.google.com/functions/docs/writing/write-http-functions) for more details.
This destination was designed to be as flexible as possible. You can exercise granular control over function triggers, batching, rate limits, concurrency limits, and even error handling. Together, these features let you integrate Hightouch with any internal tool or third-party API.
Under the hood, the Google Cloud Functions destination gives you Hightouch's powerful sync engine, so you continue to benefit from the security and observability features available in our native SaaS destinations.
Example use cases include:
- Syncing data to web APIs not yet natively supported
- Enriching data using external sources like Clearbit and ZoomInfo
- Transforming and filtering data using code instead of SQL
## Getting started
### Connect to your Google Cloud Functions
When setting up the Google Cloud Functions destination for the first time, you need to fill out the following:
- [**GCP credentials**](/security/gcp) for authenticating you function invocations.
- [**ProjectID**](https://cloud.google.com/resource-manager/docs/creating-managing-projects) of the GCP project where your function lives.
- [**Region**](https://cloud.google.com/resource-manager/docs/creating-managing-projects) for the location of your function.
- The **function name** of the function you want to invoke when sync runs.
Hightouch recommends that you follow [the principle of least privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege) when providing permissions with your service accounts.
This destination only requires a `cloudfunctions.invoker` role for the resource pointing to the function.
## Syncing data
Once you've completed setup for the Google Cloud Functions destination in your Hightouch workspace, the next step is to configure a sync that invokes your function whenever rows are added, changed, or removed in your model.
### Choose your function triggers
Hightouch monitors your data model for added, changed, and/or removed rows. In this step, you specify which of these events should invoke your function.
Suppose you want to use Google Cloud Functions to send a confirmation email whenever a customer places a new order or updates an existing order. Your model might look something like `SELECT * FROM orders`. The Google Cloud Functions should be triggered whenever rows are added—for example, when customers place new orders—or whenever rows change—when orders are updated. Therefore, you would want to enable the **Rows added** and **Rows changed** triggers. When invoking your function, Hightouch passes along metadata about *why* the function was invoked.
All function invocations are synchronous, meaning that Hightouch invokes your Google Cloud Function and waits for its response.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows, such as hydrating a CRM for the first time, may require performing a backfill of all rows during the initial sync. For other use cases, such as sending confirmation emails, you might only want to invoke your function in response to future data changes.
### Configure batch size
By default, Hightouch separately invokes your Google Cloud Function for each added, changed, or removed row.
Certain high-throughput use cases may require batching together multiple rows in the payload. Hightouch supports batches of up to 1000 rows per function invocation.
Batching isn't recommended unless absolutely necessary for performance reasons. Enabling this feature requires your Google Cloud Function to be either idempotent or significantly more fault tolerant in the event of a partial batch failure.
### Configure rate and concurrency limits
In this step, you declare whether function invocations should be throttled.
To determine the ideal limits, you should consider whether your function interacts with any downstream services that have limits of their own.
Most modern web APIs enforce rate limits, which set a maximum allowed number of requests per second, minute, or hour. Occasionally, APIs may also have concurrency limits, which set a maximum allowed number of requests that can be processed simultaneously. Rate limits and concurrency limits both affect overall sync speed.
### Configure error handling
Google Cloud Functions can fail for many reasons. Hightouch's retry logic for this destination doesn't discriminate between invocation errors, for example, missing permissions, runtime errors, for example, due to syntax issues, or function errors, for example, due to uncaught exceptions. Hightouch retries all errors eventually.
In this step, you decide whether errors should be retried immediately or during the next sync run. If you choose to retry immediately, you can specify how many retries should be attempted during the sync run. If all these retries fail, Hightouch retires the request during subsequent sync runs until it succeeds.
If an error is gracefully caught in your function, you can flag it as a rejected row and provide a custom error message using the response format specified in the [error handling section](#configure-error-handling).
You can also include retry logic inside of your Google Cloud Function. After exhausting all retry attempts during a function invocation, respond with an error so that Hightouch knows to reinvoke the function later.
### Invocation payload schema
The payload, also known as the "event document," contains your row data in JSON format, along with additional metadata. When writing your Google Cloud Function, you can assume that all function invocations include a payload following this schema:
```json
{
"operation": XXXXX,
"primary_key_column": XXXXX,
"rows": [
{
XXXXXX
}
],
"metadata": {
"api_version": XXXXX,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
#### Invocation payload properties
##### `operation`
The row operation, also known as the trigger, explains *why* your function was invoked.
Its possible values are these strings: `"add"`, `"change"`, and `"remove"`.
##### `primary_key_column`
This string refers to the column name of the primary key for your Hightouch model.
##### `rows`
This is an array of objects, with each object containing row data from your Hightouch model.
##### `metadata`
This is an object containing three key/value pairs:
- `api_version` (integer): currently set to `1`, this identifier only changes if the payload schema is modified in the future
- `sync_id` (integer): this is the unique identifier for the Hightouch sync configuration associated with the function invocation
- `sync_run_id` (integer): this is the unique identifier for the specific sync run associated with the function invocation
#### Example invocation payload
Suppose a new row is added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `john.doe@example.com` | John | Doe |
The payload for the function invocation would look like this:
```json
{
"operation": "add",
"primary_key_column": "customer_id",
"rows": [
{
"customer_id": "928713",
"email": "john.doe@example.com",
"first_name": "John",
"last_name": "Doe"
}
],
"metadata": {
"api_version": 1,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
Note that `rows` is an **array** containing a single element.
#### Example invocation payload for batching
Suppose three new rows are added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `alice.doe@example.com` | Alice | Doe |
| 283743 | `bob.doe@example.com` | Bob | Doe |
| 162352 | `carol.doe@example.com` | Carol | Doe |
The payload for the function invocation would look like this:
```json
{
"operation": "add",
"primary_key_column": "customer_id",
"rows": [
{
"customer_id": "928713",
"email": "alice.doe@example.com",
"first_name": "Alice",
"last_name": "Doe"
},
{
"customer_id": "283743",
"email": "bob.doe@example.com",
"first_name": "Bob",
"last_name": "Doe"
},
{
"customer_id": "162352",
"email": "carol.doe@example.com",
"first_name": "Carol",
"last_name": "Doe"
}
],
"metadata": {
"api_version": 1,
"sync_id": XXXXX,
"sync_run_id": XXXXX
}
}
```
When batching is enabled, rows sharing a common operation type—`add`, `change`, `remove`—are batched together. Function invocations always represent exactly one operation type.
### Response payload schema
If your Google Cloud Function fails to process any rows, it can respond with an array of "rejected rows" that encountered errors. Each rejected row should be identified by its primary key/value and may be associated with an optional error message. Hightouch retries these rows.
If there is an error, the response payload looks like this:
```json
{
"errors": [
{
"primary_key_value": XXXXX,
"reason": XXXXX
}
]
}
```
#### Response payload properties
Errors in the `errors` array have these properties:
##### `primary_key_value`
This identifies the rejected row that needs to be retried.
##### `reason` (optional)
This string may represent an error message or any other information that would be helpful for debugging and monitoring sync health.
#### Example success response payload
If all rows are processed successfully, your function responds with an empty payload.
```json
{}
```
#### Example response payload for rejected rows
Suppose three new rows are added to your model:
| `customer_id` | `email_address` | `first_name` | `last_name`|
|---------------|---------------------------|--------------|------------|
| 928713 | `alice.doe@example.com` | Alice | Doe |
| 283743 | `bob.doe@example.com` | Bob | Doe |
| 162352 | `carol.doe@example.com` | Carol | Doe |
While processing these rows, your Google Cloud Function encounters two errors:
- Alice's email can't be found in the downstream service
- Bob's email already exists in the downstream service
To surface these errors in Hightouch, your function should respond like this:
```json
{
"errors": [
{
"primary_key_value": "928713",
"reason": "Email not found"
},
{
"primary_key_value": "283743",
"reason": "Email already exists"
}
]
}
```
The primary key values (`928713` and `283743`) refer to `customer_id` values. The `customer_id` column was designated the `primary_key_column` in the function invocation payload.
#### Example response payload for other errors
If your function experiences an invocation error, for example, exceeding Google Cloud Functions rate limits, runtime error, for example, due to syntax issues, or function error, for example due to uncaught exceptions, all rows in the batch are automatically marked as rejected rows.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Google Data Manager API
**URL:** https://hightouch.com/docs/destinations/google-data-manager
**Description:** Fuel AI-powered personalization and support advanced measurement use-cases by connecting first-party audience and conversion data to Google Ads, Google Analytics, and Google Marketing Platform through a single unified API.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| **Google Ads Audiences** | Sync records as audience members in Google Data Manager via the Data Manager API | Add | [Audience Members](https://developers.google.com/data-manager/api/get-started/quickstart/send-audience-members) |
| **Display & Video 360 Audiences** | Create and update first and third party audiences in Google Display & Video 360 | Add | [Display & Video 360 Audiences](https://developers.google.com/display-video/api/guides/audiences/overview) |
| **Google Ads Conversion Events** | Send Online and Offline events to Google Data Manager via the Data Manager API | Insert | [Events](https://developers.google.com/data-manager/api/get-started/quickstart/send-events) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Google Data Manager
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button.
Select **Google Data Manager** and click **Continue**. Select whether you'd like to send data to Google Ads or Display & Video 360. You can then authenticate Hightouch to **Google Data Manager** via OAuth.
Click **Log in to Google Data Manager**, log in with your Google Account, and click **Allow**.
You should then see a success notification in the Hightouch UI.
Enter the Google Ads or Display & Video 360 account ID you'd like to use with this destination. You can find this value in Google, at the top-right corner with your account information. This should be a 10 digit value. Remove any dashes that may be present in your account ID.
## Sync configuration
Once you've set up your Google Data Manager destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data.
Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin.
Then, select the relevant model and the Google Data Manager destination you want to sync to.
### Syncing Google Ads audience members
Hightouch lets you create and maintain customer match lists via the Google Data Manager API.
It can take 6 to 48 hours for a list to be populated with members once Hightouch has sent the data to Google.
#### User list type
To begin, select which type of Customer Match User List you'd like to create or sync to. The available options are:
- **CONTACT_INFO**
- **MOBILE_ADVERTISING_ID**
#### Select an existing list or create a new one
You can create a new [Customer match user list](https://support.google.com/google-ads/answer/6276125) or use an existing one.
When creating a new list, you can optionally enter a name.
Otherwise, Hightouch defaults to the name of the associated model.
To use an existing list, enter the list's ID in the following format: `products/{product}/customers/{customer_id}/userLists/{user_list_id}`. Example: `products/GOOGLE_ADS/customers/12345678/userLists/12345678`
If you choose to create a new list, you can't update the
[account](#select-account) to which you're syncing data after the initial sync
run. If you need to change the account, configure a new sync.
#### User identifiers
Select which columns in your model you'd like to send to Google Ads as user identifiers.
For user lists with the Contact Info type, you must select *either* hashed email addresses or hashed phone numbers. If you choose to sync addresses, all fields of the address are required.
For user lists with the Mobile Advertising ID type, the mobile advertising ID is required.
#### Handling PII and hashing
You can enable PII hashing if you wish for Hightouch to detect whether your data requires normalization and hashing based on the chosen record mapping.
For example, if you select **Email** and the model column with the email data is unhashed, Hightouch hashes the value for you.
By default, Hightouch automatically detects if the following fields require hashing:
- **Email**
- **Phone**
- **Given name**
- **Family name**
#### EU user consent policy
If you are uploading data for users in the European Economic Area ([EEA]()),
you need to ensure you are passing the required consent signals to [Google](https://www.google.com/about/company/user-consent-policy/?sjid=2196795394224792925-NA).
For each sync, you can specify the consent status for these types:
- `ad_user_data`: consent for sending user data to Google for advertising purposes
- `ad_personalization`: consent for ad personalization
Check out Google's support [doc](https://support.google.com/google-ads/answer/14310715?hl=en) to learn more.
### Syncing Display & Video 360 audiences
Hightouch allows you to create and update first and third party audiences in Google Display & Video 360 as customer match user lists. These lists support adding members using mobile device ID or an array of email, phone numbers, and physical addresses.
When creating the audience, make sure the source of the data is properly selected between first or third party sources.
#### User identifiers
Mobile device IDs and other user identifiers cannot be mixed. Also, if providing the user's physical address, the zip code, country code, first and last name must be provided.
Phone numbers must be in [E.164](https://en.wikipedia.org/wiki/E.164) format.
#### User consent
Starting in March 2024, Display & Video 360 introduced the [consent object](https://developers.google.com/display-video/api/guides/audiences/upload-customer-match?sjid=10202302840984618943-NA#set_user_consent) for uploading consent with customer match data. The consent object specifies two distinct types of consent and users must grant both types of consent to use their data for customer match lists. Reference the [DV360 docs](https://support.google.com/displayvideo/answer/14673239?hl=en) for more information.
If you don't set the consent status or if the status is denied, the sync results in an error.
#### PII data and hashing
Hightouch automatically hashes all the PII data before it's sent to Google. If the data is already hashed, this option can be disabled. Reference the [DV360 docs](https://developers.google.com/display-video/api/reference/rest/v1/firstAndThirdPartyAudiences#ContactInfo) for more information.
Google's Personalized Ads Policy, Customer Match Policy and Platforms program policies include requirements for advertisers to show sufficiently established Google Ads or Display & Video 360 account history before accessing certain features. To enable uploading data to DV360, please fill [this form](https://support.google.com/displayvideo/answer/9176010) before you create the sync.
### Syncing Google Ads events
Hightouch supports syncing:
- Online events: Send event data as an additional data source for your tag conversions, to maximize ad interaction signals and strengthen your data and overall performance.
- Offline events: Send event data for [offline conversions](https://support.google.com/google-ads/answer/2998031) or [enhanced conversions for leads](https://support.google.com/google-ads/answer/15713840).
When syncing conversion events, Hightouch treats any records added to [your source](/getting-started/concepts) as new events and sends them to Google Data Manager when your sync runs.
#### Field mapping
Select which columns in your model you'd like to sync as attributes of conversion events.
Google Data Manager requires the following fields for each event:
- Event timestamp (event_timestamp): The time the event occurred.
- Transaction ID (transaction_id): The unique identifier for the event.
- Event source (event_source): The source of the event. Required for offline events. Optional for online events. If specified for an online event, must be WEB.
- Ad identifiers (GCLID, GBRAID, or WBRAID) or user data (email, phone, or address): The event must have either an ad identifier or user data. Send both if you have both for the event.
The accepted values for `event_source` are:
- `EVENT_SOURCE_UNSPECIFIED`
- `WEB`
- `APP`
- `IN_STORE`
- `PHONE`
- `OTHER`
Additional user properties mappings include:
- Customer type
- Accepted values: `CUSTOMER_TYPE_UNSPECIFIED`, `NEW`, `RETURNING`, `REENGAGED`
- Customer value bucket
- Accepted values: `CUSTOMER_VALUE_BUCKET_UNSPECIFIED`, `LOW`, `MEDIUM`, `HIGH`
#### Landing page device info (recommended for online events)
For online events, we recommend collecting and sending information about the device being used (if any) when the event happened. This will improve the likelihood that conversion events can be accurately attributed back to ad interactions, resulting in more accurate reporting and better performance as Google's AI systems learn and improve from each attributed conversion.
To map device information to your conversion events, select `Landing page device info` and map the following fields:
- `userAgent` - user-agent string of the device for the given context.
- `ipAddress` - The IP address of the device for the given context.
Google Ads does **not** support IP address matching for end users in the European Economic Area (EEA), United Kingdom (UK), or Switzerland (CH). We recommend adding logic directly in your model to conditionally exclude sharing IP addresses from users from these regions and ensure that you provide users with clear and comprehensive information about the data you collect on your sites, apps, and other properties and get consent where required by law or any applicable Google policies. See Google's [**About offline conversion imports**](https://support.google.com/google-ads/answer/2998031) page for more details.
#### EU user consent policy
If you are uploading data for users in the European Economic Area ([EEA]()),
you need to ensure you are passing the required consent signals to [Google](https://www.google.com/about/company/user-consent-policy/?sjid=2196795394224792925-NA).
For [click conversions](#syncing-click-conversions), [call conversions](#syncing-call-conversions) and [offline store conversions](#syncing-offline-store-conversions), you can specify the Google consent status for advertising purposes, by mapping these fields in the sync configuration:
- `Ad user data consent` (for the `ad_user_data` property)
- `Ad personalization consent` (for the `ad_personalization` property)
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Google Drive
**URL:** https://hightouch.com/docs/destinations/google-drive
**Description:** Transfer data at scale from your warehouse to Google Drive
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
| ------------ | ------------------------------------------------------------ | -------------------- |
| Any data set | Sync data from a source to Google Drive as JSON or CSV files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Diff mode creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Google Drive
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Google Drive** and click **Continue**. You can then connect to Google Drive via OAuth.
After authorizing, you will have successfully connected Hightouch to Google Drive.
## Sync configuration
Once you've set up your Google Drive destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Google Drive destination you want to sync to.
### Select file format
Hightouch supports syncing JSON and CSV files to Google Drive.
### Enter folder
Enter the URL of the folder in Google Drive you want to sync to. You can find a folder's URL k by right-clicking on it and then selecting **Get Link**.
### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it. To view different versions of the same file, you can right-click the file in Google Drive and then click **Manage versions**.
If you are using an audience and would like to include the audience name, you will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter '-86400' (`24 hours * 60 minutes * 60 seconds`).
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `company`, `email`, `name`, and `primaryKey` fields. Hightouch exports these fields to new fields in the file and ignores all other columns from your results.
### CSV options
If you're syncing to a CSV file, you have additional configuration options:
- **Delimiter**: Your options are comma (`,`), semicolon (`;`), pipe (`|`), tilde (`~`), and tab
- (Optional) Whether to **include a CSV header row** in the exported files
- (Optional) Whether to **include a [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark)** in the exported files, the BOM is ``
### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
## Tips and troubleshooting
### Common errors
### Sync alerts
---
## Google Cloud Pub/Sub
**URL:** https://hightouch.com/docs/destinations/google-pubsub
**Description:** Connect to your internal microservices via a message queue
**Section:** Destinations
## Overview
Hightouch integrates directly with Google Cloud Pub/Sub to support high-throughput, distributed, or asynchronous workloads, letting you build a custom connector to your internal systems.
## Getting started
### Connect to your Google Cloud account
When setting up the Google Cloud Pub/Sub destination for the first time, you need to enter your **Google Cloud Credentials** to give Hightouch access to your Google Cloud account. Hightouch needs permission to send messages to Google Cloud Pub/Sub on your behalf. Consult [our GCP documentation](/security/gcp) for guidance on how to create a Google Cloud credential.
### Project details
First, you need to enter your **GCP Project ID** and **[Regional API Endpoint](https://cloud.google.com/pubsub/docs/reference/service_apis_overview#list_of_regional_endpoints)**.
### Grant permissions
Hightouch believes in the principle of least privilege. We ask for no more permissions than necessary. For the Google Cloud Pub/Sub destination, Hightouch requires `roles/pubsub.publisher` and `roles/pubsub.viewer`. Ensure you run the command in your Google Cloud Console to grant Hightouch's service account the correct permissions.
## Syncing data
Once you've authenticated your Google Cloud account in Hightouch, entered project details, and granted the correct permissions, you can configure a sync that sends messages whenever rows are added, changed, or removed in your model.
### Configure your events trigger
Hightouch monitors your data model for added, changed, and removed rows. In this step, you specify which of these events should trigger message publishing.
### Choose your topic
In this step, you choose which topics to publish the messages to. Hightouch allows you to sync to existing topics that are already in your Google Cloud Pub/Sub.
### Customize your message
In this step, you tell Hightouch how to build the JSON message data object using data from your model.
This destination offers three methods of composing a JSON object:
- [Using a JSON editor](#use-json-editor)
- [Using one column from a model](#use-one-column-from-model)
- [Using multiple columns from a model](#use-multiple-columns-from-model)
#### Use JSON editor
With the JSON editor, you can compose any JSON object using the [Liquid template language](https://shopify.github.io/liquid/). This is particularly useful for complex message data bodies containing nested objects and arrays, which can sometimes be difficult to model entirely in SQL.
Suppose your data model looks like this:
| full_name | age | email_address | phone_number |
| --------- | --- | ------------------ | ------------ |
| John Doe | 30 | `john@example.com` | +14158675309 |
And you want your message data like this:
```json
{
"name": "John Doe",
"age": 30,
"contact_info": [
{
"type": "email",
"value": "john@example.com"
},
{
"type": "phone",
"value": "+14158675309"
}
]
}
```
Your Liquid template should look like this:
```txt
{
"name": "{{row.full_name}}",
"age": {{row.age}},
"contact_info": [
{
"type": "email",
"value": "{{row.email_address}}"
},
{
"type": "phone",
"value": "{{row.phone_number}}"
}
]
}
```
This makes it so you can reference any column using the syntax `{{row.column_name}}`. You can also use advanced Liquid features to incorporate control flow and loops into your dynamic message data.
When injecting strings into your JSON object, be sure to surround the Liquid
tag in double quotes.
#### Use one column from model
If you're already storing JSON data in your source, or if you have the ability to construct a JSON object using SQL, you can select one column in your model that already contains the full message data.
This setting is commonly used when syncing web events that have already been collected and stored as JSON objects in your database.
#### Use multiple columns from model
For the simplest use cases, Hightouch can construct a JSON object with key/value pairs based on multiple columns in your model.
Suppose your model looks like this:
| email | first_name | last_name |
| ----------------------- | ---------- | --------- |
| `alice.doe@example.com` | Alice | Doe |
| `bob.doe@example.com` | Bob | Doe |
| `carol.doe@example.com` | Carol | Doe |
The field mapping in the screenshot above would generate the following message data for the first row:
```json
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "alice.doe@example.com"
}
```
You can use the field mapper to rename fields. For example, `first_name` can
be mapped to `customer_first_name`.
### Configure optional message properties
Along with your row data in JSON format, you can also optionally include an ordering key to configure the order your topic receives your message and metadata fields as additional attributes.
##### `orderingKey`
This is a string field that ensure subscribers receive messages [in a specific order](https://cloud.google.com/pubsub/docs/ordering). Hightouch automatically tries to cast the value to a string. If we can't cast the value to a string then it is sent as `null`.
##### `attributes`
This is an object containing key/value pairs of custom mapping fields. Values can be text or byte strings. Attributes can be text strings or byte strings. You can have at most 100 attributes per message. Attribute keys should not start with `goog` and should not exceed 256 bytes.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows may require performing a backfill of all rows during the initial sync. For other use cases, you might only want to send messages in response to future data changes.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Google Retail
**URL:** https://hightouch.com/docs/destinations/google-retail
**Description:** Ensure all of your Google Retail listings match your inventory
**Section:** Destinations
## Supported content
| Type | Description | Supported Sync Modes |
|--------------|--------------------------------------------------|----------------------|
| **Products** | Sync records from your model as product listings | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Google Retail
You need to have access to a [merchant account](https://www.google.com/retail/) to sync listings.
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Google Retail** and click **Continue**. You can then authenticate Hightouch to **Google Retail** via OAuth.
For the **Authentication method**, select **Log in to Google Retail** and log into your Google Retail account. Once successful, you will be redirected back to Hightouch to enter your Merchant ID and complete setup.
## Sync configuration
Once you've set up your Google Retail destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Google Retail destination you want to sync to.
### Product category
Each sync must have a product category. This is the category that the products will be listed under in Google Retail.
The product category option can also be specified from a column in your model if your model contains products across categories.
### Field mapping
Field mapping is where you select which customer information parameters or other event parameters you want to send to the Content API.
You can map data from any of your model columns to default and custom Google fields.
Go to the [Product Specification](https://support.google.com/merchants/answer/7052112?visit_id=638422392791129370-3944862796&rd=1) to see the mappings available for all Google Retail products.
#### Vehicle mapping
Cars, Trucks & Vans (916) have an additional collection of fields available to map data to. Read more about them [here](https://support.google.com/merchants/answer/11190480?sjid=6359241266768509981-EU).
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Google Search Ads 360
**URL:** https://hightouch.com/docs/destinations/google-search-ads-360
**Description:** Search Ads 360 helps you respond to an ever-changing market in real time and at scale.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|-----------------|----------------------------------------------------------------|----------------------|---------------------------------------------------------------------------------------------|
| **Conversions** | Sync data from any source to Google Search Ads 360 conversions | Insert | [Conversions docs](https://developers.google.com/search-ads/v2/reference/conversion/insert) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Google Search Ads 360
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Google Search Ads 360** and click **Continue**. You can then authenticate Hightouch to **Google Search Ads 360**.
For the **Authentication method**, select **Log in to Google Search Ads 360** and log into your Google Search Ads 360 account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Google Search Ads 360 destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Google Search Ads 360 destination you want to sync to.
### Syncing conversions
Sync data from any source to Google Search Ads 360 conversions.
#### Record matching
You can match rows from your model to conversion in Google Search Ads 360 on any column in your model and any field in Google Search Ads 360. Ensure the data types of the model column and Google Search Ads 360 field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
The `segmentationType` property is sent with all conversions with the value 'FLOODLIGHT' which is the only acceptable value for this field.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Google Sheets - User Account
**URL:** https://hightouch.com/docs/destinations/google-sheets
**Description:** Sync warehouse data to spreadsheets used by sales, ops, and marketing teams
**Section:** Destinations
## Supported syncing
| Supported Sync Modes | Description |
|----------------------|----------------------------------------------------------------------------------------|
| **Mirror** | Sync the full result set of the model to the selected destination sheet |
| **Snapshot** | Create a new sheet on every sync with the full result set of the model |
| **Append** | Append the result set of a model to the selected sheet and add rows as the model grows |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Google Sheets
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button.
Hightouch provides two Google Sheets options:
- **Google Sheets - User Account**: Use this option if you want to authenticate access to Hightouch with a particular user's login. This gives Hightouch access to any Google Sheet the user has Editor access to. This option uses OAuth for authentication.
- **Google Sheets - Service account (Recommended)**: This option provides fine-grained control over access to your data.
Using a service account is best practice for a few reasons:
- You can grant service accounts access to specific sheets. This means that only the necessary permissions are granted to the service account, instead of allowing access to all files belonging to a user. This reduces the risk of unauthorized access to sensitive data or functionality.
- Service accounts are designed to represent applications or services, not individuals. You don't have to worry about disruptions to your syncs if the user who authenticated has their permissions changed or if they leave the organization.
### Authenticate with a user account
Select the **Google Sheets - User Account** destination option and click **Continue**.
Then select to **Log in to Google Sheets**, select your Google account, and click **Allow** to the requested permissions.
Hightouch uses the same OAuth mechanism for all Google destinations.
If you've set up other Google destinations, such as Google Ads, Campaign Manager, or Display & Video 360, they appear here.
After authorizing, Hightouch connects to Google Sheets.
Click **Continue** and complete setup by giving the destination a descriptive name.
### Authenticate with a service account
Select the **Google Sheets \(Service Account\)** destination option and click **Continue**.
Then, click **Generate a service account**.
Hightouch then generates a **Google Service Account Email**.
Click **Continue** and complete setup by giving the destination a descriptive name.
To give Hightouch access to a particular spreadsheet, share the spreadsheet with the service account email with **Editor** permissions.
You can always access the service account email from the destination's overview page, should you need to share additional sheets.
## Sync configuration
Once you've set up your Google Sheets destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Google Sheets destination you want to sync to.
The Google Sheets destination allows you to **Mirror** your data to a spreadsheet or create a new sheet on every sync with **Snapshot** mode.
You can also add data to an existing sheet with **Append** mode. In this mode, rows added to your model are added to the sheet. Rows changed and removed in your model are ignored. This may result in faster syncs.
**Mirror** mode replaces the entire selected sheet's contents with the rows from your model.
Ensure that your sheet doesn't contain data that shouldn't be overwritten if you select this mode.
### Sheet selection
You can choose to select the Google Sheets file you want to update either by its **name** using the sheets picker (recommended) or inputting the Google Sheet's **ID**.
You can find your spreadsheet's ID in its URL: `https://docs.google.com/spreadsheets/d//edit#gid=0`
If you don't see your spreadsheet's name in the dropdown, make sure you authorized the right account or invited the Hightouch service account email to your spreadsheet,
depending on your [setup mode](#connect-to-google-sheets).
If you updated a sheet's permissions, click the refresh button to see newly authorized sheets.
Then, select the sheet within the spreadsheet that you would like to sync to.
Colons (`:`) in spreadsheet and sheet titles can cause errors. Ensure your spreadsheet and sheet titles don't include any.
Avoid setting up multiple syncs to the same sheet, as this can corrupt your data.
### Data format
Google Sheets accepts inputted data in two ways:
- **Raw** Won't parse the values the user has entered and stores values as-is. Keep in mind that Hightouch does some data casting and may send numbers and dates as strings. This process can add a leading apostrophe to values in Google Sheets cells.
- **User entered** Parses values as if the user typed them into the UI. Numbers stay as numbers, but strings may convert to numbers, dates, etc. following the same rules that apply when entering text into a cell via the Google Sheets UI.
If you want to sync numbers, dates, or other non-string values in their original data type and not converted to strings, use **User entered** mode.
If you are inserting null values using the **Raw** data format,
these won't register as blank fields though they may appear to be.
Keep this in mind if you are using formulas that are looking for null values.
If any of your formulas rely on blank fields, it's best to use **User entered** as the input mode.
For more information, read the [Google documentation](https://developers.google.com/sheets/api/reference/rest/v4/ValueInputOption).
### Enable new sheet creation
Google Sheets has a limit of 10 million cells in a sheet.
If your sync contains more data than this, you can enable Hightouch to create new sheets with the additional data.
For each additional 10 million cells, Hightouch creates a new sheet with the original sheet's name suffixed with an underscore and a number.
For example, "sheetname_1" for the first additional 10 million cells, "sheetname_2" for the second 10 million cells, etc.
Ensure these names don't conflict with existing sheets' names to avoid them being overwritten.
To use this configuration option, ensure the [user you authenticated with](#connect-to-google-sheets) has access to the spreadsheet's folder, not just the spreadsheet itself.
Hightouch released enabling new sheet creation on June 22, 2023.
If you set up your Google Sheets destination before this date and want to use this option, you need to [reauthorize your connection](#connect-to-google-sheets).
### Data range selection
Hightouch supports selecting a manual range in your Google Sheets to write data to.
This allows you to preserve formulas, notes, or other data in your spreadsheets' margins.
To use a manual range enter a range into the **Custom Cell Range** input in the format `A1:Z1000`.
If you leave this field blank, Hightouch mirrors data to the entire spreadsheet, dependent on your model's query results.
## Tips and troubleshooting
### Rate limits
You may receive an error from Google if you exceed your quota for write requests during a sync: `Quota exceeded for quota metric`.
For more information, read the Google [usage limit documentation](https://developers.google.com/sheets/api/limits).
To avoid this error:
- stagger or schedule your syncs to Google Sheets to [run in sequence](/syncs/schedule-sync-with-sequences) to avoid simultaneous write requests between syncs
- set up multiple Google Sheets destinations in your workspace so as to reduce the number of syncs that use the same destination
- set up [Google Sheets (Service Account)](#authenticate-with-a-service-account) destinations rather than [authenticating with a user account](#authenticate-with-a-user-account)
- reach out to Google to request a [quota increase](https://developers.google.com/sheets/api/limits#increase)
### Find sheets originating from an Excel file upload
You may have trouble selecting that Google Sheet that was originally uploaded to Google as an Excel file.
If you are the owner of the file, but can't find the spreadsheet in the [sheet selection dropdown](#sheet-selection), confirm that it's not an `.XLS` or `.XLSX` file.
You can see the file type of a file next to its name in the Google Sheets UI.
Hightouch can only update Google Sheets files, not `.XLS` or `.XLSX` files. To convert a file to a Google Sheets file, go to **File > Save as Google Sheets**.
After converting the file to a Google sheet, it becomes available in the dropdown selector.
You may need to click the refresh button next to the dropdown selector for spreadsheets after you've converted a sheet, or updated its access.
If you want to work with Excel files, consider using the [Microsoft Excel destination](/destinations/microsoft-excel).
### Common errors
#### Action would increase the number of cells in the workbook above the limit of 10,000,000 cells
Make sure to enable Hightouch to [create new sheets](#enable-new-sheet-creation) if your data surpasses the 10 million cell limit.
#### Cannot read properties of null (reading 'properties')
This error indicates that the sheet within the spreadsheet selected in the sync configuration has either been renamed or deleted in Google Sheets. [Select a new sheet](#sheet-selection) from the dropdown menu to restore your sync.
#### Internal error encountered
This error may occur when the spreadsheet you're syncing to contains sheets with charts or pivot tables.
Try moving the data you would like to update with Hightouch to a separate spreadsheet.
Then reference this data's range using a formula like [`IMPORTRANGE`](https://support.google.com/docs/answer/3093340?hl=en) in the spreadsheet with your charts and analysis.
This way, the sheet that Hightouch updates won't have the timeout issue, and you can still see the data on the sheet that contains charts or pivot tables.
#### Invalid values [1][2]: list_value
The Google Sheets destination doesn't support syncing object or array type columns.
To resolve the error, change the type of any object or array type columns to **String** [in your model configuration](/models/data-types-casting#casting-from-your-model-configuration).
#### Range \{...\} exceeds grid limits
This error indicates that the [selected sheet](#sheet-selection) or a data range in that sheet is [protected](https://support.google.com/docs/answer/1218656).
Remove the protection from the sheet or range to enable syncing your data.
You can read more about this in [Google's documentation](https://support.google.com/docs/answer/1218656).
#### Requested writing within range ['Example:Sheet'!A1], but tried writing to column [B]
This error happens if there is a colon in the name of the sheet that you're syncing to.
Google Sheets is unable to parse names that contain colons.
Replace colons in the name of the sheet with an underscore or remove the colon.
#### The caller does not have permission
This error happens when the sheet you are syncing to is private.
To correct this, ensure you're using a sheet that is shared with the [user or service account you used set up your Google Sheets destination](#connect-to-google-sheets).
#### Unable to parse range
This error indicates that the sheet selected in the sync configuration has either been renamed or deleted. You can fix this by selecting an existing sheet in the [sheet section](#sheet-selection).
#### Permission Errors using a sheet ID
If you encounter permission errors writing to a Google sheet when using it's ID, selecting it first in the sheets picker and then using it's ID will resolve this.
### Sync alerts
---
## Great Question
**URL:** https://hightouch.com/docs/destinations/great-question
**Description:** Maintain your candidates list to reach the right audience for your Great Question studies.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
---------------|--------------------------------------------------------|----------------------|-------------------------------------------------------------------------
**Candidates** | Sync data from any source to Great Question candidates | Upsert | [Candidates docs](https://developer.greatquestion.co/reference/overview)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Great Question
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Great Question** and click **Continue**. You can then authenticate Hightouch to **Great Question**.
Enter the following fields into Hightouch:
- **Base URL**: If left empty, the default base URL (`https://api.greatquestion.co/`) will be used.
- **API key** (Required): Refer to the [Great Question docs](https://developer.greatquestion.co/reference/authentication-1) to generate your API key.
## Sync configuration
Once you've set up your Great Question destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Great Question destination you want to sync to.
### Syncing candidates
Sync data from any source to Great Question candidates.
#### Record matching
You can match rows from your model to candidates in Great Question on any column in your model and any field in Great Question. Ensure the data types of the model column and Great Question field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync candidate fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom candidate fields.
If you send data for a custom field that doesn't exist, Hightouch adds the field.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|----------------------------------------------------------------
**Do nothing** | Keep the candidate in Great Question with all its synced fields
**Delete** | Delete the synced candidates from Great Question
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Heap
**URL:** https://hightouch.com/docs/destinations/heap
**Description:** Sync your most recently captured event data to Heap to both measure and dramatically improve your digital experience
**Section:** Destinations
## Setup
Upon selecting Heap as your destination you will be prompted to
input your Heap Environment ID, which correspond to the specific project environment
you wish to sync your data to.
You can locate the appropriate Environment ID within your Heap account by navigating to
_Account > Manage > Projects_ and clicking on your desired project:
## Heap sync types
Hightouch supports syncing the following Heap sync types:
- `Events`
- `Objects`:
- `User Properties`
- `Account Properties`
## Syncing objects
### Object sync modes
- **Upsert**: pushes new objects to Heap _and_ updates fields that change in your warehouse.
### Record matching
Records can be matched from your source to your Userflow workspace by the `identity` (typically a unique user property)
or `account_id`, depending on the object you are syncing.
### A note on syncing account properties
To sync account properties you must first have users synced into your Heap workspace.
The reason for this is that to sync account properties you must first configure Account IDs.
This may be done by navigating to _Account > Manage > Features_ and reading the following:
> _"To analyze accounts in Heap, you need to specify a user property that can be used to group users into accounts (for example, Account ID)._"
## Syncing events
When syncing events, Hightouch treats any records added to [your source](/getting-started/concepts) as new events and sends them to Heap when your sync runs.
### Event sync modes
- **Insert**: _only_ pushes new events to Heap.
### Event name
To specify the name of the event you wish to sync to Heap you have the option of either
inputting the name of a particular event or using a column from your model.
### Timestamp
You may also specify the timestamp of the event in your source. However, be aware that this value
may be overridden by its respective mapping and is completely optional. If you choose not to explicitly specify
a timestamp then Heap will use the time the event arrives to the server.
### Identity
The identity of an event is a required field and is typically a unique user property. If the specified identity
doesn't exist, a new user will be created with that identity in Heap.
## Field mapping (events and objects)
You may also sync columns from your source to Heap as event, user, or account `properties`.
Any property previously created will be updated and any new properties will be created.
---
## Help Scout
**URL:** https://hightouch.com/docs/destinations/help-scout
**Description:** Help Scout is a dedicated customer support platform built for growing teams to deliver best-in-class customer service.
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes | API Reference |
|--|--|--|--|
|**Customers**|Sync data from any source to Help Scout customers|Insert, Upsert, Update| [Customer docs](https://developer.helpscout.com/mailbox-api/endpoints/customers/list/)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Help Scout
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Help Scout** and click **Continue**. You can then authenticate Hightouch to Help Scout via OAuth.
Click **Log in to Help Scout**, log in to your Help Scout account, and then click **Authorize**.
Once you've finished, Hightouch displays an **Authorization successful** message. To finish connecting, give your Help Scout destination a descriptive name.
## Sync configuration
Once you've connected your Help Scout destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Help Scout destination you want to sync to.
### Syncing customers
Hightouch supports creating new customers and updating existing ones.
#### Record matching
To match rows from your model to customers in Help Scout, you need to select a model column and corresponding Help Scout field. You can match on any of the following Help Scout fields:
- Email
- Customer ID (only with update mode)
#### Field mapping
You can sync any model columns to both native and custom Help Scout customer fields.
In insert mode, you can add new custom fields to the customer. Update and upsert mode only allow replacing existing fields on the customer. In other words, the fields you sync to must already exist in Help Scout.
For information on creating or editing customer fields, please refer to [Help Scout's docs](https://docs.helpscout.com/article/1385-customer-properties#create).
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| ----------------------------- | ----------------------------------------------------------------- |
| **Do nothing** | Keep the customer record in Help Scout with all its synced fields |
| **Delete Destination Record** | Delete the record from the Help Scout completely |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## HTTP Request
**URL:** https://hightouch.com/docs/destinations/http-request
**Description:** Connect to any internal system, third-party API, or other web service.
**Section:** Destinations
## Overview
We designed the HTTP Request destination to be as flexible as possible. We want our customers to build their own integrations with third-party APIs, internal tools, and other services not yet supported in our growing catalog of native destinations.
This destination empowers you to integrate Hightouch with almost any API without writing code. Using our visual interface, you can exercise granular control over HTTP endpoints, request payloads, rate limits, concurrency limits, error handling, and more.
Under the hood, the HTTP Request destination leverages Hightouch's powerful sync engine, so you continue to benefit from the security and observability features available in all our native destinations.
This article provides a technical overview of how the HTTP Request destination works. Skip to the [guided tutorial](#tutorial) showing how to create a custom integration with Algolia's REST API if you want to follow along with an example.
It's strictly prohibited to use the HTTP request destination to replicate
functionality already supported by a native destination.
## Setup
Hightouch queries your data source and monitors the query results for added rows, changed rows, and/or removed rows. When configuring the HTTP Request destination, you specify which of these events should trigger an HTTP request. For each trigger, you then specify an endpoint, payload, and other parameters.
To help Hightouch play nicely with other services, you can also specify constraints such as rate limits, concurrency limits, and error handling logic. You can configure all this in the Hightouch app—no code required.
Let's walk through the process to see how it works.
## Create a new destination
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **HTTP Request** and click **Continue**.
Next, the UI prompts you to enter a **base URL**, **HTTP headers**, and **certificate**. Only the base URL is required.
You'll configure specific endpoints, payloads, rate limits, and other logic during your [sync configuration](#sync-configuration).
That means you can create one HTTP destination per service and multiple sync configurations for different endpoints you want to make requests to.
### Add base URL
The **base URL** is the static part of the API endpoint you plan on making requests to.
For example, if you plan on using the [Algolia REST API](https://www.algolia.com/doc/rest-api/search/), the base URL would be `https://{Application-ID}-dsn.algolia.net`.
During [sync configuration](#sync-configuration), you provide additional path details and query parameters for the specific operations you want to perform.
### Add HTTP headers
You can optionally provide headers to include in each outbound HTTP request. These headers often include access tokens used for authentication.
To enter a header, enter its key name in the input. If you want a header property to be obscured in the UI and encrypted in storage, select the **Secret** checkbox. Click **Add key**, then enter the key's value in the input that appears to the right. Repeat this for as many headers as you want to add.
If you need to fetch a new token during each sync, consider using
[OAuth](#oauth) for authentication. You can also consider using one of our
serverless function destinations, for example, [AWS
Lambda](/destinations/aws-lambda).
### Add certificates
You can optionally add a custom CA, SSL, or TLS certificate.
Enable the relevant toggle and upload your certificate files.
Certificates can have a maximum size of 10 MB.
### OAuth
We also support OAuth authentication. OAuth implementations can vary between providers, so we recommend using a tool like [Postman](https://www.postman.com/) to test the token refresh process end-to-end. Once you've confirmed that you're able to fetch access tokens and authorize requests using [Postman](https://www.postman.com/), you can copy your [Postman](https://www.postman.com/) configuration into Hightouch verbatim.
Hightouch supports the following grant types:
- **[Authorization code](https://oauth.net/2/grant-types/authorization-code/)**
- **[Client credentials](https://oauth.net/2/grant-types/client-credentials/)**
- **[Password credentials](https://oauth.net/2/grant-types/password/)**
When setting up OAuth, you'll need to provide the following information:
- **Authorization header prefix (optional):** Specify the prefix for the Authorization header (for example, "Bearer").
- **Auth URL:** Required for authorization code grant type. The URL used to get the authorization code.
- **Access token URL:** The URL used to get an access token.
- **Refresh token URL (optional):** If different from `Access token URL`, provide the `Refresh token URL`.
- **Client ID:** The client identifier issued during the registration process.
- **Client Secret:** The client secret issued during the registration process.
- **Scopes (optional):** If your OAuth provider requires specific scopes.
- **Client authentication:** Choose to include the credentials as a header or in the request body.
To complete the OAuth setup, make sure to add the following `Callback URL` to your OAuth app: `https://api.hightouch.com/api/webhookV2/oauth/callback`.
#### Advanced configurations
During the OAuth flow, multiple requests are made to retrieve or refresh access tokens. Configure the following
options to include additional headers, URL parameters, or values in the request body.
- **Authorization request URL parameters**: URL parameters sent in the **Auth URL** for the authorization code grant type.
- **Token request**: Additional parameters and values sent with the request to the **Access token URL**.
- **Refresh request**: Additional parameters and values sent with the refresh token request to the **Refresh token URL** or
**Access token URL** if no refresh URL is provided.
## Sync configuration
Once you've created your HTTP Request destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the HTTP Request destination you previously set up.
### Choose request triggers
In this first step, you tell Hightouch when to trigger HTTP requests. You then configure each trigger separately. This lets your integration have different behaviors depending on whether a row is added, changed, and/or removed. Most APIs use different endpoints and HTTP methods for creating, updating, and deleting resources.
For example, suppose you're syncing customer data to an email marketing tool. You might want to create a contact when a row is added, update an existing contact when a row changes, and delete an existing contact when a row is removed. Having separate configuration for each trigger offers the flexibility required to support this use case.
A valid sync configuration requires at least one trigger.
The number of operations displayed on the [run details
page](/syncs/debugger#view-run-details) only counts the operations for the
request triggers you configured. For example, if you only configured the
**Rows added** trigger, the run details page doesn't display information
related to changed or removed rows.
### Batch configuration
You can send a single row or batch of multiple rows in each HTTP request.
To enable batching, select **A batch of multiple rows** in your sync configuration. The default batch size is 100, but you can enter a different value if needed.
When using the [JSON editor](#use-json-editor), make sure to correctly
[configure your payload](#batching-with-the-json-editor) for batch requests.
### Specify webhook endpoint
In this step, you tell Hightouch how to send HTTP requests to your internal system, third-party API, or other web service.
Hightouch supports `GET`, `POST`, `PUT`, `PATCH`, and `DELETE` requests to any endpoint.
When specifying the endpoint URL, you may either provide a static URL or a dynamic, templated URL.
Recall that you provided the [base URL](#add-base-url) in the initial destination setup, so you only need to provide the specific endpoint path.
If you provide a static URL, Hightouch uses that URL for all HTTP requests triggered by the sync. For example, Stripe's REST API uses a single, static URL for creating new customer objects: `POST https://api.stripe.com/v1/customers`.
If you provide a dynamic URL, Hightouch generates a different URL for each HTTP request by incorporating data from your model. For example, Stripe's REST API requires you to reference an ID when deleting specific objects. You could incorporate the `objectID` into the URL template like this: `DELETE https://api.stripe.com/v1/customers/{{customer_id}}`.
Hightouch uses the [Liquid templating language](https://shopify.github.io/liquid/) to generate dynamic content.
#### How to use Liquid templates
The [Liquid templating language](https://shopify.github.io/liquid/) supports variable injection, control flow, iteration, filters, and more. Details are available in the [Liquid docs](https://shopify.github.io/liquid/).
To insert a value from your model, use `{{row.column_name}}`, where `column_name` references any column in your model. For example, `{{row.first_name}}` or `{{row.email_address}}`.
To change the output of a Liquid object, you can append a filter with a single pipe (`|`):
`{{ row.column_name | filter }}`.
For example, let's say the value of the `first_name` column in a row is `"Alice"`. The Liquid template `{{ row.first_name | upcase }}` would generate `{{ "Alice" | upcase }}` → `ALICE`.
You can also chain multiple filters together like this: `{{ row.product_id | remove: "0" | prepend: "SKU_"}}` → `{{ "000123" | remove: "0" | prepend: "SKU_"}}` → `SKU_123`.
### Customize request payload
In this step, you tell Hightouch how to build the request payload using data from your model. Hightouch supports JSON and XML payloads, as well as URL encoded forms. You can also send an empty body, if needed.
Most modern web APIs use JSON payloads. This destination offers three methods of composing a JSON request body:
- [Using a JSON editor](#use-json-editor)
- [Using one column from a model](#use-one-column-from-model)
- [Using multiple columns from a model](#use-multiple-columns-from-model)
#### Use JSON editor
With the JSON editor, you can compose any JSON object using the [Liquid template language](https://shopify.github.io/liquid/). This is particularly useful for complex payloads containing nested objects and arrays, which can sometimes be difficult to model entirely in SQL.
Suppose your data model looks like this:
| `full_name` | `age` | `email_address` | `phone_number` |
| ----------- | ----- | ------------------ | -------------- |
| John Doe | 30 | `john@example.com` | +14158675309 |
And you want your HTTP request to have a payload like this:
```json
{
"name": "John Doe",
"age": 30,
"contact_info": [
{
"type": "email",
"value": "john@example.com"
},
{
"type": "phone",
"value": "+14158675309"
}
]
}
```
The Liquid template you enter under **Define JSON payload** should look like this:
```txt
{
"name": "{{row.full_name}}",
"age": {{row.age}},
"contact_info": [
{
"type": "email",
"value": "{{row.email_address}}"
},
{
"type": "phone",
"value": "{{row.phone_number}}"
}
]
}
```
You can also use advanced Liquid features to incorporate control flow and loops into your dynamic payloads.
When injecting strings into your JSON request body, be sure to surround the
Liquid tag in double quotes.
#### Batching with the JSON editor
When batching is enabled, the context is changed from a single record (`row`) to multiple records (`rows`).
With multiple rows, you likely want to use [loops](https://shopify.github.io/liquid/tags/iteration/) to build your payload.
Suppose your data model looks like this:
| `full_name` | `age` | `email_address` | `phone_number` |
| ----------- | ----- | ------------------ | -------------- |
| John Doe | 30 | `john@example.com` | +14158675309 |
| Jane Doe | 29 | `jane@example.com` | +14158674319 |
And you want your HTTP request to have a payload like this:
```json
{
"users": [
{
"name": "John Doe",
"age": 30,
"contact_info": [
{
"type": "email",
"value": "john@example.com"
},
{
"type": "phone",
"value": "+14158675309"
}
]
},
{
"name": "Jane Doe",
"age": 29,
"contact_info": [
{
"type": "email",
"value": "jane@example.com"
},
{
"type": "phone",
"value": "+14158674319"
}
]
}
]
}
```
The Liquid template you enter under **Define JSON payload** should look like this:
```txt
{
"users": [
{% for row in rows %}
{
"name": "{{row.full_name}}",
"age": {{row.age}},
"contact_info": [
{
"type": "email",
"value": "{{row.email_address}}"
},
{
"type": "phone",
"value": "{{row.phone_number}}"
}
]
},
{% endfor %}
]
}
```
Don't worry about unnecessary trailing commas when evaluating loops in your
payload template. Hightouch automatically strips trailing commas to conform
with the JSON specification.
#### Using Hightouch metadata inside a Liquid template
In addition to using the `{{row.column_name}}` syntax to access row-level data, you can also tap into metadata about your Hightouch syncs and models.
This metadata is provided via the `context` variable. The following table lists the available metadata options.
| Variable | Description | Example |
| ------------------------- | ------------------------------------------------------------ | ----------------- |
| `{{context.model_id}}` | The ID of the model (or audience) associated with the sync | `12345` |
| `{{context.model_name}}` | The name of the model (or audience) associated with the sync | `"VIP Customers"` |
| `{{context.sync_id}}` | The ID of the sync | `67890` |
| `{{context.sync_run_id}}` | The ID of the sync run during which the request is made | `1234567890` |
#### Use one column from model
If you're already storing JSON data in your source, or if you have the ability to construct a JSON object using SQL, you can select one column in your model that already contains the full request payload.
This setting is commonly used when syncing web events that have already been collected and stored as JSON objects in your database.
#### Batching with one column from model
When batching is enabled, the payload is an array where each element is a row value from the selected column. For example, if you select a column that includes a JSON object as each row value, the batch payload would be an array of those JSON objects.
#### Use multiple columns from model
For the simplest use cases, Hightouch can construct a JSON object with key/value pairs based on multiple columns in your model.
Suppose your model looks like this:
| `email` | `first_name` | `last_name` |
| ----------------------- | ------------ | ----------- |
| `alice.doe@example.com` | Alice | Doe |
| `bob.doe@example.com` | Bob | Doe |
| `carol.doe@example.com` | Carol | Doe |
The field mapping in the preceding screenshot would generate the following payload for the first row:
```json
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "alice.doe@example.com"
}
```
You can use the [field mapper](/syncs/mapping-data) to rename fields. For
example, you can map `first_name` to `customer_first_name`. The field mapper
offers additional capabilities, such as [inline
mapping](/syncs/mapping-data#inline-mapping) to create objects and arrays.
#### Batching with multiple columns from model
When batching is enabled, the payload is sent as an array.
Suppose your model looks like this:
| `email` | `first_name` | `last_name` |
| ----------------------- | ------------ | ----------- |
| `alice.doe@example.com` | Alice | Doe |
| `bob.doe@example.com` | Bob | Doe |
| `carol.doe@example.com` | Carol | Doe |
The field mapping in the preceding screenshot would generate the following payload for the first batch:
```json
[
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "alice.doe@example.com"
},
{
"customer_first_name": "Bob",
"customer_last_name": "Doe",
"customer_email": "bob.doe@example.com"
},
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "carol.doe@example.com"
}
]
```
### Configure rate limiting and concurrency
In this step, you configure rate limits and concurrency. In other words, you tell Hightouch how to avoid overwhelming your HTTP endpoint with requests.
Most modern web APIs enforce rate limits, which set a maximum allowed number of requests per second, minute, or hour. Occasionally, APIs may also have concurrency limits, which set a maximum allowed number of requests that can be processed simultaneously.
Hightouch defaults to 1000 requests per second or 100 concurrent requests, whichever limit is reached first. You can override these defaults to meet the requirements of your web service. Keep in mind that rate limits and concurrency limits both affect sync speed.
To ensure that Hightouch never exceeds your chosen limits, we apply a small offset to the values provided in the sync configuration form.
Hightouch waits out the entire rate limit when a sync finishes, to make sure the next run has the full number of requests and doesn't violate the rate limit configuration.
For example, if you set the rate limit window to **Requests per hour**, a sync run waits one full hour before triggering a new run.
If you want to avoid this behavior, you can set the window to **Requests per second** or **Requests per minute**.
Concurrency and rate limit controls aren't shared between syncs and don't
persist across sync runs.
### Configure error handling
In this step, you tell Hightouch how to handle errors, such as HTTP request timeouts and error responses.
These errors will be retried indefinitely until they succeed. You have a choice between retrying immediately or waiting until the next sync run. If you elect to retry immediately, you can specify how many retries should be attempted during the sync run. If all of these retries fail, the request will be retried during subsequent sync runs until it succeeds. Setting up [alerts for row and sync errors](/syncs/alerting) is always recommended.
Any HTTP response with a 400- or 500-level status code is considered an error.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows, such as hydrating a CRM for the first time, may require performing a **backfill of all rows** during the initial sync. For other use cases, such as sending product notifications, you may want to **skip existing rows** and only make HTTP requests for future data changes.
## Limitations
This destination can't perform complex operations that involve lookups or multiple requests chained together.
If you need to perform such complex operations, consider using one of our serverless function destinations: [AWS Lambda](/destinations/aws-lambda), [Google Cloud Functions](/destinations/google-cloud-functions), or [Azure Functions](/destinations/azure-functions). You could also build your own integration using our [Embedded Destination framework](/destinations/custom).
## Tutorial
This guided tutorial uses the HTTP Request destination to integrate Hightouch with [Algolia's REST API](https://www.algolia.com/doc/rest-api/search/). Specifically, it gives instructions for how to sync [synonyms](https://www.algolia.com/doc/guides/managing-results/optimize-search-results/adding-synonyms/) in a search index for each row added to a data model.
Hightouch provides an [Algolia integration](/destinations/algolia) that allows you to index records and events, but not synonyms, rules, or other Algolia objects.
### Destination configuration
First, we go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Then, we select **HTTP Request** and click **Continue**. Next, the UI prompts us to enter a **base URL**, **HTTP headers**, and **certificate**.
Algolia's Search API has the same host URL for searching, adding objects, and adding settings like synonyms.
We can enter this URL as the **base URL** in Hightouch.
According to [Algolia's docs](https://www.algolia.com/doc/rest-api/search/#authentication), you should authenticate API requests with these headers:
- `X-Algolia-Application-Id`: the ID of your Algolia application
- `X-Algolia-API-Key`: an [API key](https://www.algolia.com/doc/guides/security/api-keys/)
In Hightouch, we would provide these as the **HTTP headers**.
Be sure to replace the example headers with your actual Algolia application ID and an API Key with the write permission.
Since we don't need to use any security certificates, we can leave those options turned off and click **Continue**.
The last step of the initial configuration is giving our destination a descriptive name.
In this case, "HTTP Request - Algolia REST API" lets us know what we can use this destination for.
### Create synonyms
Next, we need to understand Algolia's API for adding synonyms to a search index. According to [Algolia's API reference](https://www.algolia.com/doc/rest-api/search/#synonyms-endpoints), we can add multiple synonyms by making a `POST` request to the [`/1/indexes/{indexName}/synonyms/batch` endpoint](https://www.algolia.com/doc/rest-api/search/#save-synonyms).
Here's how we can configure the **rows added** trigger in Hightouch to match this:
- Select to add a **batch of multiple rows** in each request, we can leave the default batch size of 100
- Select **POST** as the HTTP method
- Enter `/1/indexes/{indexName}/synonyms/batch` as the **URL**
In the preceding screenshot, the application ID and index name we want to add synonyms to have been filled out; you need to include your own Algolia application ID and index name.
[Algolia's docs](https://www.algolia.com/doc/rest-api/search/#format) say that the entire API uses JSON, so we select **JSON** as the payload type.
To construct the request payload, we need to reference our Hightouch model, which might look like this:
{/* */}
| `id` | `synonyms` |
| ----- | ----------------------------------------- |
| "sso" | "sso, single sign on, single sign-on" |
| "cdc" | "diff, diffing, change data capture, cdc" |
{/* */}
[Algolia's synonym batch endpoint reference](https://www.algolia.com/doc/rest-api/search/#save-synonyms) tells us that the synonym batch endpoint expects a JSON array of synonyms with a specific format.
```json
[
{
"objectID": "synonymID1",
"type": "synonym",
"synonyms": ["iphone", "ephone", "aphone", "yphone", "apple phone"]
},
{
"objectID": "synonymID2",
"type": "onewaysynonym",
"input": "iphone",
"synonyms": ["ephone", "aphone", "yphone", "apple phone"]
}
]
```
We can construct this expected JSON payload by using the **JSON editor** and inputing this as the **JSON payload** definition:
```json
[
{% for row in rows %}
{
"objectID": "{{ row.id }}",
"type": "synonym",
"synonyms": [ "{{ row.synonyms | replace: ', ', '", "' }}" ],
},
{% endfor %}
]
```
This example uses the Liquid [`replace`
function](https://shopify.github.io/liquid/filters/replace/) to create a
properly formatted array from a comma-separated string. If the data type in
the model results is already an array, this transformation isn't necessary.
Next, we reference [Algolia's docs regarding rate limits](https://www.algolia.com/doc/guides/sending-and-managing-data/prepare-your-data/in-depth/index-and-records-size-and-usage-limitations/#indexing-rate-limit).
Algolia starts throttling indexing operations when you have over 100 pending requests, so we set a rate limit of 100 requests per second and a concurrency limit of 100 requests at a time.
In the unlikely scenario that Algolia's API is experiencing downtime, we would want to wait a while before retrying failed rows. Therefore, we configure our sync to retry errors during the next sync run and give Algolia 30 seconds to respond to each of our HTTP requests.
Lastly, we want to make sure that our first sync run backfills our Algolia index with synonyms for all rows present in the initial model results.
That's it. Using Hightouch's UI, we've built a custom integration with Algolia to create new synonyms whenever new rows appear in our Hightouch model.
Once we've run the sync, we should check our Algolia dashboard to confirm the synonyms appear as we expect.
We can also use the Hightouch [debugger](/syncs/debugger) to inspect sync runs.
### Updating or deleting synonyms
In the future, we might want to update and delete existing synonyms so that our Algolia index always stays in sync with the latest data in our source.
To do this, we would enable the **rows changed** and **rows removed** triggers for the HTTP Request destination. The configuration for each of these triggers would be slightly different from the tutorial above, but the same concepts apply. Reference the Algolia API docs to determine the appropriate HTTP method, endpoint, and payload for each operation.
---
## HubDB
**URL:** https://hightouch.com/docs/destinations/hubdb
**Description:** Easily sync data and publish tables to always keep your website or landing page up to date.
**Section:** Destinations
## Overview
With Hightouch's HubDB integration, you no longer need to rely on others to build and maintain databases. Push and store data to tables that you can embed within websites and landing pages. Integrated into HubSpot, data is easily accessible, editable and shareable across multiple tables. You can even publish table updates in one sync.
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|**Tables**|Sync data to create or update tables in a HubDB. |Upsert, Update|
## Connect to HubDB
You must have one of the following HubSpot account types to use the Hightouch integration:
- CMS Hub Professional
- Enterprise
- Marketing Hub Professional
- Enterprise with Website Add-on
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **HubDB**. You can then authenticate Hightouch to HubDB via **OAuth**.
### Authenticate with OAuth
Click **Log in to HubDB** and log into your HubSpot account.
Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your HubDB destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the HubDB destination you want to sync to.
In the HubDB sync configuration form, select the HubDB table you want to sync to.
#### Record matching
Hightouch requires choosing one column to define how rows in your source should be matched to records in HubDB. The field to match should be unique and the data type of the field should match the data type of the corresponding column in your model.
#### Field mapping
You can choose which specific fields you want to sync to your HubDB table. Only the fields that you map will be exported. All other columns from your results are ignored. If you don't have a HubDB field to sync data to, you can add a new field to the table directly within HubDB. Then, refresh the HubDB fields and the new field should be available to map.
#### Delete behavior
You can choose how to handle data points in HubDB when the corresponding rows are deleted in your source.
| Behavior | Description |
| ----------------------------- | -------------------------------------------------------------- |
| **Do nothing** | Keep the data in HubDB |
| **Clear Fields** | Clear the mapped fields but keep the record in the HubDB table |
| **Delete Destination Record** | Delete the record from the HubDB table completely |
## Tips and troubleshooting
### Common errors
#### 400 - the value for column date was of type STRING, but we're expecting NUMERIC
The full error message looks something like this:
```json
{
"message":"Error when batch creating datatable rows in table 6058806 for portal 2661677: the value {type=string, value=2021/03/20 13:30} for column date was of type STRING, but we're expecting NUMERIC",
}
```
One possible cause for this error is sending incorrectly formatted date values to HubDB.
[HubDB's `date`](https://developers.hubspot.com/docs/api/cms/hubdb) data type must be a millisecond timestamp set to midnight UTC.
A millisecond timestamp is numerical value with 13 digits, for example `1620170285617`, as compared to a traditional [Unix timestamp](https://en.wikipedia.org/wiki/Unix_time) which has ten.
### Live debugger
### Sync alerts
---
## HubSpot
**URL:** https://hightouch.com/docs/destinations/hubspot
**Description:** Prioritize & personalize your marketing & sales outreach by adding customer data into HubSpot
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------ | ----------------------------------------------------------------------------- | ---------------------- |
| [**Objects**](#syncing-objects) | Sync records to objects such as contacts or companies in HubSpot. | Upsert, Update, Insert |
| [**Events**](#syncing-events) | Sync records as custom behavioral events in HubSpot. | Insert |
| [**Contact lists**](#syncing-contact-lists) | Create, sync, and keep contact lists up-to-date in HubSpot. | Add, Remove |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
---
## Understand your HubSpot setup
Before you begin your Hightouch to HubSpot sync configuration, you should clarify your business goals and understand your HubSpot setup. Walk through the following considerations to align your team and confirm that your sync plan matches how HubSpot is configured. If you prefer, you can download this resource as a PDF to share with your team.
### Business goals
Before configuring your Hightouch to HubSpot sync, clarify what data you want to sync, how it should be updated, and how often it needs to stay in sync. Align on these decisions with your team so your sync configuration reflects your business goals.
The table below shows how these decisions influence your sync setup.
|
Consideration
| Effect on sync configuration |
| ------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| What standard and custom objects do you want to sync to HubSpot? Contacts, companies, etc.? | The response affects your [sync types](#supported-syncing)—in other words, which objects you select to sync to. You need to set up a separate sync configuration for each object you want to sync to. |
| What standard and custom properties do these objects have that you want to sync to? | The response affects the [field mapping](#field-mapping) on each of your syncs. |
| Do you have new data to add, old data that needs updating, or a combination of both? | The response affects your [sync modes](#supported-syncing)—for example, insert mode for new data, update mode for old, and upsert mode for both. |
| How frequently do you need to update the data? | The response affects your [sync schedule](/syncs/schedule-sync-ui). This is the last step of your sync configuration. |
### Required data specifications
Once you’ve defined what you want to sync, verify that your HubSpot setup supports your plan. Use the table below to validate that your objects, properties, and permissions are configured correctly before creating your syncs.
|
Consideration
| Effect on sync configuration |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Is the property that you would like to use for [record matching](#record-matching) unique? | Non-unique values can lead to duplicates and sync errors. |
| What are the expected data types of the properties you want to sync to? | The response affects your [field mapping](#field-mapping); if the expected type differs from your model column type, you must [type cast your data](https://hightouch.com/docs/models/data-types-casting) in your model configuration. |
| Are the properties you want to sync editable? | Hightouch only displays editable fields during [field mapping](#field-mapping). |
| Are any of the properties you want to sync [calculated](https://community.hubspot.com/t5/Tips-Tricks-Best-Practices/Using-calculated-properties/m-p/372525) properties? | Calculation properties are non-editable and read-only. Change the field type if you want to sync data to it. |
| Are any of the properties you want to sync [enumeration](https://knowledge.hubspot.com/account/property-field-types-in-hubspot) properties? | Enumeration properties only accept specific values. Ensure your data matches these values or create a new property. |
| Are there any associations you want to make between objects? | You can set up [associations](#associations) while field mapping, assuming the association exists in HubSpot first. |
| Does the user you are authenticating to Hightouch have edit access? | The sync will error if the user doesn't have the correct permissions. |
Permissions depend on both the HubSpot user’s role and the OAuth scopes granted when connecting Hightouch. If objects or properties don’t appear during [record matching](#record-matching) or [field mapping](#field-mapping), re-check permissions and reconnect the destination with the appropriate scopes.
Use the steps in [Inspect objects, properties, and associations](#inspect-objects-properties-and-associations) to validate your HubSpot configuration before creating your syncs.
### Inspect objects, properties, and associations
1. In HubSpot, click the settings icon in the top navigation bar. In the left sidebar, go to **Objects** (under **Data Management**), then select the object you want to inspect (for example, **Contacts**, **Companies**, **Leads**, or a **Custom Object**).
2. In the object's overview page, under **Setup**, click **Manage [object] properties**.
3. Review both standard and custom properties you want to sync data to. Make note of their:
- **Internal name**: This is the name that appears in the Hightouch UI for [record matching](#record-matching) and [field mapping](#field-mapping). You can access it by clicking on the property name and then clicking **\** in the property inspector.
- **Property access**: The HubSpot user you use to [authenticate to Hightouch](#connect-to-hubspot) should have access to edit the property.
- **Data type**: For example, **Single-line text** or **Number**—you can view these under the property **Name**. When selecting model columns for [field mapping](#field-mapping), you must ensure the data type matches. For example, a **Single-line text** property requires text data and a **Number** requires numerical data.
4. Check whether any of the properties you plan on syncing to are an enumeration type, such as **Dropdown select**, **Radio select**, or **Multiple checkboxes**. For example, the native **Lead rating** property for contacts is a **radio select** property.
If you plan on syncing to enumeration properties, the data you sync must match the **internal value** of the options. Lead rating, for example, can accept the values "bucket_1," "bucket_2," "bucket_3," and "bucket_4," though the labels are "1 Star," "2 Stars," etc.
Since internal values can't be updated once created, you may need to create a new custom property to accept data from your model. Refer to HubSpot's guide on [property field types](https://knowledge.hubspot.com/account/property-field-types-in-hubspot) to select the appropriate field type when creating new properties.
5. Check if the property requires unique values: Click on its name, then go to **Rules**. Under **Validation rules**, see if **Require unique values for this property** is checked. Properties you want to use for [record matching](#record-matching) must have this validation.
6. Check if the property is sensitive or highly sensitive. To sync sensitive data, you must be using an Enterprise account and have permissions from your admin to read and write to sensitive and highly sensitive properties.
7. Return to the object's overview page and select **Associations**. Look for associated child objects like contacts, companies, or deals linked to the records. You can use these to map [associations](#associations) in Hightouch. Add, remove, or correct links between objects as needed.
Once you've validated the properties and associations of the objects you want to sync to, you're ready to configure your sync in Hightouch.
---
## Connect to HubSpot
1. Go to **`Integrations`** > [**`Destinations`**](https://app.hightouch.com/destinations).
2. Click **`Add destination`**.
3. Select **`HubSpot`** and click **`Continue`**.
4. Choose how to authenticate Hightouch to HubSpot:
- **`OAuth`** (recommended): Use this option to sync CRM objects (for example, contacts and companies) and update standard properties.
- **`OAuth with enterprise scopes`**: Required if you plan to sync **custom behavioral events** or use Enterprise-only features such as sensitive properties.
- Enterprise users can also enable scopes for [sensitive data](https://knowledge.hubspot.com/properties/store-sensitive-data). Select **`Sensitive`** or **`Highly sensitive`** only if your sync requires access to those properties.
For the smoothest setup, authenticate using a HubSpot user with admin permissions (typically **Super Admin**). HubSpot permissions and the OAuth scopes you approve both affect which objects and properties appear in Hightouch and whether syncs can write updates.
Only enable **`Sensitive`** or **`Highly sensitive`** scopes if your sync needs to read or write those properties.
5. Select the **HubSpot account** you want to connect to Hightouch.
6. Review and accept the scopes that the Hightouch app requests from HubSpot.
7. After authorization succeeds, you’ll be redirected back to Hightouch. Click **`Continue`**.
8. (Optional) Configure alert settings and recipients. You can edit these later from the destination settings.
9. Enter a descriptive **destination name** and configure permissions for:
- Triggered syncs
- Configuring models and syncs
- Managing the destination
10. Click **`Finish`** to complete setup.
---
At this point, you should know which objects, properties, identifiers, and associations you plan to sync, and have verified that your HubSpot permissions and property types support your configuration.
## Sync configuration
Once you've set up your HubSpot destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the HubSpot destination you want to sync to.
### Syncing objects
Hightouch can create and update standard HubSpot objects such as contacts, companies, deals, etc. Hightouch can also sync data to custom HubSpot objects. To sync to a custom object, you must [create it in HubSpot](https://knowledge.hubspot.com/crm-setup/use-custom-objects) first.
#### Record matching
To match rows from your model to objects in HubSpot, select a HubSpot property and the corresponding column from your model.
You can match on any of the available properties in the dropdown. It's best to select a property that [HubSpot automatically deduplicates on](https://knowledge.hubspot.com/crm-setup/deduplication-of-contacts-companies-deals-tickets#deduplicate-companies-by-company-domain). These include **email** (Contacts), **company domain** (Companies), or a **HubSpot record ID** (for example, `hs_object_id`) for other object types.
If you select a model column with duplicates to send to one of these properties, HubSpot [returns a 409 error](#409---contact-already-exists) and the sync errors.
If record matching isn’t unique (for example, multiple HubSpot records match the same value), HubSpot may reject updates and Hightouch may need extra lookups. This can slow syncs and cause partial failures.
The `id` property is only available for record matching when in update mode.
It is a read-only property, and HubSpot sets its value upon object creation.
The HubSpot property you select to match on should be **unique**, and its data type must match the data type of the corresponding column in your model. If the property to match on isn't unique, record matching will be slower and duplicate records may occur. You can check a property's uniqueness validation by [inspecting the property in HubSpot](#inspect-objects-properties-and-associations).
If the object doesn't have a native unique property, we recommend creating a unique custom property. Refer to [HubSpot's instructions](https://community.hubspot.com/t5/Tips-Tricks-Best-Practices/Custom-unique-property/td-p/683119#) on how to do so.
#### Field mapping
Field mapping lets you choose which properties you want to update on your HubSpot objects. If the HubSpot object you're syncing to doesn't have the specific property you want to send your data to, you need to [create a new property on the object](https://knowledge.hubspot.com/crm-setup/manage-your-properties) in HubSpot first.
You can also sync to [enumeration
properties](https://knowledge.hubspot.com/account/property-field-types-in-hubspot),
such as **Dropdown select**, **Radio select**, and **Multiple checkboxes**.
Make sure to create your property's internal values before syncing them, as
explained in the [Common errors](#400---property-values-were-not-valid)
section.
Check out [HubSpot's documentation](https://knowledge.hubspot.com/crm-setup/import-data-to-checkbox-properties)
to learn how to sync **Multiple checkboxes**.
After creating or editing properties, you can refresh the available properties for mapping by clicking the refresh button next to **`Field from HubSpot`**. The new property should then be available.
Hightouch displays the property's **Internal name**, not label, in the dropdown. You can find a property's internal name by [inspecting the property in HubSpot](#inspect-objects-properties-and-associations).
Read-only fields in HubSpot aren't available for mapping in Hightouch.
If a model column’s data type doesn’t match the HubSpot property type (for example, sending text to a number property), HubSpot may reject the row. If needed, type cast your data in your model before syncing.
#### Field sync behavior
Each field mapping has a sync behavior setting that controls how Hightouch handles the value for that field. Click the gear icon on a field mapping row to choose one of the following options:
| Behavior | Description |
| --- | --- |
| **Always sync** | Syncs the value from your model to HubSpot on every sync run, including null values. Use caution, as this can erase existing production data. Does not apply to nested object or array properties. |
| **Don’t sync null values** | Skips syncing any null values from your model to HubSpot. If the model value is null, the existing HubSpot value is left unchanged. |
| **Only write to empty fields** | Only writes the value if the HubSpot property is currently empty. Existing non-empty values are not overwritten. |
**Only write to empty fields** is useful when you want to populate missing data without overwriting values that have already been set in HubSpot—for example, backfilling phone numbers or other contact details without replacing manually entered data.
#### Associations
**Associations** let you express a relationship between the object you're syncing to (the **main object**) and a different HubSpot object (the **associated object**). For example, you may be syncing contacts to HubSpot and want to associate a company with each one.
Before mapping an association in Hightouch, confirm the association exists for the object pair in HubSpot (**Settings → Objects → [Object] → Associations**).
1. Click **`Add mapping`**, then select the association label under **`Field from HubSpot`**. The list of supported association labels for the object appears in the dropdown. You can find an object's supported associations and add others by [inspecting the object in HubSpot](#inspect-objects-properties-and-associations).
2. Search for the object you want to associate to (the **associated object**). For example, if you want to associate a company with each contact you're syncing, search for "company," then select the appropriate association. In this example, you would select **CONTACT_TO_COMPANY**.
The association should only be set up in the main object's sync configuration.
In this example, you would only add it to the **Contact** sync and not to the
**Company** one. You can learn more about this in the [Common
errors](#conflicting_one_to_ones_specified) section.
Make sure that the sync to the associated object always runs before the one to
the main object. After creating your syncs, you can [set up a
sequence](/syncs/schedule-sync-with-sequences) to schedule them in the correct
order.
Next, select the properties to match on. For example, if you're associating companies with contacts and a company's `name` matches the `COMPANY_NAME` in your contact data, you would select those properties.
If your data contains the `id` of the object you want to associate to, it's best to use this column for matching. Since the sync doesn't have to perform a lookup, sync performance is significantly faster.
The example below shows using contact data that includes the associated company's id in a column called `ACCOUNT_ID`.
You can create multiple HubSpot associations in a single sync. Click **`Add mapping`** for each association you want to make.
For more information on associations, check out [HubSpot's association docs](https://knowledge.hubspot.com/crm-setup/associate-records).
#### Association modes
##### **Create added associations**
Associations will only be appended to the record. This setting will not remove previous associations if the associated object is updated or removed.
##### **Overwrite existing associations**
When there are changes to associations in the source, remove all associations in HubSpot before creating the associations present in the source. If there are no changes to associations in the source, do not remove any associations.
This setting will also remove any associations that were created outside of
Hightouch.
##### **Delete removed associations and create added associations**
This mode will find the differences between sync runs to delete associations in HubSpot that have been removed from the model, then create associations that have been added to the model.
#### One-to-many relationships
You might want to associate the object you're syncing to with multiple objects of the same type. For example, if you're syncing company objects, you may want to link several contacts to each company.
For Hightouch to associate multiple objects, the model column you select for the association must be in one of these formats:
- an array of string or number values, for example, `["id1", "id2", "id3"]`
- a comma-separated string, for example, `"id1, id2, id3"`
Confirm that the association cardinality in your HubSpot settings is
compatible with the data you're syncing to HubSpot. You can learn more about
this in the [Common errors](#conflicting_one_to_ones_specified) section.
#### Association labels
If you have a Professional or Enterprise HubSpot account, you can also use HubSpot's [association labels feature](https://knowledge.hubspot.com/crm-setup/create-and-use-association-labels) to describe relationships between objects.
Association labels let you add descriptive text to an association that provides more context about the relationship. For example, instead of just linking a contact to a company, you can use an association label like "Employee at" or "Customer of" to clarify the nature of the relationship.
Before configuring a sync to use association labels, check out HubSpot's
association label limitations at the bottom of this [HubSpot
doc](https://developers.hubspot.com/docs/api/crm/associations/v4).
To add association labels to a Hightouch sync, you must first [set up your labels in your HubSpot account](https://knowledge.hubspot.com/crm-setup/create-and-use-association-labels). Then, you need to update the [model](/getting-started/concepts#models) column you use for associations to contain an object with this schema:
```
{ "value": string | number, "label": string | Array }
```
Make sure to set the model column's [data
type](/models/data-types-casting#view-model-data-types) to **Object / Array**.
Hightouch uses the **value** to match the associated object and the **label** as the label on the association in HubSpot. For example, if you're associating companies with contacts, you may set the company name as the value and the contact's role as the label.
```
{ "value": "ABC Corp", "label": "Employee at" }
```
If you want to add more than one label, the label can be an array of strings or a comma-separated list. For example, if you're associating companies with a contact and the contact has multiple roles, a row from your model might look like this:
```
{ "value": "ABC Corp", "label": ["Employee at", "Decision maker"] }
```
If you want to add more than one association label to an object, you can sync an array of association label objects.
For example, if you're associating billing data with multiple contacts, a row from your model might look like this:
```
[{ "value": "John Doe", "label": "Buyer" }, { "value": "Jane Doe", "label": "Seller" }]
```
Ensure **all** rows follow this schema to avoid inconsistent behavior with HubSpot's API. If a row doesn't have a label, you can set the label as `null`.
For example, suppose you want to associate a company with a contact but don't have any information about the contact's role there. The model row should look like this:
```
{ "value": "ABC Corp", "label": null }
```
You can use some SQL helper functions to create an object or array of objects with this schema. Below is an example of a SQL query that creates an object for an association label:
```sql
SELECT
id,
first_name,
last_name,
email,
JSONB_BUILD_OBJECT('value', company_name, 'label', role) as company_obj
```
The query creates a column called `company_obj` and sets the `company_name` name as the value in the object and the `role` as the label in the object.
Once you've formatted the data correctly, ensure the association in your sync configuration uses the column with the properly formatted object.
#### Sensitive data
HubSpot allows for Enterprise users to store [sensitive data](https://knowledge.hubspot.com/properties/store-sensitive-data). Prior to creating Sensitive Data properties, a user with super admin permissions must turn on Sensitive Data in your Privacy & Consent
settings and accept the related terms and conditions.
A property can be defined as sensitive or highly sensitive upon creation in the UI or through the API. If syncing to these sensitive data properties, ensure that you have the correct scopes selected when you authorize your account.
Currently, Hightouch supports using the following sensitive data scopes:
- `crm.objects.custom.sensitive.read`
- `crm.objects.custom.sensitive.write`
- `crm.objects.custom.highly_sensitive.read`
- `crm.objects.custom.highly_sensitive.write`
- `crm.objects.appointments.sensitive.read`
- `crm.objects.appointments.sensitive.write`
Sensitive scopes are object-specific. If you don’t see the object you need covered by a sensitive scope, confirm HubSpot supports sensitive properties for that object type and reconnect with the required scopes.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. Depending on the [sync mode](#syncing-objects) you're using, you can select from some or all of these options:
| Behavior | Description |
| ---------------------------------------- | --------------------------------------------------- |
| **Do nothing** | Keep the record in HubSpot |
| **Clear fields** | Keep the record, but clear the mapped fields |
| **Archive record** | Archive the record in HubSpot |
| **Delete record (Contacts only)** | GDPR Delete the record in HubSpot (Permanent delete)|
In HubSpot, **archived** records are removed from active views but are not permanently deleted. **GDPR delete** is permanent and only available for Contacts.
In Upsert mode, you can select from all of the above (**Delete record** for contacts only, performs a permanent delete using the [**GDPR delete**](https://developers.hubspot.com/docs/reference/api/crm/objects/contacts/v3#post-%2Fcrm%2Fv3%2Fobjects%2Fcontacts%2Fgdpr-delete) endpoint). In Update mode, you can only select between **Do nothing** and **Clear fields**. Insert mode doesn't support delete behavior.
#### Split retries
HubSpot counts all objects in a batch as rejected if the request contains a single invalid record. To pinpoint which records are getting rejected with which errors and reduce the number of valid records that get retried, you can enable [**split retries**](/syncs/retries#split-retries).
### Syncing events
Hightouch supports syncing [**custom behavioral events**](https://knowledge.hubspot.com/analytics-tools/create-codeless-custom-behavioral-events) to HubSpot with custom properties. To sync events, ensure you logged in via **OAuth with enterprise scopes** when you [connected Hightouch to HubSpot](#connect-to-hubspot).
#### Event name
Providing an event name is required to log an event in HubSpot. You can either provide a static value or select to **use a column** from your model data.
The event name should match the event's **internal name**. HubSpot assigns the internal name automatically when you create the event. You can find an event's internal name either in [HubSpot](https://knowledge.hubspot.com/analytics-tools/create-custom-behavioral-events#define-the-api-call) or by using the HubSpot [events API](https://legacydocs.hubspot.com/docs/methods/events/get_events).
Follow the instructions in HubSpot's docs to learn how to [find the internal name](https://knowledge.hubspot.com/analytics-tools/create-custom-behavioral-events#define-the-api-call) for an event.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server.
#### Assigning events to contacts
Hightouch lets you choose how you assign events to contacts. Select a column in your events data that matches one of these HubSpot contacts identifiers:
- email
- object (contact) ID
- user token
If the identifier column is null or doesn’t match an existing HubSpot contact, HubSpot can’t attribute the event to a record.
#### Field mapping
In the field mapping section, you can select columns from your model to sync to the default or custom properties of a HubSpot custom behavioral event.
Custom properties must exist on the event before you can sync to them. See
HubSpot's docs on [how to create a custom
property](https://knowledge.hubspot.com/analytics-tools/create-custom-behavioral-events-with-the-code-wizard)
on an event.
### Syncing contact lists
Hightouch supports syncing contacts to [contact lists](https://knowledge.hubspot.com/lists/create-active-or-static-lists) in HubSpot. You can sync data to an existing contact list or create a new one on the first sync run. Select **Segment** as the sync type.
#### User identifiers
To identify which contacts to add or update in a contact list, select a model column and the corresponding HubSpot field.
You can match on either **Email** or **HubSpot Contact ID**.
#### List creation or selection
Hightouch supports automatically creating a new list for your sync. You can specify a custom name for this list; if left empty, Hightouch defaults to the model name. The example below shows defaulting to the model name "Newsletter subscribers."
If you opt to sync to an existing list, you can select the list name from the dropdown.
#### Migrating from v1 to v3
If you had existing contact list syncs that use HubSpot's v1 API before April 30th, 2026, you need to migrate them to v3. The v1 and v3 APIs use different list IDs, so you must re-select your list after switching.
To migrate a sync:
1. Open your sync configuration.
2. Under **Object type**, select **Contacts**.
3. Select **Use an existing list** and **re-select your list from the dropdown.**
4. Save the sync.
Your list contents and membership are unchanged by this migration. Only the API version is updated.
---
## Disconnecting HubSpot
### Disconnect HubSpot from Hightouch
To disconnect HubSpot from Hightouch:
1. Open [**`Destinations`**](https://app.hightouch.com/destinations).
2. Select your HubSpot destination.
3. Choose **`Actions`** > **`Delete`**.
Deleting the destination also deletes any syncs that use that destination. Previously synced data in HubSpot is not changed.
### Uninstall Hightouch from HubSpot
You can uninstall the Hightouch app from your HubSpot account by referring to these [instructions](https://knowledge.hubspot.com/integrations/connect-apps-to-hubspot#uninstall-an-app). No data previously synced from Hightouch to HubSpot will be impacted.
---
## Tips and troubleshooting
### Date values
When syncing date or timestamp values to HubSpot, you need to consider which data type the HubSpot field should use: `date` or `datetime`.
HubSpot `date` fields support date values which store a date without a time. `datetime` fields support timestamps. For more information, check out [HubSpot's timestamp FAQ](https://developers.hubspot.com/docs/api/faq#timestamps).
Values you send to a `date` field must follow one of these formats:
- date-only ISO 8601 strings (`YYYY-MM-DD`), for example, `2020-02-29`
- EPOCH-millisecond [timestamps](https://en.wikipedia.org/wiki/Unix_time), **with the time set to midnight UTC**: for example, `1430438400000`
If you're syncing a timestamp to a date field, Hightouch automatically sets its time to midnight UTC, as required by HubSpot.
Values you send to a `datetime` field must follow one of these formats:
- ISO 8601 formatted strings (`YYYY-MM-DDThh:mm:ss.sTZD`), for example, `2020-02-29T03:30:17.000Z`
- EPOCH-millisecond [timestamps](https://en.wikipedia.org/wiki/Unix_time): for example, `1427997766000`
HubSpot displays `datetime` values in the timezone of the user viewing the record.
If you notice the timestamp from your Hightouch model column isn't appearing in HubSpot for your `datetime` field, edit the JSON mapping in your sync configuration to include `"skipDateParse": true`. See example below:
```json
mappings: [
{
"from": "hightouch_datetime_column",
"to": "hubspot_datetime_field",
"skipDateParse": true
}
]
```
### Common errors
#### 429 - You have reached your secondly limit
You may receive a `429 - You have reached your secondly limit` error message for two reasons:
- You have another sync to HubSpot running simultaneously, which could make you reach HubSpot's secondly limit.
- You've set up [associations](#associations) in your sync configuration. The HubSpot API has constraints that cause sync slowness when using associations.
To reduce the occurrence of this error, ensure you're following [record matching configuration suggestions](#record-matching).
For more information about HubSpot's API limits and other 429 errors, [see HubSpot's API usage guidelines](https://developers.hubspot.com/docs/api/usage-details).
{/* */}
#### Missing permissions or OAuth scopes
If objects, properties, or events don’t appear in Hightouch, or syncs fail immediately after connecting, the HubSpot user may not have the required permissions or the connection may be missing required scopes.
To resolve this:
- Reconnect HubSpot using a user with admin permissions (typically Super Admin).
- Approve the required scopes (use **OAuth with enterprise scopes** for events and sensitive properties).
#### 409 - Contact already exists
{/* */}
You may receive a `409 - Contact already exists` or `400 - Duplicate IDs found in batch input` error for three reasons:
- the model column you are using for [record matching](#record-matching) contains duplicate values
- the model column [mapped](#field-mapping) to HubSpot's `email` field contains duplicate values
- you're attempting to create a **new** contact in HubSpot that has the same `email` value as an already existing HubSpot contact, but a different value in the HubSpot field used for record matching (for example, `user_id`)
HubSpot uses email as the primary unique identifier to avoid duplicate contacts.
In other words, two different contacts can't share the same email.
See [HubSpot's Contacts API Reference](https://developers.hubspot.com/docs/api/crm/contacts#properties) for more information.
To resolve these errors, ensure:
- the column you selected for record matching doesn't contain duplicate values
- any email information you are syncing also doesn't include duplicate values
- you aren't attempting to create a **new** HubSpot contact with the same email as an already existing one, but a different value in the record-matching field
The `409 - Contact already exists` error message contains an `Existing ID`
value returned by HubSpot. This value refers to HubSpot's ID for the contact
and doesn't necessarily correspond to any values in your model data.
{/* */}
#### Property does not exist or is not writable
This can happen if the HubSpot property doesn’t exist on the target object, is read-only, or the connected HubSpot user/scopes don’t have permission to write to it.
To resolve this:
- Confirm the property exists on the correct HubSpot object (**Settings → Objects → [Object] → Properties**).
- Confirm the property is editable (calculated properties are read-only).
- Reconnect the destination if you recently added custom objects or changed permissions/scopes.
#### 400 - Cannot update object with type ... the following properties were cleared ...
{/* */}
You may receive a `400 - Cannot update object with type: ... the following properties were cleared: ...` because there is no data being sent to a required property in HubSpot. The properties listed after `the following properties were cleared` are the required property or properties.
You can resolve the error by either:
- Removing records with empty values in the required property; you can update your [model definition](/models/creating-models) to exclude rows like this
- Adding a value to the required property
#### 400 - Property values were not valid
You may receive a `400 - {"status":"error","message":"Property values were not valid: [{"isValid":false,...` error message because you are attempting to send a value to an [enumeration property](https://knowledge.hubspot.com/account/property-field-types-in-hubspot) that isn't one of the valid options.
Enumeration properties include **Dropdown select**, **Radio select**, and **Multiple checkboxes**.
Ensure that the values exactly match the HubSpot's **Internal values** and not the **Labels**.
These values are case sensitive.
The error message may contain lines like this:
- `"error":"INVALID_OPTION","name":"custom_dropdown_field"...`
- `"error":"INVALID_OPTION","message": {...} was not one of the allowed options...`
These can help pinpoint which HubSpot field is in question. Refer to step 4 of the [Inspect objects, properties, and associations](#inspect-objects-properties-and-associations) section to find the **Internal Values** of your HubSpot field.
As explained in the [field mapping](#field-mapping) section, make sure to
follow [HubSpot's
guidelines](https://knowledge.hubspot.com/crm-setup/import-data-to-checkbox-properties)
when syncing to **Multiple checkboxes** fields.
#### Invalid property value due to type mismatch
This can happen when the data type in your model doesn’t match the HubSpot property type (for example, sending text to a number property).
To resolve this:
- Check the property’s field type in HubSpot.
- Type cast the model column to the expected type before syncing.
- Re-run the sync after updating the model or mapping.
#### CONFLICTING_ONE_TO_ONES_SPECIFIED
This error can occur if you set up an [association](#associations) both in the main object's and the associated object's sync configuration.
In this case, make sure to remove the association from the associated object's sync configuration.
This error can also happen if you add an association to your sync configuration, but the cardinality of your association or [association label](#association-labels) in HubSpot isn't compatible with the data you're syncing.
For example, if the cardinality is set to `1-to-1` for the given association, your sync can't [associate multiple records](#one-to-many-relationships) on the main object with one record on the associated object, or vice versa.
To resolve this, access the main object's HubSpot settings, open the **Associations** tab, and check your associations' cardinalities.
In your Hightouch sync configuration, make sure to use an association or association label that has the correct cardinality.
#### Row with ID email...could not be added to the list.
{/* */}
When syncing [contact lists](#syncing-contact-lists) and [matching on **Email**](#user-identifiers), you may receive a `Row with ID email...could not be added to the list` error. This occurs when the contact you're trying to sync doesn't exist in your HubSpot workspace. To resolve this, ensure that the contact exists in HubSpot prior to syncing it to a contact list.
#### Association label missing or misformatted
You may see an error like `Associating 0-2 object with {…} was not found` if an association requires a label but the payload doesn’t include one.
To resolve this:
- Ensure the model column for association labels uses the required object schema described in [Association labels](#association-labels).
- Ensure the `label` key is present (set to a string, array of strings, or `null`).
### Live debugger
### Sync alerts
---
## Impact.com
**URL:** https://hightouch.com/docs/destinations/impact-com
**Description:** Impact.com - One platform. All partnerships.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| -------------------------------- | ----------------------------------------------- | -------------------- | -------------------------------------------------------------------------------------------- |
| **Conversion Events** | Create and send conversion events to Impact.com | Insert | [Conversions docs](https://integrations.impact.com/impact-brand/reference/conversions-model) |
| **Objects (Catalogs / Actions)** | Sync catalog items or action records | Insert, Update | [Catalog docs](https://integrations.impact.com/impact-brand/reference/catalogs-overview) |
See [sync modes](/syncs/types-and-modes#sync-modes) for more details.
---
## Add Impact.com as a Destination
1. Go to the [**Destinations**](https://app.hightouch.com/destinations) page and click **Add destination**.
2. From the destination list, search for **Impact.com** and select it.
3. Click **Continue**.
---
### 1. Connect to Impact.com
1. Enter your **Account SID** and **Auth Token**.
You can authenticate with either a legacy account token or a scoped token. Both are available in your Impact.com account under **Settings → API → API Access**.
- **Legacy account token** — find under **Legacy Account Tokens**.
- **Scoped token** — find under **Scoped Tokens**. When creating the token, grant the following scopes:
- `Accounts`
- `Actions`
- `Campaigns`
- `Catalogs`
- `Conversions`
- `Jobs Management`
- `Submissions`
From July 1, 2026, Impact.com legacy read/write and read-only tokens can no
longer be copied. Existing integrations will continue to work, but new
integrations or token resets will require scoped tokens.
2. Click **Test connection** to confirm the connection is valid.
3. Click **Continue**.
---
### 2. Configure alerts (optional)
You can configure default alerting rules for all syncs using this destination.
You may add recipients and enable alerts for:
- Fatal errors
- Rejected rows
- Sync throughput
- Model or audience size
- Sync duration
- Excessive idle time
Click **Continue**.
---
### 3. Finalize destination
Name your destination and assign permissions.
You can choose which user groups can:
- Trigger syncs
- Configure models & syncs
- Manage the destination
Click **Finish**.
---
## Create a sync
After creating and connecting your Impact.com destination, navigate to:
**`Integrations → Destinations → Impact.com → Syncs`** and click **Add a sync**.
You’ll follow the standard Hightouch sync wizard:
1. **Select model**
2. **Select destination**
3. **Configure sync**
4. **Finalize sync**
The following steps describe the **Configure sync** settings unique to Impact.com.
---
## Configure sync
At the top of the configuration page, choose what you want Hightouch to send:
- **Conversions** — send conversion events
- **Objects** — send catalogs or actions
This choice determines which mapping sections appear.
---
## Conversions sync configuration
Choose **Conversions** when sending conversion events to Impact.com.
#### 1. Campaign ID
Select a value that identifies the program/campaign on Impact.com. You can:
- Select a campaign ID returned from Impact.com, or
- Toggle **Use column** to map a value from your model
This ID is required for all conversion event syncs.
#### 2. Event attribution keys
These fields identify **who or what** generated the conversion.
At least one attribute is required.
Common attribution keys include:
- Email (may require automatic hashing)
- Customer ID
- Click ID
- Partner or affiliate ID
- Custom fields configured in Impact.com
You can add multiple attribution fields.
#### 3. Event ID mapping
This mapping determines the **type of event** being sent.
Event types are created and managed in Impact.com under:
**Event Types → Actions → View/Edit → “Codes”**
Map at least one column that identifies the event type.
Examples include: `purchase`, `lead`, `signup`, or custom codes defined in Impact.com.
#### 4. Event timestamp
Choose a column that contains the timestamp of the event.
If you leave this unmapped, Impact.com will use the server arrival time.
Hightouch converts timestamps automatically to the format required by Impact.com (ISO 8601 or Unix milliseconds).
#### 5. Additional event information
Use this section to include extra metadata with your event, such as:
- Order amount
- Currency
- Product identifiers
- SKU or item lists
- Discounts
- Customer metadata
- Device or session information
If you need to include multiple items in the same conversion, manually create
additional fields by appending the index (e.g. `ItemName1`, `ItemName2`,
`ItemName3`).
Each mapping section requires at least one field. Providing rich event
metadata leads to better reporting in Impact.com.
Learn more about the Impact.com Conversion object in their [API documentation](https://integrations.impact.com/impact-brand/reference/the-conversion-object).
#### 6. Automatic PII hashing
Impact.com often requires that PII such as email addresses be **SHA-1 hashed**.
Toggle **Yes** to allow Hightouch to automatically detect and hash fields when required.
Fields commonly hashed include:
- Customer Email
- Partner identifiers that contain PII
Only fields required by Impact.com will be hashed.
---
## Object sync configuration
Choose **Objects** when syncing **catalogs** or **actions**.
After selecting **Objects**, choose the Impact.com object:
- **Catalog**
- **Action**
Each object type has its own configuration sections.
---
### Catalog object configuration
Choose **Catalog** when syncing product catalog data such as SKUs, descriptions, or prices.
#### 1. Record update mode
Choose how Hightouch should write records:
- **Update** — update existing catalog items
- **Insert** — create new items without updating existing ones
Use **Update** if you want ongoing sync behavior.
#### 2. Catalog ID
Choose which catalog to sync to.
You can:
- Select a catalog returned by Impact.com
- Enter a static ID
- Toggle **Use column** to map a dynamic catalog ID
If your catalog does not appear, ensure it is activated and available in Impact.com.
#### 3. Record matching
Define how Hightouch should match records in your model to the records in Impact.com.
Impact.com typically matches catalog entries using an identifier such as:
- `Item ID`
- `SKU`
- `Product Code`
The left side of your mapping is your model field; the right side is the corresponding Impact.com identifier field.
Example:
| Model field | Impact.com field |
| ----------- | ---------------- |
| `sku` | `Item ID` |
Incorrect identifiers will cause mismatched updates or unintended new record creation.
#### 4. Catalog field mapping
Map your model columns to Impact.com catalog fields.
Common catalog attributes include:
- Product name
- Category
- Price
- Brand
- Description
- Primary image URL
- Inventory status
- Custom metadata fields
You may add as many mappings as needed.
#### 5. Handling deleted records
Choose what happens when a row disappears from your query result:
- **Do nothing** — Hightouch leaves the Impact.com record unchanged
- **Delete the Impact.com record**
- This permanently removes the catalog entry from Impact.com
- This cannot be undone
Choose deletion only if you want your catalog in Impact.com to stay in sync with your dataset.
---
### Action object configuration
Choose **Action** when syncing Impact.com Actions (non-conversion activities).
#### 1. Sync modes
The following modes are available:
**Operation**
- **Update** — update existing action records
- **Reverse** — reverse previously submitted action data
- **Insert** — create new action records (if supported)
**Match mode**
Choose which identifiers are used to match action records, such as:
- `Action ID`
- Custom external IDs
These must correspond to identifiers configured in your Impact.com action settings.
#### 2. Action field mapping
Map model fields to the relevant Impact.com action fields.
Common fields include:
- Action ID
- Action amount
- Status
- Event/action code
- Custom metadata
- Any additional fields your Impact.com account is configured to accept
#### 3. Disposition code (metadata)
You can optionally map a Disposition Code, which determines how Impact.com categorizes the action (e.g., approved, rejected, pending).
You can:
- Select a static value
- Toggle **Use column** to map a value from your model
---
## InMobi
**URL:** https://hightouch.com/docs/destinations/inmobi
**Description:** InMobi help brands understand, identify, engage and acquire consumers.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------- | ---------------------------------- | -------------------- |
| **Audiences** | Create and update custom audiences | All |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to InMobi
This integration requires prior approval from the InMobi team and the Hightouch
team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your InMobi rep who will verify approval. Once we have confirmed, we will
unlock the integration for you to connect.
To connect, input the following provided by your InMobi rep:
- Access key
- Secret key
- Bucket name
### Syncing custom audiences
Hightouch will upload a file to the specified bucket with a naming pattern agreed upon by Hightouch and InMobi for your use case.
#### User identifiers
To identify which users to add or update in an audience, select model columns
and the corresponding InMobi fields. You can match on any of the following user
fields:
- MAID (preferred)
- Email
- IP Address
- Ramp ID
- ID5
#### Handling PII and hashing
InMobi expects **email** values to be hashed. For audience syncs, by default, Hightouch automatically hashes first-party user data fields before sending requests to Reddit.
If the data in your model is already hashed, you can disable PII hashing to successfully send data to Reddit.
#### Delete behavior
The only supported sync mode is `all` mode, meaning that each sync run will refresh
the entire audience. Users removed from your audience will automatically be dropped
with each refresh - only present users will be in InMobi's audience.
## Tips and troubleshooting
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Sync alerts
---
## Insider InOne
**URL:** https://hightouch.com/docs/destinations/insider
**Description:** InOne brings together all technologies marketers need to deliver personalized experiences under a single, highly usable platform. InOne makes the technology so seamless that it truly blends into the daily lives of marketers, fully empowering them behind the scenes.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes |
| ----------------- | -------------------------------------------------- | ---------------------- |
| **Users** | Sync data from any source to Insider users | Upsert, Update, Insert |
| **Events** | Sync data from any source to Insider events | Insert |
| **Segments** | Sync data from any source to Insider segments | Add, Remove |
| **Subscriptions** | Sync data from any source to Insider subscriptions | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Insider InOne
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Insider** and click **Continue**. You can then authenticate Hightouch to **Insider**.
Enter the following required fields into Hightouch:
- **Partner name**:This is your partner name. Navigate to InOne > InOne Settings > [Account Preferences](https://academy.useinsider.com/docs/account-preferences) to copy your partner name. The partner name should be lowercase.
- **Request token**:This key is required to authorize your request. Refer to [API Authentication Tokens](https://academy.useinsider.com/docs/api-authentication-tokens#generate-api-key) to generate your token.
## Sync configuration
Once you've set up your Insider destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Insider destination you want to sync to.
### Syncing users
Sync data from any source to Insider users.
#### Record matching
To match rows from your model to user in Insider, you need to select a model column and corresponding Insider field. You can match on any of the following Insider fields:
- **Email**
- **Phone number**
- **UUID**
In **Insert** mode, Insider automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync user fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom user fields.
If you send data for a custom field that doesn't exist, Hightouch adds the field.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | --------------------------------------------------------- |
| **Do nothing** | Keep the user in Insider with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the user in Insider |
### Syncing events
Sync data from any source to Insider events.
#### Record matching
You can match rows from your model to event in Insider on any column in your model and any field in Insider. Ensure the data types of the model column and Insider field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
### Syncing segments
Sync data from any source to Insider segments.
#### List field
The name of an array field that has already been created in Insider InOne. This field will contain a list of segments the user belongs to.
#### Segment name
The name of the segment the sync will handle subscriptions for. This segment name will be added to or removed from the list field entered above as users' subscription statuses change.
#### Record matching
To match rows from your model to segment in Insider, you need to select a model column and corresponding Insider field. You can match on any of the following Insider fields:
- **Phone number**
- **UUID**
#### Field mapping
Hightouch lets you sync segment fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default segment fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
### Syncing subscriptions
Sync data from any source to Insider subscriptions.
#### Record matching
To match rows from your model to subscription in Insider, you need to select a model column and corresponding Insider field. You can match on any of the following Insider fields:
- **Email**
- **Phone**
#### Field mapping
Hightouch lets you sync subscription fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default subscription fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Intercom
**URL:** https://hightouch.com/docs/destinations/intercom
**Description:** Sync product and CRM data to Intercom so that your support team has all the information they need about customers
**Section:** Destinations
## Supported syncing
Hightouch can create and update both contacts and company objects within the [Intercom Contacts Model](https://developers.intercom.com/intercom-api-reference/reference#contacts-model) and [Intercom Company Model](https://developers.intercom.com/intercom-api-reference/reference#company-model) respectively.
|Sync Type|Description|Supported Sync Modes | API Reference|
|--|--|--|--|
|**Contact objects**|Sync data from any source to Intercom contacts.|Upsert, Update, Archive|[Contact docs](https://developers.intercom.com/docs/references/rest-api/api.intercom.io/Contacts/CreateContact)|
|**Company objects**|Sync data from any source to Intercom company objects.|Upsert, Update|[Company docs](https://developers.intercom.com/intercom-api-reference/reference/createorupdatecompany)|
|**Events**|Sync records as events to Intercom. This is often in the form of a track call.|Insert|[Event docs](https://developers.intercom.com/intercom-api-reference/reference/createdataevent)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Intercom
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Intercom** and click **Continue**. You can then authenticate Hightouch to Intercom with OAuth. Click **Log in to Intercom** and log into your Intercom account.
Make sure you select the region that corresponds with your Intercom workspace from the *Data host region:* dropdown:
Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Intercom destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Intercom destination you want to sync to.
### Syncing contacts
Contact objects allow you to represent and provide information on individual users. All contact objects are represented as users or leads as marked by the role attribute.
When looking at the [People list](https://www.intercom.com/help/en/articles/318-the-people-company-and-account-list-explained#the-people-list) or when [creating an audience](https://www.intercom.com/help/en/articles/2848161-create-more-targeted-audiences-with-and-or-rules), the number of contacts (users) displayed might differ from the number of rows processed by your sync. This is because Intercom [automatically filters out unsubscribed contacts](https://www.intercom.com/help/en/articles/5537581-include-unsubscribed-contacts-in-your-email-audience#h_c8fcd619b4) from your contact list.
#### Record matching
To match rows from your model to objects in Intercom, you need to select a model column and corresponding Intercom field. You can match on any of the following Intercom fields:
- Email
- External User ID
#### Field mapping
You can map data from any of your model columns to contact fields in Intercom. Ensure the data type of your model column matches the data type of the field you want to sync to.
#### Associations
You can associate companies to contacts using this configuration section. To do so, select a column from your model to match the associated object. Hightouch uses this column to look up the associated object's ID and make the connection between the two objects.
For example, if you wanted to associate a company to your contact, you select **Company** from the right dropdown. Then, select a field to match the company on, for example Company ID.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the contact in Intercom |
| **Clear fields** | Keep the contact in Intercom but clear the synced fields |
| **Remove from list** | Remove the contact from Intercom |
### Syncing companies
Company objects allow you to represent and provide information on organizations using your product. Each company has its own description and can be associated with contacts.
Company objects' Company ID can't be updated. If you need to change a company object's Company ID you need to delete it and recreate it with a new Company ID.
When syncing company objects, make sure they have at least one [associated contact](#associations-1). Otherwise, the company won't be visible in Intercom.
#### Record matching
To match rows from your model to records in Intercom, you need to select the model column that contains values that match the **Company ID** field. If there isn't a company with the matching Company ID, Hightouch creates a new company.
#### Field mapping
You can map data from any of your model columns to company fields in Intercom. Ensure the data type of your model column matches the data type of the field you want to sync to.
#### Associations
You can associate contacts to companies using this configuration section. To do so, select a column from your model to match to the associated object. Hightouch uses this column to look up the associated object's ID and make the connection between the two objects.
For example, if you wanted to associate a company to your contact, you select **Contact** from the right dropdown. Then, select a field to match the contact on, for example `user_id`.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the company in Intercom |
| **Clear fields** | Keep the company in Intercom but clear the synced fields |
| **Remove from list** | Remove the company from Intercom |
### Syncing events
When syncing events, Hightouch treats any records added to [your source](/getting-started/concepts) as new events and sends them to Intercom when your sync runs.
#### Event name
Providing an event name is required to log an event in Intercom. You can either provide a static value or select to **use a column** from your model.
Refer to [Intercom's event docs](https://developers.intercom.com/intercom-api-reference/reference/createdataevent) for guidance on event naming.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server.
#### Field mapping
You can select model columns to sync to native and custom event parameters in the field mapping section.
## Tags
You can manage tags on companies and contacts in Intercom using Hightouch.
The data in your source should contain the name of one or more tags for each row.
For multiple tags, your source can contain an array or a comma-separated string of tag names.
#### Tag Modes
Syncs can have different tag modes based on how you would like to handle tags.
##### **Create added tags**
When there are tags added to the source, create the tags in Intercom. If the tag does not already exist, it will be created in Intercom. If the tag already exists, it will be added to the contact or company. Do nothing for tags removed from the source.
##### **Overwrite existing tags**
When there are changes to tags in the source, **remove all tags** in Intercom before creating the tags present in the source. If there are no changes to the tags, do nothing. Because this removes all tags in Intercom, this may delete data not authored by Hightouch.
##### **Delete removed tags and create added tags**
When there are changes to tags in the source, removes tags in Intercom that have been removed from the source, and creates tags that have been added to the source.
Read more about how Intercom handles tags [here](https://developers.intercom.com/docs/references/rest-api/api.intercom.io/tags).
## Tips and troubleshooting
### Common errors
{/* */}
#### 429 Rate limit exceeded
The full error message looks something like this:
```
Custom data attributes could not be created: StatusCodeError: 429 -
{
"type": "error.list",
"request_id": "003drrt76xkssahyhgd41",
"errors":[{
"code": "rate_limit_exceeded",
"message": "Exceeded rate limit of 167 in 10_seconds"
}]
}
```
This error means you've exceeded [Intercom's rate limit](https://developers.intercom.com/intercom-api-reference/reference/rate-limiting).
One reason for this could be running two Hightouch syncs to Intercom simultaneously.
Try staggering the syncs so that they don't overlap.
{/* */}
#### Missing companies
If your Hightouch sync processes successfully, but you don't see the companies
in Intercom, check if there is at least one contact [associated to the company](#associations-1). Intercom only displays companies with at least one associated contact. For more information, read the Intercom [company API documentation](https://developers.intercom.com/intercom-api-reference/reference/create-or-update-a-company).
### Live debugger
### Sync alerts
---
## iSpot.tv
**URL:** https://hightouch.com/docs/destinations/ispot-tv
**Description:** Empower your brand, marketing, and advertising teams to make data-driven decisions with iSpot.tv’s TV measurement solutions.
**Section:** Destinations
## Overview
Onboard your first-party audiences to iSpot.tv to provide visibility into tune-in
metrics across your audience segments and illuminate unique media buying
opportunities.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| --------------- | ---------------------------------- | -------------------- |
| **Audiences** | Create and update custom audiences | All |
| **Conversions** | Upload TV conversion data | Insert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to iSpot.tv
This integration requires prior approval from the iSpot.tv team and the Hightouch
team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your iSpot rep who will verify approval. Once we have confirmed, we will unlock
the integration for you to connect.
To connect, input the `Advertiser Name` provided by your iSpot rep.
### Syncing custom audiences
You can input the name of the audience you plan on delivering to iSpot.
#### User identifiers
To identify which users to add or update in an audience, select model columns
and the corresponding iSpot fields. You can match on any of the following user
fields:
- Email (Sha256 hashed)
- MAID
- IP Address
#### Delete behavior
The only supported sync mode is `all` mode, meaning that each sync run will refresh
the entire audience. Users removed from your audience will automatically be dropped
with each refresh - only present users will be in iSpot's audience.
### Syncing conversions
Sync data from any source to iSpot.tv as conversion events.
#### Field mapping
Hightouch lets you sync event properties via field mappings.
## Tips and troubleshooting
### Refresh rate
iSpot.tv has a max refresh rate of monthly. You should not have your iSpot syncs
running more than every 4 weeks.
### Matched device count
Match rates are not available to preview in the Hightouch platform. Contact your
iSpot rep for insights into match rates and audience reach.
Keep in mind that audience counts reflect the number of devices match, not
the number of users matched. Also keep in mind that matching occurs at the
household level, and that a single household may own multiple devices.
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Sync alerts
---
## Iterable
**URL:** https://hightouch.com/docs/destinations/iterable
**Description:** Run better email, push, and SMS campaigns on Iterable with up-to-date customer data from your data warehouse
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API reference |
|--------------------------------------------------------|--------------------------------------------------------------------------------|----------------------|---------------------------------------------------------------------------------------|
| [**Users**](#syncing-users) | Sync data from any source to your Iterable users | Upsert, Update | [User docs](https://api.iterable.com/api/docs#users_bulkUpdateUser) |
| [**Events**](#syncing-events) | Sync records as events to Iterable. This is often in the form of a track call. | Upsert | [Event docs](https://api.iterable.com/api/docs#events_track) |
| [**User shopping cart items**](#updating-shopping-cart-items) | Update a user's shopping cart and create an event for it | Insert | [Commerce docs](https://api.iterable.com/api/docs#commerce_updateCart) |
| [**Catalog items**](#syncing-catalogs) | Sync records as catalog items in Iterable. | Upsert | [Catalog docs](https://api.iterable.com/api/docs#catalogs_partialUpdateCatalogItem) |
| [**Static list**](#syncing-segments) | Subscribe and unsubscribe users from lists | Add, Remove | [List docs](https://api.iterable.com/api/docs#lists_subscribe) |
| [**Update email address**](#updating-user-email-addresses) | Update a user's email address | Update | [User email update docs](https://api.iterable.com/api/docs#users_updateEmail) |
| [**Other user actions**](#user-api-actions) | Perform API actions like delete, forget and unforget users. | -- | [User docs](https://api.iterable.com/api/docs#users_delete) |
| [**Subscriptions**](#subscribing-users) | Subscribe and unsubscribe users from lists, message types and message channels | Upsert, Remove | [Subscription docs](https://api.iterable.com/api/docs#subscriptions_Bulk_subscription_action) |
| [**Trigger email**](#trigger-email) | Send an email to a user | -- | [Email docs](https://api.iterable.com/api/docs#email_target) |
| [**Trigger push**](#trigger-push) | Send a push to a user | -- | [Push notification docs](https://api.iterable.com/api/docs#push_target) |
| [**Trigger journey (formerly workflow)**](#trigger-journey) | Trigger a journey for a user | -- | [Journey docs](https://api.iterable.com/api/docs#workflows_triggerWorkflow) |
| [**Merge users**](#merge-users) | Merge two Iterable user profiles into one. | Insert | [Merging user profiles](https://support.iterable.com/hc/en-us/articles/33744314673172-Merging-User-Profiles) |
| [**Register device tokens**](#register-device-tokens) | Register a device to a user using a token | Insert | [Device token docs](https://api.iterable.com/api/docs#users_registerDeviceToken) |
## Connect to Iterable
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Iterable** and click **Continue**. You can then authenticate Hightouch to Iterable with an Iterable **API key** and [**Project Type Identifier**](https://support.iterable.com/hc/en-us/articles/9216719179796#checking-the-unique-identifier-for-a-project).
You can follow [Iterable's instructions](https://support.iterable.com/hc/en-us/articles/360043464871-API-Keys-#creating-api-keys) for how to create an API key. Your API key needs to have [server-side permissions](https://support.iterable.com/hc/en-us/articles/360043464871-API-Keys-#server-side-keys) for all sync types above, except for **Register device tokens** which needs to have [mobile client-side permissions](https://support.iterable.com/hc/en-us/articles/360043464871-API-Keys#client-side-keys). Hightouch does not support mobile client-side permissions using JSON web tokens (JWT).
To see your project type identifier, go to **Settings > Project Settings** and see the identifier(s) underneath the project's cluster ID.
Check out Iterable's [docs](https://support.iterable.com/hc/en-us/articles/9216719179796#checking-the-unique-identifier-for-a-project) for more information.
## Sync configuration
Once you've set up your Iterable destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Iterable destination you want to sync to.
### Syncing users
Sync Iterable users and user properties with data from any source.
#### Record matching
You can either use **email** or **user ID** to match rows from your model to users in Iterable.
In general, it's best to select the one that corresponds to the [project type](https://support.iterable.com/hc/en-us/articles/9216719179796) you selected when first setting up your Iterable instance.
There's one exception when you _shouldn't_ match your record matching column to your project type:
If you have an **email-based** Iterable project and you're using **update** mode, selecting **user ID** for record matching can increase sync speed.
You should do this if you're confident that your user IDs are unique. (In email-based projects, Iterable doesn't require uniqueness for user IDs.)
Ensure you [set your project identifier](#unset-project-identifier) appropriately.
If you select user ID for record matching in email-based projects, Hightouch uses the [bulk user update endpoint](https://api.iterable.com/api/docs#users_bulkUpdateUser) with the `preferUserId` parameter.
The `preferUserId` parameter dictates whether new users should be created if the request includes a `userId` that doesn't yet exist in the Iterable project.
This allows Hightouch to update only existing users without first having to perform a [lookup](https://api.iterable.com/api/docs#users_getUserById_0).
Iterable only respects the `preferUserId` parameter in API calls for email-based projects.
For more information on how Iterable's API works differently according to project-type, refer to [Iterable's docs](https://support.iterable.com/hc/en-us/articles/9216719179796#email-based-projects).
**Upsert** mode is performant regardless of the project-type and column used for record matching.
#### Field mapping
You can sync columns from your model to Iterable user's data fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
When creating new fields, Iterable automatically generates a schema for that field based on the first data sent.
You cannot change the schema once it's set. See [Iterable's article](https://support.iterable.com/hc/en-us/articles/206430145-User-Profile-Fields-#data-types-for-user-profile-fields) for more information.
#### Nested objects and object arrays
Iterable supports user and event fields containing [JSON objects](https://support.iterable.com/hc/en-us/articles/208183076-Field-Data-Types#objects) or [arrays of objects](https://support.iterable.com/hc/en-us/articles/208183076-Field-Data-Types#arrays).
These objects and object arrays are commonly used to support "entities" related to each user—for example, devices, medications, pets, etc.—especially when marketing communication needs to include metadata about these related entities.
For example, say you want to sync an array of a customer's pets to be an attribute named `pets` in Iterable:
```json
{
"email": "docs@hightouch.com",
"dataFields": {
"pets": [
{
"name": "Nacho",
"color": "Ginger",
"age": 1,
"favoriteToy": "Fuzzy spring"
},
{
"name": "Rami",
"color": "Brown Tabby",
"age": 2,
"favoriteToy": "Feather wand"
}
]
}
}
```
Hightouch's [inline mapper](/syncs/mapping-data#inline-mapping) is purpose-built to create nested JSON objects and arrays like this.
Refer to the [data format requirements](/syncs/mapping-data#required-data-format) and [array creation](/syncs/mapping-data#create-an-array) instructions for details.
#### Delete behavior
You have these options for how to treat records that leave your model's query results:
| Behavior | Description |
| -------------- | ----------------------------------------------- |
| **Do nothing** | Keep the users in Iterable unchanged |
| **Clear** | Keep the user in Iterable and clear data fields managed by Hightouch |
| **Delete** | Delete the user from Iterable |
### Syncing events
The integration lets you sync [custom events](https://api.iterable.com/api/docs#commerce_trackPurchase), [purchase events](https://api.iterable.com/api/docs#commerce_trackPurchase), and `updateCart` events to Iterable.
This section describes custom and purchase events. Skip to the [syncing update shopping cart items](#updating-shopping-cart-items) for more information on this sync type.
When syncing custom and purchase events, the integration inserts new events into Iterable and lets you modify existing events as long as you [provide their event ID](#additional-track-configuration).
#### Event name
Providing an event name is required to send an event to the Iterable API.
You can either provide a static value or select to **use a column** from your model.
Hightouch syncs the static or column value as the [`eventName` body parameter](https://api.iterable.com/api/docs#events_trackBulk) the API requires.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred.
If this field is empty, Iterable uses the time the event arrives at the server.
#### Additional track configuration
You can also optionally include this information as part of your event information:
- Name of the campaign associated with the event: choose a static value or select to **use a column** from your model.
- Event template: choose a static value or select to **use a column** from your model.
- Event ID: select a column from your model. You **must** include this configuration if you want to modify existing events.
If you don't select a column from your model to map as the event ID,
Hightouch considers it a new event and Iterable generates a new event ID.
#### Field mapping
You can use field mapping to add context to your events.
You must include either an **email** or **user ID** in your mappings.
Which you include depends on [how your project identifies users](https://support.iterable.com/hc/articles/204780579#identifying-users).
If you provide both email and user ID, your project type takes precedence.
The integration uses these fields for identifying the user associated with this event.
For purchase events, you also need to provide the shopping cart items as **Items** and the **Total** order dollar amount in your mappings.
You can also include event data fields, including custom ones. Hightouch sends any custom fields you include as part of the in the `dataFields` property of event's body.
### Updating shopping cart items
This sync mode both updates the [`shoppingCartItems`](https://api.iterable.com/api/docs#commerce_updateCart) field on a user's profile and tracks `updateCart` events in Iterable.
The integration inserts new events into Iterable when they first appear in your query results.
#### Field mapping
You must include shopping cart items as **Items** and either an **email** or **user ID** in your mappings.
Which you include depends on [how your project identifies users](https://support.iterable.com/hc/articles/204780579#identifying-users).
If you provide both email and user ID, your project type takes precedence.
The integration uses these fields for identifying the user associated with this event.
You can also include user data fields for the user who updated their cart.
### Syncing catalogs
You can keep your Iterable catalogs up-to-date with **Upsert** mode.
Begin by selecting the Iterable catalog you want to sync items to.
#### Record matching
To match rows from your model to catalog items, you need to select the model column that contains values that match the **Unique Product ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
The Unique Product ID must contain only alphanumeric characters and dashes and has a maximum length of 255 characters.
See Iterable's [API documentation](https://api.iterable.com/api/docs#catalogs_bulkUpdateCatalogItems) for more information.
#### Field mapping
Once you've selected a catalog, Hightouch automatically pulls existing fields from your Iterable catalog for you to map data into.
#### Delete behavior
You have these options for how to treat records that leave your model's query results:
| Behavior | Description |
| -------------- | ----------------------------------------------- |
| **Do nothing** | Keep the catalog items in Iterable unchanged |
| **Delete** | Delete the catalog item in Iterable |
### Syncing segments
Sync audiences into your Iterable [static lists](https://support.iterable.com/hc/en-us/articles/204780549-Static-and-Dynamic-Lists-Overview-) of users. Hightouch automatically removes users from the Iterable list when they leave the query result.
#### User identifiers
To identify which contacts to add or update in a segment, select a model column and the corresponding Iterable field.
You can match on either **Email** or **User ID**.
It's best to select the one that corresponds to the [project type](https://support.iterable.com/hc/en-us/articles/9216719179796) you selected when first setting up your Iterable instance.
If you provide both email and user ID, your project type takes precedence.
#### Field mapping
You can sync columns from your model to user's default and custom data fields.
If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
#### Create or use an existing list
Your can create a new list or select an existing one.
If you choose to create a new list, Hightouch creates it on the initial sync run and then keeps it updated on existing sync runs.
You can provide a name for the new list, or Hightouch defaults to the name of the model used to create the sync.
### Updating user email addresses
Use this sync mode to update existing users' email addresses in [email-based projects](https://support.iterable.com/hc/articles/9216719179796).
To update an `email` in a userID-based or hybrid project, use the [user sync](#syncing-users) type.
#### Record matching
You can use either **email** or **user ID** to match rows from your model to users in Iterable, regardless of your Iterable project type.
If you match by email, the update call is only made the first time the record appears in your query results.
Matching by user ID updates the email the first time the record appears in your query results and every time changes are detected.
#### Field mapping
The only field you can include in field mapping is the new email.
If you want to update other user data fields, use the [user sync type](#syncing-users).
### User API actions
Hightouch lets you perform the following actions on your Iterable users:
- [Delete](https://support.iterable.com/hc/en-us/articles/360029174171-Responding-to-GDPR-Requests-#deleting-users)
- [Forget](https://support.iterable.com/hc/en-us/articles/360029174171-Responding-to-GDPR-Requests-#forgetting-users)
- [Unforget](https://support.iterable.com/hc/en-us/articles/360029174171-Responding-to-GDPR-Requests-#unforgetting-users)
This destination only triggers delete, forget, or unforget when records *first* appear in your model.
In other words, if a row with the same [primary key](/getting-started/concepts#unique-primary-key-requirement) appears in the model's query results in multiple syncs, Hightouch only triggers the selected API action on it the first time.
#### Record matching
You can either use **email** or **user ID** to match rows from your model to users in Iterable.
It's best to select the one that corresponds to the [project type](https://support.iterable.com/hc/en-us/articles/9216719179796) you selected when first setting up your Iterable instance.
### Subscribing users
Set all existing users from a query to be subscribed to a specific list, message type or message channel.
Hightouch automatically unsubscribes users from the target when they leave the query result.
### Trigger email
Send an email to a user. Use an existing campaign that is triggerable and running.
### Trigger push
Send a push notification to a user. Use an existing campaign that is triggerable and running.
### Trigger journey
Iterable previously called these flows **Workflows**, and the API still uses workflow terminology (for example, `workflowId` and the `triggerWorkflow` endpoint).
Trigger a Journey for a user in Iterable. This behaves similarly to triggering an email or push campaign. You must supply a user identifier (email or userId), and optionally include Journey-specific trigger properties if configured in Iterable.
### Merge users
Use this sync type to merge one Iterable user profile into another. Each row in your model represents a single merge operation.
Iterable removes the source user after the merge and retains the destination user.
See Iterable’s documentation on [Merging User Profiles](https://support.iterable.com/hc/en-us/articles/33744314673172-Merging-User-Profiles) for details on how merges are handled.
Merges in Iterable are irreversible.
#### Record matching
This sync type requires two identifiers:
- **Source user** — the user to merge *from*
- **Destination user** — the user to merge *into*
For each, select a model column and map it to either **email** or **userId** in Iterable. Both identifiers must be present in every row.
#### Field mapping
This sync type does not support field mapping.
The identifiers you configure are sent to Iterable as the `fromUser` and `intoUser` values.
### Register device tokens
Register mobile device tokens to enable push notifications. Use either `email` or `userId` (but not both) and a `device`
object that contains a valid `token`, `platform` and `applicationName`.
#### User identifiers
To identify which contacts to add or update in a segment, select a model column and the corresponding Iterable field.
You can match on either **Email** or **User ID**, regardless of your project type.
## Tips and troubleshooting
### Unset project identifier
Before March 2023, Hightouch didn't require you to set your Iterable project identifier [when connecting to Iterable](#connect-to-iterable).
Setting it may enhance sync performance if you're using **update** mode with the **users** sync type for an email-based project.
See the [record matching section](#record-matching) for more information.
You can set the project-type from the configuration tab of your Iterable destination in Hightouch.
### Common errors
#### Fewer records in Iterable than synced
If you see a discrepancy between the number of records that have passed through your sync and the number of records displaying in Iterable, check your data set for duplicates. Iterable automatically dedupes records based on primary key so the number of records in Iterable might be less than the number of records in your Hightouch sync if your sync unintentionally contains duplicates.
{/* */}
#### 413 - Request Entity Too Large
If you see this error or notice that your syncs are slow, you can try lowering the batch size in your sync configuration.
#### 400 - Request does not have the same data types as the data previously submitted
{/* */}
The full error message looks something like this:
```json
{
"msg":"Project 13691: The request does not have the same data types as the data previously submitted, or the input value is not within the correct range. Field 'date_field' already exists for type 'user' and has a data type of 'date' but possible types 'string, keyword' in the request.",
"code":"RequestFieldsTypesMismatched",
"params":
{
"validationErrors":
{
"date_field":
{
"incomingTypes":["string","keyword"],
"expectedType":"date",
"category":"user",
"offendingValue":["2023-04-19 12:15:30","0000-00-00 00:00:00"],
"_type":"UnexpectedType"
}
}
}
}
```
The first value in the `offendingValue` array is associated with the first record in the batch sent to Iterable. The second value in the array is the value causing the issue. One instance of this value (`"0000-00-00 00:00:00"`) causes the entire batch of records being sent to Iterable to fail.
To resolve the error, check for instances of (`"0000-00-00 00:00:00"`) in the `date_field` column of your model and replace these values with `null` or a valid date.
### Live debugger
### Sync alerts
---
## Jira
**URL:** https://hightouch.com/docs/destinations/jira
**Description:** Keep your team's projects and tickets up-to-date with fresh customer data
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
| ---------- | ------------------------------------------------------------------- | ---------------------- | ------------------------------------------------------------------------------------------------------- |
| **Issues** | Sync data from any source to Jira issues or Product Discovery ideas | Upsert, Update, Insert | [Issues](https://developer.atlassian.com/cloud/jira/platform/rest/v2/api-group-issues/#api-group-issues)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Jira
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Jira** and click **Continue**. You can then authenticate Hightouch to Jira by entering your **Jira Domain**, **Username**, and **API Token** from your Jira account:
- the **Jira Domain** is the text before `.atlassian.net` in the URL,
- **Username** is the email you use to login to Jira,
- and if you don't already have an **API Token**, you can create one in your Jira account by navigating to **Account Settings** > **Security** > **Create and manage API tokens**.
## Sync configuration
Once you've set up your Jira destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Jira destination you want to sync to.
### Syncing issues and ideas
Hightouch supports syncing to Jira [issues](https://support.atlassian.com/jira-software-cloud/docs/what-is-an-issue/) and [Product Discovery ideas](https://support.atlassian.com/jira-product-discovery/docs/what-is-an-idea/). If you want to sync to Product Discovery ideas, select a [**Product Discovery Project**](https://www.atlassian.com/software/jira/product-discovery) in your sync configuration and use [Insert](#supported-syncing) mode.
#### Record matching
Upsert and Update mode are only supported for Jira issues. They require you to create a **custom
field** in Jira to use for record matching in Hightouch. This field needs to be set to the [**Text Field (read only)**](https://support.atlassian.com/jira-cloud-administration/docs/custom-fields-types-in-company-managed-projects/#Advanced-custom-field-types) type.
To create a **custom field** in Jira:
- Navigate to **Settings** > **Issues** > **Custom Fields**.
- Click **Create custom field** and select the [**Text Field (read only)**](https://support.atlassian.com/jira-cloud-administration/docs/custom-fields-types-in-company-managed-projects/#Advanced-custom-field-types) field type.
Ensure all issue fields and custom fields that you want to sync to are
included on your project's screen. To add custom fields to the
screen, navigate to the custom field, click the three-dot menu and then **Associate to Screens**.
#### Field mapping
You can map model columns to all Jira issue fields that are available to the **Project** and **Issue Type** you selected in your sync configuration. To sync successfully, ensure the **Description** field is in [Atlassian Document Format](https://developer.atlassian.com/cloud/jira/platform/apis/document/structure/) and that these other fields are of these types:
- `parent` > `id`
- `priority` > `name` (for example, Highest, High, Medium, Low, or Lowest)
- `creator` > `accountId`
- `assignee` > `accountId`
- `reporter` > `accountId`
Hightouch doesn't currently support syncing to [Status fields](https://confluence.atlassian.com/adminjiraserver/defining-status-field-values-938847108.html), so this needs to be done manually in the Jira dashboard. Please {" "} if you're interested in this feature.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Kameleoon
**URL:** https://hightouch.com/docs/destinations/kameleoon
**Description:** Empower rich personalization
**Section:** Destinations
## Overview
In Hightouch, you can easily sync data from any source to visitors in Kameleoon, much like with other platforms such as Salesforce and Meta Ads. Simply connect your data source, specify the syncing models, and create them directly within Hightouch using SQL, our user-friendly audience builder, or compatible tools like dbt or Looker.
## Supported syncing
| Type | Description | Supported Sync Modes |
| ---------------------- | -------------------------------------------------- | -------------------- |
| **Visitor properties** | Sync data from any source to visitors in Kameleoon | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Kameleoon
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination**
button. Select **Kameleoon** and click **Continue**.
Enter the following required fields into Hightouch:
- **Project ID** (Site Key)
- **Client ID**
- **Client Secret**
Follow [these instructions](https://help.kameleoon.com/api-credentials/) to retrieve your API keys.
## Sync configuration
Once you've set up your Kameleoon destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and
the Kameleoon destination you want to sync to.
#### Record matching
You must identify a unique key from your model to use as a property key in Kameleoon. This is often a visitor code from
Kameleoon, but any other unique external ID is supported.
#### Field mapping
Select any properties to sync to Kameleoon. These properties can then be used in Kameleoon to create custom segments.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Kayzen
**URL:** https://hightouch.com/docs/destinations/kayzen
**Description:** Kayzen is a mobile DSP that enables app developers, agencies, media buyers, and brands to run programmatic acquisition, retargeting, and branding campaigns.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
--------------|-----------------------------------------------|----------------------|-----------------------------------------------------------------------------------
**Audiences** | Sync data from any source to Kayzen audiences | Add, Remove | [Audiences docs](https://developers.kayzen.io/reference/audience-creation-via-api)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Kayzen
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Kayzen** and click **Continue**. You can then authenticate Hightouch to **Kayzen**.
Enter the following fields into Hightouch:
- **API key**
## Sync configuration
Once you've set up your Kayzen destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Kayzen destination you want to sync to.
### Syncing audiences
Sync data from any source to Kayzen audiences.
#### Record matching
To match rows from your model to audiences in Kayzen, you need to select a model column and corresponding Kayzen field. You can match on any of the following Kayzen fields:
- **IDFA**
- **GAID**
#### Field mapping
Hightouch lets you sync audience fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default audience fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Klaviyo
**URL:** https://hightouch.com/docs/destinations/klaviyo
**Description:** Power e-commerce focused email marketing with your warehouse data in Klaviyo
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API reference |
| --------------------- | ----------------------------------------------------------------------------- | -------------------- | ----------------------------------------------------------------------------------------------- |
| **User Profiles** | Sync data from any source to your Klaviyo users. | Upsert | [Batch profile docs](https://developers.klaviyo.com/en/reference/spawn_bulk_profile_import_job) |
| **Events** | Sync records as events to Klaviyo. This is often in the form of a track call. | Add | [Track docs](https://developers.klaviyo.com/en/reference/create_event) |
| **List** | Add and remove users from lists without explicit consent. | Add, Remove | [List docs](https://developers.klaviyo.com/en/reference/create_list_relationships) |
| **Subscription List** | Subscribe or unsubscribe profiles to email marketing, SMS marketing, or both, | Add, Remove | [Subscribe Profiles docs](https://developers.klaviyo.com/en/reference/subscribe_profiles) |
The Klaviyo destination has been updated to latest version of the
[API](https://developers.klaviyo.com/en/reference/api_overview). There have
been several changes that may affect syncs. See them [here](#api-changes).
## Setup
Hightouch supports authenticating Klaviyo with either OAuth or API key. When setting up the Klaviyo integration, select which you prefer to use.
### OAuth
At the destination configuration page, select OAuth and click **Log in to Klaviyo**. Allow the Hightouch app access to your account's information,
including access to **Events**, **List**, **Profiles**, and **Subscriptions**.
### API key
To find your API keys, navigate to the [API keys page](https://www.klaviyo.com/settings/account/api-keys) and create a private API key.
Hightouch may require the following permissions based on which Sync Types you will be using.
_Note: You cannot edit an API key after it's created._
| Sync Type | Permissions (Read/Write) |
| --------------------- | ------------------------------ |
| **Events** | Events, Profiles |
| **User Profiles** | Profiles |
| **List** | Lists, Profiles |
| **Subscription List** | Lists, Profiles, Subscriptions |
### Record matching
Records can be matched from your source to your Klaviyo workspace by using one of email, phone number, or Klaviyo unique ID.
### Field mapping
You can sync columns from your source to Klaviyo's default and custom fields.
## Syncing events
Hightouch supports syncing events to Klaviyo and assigning them to a user.
### Track configuration
Hightouch allows you to choose the event name and event timestamp for a track event.
Note: Hightouch accepts a normal date time format for the timestamp, and automatically convert it to the format that Klaviyo requires.
### Assigning events to profiles
Hightouch allows you to choose how you assign events to profiles. This is done by matching a column to a Klaviyo user identifier such as phone number or email.
### Field mappings
Hightouch allows you to pass data to the event properties of a Klaviyo event.
## Syncing lists
Hightouch supports syncing profiles to Klaviyo Lists.
Syncing profiles using emails or phone numbers to lists (regular and subscription lists) will automatically create any profiles that do not exist.
Note: When adding profiles to regular lists a [Batch Profile Job](https://developers.klaviyo.com/en/reference/spawn_bulk_profile_import_job) is used.
### Klaviyo lists
Hightouch supports automatically creating a new list for your sync. You can specify a custom name for this list, or Hightouch will use the name of the model.
Here, the model name List 1 will be used as the list name:
### Sync modes
This integration supports Segment mode. New profiles will be inserted into the Klaviyo Lists. Profiles that are removed from the model will be removed from the list.
### User identifiers
To identify which contacts to add or update in a list, select a model columns and the corresponding Klaviyo fields.
You can match on either **Email** or **Phone number**.
### User consent
Previously, user consent could be added with any profile update/create request, like in `Events`. Now, user consent is configured at the `Subscription List` level.
## API changes
After recent changes to the Klaviyo API, the following issues may affect your syncs.
| Issue | Solution |
| -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| API key permissions may not be correct | Create a new API key and asign correct permissions based on [this chart](#setup). |
| Consent is no longer set on `Events` | Consent is setup through the use of `Subscription Lists` so it needs to be removed from any `Event` syncs. [Klaviyo Docs](https://developers.klaviyo.com/en/docs/collect_email_and_sms_consent_via_api#subscribe-a-profile-via-api) |
| Invalid phone numbers | Phone numbers should be in the format: +12345678901 |
| Invalid dates | Dates should be in ISO format |
| Profile not found by ID | The ID field on profiles should only contain the Klaviyo generated profile ID |
---
## Knock
**URL:** https://hightouch.com/docs/destinations/knock
**Description:** Knock lets you send and manage notifications across email, push, SMS, and in-app from one API—without building the infrastructure yourself.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|---------------|-------------------------------------------------------------------------------------------|----------------------|------------------------------------------------------------------|
| **Users** | Sync data from any source to Knock [users](https://docs.knock.app/concepts/users) | Upsert | [Users docs](https://docs.knock.app/api-reference/users) |
| **Objects** | Sync data from any source to Knock [objects](https://docs.knock.app/concepts/objects) | Upsert | [Objects docs](https://docs.knock.app/api-reference/objects) |
| **Audiences** | Sync data from any source to Knock [audiences](https://docs.knock.app/concepts/audiences) | Add, Remove | [Audiences docs](https://docs.knock.app/api-reference/audiences) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Knock
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Knock** and click **Continue**. You can then authenticate Hightouch to **Knock**.
Enter the following fields into Hightouch:
- **Secret API key**: You can find this in the Knock dashboard under `API keys`.
- (Optional) **Service token**: This is only required if you plan to sync audiences. You can find this in the Knock dashboard under `Service tokens`.
## Sync configuration
Once you've set up your Knock destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Knock destination you want to sync to.
### Syncing users
Sync data from any source to Knock [users](https://docs.knock.app/concepts/users).
#### Record matching
To match rows from your model to users in Knock, you need to select the model column that contains values that match the **ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync user fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom user fields.
If you send data for a custom field that doesn't exist, Hightouch adds the field.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
|----------------|---------------------------------------------------|
| **Do nothing** | Keep the user in Knock with all its synced fields |
| **Delete** | Delete the synced users from Knock |
### Syncing objects
Sync data from any source to Knock [objects](https://docs.knock.app/concepts/objects).
#### Record matching
To match rows from your model to objects in Knock, you need to select the model column that contains values that match the **ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
You can also optionally map the following fields:
- **Email**
- **Phone number**
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
|----------------|-----------------------------------------------------|
| **Do nothing** | Keep the object in Knock with all its synced fields |
| **Delete** | Delete the synced objects from Knock |
### Syncing audiences
Sync data from any source to Knock [audiences](https://docs.knock.app/concepts/audiences).
#### Record matching
To match rows from your model to audiences in Knock, you need to select the model column that contains values that match the **ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync user fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom user fields.
If you send data for a custom field that doesn't exist, Hightouch adds the field.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Kognitiv
**URL:** https://hightouch.com/docs/destinations/kognitiv
**Description:** Sync data from your warehouse to Kognitiv's loyalty platform
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|Any data set|Sync data from a source to a file-import specification in Kognitiv|Mirror|
Mirror mode means one file with all the rows in the query results will be created, every time the sync runs.
## Getting started
### Connect to Kognitiv
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Kognitiv** and click **Continue**.
Hightouch syncs to Kognitiv via their UploadFile Web Service, which uses a security token. Enter your security token.
If you do not have a security token, contact Kognitiv Loyalty Support to set one up.
## Syncing data
### File import specifications
Choose which file import specification you'd like to import to Kognitiv. Hightouch offers templates for:
- Member descriptor file
- Loyalty member custom fields
Access the complete list of Kognitiv's file import options [here](https://info.kognitivloyalty.com/File_Import_Specifications.html).
If you'd like to sync to a file import specification that isn't listed, select **Custom** and enter in the column fields as mappings.
### Field mapping
If you selected one of the file import specifications listed above, column mapping options will be available to select from. You can add additional mappings by creating a new field.
If you selected **Custom** file import specification, enter the field that you'd like to map data to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Kustomer
**URL:** https://hightouch.com/docs/destinations/kustomer
**Description:** Manage and contextualize data to deliver hyper-personalized support and drive smarter processes that scale your business.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
---------------|-------------------------------------------------|----------------------|-------------------------------------------------------------------------------------------------
**Companies** | Sync data from any source to Kustomer companies | Upsert, Update | [Companies docs](https://developer.kustomer.com/kustomer-api-docs/reference/bulkcreatecompanies)
**Customers** | Sync data from any source to Kustomer customers | Upsert, Update | [Customers docs](https://developer.kustomer.com/kustomer-api-docs/reference/bulkcreatecustomers)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Kustomer
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Kustomer** and click **Continue**. You can then authenticate Hightouch to **Kustomer** by entering the following required fields into Hightouch:
- **Region**
- **API Key**
You can generate an API key by navigating to **Settings > Security > API Keys > Add API Key** in your Kustomer settings.
For more information, check out the [Kustomer's API key documentation](https://help.kustomer.com/api-keys-SJs5YTIWX).
You need to grant the following roles to your API key:
- `org.permission.bulk`
- `org.permission.company`
- `org.permission.customer`
## Sync configuration
Once you've set up your Kustomer destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Kustomer destination you want to sync to.
### Syncing companies
Sync data from any source to Kustomer companies.
#### Record matching
To match rows from your model to companies in Kustomer, you need to select a model column and corresponding Kustomer field. You can match on any of the following Kustomer fields:
- **External ID**
- **Company Name**
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync company fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom company fields.
Kustomer requires the **Name** field, so you must map it to complete your configuration.
Ensure your model columns data types match the data types of the fields you want to sync to.
You must suffix custom field names with their data type.
For example, if you were syncing to a custom field called **Age** that accepted numerical values, the Kustomer field name would be **AgeNum**.
For more information, check out the [Kustomer documentation](https://developer.kustomer.com/kustomer-api-docs/reference/createacompany).
#### Split retries
Kustomer counts all companies in a batch as rejected if the request contains a single invalid company. To pinpoint which companys are getting rejected with which errors and reduce the number of valid companys that get retried, you can enable [split retries](/syncs/retries#split-retries).
### Syncing customers
Sync data from any source to Kustomer customers.
#### Record matching
To match rows from your model to customers in Kustomer, you need to select a model column and corresponding Kustomer field. You can match on any of the following Kustomer fields:
- **External ID**
- **Home Email**
- **Work Email**
- **Other Email**
- **Work Phone**
- **Mobile Phone**
- **Home Phone**
- **Fax Phone**
- **Other Phone**
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync customer fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom customer fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
You must suffix custom field names with their data type. For example, if you were syncing to a custom field called **Age** that accepted numerical values, the Kustomer field name would be **AgeNum**.
For more information, check out the [Kustomer documentation](https://developer.kustomer.com/kustomer-api-docs/reference/createacustomer).
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|--------------------------------------------------------
**Do nothing** | Keep the customer in Kustomer with all its synced fields
**Delete** | Delete the synced customers from Kustomer
#### Split retries
Kustomer counts all customers in a batch as rejected if the request contains a single invalid customer. To pinpoint which customers are getting rejected with which errors and reduce the number of valid customers that get retried, you can enable [split retries](/syncs/retries#split-retries).
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## La Growth Machine
**URL:** https://hightouch.com/docs/destinations/la-growth-machine
**Description:** Automate your outreach campaigns to send messages through LinkedIn, Email, Voice Messages, and more with La Growth Machine.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
----------|---------------------------------------------------------|----------------------|-----------------------------------------------------------------------------------------------------------
**Leads** | Sync data from any source to Leads in La Growth Machine | Upsert, Update | [Leads docs](https://documenter.getpostman.com/view/2071164/TVCmSkH2#45f9a866-2afe-4e3d-9e0f-4a63c1b32612)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to La Growth Machine
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **La Growth Machine** and click **Continue**. You can then authenticate Hightouch to **La Growth Machine**.
Enter the following fields into Hightouch:
- **API key**
## Sync configuration
Once you've set up your La Growth Machine destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the La Growth Machine destination you want to sync to.
### Syncing leads
Sync data from any source to Leads in La Growth Machine.
#### Record matching
To match rows from your model to leads in La Growth Machine, you need to select a model column and corresponding La Growth Machine field. You can match on any of the following La Growth Machine fields:
- **Professional email**
- **Personal email**
- **LinkedIn Url**
- **Company name**
- **Company URL**
When matching on `Company name` or `Company URL`, La Growth Machine requires the following fields, so you must map them to complete your configuration:
- **First name**
- **Last name**
#### Field mapping
Hightouch lets you sync lead fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom lead fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------------|--------------------------------------------------------------------
**Do nothing** | Keep the lead in La Growth Machine with all its synced fields
**Clear** | Clear all the mapped fields, but keep the lead in La Growth Machine
**Remove from list** | Remove the lead from the audience, but keep all lead fields unchanged
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## LaunchDarkly
**URL:** https://hightouch.com/docs/destinations/launchdarkly
**Description:** Keep your team's feature flags up-to-date with fresh targeting rules and lists of users
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|---------------------|-------------------------------------------------------------------|----------------------|-------------------------------------------------------------------------------------------------|
| **Segment rules** | Update inclusion or exclusion rules in an existing segment | Add, Remove | [Segments docs](https://apidocs.launchdarkly.com/tag/Segments) |
| **Synced segments** | Sync a full segment, or set of contexts, to LaunchDarkly | Add, Remove | [Synced segments docs](https://docs.launchdarkly.com/guides/integrations/build-synced-segments) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to LaunchDarkly
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **LaunchDarkly** and click **Continue**. You can then authenticate Hightouch to **LaunchDarkly**.
Enter the following required fields into Hightouch:
- **Access Token**
#### Create an access token
1. Log in to LaunchDarkly
2. Visit your **Account settings** page
3. Click on the **Authorization** tab
4. Click on **Create token**
5. Give your token a name and assign it the **Writer** role
6. Click **Save token** and copy the new token that appears in the **Authorization** page
## Sync configuration
Once you've set up your LaunchDarkly destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the LaunchDarkly destination you want to sync to.
### Syncing segment rules
Sync data from any source to LaunchDarkly segments by updating inclusion or exclusion rules for an existing segment.
Hightouch can sync data to a LaunchDarkly segment in three ways:
1. Target users for inclusion
2. Target users for exclusion
3. Target users who match a custom rule
#### Record matching
You can match rows from your model to a segment in LaunchDarkly on any column in your model and any field in LaunchDarkly. Ensure the data types of the model column and LaunchDarkly field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Targeting users for inclusion or exclusion
The most common way of using our LaunchDarkly destination is to sync a model containing a list of user IDs for inclusion or exclusion in a particular segment.
#### Targeting users who match a custom rule
Hightouch can also sync your data into a LaunchDarkly rule clause.
### Syncing synced segments
Hightouch can directly sync a segment or set of contexts to LaunchDarkly.
#### Record matching
To match rows from your model to a synced segment in LaunchDarkly, you need to select the model column that contains values that match the **id** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
Users removed from the model get removed from the segment.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## LG Ad Solutions
**URL:** https://hightouch.com/docs/destinations/lg-ad-solutions
**Description:** Empower your advertising team to run targeted campaigns across LG’s premium Smart TV inventory.
**Section:** Destinations
## Overview
Deliver more relevant ads across LG’s premium, brand-safe inventory. Reach
addressable audiences across LG Smart TV Native and CTV ad experiences by
matching your first-party identifiers with LG IDs using Match Booster.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------- | ---------------------------------- | -------------------- |
| **Audiences** | Create and update custom audiences | All |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to LG Ad Solutions
This integration requires prior approval from the LG Ad Solutions team and the
Hightouch team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your LG Ad Solutions rep who will verify approval. Once we have confirmed,
we will unlock the integration for you to connect.
To connect, input the `Advertiser Name` provided by your LG rep.
### Syncing custom audiences
You can input the name of the audience you plan on delivering to LG.
Additionally, you can specify other properties for your audience.
#### User identifiers
LG only accepts LG Ad IDs to match on. We recommend using Match Booster to
resolve your identifiers to these LG Ad IDs if you do not have them.
#### Delete behavior
The only supported sync mode is `all` mode, meaning that each sync run will refresh
the entire audience. Users removed from your audience will automatically be dropped
with each refresh - only present users will remain in the LG audience.
## Tips and troubleshooting
### Refresh rate
LG has a max refresh rate of monthly. You should not have your LG syncs
running more than every 4 weeks.
### Matched device count
Match rates are not available to preview in the Hightouch platform. Contact your
LG rep for insights into match rates and audience reach.
Keep in mind that audience counts reflect the number of LG devices match, not
the number of users matched. Also keep in mind that matching occurs at the
household level, and that a single household may own multiple devices.
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Sync alerts
---
## Liftoff
**URL:** https://hightouch.com/docs/destinations/liftoff
**Description:** Empower your marketing team to run advertising on Liftoff
**Section:** Destinations
## Overview
[Liftoff](https://liftoff.io/) is a mobile app marketing platform that helps companies acquire and retain high-quality app users at scale. Hightouch's Liftoff destination allows you to sync audiences to Liftoff for ad targeting.
## Supported Syncing
| Type | Description | Supported Sync Modes | API Reference |
| ------------- | ------------------------------------------ | -------------------- | -------------- |
| **Audiences** | Sync audiences from any source to Liftoff. | Add and Remove | Not applicable |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Liftoff
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Liftoff** and click **Continue**. You can then authenticate Hightouch to **Liftoff**.
You need to enter the following required fields:
- **API key**: Ask your Customer Support Representative at Liftoff for your API key.
## Sync Configuration
Once you've set up your **Liftoff** destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Liftoff destination you want to sync to.
This integration supports syncing audiences.
### Record Matching
You can match rows from your model to users in Liftoff on their **Device ID**. Refer to the [record matching docs](/syncs/record-matching) for more information.
### Delete Behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| ------------------------ | ------------------------------------------ |
| **Do nothing** | Keep the user in the Liftoff audience. |
| **Remove from audience** | Remove the user from the Liftoff audience. |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## LINE Ads
**URL:** https://hightouch.com/docs/destinations/line-ads
**Description:** Boost your business growth by finding the right customers on LINE. Reach more audiences and achieve better results by managing campaign objectives, targeting, budget, and measurement anytime through LINE Ads
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|--------------|-------------------------------------------------|----------------------|------------------------------------------------------------------------------------------------------|
| **Segments** | Sync data from any source to LINE Ads audiences | Add | [Audience docs](https://ads.line.me/public-docs/v3/3.9.7/data-general-partner#_uploadcustomaudience) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to LINE Ads
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **LINE Ads** and click **Continue**. You can then authenticate Hightouch to **LINE Ads**.
Enter the following required fields into Hightouch:
- **Ad Account ID**: Enter your ad account ID.
- **API access key**: Enter a valid API access key.
- **API secret key**: Enter a valid API secret key.
For more information about LINE's Ads Management API and requesting credentials, go to LINE's [authentication docs](https://ads.line.me/public-docs/v3/3.9.7/data-general-partner#_authentication).
## Sync configuration
Once you've set up your LINE Ads destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the LINE Ads destination you want to sync to.
### Syncing segments
Sync data from any source to LINE Ads custom audiences.
#### Record matching
To match rows from your model to segment in LINE Ads, you need to select a model column and corresponding LINE Ads field. You can match on any of the following LINE Ads fields:
- **Email**
- **Phone number**
- **IFA**
If you are syncing to an existing segment, only the ID type for that segment is available.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Linear
**URL:** https://hightouch.com/docs/destinations/linear
**Description:** Enrich and create tickets in Linear
**Section:** Destinations
## Overview
Linear is a popular issue-tracking tool designed for high-performance teams. It helps streamline workflows for software projects, sprints, and tasks.
Hightouch's Linear destination allows you to sync data from your warehouse to create and update issues in Linear. This can be useful for automating bug reports, feature requests, and other tasks based on events or data in your own systems.
## Supported Syncing
| Type | Description | Supported Sync Modes |
| ---------- | ------------------------------------------- | ------------------------------ |
| **Issues** | Sync data from any source to Linear issues. | Upsert, Update, Insert, Delete |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Linear
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Linear** and click **Continue**. You can then authenticate Hightouch to **Linear** by providing your API Key.
To get your API Key:
1. Log in to your Linear account.
2. Navigate to **Settings** > **Security & access**.
3. Under **Personal API keys**, create a new key.
4. Copy the key and paste it into the Hightouch configuration.
## Sync Configuration
Once you've set up your **Linear** destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Linear destination you want to sync to.
### Syncing Issues
Hightouch allows you to create, update, or delete issues in Linear based on your model data.
#### Record Matching
You can match rows from your model to issues in Linear using the **Issue ID**. **Issue ID** must be a UUIDv4.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Linear automatically generates an identifier for every new
record synced if one is missing.
#### Field Mapping
You can map data from any of your model columns to Linear issue fields. Hightouch only requires you to map the **Title** field.
These are the additional fields you can map:
- Description
- Priority
- Assignee ID
- Project ID
- State ID
- Label IDs
- Due Date
- Estimate
#### Delete Behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | --------------------------------------------------- |
| **Do nothing** | Keep the issue in Linear with all its synced fields |
| **Delete** | Delete the synced issues from Linear |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## LinkedIn Ads
**URL:** https://hightouch.com/docs/destinations/linkedin
**Description:** Run campaigns with custom audiences, retargeting, and lookalikes in LinkedIn
**Section:** Destinations
## Overview
Deliver even more relevant ads by combining data from various sources within your
data warehouse to build custom audiences for LinkedIn Ads. By keeping your
custom audiences updated automatically, you can avoid showing irrelevant ads.
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| --------------- | ------------------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audiences** | Create new audience to sync users or companies | Upsert | [Audiences docs](https://learn.microsoft.com/en-us/linkedin/marketing/integrations/matched-audiences/matched-audiences?view=li-lms-2023-11) |
| **Conversions** | Sync data from any source to LinkedIn conversions | Insert | [Conversions docs](https://learn.microsoft.com/en-us/linkedin/marketing/integrations/ads-reporting/conversions-api?view=li-lms-2023-11&tabs=http) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to LinkedIn Ads
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations)
and click the **Add destination** button. Select **LinkedIn Ads** and click **Continue**.
You can then authenticate Hightouch to **LinkedIn Ads** using OAuth.
### Authenticate with OAuth
For the **Authentication method**, select **Log in to LinkedIn Ads** and log into
your LinkedIn Ads account.
Once successful, you will be redirected back to Hightouch to enter a descriptive name
for your destination and complete setup.
## Sync configuration
Once you've set up your LinkedIn Ads destination and have a
[model](/getting-started/concepts#models) to pull data from,
you can set up your sync configuration to begin syncing data. Go to the
[**Syncs** overview page](https://app.hightouch.com/syncs) and click the
**Add sync** button to begin. Then, select the relevant model and the LinkedIn
Ads destination you want to sync to.
### Syncing audiences
To get started, select an ad account to sync your data to, then select
which audience object you want to sync data to: Company or User.
Hightouch supports automatically creating a new audience for your sync. You can
specify a custom name for this audience, or Hightouch will use the name of the
model.
After the initial run of the sync, you can't update the ad account, audience
object, or name of the audience. To make these changes, create a new sync.
#### Sync mode
This integration supports Upsert mode. New users will be inserted into the LinkedIn
audience. Users that are removed from the model will be removed from the
audience.
#### Field mapping
The sync configuration has two field mapping sections:
- **ID fields**: These are the primary identity keys that LinkedIn uses to match users, such as hashed email or Google Advertising ID (GAID). These values become `userIds` in the LinkedIn API payload. You must map at least one ID field.
- **Additional fields**: These are supplemental fields—such as first name, last name, title, company, and country—that improve LinkedIn's match rate but aren't primary identifiers.
LinkedIn requires emails to be hashed using SHA256 or SHA512. Hightouch handles this automatically on the backend.
LinkedIn enforces the following input data requirements: - All IDs provided by
`userId` or `userIds` fields must have a [supported type and valid
value](https://learn.microsoft.com/en-us/linkedin/marketing/integrations/matched-audiences/create-and-manage-segment-users?view=li-lms-unversioned&tabs=http#input-data-requirements).
- Each row of data must provide: (1) at least one valid ID; or (2) A valid
`firstName` and `lastName`. See LinkedIn's [user data
schema](https://learn.microsoft.com/en-us/linkedin/marketing/integrations/matched-audiences/create-and-manage-segment-users?view=li-lms-unversioned&tabs=http#schema)
for more information.
### Syncing conversions
Sync data from any source to LinkedIn as conversion events.
Create a [conversion rule](https://www.linkedin.com/help/lms/answer/a1657171) in LinkedIn Campaign Manager.
Select "Conversions" in your sync in Hightouch.
Select your LinkedIn account.
Choose your event timestamp or let LinkedIn use the timestamp the event arrives at the server.
#### Record matching
You can match rows from your model to conversions in LinkedIn on the following user fields:
- Email (SHA256)
- LinkedIn tracking UUID
- Acxiom ID
- Oracle moat ID
## Handling PII and hashing
You can enable PII hashing if you want Hightouch to automatically detect, normalize, and hash fields that require it. For example, if you map to a `Email (SHA256)` field and the model column with the email data is unhashed, Hightouch hashes the value for you.
By default, Hightouch automatically detects if the following fields require hashing:
- **Email (SHA256)**
- **Email (SHA512)**
#### View conversion tracking and reporting metrics
Once your sync is set up and published, you can view conversions tracking and reporting metrics in your LinkedIn Campaign Manager.
## Tips and troubleshooting
### Matched users count
Below only applies to the audiences sync type.
### Common errors
#### 400 - MISSING_REQUIRED_FIELD
You may receive a `400 - type=MISSING_REQUIRED_FIELD` because the sync is sending a required field `null` values.
You can resolve this error by either:
- [Removing rows containing `null` values](/models/sql-editor#exclude-rows-with-null-values) from your model's query results
- Replacing any `null` values in your model's query results with another value
- Using the [advanced mapper](/syncs/mapping-data#advanced-mapper) to [ignore any `null` values](/syncs/mapping-data#dont-sync-null-values)
### Live debugger
### Sync alerts
---
## LiveIntent
**URL:** https://hightouch.com/docs/destinations/liveintent
**Description:** Empower your marketing team to match audiences across LiveIntent's publisher network for targeting and suppression.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
| ---------- | -----------------------------------------------------| ---------------------- | ------------------------------------------------------------------------------------------------------- |
| **Custom Audiences** | Sync data from any source to an LiveIntent list as user identifiers | Upsert | [Custom Audience](https://audiences.liveintent.com/api-guide#tag/What-is-a-Custom-Audience)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to LiveIntent
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **LiveIntent** and click **Continue**.
You can connect to LiveIntent with either an API Token, or a username and password. Contact your account team at LiveIntent for credentials.
## Sync configuration
Once you've set up your LiveIntent destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the LiveIntent destination you want to sync to.
### Audience selection
Select whether you'd like to create a new audience or sync to an existing audience. If you would like to create a new audience, enter the name of the audience in the sync configuration form.
### Field mapping
LiveIntent uses email, IDFA, and AAID to identify users. Select which of these identifiers you would like to map your records as and send to LiveIntent.
#### Handling PII and hashing
LiveIntent expects **email** values to be hashed.
For audience syncs, by default, Hightouch automatically hashes first-party user data fields before sending requests. If the data in your model is already hashed, you can disable PII hashing to successfully send data to LiveIntent.
### Delete behavior
When records leave your query result, they are removed from your audience by default. If you would not like to delete any records from your audience, you can choose to do nothing when records leave your query result.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## LiveRamp
**URL:** https://hightouch.com/docs/destinations/liveramp
**Description:** LiveRamp helps you unify customer data across platforms, activate it securely everywhere you need it, and measure performance with precision—without compromising privacy. It turns fragmented identifiers into actionable insights so you can target smarter and drive higher ROI.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
------------------------|--------------------------------------------|---------------------
**SFTP file upload** | Upload audience files to LiveRamp via SFTP | Insert, All
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to the LiveRamp SFTP server
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **LiveRamp** and click **Continue**. You can then authenticate Hightouch to **LiveRamp**.
Enter the following fields into Hightouch:
- **Host**
- **Port**
- **Username**
- **Password**
- (Optional) **Private Key**
- (Optional) **Private Key Passphrase**
## Sync configuration
Once you've set up your LiveRamp destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the LiveRamp destination you want to sync to.
### Syncing audiences via file upload
Upload audience files to LiveRamp via SFTP
#### Field mapping
Hightouch lets you sync batch file delivery fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default batch file delivery fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
For more information on file formatting, refer to LiveRamp's [docs](https://docs.liveramp.com/connect/en/formatting-column-based-files.html).
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Loops
**URL:** https://hightouch.com/docs/destinations/loops
**Description:** Send your product, marketing, and transactional emails for your app with Loops, the email platform built for SaaS. Empower your marketing teams to create, send and track beautiful email campaigns.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-------------|---------------------------------------------|------------------------|-----------------------------------------------------------
**Contacts** | Sync data from any source to Loops contacts | Upsert | [Contacts docs](https://loops.so/docs/api-reference/intro#contacts)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Loops
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Loops** and click **Continue**. You can then authenticate Hightouch to **Loops**.
Enter the following fields into Hightouch:
- **API key**
## Sync configuration
Once you've set up your Loops destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Loops destination you want to sync to.
### Syncing contacts
Sync data from any source to Loops contacts.
#### Record matching
To match rows from your model to contacts in Loops, you need to select a model column and corresponding Loops field. You can match on any of the following Loops fields:
- **Email**
- **User Id**
#### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom contact fields.
Loops requires the **Email** field, so you must map it to complete your configuration.
If you send data for a custom field that doesn't exist, Hightouch adds the field.
Ensure your model's columns have the same data types as the fields you want to sync to.
For the **mailingLists** object, the API expects JSON objects with key-value pairs where the key is a mailing list ID and the value is a `boolean` denoting if the contact should be added (`true`) or removed (`false`) from the list. Go to [API Docs for Mailing Lists](https://loops.so/docs/api-reference/create-contact#param-mailing-lists) to learn more.
#### Trigger Loops
Hightouch can trigger a Loop to start sending emails to your contacts. Build a Loop using the [Loop builder](https://loops.so/docs/loop-builder), and configure whether you'd like to trigger Loops in the sync form.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|-----------------------------------------------------------
**Do nothing** | Keep the contact in Loops with all its synced fields
**Clear** | Clear all the mapped fields, but keep the contact in Loops
**Delete** | Delete the synced contacts from Loops
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Magnite
**URL:** https://hightouch.com/docs/destinations/magnite
**Description:** Magnite is the world’s largest independent sell-side advertising company. We’ll help you win across every channel, every format and on your terms.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
--------------|------------------------------------------------|---------------------
**Access Data Audiences** | Sync data from any source to Magnite Data audiences (`ctvid`, `ip`, `maid`, `cookie`) | All
**Match Audiences** | Sync data from any source to Magnite Match audiences (`hem`) | All
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Magnite
This integration requires prior approval from the Magnite team and the Hightouch team.
To gain approval, please contact the Hightouch support team and connect us directly with your Magnite rep who will verify approval. Once we have confirmed, we will unlock the integration for you to connect.
To use the Magnite Data integration, enter the following fields into Hightouch:
- **Client subfolder**
- **Owner**
To use the Magnite Match integration, enter the following fields into Hightouch:
- **Client name**
- **Account ID**
Once you've set up your Magnite destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin.
## Magnite Data integration
### Sync configuration
Select the relevant model and the Magnite Data destination you want to sync to. Then, enter the following fields into Hightouch:
- **Segment name**
- **Price type** (acceptable values are `cpm` and `revshare`)
- **Charge for use of the segment**
- **Segment owner**
- **Description for the segment** (optional)
### Syncing audiences
Sync data from any source to Magnite Data audiences.
#### Field mapping
Hightouch lets you sync audience fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default audience fields.
Magnite requires the following fields, so you must map them to complete your configuration:
- **User ID**
- **User ID type**
- **Update type**
Ensure your model's columns have the same data types as the fields you want to sync to.
## Magnite Match integration
### Sync configuration
Select the relevant model and the Magnite Match destination you want to sync to. Then, enter the following fields into Hightouch:
- **Segment name**
- **Description for the segment** (optional)
- **CPM for the segment** (optional)
- **Platform for the segment**
- **Membership timestamp column** (optional)
- **Full resync interval (in days)**
### Syncing audiences
Sync data from any source to Magnite Match audiences.
#### Field mapping
Hightouch lets you sync audience fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default audience fields.
Magnite requires the following fields, so you must map them to complete your configuration:
- **Email (SHA256)**
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Mailchimp
**URL:** https://hightouch.com/docs/destinations/mailchimp
**Description:** Run better email campaigns on Mailchimp with up-to-date customer data from your data warehouse
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| ------------ | ------------------------------------------------------------------------------------- | ---------------------- |----------------------------------------------------------------------------------------------------------------------------------- |
| **Object** | Sync members to audiences in Mailchimp. | Upsert | [Members Docs](https://mailchimp.com/developer/marketing/api/list-members/add-or-update-list-member/) |
| **Tag** | Create and apply a new tag or apply an existing tag to contacts within your audience. | Add, Remove | [Static Segments (Tags) Docs](https://mailchimp.com/developer/marketing/api/list-segments/batch-add-or-remove-members/) |
## Setup
To setup the Mailchimp destination, you will need your data center and an active API Key.
To find the data center and API key, navigate to the [Mailchimp Dashboard](https://admin.mailchimp.com).
### Data center
To find the data center, look at the prefix for your admin dashboard URL. In this example, the URL is `us1.admin.mailchimp.com`. Your prefix would be `us1`.
### API key
To find your API key, navigate to the [API Keys section](https://admin.mailchimp.com/account/api/) of your Mailchimp account.
Go down to the section Your API Keys, and press the Create A Key button if none exist. Otherwise, you'll see your API key in the table.
Mailchimp has API Keys and Mobile SDK API Keys. The Mobile SDK API Key will
not work with this integration. You will need to create a regular API Key.
## How Hightouch connects
Hightouch connects to Mailchimp through the [Mailchimp Marketing API](https://mailchimp.com/developer/marketing/api/), specifically the [Lists API](https://mailchimp.com/developer/marketing/api/lists/) to read, update, and manage audience data.
This integration handles:
- Retrieving your available audiences for setup and testing
- Syncing audience members and custom merge fields
- Managing segment memberships
- Archiving or permanently deleting members when records leave your query
## Syncing Mailchimp contacts
Hightouch supports syncing to the following Mailchimp resources:
- Audience members
Hightouch requires an existing audience in Mailchimp to sync audience members.
### Sync modes
This integration supports Upsert mode.
In the Upsert mode, new users will be inserted into Mailchimp and all attributes will be kept up-to-date within Mailchimp.
### Record matching
Records can be matched from your source to your Mailchimp workspace by your Mailchimp Audience Member Email in the Upsert mode.
### Field mapping
You can sync columns from your source to Mailchimp's default and custom merge fields. Hightouch automatically detects existing merge fields from your Mailchimp audience.
You can add custom merge fields in your Mailchimp dashboard from the Settings page dropdown and selecting Audience fields and \*\|MERGE\|\* tags
Once on the Audience fields page you can drop to the bottom of the page and hit the Add A Field button to create a custom merge field.
### Delete mode
You can choose what Hightouch's behavior is when records leave the query result set. The default is doing nothing, but you can also set Hightouch to archive or permanently delete the Mailchimp audience member on record exit.
- `ARCHIVE` : Unsubscribe a user from your Mailchimp audience, but do not delete the record. Archived members can be re-added at a later date.
- `DELETE` : **Permanently** remove an audience member and delete their information. This action can't be undone, and the user **can't** be re-added to the list at a later date.
---
## MariaDB
**URL:** https://hightouch.com/docs/destinations/mariadb
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Overview
This destination combines the analytical power of your data warehouse with the low-latency performance of a transactional database.
## Supported syncing
|Sync Type|Description |Supported Sync Modes |
|-----------|----------------------------------------------|---------------------|
| **Table** |Sync data from any source to a MariaDB table | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to MariaDB
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **MariaDB** and click **Continue**.
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your MariaDB.
- **Port**: The port number of your MariaDB. The default is 3306, but yours may be different.
- **Database**: The name of the database in your MariaDB.
- **User**: This can be your personal MariaDB login or a dedicated user for Hightouch.
- **Password**: The password for the user specified above.
You can optionally create or select a [previously configured tunnel](/security/ssh-tunneling) or upload a `.pem` CA certificate.
### SSH tunneling
Hightouch can connect directly to MariaDB over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your MariaDB instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
## Sync configuration
Once you've set up your MariaDB destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the MariaDB destination you want to sync to.
### Select table
Select the existing table in MariaDB you want to sync to.
### Record matching
You can match rows from your model to rows in MariaDB on any column as long as it is a [primary key or unique key](https://mariadb.com/kb/en/create-table/#index-categories).
### Field mapping
Select which model columns you want to sync to your MariaDB table columns. Hightouch automatically pulls the columns from your table to make them available for you to map.
When syncing to [binary](https://dev.mysql.com/doc/refman/8.0/en/binary-varbinary.html) destination fields, Hightouch assumes the model column values to be [base64](https://en.wikipedia.org/wiki/Base64) strings. If they aren't, you can use [template mapping](/syncs/mapping-data#template-mapping) to [cast](/models/data-types-casting#casting) them if needed.
### Upsert query optimization
You can choose to use the [`LOAD DATA` statement](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_local_infile) in the upsert query during your sync. This option loads data from a text file into your database, which can be up to twenty times faster than the default `INSERT INTO` query Hightouch uses.
If you enable the `LOAD DATA` option, Hightouch replaces databases rows with the mapped model row, if it exists. This means that if any MariaDB fields aren't mapped in Hightouch they receive `NULL` values.
For Hightouch to successfully sync with the `LOAD DATA` option, you need to enable the [`local_infile`](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_local_infile) system variable.
Review the [security considerations](https://dev.mysql.com/doc/refman/8.0/en/load-data-local-security.html) before enabling `local_infile`.
### Delete behavior
When rows no longer shows up from your model's query results, you can configure the sync to **Do nothing** or **Delete** those rows from the MariaDB.
| Behavior | Description |
| -------------- | ----------------------------------------------- |
| **Do nothing** | Keep the synced rows in the table in MariaDB |
| **Delete** | Delete the synced rows from the table in MariaDB|
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Marigold Engage (Selligent)
**URL:** https://hightouch.com/docs/destinations/marigold-engage
**Description:** Targeting marketing to personalize customer experiences.
**Section:** Destinations
## Supported content
| Type | Description | Supported Sync Modes |
|-----------|----------------------------------------------------|------------------------|
| **Users** | Sync records from your model to lists and segments | Upsert, Update, Insert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Marigold Engage
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Marigold Engage** and click **Continue**. You can then authenticate Hightouch to **Marigold Engage** via a Service Account.
To create a Service Account, click the settings gear in the bottom left corner of the Marigold Engage dashboard. Then, navigate to **Access Management > Service Accounts** and click **New**. Make sure to set an expiration date and configure the following endpoints:
- `/organizations/{organization}/lists`
- `/organizations/{organization}/lists/{list}/fields`
- `/organizations/{organization}/lists/{list}/data/search`
- `/organizations/{organization}/lists/{list}/data/load`
- `/organizations/{organization}/lists/{list}/segments`
- `/organizations/{organization}/lists/{list}/segments/{segment}`
- `/organizations/{organization}/lists/{list}/segments/{segment}/data/load`
Then in Hightouch:
1. Enter the Marigold hostname used to access the Marigold control panel, this is usually `{organization}.slgnt.eu`
2. Enter the API and Secret keys from the Service Account you created
3. Enter the organization the lists should be created in
## Sync configuration
Once you've set up your Marigold Engage destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Marigold Engage destination you want to sync to.
### Field mapping
Field mapping is where you select which customer information parameters or other record parameters you want to send to Marigold.
You can map data from any of your model columns to the list's preconfigured fields.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Marigold Engage+
**URL:** https://hightouch.com/docs/destinations/marigold-engage-plus
**Description:** Sync data and run campaigns on Marigold Engage+
**Section:** Destinations
## Supported content
| Type | Description | Supported Sync Modes |
|--------------|----------------------------------------------------------------|----------------------|
| **Audience** | Sync records from your model to audiences in real-time or bulk | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
Marigold Engage+ supports both real-time and bulk sync modes.
Real-time syncs are recommended only for small data sets,
while bulk syncs are recommended for any data set over 1 million.
## Connect to Marigold Engage+
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Marigold Engage+** and click **Continue**. You can then authenticate Hightouch to **Marigold Engage+** with help from your Marigold agent.
Then in Hightouch:
1. Enter the Customer ID that your Marigold agent provides
2. Enter the region your Marigold account is in
3. Enter the API and Secret keys from Marigold
4. If you would like to sync in bulk, enter the SFTP credentials your Marigold agent provides
## Advanced configuration
If you are syncing in real-time to joined tables you must specify which fields should be sent to which forms in the destination config.
If you are syncing in bulk you must specify the available mapping options in the destination config.
Your Marigold agent will help you configure the available options for your use case.
## Sync configuration
Once you've set up your Marigold Engage+ destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Marigold Engage+ destination you want to sync to.
### Field mapping
Field mapping is where you select which customer information parameters or other record parameters you want to send to Marigold.
You can map data from any of your model columns to the list's preconfigured fields.
### Form IDs
When syncing in real-time mode you must also create an API post form in Marigold and enter the form's ID in Hightouch.
The form will control which campaigns are triggered as Highouch sends data to Marigold.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Marketo
**URL:** https://hightouch.com/docs/destinations/marketo
**Description:** Build robust marketing campaigns on Marketo using rich customer data from your data warehouse
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| ------------------------------- | ------------------------------------------------------------------------------------- | -------------------- | ------------------------------------------------------------------------------------ |
| **Standard and custom objects** | Sync data to custom or standard objects including leads, companies, and opportunities | Upsert, Update | [Object docs](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/lead-database/custom-objects) |
| **Custom activities** | Sync data to custom activities | Insert | [Activities docs](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/lead-database/activities) |
| **Static lists** | Update membership in static lists | Add, Update | [Static lists docs](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/assets/static-lists) |
| **Campaign triggers** | Trigger campaigns using source data | Insert | [Campaign docs](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/assets/smart-campaigns) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Marketo setup
Giving Hightouch access to Marketo's API requires you to create a [**Custom Service**](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/custom-services) and [**API Only User**](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/custom-services) for that service in Marketo.
Even if you already have a Marketo API User, it's best to create one that's solely for the Hightouch service you'll create.
Having a dedicated user for the Hightouch integration lets you more easily track changes from Hightouch syncs.
### Create Marketo service
Before you can create a Custom Service, you need to create a Marketo [role](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/custom-services) with API Access and a user with that role.
None of the default Marketo Roles have API access, so if this is your first API integration in Marketo, you first need to create an API access role.
#### Create Marketo API access role
1. In Marketo, go to **Admin > Users & Roles > Roles** and click **New Role**.
2. Enter a **Role Name** (for example, "Hightouch Integration Role") and **Description**. Then assign **all** permissions listed in this table to the role:
| Sync Type | Necessary permissions |
| ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Standard and custom objects** | **Read-Write Company** (for company objects), **Read-Write Opportunity** (for opportunities), **Read-Write Person** (for leads), and **Read-Write Custom Object** and **Read-Write Custom Object Type** (for any custom objects) |
| **Custom activities** | **Read-Write Activity**, **Read-Write Activity Metadata** |
| **Static lists** | **Access Marketing Activities > List Import** |
| **Campaign triggers** | **Read-Write Assets**, **Activate Campaign** |
3. Click **Create**.
If you try setting up a Marketo destination with an API access role that wasn't assigned **all** permissions in the table above, the destination setup fails with a [`603 - Access denied`](#code-603-access-denied) error.
#### Create Marketo API user
After creating a role, you need to create an API-Only user.
API-Only users are a special type of user in Marketo, as they are administrated by other users and cannot be used to log in to Marketo.
1. In Marketo, go to **Admin > Users & Roles > Users** and click **Invite New User**.
2. Enter an **email** address, **first name**, and **last name** for the user. (These don't need to be valid.) Click **Next**.
3. Assign the [previously created role](#create-marketo-api-access-role), click the **API Only** checkbox, and click **Next**.
4. The last screen for user creation says "An invitation is not required for API only." Click **Send** to create the API-Only user.
Ensure that this user has access to the [workspaces and partitions](https://experienceleague.adobe.com/en/docs/marketo/using/product-docs/administration/workspaces-and-person-partitions/understanding-workspaces-and-person-partitions) you want to use in Hightouch. You can read more about this in the [`603 - Access denied`](#code-603-access-denied) section.
#### Create a custom service
Custom Services provide the actual credentials required to authenticate Marketo to Hightouch.
To provision one, follow these instructions:
1. In Marketo, go to **Admin > LaunchPoint** and select **New Service**.
2. Give your service a descriptive name. From the **Service** list, select the **Custom**. Give your service a description, for example, "Hightouch Integration" and select [the API Only user you previously created](#create-marketo-api-user), then click **Create**.
3. This adds a new service to your list of LaunchPoint services, and the option to **View Details**. Click on **View Details** and save the **Client Id** and **Client Secret** for use in Hightouch.
You're now ready to provide Hightouch access to Marketo.
## Connect to Marketo
If you created an [IP Allowlist](https://experienceleague.adobe.com/docs/marketo/using/product-docs/administration/additional-integrations/create-an-allowlist-for-ip-based-api-access.html) in your Marketo settings, make sure to allowlist Hightouch IP addresses so Hightouch can reach your cluster. Refer to our
[IP address docs](/security/networking#ip-addresses) to find the relevant IP addresses for your Hightouch region.
In Hightouch, go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Marketo** and click **Continue**. You can then authenticate Hightouch to Marketo by entering the following fields:
- Client ID
- Client Secret
- Base URL
To find the Client ID and Secret, navigate to **Admin > LaunchPoint** in your Marketo account and select [the service you created for Hightouch](#create-marketo-service).
You can find the Base URL in **Admin > Integration > Web Services** under **REST API**.
Only use the part after `https://` and before `/rest` as your base URL, for example, `663-XSQ-200.mktorest.com`.
## Sync configuration
Once you've set up your Marketo destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Marketo destination you want to sync to.
### Syncing objects
Hightouch supports syncing to both standard and custom Marketo objects.
The standard objects that Hightouch supports are:
- **Leads**
- **Companies**
- **Opportunities**
#### Record matching
You can match rows from your model to objects in Marketo on any column in your model and certain fields in Marketo. To avoid duplications or other data issues, it's imperative to choose a column with unique values. Refer to the [record matching docs](/syncs/record-matching) for more information.
{/* */}
{/* */}
The Marketo fields available for record matching depend on the object:
- for the [Lead object](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/lead-database/lead-database), Hightouch lists all `person` type [custom fields](https://experienceleague.adobe.com/docs/marketo/using/product-docs/administration/field-management/create-a-custom-field-in-marketo.html) for record matching,
- for [custom objects](https://experienceleague.adobe.com/docs/marketo/using/product-docs/administration/marketo-custom-objects/understanding-marketo-custom-objects.html), [Upsert](#supported-syncing) mode lists all [dedupe fields](https://experienceleague.adobe.com/docs/marketo/using/product-docs/administration/marketo-custom-objects/add-marketo-custom-object-fields.html), whereas [Update](#supported-syncing) mode lists all dedupe fields and `MarketoGUID`.
{/* */}
{/* */}
#### Field mapping
Hightouch lets you sync object fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to an object's default fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
-------------------|-----------------------------------------------------------------
**Do nothing** | Keep the record in Marketo with all its synced fields
**Delete record** | Delete the record from Marketo
**Clear fields** | Keep the record, but clear the mapped fields
In Upsert mode, you can select between **Do nothing** and **Delete record**. In Update mode, you can select between **Do nothing** and **Clear fields**.
#### Batch size
By default, Hightouch sends 300 objects in a batch.
You can lower the batch size to mitigate [413 Request Entity Too Large](#code-413-request-entity-too-large) errors.
### Syncing custom activities
When syncing custom activities, Hightouch treats any records added to your [model](/getting-started/concepts#models) as new activities and sends them to Marketo when your sync runs.
To sync custom activities, the custom activity type must first be created and approved in Marketo.
You can learn more about creating custom activities in [Marketo's docs](https://experienceleague.adobe.com/docs/marketo/using/product-docs/administration/marketo-custom-activities/create-a-custom-activity.html).
#### Activity selection
To start, select a custom activity type you've already created in Marketo.
If you don't see it in the dropdown, make sure it's approved in Marketo and then click **Refresh** in Hightouch.
#### Event timestamp
You can optionally select a column that contains timestamps of when activities occurred. If this field is empty, Marketo uses the time the activity arrives at the server.
#### Lead matching
To associate activities to leads, select the model column that contains either the leads' **Marketo GUID** or **Email**.
#### Activity primary attribute
Select the model column to map to the [primary attribute field](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/lead-database/activities) for your custom activity.
#### Field mapping
You can also map data from any of your model columns to a custom activity's non-primary attributes.
Ensure your model columns data types match the data types of the fields you want to sync to.
### Syncing static lists
Static lists are a list of names in Marketo.
You can configure Hightouch syncs to manage these lists by adding and updating users.
#### User identifiers
To identify which users to add or update in an static list, select a model column and the corresponding Marketo attribute, for example **Lead ID**.
#### Select an existing static list or create a new one
You can create a new static list or use an existing one. When creating a new static list, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model. To use an existing static list, select the desired static list from the dropdown.
#### Folder selection
If you're creating a new static list in Marketo, you need to select the folder/program to create the list in. Static lists can only be created in
folders and programs of the following types:
- Email Batch Program
- Marketing Program
- Nurture Program
- List
In the Marketo UI, you can tell whether the folder is one of the above types by right clicking the folder in the Marketing Activities sidebar. If the "New local asset"
or "Create list" options are available, then the folder supports static lists.
#### Static list description
If you're creating a new static list in Marketo, you can optionally Hightouch add a description for the static list.
#### Split retries
By default, Hightouch retries all records in a rejected batch. To pinpoint which users are getting rejected with which errors and reduce the number of valid users that get retried, you can enable [split retries](/syncs/retries#split-retries).
### Trigger campaigns
Hightouch lets you use records from your models to trigger campaigns through the [Marketo Campaign API](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/endpoint-reference).
#### Prerequisites
Before you can set up a Hightouch sync to trigger a campaign, you need to have an active [trigger campaign](https://experienceleague.adobe.com/en/docs/marketo/using/product-docs/core-marketo-concepts/smart-campaigns/creating-a-smart-campaign/understanding-batch-and-trigger-smart-campaigns) with **Web Service API** as the source in Marketo.
For details, follow the setup instructions below.
#### Trigger campaign setup
1. If you haven't already, go to **Design Studio** to [draft an email template](https://experienceleague.adobe.com/docs/marketo/using/product-docs/email-marketing/general/email-editor-2/create-an-email-template.html) in your Marketo instance. Before your email template you can use the email template in Hightouch, your draft must be approved.
You can use custom [tokens](https://experienceleague.adobe.com/docs/marketo/using/product-docs/demand-generation/landing-pages/personalizing-landing-pages/tokens-overview.html) to populate your email template with data from your model.
In the example screenshot above, a custom token makes it possible to dynamically populate the email subject.
2. Next, you need a **Smart Campaign** to trigger. Follow [these steps](https://experienceleague.adobe.com/docs/marketo/using/product-docs/core-marketo-concepts/smart-campaigns/creating-a-smart-campaign/create-a-new-smart-campaign.html?lang=en) if you need to create a new one. Make sure that your campaign belongs to a program.
3. Open the **Smart List** tab in the top nav to configure the trigger. Drag and drop **Campaign is Requested** from the triggers list on the right.
4. Select that the source **is** **Web Service API**.
5. Open the **Flow** tab in the top nav to configure your flow. Drag and drop **Send Email** from the actions on the right.
6. Select the email template you want to send.
7. Open the **Schedule** tab in the top nav to configure your qualifications. Click on **Edit** to edit your qualifications.
Hightouch doesn't check if an email has been sent to
a distinct lead before trigger a campaign. For example, if a lead with email `janedoe@example.com`
appears in two of the queried rows from your model, then Hightouch triggers
an email for each of these rows. Two emails would be sent to `janedoe@example.com`. If you want to only send
one email per lead, set your qualifications in Marketo to **only
once**. 
8. Click **Activate** when you're finished.
For additional reference, see [Marketo's how-to article](https://experienceleague.adobe.com/en/docs/marketo-developer/marketo/rest/assets/transactional-email).
#### Sync configuration
Each new row in your model triggers your Marketo Smart Campaign.
That means that Hightouch triggers the campaign for every row in your model on your first sync run, or if you trigger a [full resync](/syncs/overview#resync-full-query).
In subsequent sync runs, only new rows in your model's query results trigger the campaign.
#### Campaign selection
To start, select a campaign you've created in Marketo.
Only campaigns triggerable via web service API are listed here.
If you don't see the campaign you want to trigger in the dropdown, make sure it's follows the [prerequisites](#prerequisites) and then click **Refresh** in Hightouch.
#### Lead matching
Marketo Smart Campaigns can only be triggered for leads that already exist in your Marketo instance.
To trigger an email for the intended leads, select the model column that contains either the leads' **Marketo GUID** or **Email**.
If you use email for your lead matching and your leads aren't
de-duplicated by email, Hightouch uses the latest lead based on the
`createdAt` timestamp.
#### Token mappings
Use this section to map custom token values.
You only need to add mappings for tokens used in your email template.
#### Testing
Before activating your campaign trigger sync, it's best to test it with with an example lead whose email inbox you have access to.
This lets you inspect the test email.
Click **Test** from the sync configuration page.
You can manually overwrite columns of a test row.
In the example below, since the sync uses the **Email** column to match leads, you can manually overwrite the **Email** field to test lead's email address.
If you're using the **Marketo GUID** column to match leads, you need to manually overwrite the **Marketo GUID** field to the test lead's **Marketo GUID** to send a test email.
Once you've altered the email or Marketo GUID, press **Sync as added row**. This triggers the campaign for the test lead and sends an email to their inbox.
Check the test lead's email inbox to verify that your configuration sent the intended email with custom tokens populated as expected.
## Tips and troubleshooting
### Common errors
{/* */}
#### Code 413 "Request Entity Too Large"
This error means that size of Hightouch's API request to Marketo was too large.
Lower the [batch size](#batch-size) to mitigate it.
#### Code 603 "Access denied"
This error means that authentication to Marketo was successful but the account you authenticated with doesn't have sufficient permission to call the Marketo API.
For more information, see [Marketo's help article](https://experienceleague.adobe.com/en/docs/experience-cloud-kcs/kbarticles/ka-29391).
As explained in the [Create Marketo service](#create-marketo-service) section, ensure that:
- you assigned all permissions listed in the table to the API access role (especially the [**Read-Write Activity** permission](https://experienceleague.adobe.com/docs/marketo/using/product-docs/administration/users-and-roles/descriptions-of-role-permissions.html#access-api)),
- and that the Marketo API user has access to the Marketo [workspaces and partitions](https://experienceleague.adobe.com/en/docs/marketo/using/product-docs/administration/workspaces-and-person-partitions/understanding-workspaces-and-person-partitions) you want to use in Hightouch.
#### Code 1003 "Campaign is not part of a program"
To resolve this error, move your campaign into a program.
#### Code 1007 "Multiple lead match lookup criteria"
To resolve this error, remove any duplicate rows with the same primary key in your model.
#### Code 1013 "Record not found"
This error can happen if you select [Update mode](#supported-syncing) when syncing to standard and custom objects. If a record doesn't exist in Marketo yet, you need to select [Upsert mode](/syncs/types-and-modes#sync-modes) instead, which updates existing records and creates new ones, if needed.
{/* */}
### Live debugger
### Sync alerts
---
## Mattermost
**URL:** https://hightouch.com/docs/destinations/mattermost
**Description:** Sync your warehouse with Mattermost and push business metrics and user activity data to Mattermost in real-time
**Section:** Destinations
## Overview
With the Mattermost destination, Hightouch can serve as a notifier for various changes. This is a very flexible integration, but here are a few example use cases:
1. Notify your customer success teams when product usage for a key account suddenly drops
2. Notify an account executive when product usage surpasses a certain threshold
3. Send a daily summary of the number of sign ups, compared to the same day last month
4. Send a daily summary on the adoption of a core feature
## Setup
Use personal access tokens to connect to Mattermost. Check with your Mattermost administrator that personal access tokens are enabled.
1. In Mattermost, click your icon in the top right and go to Profile to access your personal settings
2. Click the Security tab in the modal that appears
3. Click Personal Access Tokens and create the token
## Sending Mattermost messages \(added / changed / removed\)
Hightouch will not send any messages on the initial run to prevent a flood of
messages to your channel. Messages will start after the this initial sync.
Hightouch supports sending messages based on rows that have been added, changed, or removed since the last sync run. Each row is sent as an individual message and can be sent to different channels and templated with Liquid.
Message formatting is best when the following conditions are met:
1. You want to write a query, and have Hightouch detect the added and removed rows
2. You want a custom message sent per each added and removed rows
### Channel selection
Hightouch supports both sending messages to a specific channel that the Hightouch bot is part of, or by pulling the channel ID each row. Enable the "Use column" toggle to dynamically set the target channel.
### Added, changed, and removed messages
Hightouch allows you to template the content you would like to send in your message.
- When a row is added in your query results, Hightouch will send the row added content.
- When a row is changed in your query results, Hightouch will send the row changed content.
- When a row is deleted in your query results, Hightouch will send the row deleted content.
This content can be [templated](#liquid-templating).
## Sending Mattermost messages \(all results\)
The batch message mode is similar to the normal message mode, except it sends all rows in the query results rather than only added / changed / removed rows since the last sync.
This mode also allows you to specify how many rows are sent per message, which can range from 1 to 20. With 1 row per message, it would be similar to the message mode, whereas 20 rows per message would be similar to a tabular representation.
This format is good for the following scenarios.
1. You want to write a query, and have Hightouch push a message containing **all** rows, formatted as a message, to Mattermost
2. You don't want Hightouch to detect added and removed rows, but instead want to send all rows on each query run
### Channel selection
The batch message only supports sending your messages to a single channel.
### Header, body, and footer content
For each message, Hightouch allows setting the header, footer, and row content. Only the row content is required. The row content is repeated per row in the message and has access to the row values via templating.
This content can be [templated](#liquid-templating).
## Templating messages
### Liquid templating
For both the message and batch message modes, [Liquid templating ](https://shopify.github.io/liquid/)is supported. The most important implication of this is that columns from the added or removed row can be accessed as part of the message body. Here is an example message making use of Liquid to insert a column into the message.
```text
User with name {{ row["name"] }} and email {{ row["email"] }} has signed up
```
### Tagging Mattermost users \(@...\)
Hightouch supports tagging users by either their Mattermost username \(if you have it in your warehouse\) or by email.
- **Tagging by username:** tag by username by writing `<@{{ row["username"] }}>` where username is the column that contains the user's Mattermost username, just as using Liquid templating. If the username is a static value, you can write something like `<@exampleuser>`.
- **Tagging by Email:** tag by email by writing `<@{{ row["email"] | mattermost_user_lookup }}>` where email is the column that contains the user's email used in Mattermost. If the email is a static value, you can write something like `<@{{ "example@gmail.com" | mattermost_user_lookup }}>`.
To tag by email, we require the permission users:read.email.
### Unfurling links and blocking unfurl
By default, Mattermost automatically unfurls (displays a preview) URL links in your messages. To prevent automatic unfurling, surround the URL with backticks, for example:
`https://www.hightouch.com/`
(Note: The backtick is usually located above the tab key on the left-hand side of the keyboard).
## Table
Table formatting sends all of your query results formatted as a text table as messages to Mattermost. This mode is recommended only for small sets of data.
This format is good for the following scenarios.
1. You want to write a query, and have Hightouch push a message containing **all** rows, formatted as a table, to Mattermost
2. You don't want Hightouch to detect added and removed rows, but instead want to send all rows on each query run
### Channel selection
The batch message only supports sending your messages to a single channel.
## Using webhooks
If you enable the option to use webhooks for sending messages, ensure the user whose access token was used has permissions to create and get webhooks. Using webhooks allows you to set how the sender is displayed when sending the message. This can be configured with the following fields
| Name | Description |
| :------------- | :---------------------------------------------------------------------------- |
| Icon Image URL | URL of 48x48 image of the icon that you wish to use |
| Icon Emoji | Mattermost emoji \(for example, `:sweat_smile:`\), this will overwrite icon image URL |
| Bot Name | Display name of bot |
(Note: Using webhooks for the CSV mode is not supported since files can't be uploaded using webhooks)
---
## Maxio
**URL:** https://hightouch.com/docs/destinations/maxio
**Description:** Unlock the full potential of your subscription business by syncing data from your data warehouse to Maxio
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
------------------|-----------------------------------------------------------------|----------------------|------------------------------------------------------------------------------------------------------
**Customers** | Sync data from any source to Customers | Upsert, Update, Insert | [Customers docs](https://developers.chargify.com/docs/api-docs/18237bcfe5cbb-create-customer)
**Subscriptions** | Sync data from any source to Subscriptions | Upsert, Update, Insert | [Subscriptions docs](https://developers.chargify.com/docs/api-docs/d571659cf0f24-create-subscription)
**Usage** | Sync data from any source to Usage | Insert | [Usage docs](https://developers.chargify.com/docs/api-docs/9954fb40d117d-create-usage)
**Events** | Sync data from any source to Events-Based Billing | Insert | [Event docs](https://developers.chargify.com/docs/api-docs/b8ffb590f236a-event-ingestion)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Maxio
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Maxio** and click **Continue**. You can then authenticate Hightouch to **Maxio** by entering the following required fields into Hightouch:
- **Subdomain**: Part of your URL, for example, `example-subdomain` in `https://example-subdomain.chargify.com`
- **API key**: A valid Maxio API key, follow [Maxio's instructions](https://maxio-chargify.zendesk.com/hc/en-us/articles/5405281550477#api) to view or create a new API key
- **Password**: The same one you used to log into your account
## Sync configuration
Once you've set up your Maxio destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Maxio destination you want to sync to.
### Syncing customers
When syncing customers, you can choose from upsert, update, or insert modes.
#### Record matching
To match rows from your model to customers in Maxio, you need to select a model column and corresponding Maxio field. You can match on any of the following Maxio fields:
- **email**
- **id**
- **reference**
In **Insert** mode, Maxio automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in Maxio. Maxio requires the following fields, so you must map them to complete your configuration:
- **first_name**
- **last_name**
- **email**
Ensure the data types of your model columns match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|----------------------------------------------------------
**Do nothing** | Keep the record in Maxio with all its synced fields
**Clear** | Unset all the mapped fields, but keep the record in Maxio
**Delete** | Delete the synced record from your Maxio customer
### Syncing subscriptions
When syncing subscriptions, you can choose from upsert, update, or insert modes.
#### Record matching
To match rows from your model to records in Maxio, you need to select the model column that contains values that match the **reference** field.
In **Insert** mode, Maxio automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in Maxio. Maxio requires the following fields, so you must map them to complete your configuration:
- **customer_attributes**, **customer_id** or **customer_reference**
- **payment_profile_id**
- **product_handle** or **product_id**
Ensure the data types of your model columns match the data types of the fields you want to sync to.
### Syncing usage data
Hightouch lets you report usage events to Maxio. To do so, you must select the Maxio subscription and component you would like to sync to.
#### Record matching
Maxio automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
You can sync columns from your model to Maxio default usage fields. Check out Maxio's [usage docs](https://developers.chargify.com/docs/api-docs/9954fb40d117d-create-usage) to learn more.
### Syncing events
Maxio requires the following event parameters:
- `api-handle`: You can either provide a static value or select to **use a column** from your model.
- `subdomain`: Hightouch uses the subdomain you entered when [connecting to Maxio](#connect-to-maxio)
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server. Hightouch syncs the timestamp as the `event_time` parameter the Maxio requires.
#### Record matching
Maxio automatically generates an identifier for every new event synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync event properties via default and custom mappings.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Medallia
**URL:** https://hightouch.com/docs/destinations/medallia
**Description:** The leading experience management platform used by companies around the world to deliver exceptional experiences
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|-----------|------------------------------------------------------|----------------------|--------------------------------------------------------------------------------|
| **Users** | Sync data from any source to Medallia users | Upsert | [Users docs](https://developer.medallia.com/medallia-apis/reference/listusers) |
| **Data** | Sync data from any source to the Medallia Import API | Insert, All | [Import docs](https://developer.medallia.com/medallia-apis/reference/import-api) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Medallia
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Medallia** and click **Continue**. You can then authenticate Hightouch to **Medallia**.
Enter the following required fields into Hightouch:
- (Optional) **Company name**: Enter your company name.
- **Tenant name**: Enter your Medallia tenant name.
- **Client ID**: Enter your client ID.
- **Client secret**: Enter your client secret.
To obtain your client ID and client secret, go to your Medallia setup console and follow these steps:
- Navigate to **Integrations** → **Oauth**.
- Find the client you want to use and copy your **Client ID**.
- If you don't have a client secret for the selected client, generate a new client secret.
- Navigate to **Company** → **Users**
- Go to the **User Edit Roles** tab and ensure the client has sufficient permissions to edit the roles you will be syncing.
## Sync configuration
Once you've set up your Medallia destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Medallia destination you want to sync to.
### Syncing users
Sync data from any source to Medallia users.
#### Record matching
To match rows from your model to users in Medallia, you need to select the model column that contains the **Username** values.
#### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default contact fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
### Syncing to an Import API
Sync data from any source to any Medallia Import API.
In the **Which destination would you like to import to?** field, enter the **URL Unique Name** of the Medallia Web Feed you want to target. This value must exactly match the URL Unique Name configured in Medallia’s Web Feeds configuration screen. Medallia doesn’t automatically generate this value, so an admin can choose any string, but the value in Hightouch and the value in Medallia must match for imports to succeed.
This field is specific to Medallia and is separate from the Hightouch **Sync slug**, which is only used inside Hightouch for URLs and logging and is not read by Medallia.
#### Field mapping
Hightouch lets you sync import fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default contact fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
## Tips and troubleshooting
### Common errors
If your sync runs successfully in Hightouch but no data appears in Medallia, confirm that the value in **Which destination would you like to import to?** exactly matches the Web Feeds **URL Unique Name** in Medallia.
### Live debugger
### Sync alerts
---
## Meta Custom Audiences
**URL:** https://hightouch.com/docs/destinations/meta
**Description:** Sync first-party audiences to Meta Ads Manager for better targeting, lookalikes, and suppression across your campaigns.
**Section:** Destinations
## Overview
Deliver even more relevant ads by combining data from various sources within your data warehouse to build custom audiences for Meta Ads. By keeping your custom audiences updated automatically, never show an ad to someone after they purchase the item you were promoting.
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API reference |
| ------------- | -------------------------------------------------------------------------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------- |
| **Audiences** | Create and update [custom audiences](https://www.facebook.com/business/help/744354708981227) in Meta | Add, Remove | [Custom Audiences docs](https://developers.facebook.com/docs/marketing-api/audiences/guides/custom-audiences) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Meta Custom Audiences
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Meta Custom Audiences** and click **Continue**. You can then authenticate Hightouch to Meta Custom Audiences either with a system user token or with OAuth.
Since the OAuth flow requires you to manually refresh this connection every 60 days, we strongly recommend using a system user token for indefinite access.
Make sure to select an **Ad Account** on the destination configuration page.
Otherwise, your sync fails with an [`'undefined' does not
exist`](#unsupported-post-request-object-with-id-undefined-does-not-exist)
error.
### Authenticate with a system user token
When authenticating with a system user token you can choose to either use an existing one or create a new one for the Hightouch integration.
To use an [existing system user token](#use-an-existing-system-user-token), you must meet the following prerequisites:
- The system user whose token you'd like to use has `Admin` access.
- You've created an app to add as an assigned asset to the system user. For detailed instructions, see steps 1 - 7 from [creating a new system user token](#create-a-new-system-user-token).
- You've assigned the ad account that you want to create audiences for to the system user. For detailed instructions, see step 9 from [creating a new system user token](#create-a-new-system-user-token).
If you haven't met these prerequisites, follow the instructions for [creating a new system user token](#create-a-new-system-user-token).
#### Create a new system user token
To create a new token, you first need to add a new app.
1. Open your Meta Business account and navigate to **Business Settings** > **Accounts** > **Apps**. Click **Add**.
2. Choose **Create a new app ID**.
3. Name your app.
4. Select the use case as **Other**.
5. Set the app type to **Business**.
6. Click **Create app**.
7. Make sure the app has `Ads Management Standard Access` permissions. You can find this setting in **App Review** > **Permissions and Features**.
8. Navigate to **Business Settings** > **Users** > **System users** and add a new system user with `Admin` access.
9. Use the **Add assets** button to assign **_both_** the app you created and the ad account that you want to create audiences for to the system user. Be sure to give **_both_** the app and the ad account **Full Control**, not **Partial Access**.
10. Once a new system user account is created and assets are assigned, select **Generate token**.
11. Choose the app you created to generate the token.
12. Select the expiration policy for the token.
13. Ensure that the `ads_management` permission is selected, then click **Generate token**.
14. Copy the generated token. Since this token won't be stored in Meta, you may want to consider storing it in a secure password vault as well.
15. Back in Hightouch, select **Use system user token** as the **Authentication method**, then paste the token you generated as the **System User Token**, select the relevant **Ads Account**, and click **Continue**.
#### Use an existing system user token
If you already have a Meta app and system user setup, open your Meta Business account and navigate to **Business Settings** > **Users** > **System users**.
1. Once you've selected the existing user and assets are assigned, select **Generate token**.
2. Choose the app you created to generate the token.
3. Select the expiration policy for the token.
4. Ensure that the `ads_management` permission is selected, then click **Generate token**.
5. Copy the generated token. Since this token won't be stored in Meta, you may want to consider storing it in a secure password vault as well.
6. Back in Hightouch, select **Use system user token** as the **Authentication method**, then paste the token you generated as the **System User Token**, select the relevant **Ads Account**, and click **Continue**.
### Authenticate with OAuth
For the **Authentication method**, select **Log in to Meta** and log into your Meta account.
Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Meta Custom Audiences destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Meta Custom Audiences destination you want to sync to.
### Syncing custom audiences
Hightouch lets you create and maintain custom audiences via the [Meta Custom Audience API](https://developers.facebook.com/docs/marketing-api/reference/custom-audience/).
Each sync adds new users and updates existing users' identifiers you include in [record matching](#record-matching).
You can remove users that leave your model's query result by selecting this option in the sync's [delete behavior](#delete-behavior).
#### Select an existing audience or create a new one
You can create a new audience or use an existing one. When creating a new audience, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model. To use an existing audience, select the desired audience from the dropdown.
#### User identifiers
To identify which users to add or update in an audience, select model columns and the corresponding Meta Custom Audience fields.
You can match on default, custom, and Meta-specific fields.
Meta-specific fields include:
- [**External ID**](https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/external-id/): a custom identifier that you send to Meta via its APIs or tracking pixels
- **Page-scoped ID**: an ID used for Meta messenger apps and Meta pages
To increase the match rate for your records, include as many fields as possible.
#### Value-based audiences
If you're creating a new Meta Audience, you can create a [value-based audience](https://www.facebook.com/business/help/917879191754763?id=401668390442328) by mapping a numerical model column containing lifetime values to the **Lookalike Value** field.
You can then create a lookalike audience in Meta that finds the people most similar to your highest value customers. For more information refer to [Meta's docs](https://www.facebook.com/business/help/458132024732845?id=401668390442328).
#### Handling PII and hashing
By default, Hightouch automatically hashes the following fields before sending
requests to Meta:
- **Email**
- **Phone**
- **Gender**
- **First Name**
- **Last Name**
- **First Initial**
- **State**
- **City**
- **Zip Code**
- **Country**
You can disable this behavior in the sync configuration. If disabled, the data from the model should be appropriately normalized and hashed according to [Meta's hashing requirements](https://developers.facebook.com/docs/marketing-api/audiences/guides/custom-audiences/#hash).
#### Share your audience
Optionally share this audience with another Meta advertising account.
#### Delete behavior
You can choose how to handle user records in Meta when the corresponding rows are deleted in your source.
| Behavior | Description |
| -------------------- | ----------------------------------------------------- |
| **Do nothing** | Keep the contact in your Meta Custom Audience |
| **Remove from list** | Remove the contact from your Meta Custom Audience |
## Tips and troubleshooting
### Matched users count
Meta returns 1k as the default when there is no data or audience is too
small
### Common errors
{/* */}
#### Custom Audience Terms not yet accepted
{/* */}
The full error message is:
```txt
An error occurred when creating the audience. Meta Custom Audiences API returned error with code 400: Custom Audience Terms not yet accepted: You'll need to agree to the Custom Audience terms before you can create or edit an audience or an ad set. See Meta, Custom Audience Terms.
```
To resolve this error, go to [Audiences](https://adsmanager.facebook.com/adsmanager/audiences) in your Meta Ads Manager account and click **Create a Custom Audience** to create a list a manually. This prompts the Terms of Service to appear so you can agree to them.
You can also go to `https://business.facebook.com/ads/manage/customaudiences/tos/?{ACCOUNT_ID}` to accept the Terms of Service for each ad account that you want to use. Replace `{ACCOUNT_ID}` with the ad account ID in `act_xxxx` format.
Once you've accepted the Terms of Service, rerun your sync with the error.
#### Error Code 400: Missing schema attribute in payloads
By default, Hightouch automatically [hashes certain fields](#handling-pii-and-hashing) before sending them to Meta.
The error occurs when Hightouch attemps to hash a `null` value or `null` values are being sent to Meta.
To resolve this, [remove rows containing `null` values](/models/sql-editor#exclude-rows-with-null-values) from your model's query results or enable the [**Don't sync null values** option](/syncs/mapping-data#dont-sync-null-values) in the [advanced mapper](/syncs/mapping-data#advanced-mapper).
#### Unsupported post request. Object with ID 'undefined' does not exist
This error occurs if you don't select an **Ad Account** on the destination configuration page.
Make sure to follow the destination setup procedure outlined in the [Connect to Meta Custom Audiences](#connect-to-meta-custom-audiences) section.
#### 401 - Error validating access token
As explained in the [Connect to Meta Custom Audiences](#connect-to-meta-custom-audiences) section, authenticating with OAuth requires you to reauthorize your destination **every 60 days**. Otherwise, your syncs fail with this error message.
Authenticate with a [system user
token](#authenticate-with-a-system-user-token) to avoid this reauthorization
requirement.
#### 400 - (#100) unsupported get request
The full error may look like: `400 - {"error":("message":"(#100) Unsupported get request. Please read the Graph API documentation at https://developers.facebook.com/docs/graph-api {...}`.
This error usually happens when one or multiple Meta pages that were granted access to the [system user](#authenticate-with-a-system-user-token) have **age restrictions** or **country restrictions** enabled. To resolve the error, edit your system user to remove all pages that have these age or country restrictions. Otherwise, you can [edit the page settings](https://www.facebook.com/help/www/778445532225441) to remove the restrictions from these pages.
#### Clear and fill
Upon request, [clear and fill](/syncs/overview#clear-and-fill) can be enabled for Meta. Running the clear operation for Meta may take up to 24 hours to complete.
### Live debugger
### Sync alerts
---
## Meta Product Catalog
**URL:** https://hightouch.com/docs/destinations/meta-catalog
**Description:** Power Meta's Advantage+ and dynamic advertising campaigns with up-to-date product data synced directly from your data warehouse.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| --------- | ------------------------------------------ | -------------------- |
| Products | Sync records to products in your catalogs. | Upsert |
## Setup
If you don't have a product catalog already, navigate to Meta [Commerce Manager](https://business.facebook.com/commerce/) and click "Add Catalog" to create one.
## Connect to Meta Product Catalog
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Meta Product Catalog** and click **Continue**.
You can then authenticate Hightouch to your Meta Product Catalog either with a system user token or with OAuth.
Since the OAuth flow requires you to manually refresh this connection every 60 days, we strongly recommend using a system user token for indefinite access.
Make sure to select an **Ad Account** on the destination configuration page.
Otherwise, your sync fails with an [`'undefined' does not
exist`](#unsupported-post-request-object-with-id-undefined-does-not-exist)
error.
### Authenticate with a system user token
When authenticating with a system user token you can choose to either use an existing one or create a new one for the Hightouch integration.
To use an [existing system user token](#use-an-existing-system-user-token), you must meet the following prerequisites:
- The system user whose token you'd like to use has `Admin` access.
- You've created an app to add as an assigned asset to the system user.
- You've assigned the ad account where your catalog lives for to the system user.
### Authenticate with OAuth
For the **Authentication method**, select **Log in to Meta** and log into your Meta account.
Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Meta Product Catalog destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data.
Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Meta Product Catalog destination you want to sync to.
### Syncing products
Hightouch supports syncing to the following Meta catalog products:
- `Product items`
- `Vehicles`
### Record matching
Product records can be matched from your source to your Meta catalog workspace by the product ID in the Upsert mode.
### Field mappings
You can sync columns from your source to Meta catalog's default and custom fields. Hightouch automatically detects existing custom fields from your Meta catalog account.
Review the [documentation](https://developers.facebook.com/docs/marketing-api/catalog-batch/reference#supported-fields-items-batch) on what fields are required to sync to your catalog type.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Meta Conversions API
**URL:** https://hightouch.com/docs/destinations/meta-conversions
**Description:** Optimize campaigns and gain insight into ad performance by sending web, app, phone, and in-store conversion events to Meta from your data warehouse.
**Section:** Destinations
## When to use Meta Conversions API
Use this destination to improve Meta campaign performance and measurement by sending offline or server-side conversion events (for example, purchases, form fills, or calls) to Meta.
**Use Meta Conversions API when you want to:**
- Attribute conversions that happen outside the browser (for example, in your CRM or POS system)
- Improve ad delivery and targeting with more complete conversion data
- Measure performance when cookies or browser signals are limited
---
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-----------|-------------------------------------------------------------|----------------------|-----------------------------------------------------------------------------------
**Events** | Sync conversion events to Meta’s Conversions API | Insert | [Conversions API reference](https://developers.facebook.com/docs/marketing-api/conversions-api/)
See [Sync modes](/syncs/types-and-modes#sync-modes) for more information.
## Connect to Meta Conversions API
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Meta Conversions** and click **Continue**. You can then authenticate Hightouch to the Meta Conversions API by entering the following required fields into Hightouch:
1. In Hightouch, go to [**Destinations**](https://app.hightouch.com/destinations) and click **Add destination**.
2. Select **Meta Conversions API** and click **Continue**.
3. Enter the following credentials:
- **Pixel ID** or **Database ID**
- **Access Token**
Find these in [Meta Events Manager](https://business.facebook.com/events_manager2/):
1. Open the **Data Sources** tab and select your Pixel.
2. Under **Settings**, find your **Pixel ID** in the **Details** section.
3. To create an Access Token, scroll to **Conversions API** and click **Generate access token**.
---
## Configure the sync
After connecting Meta Conversions API and selecting a [model](/getting-started/concepts#models), create a sync to start sending data.
1. Go to [**Syncs**](https://app.hightouch.com/syncs).
2. Click **Add sync**, choose your model, and select **Meta Conversions API** as the destination.
### Required event fields
The Meta Conversions API requires the following event parameters:
- `event_name`
- `event_time`
- At least one [customer information (`user_data`)](https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/customer-information-parameters) field
The sync configuration form ensures all these are set and provides some additional options.
#### Event name
Providing an event name is required to send an event to the Meta Conversions API. You can either provide a static value or select to **use a column** from your model. Hightouch syncs the static or column value as the `event_name` parameter that the Conversions API requires.
Use Meta’s [Standard Events](https://www.facebook.com/business/help/402791146561655?id=1205376682832142) when possible (for example, `Purchase`, `Lead`, or `AddToCart`) to improve reporting accuracy.
#### Event time
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server. Hightouch syncs the timestamp as the `event_time` parameter the Conversions API requires.
If you select a column, it should be in [ISO 8601](https://www.iso.org/iso-8601-date-and-time-format.html) format. Hightouch will then automatically convert the data into the format that Meta expects.
#### Action source
The [`action_source`](https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/server-event#action-source) parameter specifies where your conversions occurred (e.g., web, app, email).
Knowing where your events took place helps Meta optimize delivery and attribution.
Certain source types require specific fields:
- **Web** events must include `event_source_url` and `client_user_agent`.
- **App** events must include `advertiser_tracking_enabled`, `application_tracking_enabled`, and `extinfo`.
If not all rows contain this field, you can set a **fallback action source** in your sync configuration.
#### Field mappings
Map your customer information and event parameters here.
The Meta Conversions API requires at least one [`user_data`](https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/customer-information-parameters) parameter for attribution.
Include as many `user_data` fields as possible to improve [Event Match Quality](https://www.facebook.com/business/help/765081237991954?id=818859032317965).
If using a **Standard Event** (like `Purchase`), ensure you include all required fields (e.g., `value`, `currency`) and any relevant optional ones.
Custom fields are sent in the `custom_data` parameter.
#### Advanced analytics
To send data through the Conversions API to the Advanced Measurement API (AMAPI), map the following fields:
- `data_processing_options`: an array of strings like `["AMO"]`
- `advanced_measurement_table`: a string
#### Test event codes
Test event codes are special codes you can include in your event payload to simulate conversion events. This allows you to test your API implementation without actually recording real conversion data.
You can optionally enter a test event code as part of your sync configuration. You can find your test event code in your Meta [Events Manager](https://business.facebook.com/events_manager2/) in the **Test events** tab.
Make sure to remove the test event code if you're running a production sync.
#### Hashing
By default, Hightouch automatically hashes the following fields:
- **email**
- **phone**
- **gender**
- **dob**
- **last name**
- **first name**
- **city**
- **state**
- **zip**
- **country**
If you want to turn hashing off, select **No** under **Would you like Hightouch to automatically detect if your PII data requires hashing?**.
---
## Event Data Quality
Hightouch provides insight into the effectiveness of your conversion API implementation by reporting on **Event Match Quality (EMQ)** and **Additional Conversions Reported**.
Data quality metrics appear in the **Overview** tab of your Meta Conversions API sync.
Your access token must include Data Quality API permissions. If metrics don’t appear, generate a new token in Meta Events Manager.
### Event Match Quality (EMQ)
The EMQ score estimates how well your conversion events match to Meta users, ad clicks, and impressions. It’s based on the “fill rate” of key `user_data` values you send with each conversion event.
A higher EMQ score generally means better attribution accuracy and ad performance, as Meta’s algorithms can use more complete data to optimize bids, budgets, and targeting.
To improve your EMQ score:
- Collect and sync more `user_data` fields with each event.
- Use [**Match Booster**](/match-booster/overview) to supplement user data with Hightouch’s third-party identity graph.
For guidance on which user attributes to prioritize, see the [Conversion API schema guide](https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/customer-information-parameters/). For more about EMQ, see [Meta’s documentation](https://www.facebook.com/business/help/765081237991954?id=818859032317965).
### Additional Conversions Reported
The Additional Conversions Reported metric helps you understand how much your business benefits from using the Conversions API alongside the Meta Pixel. More reported conversions can help you decrease your cost per result and show your ads to people who find them relevant.
Learn more in [Meta’s Additional Conversions Reported guide](https://www.facebook.com/business/help/409195343791109).
---
## Tips and troubleshooting
### Common errors
#### Meta SDK
When setting up a new datasource for CAPI, if you don't want to use the Meta SDK in your app, choose “Web” (even if the data would include app events as well). This eliminates the need to install the SDK.
In this case, use **Dataset ID** instead of **Pixel ID** when configuring the destination.
#### App data
To send app data in your CAPI events, include [app data parameters](https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/app-data#).
### Live debugger
### Sync alerts
---
## Microsoft Dynamics 365
**URL:** https://hightouch.com/docs/destinations/microsoft-dynamics
**Description:** Sync data into Dynamics 365 by writing to Dataverse tables.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
------------|-------------------------------------------------------------|------------------------|----------------------------------------------------------------------------------------------------------------------------------
**Objects** | Sync data from any source to Microsoft Dynamics 365 objects | Upsert, Update, Insert | [Objects docs](https://learn.microsoft.com/en-us/power-apps/developer/data-platform/webapi/reference/about?view=dataverse-latest)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Microsoft Dynamics 365
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Microsoft Dynamics 365** and click **Continue**. You can then authenticate Hightouch to **Microsoft Dynamics 365**.
### Authenticate with oauth
For the **Authentication method**, select **Log in to Microsoft Dynamics 365** and log into your Microsoft Dynamics 365 account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
You must enter your **Web API endpoint** in Hightouch to complete the setup. See [How to get your **Web API endpoint**](microsoft-dynamics#how-to-get-your-web-api-endpoint) for information on how to get it.
#### Access level recommendations
We recommend that you authenticate with a user that has the correct access to the environment that you will be working on.
### Authenticate with client credentials
Enter the following required fields into Hightouch:
- [**Web API endpoint**](microsoft-dynamics#how-to-get-your-web-api-endpoint)
- [**Tenant ID**](microsoft-dynamics#setting-up-your-app-registration)
- [**Client ID**](microsoft-dynamics#setting-up-your-app-registration)
- [**Client Secret**](microsoft-dynamics#setting-up-your-app-registration)
#### Setting up your app registration
1. Go to [Azure Ad portal](https://portal.azure.com/) > **App registrations**.
2. Create a single tenant app registration.
3. Once an app registration is created, copy the **Tenant ID** and **Client ID** in Hightouch.
4. To get a **Client Secret**, navigate to **Certificates & secrets** > **New client secret**.
Copy the value of **Client Secret** in Hightouch.
5. In order to use the created app registration with Microsoft Dynamics 365 API, You must grant API permission to it by going to **API Permissions** > **Add Permissions** then select **Dynamics CRM** and add the `user_impersonation` permission.
6. Finally, create a non-interactive application user with the app registration in the [Power Platform admin center](https://admin.powerplatform.microsoft.com/) with `Service Writer` role. To do this **Environments** > **[the environment you want to connect]** > **Settings** > **Application users** > **New app user**.
## How to get your Web API endpoint
In order to complete your Dynamics 365 connection setup, you will need to provide your service URL. Follow these steps to get your Web API endpoint:
1. Sign in to your [Microsoft Power Apps](https://powerapps.microsoft.com/).
2. Click the settings icon on the top right corner to open the settings sidebar.Then, click on *Developer resources*.
3. Copy the *Web API endpoint* then go back to Hightouch and paste the URL where it's required.
## Prerequisites
Before you set up a sync, we recommend you to skim through these concepts to gain an insight on how to effectively sync your data into Microsoft Dynamics 365.
### Dataverse
Dataverse is the source of truth for your Dynamics 365 applications.
Microsoft Dynamics 365 destination will sync your data into the Dataverse which will then allows you to use the data directly in your Dynamics 365 applications.
For more information about Dataverse see [Microsoft's documentation](https://docs.microsoft.com/en-us/power-apps/maker/data-platform/data-platform-intro).
#### What is a GUID?
GUID is the auto-generated primary key in your Dataverse tables. You are allowed to use GUID to match query results in Hightouch with rows in your Dataverse tables by default.
{" "}
Although GUIDs are available by default, we recommend the use of alternate keys
for record matching in your Hightouch syncs.
#### What are alternate keys?
[Alternate keys](https://docs.microsoft.com/en-us/dynamics365/customerengagement/on-premises/developer/define-alternate-keys-entity) are custom unique identifiers that can be derived from one or more columns in your Dataverse table.
it's a flexible way to set up a unique identifier in your Dataverse table to be use for record matching between your query results in Hightouch and your rows in Dataverse.
In most cases, you won't have access to Dataverse GUID in your external data warehouse for record matching so alternate keys are convenient solution.
#### Steps to setup and use an alternate key
1. Sign in to your [Microsoft Power Apps](https://powerapps.microsoft.com/).
2. In the Power Apps, navigate to tables in Dataverse.  Select the table you want to sync to. In this walk through, we will use the `Account` table.
Please make sure that you are always working on the correct environment in
Power Apps. If you haven't set up your environment yet, go to this
[link](https://docs.microsoft.com/en-us/power-platform/admin/create-environment)
for more information. Here is an
[article](https://powerapps.microsoft.com/en-us/blog/establishing-an-environment-strategy-for-microsoft-power-platform)
from Microsoft on environment strategy.
3. Once you are in the table view, select `keys` to open the alternate keys view. 
4. In the keys page:
(1) Click `New Key` then in the pop-up on the right, enter the configuration for the key you are about to create.
(2) Enter a label for this key. This label is the label that will be displayed in Hightouch's ID mapping section.
(3) Select a column that you want to use as an alternate key.
Note that alternate keys must be unique and must meet the requirements described in the [Dynamics 365 docs](https://docs.microsoft.com/en-us/dynamics365/customerengagement/on-premises/developer/define-alternate-keys-entity?view=op-9-1). If you do not have a column that meets these requirements, you can manually add a column and use it. In our example below, We're using a column that we created called `Custom External ID`.
{" "}
{" "}
If you are creating a new column to be used as external ID, you may have to
backfill the new column first before they can be used to avoid accidental duplicates.
(4) Once configured, hit save button to create the alternate key.
5. The created alternate keys will appear as an option in the identifier section of your Hightouch sync configuration. Select the name of your alternated key then map the columns for that alternate key in the **ID fields mappings** section below.
If your alternate key is composed of multiple columns, you will have to map those columns in the **ID fields mappings** section.
## Sync configuration
Once you've set up your Microsoft Dynamics 365 destination and have a [model](/getting-started/concepts#models) to pull data, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Microsoft Dynamics 365 destination you want to sync.
### Syncing objects
Sync data from any source to Microsoft Dynamics 365 objects.
#### Record matching
To match rows from your model to objects in Microsoft Dynamics 365, you need to select a model column and corresponding Microsoft Dynamics 365 field. You can match on any of the following Microsoft Dynamics 365 fields:
- [**GUID**](microsoft-dynamics#what-is-a-guid)
- [**Alternate Keys**](microsoft-dynamics#what-are-alternate-keys)
In **Insert** mode, Microsoft Dynamics 365 automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync object fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default object fields.
**Field types**
Ensure your model columns data types match the data types of the fields you want to sync to. Microsoft Dynamics 365 destination will not automatically cast your data to the data types they are expected to be in your table in Dataverse. This means that you are responsible for casting them to the valid type they need to be in. The image below shows an example of fields we pulled from Dataverse along with their types. The icons on the left indicate the type of the column. As you may notice, there are icons with `?`. This indicates that the type is either complex or unexplored by Hightouch. Reference the [Dynamics 365 docs](https://docs.microsoft.com/en-us/dynamics365/customerengagement/on-premises/customize/types-of-fields?view=op-9-1) for more details on types.
**Required fields**
You may have noticed that `First Name` and `Last Name` appear to be read-only in the destination side while `Description` is editable.
These read-only fields are fields that are marked as `Business required` in Dataverse.
This means that whenever you create a sync, you must send data for these fields.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|---------------------------------------------------------------------------
**Do nothing** | Keep the object in Microsoft Dynamics 365 with all its synced fields
**Clear** | Clear all the mapped fields, but keep the object in Microsoft Dynamics 365
**Delete** | Delete the synced objects from Microsoft Dynamics 365
## Best practices
#### Validate your configuration
To validate that your configuration is working, we highly recommend that your to run a test.
Click on `Test` button to open the test pop-up.
Click on the `Sync as added row` to make the request.
The result on the right will indicate if the request to Dataverse is successful.
Be aware that requests made through here will persist data into your
destination as if you ran an actual sync. When working on a production
environment, please make sure that you fully understand the risks.
## Tips and troubleshooting
### Need admin approval
If you see a **Need admin approval** page appear when [accessing Microsoft's OAuth login flow](#setup), your Azure Active Directory user account might not have the necessary permissions to connect applications to your organization.
Ask one of your Azure organization's admins to edit your user account permissions.
You can read more about [Azure user permissions](https://learn.microsoft.com/en-us/azure/active-directory/develop/active-directory-how-applications-are-added#who-has-permission-to-add-applications-to-my-azure-ad-instance) and [configuring user settings](https://learn.microsoft.com/en-us/azure/active-directory/manage-apps/configure-user-consent?pivots=portal#configure-user-consent-settings) in Azure's documentation.
### Common errors
### Live debugger
### Sync alerts
---
## Microsoft Excel
**URL:** https://hightouch.com/docs/destinations/microsoft-excel
**Description:** Sync warehouse data to Excel worksheets used by sales, ops, and marketing teams
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
-------------|-----------------------------------------------------------------------------|---------------------
**Mirror** | Sync the full result set of the model to the selected destination worksheet | All
**Snapshot** | Create a new worksheet on every sync with the full result set of the model | All
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Microsoft Excel
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination**
button. Select **Microsoft Excel** and click **Continue**. You can then authenticate Hightouch to Microsoft Excel by logging into your Microsoft Excel account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Microsoft Excel destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and
the Microsoft Excel destination you want to sync to.
### Sync mode
Your first configuration is the [sync mode](/syncs/types-and-modes#sync-modes). You can select either:
- **Mirror**: if you want the full result set to overwrite a particular sheet's contents
- **Snapshot**: if you want to create a new sheet on every sync; the created worksheet's name is composed of the model name and sync run timestamp, for example, `Model Name: YYYY-MM-DD T hh mm ss`.
If you select **Mirror** mode, please ensure that your sheet doesn't contain data you don't want be overwritten.
### Drive and workbook selection
You can select any drives the user you authenticated with has access to. When selecting a workbook, you can either select the workbook from a dropdown or choose to enter a workbook ID. Once you've selected the workbook, its worksheets populate the next dropdown.
### Field mapping
You can choose to **sync all columns** or select only certain model columns to sync to your Excel worksheet. When syncing all columns, the column names in Excel mirror the column names in your source. When selecting specific columns, you can enter new columns names. If you'd like to rename columns, even if you want to sync all columns, you need to map each one individually.
### Custom cell range
By default, Hightouch inserts data in a worksheet starting at cell A1. The inserted data clears and updates the entire sheet. You can optionally select a range of cells to insert the data to. This capablity lets your preserve formulas or notes in your spreadsheet's margins.
When entering a range into the **Custom Cell Range** use the format `A1:Z1000`, where `A1` is the top left cell to insert data into and `Z1000` is the bottom right cell.
Ensure you the custom cell range is large enough to contain the data you want to sync. If it isn't, the sync will fail with a [**Query results are too large for provided range**]() error.
## Tips and troubleshooting
### Need admin approval
If you see a **Need admin approval** page appear when [accessing Microsoft's OAuth login flow](#connect-to-microsoft-excel), your Azure Active Directory user account might not have the necessary permissions to connect applications to your organization.
Ask one of your Azure organization's admins to edit your user account permissions.
You can read more about [Azure user permissions](https://learn.microsoft.com/en-us/azure/active-directory/develop/active-directory-how-applications-are-added#who-has-permission-to-add-applications-to-my-azure-ad-instance) and [configuring user settings](https://learn.microsoft.com/en-us/azure/active-directory/manage-apps/configure-user-consent?pivots=portal#configure-user-consent-settings) in Azure's documentation.
### Common errors
#### My data isn't appearing in Excel
Make sure to refresh your Excel worksheet after your sync has completed to see the updated values.
#### Query results are too large for provided range
This error occurs if you've set a [custom cell range](#custom-cell-range) that isn't large enough for the dataset you're syncing. The error specifies if you need to update your row or column range.
### Live debugger
### Sync alerts
---
## Microsoft Teams
**URL:** https://hightouch.com/docs/destinations/microsoft-teams
**Description:** Teams is a business communication platform where people can work together more effectively, connect all their software tools and services, and find the information they need to do their best work — all within a secure, enterprise-grade environment.
**Section:** Destinations
## Overview
With the Microsoft Teams destination, Hightouch can serve as a notifier for various changes. This is a very flexible integration, but here are a few example use cases:
1. Notify your customer success teams when product usage for a key account suddenly drops
2. Notify an account executive when product usage surpasses a certain threshold
3. Send a daily summary of the number of sign ups, compared to the same day last month
4. Send a daily summary on the adoption of a core feature
## Supported syncing
| Sync type | Description |
| :---------------- | :--------------------------------------------------------------------------------------------------------------------------------------------- |
| **Message** | Sends individual messages for each row that was added, changed, or removed since the last sync. There will be no messages for the initial run. |
| **Batch Message** | Sends a formatted message (or multiple messages) of all the rows in the query results. |
| **Table** | Sends a formatted table of all rows in the query results as a single message. |
## Sending Teams messages \(added / changed / removed\)
Hightouch will not send any messages on the initial run to prevent a flood of
messages to your channel. Messages will start after the this initial sync.
Hightouch supports sending messages based on rows that have been added, changed, or removed since the last sync run. Each row is sent as an individual message and can be sent to different channels and templated with Liquid.
Message formatting is best when the following conditions are met:
1. You want to write a query, and have Hightouch detect the added and removed rows
2. You want a custom message sent per each added and removed rows
### Channel selection
Hightouch supports both sending messages to a specific channel that the Hightouch bot is part of, or by pulling the channel ID each row. Enable the "Use column" toggle to dynamically set the target channel.
### Added, changed, and removed messages
Hightouch allows you to template the content you would like to send in your message.
- When a row is added in your query results, Hightouch will send the row added content.
- When a row is changed in your query results, Hightouch will send the row changed content.
- When a row is deleted in your query results, Hightouch will send the row deleted content.
This content can be [templated](#liquid-templating).
## Sending Teams messages \(all results\)
The batch message mode is similar to the normal message mode, except it sends all rows in the query results rather than only added, changed, and removed rows since the last sync.
This mode also allows you to specify how many rows are sent per message, which can range from 1 to 20. With 1 row per message, it would be similar to the message mode, whereas 20 rows per message would be similar to a tabular representation.
This format is good for the following scenarios.
1. You want to write a query, and have Hightouch push a message containing **all** rows, formatted as a message, to Microsoft Teams
2. You don't want Hightouch to detect added, changed, and removed rows, but instead want to send all rows on each query run
### Channel selection
The batch message only supports sending your messages to a single channel.
### Subject and body content
For each message, Hightouch allows setting the subject and row content. Only the row content is required. The row content is repeated per row in the message and has access to the row values via templating.
This content can be [templated](#liquid-templating).
## Templating messages
### Liquid templating
For both the message and batch message modes, [Liquid templating ](https://shopify.github.io/liquid/)is supported. The most important implication of this is that columns from the added or removed row can be accessed as part of the message body. Here is an example message making use of Liquid to insert a column into the message.
```text
User with name {{ row["name"] }} and email {{ row["email"] }} has signed up
```
## Table
Table formatting sends all of your query results formatted as a text table as messages to Microsoft Teams. This mode is recommended only for small sets of data.
This format is good for the following scenarios.
1. You want to write a query, and have Hightouch push a message containing **all** rows, formatted as a table, to Microsoft Teams
2. You don't want Hightouch to detect added and removed rows, but instead want to send all rows on each query run
### Channel selection
The batch message only supports sending your messages to a single channel.
## Tips and troubleshooting
### Need admin approval
If you see a **Need admin approval** page appear when accessing Microsoft's OAuth login flow, your Azure Active Directory user account might not have the necessary permissions to connect applications to your organization.
Ask one of your Azure organization's admins to edit your user account permissions.
You can read more about [Azure user permissions](https://learn.microsoft.com/en-us/azure/active-directory/develop/active-directory-how-applications-are-added#who-has-permission-to-add-applications-to-my-azure-ad-instance) and [configuring user settings](https://learn.microsoft.com/en-us/azure/active-directory/manage-apps/configure-user-consent?pivots=portal#configure-user-consent-settings) in Azure's documentation.
### Common errors
### Live debugger
### Sync alerts
---
## MinIO
**URL:** https://hightouch.com/docs/destinations/minio
**Description:** Transfer data at scale from your warehouse to MinIO
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
| ------------ | -------------------------------------------------------------| -------------------- |
| Any data set | Sync data from a source to MinIO as CSV, JSON, or Parquet files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Prerequisites
To get started, you need:
- a [MinIO bucket](https://charts.min.io/)
- MinIO access key and secret key for use by Hightouch
## Connect to MinIO
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **MinIO** and click **Continue**. You can then authenticate Hightouch to MinIO by entering your **Bucket Name**. The Bucket Name should just be the name of the bucket, not a URL.
## Sync configuration
Once you've set up your MinIO destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the MinIO destination you want to sync to.
### Select file format
Hightouch supports syncing JSON, CSV, and Parquet files to Amazon MinIO.
### Enter filename
The filename or **object key** field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file.
Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it.
To keep different versions of the same results file, you can enable [versioning](https://docs.aws.amazon.com/AmazonMinIO/latest/userguide/Versioning.html) in your bucket, or your app can copy the data to another location.
If you are using an audience and would like to include the audience name, you will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter `-86400` (`24 hours * 60 minutes * 60 seconds`). If you want the filename's data to be one hour after the sync takes place, you would enter `3600` (`60 minutes * 60 seconds`).
### CSV options
If you're syncing to a CSV file, you have additional configuration options:
- **Delimiter**: Your options are comma (`,`), semicolon (`;`), pipe (`|`), tilde (`~`), and tab
- (Optional) Whether to **include a CSV header row** in the exported files
- (Optional) Whether to **include a [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark)** in the exported files, the BOM is ``
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `customer_id`, `email`, `first_name` and `last_name`, columns. These columns are mapped to new fields in the destination file as `id`, `email`, and `name`—a [templated](/syncs/mapping-data#template-mapping) concatenation of `first_name` and `last_name`—respectively. Hightouch exports these fields to new fields in the file and ignores all other columns from your results.
### Batch size
By default, and depending on your [sync mode](#supported-syncing), Hightouch sends one file to your MinIO bucket for each export. (In diff mode, Hightouch sends three files.)
By enabling batching, you allow Hightouch to send multiple files if your model query results are large. This can help avoid file size errors from MinIO.
### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
### Gzip compression
You can enable gzip compression to reduce the size of your exported files. Note
that Hightouch does not automatically add the `.gz` extension to your files. You
can add this or any other extension to your filename.
## Tips and troubleshooting
### Common errors
#### Access denied
The **ACCESS_DENIED** error is a permissions-related error. Ensure the crendtials in Hightouch match the credentials used during MinIO setup.
### Sync alerts
---
## Mixpanel
**URL:** https://hightouch.com/docs/destinations/mixpanel
**Description:** Perform deeper analysis and better understand user behavior by bringing enriched customer data from your data warehouse into Mixpanel
**Section:** Destinations
## Overview
Ideally, your data warehouse is the single source of truth for all your customer data.
When that's the case, using your warehouse as the source for your Mixpanel data enables richer product-usage analysis.
You can sync enriched data from your warehouse to create more granular segments on Mixpanel as well as bring in data from your engagement tools.
Moreover, you can ensure that your product and growth teams work with data that's always up-to-date and in the expected format.
Lastly, if your company also uses a Business Intelligence tool such as Looker or Mode, you can ensure that the data is consistent across both systems.
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|**Objects**|Sync records to objects such as users or organizations (groups) in your destination.|Upsert|
|**Events**|Sync records as events to your destination. This is often in the form of a track call.|Insert|
|**Lookup tables**|Sync records as a CSV file to enrich event and profile properties within Mixpanel.|All|
- In **Upsert** mode, new users and groups are inserted into Mixpanel and all attributes are kept up-to-date.
- In **Insert** mode, new events are inserted as track calls.
- In **All** sync mode, the entire lookup table is replaced with each sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Getting started
To configure your Mixpanel destination you need to provide the following information:
- Project ID
- Project Token
- Service Account Username and Secret
- Group Keys, when syncing group objects
### Project ID
To find your Project ID, click the Settings icon, then Project Settings.
Then navigate to Overview > Project Details.
### Project token
To find your Project Token, click the Settings icon, then Project Settings.
Then navigate to Overview > Access Keys.
### Service account
To receive your Service Account Username and Secret you must first create a Service Account.\
Click the Settings icon, then Project Settings.
Then navigate to Service Accounts > Add Service Account and copy the **Username** and **Secret**.
Be sure to give your Service Account **Owner** or **Admin** permissions.
### Group keys
If you are syncing group profiles or assigning users to groups, you need to indicate which Group Key to use.
To find your Group Keys, click the Settings icon, then Project Settings.
Then navigate to Overview > Group Keys. Copy the appropriate **Key** value.
## Syncing data
### Objects
You can decide to sync either User Objects or Group Objects.
When syncing Group Objects, enter the [appropriate group key](#group-keys) into the **Mixpanel group key** field.
#### Record matching
You can match records from your source to your Mixpanel workspace by Mixpanel User ID, email or Group ID.
#### Field mapping
You can sync columns from your source to your default and custom fields in Mixpanel.
If you're syncing User Objects, you can see two sections: one for native Mixpanel fields and one for custom fields.
If you're syncing Group Objects, you can only see one section.
#### Users location
By default, users uploaded using Hightouch won't have any associated location in Mixpanel.
To set a location, provide an IP address \(field `$ip`\), or longitude and latitude coordinates \(fields `$longitude` and `$latitude`\).
For more information, you can visit [Mixpanel's official documentation](https://help.mixpanel.com/hc/en-us/articles/115004499343-Tracking-Geolocation-with-Server-Side-Implementation).
#### Assigning users to groups
To assign users to groups, create a sync for User Objects.
Then, create a column in your model which contains the group IDs that you want to assign the user to.
For example, to sync organizations, use a column called `organizations` containing either a group ID \(`google`\) or an array of group IDs \(`[google, apple, microsoft]`\).
Then, [find your group key](#group-keys), and use it as the destination field to sync to.
### Events
Hightouch supports sending events of a given name, for example, `Signed Up`.
Hightouch lets you choose the column containing the event timestamp.
Hightouch accepts various date time formats for the timestamp and automatically converts them to the format required by Mixpanel.
#### Record matching
You can match records from your source to your Mixpanel workspace by Mixpanel User ID.
#### Field mapping
Hightouch allows you to pass data to the event properties of a Mixpanel event.
#### Assigning events to groups
To assign events to groups, create a sync to events.
Then, create a column in your model which contains the group IDs that you want to assign the event to.
\
For example, to sync organizations, use a column called `organizations` containing either a group ID \(`google`\) or an array of group IDs \(`[google, apple, microsoft]`\).
Then, [find your group key](#group-keys), and use it as the destination field to sync to.
### Lookup tables
To sync data to your Lookup tables, choose **Enrichment**.
Then choose the specific Lookup table you want to replace.
#### Record matching
You can match records from your source to your Mixpanel workspace by the ID mapping of your choice.
The ID mapping should reference the unique identifier field of the events or profiles you are seeking to enrich.
#### Field mapping
You may also sync columns from your source to your Mixpanel Lookup table's custom fields.
#### Creating lookup tables in Mixpanel
To create Lookup tables, you can import them directly in Mixpanel.
You can do this by clicking on the Data Management icon and then on Lexicon.
Navigate to Lookup Tables > Import > Lookup Table.
#### Limitations
Mixpanel's Lookup table endpoint comes with the following limits:
- Only 100 requests are allowed in a rolling 24 hour period. Please be mindful of this when choosing your sync schedule. Mixpanel recommends updating Lookup tables at most hourly.
- A Lookup table must not exceed 100 MB uncompressed (roughly 1-2M rows).
## Tips and troubleshooting
### Common errors
#### Changing projects
If you need to change your Project ID, you also need to update the API Key that you originally provided.
Project ID and API keys are paired together in the Mixpanel platform.
### Live debugger
### Sync alerts
---
## MoEngage
**URL:** https://hightouch.com/docs/destinations/moengage
**Description:** Build more robust campaigns on MoEngage with up-to-date customer data from your data warehouse
**Section:** Destinations
## Overview
Providing relevant and timely campaigns relies on fresh and accurate data in your MoEngage account. By automatically syncing customer data from your data warehouse into MoEngage, you don't need to worry about data consistency or staleness. Instead, you can focus on building world-class customer experiences.
## Supported syncing
| Type | Description | Supported Sync Modes |
| ------------ | ------------------------------------------- | -------------------- |
| **Users** | Sync data from any source to MoEngage users | Upsert |
| **Events** | Sync events to MoEngage | Insert |
| **Segments** | Add or remove users from MoEngage cohorts | Add, Remove |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to MoEngage
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **MoEngage** and click **Continue**. You can then authenticate Hightouch to **MoEngage**.
You need to provide the following information:
- **API ID**: You can find this in your MoEngage dashboard under **Settings** > **APIs** > **General**.
- **API Secret**: You can find this in your MoEngage dashboard under **Settings** > **APIs** > **General**.
- **Business Events API Key**: This is required for syncing business events. You can find this in your MoEngage dashboard under **Settings** > **APIs** > **General**.
- **Region**: Select the region your MoEngage account is hosted in.
## Sync configuration
Once you've set up your **MoEngage** destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the MoEngage destination you want to sync to.
### Syncing users
This sync type lets you sync data to MoEngage user profiles.
#### Record matching
You can match rows from your model to users in MoEngage on their **User ID**.
#### Field mapping
Hightouch lets you sync user fields via [field mapping](/syncs/mapping-data). You can map data from any of your model columns to standard or custom user attributes in MoEngage. MoEngage expects custom properties to follow a `snake_case` format.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ---------------------------------------------------------- |
| **Do nothing** | Keep the user in MoEngage with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the user in MoEngage |
### Syncing events
This sync type lets you send events to MoEngage. Hightouch treats any records added to your model as new events to be sent to MoEngage.
When creating a sync for events, you can specify the name of the event being tracked. You can either input a static name, or use a column from your model. You can also select a column that is used as the timestamp of when the event happened. If this field is empty, Hightouch will use the time of when the event arrives at the server.
#### Record matching
You can match rows from your model to users in MoEngage on their **User ID**.
#### Field mapping
You can map data from your model columns to event properties. Any mappings from this section will be synced as attributes in the event. MoEngage expects custom properties to follow a `snake_case` format.
### Syncing segments
This sync type lets you add or remove users from MoEngage cohorts. You can either sync to an existing cohort or create a new one.
Users must already exist in MoEngage before they can be added to a cohort.
#### Record matching
You can match rows from your model to users in MoEngage on their **User ID**.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| ---------------------- | --------------------------------------------------------------- |
| **Do nothing** | Keep the user in the MoEngage cohort with all its synced fields |
| **Remove from cohort** | Remove the user from the MoEngage cohort |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Moloco
**URL:** https://hightouch.com/docs/destinations/moloco
**Description:** Moloco is the AI-powered growth engine that unlocks the value of a business's first-party data.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
------------------|---------------------------------------------------|----------------------|--------------------------------------------------------------------------
**Customer Sets** | Sync data from any source to Moloco customer sets | Add | [Customer Sets docs](https://developer.moloco.cloud/docs/getting-started)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Moloco
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Moloco** and click **Continue**. You can then authenticate Hightouch to **Moloco**.
Enter the following fields into Hightouch:
- **Ad Account ID**
- **API key**
## Sync configuration
Once you've set up your Moloco destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Moloco destination you want to sync to.
### Syncing customer sets
Sync data from any source to Moloco customer sets.
#### Creating a new customer set
You can create a new customer set via Hightouch by selecting `Create a new customer set`. If selected, Hightouch will generate a new customer set. The customer set will not be ready for 30 minutes, so the initial sync will wait 30 minutes for the customer set to be ready, and then sync your data over.
#### Record matching
To match rows from your model to customer sets in Moloco, you need to select a model column and corresponding Moloco field. You can match on any of the following Moloco fields:
- **Apple IDFA**
- **Google ADID**
#### Field mapping
Hightouch lets you sync customer set fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default customer set fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Monday
**URL:** https://hightouch.com/docs/destinations/monday
**Description:** Keep your team's projects and tickets up-to-date with updated items for your Monday boards
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
| ---------- | -----------------------------------------------------| ---------------------- | ------------------------------------------------------------------------------------------------------- |
| **Items** | Sync data from any source to a Monday board as items | Update, Insert | [Items](https://developer.monday.com/api-reference/docs/items_page)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Monday
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Monday** and click **Continue**. Authenticate Hightouch to Monday by entering your API token from your Monday account.
If you don't have an API token, follow the [instructions on **Monday** to access API tokens](https://developer.monday.com/api-reference/docs/authentication#accessing-api-tokens) depending on your role. Ensure your API token has write access to the board you are syncing to.
## Sync configuration
Once you've set up your Monday destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Monday destination you want to sync to.
Select which board you'd like to sync to. Each sync sends data to one board.
### Sync mode
Hightouch supports **Update** and **Insert** mode by using Monday's GraphQL API with version 2023-10.
**Update** mode uses the [`change_multiple_column_values`](https://developer.monday.com/api-reference/docs/columns#change-multiple-column-values) mutation.
**Insert** mode uses the [`create_item`](https://developer.monday.com/api-reference/docs/items#create-an-item) mutation.
### Field mapping
Hightouch supports the following column types:
* Checkbox
* Country
* Date
* Dependency
* Email
* Hour
* Link
* Location
* Name
* Numbers
* People
* Phone
* Rating
* Status
* Tags
* Text
* Timeline
* Week
* World Clock
You may use Hightouch's object inline mapping to construct Monday's object values from your source. Below is an example sync configuration:
Monday uses **YYYY-MM-DD** format for dates. For the most success in your sync, ensure that you use that format for your dates. You can also use Hightouch's template option to format your data.
### Record matching
In **Update mode**, Hightouch requires a unique identifier in your model with item IDs from Monday. Rows without item IDs will fail to sync.
In **Insert mode**, the source must contain a column that maps to the item name. Monday rejects any rows without an item name.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## MongoDB
**URL:** https://hightouch.com/docs/destinations/mongodb
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Overview
The MongoDB destination combines the analytical power of your data warehouse with the low-latency performance of a transactional database. It's been battle-tested with syncs up to hundreds of millions of rows.
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|Documents|Sync data to documents in a MongoDB collection |Upsert|
## Connect to MongoDB
Make sure to allowlist Hightouch IP addresses so Hightouch can reach your cluster. Refer to our
[IP address docs](/security/networking#ip-addresses) to find the relevant IP addresses for your Hightouch region. If you are using Atlas, you can add allowlist IP addresses from **Security > Network Access**.
When creating a MongoDB destination, you can either enter the **Host**, **Port**, **Database**, **User**, and **Password**, or you can provide the full URI connection string. For more information, go to the [official connection guide](https://docs.mongodb.com/drivers/node/current/fundamentals/connection/).
### SSH tunneling
Hightouch can connect directly to MongoDB over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your MongoDB instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
### Retrieve MongoDB credentials
Assign the user whose credentials you use the built-in [`readWriteAnyDatabase`](https://www.mongodb.com/docs/atlas/security-add-mongodb-users/#mongodb-atlasrole-Read-and-write-to-any-database) MongoDB role or create a [custom role](https://www.mongodb.com/docs/atlas/security-add-mongodb-roles/) and assign these actions at minimum:
| Action | Details |
| :----------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------- |
| [`insert`](https://www.mongodb.com/docs/manual/reference/privilege-actions/#mongodb-authaction-insert) | Grants permission to perform `insert` and `create` commands|
| [`remove`](https://www.mongodb.com/docs/manual/reference/privilege-actions/#mongodb-authaction-remove) | Grants permission to perform the `delete` command |
| [`update`](https://www.mongodb.com/docs/manual/reference/privilege-actions/#mongodb-authaction-update) | Grants permission to perform the `update` command |
| [`listIndexes`](https://www.mongodb.com/docs/manual/reference/privilege-actions/#mongodb-authaction-listIndexes) | Grants permission to perform the `listIndexes` command |
| [`listCollections`](https://www.mongodb.com/docs/manual/reference/privilege-actions/#mongodb-authaction-listCollections) | Grants permission to perform the `listCollections` command |
To create a user in MongoDB, use [`db.createUser()`](https://www.mongodb.com/docs/manual/reference/method/db.createUser/).
#### Authenticate with connection parameters
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your MongoDB server. For help finding this, go to the [official connection guide](https://www.mongodb.com/docs/drivers/node/current/fundamentals/connection/connect/#std-label-node-connect-to-mongodb).
- **Port**: The port number of your MongoDB server. The default port number is 27017, but yours may be different. You don't need to specify the port if you are using SRV protocol.
- **Database**: The name of the your MongoDB database.
- **Username**: This can be your personal MongoDB login or a dedicated user for Hightouch.
- **Password**: This is the password for the user specified above. If you're using [Atlas](https://www.mongodb.com/atlas/database), the user password might differ from the one you use to login to your MongoDB Atlas instance.
{/* */}
- **Is the host name a seedlist record?**: To use [SRV](https://www.mongodb.com/docs/manual/reference/connection-string/#std-label-connections-dns-seedlist) protocol (Hightouch defaults to [standard](https://www.mongodb.com/docs/manual/reference/connection-string/#std-label-connections-standard-connection-string-format)), you must toggle this setting. If you are using MongoDB Atlas and the connection string begins with protocol `mongodb+srv`, toggle this setting to connect.
{/* */}
#### Authenticate with connection string
The connection string is available in multiple locations. Follow these instructions for one way to find it.
1. From the [MongoDB UI](https://cloud.mongodb.com/v2/) click the **Connect** button next to the name of your cluster:
2. From the options displayed, select **Connect your application**:
Follow the instructions for changing the password and database name in the string. Copy the string by clicking the copy icon:
In Hightouch, in the MongoDB setup configuration, paste your saved connection string into the provided field.
## Sync configuration
### Record matching
Hightouch requires choosing one column to define how to match data in your source to documents in MongoDB. It's usually best to use the [`_id` field](https://www.mongodb.com/docs/manual/core/document/#the-_id-field) in MongoDB. Otherwise, you can also select `_id (Object ID)` if you already have [Object ID](https://www.mongodb.com/docs/manual/reference/bson-types/#objectid) values in your source data.
You can also choose to record match on a custom MongoDB field. To do this, you need to create a [unique index on a single field](https://www.mongodb.com/docs/manual/core/index-unique/#unique-index-on-a-single-field), which is a specific type of [single field index](https://www.mongodb.com/docs/upcoming/core/indexes/index-types/index-single/#single-field-indexes). Before creating this index, make sure that the field doesn't contain duplicate values, as this makes the index build fail with an `Unknown error` message.
If you record match on a custom field, MongoDB automatically generates an Object ID to insert in the `_id` field.
### Field mapping
You can map the columns you want to include when creating new MongoDB documents or updating existing ones by selecting the model column and typing the respective MongoDB field name. Otherwise, you can include every column from your model by checking the **Sync all columns** box.
### Delete behavior
You can choose how to handle documents in MongoDB when the corresponding rows are deleted in your source.
| Behavior | Description |
|-------------------------------|-----------------------------------------------|
| **Do nothing** | Keep the document in MongoDB |
| **Clear Fields** | Clear the mapped fields but keep the document |
| **Delete Destination Record** | Delete the MongoDB document completely |
## Tips and troubleshooting
### Best practices
Create a user in MongoDB specifically for Hightouch and only allow that user to access the collections you want Hightouch to access. Don't use the 'root' user.
### Compatible versions
The minimum supported MongoDB version is `4.0`. Earlier versions aren't officially supported but may work regardless.
### Common errors
{/* */}
#### MongoServerError: bad auth: Authentication failed
{/* */}
This error can occur if the destination wasn't [configured correctly](#authenticate-with-connection-parameters). Make sure to use the correct password for the user specified in the configuration. If the error is still happening, try changing your MongoDB user's password and attempting the setup again.
### Live debugger
### Sync alerts
---
## mParticle
**URL:** https://hightouch.com/docs/destinations/mparticle
**Description:** Sync customer attributes and events from your data warehouse to hundreds on integrations through mParticle
**Section:** Destinations
## Overview
Through mParticle's diverse integration library, Hightouch users can sync enriched customer profiles and events to hundreds of partner integrations. Use your data warehouse as the source of truth and keep your team's tools up to date with the latest data using Hightouch and mParticle.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ----------- | -------------------------------------------------------------- | -------------------- |
| **Users** | Sync data from any source as user attributes | Upsert |
| **Events** | Sync data from any source as commerce events and custom events | Insert |
## Getting started
To give Hightouch access to mParticle, you need to enter:
- API key
- API secret
Follow the instructions in the [following sections](#retrieve-api-key-and-api-secret) to retrieve your API key and API secret.
### Retrieve API key and API secret
1. Navigate to [Inputs](https://app.mparticle.com/setup/inputs/feeds) in your mParticle account.
2. Add a feed under the **Hightouch** tab by clicking the **+** button.
3. Give your configuration a name and select **Web** under **Act as Platform**.
4. Copy your generated keys. Your **Server to Server Key** is your **API key** and your **Server to Server Secret** is your **API secret**.
## Syncing data
### Users
Sync data from any source as user attributes without an event to mParticle.
#### Record matching
You can match rows in your model with users in mParticle with the provided identifiers. Hightouch uses the identifier you select in this section and as the `external Id` for this sync.
#### Additional identifiers
In addition to the column selected for [record matching](#record-matching), you can choose to provide more user identifiers in this section. There is no limit to the number of columns you can select here.
#### Field mapping
You can sync columns from your source to mParticle as user attributes. If the customer attribute doesn't exist, Hightouch creates it.
#### Custom field mapping
You can sync columns from your source to mParticle as custom user attributes. If the customer attribute doesn't exist, Hightouch creates it.
#### Environment
In this section, select the environment you would like to sync your data to. The available choices are:
| Environment | Description |
| --------------- | -------------------------------------------------------------------------------------------------------------- |
| **Production** | All data synced is treated as production data and is forwarded to integrations for your application |
| **Development** | All data synced is treated as development data and is siloed from your production data |
mParticle recommends using **Development** mode first to ensure the data meets your expectations before moving to **Production**.
#### Delete behavior
You have the following options for how Hightouch should handle rows leaving your query results:
| Behavior | Description |
| ---------------- | ---------------------------------------------------- |
| **Do nothing** | Keep the customer in mParticle |
| **Clear Fields** | Clear the mapped fields but keep the customer record |
### Events
You can sync the following event types:
- **Custom Event**
- **Commerce Event**
#### Events and historical events
Given a valid `Date`, `DateTime`, or `timestamp` column as a `timestamp` for events, Hightouch separates events based on the age of the data.
- Hightouch sends events that are older than **28** days to the `/v2/bulkevents/historical` endpoint.
- Hightouch sends events that are newer than **28** days to the `/v2/bulkevents` endpoint.
If you choose not to provide a timestamp for your events, Hightouch uses the time the event arrives at the server. That means all events are treated as newer than 28 days.
### Custom event
Sync data from any source as a custom event.
#### Event name
Provide an event name. This is how you can reference the event in mParticle campaigns or segments.
#### Event timestamp
Provide a timestamp of when the event occurred. If this field is empty, Hightouch uses the time the event arrives at the server.
Any `DateTime` value provided from your source is converted to Unix timestamp as expected by mParticle.
#### Custom event type
Select the type of custom event you would like to sync out of the provided selections.
#### User identifiers
In this section, you can provide user identifiers for each event that occurred.
#### Event attributes field mapping
You can sync columns from your source to mParticle as event attributes. If the event attribute doesn't exist, Hightouch creates it.
#### Event attributes custom field mapping
You can sync columns from your source to mParticle as custom event attributes. If the custom event attribute doesn't exist, Hightouch creates it.
#### User attributes field mapping
You can sync columns from your source to mParticle as user attributes. If the customer attribute doesn't exist, Hightouch creates it.
#### User attributes custom field mapping
You can sync columns from your source to mParticle as custom user attributes. If the customer attribute doesn't exist, Hightouch creates it.
#### Environment
In this section, select the environment you would like to sync your data to. The available choices are:
| Environment | Description |
| --------------- | -------------------------------------------------------------------------------------------------------------- |
| **Production** | All data synced is treated as production data and is forwarded to integrations for your application |
| **Development** | All data synced is treated as development data and is siloed from your production data |
mParticle recommends using **Development** mode first to ensure the data meets your expectations before moving to **Production**.
### Commerce event
Sync data from any source as a commerce event.
#### Event timestamp
Provide a timestamp of when the event occurred. If this field is empty, Hightouch uses the time the event arrives at the server.
Any `DateTime` value provided from your source is converted to Unix timestamp as expected by mParticle.
#### User identifiers
In this section, you can provide user identifiers for each event that occurred.
#### Event attributes field mapping
You can sync columns from your source to mParticle as event attributes. If the event attribute doesn't exist, Hightouch creates it.
#### Event attributes custom field mapping
You can sync columns from your source to mParticle as custom event attributes. If the custom event attribute doesn't exist, Hightouch creates it.
#### User attributes field mapping
You can sync columns from your source to mParticle as user attributes. If the customer attribute doesn't exist, Hightouch creates it.
#### User attributes custom field mapping
You can sync columns from your source to mParticle as custom user attributes. If the customer attribute doesn't exist, Hightouch creates it.
#### Environment
In this section, select the environment you would like to sync your data to. The available choices are:
| Environment | Description |
| --------------- | -------------------------------------------------------------------------------------------------------------- |
| **Production** | All data synced is treated as production data and is forwarded to integrations for your application |
| **Development** | All data synced is treated as development data and is siloed from your production data |
mParticle recommends using **Development** mode first to ensure the data meets your expectations before moving to **Production**.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## MySQL
**URL:** https://hightouch.com/docs/destinations/mysql
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Overview
This destination combines the analytical power of your data warehouse with the low-latency performance of a transactional database.
## Supported syncing
|Sync Type|Description |Supported Sync Modes |
|-----------|----------------------------------------------|---------------------|
| **Table** |Sync data from any source to a MySQL table | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to MySQL
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **MySQL** and click **Continue**.
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your MySQL.
- **Port**: The port number of your MySQL. The default is 3306, but yours may be different.
- **Database**: The name of the database in your MySQL.
- **User**: This can be your personal MySQL login or a dedicated user for Hightouch.
- **Password**: The password for the user specified above.
You can optionally create or select a [previously configured tunnel](/security/ssh-tunneling) or upload a `.pem` CA certificate.
### SSH tunneling
Hightouch can connect directly to MySQL over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your MySQL instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
## Sync configuration
Once you've set up your MySQL destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the MySQL destination you want to sync to.
### Select table
Select the existing table in MySQL you want to sync to.
### Record matching
You can match rows from your model to rows in MySQL on any column as long as it is a [primary key or unique key](https://dev.mysql.com/doc/refman/8.0/en/partitioning-limitations-partitioning-keys-unique-keys.html).
### Field mapping
Select which model columns you want to sync to your MySQL table columns. Hightouch automatically pulls the columns from your table to make them available for you to map.
When syncing to [binary](https://dev.mysql.com/doc/refman/8.0/en/binary-varbinary.html) destination fields, Hightouch assumes the model column values to be [base64](https://en.wikipedia.org/wiki/Base64) strings. If they aren't, you can use [template mapping](/syncs/mapping-data#template-mapping) to [cast](/models/data-types-casting#casting) them if needed.
### Upsert query optimization
You can choose to use the [`LOAD DATA` statement](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_local_infile) in the upsert query during your sync. This option loads data from a text file into your database, which can be up to twenty times faster than the default `INSERT INTO` query Hightouch uses.
If you enable the `LOAD DATA` option, Hightouch replaces databases rows with the mapped model row, if it exists. This means that if any MySQL fields aren't mapped in Hightouch they receive `NULL` values.
For Hightouch to successfully sync with the `LOAD DATA` option, you need to enable the [`local_infile`](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_local_infile) system variable.
Review the [security considerations](https://dev.mysql.com/doc/refman/8.0/en/load-data-local-security.html) before enabling `local_infile`.
### Delete behavior
When rows no longer shows up from your model's query results, you can configure the sync to **Do nothing** or **Delete** those rows from the MySQL.
| Behavior | Description |
| -------------- | ----------------------------------------------- |
| **Do nothing** | Keep the synced rows in the table in MySQL |
| **Delete** | Delete the synced rows from the table in MySQL|
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## NetSuite (SOAP)
**URL:** https://hightouch.com/docs/destinations/netsuite
**Description:** Sync business-critical data from your data warehouse to Netsuite with Hightouch
**Section:** Destinations
Check out our [playbook](https://hightouch.com/playbooks/automating-usage-based-billing) and [blog post](https://hightouch.com/blog/automated-billing) to learn how to automate usage-based billing to streamline your finance operations.
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ------------ | ---------------------------------------------------------------------------------------------------------------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| **Entities** | Sync data from any source to entities in NetSuite, for example, Customers, Invoices, Sales Orders, and Journal Entries | Upsert, Update, Insert | [NetSuite schema browser](https://system.netsuite.com/help/helpcenter/en_US/srbrowser/Browser2022_2/schema/record/account.html)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## REST vs SOAP
The Hightouch NetSuite REST destination offers a few advantages over the SOAP destination:
- The NetSuite REST API uses JSON for web service requests and responses, which is easier to read and debug in the [live debugger](/syncs/debugger) compared to XML.
- Hightouch prioritizes supporting the latest REST API features and versions.
- The [NetSuite REST](/destinations/netsuite-rest#field-mapping) destination supports the [inline mapper](/syncs/mapping-data#inline-mapping) to build objects and arrays
Unless you have a particular entity that's only supported by the SOAP API, we recommend using the REST NetSuite destination.
## Setup in NetSuite
Before you can use Hightouch to sync data to NetSuite, you need to create and and configure a NetSuite integration. Then, to grant access to Hightouch to synchronize data between your source and NetSuite, you need to create a role, a user, and an authentication token for the integration.
Depending on your NetSuite account's setup, you may need an **admin** role to complete setup in NetSuite.
Netsuite permission changes to a role or user don't occur immediately. These changes can take a few minutes or hours to go into effect.
### Create an integration
1. Go to **Setup > Integration > Manage Integrations** or use the global search to find and go to the **Manage Integrations** page.
This page shows a list of enabled integrations for your account.
2. Click **New** to create a new integration with the following properties:
- A **Name** that can easily be associated with Hightouch, for example, "Hightouch Integration"
- Set **State** to **Enabled**
- Check **Token-Based Authentication**
- Uncheck **TBA:Authorization flow**
- Uncheck **Authorization Code Grant**
- Leave other authentication options unchanged.
3. Scroll to the bottom of the page and click **Save**.
4. Scroll to **Client credentials** and Copy and store the **Consumer Key / Client ID** and **Consumer Secret / Client secret** in a secure location. You'll need them to connect to Hightouch.
### Create a role
1. Go to **Setup > Users/Roles > Manage Roles** and click **New** or use the global search to find and go to the **New Role** page.
2. Create a role with the following properites:
- A descriptive **Name**, for example, "Hightouch Role."
- Check **Do not restrict employee fields**
- For multi-subsidiary NetSuite users, check **All** under **Accessible subsidiaries**
- Under **Authentication**, check **Web Services only Role**.
3. Add the following permissions under **Permissions > Setup**
- Access Token Management: Full
- Custom Body Fields: Full
- Custom Column Fields: Full
- Custom Entity Fields: Full
- Custom Fields: Full
- Custom Item Fields: Full
- SOAP Web Services: Full
- User Access Tokens: Full
4. Under **Permissions > Lists** and **Permissions > Transactions**, provide **Full** permissions for the NetSuite entities you plan on syncing to. For example, if you want to update sales orders, you need to go to the **Transactions** tab and select **Sales Order** from the list. If you want to update accounts, you need to go to **Lists** and select **Accounts** from the list.
### Create a user
Now you can create a new user that uses this role.
1. Go to the **Manage Users** page. Create a new user or select the user you will be using Hightouch with.
2. For new users: fill the **Employee** form with all required fields.
3. On the **Access** tab, check **Manually assign or change password** and assign a secure password.
4. Under **Roles** Assign the role, for example, "Hightouch Role," you previously created.
5. Click **Save**.
### Create an access token
1. Go to the **Setup > Users/Roles > Access Tokens** page, or use the global search to find it.
2. Click **New Access Token**.
3. Select the integration, user, and role you previously configured.
4. When you click **Save**, NetSuite displays a **Token ID** and a **Token Secret**. Copy and store them in a secure location. You'll need them to connect to Hightouch.
### Enable SOAP web services
Finally, go to **Setup > Company > Enable Features > SuiteCloud > SuiteTalk (Web Services)** and enable **SOAP Web Services**. Once you check this box, NetSuite prompts you to agree to terms of service.
On the same page, under **Manage Authentication**, enable **Token Based Authentication**. It should look similar to the screenshot below.
## Connect to NetSuite
You're now ready to create a NetSuite destination in Hightouch. Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **NetSuite** and click **Continue**.
You can then authenticate Hightouch to NetSuite by entering the following required fields into Hightouch:
- **Account ID**: You can find this on your **Company information** page in NetSuite.
- **Consumer Key**: From your new integration
- **Consumer Secret**: From your new integration
- **Token Key**: From your new access token
- **Token Secret**: From your new access token
If your account is a Sandbox account, you need to include the suffix `_SB` as part of the **Account ID**:
- `SB` should be in uppercase.
- The Sandbox suffix should be included with an underscore (`_`).
For example, if the Account ID in your sandbox is `1234567-sb1`, then you need to set your Account ID as `1234567_SB1` in your sync configuration.
## Sync configuration
Once you've set up your NetSuite destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the NetSuite destination you want to sync to.
### Syncing entities
The Hightouch NetSuite destination allows you to sync data to supported NetSuite entities, for example, customers, invoices, vendors, etc. Hightouch also supports syncing custom records types—all custom records are part of the
**CustomRecords** namespace.
The specific entities you can sync to depend on your NetSuite implementation. You can view your available entities in the dropdown. Because of the large catalog size, it's best to search for the entities you're interested in.
#### Related records
If you like, you can choose to initialize the entity with a related record. For example, an invoice can be created from a sales order. If you enable this, you then need to select the base record.
#### Record matching
To match source rows to entities in Netsuite, you need to select a source column and corresponding Netsuite field.
Depending on the sync mode you select, you can match on the following Netsuite fields:
- **External ID**
- **Internal ID**
**Update** mode supports both, whereas **Upsert** mode only supports **External ID**.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, NetSuite automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
You can sync columns from your model to NetSuite default and custom fields.
Mappings mirror the NetSuite SOAP schema.
That means some of your model columns might need to be JSON objects.
NetSuite generally expects JSON objects when an entity references another entity. For example, in an [invoice](https://system.netsuite.com/help/helpcenter/en_US/srbrowser/Browser2022_2/schema/record/invoice.html), the `entity` field can reference a customer, partner, vendor, employee, contact, etc. by their ID. If you select a numerical model column with external or internal IDs, Hightouch automatically converts this to the JSON object NetSuite expects.
For other JSON objects, such as nested objects or arrays, you need to ensure your model columns match NetSuite's schema expectations.
The [NetSuite REST](/destinations/netsuite-rest#field-mapping) destination supports the [inline mapper](/syncs/mapping-data#inline-mapping), which can help you build objects and arrays.
NetSuite arrays are often wrapped in a JSON object.
For example, an invoice supports an `item` field that expects an array of items:
```json
{
"items": [
{ "item": { "id": "8" }, "line": 1, "amount": 1, "price": { "id": "-1" } }
]
}
```
To see the schema that NetSuite expects for different entities's fields, go to the [SOAP browser](https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/section_N3639052.html).
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|--------------------------------------------------------------
**Do nothing** | Keep the record in NetSuite with all its synced fields
**Delete** | Delete the synced record from your NetSuite instance
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## NetSuite (REST)
**URL:** https://hightouch.com/docs/destinations/netsuite-rest
**Description:** Sync business-critical data from your data warehouse to Netsuite with Hightouch
**Section:** Destinations
Check out our
[playbook](https://hightouch.com/playbooks/automating-usage-based-billing) and
[blog post](https://hightouch.com/blog/automated-billing) to learn how to
automate usage-based billing to streamline your finance operations.
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| -------------- | ---------------------------------------------------------------------------------------------------------------------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| **Entities** | Sync data from any source to entities in NetSuite, for example, Customers, Invoices, Sales Orders, and Journal Entries | Upsert, Update, Insert | [NetSuite schema browser](https://system.netsuite.com/help/helpcenter/en_US/srbrowser/Browser2022_2/schema/record/account.html) |
| **Transforms** | Transform records from one type into another, using data from existing records. | Insert | [NetSuite record transform](https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/section_4267258715.html) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## REST vs SOAP
The Hightouch NetSuite REST destination offers a few advantages over the SOAP destination:
- The NetSuite REST API uses JSON for web service requests and responses, which is easier to read and debug in the [live debugger](/syncs/debugger) compared to XML.
- Hightouch prioritizes supporting the latest REST API features and versions.
- The [NetSuite REST](#field-mapping) destination supports the [inline mapper](/syncs/mapping-data#inline-mapping) to build objects and arrays
Unless you have a particular entity that's only supported by the SOAP API, we recommend using the REST NetSuite destination.
## Setup in NetSuite
Before you can use Hightouch to sync data to NetSuite, you need to create and and configure a NetSuite integration.
Then, to grant access to Hightouch to synchronize data between your source and NetSuite, you need to create a role, a user, and an authentication token for the integration.
These steps are the same whether you choose to [connect Hightouch to NetSuite](#connect-to-netsuite) with basic authentication or OAuth.
The only difference is in the final step of [enabling web services](#enable-rest-web-services).
Depending on your NetSuite account's setup, you may need an **admin** role to
complete setup in NetSuite. Netsuite permission changes to a role or user
don't occur immediately. These changes can take a few minutes or hours to go
into effect.
### Create an integration
1. Go to **Setup > Integration > Manage Integrations** or use the global search to find and go to the **Manage Integrations** page.
This page shows a list of enabled integrations for your account.
2. Click **New** to create a new integration with the following properties:
- A **Name** that can easily be associated with Hightouch, for example, "Hightouch Integration"
- Set **State** to **Enabled**
- Check **Token-Based Authentication**
- Uncheck **TBA:Authorization flow**
- Uncheck **Authorization Code Grant**
- Leave other authentication options unchanged.
3. Scroll to the bottom of the page and click **Save**.
4. Scroll to **Client credentials** and Copy and store the **Consumer Key / Client ID** and **Consumer Secret / Client secret** in a secure location. You'll need them to connect to Hightouch.
### Create a role
1. Go to **Setup > Users/Roles > Manage Roles** and click **New** or use the global search to find and go to the **New Role** page.
2. Create a role with the following properites:
- A descriptive **Name**, for example, "Hightouch Role."
- Check **Do not restrict employee fields**
- For multi-subsidiary NetSuite users, check **All** under **Accessible subsidiaries**
- Under **Authentication**, check **Web Services only Role**.
3. Add the following permissions under **Permissions > Setup**
- Access Token Management: Full
- Custom Body Fields: Full
- Custom Column Fields: Full
- Custom Entity Fields: Full
- Custom Fields: Full
- Custom Item Fields: Full
- REST Web Services: Full
- User Access Tokens: Full
4. Under **Permissions > Lists** and **Permissions > Transactions**, provide **Full** permissions for the NetSuite entities you plan on syncing to. For example, if you want to update sales orders, you need to go to the **Transactions** tab and select **Sales Order** from the list. If you want to update accounts, you need to go to **Lists** and select **Accounts** from the list.
### Create a user
Now you can create a new user that uses this role.
1. Go to the **Manage Users** page. Create a new user or select the user you will be using Hightouch with.
2. For new users: fill the **Employee** form with all required fields.
3. On the **Access** tab, check **Manually assign or change password** and assign a secure password.
4. Under **Roles** Assign the role, for example, "Hightouch Role," you previously created.
5. Click **Save**.
### Create an access token
1. Go to the **Setup > Users/Roles > Access Tokens** page, or use the global search to find it.
2. Click **New Access Token**.
3. Select the integration, user, and role you previously configured.
4. When you click **Save**, NetSuite displays a **Token ID** and a **Token Secret**. Copy and store them in a secure location. You'll need them to connect to Hightouch.
### Enable REST web services
Finally, go to **Setup > Company > Enable Features > SuiteTalk (Web Services)** and enable **REST Web Services**. Once you check this box, NetSuite prompts you to agree to terms of service.
On the same page, under **Manage Authentication**, enable either :
- **Token Based Authentication**, if you plan on [connecting Hightouch](#connect-to-netsuite) to NetSuite with basic authentication
- **OAuth 2.0**, if you plan on [connecting Hightouch](#connect-to-netsuite) to NetSuite with OAuth
If you're not sure which you will use, you can check both of them.
## Connect to NetSuite
You're now ready to create a NetSuite destination in Hightouch. Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **NetSuite** and click **Continue**. You can then authenticate Hightouch to NetSuite via **Basic Auth** or **OAuth**.
OAuth only requires you to login to your NetSuite account rather than enter credentials.
### Authenticate with basic auth
Basic auth requires you to enter the following required fields into Hightouch:
- **Account ID**: You can find this on your **Company information** page in NetSuite.
- **Consumer Key**: From your new integration
- **Consumer Secret**: From your new integration
- **Token Key**: From your new access token
- **Token Secret**: From your new access token
If your account is a Sandbox account, you need to include the suffix `_SB` as part of the **Account ID**:
- `SB` should be in uppercase.
- The Sandbox suffix should be included with an underscore (`_`).
For example, if the Account ID in your sandbox is `1234567-sb1`, then you need to set your Account ID as `1234567_SB1` in your sync configuration.
### Authenticate with OAuth
For the **Authentication method**, select **OAuth** and log into your NetSuite account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your NetSuite destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the NetSuite destination you want to sync to.
### Syncing entities
The Hightouch NetSuite destination allows you to sync data to supported NetSuite entities, for example, customers, invoices, vendors, etc. Hightouch also supports syncing custom records types.
The specific entities you can sync to depend on your NetSuite implementation. You can view your available entities in the dropdown. Because of the large catalog size, it's best to search for the entities you're interested in.
#### Record matching
To match source rows to entities in Netsuite, you need to select a source column and corresponding Netsuite field.
Depending on the sync mode you select, you can match on the following Netsuite fields:
- **External ID**
- **Internal ID**
**Update** mode supports both, whereas **Upsert** mode only supports **External ID**.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, NetSuite automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can sync columns from your model to NetSuite default and custom fields.
Mappings mirror the NetSuite REST schema. That means some of your model columns might need to be JSON objects.
NetSuite generally expects JSON objects when an entity references another entity. For example, in an [invoice](https://system.netsuite.com/help/helpcenter/en_US/srbrowser/Browser2022_2/schema/record/invoice.html), the `entity` field can reference a customer, partner, vendor, employee, contact, etc. by their ID. To make that reference, you need to map a JSON object like this to the `entity` field:
```json
{ "id": 1 }
```
You can use also external IDs for references, for example:
```json
{ "externalId": 1 }
```
The [inline mapper](/syncs/mapping-data#inline-mapping) is purpose-built to help you create [JSON objects](/syncs/mapping-data#create-an-object) like this.
The inline mapper can also help you create [arrays](/syncs/mapping-data#create-an-array).
NetSuite arrays are often wrapped in a JSON object.
For example, an invoice supports an `item` field that expects an array of items:
```json
{
"items": [
{ "item": { "id": "8" }, "line": 1, "amount": 1, "price": { "id": "-1" } }
]
}
```
Refer to the [inline mapping docs](/syncs/mapping-data#create-an-array) to learn how to create an array like this and what the [required data format](/syncs/mapping-data#required-data-format) for your model is. To see the schema that NetSuite expects for different entities, go to the [REST browser](https://system.netsuite.com/help/helpcenter/en_US/APIs/REST_API_Browser/record/v1/2022.1/index.html).
#### Overwrite sublists
Updating sublist properties can result in appending the mapped items to the record's existing sublist.
This may result in unwanted duplicates. You can explicitly specify in Hightouch to overwrite your
mapped sublist items in NetSuite. Overwriting the sublist removes all existing items and updates
it with incoming data.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ------------------------------------------------------ |
| **Do nothing** | Keep the record in NetSuite with all its synced fields |
| **Delete** | Delete the synced record from your NetSuite instance |
### Transforming records
The Hightouch NetSuite destination allows you to transform records from one type to another, for example, transforming purchase orders into vendor bills.
#### Record matching
To match source rows to entities in Netsuite, you need to select a source column and corresponding Netsuite field.
The ID must be the ID of the original record that will be transformed. You may use the following IDs to match on:
- **External ID**
- **Internal ID**
#### Field mapping
You can sync columns from your model to NetSuite default and custom fields of the target record type.
Mappings mirror the NetSuite REST schema. That means some of your model columns might need to be JSON objects.
NetSuite generally expects JSON objects when an entity references another entity. For example, in an [invoice](https://system.netsuite.com/help/helpcenter/en_US/srbrowser/Browser2022_2/schema/record/invoice.html), the `entity` field can reference a customer, partner, vendor, employee, contact, etc. by their ID. To make that reference, you need to map a JSON object like this to the `entity` field:
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Nextdoor
**URL:** https://hightouch.com/docs/destinations/nextdoor
**Description:** Elevate your local marketing efforts with Nextdoor Ads and drive business growth by reaching nearby customers in their communities
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
-----------|----------------------------------------------|---------------------
**Events** | Sync data from any source to Nextdoor events | Insert
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Nextdoor
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Nextdoor** and click **Continue**. You can then authenticate Hightouch to **Nextdoor**.
Enter the following required fields into Hightouch:
- **API token**: Enter a valid API token.
## Sync configuration
Once you've set up your Nextdoor destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Nextdoor destination you want to sync to.
### Syncing events
Nextdoor's Conversions API requires the following event parameters:
- `event_name`
- `event_time`
- `action_source`
- `client_id`
- at least one `customer` parameter
#### Event name
Providing an event name is required to send an event to the Conversions API. You can either provide a static value or select to **use a column** from your model. Hightouch syncs the static or column value as the `event_name` parameter the Conversions API requires.
Supported values include:
- `conversion`
- `lead`
- `purchase`
- `sign_up`
- `custom_conversion_1`
- `custom_conversion_2`
- `custom_conversion_3`
- `custom_conversion_4`
- `custom_conversion_5`
- `custom_conversion_6`
- `custom_conversion_7`
- `custom_conversion_8`
- `custom_conversion_9`
- `custom_conversion_10`
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server. Hightouch syncs the timestamp as the `event_time` parameter the Conversions API requires.
If you select a column, it should be in [ISO 8601](https://www.iso.org/iso-8601-date-and-time-format.html) format. Hightouch automatically converts the data to the format Nextdoor expects.
#### Action source
The `action_source` is a required parameter and indicates the channel through which conversion happened, for example, over email, on your website, etc. Knowing where your events took place helps ensure your ads go to the right people.
#### Client ID
The `client_id` is a required parameter, also known as the NAM Advertiser ID. To get your `client_id`, reach out to your Nextdoor account manager.
#### Test event codes
Test event codes are special codes you can include in your event payload to simulate conversion events. This allows you to test your API implementation without actually recording real conversion data.
You can optionally enter a test event code as part of your sync configuration.
Make sure to remove the test event code if you're running a production sync.
#### Hashing
By default, Hightouch automatically hashes the following customer fields:
- **email**
- **phone_number**
- **first_name**
- **last_name**
- **date_of_birth**
- **street_address**
- **city**
- **state**
- **zip_code**
- **country**
If you want to turn hashing off, select **No** under **Would you like Hightouch to automatically detect if your PII data requires hashing?**.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Notion
**URL:** https://hightouch.com/docs/destinations/notion
**Description:** Power up your Notion databases with data from your data warehouse
**Section:** Destinations
|Sync Type |Description |Supported Sync Modes |
|--------------|----------------------------------------------------------------------------------------------|---------------------|
|Database rows | Upsert or update rows in a Notion [database](https://www.notion.so/help/intro-to-databases) |Upsert, Update |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Notion
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Notion** and click **Continue**. You can then authenticate Hightouch to Notion by logging into Notion.
Next, select which Notion pages you want to give Hightouch access to.
Be sure to give access to pages that include databases you want to sync to.
If the page containing the database within Notion has restricted access, you won't be able to configure a sync to it.
Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Notion destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Notion destination you want to sync to.
### Database selection
After selecting your desired [sync mode](#supported-syncing), select the Notion database you want to sync data to.
### Record matching
To match rows from your model to database rows in Notion, you need to select a model column and corresponding Notion field.
It's best to select a unique identifier for record matching, otherwise the intended records may not be updated properly.
Refer to the [record matching docs](/syncs/record-matching) for more information.
The Notion field you select for record matching [must be a **text** field](#column-not-appearing-for-record-matching).
### Field mapping
You can sync model columns to Notion's database properties.
Hightouch automatically detects existing database properties from your Notion databases.
#### Supported Notion properties
Hightouch supports the following Notion database properties when field mapping.
- `Checkbox`
- `Date`: Hightouch supports a single date, and not a date range
- `Email`
- `Multi Select`
- `Number`: If a number field is passed an invalid number, Hightouch attempts to convert it to a number. If the conversion result is invalid, the sync errors.
- `Phone`
- `Rich Text`
- `Select`
- `Title`
- `URL`
To learn more about Notion's available properties, please refer to [Notion's docs](https://developers.notion.com/reference/property-object).
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available on **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
| Behavior | Description |
| -------------- | ------------------------------------------------------------- |
| **Do nothing** | Keep the row in the database with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the row in the database |
| **Delete** | Delete the synced row in the database |
## Tips and troubleshooting
### Common errors
#### Database not appearing in dropdown
If the page containing the database within Notion has restricted access, the database won't appear in the dropdown menu of the sync configuration, even if Hightouch is explicitly granted access during authentication.
You may also receive a `404 - Could not find database with ID: {...}` error in the [record matching](/syncs/record-matching) section of your sync configuration, or when attempting to sync to Notion.
To fix this, go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and select your Notion destination.
Click **Log in to Notion** and add access to any required pages.
Then, return to your sync configuration or create a new one from the [**Syncs** overview page](https://app.hightouch.com/syncs).
You may need to click the refresh button to see the new database.
#### Column not appearing for record matching
Hightouch only supports record matching on **text** fields in Notion.
Convert the field type to text in Notion, or create a new text field.
#### 400 - body failed validation \{...\} should be defined, instead was undefined
This error occurs if you try to sync `null` values to [Select](https://developers.notion.com/reference/property-object#select) or [Multi-select](https://developers.notion.com/reference/property-object#multi-select) fields in Notion.
To resolve this, check the [**Don't sync null values** option](/syncs/mapping-data#dont-sync-null-values) in the template mapper or [remove rows containing `null` values](/models/sql-editor#exclude-rows-with-null-values) from your model's query results.
You need to do this option for all field mappings to [Select](https://developers.notion.com/reference/property-object#select) or [Multi-select](https://developers.notion.com/reference/property-object#multi-select) fields in Notion, if you believe the mapped model columns might contain `null` values.
#### 400 - body failed validation \{...\} should be a string, instead was \{...\}
This error occurs if you try to sync non-text values to a [Text](https://developers.notion.com/reference/property-object#rich-text) field in Notion.
To resolve this, [cast](/models/data-types-casting#casting-from-your-model-configuration) the relevant column or columns from your model to a **String** type.
### Live debugger
### Sync alerts
---
## Ometria
**URL:** https://hightouch.com/docs/destinations/ometria
**Description:** Ometria is a customer data and marketing platform built for retail brands that want to turn fragmented data into revenue. It unifies customer data in real time, uses AI to predict behavior, and lets you trigger highly personalized campaigns across email, SMS, and onsite—so you can send the right message at the right time without duct-taping multiple tools together.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|--------------|--------------------------------------------------|----------------------|--------------------------------------------------------------------------------------------------------|
| **Contacts** | Sync data from any source to contacts in Ometria | Upsert | [Contacts docs](https://docs.ometria.com/docs/contacts-endpoint) |
| **Orders** | Sync data from any source to orders in Ometria | Upsert | [Orders docs](https://docs.ometria.com/docs/orders-endpoint) |
| **Products** | Sync data from any source to products in Ometria | Upsert | [Products docs](https://docs.ometria.com/docs/products-product-listings-and-product-variants#products) |
| **Events** | Sync data from any source to events in Ometria | Insert | [Events docs](https://support.ometria.com/hc/en-gb/articles/360011171077-Setting-up-custom-events) |
All sync types use the [push endpoint](https://docs.ometria.com/reference/post_push) to sync data to Ometria.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Ometria
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Ometria** and click **Continue**. You can then authenticate Hightouch to **Ometria**.
Enter the following fields into Hightouch:
- **API key:** To create a new API key, refer to Ometria's [API keys docs](https://support.ometria.com/hc/en-gb/articles/9882706389917-Creating-a-new-API-key).
## Sync configuration
Once you've set up your Ometria destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Ometria destination you want to sync to.
### Syncing contacts, orders, and products
Sync data from any source to contacts, orders, and products in Ometria.
#### Record matching
To match rows from your model to objects in Ometria, you need to select the model column that contains values that match the **ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom contact fields.
For orders, Ometria requires the following fields, so you must map them to complete your configuration:
- **Timestamp**
- **Grand total**
- **Subtotal**
- **Currency**
For products, Ometria requires the following fields, so you must map them to complete your configuration:
- **Title**
- **Price**
Ensure your model's columns have the same data types as the fields you want to sync to.
### Syncing events
Sync data from any source to events in Ometria.
#### Record matching
You can match rows from your model to events in Ometria on any column in your model and any field in Ometria. Ensure the data types of the model column and Ometria field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Microsoft OneDrive
**URL:** https://hightouch.com/docs/destinations/onedrive
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
| ------------ | ----------------------------------------------------------| -------------------- |
| Any data set | Sync data from a source to One Drive as JSON or CSV files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Diff mode creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Microsoft OneDrive
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Microsoft OneDrive** and click **Continue**. You can then connect to Microsoft OneDrive via OAuth.
After authorizing, you will have successfully connected Hightouch to Microsoft OneDrive.
## Sync configuration
Once you've set up your OneDrive destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the OneDrive destination you want to sync to.
### Select file format
Hightouch supports syncing JSON and CSV files to OneDrive.
### Write to an Owned folder or Shared folder
You can sync to a folder that is owned by or shared to the account connected to Hightouch.
Select either **Owned folders** or **Shared folders** to see available folder options.
### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it. To view different versions of the same file, you can right-click the file in Microsoft OneDrive and then click **Version history**.
If you are using an audience and would like to include the audience name, you will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter '-86400' (`24 hours * 60 minutes * 60 seconds`).
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `company`, `email`, `name`, and `primaryKey` fields. Hightouch exports these fields to new fields in the file and ignores all other columns from your results.
### CSV options
If you're syncing to a CSV file, you have additional configuration options:
- **Delimiter**: Your options are comma (`,`), semicolon (`;`), pipe (`|`), tilde (`~`), and tab
- (Optional) Whether to **include a CSV header row** in the exported files
- (Optional) Whether to **include a [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark)** in the exported files, the BOM is ``
### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
## Tips and troubleshooting
### Need admin approval
If you see a **Need admin approval** page appear when [accessing Microsoft's OAuth login flow](#connect-to-microsoft-onedrive), your Azure Active Directory user account might not have the necessary permissions to connect applications to your organization.
Ask one of your Azure organization's admins to edit your user account permissions.
You can read more about [Azure user permissions](https://learn.microsoft.com/en-us/azure/active-directory/develop/active-directory-how-applications-are-added#who-has-permission-to-add-applications-to-my-azure-ad-instance) and [configuring user settings](https://learn.microsoft.com/en-us/azure/active-directory/manage-apps/configure-user-consent?pivots=portal#configure-user-consent-settings) in Azure's documentation.
### Common errors
### Sync alerts
---
## OneSignal
**URL:** https://hightouch.com/docs/destinations/onesignal
**Description:** Deliver mobile and web push notifications using OneSignal with fresh data from your data warehouse
**Section:** Destinations
## Overview
With the OneSignal destination, Hightouch can:
- Update tags on existing devices via the external unique ID
## Setup
To set up the OneSignal destination, navigate to Destinations and Add New. We can find the keys needed to access OneSignal at OneSignal > Settings > Keys & IDs.
## Modes
### Updating tags via external unique ID
The destination is configured by default to update tags via external unique ID. In this mode, tags are added or updated on existing devices using the external unique ID.
This mode is good for the following scenarios:
- Adding LTV based on external unique ID to a device
- Assigning segment data to a device
| Name | Description |
| :----------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------- |
| Hightouch Column Containing OneSignal External User ID | The column containing the external user ID of the OneSignal device, which is contained in the OneSignal field `external_user_id` |
| Hightouch Column to OneSignal Tag Mappings | A list of columns that will be pushed to tags in OneSignal. Each column and it's value for each row will be a key:value pair in the device's tags. |
---
## OpenSearch
**URL:** https://hightouch.com/docs/destinations/opensearch
**Description:** OpenSearch is a distributed, free and open search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured.
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes|
|--|--|--|
|**Index documents**|Sync data from any source to an OpenSearch index or index alias|Upsert, Update|
## Connect to OpenSearch
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **OpenSearch** and click **Continue**. You can then authenticate Hightouch to OpenSearch by entering your:
- **Host**: The hostname or IP address of your OpenSearch server. Don't include `https://` as part of the hostname.
- **Port**: The port number of your OpenSearch server's HTTP API. The default is 3306, but yours may be different.
- **Username**: This can be your personal OpenSearch login or a dedicated user for Hightouch.
- **Password**: The password for the user specified above.
Once you've entered these fields, click **Continue**. Give your destination a descriptive name and **Finish** the destination connection setup.
## Sync configuration
Once you've set up your OpenSearch destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the OpenSearch destination you want to sync to.
### Syncing to an index
An OpenSearch [index](https://opensearch.org/docs/latest/im-plugin/index/) is the place where the data used by a search engine is stored.
You can use Hightouch to update individual [documents](https://opensearch.org/docs/latest/api-reference/document-apis/index-document/) in an index.
You can also sync your document data via [index aliases](https://opensearch.org/docs/latest/im-plugin/index-alias/).
Index aliases will only be displayed if the [`is_write_index` option](https://opensearch.org/docs/latest/im-plugin/index-alias/#index-alias-options) is configured.
#### Record matching
To match rows from your model to documents in OpenSearch, you need to select a model column and corresponding OpenSearch field. You can match on any OpenSearch field.
#### Field mapping
You can sync data to both existing fields and fields that don't yet exist in OpenSearch. If you send data to a field that doesn't exist yet, Hightouch adds the field and automatically detects its type.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ------------------------------------------------------------------ |
| **Do nothing** | Keep the document in OpenSearch |
| **Clear** | Keep the document in OpenSearch, but set mapped fields to empty |
| **Delete** | Remove the document from OpenSearch |
#### Advanced configuration
You can configure how many rows you'd like to send per batch and how many batches you'd like to send in parallel.
If your sync is rejecting all or most rows in your model with vague error messages, try decreasing the batch size.
## Tips and troubleshooting
### Common errors
#### Failed to connect to your OpenSearch client. Check your credentials.
This error can occur if your credentials are incorrect.
Ensure your hostname doesn't include `https://` and that your port is correct.
### Live debugger
### Sync alerts
---
## Optimizely
**URL:** https://hightouch.com/docs/destinations/optimizely
**Description:** Power digital experiences your customers will love with rich customer data
**Section:** Destinations
## Overview
You want to provide relevant, timely interactions to your customers, and doing so heavily relies on data in your Optimizely account to be accurate and fresh. By automatically syncing customer data and events from your data warehouse into Optimizely, you no longer need to worry about data consistency because data is no longer flowing in from multiple sources in different formats, and you can focus on building world-class customer experiences.
## Supported syncing
| Event Type | Description | Supported Sync Modes |
| --------------------- | -------------------------------------------------- | -------------------- |
| **[Log Events](https://docs.developers.optimizely.com/experimentation/v20.0.0-data/reference/post_events)** | Send any event-based data associated with a user | Upsert |
| **[Visitor Attributes](https://docs.developers.optimizely.com/experimentation/v20.0.0-data/reference/post_events)** | Add customizable data or characteristics to a user | Upsert |
| **[ODP Audiences](https://docs.developers.optimizely.com/optimizely-data-platform/reference/create-list-1)** | Maintain subscription statuses of users in ODP audiences | Add, Remove |
## Connect to Optimizely
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Optimizely** and click **Continue**. You can then authenticate Hightouch to Optimizely with a personal access token and an account ID.
### Web experimentation
#### Authenticate with a personal access token
To use the Optimizely web experimentation APIs, generate a personal access token:
1. Navigate to your [Optimizely profile](https://app.optimizely.com/v2/profile/api) and click **Generate New Token**.
2. Enter a name for the token and click **Create**.
You will also need the `Project ID`. The project ID can be found in the URL by going to your [Optimizely account](https://app.optimizely.com/v2/projects).
### Optimizely data platform
#### Authenticate with a private API key
To use the Optimizely Data Platform APIs, get your API key by following [Optimizely's guide](https://docs.developers.optimizely.com/optimizely-data-platform/reference/authentication) on ODP private keys.
## Syncing configuration
### Log events
Syncing log events lets you send user actions from any source such as offline experiment data, web-page interactions, or third-party sources.
#### Record matching
You can associate events with the visitors who made them on the `Visitor ID` field.
#### Timestamps
Hightouch allows you to send a timestamp of when an event occurred. Timestamps must be sent as milliseconds since the epoch. If an event doesn't include a timestamp, the time of the API request is used.
#### Field mapping
You can sync columns from your source to Optimizely event fields. Optimizely events require a UUID that will be used by Optimizely to de-duplicate events that are accidentally or erroneously replayed.
### Visitor attributes
Syncing visitor attributes lets you maintain up-to-date customer data and create better audiences for personalized experiences in Optimizely. Note that you cannot create net-new properties with this API: attributes must be created within Optimizely before using this endpoint. Since Optimizely audience membership is handled by whether a visitor's attributes satisfy the conditions required for the audience, sending data to this endpoint controls audience membership in Optimizely.
#### Record matching
You can associate attributes to visitors on the `Visitor ID` field.
#### Field mapping
You can sync columns from your source to Optimizely visitor attributes. Attributes must be created within Optimizely before creating mappings.
### ODP audiences
Sync data from any source to maintain subscription statuses of audiences in Optimizely.
#### Select an existing audience or create a new one
You can create a new audience segment or use an existing one. When creating a new audience, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model. To use an existing audience, select the desired audience from the dropdown.
#### User identifier
To identify which users to add or update in an audience, select a model column containing the user identifier. This identifier will be mapped to the `fs_user_id` and `web_user_id`.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Oracle DB
**URL:** https://hightouch.com/docs/destinations/oracle-db
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------ | ----------------------------------------------- | -------------------- |
| Any data set | Sync data from any source to an Oracle DB table | Upsert |
## Connect to Oracle DB
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Oracle DB** and click **Continue**. If you're not using a [tunnel](#ssh-tunneling), you can then authenticate Hightouch to Oracle DB by entering the following fields:
- **Host**: The hostname or IP address of your Oracle server.
- **Port**: The port number of your Oracle server. The default port number is 1521, but yours may be different.
- **Database**: This specifies the database to use when Hightouch executes queries in Oracle DB. This is different than the host, but your host address might contain your database name.
- **Username**: The user that has access through Hightouch to the database and tables you want to sync to. It's best to [create a new user](https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/CREATE-USER.html) specifically for Hightouch access. _Do not use the root user._
- **Password**: The password for the specified user.
To ensure your [credentials](#required-permissions) are correct, click **Test connection**. This confirms if Hightouch can connect to your database by running a basic `SELECT` query.
### Required permissions
The user you use to authenticate must have the following permissions:
- Add, update, and delete ([if applicable](#delete-behavior)) rows in your sync's table.
- Create tables in your database, which are kept only temporarily and used to perform `MERGE` operations. These tables are cleaned up at the end of each sync.
- View the following tables which are used for gathering metadata to set up the sync:
- `USER_CONSTRAINTS`
- `USER_CONS_COLUMNS`
- `USER_CATALOG`
- `USER_TAB_COLUMNS`
### SSH tunneling
Hightouch can connect directly to Oracle DB over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your Oracle DB instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
## Sync configuration
Once you've set up your Oracle DB destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Oracle DB destination you want to sync to.
Hightouch supports **Upsert** mode, with the option to [delete removed rows](#delete-behavior).
### Record matching
Hightouch requires you select a unique identifier in the table you are syncing to. The model column you select must match a `UNIQUE`, `PRIMARY KEY` column within Oracle DB.
You can see columns that fit this criteria as available options in records matching section.
If there are no fields in the dropdown, you must add a unique type column to your Oracle DB table. Then, click the refresh icon to access the newly created column.
### Column types
Hightouch works out of the box with most standard column types, including:
- `VARCHAR2`
- `NUMBER`
- `TIMESTAMP`
- `DATE`
- `BINARY_FLOAT`
- `BINARY_DOUBLE`
If you see type errors, it may be because your model is producing the wrong format. If so, use [typecasting](/models/data-types-casting) to resolve the issue.
### Field mapping
Select which model columns you want to sync to your Oracle DB table columns. Hightouch automatically pulls the columns from your table to make them available for you to map.
### Batch size
You can tune the number of rows that Hightouch upserts per query based on your needs and database threshold. The default is 1000 rows per batch.
### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | -------------------------------------- |
| **Do nothing** | Keep the row in Oracle DB |
| **Delete row** | Remove the row from Oracle DB entirely |
## Tips and troubleshooting
### Common errors
### Compatible versions
The supported Oracle DB version is `19c`. Other versions aren't officially supported but may work regardless.
### Live debugger
### Sync alerts
### Unsupported data types
Hightouch does not support `INTERVAL` types for Oracle.
---
## Orbit
**URL:** https://hightouch.com/docs/destinations/orbit
**Description:** Enrich data of your community members and automate actions
**Section:** Destinations
## Setup
To enable syncing with [Orbit](https://orbit.love) first you need to get an authentication token. Go to your Account Settings (on the menu
under your profile picture) and click API Token:
Then copy the value, and create a new Destination in Hightouch, selecting Orbit as the destination type. Lastly, paste
the API token into the field. Save the destination and it's ready to start syncing data.
## Syncing activities
When syncing activities, Hightouch treats any records added to [your source](/getting-started/concepts) as new activities and sends them to Orbit when your sync runs.
Select the _Activities_ object from the drop down menu. Then you can
define a static value for the title and the identity source for all the
activities, or define the column that will contain the information.
Lastly, choose the mappings for the columns in your model.
## Syncing members
Select the _Members_ object from the drop down menu.
### Record matching
Records can be matched from your source to your Orbit workspace by using the email field.
### Field mapping
You can sync columns from your source to Orbit Member fields.
### Rate limits
By default, Orbit limits 120 requests per IP per minute. If you experience slow syncs, please contact them to increase that limit,
and let us know.
---
## Ortto
**URL:** https://hightouch.com/docs/destinations/ortto
**Description:** Ortto brings the customer data platform, customer journey marketing, and customer journey analytics together to grow your SaaS business.
**Section:** Destinations
## Overview
Enrich people and organization objects in Ortto and keep them up-to date with changes from your data warehouse. This destination also empowers you to customize your audiences by subscribing or unsubscribing people from them based on custom criteria.
## Supported syncing
|Sync Type |Description |Supported Sync Modes |
|------------------|-----------------------------------------------------------------------------------------------|---------------------|
|**Person** |Sync records to Ortto [people](https://help.ortto.com/user/latest/people/) | Upsert, Update |
|**Organization** |Sync records to Ortto [organizations](https://help.ortto.com/user/latest/organizations/) | Upsert, Insert |
|**Audience** |Subscribe or unsubscribe people from [audiences](https://help.ortto.com/user/latest/audiences/)| Insert |
## Getting started
Connecting Ortto to Hightouch requires an Ortto **API Key**. You can retrieve it by following these steps:
1. Navigate to **Data sources** under **More** in the Ortto UI.
2. Create a new data source by selecting **Custom API (advanced)**.
3. Enter an **API name**, for example "Hightouch."
4. Copy your **Private API key** into Hightouch.
5. Select the region of your Ortto instance from the dropdown menu.
If you need further details on creating an Ortto API key, check out the [Ortto documentation](https://help.ortto.com/user/latest/data-sources/configuring-a-new-data-source/other-integrations/custom-api.html).
## Syncing data
Hightouch supports syncing to the following to Ortto [people](#people), [organizations](#organizations), and [audiences](#audiences). You need to setup a separate sync for each object type you want to sync to.
### People
You can upsert, update, or insert [people](https://help.ortto.com/user/latest/people/) in Ortto based on records in your query results.
#### Record matching
You can match rows in your model with people on the following fields:
- Email
- External ID
#### Field mapping
You can sync columns from your query results to [people's](https://help.ortto.com/user/latest/people/) default and custom fields.
#### Delete behavior
You have the following options for how Hightouch should handle rows leaving your query results:
| Behavior | Description |Supported Sync Modes |
| ---------------------| -----------------------------------------------------------------------------|---------------------|
| **Do nothing** | Keep the person object in Ortto | Any |
| **Clear** | Clear the mapped fields but keep the person object in Ortto | Update |
| **Delete** | Delete the person object from Ortto completely | Upsert |
### Organization
You can perform upsert or insert [organizations](https://help.ortto.com/user/latest/organizations/) in Ortto based on records in your query results.
#### Record matching
You can match rows in your model with organization on the following fields:
- Name
- Website
#### Field mapping
You can sync columns from your query results to an [organization's](https://help.ortto.com/user/latest/organizations/) default and custom fields.
### Audiences
Subscribe people to [audiences](https://help.ortto.com/user/latest/audiences/) in Ortto when records for them appear in your query results.
#### Record matching
You can match rows in your model with people in Audience on email only.
#### Delete behavior
Records that no longer appear in your query results are unsubscribed from Ortto audiences.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Outreach
**URL:** https://hightouch.com/docs/destinations/outreach
**Description:** Empower your sales team to act upon customer activity by pushing enriched customer data from your warehouse into Outreach
**Section:** Destinations
## Setup
After selecting Outreach from our Destinations catalog, you will be prompted to
initiate an OAuth flow. You will then be redirected back to Hightouch.
## Syncing
Hightouch supports syncing to the following Outreach objects:
- `Accounts`
- `Prospects`
- `Sequence States` (instance of a prospect within a sequence)
### Sync modes
Here are the possible modes for how to alter Outreach objects:
- **Upsert**: pushes new objects to Outreach and updates fields that change in your warehouse.
- **Update**: updates particular fields on existing objects in Outreach. It doesn't add new objects.
`Sequence States` will only have _insert_ mode
- **Insert**: pushes new objects to Outreach. It doesn't update any objects.
### Record matching
Records can be matched from your source to your Outreach workspace by the given fields in the dropdown.
The record should be matched using a unique identifier, otherwise the intended
records may not be updated properly.
### Field mapping
You can sync columns from your source to Outreach's object properties, including custom fields, and relationships.
You will still need to refer to the custom field number, despite having a different custom display name.
### Syncing sequence states
Syncing sequence state objects is adding prospects to their corresponding sequences. To successfully start a sequence and engage
with the prospect, you must make sure your sequence has the correct settings and sending and syncing messages from mailbox are enabled (if emailing).
Creating a sequence state requires that you have information on all of these Outreach objects:
- prospects
- sequences
- mailboxes
Here is an example on how to set up the configurations:
You will select a field from your model to match to the corresponding object. You need to make sure you are using a unique identifier.
Hightouch will use that to look up each of the corresponding object's ID and combine them to create a sequence state.
## Sync performance and parallelization
Outreach has a fixed limit of 10,000 requests per hour per user. This limit resets every hour on the dot.
To accommodate this limit and avoid potentially impacting your usage of the Outreach desktop app during the sync run, Hightouch will draw out the sync by sending just one request at a time.
If the limit is hit during a sync run, Hightouch will wait until the next hour to continue the rest of the sync.
Note that the wait time can vary depending on when the rate limit is hit.
If you want to speed up your sync, you may increase the parallelization to send more requests in parallel.
Keep in mind that this will increase the risk of hitting the rate limit. We recommend that you run a test through the "Test a row"
feature in the sync configuration form page and spot check how many requests are made. You can then estimate how many requests
each sync run will make based on how many rows you expect to be added, changed, or removed. If you estimate that you will exceed 10,000 requests,
it's best to not increase the parallelization. Another safety net is to create a Hightouch specific user
for the sync so it doesn't lock out other users from using the Outreach app.
---
## Destinations overview
**URL:** https://hightouch.com/docs/destinations/overview
**Description:** Learn how to connect destinations in Hightouch so your models can be synced into the tools where teams run campaigns, measure results, and engage customers.
**Section:** Destinations
A **destination** is any system where teams consume activated data.
Common examples include:
- **CRMs** (Salesforce, HubSpot)
- **Marketing platforms** (Braze, Iterable, Marketo)
- **Ad platforms** (Meta Ads, Google Ads, LinkedIn)
- **Support tools** (Zendesk, Intercom)
- **Analytics platforms** (Amplitude, Mixpanel)
- **Custom APIs** (via Webhook/HTTP Request destinations)
Destinations are the endpoint of Hightouch’s [**Reverse ETL**](/getting-started/concepts) pipeline. Once a [**model**](/models/creating-models) is defined, you can configure one or more [**syncs**](/syncs/overview) to deliver data into destinations.
**For marketers**: a destination is the tool where you build campaigns, measure results, or engage with customers—like an ads platform, email tool, or CRM.
---
## Add a destination
To create a new destination:
1. Go to **Integrations → [Destinations](https://app.hightouch.com/destinations)**.
2. Click **Add destination**.
You’ll be prompted to choose from supported destination types and configure:
- **Authentication** (OAuth, API key, service account, etc.)
- **Object type** (e.g. user, event, list, custom object)
- **Sync type** supported by that destination (upsert, insert, update, event streaming, etc.)
Each destination has unique configuration options. See catalog and refer to **integration-specific documentation** for setup details and supported sync types.
---
## Security and data handling
Hightouch writes data into destinations using secure APIs:
- All outbound traffic is encrypted via TLS 1.2+
- Data is batched and delivered according to the sync configuration (mode, type, mapping)
- Sensitive fields can be masked or excluded before syncing to destinations
See [**Security and compliance overview →**](https://hightouch.com/docs/security/overview) for more details.
---
## How destinations fit into data activation
- **Data teams**: configure destinations and map model columns to the correct destination fields.
- **Marketers**: access fresh, trusted data directly in the tools where they run campaigns or measure results.
**Example:**
- **Model**: SQL query for “users who abandoned checkout”
- **Destination**: Braze (Email + Push)
- **Sync**: upsert new/changed users nightly, send them into a “Cart Abandonment” campaign
---
## Next steps
- [Add your first destination →](https://app.hightouch.com/destinations)
- Learn about [Syncs →](https://hightouch.com/docs/syncs/overview) and how they deliver data into destinations
---
## Salesforce Pardot
**URL:** https://hightouch.com/docs/destinations/pardot
**Description:** Run better email campaigns on Salesforce Pardot with up-to-date customer data from your data warehouse
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| -------------------------------- | ------------------------------------------------ | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| **Prospect** | Sync Prospects to update list membership objects | Upsert, Update | [Prospects v5](https://developer.salesforce.com/docs/marketing/pardot/guide/prospect-v5.html), [Prospects v4](https://developer.salesforce.com/docs/marketing/pardot/guide/prospects-v4.html), [Prospects v3](https://developer.salesforce.com/docs/marketing/pardot/guide/prospects-v3.html) |
| **Lists** | Sync Prospects to update list membership objects | Update | [Lists v5](https://developer.salesforce.com/docs/marketing/pardot/guide/list-membership-v5.html), [Lists v4](https://developer.salesforce.com/docs/marketing/pardot/guide/list-memberships-v4.html), [Lists v3](https://developer.salesforce.com/docs/marketing/pardot/guide/list-memberships-v3.htm)|
| **Campaign Trigger** | Trigger email sends to Prospects | Update | [One-To-One](https://developer.salesforce.com/docs/marketing/pardot/guide/email-v5.html#one-to-one-email-create), [List](https://developer.salesforce.com/docs/marketing/pardot/guide/list-email-v5.html#list-email-create)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Salesforce Pardot
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button.
Select **Salesforce Pardot** and click **Continue**.
You can then authenticate Hightouch to **Salesforce Pardot** via OAuth.
Connect to Pardot via OAuth by logging into your Pardot account.
Then, input your **Business Unit ID**, which can be found in your Pardot App by navigating to **Setup > Pardot > Pardot Account Setup**.
Next, select the version of the Pardot API you wish to utilize. The v5 option is recommended for modern accounts.
Please note that syncs which have been configured for v3 / v4 are not currently cross-compatible with v5. For this reason, we recommend leaving your version *unchanged* once the destination is configured.
If your Pardot account type is **Pardot Developer Org**, input `pi.demo.pardot.com` in the optional **Pardot Custom Domain** field.
When left unset, this will default to `pi.pardot.com`.
## Sync configuration
Once you've set up your Salesforce Pardot destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data.
Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin.
Then, select the relevant model and the Salesforce Pardot destination you want to sync to.
### Syncing Prospects
#### Record matching
To match source rows to Prospects in Salesforce Pardot, you need to select a source column and corresponding Salesforce Pardot field.
Depending on the [sync mode](/syncs/types-and-modes#sync-modes) you select, you can match on the following Salesforce Pardot fields:
- **Prospect Email**
- **Prospect Salesforce ID**
- **Prospect ID**
In Pardot API v3 / v4, all of these IDs are valid first-order identifiers.
In large v3 / v4 syncs(\>=7500 rows per run), the `batchUpdate` and `batchUpsert` endpoints will be used, which enable us to send 50 records per API request.
Large v5 syncs automatically switch to the CSV processing endpoint, which is identifier-agnostic and uses very few API requests per 100,000 rows.
Pardot only processes one file at a time, so if you have many syncs which are sending CSV files at once, please know that you may experience higher sync times as Hightouch waits for Pardot's CSV queue to open up again.
You can examine your [Pardot CSV Console](https://pi.pardot.com/import/wizardStep1) (Admin -\> Import -\> Prospects) to view the CSV files that Hightouch is syncing to Pardot, as well as the Prospects which were created and updated as a result of them.
By contrast, small v5 syncs(\<7500 rows per run) do not utilize the CSV endpoint. These Prospect by Prospect endpoints require the Prospect Id to be
present in update requests and require the Prospect Email to be present in upsert requests.
Therefore:
- If you select Prospect Id to match records in small v5 upsert syncs, extra API calls will be made to look up the Prospect's email
- If you select Prospect Email to match records in small v5 update syncs, extra API calls will be made to look up the Prospect's Prospect Id
#### Field mapping
You can sync columns from your source to Salesforce Pardot's default and custom fields.
If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
Refer to the [v3 / v4 object docs](https://developer.salesforce.com/docs/marketing/pardot/guide/object-field-reference.html) or [v5 overview](https://developer.salesforce.com/docs/marketing/pardot/guide/version5overview.html) for more information about object fields.
#### Match list
You can select if you want to add the Prospects a **match list** (new or existing), or if you prefer not adding them to any list.
Read about [finding IDs](#finding-ids) if you can't find your match list.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results.
You have the following options:
| Behavior | Description |
| -------------------- | ------------------------------------------------------------------ |
| **Do nothing** | Keep the Prospect in Pardot |
| **Clear fields** | Keep the Prospect in Pardot but clear the synced fields |
| **Remove record** | Remove the Prospect from the specified List |
| **Delete record** | Delete the Prospect from Pardot entirely |
### Syncing Lists
#### Record Matching
To match source rows to lists in Salesforce Pardot, you need to select a source column and corresponding Salesforce Pardot field.
You can match on the following Salesforce Pardot fields:
- **Prospect Email**
- **Prospect ID**
All of the endpoints involved in List syncing prioritize Prospect Id as an identifier. List syncing is currently only supported on v5 of our integration.
Therefore, using Prospect Email in record matching will add aditional API requests to your sync runs.
#### CSV Mode
Here, we give you the ability to explicitly opt into or out of the CSV endpoint methodology.
As outlined in the Prospects section, this is a tradeoff where CSV processing can at times be slower but also enables bulk processing of data and a diminished API request count.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results.
You have the following options:
| Behavior | Description |
| -------------------- | ------------------------------------------------------------------ |
| **Do nothing** | Leave the Prospect on the List |
| **Remove record** | Remove the Prospect from the specified List |
### Sending Emails
To match source rows to Emails in Salesforce Pardot, you need to select a source column and corresponding Salesforce Pardot field.
You can match on the following Salesforce Pardot fields:
- **Prospect Email**
- **Prospect ID**
The notion of sync mode is less applicable here, because the Campaign Trigger sync type does not create an object in the destination.
Instead, it simply references a Prospect and the Email Template to send them. If that Prospect does not exist, the request will fail; this sync type does not enable the upserting of Prospects at the same time.
All of the endpoints involved in List syncing prioritize Prospect Id as an identifier.
Therefore, using Prospect Email in record matching will add aditional API requests to your sync runs.
Use the Email Template selector to preview your email content.
Read about [finding IDs](#finding-ids) if you can't find your Email Template.
Optionally, include the Folder to save this Sent Email in once it is processed by Pardot.
Read about [finding IDs](#finding-ids) if you can't find your Folder.
If you are sending Emails to Prospects which already exist in an updated state in Pardot, their template system will automatically fill out data in your template so long as it is formatted correctly.
#### List Mode
| Behavior | Description |
| -------------------- | ------------------------------------------------------------------ |
| **One-To-One** | As each row is processed, an email is immediately sent to that Prospect |
| **List** | Each batch of rows is assigned to an ephemeral List. An email is then triggered for all Prospects on that List. |
##### Advanced List Options
List mode creates new Prospect Lists that can have a single trigger send an email to the entire list.
Each List is constructed using 10,000 Prospects by default, but this number can be modified in the advanced options at the bottom of the form.
Selecting a lower number of Prospects will send emails more often, but also generate more Prospect Lists in Pardot to facilitate this.
##### Delete Mode
| Behavior | Description |
| -------------------- | ------------------------------------------------------------------ |
| **Do nothing** | Leave the Prospect List used to send the batch emails intact in Pardot |
| **Delete** | Delete the Lists associated with each batch after a Prospect List is sent an Email. |
If desired, Hightouch offers the ability to clean up Lists in Pardot after email campaigns are successfully triggered to them.
Therefore we also offer the ability to clean these Lists up after the email campaign is successfully triggered to them.
## Tips and troubleshooting
### Common errors
**"Invalid Campaign Id"**
This error happens when the **required Pardot campaign ID** is missing when inserting a **prospect record**. A Pardot campaign tracks the first interaction that a prospect has with your online marketing materials and functions similarly to a source field. This field is required when creating a Pardot prospect [(see Salesforce docs)](https://help.salesforce.com/s/articleView?id=mktg.pardot_campaigns_salesforce_campaigns.htm&type=5).
Pardot campaigns are not the same as Salesforce campaigns. In Pardot, each prospect has exactly one campaign **(1:1)**. In Salesforce, a lead or contact can belong to many campaigns **(1:many)**.
Because both systems use “Campaign Id,” it’s easy to confuse them. Be sure your models are pulling the correct ID. This integration currently **only accepts the *numeric* `campaignId`** from Pardot.
**"CRM Connector"**
If there is a CRM connector already present between your Salesforce account and Salesforce Pardot accounts, Pardot may explicitly disallow you from attaching `campaignId` to Prospect requests.
This is because the Prospects are already being automatically associated with campaigns based on the rules defined by your integration. To modify this behavior, consider reaching out to Salesforce Pardot support directly.
### Finding IDs
If you're trying to select an object from a dropdown in Hightouch but find that it is not present, you may need to find it manually.
Pardot's API is capable of enumerating roughly 3000 objects, but beyond this threshhold, they ask that you use a CSV export.
See Pardot's note about the `offset` parameter [here](https://developer.salesforce.com/docs/marketing/pardot/guide/version5overview.html#query-using-get).
Queries to enumerate Lists, Email Templates, Folders, and more are all met with this limitation.
Folders can only be sorted by their internal Pardot ID, but for all other objects we ask Pardot to enumerate them in the order they were most recently updated.
If you're trying to find an ID, an easy first step is to simply edit and update the object in Pardot, bringing it to the top of the dropdown in our form alongside anything else you've been working on recently.
If you have the object open in Pardot, however, you can often find its ID simply by looking at the URL which references it.
If this still proves ineffective, consider using Pardot's inbuilt CSV export tool which should be visible from any table of objects.
The exported CSV will give you data and metadata about the objects in question.
Once you have your ID, bring it back to the Hightouch sync configuration and use the "Edit as JSON" button to change the ID value manually.
### Live debugger
### Sync alerts
---
## PartnerStack
**URL:** https://hightouch.com/docs/destinations/partnerstack
**Description:** Empower your partnership programs using Partnerstack with Hightouch
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-----------------|-----------------------------------------------------------------|----------------------|----------------------------------------------------------------------------------
**Customers** | Sync data from any source to PartnerStack customers | Upsert | [Customers docs](https://docs.partnerstack.com/reference/post_v2-customers-1)
**Transactions** | Sync data from any source to PartnerStack transactions | Insert | [Transactions docs](https://docs.partnerstack.com/reference/post_v2-transactions)
**Events** | Sync data from any source to PartnerStack S2S conversion events | Insert | [Events docs](https://docs.partnerstack.com/docs/server-to-server-s2s-tracking)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to PartnerStack
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **PartnerStack** and click **Continue**. You can then authenticate Hightouch to **PartnerStack**.
Enter the following fields into Hightouch:
If you want to use the PartnerStack public API, the following keys are required:
- **Public key**
- **Secret key**
If you want to send PartnerStack S2S conversion events, the following key is required:
- **S2S tracking token**
To find the above keys, navigate to your [PartnerStack Integration Settings](https://app.partnerstack.com/settings/integrations).
## Sync configuration
Once you've set up your PartnerStack destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the PartnerStack destination you want to sync to.
### Syncing customers
Sync data from any source to PartnerStack customers.
#### Record matching
You can match rows from your model to customers in PartnerStack on any column in your model and any field in PartnerStack. Ensure the data types of the model column and PartnerStack field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync customer fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom customer fields.
PartnerStack requires the following fields, so you must map them to complete your configuration:
- **Email**
- **Partner Key**
- **Customer Key**
If you send data for a custom field that doesn't exist, Hightouch adds the field.
Ensure your model's columns have the same data types as the fields you want to sync to.
### Syncing transactions
Sync data from any source to PartnerStack transactions.
#### Record matching
You can match rows from your model to transactions in PartnerStack on any column in your model and any field in PartnerStack. Ensure the data types of the model column and PartnerStack field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, PartnerStack automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync transaction fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default transaction fields.
PartnerStack requires the following fields, so you must map them to complete your configuration:
- **Customer Key**
- **Transaction Key**
Ensure your model's columns have the same data types as the fields you want to sync to.
### Syncing events
Sync data from any source to PartnerStack S2S conversion events.
#### Record matching
You can match rows from your model to events in PartnerStack on any column in your model and any field in PartnerStack. Ensure the data types of the model column and PartnerStack field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Pendo
**URL:** https://hightouch.com/docs/destinations/pendo
**Description:** Perform deeper analysis and better understand user behavior by bringing enriched customer data into Pendo from your data warehouse
**Section:** Destinations
## Overview
Ideally, your data warehouse should contain all your customer data. When that's the case, using your warehouse as the source for your Pendo data enables richer, more granular analysis. You can sync enriched data from your warehouse and data from other engagement tools. Moreover, you can ensure that the data in Pendo consumed by your product and growth teams is always up-to-date and in the expected format.
If your company also uses a Business Intelligence tool such as Looker or Mode alongside Pendo, you can ensure that the data is consistent across both systems since your data warehouse is the only true source of data.
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| ----------- | ------------------------------------------------------------------ | -------------------- | --------------------------------------------------------------- |
| **Events** | Sync records as events to Pendo, often in the form of a track call | Insert | [Track API reference](https://engageapi.pendo.io) |
| **Objects** | Update visitor, account, and parent account objects | Update | [API overview](https://engageapi.pendo.io) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Pendo
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Pendo** and click **Continue**. You can then authenticate Hightouch to Pendo with an **Integration Key** and **Track Event Shared Secret**.
{/* */}
### Create Pendo Integration Key
{/* */}
In your [Pendo admin](https://app.pendo.io/admin/) go to **Settings** > **Integrations** > **Integration Keys**. Then follow the instructions below:
1. Select **+ Add Integration Key**.
2. Enter a meaningful **description** for your Integration Key, for example, "Hightouch Integration."
3. Provide the key **Allow writes** access.
4. Select **Create**. Securely store this key for use in Hightouch.
For more information about Integration Keys, refer to Pendo's [Integration Key article](https://support.pendo.io/hc/en-us/articles/9491198203547-Pendo-Integration-Key#requirements-0-0).
{/* */}
### Access Track Event Shared Secret
{/* */}
To access your Track Event Shared Secret go to **Subscription Settings** > **Choose your App** > **App Details** in your [Pendo admin](https://app.pendo.io/admin/). You must be a Pendo Admin to have access to this Secret Key. You can find more information in Pendo's [events tracking documentation](https://support.pendo.io/hc/en-us/articles/360032294291-Track-Events-Configuration#segment-com-0-5).
## Sync configuration
Once you've set up your Pendo destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Pendo destination you want to sync to.
### Events
The Pendo integration lets you send events to Pendo with custom properties. You must input the following information:
- **Event name**: either a model column containing the event name or a static value.
- **Timestamp**: a model column containing the time the event occurred; if you leave this empty, Pendo uses the time the event arrives at the server.
#### Field mapping
Pendo suggests that you always supply both a Visitor ID (required) and Account ID (strongly recommended).
In addition, you can choose to map additional fields to your Pendo event payload in the custom properties.
### Objects
The Pendo integration allows users to update existing visitors, accounts, and parent accounts with metadata—both agent and custom fields.
#### Record matching
You have to choose how to match records from your model to objects in Pendo. Select a column from your model that corresponds to your Pendo profile ID. The following example screenshot shows mapping the model's **id** column to Pendo's **Visitor ID** field.
If users aren't found in Pendo, Hightouch automatically skips the row.
#### Field mapping
Pendo has two field types:
- **Agent fields** are API-created fields the agent integrated on your website or app.
- **Custom fields** are editable in the UI and through the API,
You can choose which fields you want to update on your profiles in Pendo. The following example screenshot shows mapping the model's **firstname** column to the **firstname** field in Pendo.
You cannot create agent fields in the Pendo UI. If you want to write to a new field in Pendo that doesn't exist, you can pass in the desired field's name in Hightouch. The option to create that new field will appear, and if you select it, the field will be created during the next sync.
You can create custom fields in Pendo by going to [**Settings** > **Data Mappings**](https://app.pendo.io/admin/mappings).
#### Split retries
Pendo counts all events in a batch as rejected if the request contains a single invalid event.
To pinpoint which records are getting rejected with which errors and reduce the number of valid records that get retried,
you can enable [**split retries**](/syncs/retries#split-retries).
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ---------------------------------------------------------- |
| **Do nothing** | Keep the document in Pendo |
| **Delete** | Remove the document from Pendo |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Pinterest Ads
**URL:** https://hightouch.com/docs/destinations/pinterest-ads
**Description:** Deliver better ads by syncing conversion events and customer list audiences to Pinterest Ads
**Section:** Destinations
## Overview
Manage your conversion tracking by syncing conversion events and promote your pins to a specific Pinterest audience by syncing customer lists from your data warehouse. Deliver more relevant ads, boost marketing ROI, and retarget your most loyal customers.
## Supported syncing
| Object Type | Description | Supported Sync Modes |
| --------------------------- | --------------------------------------------------------------------------- | -------------------- |
| **Conversion Events** | Sync conversion events from any source to Pinterest Ads | Insert |
| **Customer List Audiences** | Sync your PII data from any source to Pinterest Ads customer list audiences | Add, Remove |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Pinterest Ads
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Pinterest Ads** and click **Continue**. You can then authenticate Hightouch to Pinterest Ads by selecting **Log in to Pinterest Ads** and logging into your Pinterest Ads account.
Once successful, you will be redirected back to Hightouch to enter your **advertiser account ID** and **conversion access token**.
- **Advertiser account ID**: You can locate your advertiser account ID within your Pinterest business account by navigating to
the dropdown button on the top right of the page, then **Business Access** > **Ad Accounts**. You can find your ID under your the **Asset name**.
- **Conversion access token**: If you want to send conversion events to Pinterest Ads, you also need to input a conversion access token. You can create a conversion access token within your Pinterest business account by navigating to the **Ads** dropdown then **Conversions** > **Conversion access token**. Click **Generate new token** and enter the token in Hightouch.
## Sync configuration
Once you've set up your Pinterest Ads destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Pinterest Ads destination you want to sync to.
### Syncing conversion events
Hightouch supports syncing conversion events to Pinterest Ads.
#### Record matching
You can match events from your source to Pinterest Ads using the Event ID.
#### Field mapping
You can use [field mapping](/syncs/mapping-data) to sync data to the event properties of a Pinterest Ads conversion event. Refer to the [Pinterest Ads developer docs](https://developers.pinterest.com/docs/api/v5/#operation/events/create) for descriptions of each property.
Ensure your model columns' data types match the data types of the fields you want to sync to.
For fields that expect an array , Hightouch automatically converts your row values to arrays if they aren't already. For example, you can map [string model columns](/models/creating-models#data-types-and-casting) to array type Pinterest Ads fields.
The Pinterest Ads API requires the **Action Source** field, so you must map it to complete your configuration.
**Action Source** is an enumerated field which only accepts the following `string` values:
- "app_android"
- "app_ios"
- "web"
- "offline"
#### Handling PII
Pinterest Ads requires some identity information to be normalized and hashed. We support hashing for all fields that require values to be SHA256 hashed. Disabling PII hashing requires you to do this yourself.
If you're handling hashing yourself, make sure that email addresses are in lowercase before hashing, as explained in [Pinterest's documentation](https://developers.pinterest.com/docs/api/v5/#operation/events/create).
#### Cleaning phone numbers
You can select whether Hightouch should clean your phone numbers. By default, Hightouch doesn't clean phone numbers.
Pinterest Ads requires users' phone numbers to be normalized and hashed. The phone number should only include digits with country code, area code, and number without any symbols, letters, spaces, and leading zeros.
If you enable cleaning phone numbers, Hightouch tries to convert the phone number into E.164 format and remove the leading plus (`+`) symbol.
#### Using the test events endpoint
Pinterest Ads provides a [test events endpoint](https://help.pinterest.com/en/business/article/validating-your-set-up-with-event-testing) to validate your set up for conversion events. This endpoint does not affect
production conversion data, and returns warnings and recommendations based on your set up. Syncs can be run using this endpoint and results can be seen by clicking on **Test Events** under the the **Conversions Tab**.
### Syncing customer list audiences
A **customer list audience** is a [customer list that is added to an audience](https://help.pinterest.com/en/business/article/audience-targeting).
Depending on your model query results, Hightouch adds new user records into the Pinterest Ads customer list audience or removes them.
#### User identifiers
To identify which users to add or update in an audience, select model columns and the corresponding Pinterest Ads user fields. You can match on any of the following Pinterest Ads user fields:
- Email
- IDFA
- MAID
- LR_ID
- DLX_ID
- Hashed Pinner ID
Emails must be lowercase and can be plain text or hashed using SHA1, SHA256,
or MD5. MAIDs and IDFAs must be hashed with SHA1, SHA256, or MD5.
#### New audience creation
In your sync configuration, you can tell Hightouch to create a new customer list audience for you. You can optionally assign a custom name to your audience. If no name is provided, Hightouch defaults to using your model name.
To help differentiate the name of your customer list and audience, we save the customer list name as `customer list-{customer list name}` and audience name as `audience-{audience name}`.
The following example screenshot would create a new customer audience list called "Users 1."
After creation, the customer list name will be "customer list-Users 1," and the audience name will be "audience-Users 1."
#### Using an existing audience
Instead of creating a new data customer list audience, Hightouch can also sync users to existing audiences that are already populated in your Pinterest Ads account.
The Hightouch UI shows the customer list and audience as a comma-separated string.
In the example screenshot below, the customer list audience shows as "customer list-Users 1, audience-Users 1."
#### Field mapping
You can sync columns from your model to the following identifiers in Pinterest:
- Email
- IDFA
- MAID
- LR_ID
- DLX_ID
- Hashed Pinner ID
Hightouch [Match Booster](/match-booster/overview) can help to provide additional identifiers and increase your match rates.
#### Handling PII
Pinterest Ads requires some identity information to be normalized and hashed. Hightouch supports hashing for all user types except for **Email**, since Pinterest Ads doesn't require hashing this field.
Enable PII hashing if you wish for Hightouch to normalize and hash PII fields using a SHA256 hash. Disabling PII hashing requires to do this yourself.
## Tips and troubleshooting
### Matched users count
Below only applies to the audiences sync type.
### Common errors
#### Delayed syncing
Pinterest takes around 24 hours to fully sync after Hightouch has synced data to their API. You can find more details under **customer list** on [Pinterest's audience targeting help page](https://help.pinterest.com/en/business/article/audience-targeting).
#### Smaller audience size
You may notice that the audience size may be smaller than the total number of records synced to Pinterest Ads. This is because [Pinterest filters out
records that don't have a Pinterest account](https://developers.pinterest.com/docs/api/v5/#tag/customer_lists).
#### 400 - Invalid request
You may receive a `400 - {"code":1,"message":"Invalid request: {} is not valid under any of the given schemas"}` error message because a row being sent to Pinterest contains a `null` or blank value.
Ensure that mapped fields do not contain any`null` or blank values. You can resolve this error by:
- Filtering these rows in the model's query
- Selecting the [Don't sync null values](https://hightouch.com/docs/syncs/mapping-data#dont-sync-null-values) checkbox in of the field mapping configurations.
### Live debugger
### Sync alerts
---
## Pipedrive
**URL:** https://hightouch.com/docs/destinations/pipedrive
**Description:** Enable your sales team to engage with prospects better by syncing enriched data from your warehouse to Pipedrive CRM
**Section:** Destinations
## Overview
Your leads and prospects expect you to have the necessary context before engaging with them. You can accomplish this by syncing lead and prospect data from your data warehouse to Pipedrive CRM. You no longer need to worry about executives chasing the wrong leads or engaging with warm ones without proper context.
## Supported syncing
|Sync Type|Description|Supported Sync Modes | API Reference|
|--|--|--|--|
|**Organization**|Sync data from any source to organizations|Upsert, Update| [Organization docs](https://developers.pipedrive.com/docs/api/v1/Organizations/)|
|**Person**|Sync data from any source to person objects|Upsert, Update|[Person docs](https://developers.pipedrive.com/docs/api/v1/Persons#getPersonActivities)|
|**Deal**|Sync data from any source to deals|Upsert, Update|[Deal docs](https://developers.pipedrive.com/docs/api/v1/Deals)|
|**Lead**|Sync data from any source to leads|Upsert, Update| [Lead docs](https://developers.pipedrive.com/docs/api/v1/Leads)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Pipedrive
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Pipedrive** and click **Continue**. You can then authenticate Hightouch to Pipedrive with an API key.
You can retrieve your Pipedrive API Key under **Settings > Personal preferences > API**.
For more information, see [Pipedrive's docs](https://support.pipedrive.com/en/article/how-can-i-find-my-personal-api-key).
## Sync configuration
Once you've set up your Pipedrive destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Pipedrive destination you want to sync to.
### Record matching
You can match rows in your model with objects in Pipedrive on any unique property. Pipedrive's `id` field is only available in the dropdown menu when using **Update** mode.
To match Deal objects, the [Pipedrive
API](https://developers.pipedrive.com/docs/api/v1/Deals#searchDeals) requires
a search term that is *not* the Deal ID. Available search terms
pre-populate your Deal matching field drop-down.
### Field mapping
You can sync columns from your source to Pipedrive default and custom fields.
### Delete behavior
| Behavior | Description |
| -------------- | -------------------------------------------------------------------- |
| **Do nothing** | Keep the record in Pipedrive |
| **Delete** | Remove the record from Pipedrive entirely |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## PlanetScale
**URL:** https://hightouch.com/docs/destinations/planetscale
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Supported syncing
|Sync Type|Description |Supported Sync Modes |
|-----------|--------------------------------------------------|---------------------|
| **Table** |Sync data from any source to a PlanetScale table | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to PlanetScale
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **PlanetScale** and click **Continue**.
Enter the following required fields into Hightouch:
- **Host**: The hostname or IP address of your PlanetScale server.
- **Port**: The port number of your PlanetScale server. The default is 3306, but yours may be different.
- **Database**: This specifies the database to use when Hightouch executes queries in Planetbase. This is different than the host, but your host address might contain your database name.
- **User**: This can be your personal PlanetScale login or a dedicated user for Hightouch.
- **Password**: The password for the user specified above.
## Sync configuration
Once you've set up your PlanetScale destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the PlanetScale destination you want to sync to.
### Select table
Select the existing table in PlanetScale you want to sync to.
### Record matching
You can match rows from your model to rows in PlanetScale on any column as long as it is a primary key or unique key.
You can see columns that fit this criteria as available options in records matching section.
If there are no fields in the dropdown, you must add a unique type column to your PlanetScale table. Then, click the refresh icon to access the newly created column.
### Field mapping
Select which model columns you want to sync to your PlanetScale table columns. Hightouch automatically pulls the columns from your table to make them available for you to map.
When syncing to [binary](https://dev.mysql.com/doc/refman/8.0/en/binary-varbinary.html) destination fields, Hightouch assumes the model column values to be [base64](https://en.wikipedia.org/wiki/Base64) strings. If they aren't, you can use [template mapping](/syncs/mapping-data#template-mapping) to [cast](/models/data-types-casting#casting) them if needed.
### Batch size
You can tune the number of rows Hightouch upserts per query based on your needs and database threshold. The default is 1000 rows per batch.
If you want to increase the sync's speed, you can increase the batch size. Keep in mind that PlanetScale fails the entire batch of rows if it detects any erroneous row. If you suspect that you will have many bad rows, don't use a high batch size. To avoid locks, ensure you account for your database's capacity when increasing the batch size.
### Delete behavior
When rows no longer shows up from your model's query results, you can configure the sync to **Do nothing** or **Delete** those rows from the PlanetScale.
| Behavior | Description |
| -------------------- | --------------------------------------------------- |
| **Do nothing** | Keep the synced rows in the table in PlanetScale |
| **Delete** | Delete the synced rows from the table in PlanetScale|
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Planhat
**URL:** https://hightouch.com/docs/destinations/planhat
**Description:** Sync real-time analytics and enrich customer data in Planhat to optimize your customer management and growth
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ------------------- | ---------------------------------------------------- | ---------------------- | ------------------------------------------------------------------- |
| **Assets** | Sync data from any source to Planhat assets | Upsert, Update, Insert | [Assets endpoint](https://docs.planhat.com/#assets) |
| **Campaigns** | Sync data from any source to Planhat campaigns | Upsert, Update, Insert | [Campaigns endpoint](https://docs.planhat.com/#campaigns) |
| **Company** | Sync data from any source to Planhat company | Upsert, Update, Insert | [Company endpoint](https://docs.planhat.com/#companies) |
| **Conversations** | Sync data from any source to Planhat conversations | Upsert, Update, Insert | [Conversations endpoint](https://docs.planhat.com/#conversations) |
| **Endusers** | Sync data from any source to Planhat endusers | Upsert, Update, Insert | [Endusers endpoint](https://docs.planhat.com/#endusers) |
| **Invoices** | Sync data from any source to Planhat invoices | Upsert, Update, Insert | [Invoices endpoint](https://docs.planhat.com/#invoices) |
| **Issues** | Sync data from any source to Planhat issues | Upsert, Update, Insert | [Issues endpoint](https://docs.planhat.com/#issues) |
| **Licenses** | Sync data from any source to Planhat licenses | Upsert, Update, Insert | [Licenses endpoint](https://docs.planhat.com/#licenses) |
| **Metrics** | Sync data from any source to Planhat metrics | Insert | [Metrics endpoint](https://docs.planhat.com/#bulkinsert_metrics) |
| **Notes** | Sync data from any source to Planhat notes | Upsert, Update, Insert | [Notes endpoint](https://docs.planhat.com/#notes) |
| **Opportunities** | Sync data from any source to Planhat opportunities | Upsert, Update, Insert | [Opportunities endpoint](https://docs.planhat.com/#opportunities) |
| **Projects** | Sync data from any source to Planhat projects | Upsert, Update, Insert | [Projects endpoint](https://docs.planhat.com/#projects) |
| **Sales** | Sync data from any source to Planhat sales | Upsert, Update, Insert | [Sales endpoint](https://docs.planhat.com/#sales) |
| **User Activities** | Sync data from any source to Planhat user activities | Insert | [User Activities endpoint](https://docs.planhat.com/#activity_bulk) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Planhat
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Planhat** and click **Continue**. You can then authenticate Hightouch to **Planhat** by entering the following required fields into Hightouch:
- **API Access Token**: You can generate an API token under the **Settings > Service Accounts** section in Planhat.
- **Tenant UUID**: You only need to provide a Tenant UUID if you plan to sync to **Metrics** or **User Activities**. You can get your Tenant UUID under in the **Developer** section in Planhat.
- **Region**: You only need to provide a region if your Planhat app subdomain isn't `app`. For example, if your app URL is `app-us1.planhat.com`, then enter `us1`. If your app URL is `app.planhat.com`, leave the region blank.
## Sync configuration
Once you've set up your Planhat destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Planhat destination you want to sync to.
### Syncing companies
Companies or accounts are your customers.
Depending on your business these might be agencies, schools, other businesses or something else.
Companies can also be your previous customers and or prospects.
#### Record matching
To match rows from your model to company in Planhat, you need to select a model column and corresponding Planhat field. You can match on any of the following Planhat fields:
- **ID**
- **Source ID**
- **External ID**
In **Insert** mode, Planhat automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync company fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom company fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available on **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
| Behavior | Description |
| -------------- | ------------------------------------------------------------ |
| **Do nothing** | Keep the company in Planhat with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the company in Planhat |
| **Delete** | Delete the synced companys from Planhat |
### Syncing endusers
An enduser represents an individual at one of your customers, typically a user of your product, a business contact or both.
#### Record matching
To match rows from your model to enduser in Planhat, you need to select a model column and corresponding Planhat field. You can match on any of the following Planhat fields:
- **ID**
- **Source ID**
- **External ID**
In **Insert** mode, Planhat automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync enduser fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom enduser fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available on **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
| Behavior | Description |
| -------------- | ------------------------------------------------------------ |
| **Do nothing** | Keep the enduser in Planhat with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the enduser in Planhat |
| **Delete** | Delete the synced endusers from Planhat |
### Syncing sales
Sales represents not recurring revenue (NRR), like an onboarding fee, or a one-off professional services project.
#### Record matching
To match rows from your model to sale in Planhat, you need to select a model column and corresponding Planhat field. You can match on any of the following Planhat fields:
- **ID**
- **Source ID**
- **External ID**
In **Insert** mode, Planhat automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync sale fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom sale fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available on **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
| Behavior | Description |
| -------------- | --------------------------------------------------------- |
| **Do nothing** | Keep the sale in Planhat with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the sale in Planhat |
| **Delete** | Delete the synced sales from Planhat |
### Syncing invoices
Planhat automatically generates invoices when a license is created or renewed.
Invoices can include multiple line items.
#### Record matching
To match rows from your model to invoice in Planhat, you need to select a model column and corresponding Planhat field. You can match on any of the following Planhat fields:
- **ID**
- **Source ID**
- **External ID**
In **Insert** mode, Planhat automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync invoice fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom invoice fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available on **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
| Behavior | Description |
| -------------- | ------------------------------------------------------------ |
| **Do nothing** | Keep the invoice in Planhat with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the invoice in Planhat |
| **Delete** | Delete the synced invoices from Planhat |
### Syncing licenses
Licenses represent your customers' subcriptions to your service and is the base for MRR or ARR calculations and most revenue reports.
#### Record matching
To match rows from your model to license in Planhat, you need to select a model column and corresponding Planhat field. You can match on any of the following Planhat fields:
- **ID**
- **Source ID**
- **External ID**
In **Insert** mode, Planhat automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync license fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom license fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available on **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
| Behavior | Description |
| -------------- | ------------------------------------------------------------ |
| **Do nothing** | Keep the license in Planhat with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the license in Planhat |
| **Delete** | Delete the synced licenses from Planhat |
### Syncing metrics
Syncing metrics lets you track dimensional data in Planhat.
While [user activity](#syncing-user-activites) says a lot about user engagement, it doesn't always reflect the value created.
Dimension data is a set of model level metrics (Company by default) to understand how well your customers are doing, and the value they get out of your service.
#### Field mapping
Hightouch lets you sync event properties via field mapping. For more information on event properties you can sync into, visit [Planhat's docs](https://docs.planhat.com/#bulkinsert_metrics).
### Syncing user activities
Syncing user activites lets you track endusers's actions in Planhat.
Keeping track of your endusers activity is an important part of the Customer Success effort.
#### Field mapping
Hightouch lets you sync event properties via field mapping. For more information on event properties you can sync into, visit [Planhat's docs](https://docs.planhat.com/#activity_bulk).
## Tips and troubleshooting
### Failed user activities
When receiving [user activities](#syncing-user-activities), Planhat's API always returns an `OK` successful response even if the user activity payload contains invalid data.
You can find these failed user events in Planhat in **Customer Intelligence > Trends & Analytics > User Activities** by clicking the **Show failed activities** icon.
### Common errors
### Live debugger
### Sync alerts
---
## Podscribe
**URL:** https://hightouch.com/docs/destinations/podscribe
**Description:** Simple, powerful podcast and audio ad analytics + tools that help everyone make better decisions.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-----------|-----------------------------------------------|----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
**Events** | Sync data from any source to Podscribe events | Insert | [Events docs](https://help.podscribe.com/sending-podscribe-conversion-events-/g4g8w3hppouCfkXZdcx6FV/sending-events-server-to-server-s2s-in-batch-or-as-an-img-tag/67UdckfSLHXAPtUAfWDyud)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Podscribe
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Podscribe** and click **Continue**. You can then authenticate Hightouch to **Podscribe**.
Enter the following fields into Hightouch:
- **Advertiser ID**
- **User ID**
## Sync configuration
Once you've set up your Podscribe destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Podscribe destination you want to sync to.
### Syncing events
Sync data from any source to Podscribe events.
#### Record matching
You can match rows from your model to events in Podscribe on any column in your model and any field in Podscribe. Ensure the data types of the model column and Podscribe field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Poplar
**URL:** https://hightouch.com/docs/destinations/poplar
**Description:** Keep your Poplar audiences and Do Not Mail list up-to-date and trigger campaigns with data from your warehouse
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
----------------------|-------------------------------------------------------|---------------------
**Segment** | Add members to existing Poplar segments | Add
**Campaign Trigger** | Trigger mail campaigns with context from your records | Add
**Suppression Lists** | Add members to Poplar's Do Not Mail list | Add
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Poplar
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination**
button. Select **Poplar** and click **Continue**. You can then authenticate Hightouch to **Poplar** by entering a Poplar **API key**.
To access your API keys, navigate to **Integrations** > **API** in Poplar. If you want to create a new one, click **Generate Access Token**.
We recommend using a **Production** mode API key. If you use a **Test** mode API key, you can only use the Campaign Trigger sync type due to Poplar API limitations. To learn more about Poplar API keys, check out [Poplar's docs](https://docs.heypoplar.com/article/246-test-production-tokens).
## Sync configuration
Once you've set up your Poplar destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and
the Poplar destination you want to sync to.
### Segment
Hightouch supports adding customer records based on rows from your model to any of your Poplar audiences.
#### Record matching
You can match rows from your model to customer records in Poplar on the fields:
- **Identifier**
- **Email**
- **Hashed Email**
- **Address**
### Campaign trigger
Hightouch lets you use records from your models to trigger mail campaigns. This sync triggers a campaign for your Poplar customers based on the recipient's information.
Once configured, your Hightouch sync triggers your campaign once rows appear in your model. Hightouch triggers your campaign for every row in your model the first time you run the sync, whenever you do a full resync, and for every new row added to your query results in subsequent syncs.
#### Creative selection
You can optionally provide a Poplar **creative ID** for the campaign you want to trigger. If you don't enter a creative ID, Poplar uses the default creative. If you haven't sent a default creative, Poplar alternates between all active creatives.
#### Field mapping
You can sync columns from your source to the following native columns in a Poplar campaign trigger:
- **Email**
- **Company**
- **First Name**
- **Last Name**
- **Address 1**
- **Address 2**
- **City**
- **State**
- **Postal Code**
- **Identifier**
- **Address**
You can also map model columns to use in [Poplar merge tags](https://docs.heypoplar.com/article/203-merge-tags).
This destination doesn't check for duplicates. Hightouch recommends that you set [suppression control](https://docs.heypoplar.com/article/204-suppression-settings) in Poplar to prevent duplicate campaign messages in case your model has duplicates.
#### Testing
You can test your campaign trigger sync by following these steps:
1. Use a **Test** mode API key within your destination configuration.
2. Set up your campaign trigger sync and run it on Hightouch.
3. Download mailing data by navigating to **Integrations** > **Mailing Data** in Poplar.
4. Verify that the trigger was set off by comparing the synced records with mailing data.
If you need to make corrections and thus want to re-run the sync, click **Resync full query** next to the **Run** button.
### Suppression lists
Hightouch supports syncing rows from your model to customer records in your Poplar **Do Not Mail** list. Members of this list are automatically suppressed across all campaigns.
#### Field mapping
You can add customer records to the Poplar' **Do Not Mail** using these fields:
- **Identifier**
- **Email**
- **Hashed Email**
- **Address**
You can select one or more of the identifiers for a single record.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## PostgreSQL
**URL:** https://hightouch.com/docs/destinations/postgres
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
The minimum supported PostgreSQL version is `9.5` since Hightouch relies on [`ON CONFLICT`](https://www.postgresql.org/docs/current/sql-insert.html) for updating rows.
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|Any data set|Sync data from any source to a PostgreSQL table|Upsert, Insert|
## Connect to PostgreSQL
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **PostgreSQL** and click **Continue**. If you're not using a [tunnel](#tunneling), you can then authenticate Hightouch to PostgreSQL by entering the following fields:
- **Host**: The hostname or IP address of your PostgreSQL server. You don't need to include the `https://`.
- **Port**: The port number of your PostgreSQL server. The default port number is 5432, but yours may be different.
- **Database**: This specifies the database to use when Hightouch executes queries in PostgreSQL. This is different than the host, but your host address might contain your database name.
- **User**: The user that has access through Hightouch to the database and tables you want to sync to. It's best to [create a new user](https://www.postgresql.org/docs/8.0/sql-createuser.html) specifically for Hightouch access. _Do not use the root user._
- **Password**: The password for the user specified above.
To ensure your [credentials](#required-permissions) are correct, click **Test connection**. This confirms if Hightouch can connect to your database by running a basic `SELECT` query.
### Required permissions
The user you use to authenticate must have the following permissions:
- Add, update, and delete ([if applicable](#delete-behavior)) rows in your sync's table.
- View the following tables which are used for gathering metadata to set up the sync:
- `INFORMATION_SCHEMA.COLUMNS`
- `INFORMATION_SCHEMA.TABLE_CONSTRAINTS`
- `INFORMATION_SCHEMA.CONSTRAINT_COLUMN_USAGE`
### SSH tunneling
Hightouch can connect directly to PostgreSQL over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your PostgreSQL instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
## Sync configuration
Once you've set up your PostgreSQL destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the PostgreSQL destination you want to sync to.
Hightouch supports **Upsert** mode, with the option to [delete removed rows](#delete-behavior), and **Insert** mode using the [`COPY FROM STDIN`](https://www.postgresql.org/docs/current/sql-copy.html)statement. This statement loads queried rows into your table faster than upsert mode. However, using this statement, PostgreSQL rejects the entire batch if any row contains a primary key or unique value that already exists.
Use insert mode if you are _only_ inserting rows and want to sync your data faster.
### Record matching
Hightouch requires you select a unique identifier in the table you are syncing to. The model column you select must match a `UNIQUE`, `PRIMARY KEY` column within PostgreSQL. If a field that satisfies these conditions still doesn't appear in the dropdown menu, ensure that you assigned the [required permissions](#required-permissions) to the Postgres user.
You can see columns that fit this criteria as available options in record matching section.
If there are no fields in the dropdown, you must add a unique type column to your PostgreSQL table. Then, click the refresh icon to access the newly created column.
### Column types
Hightouch works out of the box with all standard column types, including:
- `BIGINT`
- `INT`
- `TEXT`
- `VARCHAR`
- `TIMESTAMPTZ`
- `BOOLEAN`
- `DECIMAL`
If you see type errors, it may be because your model is producing the wrong format. If so, use [typecasting](/models/data-types-casting) to resolve the issue.
#### Array support
Hightouch supports arrays of standard JSON objects (JSON or JSONB) but doesn't support arrays of [`USER-DEFINED` type JSON objects](https://www.postgresql.org/docs/current/xtypes.html).
Because **insert** mode uses the `COPY FROM STDIN` statement, it doesn't support the array of JSONB column type (`jsonb[]`). Instead, use a JSONB column type, which natively supports arrays.
### Field mapping
You can sync columns from your model to PostgreSQL fields.
### Batch size and sleep interval
You can tune the number of rows Hightouch upserts or inserts per query based on your needs and database threshold. The default is 1000 rows per batch. You can also optionally enter a custom sleep interval in milliseconds (ms) between batches. This default is 100 ms.
If you want to increase the sync's speed, you can increase the batch size and lower the sleep interval. Keep in mind that PostgreSQL fails the entire batch of rows if it detects any erroneous row. If you suspect that you will have many bad rows, don't use a high batch size. To avoid locks, ensure you account for your database's capacity when increasing the batch size.
### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | --------------------------------------- |
| **Do nothing** | Keep the row in PostgreSQL |
| **Delete row** | Remove the row from PostgreSQL entirely |
## Tips and troubleshooting
### Common errors
#### Canceling statement due to statement timeout
This error occurs when the required execution time of your PostgreSQL query exceeds the timeout limit for the database.
To address the error, increase the timeout by executing the following query:
```json
set statement_timeout = '300 s'; -- 300 seconds, 5 minutes
```
Be sure to adjust the time to be as long as it takes for your query to execute.
### Live debugger
### Sync alerts
---
## PostHog
**URL:** https://hightouch.com/docs/destinations/posthog
**Description:** Push user behavior data from your warehouse into PostHog
**Section:** Destinations
## Setup
After selecting PostHog from our destinations catalog, you will be required to enter your PostHog [_Project API Key_](https://posthog.com/docs/api/post-only-endpoints) and _Public URL_.
- You can locate your Project API key within your PostHog account by navigating to _your PostHog project_ > _Project Settings_ > _Project API key_.
- If you are using the cloud-hosted version of PostHog, your Public URL will be either:
- For US: [https://us.i.posthog.com/batch/](https://us.i.posthog.com/batch/)
- For EMEA: [https://eu.i.posthog.com/batch/](https://eu.i.posthog.com/batch/)
- If you are using a self-hosted version of PostHog, replace [https://app.posthog.com/](https://app.posthog.com/) with the URL of your instance.
Hightouch supports syncing both `Objects` and `Events` to PostHog.
## Syncing
Hightouch supports syncing to the following PostHog objects:
- `People` (`Persons`)
### Sync modes
Hightouch supports the following sync modes:
- **Upsert** - pushes new `Person` objects to PostHog and updates properties that change in your warehouse.
### Record matching
`Person` records can be matched from your source to your PostHog workspace by Distinct ID (`distinct_id`), which is a unique identifier
in PostHog for each `Person`.
### Field mapping
You may also sync columns from your source to PostHog's `Person` properties. PostHog doesn't provide information on properties set for `People` objects,
so you will need to type the exact name of the property you would like to map to in PostHog. These property names must be spelled correctly and are case-sensitive.
## Sending events to PostHog
The PostHog integration also allows you to send events to PostHog with custom properties.
The four elements of any PostHog event are the _Event Name_ (**required**, which can be a single string used to name all events, or pulled from your warehouse per event), a _Timestamp_ (**optional**, in ISO 1806 format, or a Unix timestamp in **milliseconds**), a **Distinct ID** (**required**, the unique ID of the user corresponding to the event), and event properties.
You can also map properties that will become event `properties` in PostHog, just as you would for object properties.
---
## Qualified
**URL:** https://hightouch.com/docs/destinations/qualified
**Description:** Convert inbound leads at scale
**Section:** Destinations
## Overview
Qualified is a conversational sales and marketing platform that helps revenue teams identify and engage with website visitors. With the Qualified destination, you can sync lead and company data from your data warehouse to Qualified, ensuring your sales team has the most up-to-date information when engaging with prospects.
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| --------- | ----------- | -------------------- | ------------- |
| **Leads** | Sync data from any source to leads in Qualified | Upsert | [Qualified API docs](https://www.qualified.com/api) |
| **Companies** | Sync data from any source to companies in Qualified | Upsert | [Qualified API docs](https://www.qualified.com/api) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Qualified
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Qualified** and click **Continue**.
You will be required to enter your Qualified API key. Generate a Qualified API key with access to the Enterprise bulk endpoints in your Qualified account. The API key must have permissions to access the bulk API endpoints for leads and companies.
The API key must have `Enterprise bulk access` to use the Qualified destination. Confirm that your API key has the necessary permissions before connecting.
Paste the API key exactly as provided; Hightouch passes it as a Bearer token in API requests.
## Sync configuration
Once you've set up your Qualified destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Qualified destination you want to sync to.
### Object selection
Choose which Qualified object to sync to:
- **Leads**: Sync lead data to Qualified
- **Companies**: Sync company data to Qualified
### Record matching
Records are matched from your model to Qualified based on the object type:
- **Leads**: Match by **Email**. The email address is the primary identifier for leads in Qualified. Qualified upserts leads by email address.
- **Companies**: Match by **Domain**. The company domain is the primary identifier for companies in Qualified. Qualified upserts companies by domain.
The record should be matched using a unique identifier. For leads, ensure the email column contains unique values. For companies, ensure the domain column contains unique values. Duplicate identifiers may cause sync errors or unexpected behavior.
Leads synced to Qualified will not appear in the Qualified interface until they have activity associated with them (such as website visits or engagement). The data is successfully synced, but Qualified requires activity before displaying leads in the platform.
### Field mapping
You can sync columns from your source to Qualified's default and custom fields. Map Hightouch columns to Qualified fields in the field mapping section.
When syncing **Leads**, you can map both:
- **Lead fields**: Fields that belong to the lead record
- **Company fields**: Fields that belong to the associated company record
When syncing **Companies**, you can map:
- **Company fields**: Fields that belong to the company record
Qualified requires object fields to be created in advance. After adding new fields in Qualified, click "Refresh fields" in the mapper to pull the latest definitions. Hightouch cannot provision fields automatically.
## Tips and troubleshooting
### Sync performance
Qualified uses bulk API endpoints to efficiently sync large volumes of data. The destination automatically:
- Groups rows into bulk jobs (up to 500 rows per job)
- Manages job polling and status checks
- Handles rate limiting with automatic retries
- Processes job results and reports rejected rows
If you experience rate limiting, contact Qualified support to understand your account's rate limits.
### Common errors
#### API key authentication errors
If you receive authentication errors, verify that:
- Your API key is correct and has not expired
- The API key has Enterprise bulk access permissions
- The API key has access to both lead and company endpoints
#### Rate limiting
If you encounter rate limit errors (429 or 503 status codes), the destination will automatically retry with exponential backoff. Contact Qualified support to understand your account's specific rate limits.
#### Field mapping errors
If you receive errors about invalid fields:
- Ensure custom fields exist in Qualified before mapping to them
- Verify field names match exactly (field names are case-sensitive)
- Check that the data types in your model match the expected types in Qualified
### Live debugger
### Sync alerts
---
## Qualtrics
**URL:** https://hightouch.com/docs/destinations/qualtrics
**Description:** Improve user experiences with rich data by syncing warehouse data to Qualtrics
**Section:** Destinations
## Overview
This integration allows you to import contacts to Qualtrics XM Directory using
endpoint for your automation. After the sync, Qualtrics will asynchronously
process the files.
## Setup
After selecting Qualtrics from our Destinations catalog, you will be asked to
provide your Qualtrics data center ID and API token.
Both of these can be found in the Qualtrics console under Account Settings >
Qualtrics IDs.
## Sending contacts
Hightouch supports syncing contacts to Qualtrics XM Directory. Rows will be
synced when they are added or changed. Rows that are removed from your model
will be ignored.
Hightouch sends your contacts to Qualtrics in batches of 10,000 rows. Each batch
is posted with a unique filename, based on the current time.
## Sync status
Because Qualtrics asynchronously processes the files on the automation's
schedule, Hightouch only shows whether the files were posted to Qualtrics
successfully. Hightouch doesn't wait for Qualtrics to process the files.
We recommend that you enable email notifications in Qualtrics so that you know
when Qualtrics is done processing the files.
---
## QuickBooks
**URL:** https://hightouch.com/docs/destinations/quickbooks
**Description:** Gain financial visibility and empower your accounting team by keeping your financials up to date
**Section:** Destinations
Hightouch uses minor version `65` of the **QuickBooks Online Accounting API**.
See QuickBooks's
[docs](https://developer.intuit.com/app/developer/qbo/docs/learn/explore-the-quickbooks-online-api/minor-versions)
to learn more about minor versions.
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ------------------------ | -------------------------------------------------------------------- | ---------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| **Account** | Sync data from any source to QuickBooks Accounts | Upsert, Update, Insert | [Account docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/account) |
| **Bill** | Sync data from any source to QuickBooks Bills | Upsert, Update, Insert | [Bill docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/bill) |
| **Class** | Sync data from any source to QuickBooks Classes | Upsert, Update, Insert | [Class docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/class) |
| **Customer** | Sync data from any source to Quickbook Customers | Upsert, Update, Insert | [Customer docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/customer) |
| **Deposit** | Sync data from any source to QuickBooks Deposits | Upsert, Update, Insert | [Deposit docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/deposit) |
| **Invoice** | Sync data from any source to QuickBooks Invoices | Upsert, Update, Insert | [Invoice docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/invoice) |
| **JournalEntry** | Sync data from any source to QuickBooks Journal Entries | Upsert, Update, Insert | [JournalEntry docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/journalentry) |
| **Payment** | Sync data from any source to QuickBooks Payments | Upsert, Update, Insert | [Payment docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/payment) |
| **PaymentMethod** | Sync data from any source to QuickBooks Payment Methods | Upsert, Update, Insert | [PaymentMethod docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/paymentmethod) |
| **RecurringTransaction** | Sync data from any source to QuickBooks RecurringTransaction objects | Insert | [RecurringTransaction docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/recurringtransaction) |
| **TimeActivits** | Sync data from any source to QuickBooks TimeActivity objects | Update, Insert | [TimeActivity docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/timeactivity) |
| **Vendor** | Sync data from any source to QuickBooks Vendors | Upsert, Update, Insert | [Vendor docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/vendor) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to QuickBooks
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **QuickBooks** and click **Continue**. You can then authenticate Hightouch to **QuickBooks** via OAuth.
You can choose to sync your data to a Sandbox or Production Quickbooks environment. Select your preference, click **Log in to QuickBooks**, and log into your QuickBooks account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your QuickBooks destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the QuickBooks destination you want to sync to.
### Syncing accounts
Sync data from any source to accounts in QuickBooks.
#### Record matching
You can match rows from your model to accounts in Quickbooks on any column in your model and any account field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/account) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **Name**: User recognizable name for the Account. 100 character max.
Ensure the data types of your model columns match the data types of the fields you want to sync to. Refer to QuickBooks [account object docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/account) for more information.
### Syncing bills
Sync data from any source to bills in QuickBooks.
#### Record matching
You can match rows from your model to bills in Quickbooks on any column in your model and any bill field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/customer) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **VendorRef**: Reference to the vendor for this transaction. Query the Vendor name list resource to determine the appropriate Vendor object for this reference. Use `Vendor.Id` and `Vendor.Name` from that object for `VendorRef.value` and `VendorRef.name`, respectively.
- **Line**: Individual line items of a transaction. Valid Line types include: `ItemBasedExpenseLine` and `AccountBasedExpenseLine`
Ensure the data types of your model columns match the data types of the fields you want to sync to. Refer to QuickBooks [bill object docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/bill) for more information.
### Syncing classes
Sync data from any source to classes in QuickBooks.
#### Record matching
You can match rows from your model to classes in Quickbooks on any column in your model and any class field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/class) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **Name**: User recognizable name for the Class. 100 character max.
Ensure the data types of your model columns match the data types of the fields you want to sync to. Refer to QuickBooks [class object docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/class) for more information.
### Syncing customers
Sync data from any source to customers in QuickBooks.
#### Record matching
You can match rows from your model to customers in Quickbooks on any column in your model and any customer field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can sync columns from your model to QuickBooks default and custom fields.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/customer) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
### Syncing deposits
Sync data from any source to deposits in QuickBooks.
#### Record matching
You can match rows from your model to deposits in Quickbooks on any column in your model and any deposit field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/deposit) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **DepositToAccountRef**: Identifies the account to be used for this deposit.
- **Line**: Atleast 1 line item must be included to create a deposit.
Ensure the data types of your model columns match the data types of the fields you want to sync to. Refer to QuickBooks [deposit object docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/deposit) for more information.
### Syncing invoices
Sync data from any source to invoices in QuickBooks.
#### Record matching
You can match rows from your model to invoices in Quickbooks on any column in your model and any invoice field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/invoice) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **DepositToAccountRef**: Identifies the account to be used for this deposit.
- **Line**: Atleast 1 line item must be included to create a deposit.
Ensure the data types of your model columns match the data types of the fields you want to sync to. Refer to QuickBooks [invoice object docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/invoice) for more information.
### Syncing journal entries
Sync data from any source to journal entries in QuickBooks.
#### Record matching
You can match rows from your model to jounal entries in Quickbooks on any column in your model and any journal entry field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/journalentry) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **Line**: Individual line items of a transaction. There must be at least one pair of Journal Entry Line elements, representing a debit and a credit, called distribution lines.
Ensure the data types of your model columns match the data types of the fields you want to sync to. Refer to QuickBooks [journal entry object docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/journalentry) for more information.
### Syncing payments
Sync data from any source to payments in QuickBooks.
#### Record matching
You can match rows from your model to payments in Quickbooks on any column in your model and any payment field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/payment) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **TotalAmt**: Indicates the total amount of the transaction. This includes the total of all the charges, allowances, and taxes.
- **CustomerRef**: Reference to a customer or job.
Ensure the data types of your model columns match the data types of the fields you want to sync to. Refer to QuickBooks [payment object docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/payment) for more information.
### Syncing payment methods
Sync data from any source to payment methods in QuickBooks.
#### Record matching
You can match rows from your model to payment methods in Quickbooks on any column in your model and any payment method field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/paymentmethod) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **Name**: User recognizable name for the payment method. 31 character max.
Ensure the data types of your model columns match the data types of the fields you want to sync to. Refer to QuickBooks [payment method object docs](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/paymentmethod) for more information.
### Syncing RecurringTransaction objects
Hightouch supports syncing RecurringTransaction objects to the **Bill** entity. This sets the [`type`](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/recurringtransaction) field on the ecurringTransaction objects.
If you're interested in syncing RecurringTransations to another entity, please .
#### Record matching
Since you can only insert RecurringTransaction objects, QuickBooks automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/customer) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the **RecurringInfo** field, so you must map it to complete your configuration. Ensure the data types of your model columns match the data types of the fields you want to sync to.
### Syncing TimeActivity objects
Sync data from any source to TimeActivity objects in QuickBooks.
#### Record matching
You can match rows from your model to TimeActivity objects in Quickbooks on any column in your model and any TimeActivity field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can map data from any of your model columns to fields in QuickBooks.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/customer) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
QuickBooks also requires the following fields, so you must map them to complete your configuration:
- **NameOf** - Value can be `Vendor` or `Employee`.
- **VendorRef** - Required if **NameOf** is set to `Vendor`.
- **EmployeeRef** - Required if **NameOf** is set to `Employee`.
Ensure the data types of your model columns match the data types of the fields you want to sync to.
### Syncing vendors
Sync data from any source to vendors in QuickBooks.
#### Record matching
You can match rows from your model to vendors in Quickbooks on any column in your model and any vendor field in Quickbooks. Ensure the data types of the model column and Quickbooks field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, QuickBooks automatically generates an identifier for every
new record synced, so there is no need to match an existing record.
#### Field mapping
You can sync columns from your model to QuickBooks default and custom fields.
QuickBooks requires the [`SyncToken` attribute](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/customer) when performing updates.
The `SyncToken` is the version number of the object. It's used to lock an object for use by one app at a time. As soon as an application modifies an object, its SyncToken is incremented.
By default, Hightouch pulls the latest `SyncToken` from QuickBooks before perfoming the update. You can override this behavior by providing your own `SyncToken` value via field mapping. Keep in mind that attempts to modify an object specifying an older `SyncToken` fail.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## RabbitMQ
**URL:** https://hightouch.com/docs/destinations/rabbitmq
**Description:** Connect to your internal microservices via a message queue
**Section:** Destinations
## Overview
Hightouch integrates with RabbitMQ so you can send data from your warehouse as messages to a RabbitMQ broker without writing and maintaining code, letting you build a custom connector to your internal systems.
Hightouch supports all managed RabbitMQ brokers (Amazon MQ, CloudAMQP, etc.) and can also connect to self-hosted instances.
## Getting started
### Connect to your RabbitMQ server
For Hightouch to connect to your RabbitMQ server, you need to enter the connection parameters of your [RabbitMQ URI](https://www.rabbitmq.com/uri-spec.html). These parameters include the **protocol** used, server connection details (**host address** and **port**), authentication credentials (**username** and **password**) and **vhost**.
You also need to enter your [Management Portal URL](https://www.rabbitmq.com/management.html) for Hightouch to query the queues in your RabbitMQ server,.
- **Protocol**: Select the protocol used for your RabbitMQ server. This is either `amqp` or `amqps`.
- **Host**: Enter your host address to establish the underlying TCP/IP connection to the server.
- **Port**: Enter your port number to establish the underlying TCP/IP connection to the server.
- **VHost**: Enter you vhost. The default value is `/`.
- **Management Portal**: Enter your management portal URL. This can usually be found on the details page for your hosted RabbitMQ solution.
- **Username and Password**: Enter your RabbitMQ server username and password.
## Syncing data
Once you've connected your RabbitMQ server to Hightouch, you can configure a sync that send messages whenever rows are added, changed, or removed in your model.
### Configure your events trigger
Hightouch monitors your data model for added, changed, and removed rows. In this step, you specify which of these events should trigger message publishing.
### Choose your queue
In this step, you choose which queues to publish the messages to. Hightouch allows you to sync to existing queues that are already in your RabbitMQ.
Suppose you want to sync to multiple existing queues but don't want to create a new sync for every queue. As long as your model has a column associated to queue names in your RabbitMQ server, Hightouch can sync to multiple RabbitMQ queues in just one sync.
To enable this feature, toggle **USE COLUMN**, and select a column in your model containing the queue name rows.
When syncing to multiple queues, if a queue name in the selected column of
your model doesn't exist in RabbitMQ, then the sync for that message will
fail.
### Customize your message
In this step, you tell Hightouch how to build the JSON message data object using data from your model.
This destination offers three methods of composing a JSON object:
- [Using a JSON editor](#use-json-editor)
- [Using one column from a model](#use-one-column-from-model)
- [Using multiple columns from a model](#use-multiple-columns-from-model)
#### Use JSON editor
With the JSON editor, you can compose any JSON object using the [Liquid template language](https://shopify.github.io/liquid/). This is particularly useful for complex message data bodies containing nested objects and arrays, which can sometimes be difficult to model entirely in SQL.
Suppose your data model looks like this:
| full_name | age | email_address | phone_number |
| --------- | --- | ------------------ | ------------ |
| John Doe | 30 | `john@example.com` | +14158675309 |
And you want your message data like this:
```json
{
"name": "John Doe",
"age": 30,
"contact_info": [
{
"type": "email",
"value": "john@example.com"
},
{
"type": "phone",
"value": "+14158675309"
}
]
}
```
Your Liquid template should look like this:
```txt
{
"name": "{{row.full_name}}",
"age": {{row.age}},
"contact_info": [
{
"type": "email",
"value": "{{row.email_address}}"
},
{
"type": "phone",
"value": "{{row.phone_number}}"
}
]
}
```
This makes it so you can reference any column using the syntax `{{row.column_name}}`. You can also use advanced Liquid features to incorporate control flow and loops into your dynamic message data.
When injecting strings into your JSON object, be sure to surround the Liquid
tag in double quotes.
#### Use one column from model
If you're already storing JSON data in your source, or if you have the ability to construct a JSON object using SQL, you can select one column in your model that already contains the full message data.
This setting is commonly used when syncing web events that have already been collected and stored as JSON objects in your database.
#### Use multiple columns from model
For the simplest use cases, Hightouch can construct a JSON object with key/value pairs based on multiple columns in your model.
Suppose your model looks like this:
| email | first_name | last_name |
| ----------------------- | ---------- | --------- |
| `alice.doe@example.com` | Alice | Doe |
| `bob.doe@example.com` | Bob | Doe |
| `carol.doe@example.com` | Carol | Doe |
The field mapping in the screenshot above would generate the following message data for the first row:
```json
{
"customer_first_name": "Alice",
"customer_last_name": "Doe",
"customer_email": "alice.doe@example.com"
}
```
You can use the field mapper to rename fields. For example, `first_name` can
be mapped to `customer_first_name`.
### Configure initial sync behavior
In this step, you tell Hightouch how to handle rows present in your model results during the first sync run.
Certain workflows may require performing a backfill of all rows during the initial sync. For other use cases, you might only want to send messages in response to future data changes.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Rakuten
**URL:** https://hightouch.com/docs/destinations/rakuten
**Description:** Send Ad Conversion Transactions to the Rakuten Transactions API
**Section:** Destinations
## Overview
Rakuten Advertising is a global affiliate marketing platform that connects advertisers with publishers to drive conversions and track affiliate performance. With the Rakuten destination, you can sync transaction and conversion event data from your data warehouse to Rakuten's Advertiser Transaction API, enabling accurate affiliate attribution and commission tracking.
This destination supports comprehensive transaction tracking for multiple verticals including Retail, Travel, and Financial services, with both affiliate-attributed (click-based) and Linkless transaction types.
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| --------- | ----------- | -------------------- | ------------- |
| **Events** | Sync transaction/conversion events from any source to Rakuten | Insert | [Rakuten API docs](https://api.linksynergy.com/docs) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Rakuten
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Rakuten** and click **Continue**.
You need to provide the following credentials:
- **Client ID**: Your Rakuten API client ID
- **Client Secret**: Your Rakuten API client secret
- **Advertiser ID (MID)**: Your Rakuten Merchant ID (numeric, up to 5 digits)
### Obtain API credentials
Contact your Rakuten Advertising account representative to obtain API credentials. You'll need credentials with access to the Advertiser Transaction API endpoint.
The Advertiser ID must be numeric and no more than 5 characters. Verify your credentials by testing the connection before creating syncs.
## Sync configuration
Once you've set up your Rakuten destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Rakuten destination you want to sync to.
### Event timestamp
Select the column from your model that contains the transaction timestamp. This becomes the `transaction_datetime` in Rakuten and must be in a valid datetime format. The timestamp is automatically converted to ISO 8601 UTC format.
Transaction timestamps cannot be in the future. Ensure your transaction datetime values represent when the transaction occurred.
### Field mapping
Hightouch lets you sync transaction fields via [field mapping](/syncs/mapping-data). You can map data from any of your model columns to Rakuten's transaction fields.
#### Required fields
You must map the following fields to complete your configuration:
- **Order ID**: A unique identifier for the transaction (max 40 characters)
- **Currency**: ISO 3-character currency code (e.g., `USD`, `EUR`, `GBP`)
- **Items**: An array of line item objects (see Items array section below)
#### Tracking fields
These optional fields enable affiliate attribution:
- **Click ID (ranSiteID)**: The Rakuten tracking parameter captured from affiliate redirects. This is a 34-character identifier. For Linkless transactions, this field can be omitted.
- **Landing datetime**: The UTC timestamp when the user landed on your site from the affiliate link. If not provided, this automatically defaults to 5 minutes before the transaction timestamp to ensure API compliance.
For affiliate-attributed transactions, capture the `ranSiteID` parameter from the referring URL when users arrive from Rakuten affiliate links. This enables proper commission attribution.
#### Items array
Each transaction requires at least one item. Use the [inline mapper](/syncs/mapping-data#inline-mapping) to structure the items array. Each item in the array must include:
**Required item fields:**
- `sku`: Product identifier (max 40 characters)
- `product_name`: Product name (max 512 characters)
- `amount`: Item subtotal (post-discount, excluding tax)
- `quantity`: Number of units
**Optional item fields:**
- `order_item_id`: Unique identifier for the line item
- `brand`: Product brand name
- `category`: Product category path (e.g., `Electronics>Widgets`)
- `category_id`: Category identifier
- `coupon`: Item-level coupon code applied
- `discount_amount`: Item-level discount amount
- `discount_type`: Type of discount applied
- `is_clearance`: Whether the item is on clearance
- `is_sale`: Whether the item is on sale
- `marketplace_store`: Marketplace store name
- `product_id`: Additional product identifier
Use the inline mapper to transform your source data into the required array format. If your data has one row per order, you'll need to structure the items as a JSON array.
#### Optional transaction fields
Rakuten supports extensive optional fields organized by vertical:
**General/Retail fields:**
- `alternative_order_id`, `customer_id`, `customer_status`, `customer_rank`, `customer_segment`
- `coupon`, `credit_card_type`, `commission_reason`
- `discount_amount`, `discount_type`, `order_status`
- `purchase_site`, `ship_country`, `shipped_datetime`
- `store_id`, `store_category`
**Travel fields:**
- `check_in_datetime`, `check_out_datetime`
- `reservation_start_datetime`, `reservation_end_datetime`, `reservation_length_in_days`
- `cancel_deadline_datetime`, `consumed_datetime`
**Financial fields:**
- `autopay`, `home_ownership`
- `loan_amount_requested`, `loan_apr`, `loan_approval_datetime`
- `loan_offer_apr`, `loan_offer_badge`, `loan_offer_term`
- `loan_origination_fee`, `loan_model_number`, `loan_purpose_applicant`
- `loan_term_in_months`, `monthly_housing_payment`, `publisher_lead_id`
**Offline tracking (O2O) fields:**
- `o2o_bank_partner`, `o2o_store_id`, `o2o_store_zip`
### Sync behavior
Rakuten processes transactions individually via POST requests. The destination automatically:
- Formats datetime values to Rakuten's required ISO 8601 format (without milliseconds)
- Validates the items array contains at least one item
- Handles authentication token management and refresh
- Provides detailed error messages from Rakuten's validation
## Tips and troubleshooting
### Linkless transactions
For Linkless transactions (conversions without an affiliate click), you can omit the `click_id` field. The destination automatically sets a default `landing_datetime` value (5 minutes before the transaction) to ensure API compliance.
### Date format requirements
All datetime fields must be convertible to ISO 8601 UTC format. The destination handles the conversion automatically, but ensure your source data contains valid datetime values.
The `landing_datetime` must be before the `transaction_datetime`. If you provide a landing datetime that occurs after the transaction, the Rakuten API will reject the request.
### Field length limits
Rakuten enforces strict field length limits:
- `order_id`: Maximum 40 characters
- `sku`: Maximum 40 characters
- `product_name`: Maximum 512 characters
Ensure your source data respects these limits to avoid validation errors.
### Common errors
#### Authentication errors
If you receive 401 or 403 authentication errors:
- Verify your Client ID and Client secret are correct
- Confirm the credentials have access to the Advertiser Transaction API
- Check that your Advertiser ID (MID) matches your account
#### Validation errors
If you receive 400 validation errors, check the error details for:
- Missing required fields (order_id, currency, items)
- Empty items array (at least one item is required)
- Invalid field formats (especially datetime fields)
- Field values exceeding length limits
#### Items array errors
If you receive errors about the items array:
- Ensure the items field is a valid array with at least one item
- Verify each item has all required fields: `sku`, `product_name`, `amount`, `quantity`
- Check that numeric fields (`amount`, `quantity`) contain valid numbers
### Live debugger
### Sync alerts
---
## Reddit Ads
**URL:** https://hightouch.com/docs/destinations/reddit-ads
**Description:** Run more targeted ads by syncing conversion events and audiences to Reddit
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
| ---------- | -----------------------------------------------------| ---------------------- | ------------------------------------------------------------------------------------------------------- |
| **Conversion events** | Sync data from any source to Reddit as conversion events | Insert | [Conversions](https://ads-api.reddit.com/docs/v2/#tag/Conversions)|
| **Audiences** | Sync data from any source to Reddit as a custom audience list | Upsert | [Audiences](https://ads-api.reddit.com/docs/v3/operations/Create%20Custom%20Audience)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Reddit
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Reddit Ads** and click **Continue**.
Authorize Hightouch to access Reddit by logging into your account and agreeing to the app's usage. To successfully sync Custom Audiences, Hightouch requires the `adsedit` and `adsread` scope.
## Sync setup
### Account ID
An Account ID is required to sync Conversion events and Audiences.
By default, Hightouch will automatically use the Reddit Ads account owned by the user authenticated. If you'd like to use a different account ID, you can navigate to the respective dashboard and grab the ID from the URL.
## Syncing conversion events
### Prerequisites
- **Allowlisted by Reddit:** Hightouch uses Reddit's Conversions API, which is in beta.
To send conversion events, your Reddit Ads account needs to by white listed by Reddit. Reach out to your Reddit account team to request access for your account to use the Conversions API.
- **Events within 7 days:** If you are planning to set the event timestamp yourself, the event must occur within the last 7 days otherwise the conversion event will not go through.
### Sync setup
#### Mappings
The view the schema and the field descriptions of the conversion endpoints visit the [Reddit docs](https://ads-api.reddit.com/docs/#tag/Conversions/paths/~1api~1v2.0~1conversions~1events~1{account_id}/post).
## Syncing custom audiences
Hightouch will sync data to Reddit ads as a Custom Audience with the `CUSTOMER_LIST` type.
You can choose to create a new Custom Audience through Hightouch, or sync to an existing audience.
### Sync setup
#### Mappings
Reddit accepts the following user fields: `EMAIL_SHA256` and `MAID_SHA256`. At least one identifier is required to successfully sync to Reddit ads.
#### Handling PII and hashing
The Reddit Custom Audiences API will **only** accept hashed values. For audience syncs, by default, Hightouch automatically hashes first-party user data fields before sending requests to Reddit.
If the data in your model is already hashed, you can disable PII hashing to successfully send data to Reddit.
#### Delete behavior
When records leave your query result, they are removed from your audience by default. If you would not like to delete any records from your audience, you can choose to do nothing when records leave your query result.
## Tips and troubleshooting
### Matched users count
Below only applies to the **Audiences** sync type.
You may observe match rates in Reddit Ads that are greater than 100%. Reddit Ads returns the count of matched users as a range. Reddit Ads does not reliably deduplicate users and each of the IDs can be counted separately, meaning the calculated audience size can exceed the actual number of unique individuals. The use of an audience size range is meant to help advertisers understand whether an audience segment is broad enough for effective ad delivery, without misrepresenting the precision or uniqueness of the reach.
### Don't see your data in Reddit?
Make sure you've configured your sync to use the correct Reddit environment. You might've accidentally pointed to a different environment. If that doesn't solve your issue, . We're happy to help.
### Live debugger
### Sync alerts
---
## Redis
**URL:** https://hightouch.com/docs/destinations/redis
**Description:** Redis is an in-memory data structure store, used as a distributed, in-memory key–value database, cache and message broker, with optional durability. Redis supports different kinds of abstract data structures, such as strings, lists, maps, sets, etc.
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes | API Reference|
|--|--|--|--|
|**Redis Cache**|Sync data from any source to a Redis cache database|Upsert, Update|[Redis Cache](https://redis.io/docs/manual/client-side-caching/)|
## Connect to Redis
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Redis** and click **Continue**.
### Connect to a Redis instance
Authenticate Hightouch to Redis with your hostname and port. If your Redis instance has authentication credentials, you can also input a username and password.
#### SSH tunneling
Hightouch can connect directly to Redis over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your Redis instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
### Connect to a Redis cluster
Hightouch needs permission to access all cluster nodes of your Redis cluster. If your Redis cluster exists on a publicly available network, only the primary cluster node URL is required. Provide the cluster node URLs in the specified `redis://:` format.
#### Node address map
A node address map is a mapping between the addresses in the cluster and the addresses the client should connect to. This is useful when connecting to a cluster running on a different network than the client. For more information on node address maps specifically, refer to [`node-redis`' documentation](https://github.com/redis/node-redis/blob/master/docs/clustering.md#node-address-map).
To get the addresses of your Redis cluster nodes, run `CLUSTER NODES` in your cluster. Refer to the [Redis documentation](https://redis.io/docs/latest/commands/cluster-nodes/) for more details.
A common use case of node address mapping is connecting to AWS ElastiCache. To access ElastiCache resources through Hightouch, you will need to follow the instructions defined [in this AWS tutorial](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/accessing-elasticache.html) to create a NAT instance in the same VPC as your ElastiCache instance, but on a public subnet. This NAT instance will need to forward a port to each Redis cluster node. Ensure that you can successfully connect to your Redis cluster through the NAT instance.
When configuring your connection from Hightouch, construct the Redis cluster node URLs by using the publicly accessible IP address of your NAT instance with the corresponding ports. Then, map each cluster node address to the corresponding public address in the node address mapping section.
#### Connect over a private network
If you have a Redis cluster running on a private network with no publicly accessible endpoints, you will need to create a [SSH tunnel](/security/ssh-tunneling) to _each_ cluster node.
To get the addresses of your Redis cluster nodes, run `CLUSTER NODES` in your cluster. Refer to the [Redis documentation](https://redis.io/docs/latest/commands/cluster-nodes/) for more details.
Provide the URL for each cluster node, and map it to its corresponding SSH tunnel.
## Sync configuration
Once you've set up your Redis destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Redis destination you want to sync to.
### Syncing to a cache database
Hightouch supports syncing [**keys**](#syncing-keys) and [**members**](#syncing-members) to Redis.
### Syncing keys
Keys identify, store, and retrieve data. Each key in Redis maps to a corresponding value.
#### Supported commands
|Command |Description|Supported Sync Modes | API Reference|
|---------|-----------|---------------------|--------------|
|**`SET`**|Set key to hold the string value.|Upsert, Update|[SET Command](https://redis.io/commands/set/)|
|**`HSET`**|Set specified fields to their respective values in the hash stored at key.|Upsert, Update|[HSET Command](https://redis.io/commands/hset/)|
|**`SADD`**|Add specified members to the set stored at key.|Upsert|[SADD Command](https://redis.io/commands/sadd/)|
|**`GEOADD`**|Add specified geospatial items (longitude, latitude, name) to the specified key.|Upsert|[GEOADD Command](https://redis.io/commands/geoadd/)|
#### Record matching
To match rows from your model to records in Redis, you need to select the model column that contains values that match the **Key** field in Redis.
#### Field mapping
You can sync columns from your source to custom fields in Redis.
#### Time to live (TTL)
You can set an expiration for the keys that you are syncing to Redis by setting a [TTL value](https://redis.io/commands/ttl/). This value is the amount of time in seconds that the keys will exist in Redis before they're automatically deleted.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the key in Redis with its fields untouched |
| **Clear** | Keep the key in Redis, but set mapped fields to empty |
| **Delete** | Remove the key from Redis |
### Syncing members
Members are used for managing [sorted sets](https://redis.io/docs/data-types/sorted-sets/) in Redis.
#### Supported commands
|Command |Description|Supported Sync Modes | API Reference|
|---------|-----------|---------------------|--------------|
|**`ZADD`**|Add all specified members with the specified scores to the sorted set stored at key.|Upsert, Update|[ZADD Command](https://redis.io/commands/zadd/)|
|**`GEOADD`**|Add specified geospatial items (longitude, latitude, name) to the specified key.|Upsert|[GEOADD Command](https://redis.io/commands/geoadd/)|
#### Record matching
To match rows from your model to records in Redis, select the model column that contains the unique member values of each element of the sorted set in Redis.
#### Field mapping
The fields available for mapping depend on the [command](#supported-commands-1) that you've selected.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the member in Redis with its fields untouched |
| **Clear** | Keep the member in Redis, but set mapped fields to empty |
| **Delete** | Remove the member from the sorted set from Redis |
### Transactions
Transactions are disabled by default when connected to a Redis cluster.
You can toggle transactions for your Redis instance by configuring the options as shown.
## Tips and troubleshooting
### Common errors
#### Invalid argument type
This error can happen when syncing data types that aren't [supported by Redis](https://redis.io/docs/data-types/), such as `null` values. If your model columns contain `null` values, you can enable [Don't sync null values](/syncs/mapping-data#dont-sync-null-values) to avoid syncing them.
### Live debugger
### Sync alerts
---
## Reply.io
**URL:** https://hightouch.com/docs/destinations/replyio
**Description:** Supercharge your sales team by syncing your most up-to-date customer and lead data from your warehouse
**Section:** Destinations
## Setup
Upon selecting Reply.io as your destination you will be prompted to
input your Reply.io API key.
You can locate this API key within your Reply.io account by navigating to Settings > API Key:
## Syncing
Hightouch supports syncing the following Reply.io objects:
- `People`
### Sync modes
- **Upsert**: pushes new objects to Reply.io _and_ updates fields that change in your warehouse.
- **Update**: only updates particular fields on existing objects in Reply.io. It does _not_ add new objects.
### Record matching
Records can be matched from your source to your Reply.io workspace by the **contact's email** only.
### Field mapping
You may also sync columns from your source to Reply.io's object properties.
## Other considerations
As per their [documentation](https://apidocs.reply.io/#32e27eaa-b6b1-496e-8504-04b676aec3c1),
Reply.io's API is limited to _15,000 requests per user per month_.
This is important to keep in mind when syncing large amounts of data since **Upsert** mode
will make one API call per record synced and **Update** mode will make up to two API calls per record synced
(one to check that user exists and the other to update the user). Please plan your syncs accordingly.
[If you'd like to increase this rate limit, please contact Reply.io support.](https://support.reply.io/en/articles/978063-api-rate-limits).
---
## Resonate
**URL:** https://hightouch.com/docs/destinations/resonate
**Description:** Resonate tells you who your highest-value audiences are and what they’ll do next so you can act with precision, before the competition.
**Section:** Destinations
## Overview
Resonate is an audience intelligence platform that helps brands activate first-party data for targeting and match enrichment. With the Resonate destination, you can sync customer data from your data warehouse to Resonate via SFTP, enabling audience matching and activation.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------ | -------------------------------------------------------- | -------------------- |
| **Audience** | Sync customer records to Resonate for audience matching | All, Insert |
- **All**: Sends all records in your model every time the sync runs.
- **Insert**: Sends only new records added since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Resonate
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Resonate** and click **Continue**.
You will need the following credentials from Resonate:
- **Username**: Your Resonate SFTP username.
- **Private Key**: The SSH private key for SFTP authentication. Provide the full private key including the header and footer.
- **Private Key Passphrase** _(optional)_: The passphrase for your private key, if applicable.
Resonate uses SSH key-based authentication. Contact your Resonate account representative to obtain your SFTP credentials and private key.
Hightouch connects to Resonate's SFTP server at `sftp.resonate.com` on port 22. Files are automatically placed in the `data-append/inbound/` directory.
## Sync configuration
Once you've set up your Resonate destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button. Select the relevant model and the Resonate destination.
### Field mapping
Map columns from your model to Resonate's required fields. **Customer ID is required** for every sync. You must also include at least one identifier group:
| Field | Required | Notes |
| ----- | -------- | ----- |
| **Customer ID** | Yes | Your internal customer identifier |
| **Email/HEM** | Pair with HEM Type | Emails are automatically hashed (MD5 preferred). Map both HEM and HEM Type together |
| **HEM Type** | Pair with HEM | The hash type of the email (e.g. MD5, SHA-1, SHA-256) |
| **MAID** | One of the identifier groups | Mobile advertising identifier |
| **Address Line 1** | All address fields required together | Street address |
| **Address Line 2** | Optional | Secondary address (apartment, suite, etc.) |
| **City** | All address fields required together | City |
| **State** | All address fields required together | State |
| **Zip Code** | All address fields required together | 5-digit ZIP code |
| **ZIP11** | One of the identifier groups | 11-digit ZIP+4 code |
| **IP Address** | One of the identifier groups | IPv4 format |
HEM and HEM Type must be mapped together — you cannot map one without the other. Similarly, if you map any postal address field, all required address fields (Address Line 1, City, State, Zip Code) must be mapped.
Hightouch automatically hashes email values in the HEM field using MD5 (preferred), SHA-1, or SHA-256. You do not need to pre-hash emails before syncing.
### Data quality options
**Remove Duplicates**: When enabled, duplicate rows based on Customer ID are automatically removed before writing to the file.
**Skip Invalid Rows**: When enabled, rows missing required identifiers (Customer ID + at least one identifier group) are skipped rather than causing sync errors.
## File format
Resonate requires CSV files. Hightouch automatically handles file naming and placement:
- **Directory**: `data-append/inbound/`
- **File naming**: `{ModelName}_{SegmentName}_{YYYYMMDD}_{syncRunId}.csv`
## Tips and troubleshooting
### Match rate
Resonate targets a 60–80% match rate. To maximize match rates:
- Include complete postal address (Address Line 1, City, State, Zip Code) when available — this is Resonate's strongest identifier.
- Ensure email addresses are lowercase and trimmed before syncing. Hightouch handles hashing automatically.
- Remove inactive, duplicate, and low-quality records from your model before syncing.
- Suppress international records, business addresses, and P.O. boxes.
### Data requirements
Resonate enforces the following data quality requirements:
- Emails must be lowercase with no leading or trailing spaces (Hightouch hashes them automatically).
- Records missing consent should be excluded from your model.
- International records, business addresses, and P.O. boxes should be suppressed.
### Sync alerts
---
## Responsys
**URL:** https://hightouch.com/docs/destinations/responsys
**Description:** Oracle Responsys is a cross-channel campaign management platform that delivers advanced intelligence at scale so you can create personalized messages based on customers' and prospects' individual interests and preferences.
**Section:** Destinations
## Supported syncing
|Event Type|Description|Supported Sync Modes |
|--|--|--|
|**Profile Lists**|Create base profiles with Email, phone number, or custom IDs|Upsert, Update|
|**Profile Extension Tables**|Add custom attributes and properties to the base profiles|Upsert, Update|
|**Custom / REI Events**|Add items to Events|Insert, Update|
## Getting started
To use the Responsys destination in Hightouch, you need to know your Responsys credentials and which Oracle system hosts your
Responsys account. To find your host URL refer to the [Oracle Responsys documentation](https://docs.oracle.com/en/cloud/saas/marketing/responsys-rest-api/SendRequests.html).
## Syncing data
### Profile lists
Syncing to Responsys profile lists lets you create base profiles. Base profiles primarily contain user IDs to associate
the user to records in profile extension tables.
#### Record matching
You can match rows in your model with users in Responsys on the following fields:
- Email
- Mobile number
- Customer ID
- Responsys ID (RIID)
- MD5 Hashed Email
- SHA256 Hashed Email
Hashed emails can't be used for upsert operations.
#### Field mapping
You can map additional user information such as other identifiers, country, address, or other user-created fields.
### Profile Extension Tables
Syncing to profile extension tables lets you attach additional information to base user profiles.
Profile extension tables (PETs) give you a way to store additional attributes that define behavioral, demographic or
profile preference characteristics of your contacts. Using PETs helps you organize information about your contacts
in a way that is most efficient for your queries and data imports.
Refer to the [Responsys Profile Extension Table docs](https://docs.oracle.com/en/cloud/saas/marketing/responsys-user/List_CreatePET.htm) to learn more.
#### Record matching
You can match rows in your model with users in Responsys on the following fields:
- Email
- Mobile number
- Customer ID
- Responsys ID (RIID)
- MD5 Hashed Email
- SHA256 Hashed Email
Hashed emails can't be used for upsert operations.
#### Field mapping
You can sync columns from your source to any user-created field in your Responsys PET.
### Custom / REI Events
Syncing to Responsys Custom Events (part of Responsys Event Infrastructure or REI) allows you to track and store behavioral data about your contacts.
These events capture specific actions or interactions that contacts have with your brand, such as purchases, website visits, app usage, or any custom activities you want to monitor.
API Documentation on Responsys events is available [here](https://docs.oracle.com/en/cloud/saas/marketing/responsys-rest-api/api-events.html).
#### Record matching
You can match rows in your model with users in Responsys on the following fields:
- Event Name
- Profile List
#### Field mapping
You can sync rows to Responsys events on the following fields:
- Recipient ID
- Customer ID
- Email Address
- Mobile Number
- MD5 Hashed Email
- SHA256 Hashed Email
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Retention Science
**URL:** https://hightouch.com/docs/destinations/retention-science
**Description:** AI-driven email marketing automation
**Section:** Destinations
## Overview
You want to send relevant, timely emails to your customers, and doing so heavily relies on the accuracy and freshness of the data in Retention Science. By automatically syncing customer data from your data warehouse into Retention Science, you no longer need to worry about data consistency because data is no longer flowing in from multiple sources in different formats, and your team can focus on drafting engaging emails and delivering delightful experiences.
## Supported syncing
Type | Description | Supported Sync Modes | API Reference |
----------|---------------------------------------------------------|--------------------- | ----------------- |
**Users** | Sync data from any source to Users in Retention Science | Upsert, Update | [Users docs](https://developer.retentionscience.com/?data/v3.0.0#tag/Users)|
**Lists** | Add or remove users from a list in Retention Science | Add, Remove | [Lists docs](https://developer.retentionscience.com/?data/v3.0.0#tag/Lists)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Retention Science
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination**
button. Select **Retention Science** and click **Continue**. You can then authenticate Hightouch to **Retention Science** by entering a Retention Science API key.
To get your API Key, go to your ReSci dashboard. Click on the dropdown menu in the upper right hand corner and then select **API settings**. Click **API Password** to reveal your key.
## Sync configuration
Once you've set up your Retention Science destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and
the Retention Science destination you want to sync to.
### Users
Sync data from any source to Users in Retention Science.
#### Record matching
You can match rows in your model with user in Retention Science on any of the following fields:
- **Email**
- **Record ID**: this is also called `id` or `record_id`
#### Field mapping
You can sync columns from your source to the following columns in Retention Science user:
- **Full Name**: User's full name (first and last)
- **Address Line 1**
- **Address Line 2**
- **City**
- **State**
- **Zip code**
- **Country**
- **Phone number**
- **Year of birth**
- **Gender**
- **Registartion Source**: The source of the user's registration typically used for user segmentation
- **Birthday**: User's full birthday in `YYYY-MM-DD` format
- **Send Transactional Welcome**: When `true`, ReSci sends an email to new users when there is an available template in the Welcome stage. When `false`, ReSci doesn't send welcome emails to new users. No emails are sent to existing users who are updated. The default for this setting is `true`.
- **Merge Before Upsert**: By default, a request containing a `record_id` and `email` that match two distinct users will result in an API error, with an error message stating that we cannot uniquely identify which user to update. If you set the `merge_before_upsert` field to `true`, ReSci merges these two users into one. The default for this setting is `false`.
- **Date unsubscribed**: The User's unsubscribe date. Setting this field unsubscribes a user from email. This field accepts `YYYY-MM-DD hh:mm:ss` (24-hour clock) in UTC. Note that passing a null value doesn't re-subscribe a previously unsubscribed user. If a user is unsubscribed, ReSci ignores null values.
- **Account Created On**: User's signup date in format `YYYY-MM-DD`. If this field and existing user **Account Created On** value differ, ReSci chooses the oldest date between the two.
- **Timestamps**: The timestamp is the date and time of when the data changed for a specific plugin fields. ReSci uses the newest timestamp when updating the fields. If the timestamp for the plugin data in the database is newer than the data in the call, then the data in the call will be ignored.
- **UTM**: User's UTM tags
- **Accepts Marketing**: Indicates whether the user has opted in to accepting marketing emails. For new users, the default is `true`. Existing users' marketing preferences will not be modified unless you explicitly change this field.
- **Accepts Marketing Timestamp**: The timestamp of a users subscription status change. If earlier than a user's last subscription update, the requested subscription status update will be ignored. The `date_unsubscribed` field takes precedence over this date, so if both are present this date will be ignored. If no date is passed in the current time will be used.
- **Birthday Month**: Birthday month of the user being upserted in numerical format, for example, January = 1. Note that this field is ignored if birthday is present.
- **Birthday Day of Month**: Day of the month for the users birthday. Note that this field is ignored if the **Birthday** field is present.
- **Interests**: An array of interest object representing user's interest opt-in selections.
- **Frequency Option ID**: An integer representing the ID of a user's frequency choice for subscription preferences.
The **Send Transactional Welcome** field defaults to `true`.
If a sync creates a new user record and you've enabled transactional welcome emails in Cortex, then the sync results in ReSci sending the user a welcome email.
Set **Send Transactional Welcome** to `false` if you don't want to send a welcome email.
See Retention Science's [docs](https://developer.retentionscience.com/?data/v3.0.0#tag/Users/paths/~1users/patch) for more details.
#### Delete behavior
**Delete** is only available for upsert mode.
Behavior | Description
---------------|-------------------------------------------------------------------------
**Do nothing** | Keep the customer record in Retention Science with all its synced fields
**Clear** | Unset all the mapped fields from Retention Science
**Delete** | Delete the synced record from your Retention Science user
### Lists
Add or remove users from a list in Retention Science.
#### Record matching
You can match rows in your model with list in Retention Science on any of the following fields:
- **Email**
- **Record ID**: this is also called `id` or `record_id`
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Rockerbox
**URL:** https://hightouch.com/docs/destinations/rockerbox
**Description:** Rockerbox brings clarity to the noise through accurate attribution and marketing mix modeling that helps you make the right optimizations.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes |
| ----------------------- | ---------------------------------------- | -------------------- |
| **Batch File Delivery** | Upload batch files to Rockerbox via SFTP | Insert, All, Diff |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Rockerbox
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Rockerbox** and click **Continue**. You can then authenticate Hightouch to **Rockerbox**.
Enter the following required fields into Hightouch:
- **Username**: Enter your Rockerbox batch file delivery username.
- **Password**: Enter the password for the username above.
## Sync configuration
Once you've set up your Rockerbox destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Rockerbox destination you want to sync to.
### Syncing batch file delivery
Upload batch files to Rockerbox via SFTP.
#### Field mapping
Hightouch lets you sync batch file delivery fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default batch file delivery fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Rockset
**URL:** https://hightouch.com/docs/destinations/rockset
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Setup
You will need to provide the API Key.
Once logged into your Rockset account, click the **API Keys** option under the **Manage** section in the sidebar.
You can either copy an existing key, or click the **Create API Key** button to generate one.
## Syncing
### Sync modes
Hightouch let's you select the workspace and collection to sync to.
This integration supports the Insert, Update and Upsert sync mode.
In the Insert mode, new rows will be inserted into Rockset.
In the Update mode, rows will be kept up-to-date within Rockset.
During the Upsert mode, new rows will be inserted into Rockset and existing rows will be kept up-to-date within Rockset.
### Columns to sync
For Rockset, we give you the ability to send all columns as they are represented in your model.
You can also choose to manually map fields. Only the fields that you map will be exported. All other columns from your results are ignored.
### Delete mode
The default is to do nothing, but you can also set Hightouch to delete the record when it leaves the query result.
---
## Rokt
**URL:** https://hightouch.com/docs/destinations/rokt
**Description:** Empower your marketing team by delivering up-to-date event data to Rokt straight from your warehouse
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes | API reference |
|--|--|--|--|
|**Events**|Sync data from any source to events|Insert| [Event API docs](https://docs.rokt.com/developers/api-reference/event-api)|
|**Segments**|Create and update [audiences](https://docs.rokt.com/developers/api-reference/custom-audience-import)|Add| [Custom Audience API](https://docs.rokt.com/developers/api-reference/custom-audience-import)|
## Connect to Rokt
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Rokt** and click **Continue**. You can then authenticate Hightouch to Rokt by entering the following required fields into Hightouch:
- **App ID**
- **App Secret**
- **Account ID**: You can find this in the URL of your Rokt account. Go to your profile settings and the URL will look like this: `https://my.rokt.com/accounts//profile-settings`
You can find the AppID and the App Secret at the bottom of your Rokt profile settings.
If you haven't created an app yet, create an app and generate the two values.
For more information, refer to [Rokt's docs](https://docs.rokt.com/developers/api-reference/event-api#generating-app-id-and-app-secret).
If you plan on syncing audience segements, you also need to enter your **API Key**.
You can retrieve one by following these instructions in [Rokt's docs](https://docs.rokt.com/developers/api-reference/custom-audience-import/#authorization).
## Sync configuration
Once you've set up your Rokt destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Rokt destination you want to sync to.
### Syncing events
When syncing events, Hightouch treats any records added to [your source](/getting-started/concepts#sources) as new events and sends them to Rokt when your sync runs.
[Rokt's Event API](https://docs.rokt.com/developers/api-reference/event-api#post-events) requires the following event parameters:
- `clientEventId`
- `eventType`
- `eventTime`
The sync configuration form ensures all these are set and provides some additional options.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server. Hightouch syncs the timestamp as the `eventTime` parameter the Event API requires.
If you select a column, it should be in in UTC and RFC 3339.
The request will be rejected if it contains an event with a timestamp that is older than 18 months or more than 5 minutes into the future from the time that the Event API receives it.
Events with timestamps that are outside of this range are not processed.
#### Field mapping
You can sync model columns to Rokt's event properties, including custom fields.
You must include mappings for the **Client Event ID** and **Event Type** fields.
#### Handling PII and hashing
By default, Hightouch automatically hashes the following fields before sending
them to Rokt:
- **Email**
- **Mobile Phone**
- **First Name**
- **Last Name**
- **IP Address**
You can disable this behavior in the sync configuration. If disabled, the data from the model should be appropriately normalized and hashed according to [Rokt's hashing requirements](https://docs.rokt.com/developers/integration-guides/data-and-security/pii-data-hashing/).
### Syncing segments
You can use these lists to target or exclude specific customers in your campaigns.
You can also use these lists to determine which customers experience Rokt on your site.
The Rokt API enables you to create an audience using email information.
You select which type of email field to use in the field mapping portion of sync configuration.
#### User identifiers
To identify which users to add or update in an audience, select model columns that contain the users' email, email (MD5 Hash), or email (SHA256 Hash).
#### Select an existing audience or create a new one
You can create a new audience or use an existing one. When creating a new audience, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model. To use an existing audience, select the desired audience from the dropdown.
#### Handling PII and hashing
By default, Hightouch automatically hashes unhashed emails before sending them to Rokt.
You can disable this behavior in the sync configuration.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Roku Advertising Direct
**URL:** https://hightouch.com/docs/destinations/roku-ads
**Description:** Empower your advertising team to target and suppress first-party audiences across direct buys on Roku’s premium streaming platform inventory.
**Section:** Destinations
## Overview
Deliver more relevant ads across Roku’s premium, brand-safe inventory. Reach
addressable audiences across native, destination, video, and action ad
experiences by matching your first-party identifiers with Roku-connected
devices using Match Booster.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------- | ---------------------------------- | -------------------- |
| **Audiences** | Create and update custom audiences | All |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Roku Ads
This integration requires prior approval from the Roku Ads team and the Hightouch
team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your Roku rep who will verify approval. Once we have confirmed, we will unlock
the integration for you to connect.
To connect, input the `Advertiser Name` provided by your Roku rep. This must match
exactly to what is in Roku's system.
### Syncing custom audiences
You can input or select the name of the audience you plan on delivering to Roku Ads.
#### User identifiers
To identify which users to add or update in an audience, select model columns
and the corresponding Roku Ads fields. You can match on any of the following user
fields:
- Email
- Phone
- IP Address
- Name
- Postal Address
- 1P Roku IDs
If you select 1P Roku IDs (also known as `rida`), Hightouch sends these 1P Roku IDs to the Roku segment directly, along
with other matched Roku IDs based on your other mapped identifiers.
#### Delete behavior
You can choose how to handle records in Roku when the corresponding rows are deleted in your source.
| Behavior | Description |
| -------------------- | ------------------------------------------ |
| **Do nothing** | Keep the contact in your Roku audience |
| **Remove from list** | Remove the contact from your Roku audience |
## Tips and troubleshooting
### Refresh rate
Roku Ads has a max refresh rate of daily. You should not have your Samsung
Ads syncs running more than daily.
### Matched device count
Match rates are not available to preview in the Hightouch platform. Contact your
Roku rep for insights into match rates and audience reach.
Keep in mind that audience counts reflect the number of Roku devices match, not
the number of users matched. Also keep in mind that matching occurs at the
household level, and that a single household may own multiple devices.
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Sync alerts
---
## Rudderstack
**URL:** https://hightouch.com/docs/destinations/rudderstack
**Description:** Sync rich customer data from your data warehouse to hundreds of SaaS tools via Rudderstack's large integration library
**Section:** Destinations
## Setup
To find the write key for your RudderStack destination in Hightouch:
1. Create a new HTTP API source in RudderStack
2. Navigate to Source Settings > API Keys and copy the "Write Key"
## Syncing
### Event types
- **Track \(Generic\):** Sends events of different names when a record is added, changed, or removed in the query results
- **Identify:** Sends an identify event with related properties of the user
### User and anonymous IDs
it's required to send either the user or anonymous ID to RudderStack to identify the user associated with the event.
### Field mapping
You can sync columns from your source into properties of the user \(identify\) or properties of the event \(track\).
### Sending initial results \(track only\)
For the two "Track" event types, Hightouch allows you to choose whether to:
- Send the initial set of results as track events
- Or, only send the changes between the results in the subsequent syncs as track events
Checking the "Send events for initial result set" checkbox will result in a large amount of events being sent in the first sync.
Hightouch will always send the initial set of results as identify calls when in the "Identify" event type.
### Event names and timestamps \(generic track only\)
For the generic track mode, there are further settings that are needed to determine which event name Hightouch will send to RudderStack for each added, changed, and removed record.
In this example, Hightouch will send RudderStack:
- A user signed up event when a record is added to the query results
- A user changed subscription event when a record is changed in the query results
- A user deleted account event when a record is removed in the query results
Hightouch allows you to send a timestamp that the event occurred for generic track events. This is useful for backfilling historical events.
Hightouch also allows you to send a message ID for each record to ensure that the event is deduplicated within RudderStack in the case that the same event erroneously gets sent twice.
---
## S3
**URL:** https://hightouch.com/docs/destinations/s3
**Description:** Transfer data at scale from your warehouse to S3
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
| ------------ | ------------------------------------------------------------ | -------------------- |
| Any data set | Sync data from a source to S3 as CSV, JSON, NDJSON, XML, or Parquet files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Prerequisites
To get started, you need:
- an [S3 bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html)
- AWS [credentials configured in Hightouch](/security/aws) with programmatic access enabled and permission to write to the S3 path you want to use
Hightouch needs the following IAM actions to store items in your bucket:
| Action | Details |
| :------------------------------ | :---------------------------------------------------------------------------------- |
| `s3:PutObject` | Grants permission to add an object to a bucket |
| `s3:AbortMultipartUpload` | Grants permission to to abort or cancel an ongoing multipart upload in an S3 bucket |
| `s3:ListMultipartUploadParts` | Grants permission to list the parts of a multipart upload in an S3 bucket |
| `s3:ListBucketMultipartUploads` | Grants permission to list all the ongoing multipart uploads within an S3 bucket |
For small file transfers, Hightouch only uses the `s3:PutObject` action.
For large files, multi-part uploads may be necessary, hence the other required actions.
You can use the following JSON sample to create your IAM policy:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Sample",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:AbortMultipartUpload",
"s3:ListMultipartUploadParts",
"s3:ListBucketMultipartUploads"
],
"Resource": [
"arn:aws:s3:::${bucketName}/*",
"arn:aws:s3:::${bucketName}"
]
}
]
}
```
See the guide for [configuring AWS credentials](/security/aws) for more details.
## Connect to Amazon S3
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Amazon S3** and click **Continue**. You can then authenticate Hightouch to Amazon S3 by entering your **Bucket Name**. The Bucket Name should just be the name of the bucket, not a URL.
### PGP encryption
The S3 destination supports PGP encryption. Given a **PGP Public Key**, all files transferred to the destination will be encrypted.
To use PGP encryption, provide your public key in the **PGP Public Key** text area including the header and footer in the following format:
```bash
-----BEGIN PGP PUBLIC KEY BLOCK-----
-----END PGP PUBLIC KEY BLOCK-----
```
The **PGP Private Key** is required to decrypt the files synced to your
destination. You don't need to insert this value in Hightouch.
## Sync configuration
Once you've set up your S3 destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the S3 destination you want to sync to.
### Select file format
Hightouch supports syncing CSV, JSON, NDJSON, XML, and Parquet files to Amazon S3.
### Enter filename
The filename or **object key** field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file.
Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync,
Hightouch overwrites it. To keep different versions of the same results file,
you can enable
[versioning](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html)
in your bucket, or your app can copy the data to another location.
If you are using an audience and would like to include the audience name, you
will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter `-86400` (`24 hours * 60 minutes * 60 seconds`). If you want the filename's data to be one hour after the sync takes place, you would enter `3600` (`60 minutes * 60 seconds`).
### CSV options
If you're syncing to a CSV file, you have additional configuration options:
- **Delimiter**: Your options are comma (`,`), semicolon (`;`), pipe (`|`), tilde (`~`), and tab
- (Optional) Whether to **include a CSV header row** in the exported files
- (Optional) Whether to **include a [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark)** in the exported files, the BOM is ``
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `customer_id`, `email`, `first_name` and `last_name`, columns. These columns are mapped to new fields in the destination file as `id`, `email`, and `name`—a [templated](/syncs/mapping-data#template-mapping) concatenation of `first_name` and `last_name`—respectively. Hightouch exports these fields to new fields in the file and ignores all other columns from your results.
### Batch size
By default, and depending on your [sync mode](#supported-syncing), Hightouch sends one file to your S3 bucket for each export. (In diff mode, Hightouch sends three files.)
By enabling batching, you allow Hightouch to send multiple files if your model query results are large. This can help avoid file size errors from S3.
### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
### Indicator file
For diff mode syncs or syncs with batching enabled, you can include an indicator filename. By doing so, an empty file will be uploaded after all files have been uploaded to indicate successful completion of the sync.
The indicator file will not be written in the cases of a sync failure or an empty query result.
### Gzip compression
You can enable gzip compression to reduce the size of your exported files. Note
that Hightouch does not automatically add the `.gz` extension to your files. You
can add this or any other extension to your filename.
## Tips and troubleshooting
### Common errors
#### Access denied
The **ACCESS_DENIED** error is a permissions-related error. Ensure your Hightouch AWS credentials have read/write access to the S3 bucket specified in your destination. The minimal permissions required are `s3:GetObject`, `s3:PutObject`, and `s3:ListBucket`.
### Sync alerts
---
## Sage Intacct
**URL:** https://hightouch.com/docs/destinations/sage-intacct
**Description:** Sync data from your warehouse to Sage Intacct to streamline your financial processes and gain comprehensive insights into your financial data.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
------------------|-------------------------------------------------------|------------------------|-------------------------------------------------------------------------------------------
**AR Adjustment** | Sync data from any source to the AR Adjustment object | Insert | [AR Adjustment docs](https://developer.intacct.com/api/accounts-receivable/ar-adjustments)
**AR Payment** | Sync data from any source to the AR Payment object | Insert | [AR Payment docs](https://developer.intacct.com/api/accounts-receivable/ar-payments)
**Contact** | Sync data from any source to the Contact object | Upsert, Update, Insert | [Contact docs](https://developer.intacct.com/api/company-console/contacts)
**Customer** | Sync data from any source to the Customer object | Upsert, Update, Insert | [Customer docs](https://developer.intacct.com/api/accounts-receivable/customers)
**Invoice** | Sync data from any source to the Invoice object | Update, Insert | [Invoice docs](https://developer.intacct.com/api/accounts-receivable/invoices)
**Journal Entry** | Sync data from any source to the Journal Entry object | Insert | [Journal Entry docs](https://developer.intacct.com/api/general-ledger/journal-entries)
**Transaction** | Sync data from any source to the Transaction object | Update, Insert | [Transaction docs](https://developer.intacct.com/api/order-entry/order-entry-transactions)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Sage Intacct
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Sage Intacct** and click **Continue**. You can then authenticate Hightouch to **Sage Intacct**.
Enter the following required fields into Hightouch:
- **Sender ID**: Enter a valid Sender ID.
- **Sender Password**: Enter a valid Sender Password.
- **Company ID**: Enter a valid Company ID.
- **User ID**: Enter a valid User ID.
- **User Password**: Enter a valid User Password.
## Sync configuration
Once you've set up your Sage Intacct destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Sage Intacct destination you want to sync to.
### Syncing AR Adjustment
Sync data from any source to the AR Adjustment object
#### Field mapping
Hightouch lets you sync AR Ajustment fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default AR Adjustment fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
### Syncing AR Payment
Sync data from any source to the AR Payment object
#### Field mapping
Hightouch lets you sync AR Payment fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default AR Payment fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
### Syncing Contact
Sync data from any source to the Contact object
#### Record matching
You can match rows from your model to Contact in Sage Intacct on any column in your model and any field in Sage Intacct. Ensure the data types of the model column and Sage Intacct field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Sage Intacct automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync Contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default Contact fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|------------------------------------------------------------------
**Do nothing** | Keep the Contact in Sage Intacct with all its synced fields
**Clear** | Clear all the mapped fields, but keep the Contact in Sage Intacct
**Delete** | Delete the synced Contacts from Sage Intacct
### Syncing Customer
Sync data from any source to the Customer object
#### Record matching
You can match rows from your model to Customer in Sage Intacct on any column in your model and any field in Sage Intacct. Ensure the data types of the model column and Sage Intacct field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Sage Intacct automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync Customer fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom Customer fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|-------------------------------------------------------------------
**Do nothing** | Keep the Customer in Sage Intacct with all its synced fields
**Clear** | Clear all the mapped fields, but keep the Customer in Sage Intacct
**Delete** | Delete the synced Customers from Sage Intacct
### Syncing Invoice
Sync data from any source to the Invoice object
#### Record matching
You can match rows from your model to Invoice in Sage Intacct on any column in your model and any field in Sage Intacct. Ensure the data types of the model column and Sage Intacct field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Sage Intacct automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync Invoice fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom Invoice fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|------------------------------------------------------------------
**Do nothing** | Keep the Invoice in Sage Intacct with all its synced fields
**Clear** | Clear all the mapped fields, but keep the Invoice in Sage Intacct
### Syncing Journal Entry
Sync data from any source to the Journal Entry object
#### Field mapping
Hightouch lets you sync Journal Entry fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default and custom Journal Entry fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
**Formatting entries**
Hightouch recommends using the [array builder inline mapper](/syncs/mapping-data#create-an-array) to map properties that the [Sage Intacct SDK](https://github.com/intacct/intacct-sdk-js/blob/acc333d883ade0e37fe1baee5118ff009c109458/src/Functions/GeneralLedger/JournalEntryLineCreate.ts#L30) expects.
The Sage Intacct SDK automatically determines the `TR_TYPE` based on the `AMOUNT` of the entry, so there's no need to map this field. To represent whether an entry is a debit or credit, use a positive or negative `AMOUNT` value, respectively. The sum of all entries in the array will be zero. The absolute value of the `AMOUNT` should not be used.
### Syncing Transaction
Sync data from any source to the Transaction object
#### Record matching
You can match rows from your model to Transaction in Sage Intacct on any column in your model and any field in Sage Intacct. Ensure the data types of the model column and Sage Intacct field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Sage Intacct automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync Transaction fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom Transaction fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|------------------------------------------------------------------
**Do nothing** | Keep the Transaction in Sage Intacct with all its synced fields
**Clear** | Clear all the mapped fields, but keep the Transaction in Sage Intacct
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Sailthru
**URL:** https://hightouch.com/docs/destinations/sailthru
**Description:** Run better marketing campaigns on Sailthru with up-to-date customer data from your data warehouse
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
------------------|-------------------------------------------------------|----------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
**Users** | Sync data from any source to Sailthru users | Upsert, Update | [Users docs](https://getstarted.meetmarigold.com/engagebysailthru/Content/developers/api/user.html?tocpath=Technical%20Documents%7CFor%20Developers%7CAPI%20Endpoints%7C_____21#POSTtocreateorupdateuser)
**Custom events** | Sync data from any source to Sailthru custom events | Insert | [Custom events docs](https://getstarted.meetmarigold.com/engagebysailthru/Content/developers/api/event.html?tocpath=Technical%20Documents%7CFor%20Developers%7CAPI%20Endpoints%7C_____5#POST)
**Purchases** | Sync data from any source to Sailthru purchase events | Insert | [Purchases docs](https://getstarted.meetmarigold.com/engagebysailthru/Content/developers/api/purchase.html?tocpath=Technical%20Documents%7CFor%20Developers%7CAPI%20Endpoints%7C_____14#POSTapurchase)
**Returns** | Sync data from any source to Sailthru return events | Insert | [Returns docs](https://getstarted.meetmarigold.com/engagebysailthru/Content/developers/api/return.html?tocpath=Technical%20Documents%7CFor%20Developers%7CAPI%20Endpoints%7C_____15#POSTareturn)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Sailthru
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Sailthru** and click **Continue**. You can then authenticate Hightouch to **Sailthru**.
Enter the following fields into Hightouch:
- **API Key**
- **API Secret**
To find your API key, navigate to the [Sailthru API Settings](https://my.sailthru.com/settings/api_postbacks).
## Sync configuration
Once you've set up your Sailthru destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Sailthru destination you want to sync to.
### Syncing users
Sync data from any source to Sailthru users.
#### Record matching
To match rows from your model to users in Sailthru, you need to select a model column and corresponding Sailthru field. You can match on any of the following Sailthru fields:
- **Email**
- **Sailthru ID**
#### List mapping
Hightouch supports syncing users to a single list. You can input the name of your list on the sync configuration page. If the list doesn't exist yet, Hightouch will create the list automatically in Sailthru.
#### Field mapping
Hightouch lets you sync user fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom user fields.
If you send data for a custom field that doesn't exist, Hightouch adds the field.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|-----------------------------------------------------------
**Do nothing** | Keep the user in Sailthru with all its synced fields
**Clear** | Clear all the mapped fields, but keep the user in Sailthru
**Delete** | Delete the synced users from Sailthru
### Syncing custom events
Sync data from any source to Sailthru custom events.
#### Record matching
You can match rows from your model to custom events in Sailthru on any column in your model and any field in Sailthru. Ensure the data types of the model column and Sailthru field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
### Syncing purchases
Sync data from any source to Sailthru purchase events.
#### Record matching
You can match rows from your model to purchases in Sailthru on any column in your model and any field in Sailthru. Ensure the data types of the model column and Sailthru field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
### Syncing returns
Sync data from any source to Sailthru return events.
#### Record matching
You can match rows from your model to returns in Sailthru on any column in your model and any field in Sailthru. Ensure the data types of the model column and Sailthru field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Salesforce
**URL:** https://hightouch.com/docs/destinations/salesforce
**Description:** Push enriched data from your warehouse into Salesforce CRM for your teams to act upon it
**Section:** Destinations
## Overview
With the Salesforce destination, Hightouch can both create and update standard or custom objects, update custom picklist field values on those objects, and create platform events to drive more automation in Salesforce.
Use the Salesforce destination to sync to:
- Salesforce Sales Cloud (CRM)
- Salesforce Financial Cloud
- Salesforce Health Cloud
- Salesforce Service Cloud
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| -------------------------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Standard and custom objects** | Sync records to Salesforce objects such as accounts, contacts or custom objects | Insert, Update, Upsert, Archive | [Salesforce object reference](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_list.htm) |
| **Metadata for picklist fields** | Sync metadata to your Salesforce Picklist fields. | All | [Metadata field types](https://developer.salesforce.com/docs/atlas.en-us.api_meta.meta/api_meta/meta_field_types.htm) |
| **Platform events** | Publish event messages that can be received by Salesforce flows, processes, Apex triggers, and more. | Insert | [Platform events developer guide](https://developer.salesforce.com/docs/atlas.en-us.platform_events.meta/platform_events/platform_events_intro.htm) |
- **Insert**: Insert mode pushes new objects or events to Salesforce and doesn't update them as they change in your data source. Insert is the only available mode for platform events.
- **Update**: Update mode updates fields on existing objects in Salesforce.
- **Upsert**: Upsert mode pushes new objects to Salesforce and updates fields that change in your data source.
- **Archive**: Archive mode [deletes](https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/dome_delete_record.htm) all records that appear in your model's query results.
- **All**: All mode is the only available mode for metadata for picklists. Each time you sync metadata values for a particular picklist, it overwrites all existing values.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Understand your Salesforce setup
Before you begin your Hightouch to Salesforce sync configuration, clarifying your business goals and understanding your Salesforce setup is crucial. Walk through the following considerations to ensure your team is aligned and your sync plans are clear. If you prefer, you can download this resource as a PDF to share with your team.
### Business goals
First, you need to plan what data you want to sync to Salesforce, how you want to update it, and how frequently you want to update it. Consider your business needs and consult with your team to align on the answers to these questions.
The following table shows how the responses affect your sync configuration.
|
Consideration
| Effect on sync configuration |
| ------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| What standard and custom objects do you want to sync to Salesforce? Contacts, Leads, etc.? | The response affects your [sync types](#supported-syncing)—in other words, which objects you select to sync to. You need to set up a separate sync configuration for each object you want to sync to. |
| What standard and custom data fields do these objects have that you want to sync to? | The response affects the [field mapping](#field-mapping) on each of your syncs. |
| Do you have new data to add, old data that needs updating, or a combination of both? | The response affects your [sync modes](#supported-syncing)—for example, insert mode for new data, update mode for old, and upsert mode for both. |
| How frequently do you need to update the data? | The response affects your [sync schedule](/syncs/schedule-sync-ui). This is the last step of your sync configuration. |
To learn what objects and fields you are using in your Salesforce instance, you can [inspect your Object Manager](#inspect-objects-relationships-and-fields).
### Required data specifications
Once you've confirmed the objects and fields you want to sync, you need to check that they have the correct data types and that you have permission to edit them. You can find the answers to these questions by [using the Object Manager in Salesforce](#inspect-objects-relationships-and-fields).
|
Consideration
| Effect on sync configuration |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Is the field that you would like to use for [record matching](#record-matching) an [**external ID**](https://help.salesforce.com/articleView?id=inserting_updating_or_deleting_data.htm&type=5) and [**unique**](https://help.salesforce.com/s/articleView?id=000385174&language=en_US&type=1)? | The response affects your [record matching](#record-matching); only fields that meet this criteria appear in the dropdown for record matching. |
| What are the expected data types of the fields you want to sync to? | The response affects your [field mapping](#field-mapping); if the expected type is different than your model column type, you need to [type cast your data](https://hightouch.com/docs/models/data-types-casting) in your model configuration. |
| Are the fields you want to sync to editable? | Hightouch only displays editable fields during [field mapping](#field-mapping). |
| Does the user you are authenticating to Hightouch have edit access? | The sync will error if the user doesn't have the correct permissions. |
| Are any of the data fields lookup fields to another Salesforce object | You can set up [Lookup fields](#relationships-object-id-lookup) while field mapping. |
| Are any of the data fields formula fields? | Formula fields are non-editable and read-only, so the sync will throw an error; change the field type if you want to sync data to it. |
### Other considerations
These last considerations examine the wider context of your Salesforce instance to ensure that your sync won't conflict with other Salesforce automations or overuse your API call quota.
You can [inspect an objects' triggers and flows](#inspect-apex-triggers-and-flows) using the Object Manager. You can inspect your API usage and quota by going to **Setup** > **Environments** > **System Overview**, or by checking Salesforce's [quick reference](https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_api.htm).
Refer to your Salesforce contract in case of any custom limits.
|
Consideration
| Effect on sync configuration |
| --------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| What Salesforce workflows or APEX triggers are you running? What objects do they touch, when do they run? | The response affects your [sync schedule](/syncs/schedule-sync-ui); syncs will error if there are conflicting processes. |
| What Salesforce plan do you have and what are your API limits? | The response affects your [sync schedule](/syncs/schedule-sync-ui); syncs will error if you use up your API call quota. |
### Inspect objects, relationships, and fields
1. Login to your Salesforce instance and make sure you are in the **Sales** module.
2. Click on the gear icon in the top right and open the **Setup** page.
3. Search for **Object Manager** in the left search input and open it.
4. Search for the object you are interested in syncing to the **Quick find** input, for example **Contact**. Click on the label name to inspect the object.
5. From the left sidebar, click **Fields & Relationships**. Inspect the field names and data types to confirm which you plan to sync via Hightouch. Any fields you want to use for [record matching](#record-matching) must have the **External ID** and **Unique** types.
6. If necessary, **create a new custom field** and set it's type to Unique and an External ID.
- Click **New** in your chosen object's **Fields & Relationships**.
- Choose the field type, for example **Email** or **Text**, then click **Next**. (You'll set the Unique and an External ID types in the next step.)
- Give your new field a **Field Label** and description. Enable the **Unique** and **External ID** checkboxes. Click **Next**.
- Select the profiles to which you want to grant edit access to this field via field-level security. The field will be hidden from all profiles if you don't add it to field-level security. Click **Next**.
- Add the field to the page layouts you like and click **Save**.
### Inspect APEX triggers and flows
1. Login to your Salesforce instance and make sure you are in the **Sales** module.
2. Click on the gear icon in the top right and open the **Setup** page.
3. Search for **Object Manager** in the left search input and open it.
4. Select the object you want to inspect the APEX triggers and workflows for, for example **Contact**. Click on the object to open its detail page.
5. From the left sidebar, click **Triggers** or **Flow Triggers**, depending on which one you want to inspect first.
- The Triggers page displays a list of all the APEX triggers that are associated with the object. To inspect a specific trigger, click its name.
- The Flow Triggers page displays list of all the workflows that are associated with the object. To inspect a specific flow, click its name.
6. Once you've opened a specific workflow or trigger, you can inspect its properties, such as its name, status, and other details.
You can also edit or delete the workflow or trigger from this page, as well as view the history of changes made to it, if you have the correct permissions.
If you have any triggers or flows on objects you plan on syncing to, make sure to [schedule your Hightouch syncs](/syncs/schedule-sync-ui) so as to not conflict with them.
## Connect to Salesforce
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Salesforce** and click **Continue**. You can then authenticate Hightouch to **Salesforce** via OAuth.
We recommend that you authenticate with a user that has access to update all fields, records, and objects that you need to interact with. The easiest way to do this is to authenticate with a [system administrator](https://www.salesforce.com/blog/what-is-a-salesforce-admin/) or [integration account user](https://help.salesforce.com/s/articleView?id=sf.create_sf_user.htm&language=en_US&type=5).
Click **Log in to Salesforce**, log in with a Salesforce account with the correct access level, and click **Allow**. You should then see a success notification in the Hightouch UI.
If you have Salesforce Authenticator setup as your 2FA, you may have to accept
the request from your device.
After you've authorized access to Salesforce click **Continue**. Give your destination a descriptive name, for example, "Salesforce Production," and click **Finish**.
## Sync configuration
Once you've set up your Salesforce destination, have a [model](/getting-started/concepts#models) to pull data from, and have a clear idea of the data you want to sync, you can set up your sync. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Salesforce destination you want to sync to.
### Syncing objects
When syncing objects, you can choose to insert, update, upsert, or delete (archive) them. Consult the following table to understand which is best for your use case.
| Sync mode | Use case |
| --------- | --------------------------------------------------------------------------------------------------------------------- |
| Insert | You want to push objects into Salesforce, but don't care if the data within each object remains up to date. |
| Update | You have objects in Salesforce that you want to add information to, but you don't want to create any new objects. |
| Upsert | You want to push objects into Salesforce and want to keep the data up-to-date. |
| Archive | You have objects in Salesforce you would like to delete and your model contains those records and only those records. |
#### Record matching
Because inserts are non-idempotent when using Salesforce ID as an identifier,
Hightouch doesn't support matching on Salesforce ID when upserting. **If you'd
like to sync records based on Salesforce ID, use update mode**.
To match rows from your model to record in Salesforce, you need to select the model column and corresponding Salesforce field. Hightouch requires you use an [**external ID**](https://help.salesforce.com/articleView?id=inserting_updating_or_deleting_data.htm&type=5) type field as the field to match on. It should be an explicit external ID type in Salesforce, not just an external ID field by practice.
Using an external ID field for matching lets Hightouch use Salesforce's native [upsert API](https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/dome_upsert.htm), meaning Hightouch doesn't need to perform a search to determine if a record should be updated or inserted. As a result, sync performance improves and avoids inadvertent duplicates.
You can create a custom field and set it to the external ID type by following the [steps outlined above](#inspect-objects-relationships-and-fields). To learn more, check out the [Salesforce article on externalID field creation](https://help.salesforce.com/s/articleView?id=000383278&type=1). The information regarding audit fields in the Salesforce article doesn't apply.
Once you've created the custom field, click the refresh icon to access the newly created field.
An explicit external ID field will have **(recommended)** written next to its name.
#### Field mapping
You can map data from any of your model columns to native and custom fields in your Salesforce objects. Ensure the data type of your model column matches the data type of the field you want to sync to in Salesforce. For example, a [string-type](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/primitive_data_types.htm) Salesforce field like `FirstName` expects a [string-type](/models/data-types-casting#data-types) model column.
Certain Salesforce fields can be set upon record creation but can't be edited on existing records. Since Salesforce rejects updates to fields that can't be edited:
- in Update mode, the field mapping section only lists editable Salesforce fields;
- in Upsert mode, you can also map to non-editable Salesforce fields, but Hightouch omits those mappings when updating existing records, to make sure that your sync is successful.
To see if a field is editable, check its [`updateable`](https://developer.salesforce.com/docs/atlas.en-us.uiapi.meta/uiapi/ui_api_responses_field.htm) field property via the Salesforce API. You can check this property by [generating an initial access token](https://help.salesforce.com/s/articleView?id=sf.remoteaccess_oidc_initial_access_token.htm&type=5) and calling the [sObject Describe](https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/resources_sobject_describe.htm) API endpoint. The `updateable` value is included in Salesforce's API response, as shown in [this example](https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/dome_sobject_describe.htm).
When syncing to a [multi-select
picklist](https://help.salesforce.com/s/articleView?id=sf.siteforce_data_form_field_types.htm&type=5)
type field, make sure to sync `string`-type values containing
semicolon-delimited lists, as explained in [Salesforce's
documentation](https://help.salesforce.com/s/articleView?id=sf.picklist_limitations.htm&type=5).
#### Relationships (object ID lookup)
In Salesforce, you can set up different [types of relationships](https://help.salesforce.com/s/articleView?id=sf.overview_of_custom_object_relationships.htm&type=5) to associate records on a **main object** with records on a **linked object**.
Using Hightouch, you can relate your Salesforce data through **lookup relationships**, which associate records by using a foreign key.
This relationship type requires you to copy the [Salesforce ID](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/field_types.htm#i1435616) of the associated record (on the linked object) and insert it in a specific [reference field](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/field_types.htm#i1435823) in the record you're syncing (on the main object).
This reference field needs to be an [**associated field** data type](/models/data-types-casting#data-types), which corresponds to Salesforce's [**lookup** type](https://help.salesforce.com/s/articleView?id=sf.external_object_lookup_relationships.htm&type=5).
You can confirm the field is a lookup data type in Salesforce by [using the Object Manager](#inspect-objects-relationships-and-fields).
For example, you may want to associate contacts you're updating with their respective Salesforce accounts.
In this case, the **Contact** object is the main object and the **Account** object is the linked object.
Records on the **Contact** object have an [`AccountId` field](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_contact.htm) which expects the Salesforce ID of the related records on the **Account** object.
However, your data model might not contain the required Salesforce ID values.
As a workaround, Hightouch allows you to perform an **Object ID lookup** by:
1. looking up the Account object based on another unique Salesforce field,
2. retrieving the Salesforce ID of the associated record on the Account object,
3. syncing that value to the `AccountId` field on the Contact object.
Make sure to use a unique Salesforce field for the lookup.
If you already created a separate sync to the linked object, you can select the Salesforce field used for [record matching](#record-matching) in that sync.
If your source data already contains the required Salesforce ID values, you
can skip the lookup by selecting the **Salesforce ID** field in the
relationship mapping. This will also increase your sync performance.
Make sure that the sync to the linked object always runs before the one to the
main object. After creating your syncs, you can [set up a
sequence](/syncs/schedule-sync-with-sequences) to schedule them in the correct
order.
In your sync configuration, start by selecting the Salesforce reference field you want to sync to.
The left side of the mapper displays the text and inputs:
- **find [linked object]**
- **where [Salesforce field to use for lookup]**
- **equals [model column to match on]**
You then need to select the Salesforce field and model column to match on.
In this example, Hightouch looks up the corresponding record on the **Account** object by matching the value in the `unique_key` model column to the values in the `Unique_Key__c` Salesforce field.
The lookup returns the associated Salesforce ID, which Hightouch syncs to the `AccountId` field on the **Contact** object.
You can set up a lookup relationship between records on the same object, such
as relating two records on the **Account** object by mapping the [`ParentId`
lookup
field](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_account.htm).
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. Depending on the [sync mode](#syncing-objects) you're using, you can select from some or all of these options:
| Behavior | Description |
| ----------------- | -------------------------------------------- |
| **Do nothing** | Keep the record in Salesforce |
| **Clear fields** | Keep the record, but clear the mapped fields |
| **Delete record** | Delete the record in Salesforce |
In Upsert mode, you can select from all three. In Update mode, you can only select between **Do nothing** and **Clear fields**. Insert mode doesn't support delete behavior.
#### Batching and parallelization
By default, Hightouch sends batches of 1,000 records to Salesforce and sends ten batches in parallel.
You can increase the batch size up to 10,000 records per batch and parallelization up to 50 batches. Higher batch sizes and batch parallelization can increase the speed of your sync, but may eventually cause errors due to the amount of processing power they require from Salesforce.
If you run into APEX errors such as `TOO_MANY_APEX_REQUESTS` it can be helpful to lower and tune these numbers. For example, you can decrease your batch size by half and if you no longer encounter errors, you can raise the batch size limit slowly.
#### Bulk API version
Hightouch supports both legacy [Bulk API](https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asynch/api_asynch_introduction_bulk_api.htm) and [Bulk API 2.0](https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asynch/bulk_api_2_0.htm). Hightouch uses the original Bulk API by default. You can opt in to use V2.0 in the sync configuration.
The API versions have different batch and parallelization limits.
Specifically, the original Bulk API counts the number of calls and Bulk API 2.0 counts the number of records.
You can learn more about different limits and allocations between the two versions in [Salesforce's docs](https://developer.salesforce.com/docs/atlas.en-us.244.0.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_bulkapi.htm).
You should select the version that best suits your Salesforce agreement.
[**Insert** mode](#syncing-objects) doesn't support Bulk API 2.0.
#### REST API Support
Hightouch provides the option to use Salesforce's [REST API](https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/intro_rest.htm) instead of the Bulk API. This can be useful if your account runs into usage limits with the Bulk API and has a higher usage limit for the REST API. You can enable the REST API in the `Advanced` section in your sync configuration.
We currently use `v58.0` for the REST API.
The REST API uses a default batch size of 200 records per request. If you have automations that require specific batch sizes, you can adjust this in the `Advanced` settings configuration as well.
#### Split retries
Salesforce return a `200` successful response even when part of a batch of updates failed.
In some cases, you may want to retry the failed records, like for the [`UNABLE_TO_LOCK_ROW` error](#unable_to_lock_row).
You can enable [**split retries**](/syncs/retries#split-retries) to filter out the records
that received the relevant errors and retry them again.
#### Merging objects
You can only merge **Account**, **Contact**, **Individual**, and **Lead**
objects. Only [upsert and insert](#syncing-objects) modes support merging, and
you must set [parallelization](#batching-and-parallelization) to **1**.
If your model contains child records that should be merged into a parent record, Hightouch handles merging the records into their respective parents based on two fields:
- `shared_id`
- `merge_id` or `is_child`
Hightouch treats records with the same `shared_id` as the same record and merges them together.
All child records in a group with the same `shared_id` must have a `merge_id` or `is_child` field populated.
For example, suppose you have the following records in your model's query result:
| Account | Shared ID | Is child |
| --------- | --------- | -------- |
| Account A | 1 | true |
| Account B | 1 | true |
| Account C | 1 | false |
| Account D | 2 | false |
| Account E | 3 | false |
| Account F | 3 | true |
Hightouch would merge Account A, Account B, and Account C together with Account C as the parent.
Accounts E and F will be merged together with Account E as the parent.
Select the appropriate model columns and corresponding Salesforce fields for this logic.
#### Multi-object syncs
Hightouch supports syncing to multiple objects, such as **Contact or Lead** and **Account or Lead**. When syncing to multiple objects, Hightouch checks to see if your **external ID mapping** maps to either object. If so, it updates the appropriate object.
To use multi-object syncs, make sure that your Lead's external ID field is **mapped** to the matching field on your Contact or Account. You can check by following these steps.
1. Find the **Lead** in the Salesforce **Object Manager**. Refer to the above documentation for information on [how to use and access the Object Manager](#inspect-objects-relationships-and-fields).
2. Go to **Fields & Relationships**.
3. Click **Map Lead Field**.
4. Map your Lead's external ID to the matching field on the Contact or Account.
You may also map other fields that you want to carry over from Leads when they're converted.
#### Salesforce person accounts syncs
If you need to send data to a Salesforce [person account](https://help.salesforce.com/s/articleView?id=sf.account_person.htm&type=5), follow these steps:
1. Contact Salesforce to enable person accounts if they aren't already.
1. In your Hightouch sync configuration, select **Account** as the object to sync data to. A person account is a copy of the account object.
1. Select your desired [sync mode](#syncing-objects).
1. Ensure you map at least a **LastName** field instead of **FullName**. Without this level of specificity, the sync fails.
1. Complete your configuration as usual.
### Syncing custom picklist values
Hightouch lets you update picklist values on any object. You need to create a separate sync configuration for each picklist you want to update, even if the picklists exist on the same object.
When syncing custom picklist values, _all active picklist values_ are **replaced** with each sync. For example, if your model returns four records, all four records become values for your picklist. Hightouch doesn't check which values are already there or append the new values. Each sync completes overwrites existing picklist values.
First, select the object whose picklist you want to update, then specify the picklist whose values you wish to update:
#### Record matching
Hightouch matches rows from your model to picklist values on the values' **API name**, also referred to as `valueName`. Select the model column that includes the appropriate API names for matching. API names must be unique across both inactive and active values within a picklist.
#### Field mapping
You can optionally select a model column that custom labels. Label values must be unique amongst active values in a picklist.
#### Limitations
Custom picklists may only contain a **total of 1000 values** (active and inactive) per picklist.
### Syncing platform events
Salesforce platform events are immutable, which means that you cannot update or delete them after they have been published. As a result, platform event syncs always use insert mode. This means that Hightouch ignores row changes and removals in your model's query results, and publishes a new platform event whenever a new row appears in your model results.
Before events appear in your Hightouch sync configuration, you first need to create platform events in your Salesforce instance. Hightouch then surfaces them within the sync configuration in a dropdown.
#### Record matching
Since platform events are **Insert** only, you don't need to match them to existing records.
#### Field mapping
You can map any column in your source to a platform event. Hightouch doesn't support creating platform event fields. When creating your event in Salesforce, ensure you're including all the relevant fields so that Hightouch can sync to them.
## Tips and troubleshooting
### Sync volume and performance
Hightouch can process millions of rows to Salesforce in a single sync, given you have the available Salesforce API calls. In other words, sync volume limitations are a factor of your [Salesforce API request limits and allocations](https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_api.htm), rather than Hightouch. If you're syncing large volumes of data, you may want to finetune your [batching and parallelization configuration](#batching-and-parallelization) for optimal performance.
[Upsert mode](#syncing-objects) calls Salesforce's upsert endpoint, which can improve sync performance and lower API datapoint consumption since it only requires one single API request per batch of rows. However, there are still some additional API calls for [rejected row retries](/syncs/retries) and miscellaneous data that needs to be sent to Salesforce. This efficient Upsert mode only works when the Salesforce field used for record matching is an [explicit external ID field](#record-matching).
### Common errors
| Error Message | Explanation |
| :--------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `ApiBatchItems Limit exceeded.` | This error is returned directly by Salesforce when your org has used up its daily **ApiBatchItems** Bulk API limit. Check your Salesforce Bulk/API limits, review any other high‑volume jobs, then retry the sync after the limit resets. See [Salesforce Bulk API limits](https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_bulkapi.htm) for more details. |
| `CANNOT_UPDATE_` `CONVERTED_LEAD` | When you convert a lead to a contact, account, or opportunity, it can't be updated as it doesn't exist anymore. The minute a lead is converted it's "soft" deleted from the leads object. You are no longer able to update this record. |
| `DUPLICATE_EXTERNAL_ID` | This error occurs if you have records with the same IDs coming from an external system. Unique fields are unique to a record and can't be duplicated.
This can happen:
when a lead or contact was imported with the wrong External CRM ID
when you have null values in your primary key
when the Salesforce field used for record matching contains duplicates
|
| `DUPLICATES_DETECTED` | Verify that your batch of data doesn't include duplicates within the batch. Unique fields are unique to a record and can't be duplicated.
This can happen:
when you have null values in your primary key
when the Salesforce field used for record matching contains duplicates
|
| `INVALID_CROSS_` `REFERENCE_KEY` | This error generally occurs when someone tries to sync ids from a different object or a different production or sandbox environment. It's a mismatch that Salesforce identifies for the user. |
| `INVALID_FIELD_` `FOR_INSERT_UPDATE` | This error often occurs when you try to update a read-only field. Visit your Salesforce Object Manager to confirm or change field types. It can also happen if you're using record types and you don't set a [default record type](https://help.salesforce.com/s/articleView?id=sf.user_recordtype.htm&type=5). |
| `INVALID_OR_NULL_` `FOR_RESTRICTED_PICKLIST` | This error occurs when you attempt to sync invalid data into a picklist field, for example, the length is too long or the data type incorrect. Review the field's settings in Salesforce. |
| `INSUFFICIENT_ACCESS_ON_` `CROSS_REFERENCE_ENTITY` | Check that your Salesforce credentials have access to the Salesforce object you are cross-referencing in your sync. |
| `INSUFFICIENT_ACCESS_` `OR_READONLY` | Check that your Salesforce credentials have access and/or write access to the Salesforce object you are referencing in your sync. |
| `INVALID_TYPE_ON_` `FIELD_IN_RECORD` | See [below](#invalid_type_on_field_in_record). |
| `OAUTH_APP_BLOCKED` | See [below](#oauth_app_blocked). |
| `UNABLE_TO_LOCK_ROW` | See [below](#unable_to_lock_row). |
| `ValidationError:` `Reference not found` | See [below](#validationerror-reference-not-found). |
See Salesforce's comprehensive [list of error messages and error codes](https://developer.salesforce.com/docs/atlas.en-us.api.meta/api/sforce_api_calls_concepts_core_data_objects.htm#i1421192) for more.
#### UNABLE_TO_LOCK_ROW
This error occurs when one or more users or processes—including Hightouch syncs—are trying to modify the same record at the same time.
Salesforce can't work on a row while another process is touching it. To mitigate this error, you can try the following solutions:
- Remove conflicting processes: If there is an [APEX trigger or workflow](#inspect-apex-triggers-and-flows) that conflicts with your sync, delete or pause the process or rerun sync when the process finishes. Usually, this error is a temporary issue that can be resolved by retrying the operation after a few minutes.
- Reduce the number of parallel processes: lowering your [parallelization](#batching-and-parallelization) to five or less minimizes the chances of two requests trying to modify the same record at the same time.
- Check for long-running processes: Identify any long-running processes in your Salesforce environment that could be holding locks on records for an extended period. Try to optimize these processes or break them up into smaller batches to reduce the time they take to complete.
- Remove duplicated records: This error can also be caused by duplicates in your [model's](/getting-started/concepts#models) query results. Ensure your model doesn't return any duplicated rows and try the sync again.
You can find more information from [Salesforce's Help Center](https://help.salesforce.com/s/articleView?id=000387767&type=1).
#### INVALID_TYPE_ON_FIELD_IN_RECORD
This error occurs when sending a `null` value to a Boolean field in Salesforce or when a [before-save update flow](https://help.salesforce.com/s/articleView?id=sf.flow_concepts_trigger_record.htm&type=5) tries to set a value that conflicts with the value sent from Hightouch.
To mitigate this error, you can try disabling or modifying the flow.
#### OAUTH_APP_BLOCKED
This error occurs when Hightouch is installed in Salesforce. Ensure that it's unblocked on the Connected Apps OAuth Usage page. The user verifying OAuth permissions must be an admin in Salesforce. Additionally, Hightouch should be granted read and write access to all the objects and fields that you intend to sync.
Keep in mind that Salesforce requires the "View Setup and Configuration" permission for the user account. Ensure that this permission is enabled in the Profile assigned to the user, under the "Administrative Permissions" section of their profile.
{/* */}
#### ValidationError: Reference not found
{/* */}
This error indicates that a [relationship lookup mapping](#relationships-object-id-lookup) in the sync configuration isn't able to successfully perform the lookup. The error message contains the Salesforce field name used for the lookup and the value that is causing the failure. Ensure that the model column used for the lookup mapping only contains values that already exist in Salesforce.
### Missing fields in sync configuration
If you are missing an expected field when configuring syncs, try these tips:
- Ensure fields you want to match against are set as external identifiers.
- If you authenticated with your personal user ID, make sure you have permission to view and update the fields you want.
- If you're missing the Salesforce ID field for matching on, make sure you're in the correct sync mode. Matching on Salesforce ID [isn't supported for upserting](#supported-syncing)—use update mode instead.
- If you set your sync to [Update mode](#syncing-objects), the [field mapping](#field-mapping) section only lists [`updateable`](https://developer.salesforce.com/docs/atlas.en-us.uiapi.meta/uiapi/ui_api_responses_field.htm) Salesforce fields.
When syncing to multi-objects, for example, **Contact or Lead** or **Account or Lead**, ensure _both_ object types include a shared unique field marked as a [`Unique ID`](https://help.salesforce.com/s/articleView?id=000385174&type=1) or a unique `external ID`. Without meeting this requirement, you won't see any fields populate for record matching.
### Duplicate records in Salesforce
Make sure that the Salesforce field used for record matching is an **explicit external ID** field. You can learn more about this in the [record matching](#record-matching) section.
### Limitations on sandboxes
If you are building Hightouch syncs to Salesforce sandbox environments and encounter processing delays or errors, check the version of your Salesforce sandbox and upgrade to a higher version if necessary. You can find more information here: [Salesforce Sandbox Licenses and Storage Limits by Type](https://help.salesforce.com/s/articleView?id=sf.data_sandbox_environments.htm&type=5)
### Enhanced domains
If you are enabling [enhanced domains](https://help.salesforce.com/s/articleView?id=sf.domain_name_enhanced.htm&language=en_US&type=5) for your Salesforce instance, you need to [reauthorize](#connect-to-salesforce) your Salesforce destination to avoid sync disruptions.
To do so, go to your Salesforce destination configuration page, and click **Log in to Salesforce** under **Reauthorize connection**.
To learn more about enhanced domains, check out [Salesforce's enhanced domain FAQ](https://help.salesforce.com/s/articleView?id=000393816&type=1).
### Live debugger
The debugger isn't available for **All** and **Insert** sync modes.
### Sync alerts
---
## Salesforce Commerce Cloud
**URL:** https://hightouch.com/docs/destinations/salesforce-commerce-cloud
**Description:** Keep your storefront up-to-date with the latest customer info from your data warehouse
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ------------- | ---------------------------------------------------------------- | ---------------------- | ---------------------------------------------------------------------------------------------------------- |
| **Customers** | Sync data from any source to Salesforce Commerce Cloud customers | Upsert, Update, Insert | [Customers](https://developer.salesforce.com/docs/commerce/commerce-api/references/customers?meta=Summary) |
## Connect to Salesforce Commerce Cloud
If you don't have an API client already, follow these [steps](https://developer.salesforce.com/docs/commerce/commerce-api/guide/authorization-for-admin-apis.html#create-an-api-client) to create one.
Make sure you include the `sfcc.customerlists.rw` scope to your list of **Allowed Scopes**.
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Salesforce Commerce Cloud** and click **Continue**. You can then authenticate Hightouch to **Salesforce Commerce Cloud**.
Enter the following fields into Hightouch:
- **Organization ID**
- **Short code**
- **Site ID**
- **Client ID**
- **Password**
## Sync configuration
Once you've set up your Salesforce Commerce Cloud destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data.
Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Salesforce Commerce Cloud destination you want to sync to.
### Syncing customers
Sync data from any source to Salesforce Commerce Cloud customers.
#### Record matching
To match rows from your model to customers in Salesforce Commerce Cloud, you need to select a model column and corresponding Salesforce Commerce Cloud field. You can match on any of the following Salesforce Commerce Cloud fields:
- **customerNo (customer number)**
- **email**
If you decide to match on **email**, you can still map the `customerNo` field if you prefer to generate this yourself.
If `customerNo` is not mapped, Salesforce Commerce Cloud generates one for you when inserting new customers.
#### Field mapping
Hightouch lets you sync customer fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom customer fields.
Ensure the data types in your model match the data types of the fields you want to sync to.
You can refer to this API [doc](https://developer.salesforce.com/docs/commerce/commerce-api/references/customers?meta=createCustomerInCustomerList) for the customer's schema.
You can upsert customer addresses by mapping the `addresses` field in the mappings section.
Hightouch expects an array of address objects from your model. You can refer to this API [doc](https://developer.salesforce.com/docs/commerce/commerce-api/references/customers?meta=createAddressForCustomerInCustomerList)
on how to build your address object in your model or in our [inline mapper](/syncs/mapping-data#inline-mapping).
For each customer record, Hightouch gets all addresses in Salesforce Commerce
Cloud tied to the customer to match with the record's addresses. Hightouch
updates the ones that exist and inserts the ones that don't.
You can map your custom fields by entering your custom field's key, which is case sensitive (e.g. `customField`), in the custom mappings section.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| -------------- | ------------------------------------------------------------------------- |
| **Do nothing** | Keep the customer in Salesforce Commerce Cloud with all its synced fields |
| **Delete** | Delete the synced customers from Salesforce Commerce Cloud |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Salesloft
**URL:** https://hightouch.com/docs/destinations/salesloft
**Description:** Empower your Salesloft usage with up to date data from your data warehouse
**Section:** Destinations
## Setup
Salesloft requires an API key. To find the API key, navigate to the [Salesloft Dashboard Settings](https://accounts.salesloft.com/oauth/applications).
In the Salesloft settings, an API Key can be created on this page.
If an API key has never been created, you can create one here.
## Syncing
Hightouch supports syncing to the following Salesloft resources:
- `People`
- `Accounts`
### Sync modes
This integration supports Upsert and Update mode.
In the Upsert mode, new people or accounts will be inserted into Salesloft and all attributes will be kept up-to-date within Salesloft.
In Update mode, people and accounts will be kept up-to-date, but new entries will not be inserted.
### Record matching
Person records can be matched from your source to your Salesloft workspace by the person's email in the Upsert mode.
Account records can be matched from your source to your Salesloft workspace by your Salesloft Account Domain or CRM ID in the Upsert mode.
In Update mode, records can be matched from your source to your Salesloft workspace by ID.
### Field mapping
You can sync columns from your source to Salesloft's default and custom fields. Hightouch automatically detects existing custom fields from your Salesloft account.
You can add custom attributes for people and accounts in your Salesloft account.
---
## Samsung Ads
**URL:** https://hightouch.com/docs/destinations/samsung-ads
**Description:** Empower your advertising team to run targeted campaigns across Samsung Ads’ premium inventory.
**Section:** Destinations
## Overview
Deliver more relevant ads across Samsung’s premium, brand-safe inventory and ad
formats that are not available through programmatic buys. Reach addressable
audiences across their ecosystem of Smart TVs, Smart Phones, and Digital Signage
by matching your first-party identifiers with Samsung IDs using Match Booster.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------- | ---------------------------------- | -------------------- |
| **Audiences** | Create and update custom audiences | All |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Samsung Ads
This integration requires prior approval from the Samsung Ads team and the
Hightouch team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your Samsung rep who will verify approval. Once we have confirmed,
we will unlock the integration for you to connect.
### Syncing custom audiences
You can select the name of the audience you plan on delivering to Samsung Ads.
You will need to reach out to us for every new audience you want to target. We
will set that up for you before syncing.
#### User identifiers
To identify which users to add or update in an audience, select model columns
and the corresponding Samsung Ads fields. You can match on any of the following user
fields:
- Hashed email (SHA256)
- MAID
#### Delete behavior
The only supported sync mode is `all` mode, meaning that each sync run will refresh
the entire audience. Users removed from your audience will automatically be dropped
with each refresh - only present users will be in Samsung Ads's audience.
## Tips and troubleshooting
### Refresh rate
Samsung Ads has a max refresh rate of monthly. You should not have your Samsung
Ads syncs running more than every 4 weeks.
### Matched device count
Match rates are not available to preview in the Hightouch platform. Contact your
Samsung rep for insights into match rates and audience reach.
Keep in mind that audience counts reflect the number of Samsung devices match,
not the number of users matched. Also keep in mind that matching occurs at the
household level, and that a single household may own multiple devices.
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Sync alerts
---
## Sanity
**URL:** https://hightouch.com/docs/destinations/sanity
**Description:** Empower your content management through Sanity - a customizable all-code backend for content-driven websites and apps.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
--------------|-------------------------------------|------------------------|------------------------------------------------------------
**Documents** | Sync data from any source to Sanity | Upsert, Update, Insert | [Documents docs](https://www.sanity.io/docs/http-mutations)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Sanity
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Sanity** and click **Continue**. You can then authenticate Hightouch to **Sanity**.
Enter the following fields into Hightouch:
- **Project ID**: Find your Project ID on the project's main dashboard.
- **API token**: Generate or find your API token for your project under the API tab.
## Sync configuration
Once you've set up your Sanity destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Sanity destination you want to sync to.
### Syncing documents
Sync data from any source to Sanity documents. Define document schemas and `types` in your Sanity project, and sync document properties and fields to Sanity using Hightouch.
#### Record matching
To match rows from your model to documents in Sanity, you need to select the model column that contains values that match the **ID** field. You can match on any of the following Sanity fields:
- ID
- *custom matching field (recommended unique)*
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Sanity automatically generates an identifier for every new record synced.
#### Field mapping
Hightouch lets you sync document fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom document schema fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
Prior to syncing, define your document type's schema in your Sanity project. To write to Sanity documents, map your source fields
to your defined custom document schema fields.
#### Update mode and Upsert mode
Hightouch leverages Sanity's [patch mutation API ](https://www.sanity.io/docs/content-lake/http-patches) to support both Update and Upsert modes. The integration currently uses the `set`
operation, which replaces the values of the fields mapped in your configuration.
We recommend maintaining a clear separation between fields managed by Hightouch and those edited manually in Sanity. Any field mapped to
Hightouch will be overwritten during updates and upserts. To preserve human-managed content, exclude those fields from your Hightouch mappings.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## SAP HANA
**URL:** https://hightouch.com/docs/destinations/sap-hana
**Description:** Move your data into HANA
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
|--------------|-------------------------------------------|----------------------|
| Any data set | Sync data from any source to a HANA table | Upsert |
## Connect to SAP HANA
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **SAP HANA** and click **Continue**. If you're not using a [tunnel](#tunneling), you can then authenticate Hightouch to SAP HANA by entering the following fields:
- **Host**: The hostname or IP address of your SAP HANA server. You don't need to include the `https://`.
- **Port**: The port number of your SAP HANA server.
- **User**: The user that has access through Hightouch to the database and tables you want to sync to.
- **Password**: The password for the user specified above.
To ensure your [credentials](#required-permissions) are correct, click **Test connection**. This confirms if Hightouch can connect to your database by running a basic `SELECT` query.
## Record matching
Hightouch requires a unique identifier in the destination HANA table to add, remove, and update rows. The column must be either a `UNIQUE` or `PRIMARY KEY` column. At least one Hightouch field must map to a unique column.
If you map to an incompatible column, Hightouch will fail the sync with an error message.
## Column types
Hightouch works out of the box with all standard column types.
If you see type errors, it may be because your SQL query is producing the wrong format. Message us on Intercom if you need help.
## Required permissions
Your HANA credentials must be able to:
- `MERGE INTO`, `UPDATE`, `INSERT` and `DELETE` rows from the destination table.
- Create a local temporary table
- Perform select operations on system tables:
- `SCHEMAS`
- `TABLES`
- `TABLE_COLUMNS`
- `CONSTRAINTS`
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## SAP IBP
**URL:** https://hightouch.com/docs/destinations/sap-ibp
**Description:** Forecast and plan across your business by loading fresh data into SAP IBP
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes |
| ------------ | -------------------------------------------------- | -------------------- |
| **Entities** | Sync data to any of your OData entities in SAP IBP | Upsert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to SAP IBP
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **SAP IBP** and click **Continue**. You can then authenticate Hightouch to **SAP IBP**.
Enter the following required fields into Hightouch:
- **Base URL**: The full URL for the data service. For example, `https://myibp.com/sap/opu/odata/sap/IBP/MASTER_DATA_API_SRV/`.
- **Username**: The username for the communication arrangement, rather than your personal username.
- **Password**: The password for the username.
## Sync configuration
Once you've set up your SAP IBP destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the SAP IBP destination you want to sync to.
### Syncing entities
Sync data to any of your OData entities in SAP IBP.
#### Record matching
You can match rows from your model to entity in SAP IBP on any column in your model and the key property of your entity in SAP IBP. Ensure the data types of the model column and SAP IBP field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync entity fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to any available properties.
Ensure your model columns data types match the data types of the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## SAP S/4HANA
**URL:** https://hightouch.com/docs/destinations/sap-s4hana
**Description:** Explore top predictions for intelligent ERP.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
| ---------- | -----------------------------------------------------| ---------------------- | ------------------------------------------------------------------------------------------------------- |
| **Sales Orders** | Sync data from any source to SAP S/4HANA as sales orders | Upsert, Update, Insert | [Sales Orders](https://api.sap.com/api/API_SALES_ORDER_SRV/resource/Sales_Order_Header)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to SAP S/4HANA
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **SAP S/4HANA** and click **Continue**.
Choose the environment that you would like to connect to SAP S/4HANA in. This will be either **Sandbox** or **Production**.
For **Sandbox** environments, input your API key. You can find your sandbox API key in any of the API resource pages, such as [this page for sales orders](https://api.sap.com/api/API_SALES_ORDER_SRV/tryout).
Make sure that your API key has read and write access to your SAP S/4HANA instance.
For **Production** environments, you will need to set up the following in your Communication Management app:
- Communication Users
- Communication Systems
- Communication Arrangements
Use these guides from SAP S/4HANA to create communication arrangements with the necessary permissions and a user:
- [Maintaining communication users](https://help.sap.com/docs/SAP_S4HANA_CLOUD/0f69f8fb28ac4bf48d2b57b9637e81fa/eef80dda3867461c92ac1273689ed36f.html)
- [Creating communication arrangement](https://help.sap.com/docs/SAP_S4HANA_CLOUD/0f69f8fb28ac4bf48d2b57b9637e81fa/fab3fd449cf74c6384622b98831e989e.html)
- [Introductory guide to testing your API](https://help.sap.com/doc/e9d1daf327b943c99ae274c8a9871509/Cloud/en-US/Testing%20SAP%20S4HANA%20Cloud%20API%20Services.pdf)
Once you have completed setting up the above, enter the following fields to connect to SAP S/4HANA:
- **Host**: You can get this from the API environment that you can see in the communication arrangement you activated earlier. In your SAP S/4HANA URL, this is the first section of the URL after the `https://` prefix, i.e https://HOSTNAME.s4hana.ondemand.com.
- **Port**: The default value is 443. You can also find this value the communication arrangement.
- **User**: Username of the user created in Communication Management.
- **Password**: The password for the user specified above.
## Sync configuration
Once you've set up your SAP S/4HANA destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the SAP S/4HANA destination you want to sync to.
### Sync mode
Hightouch supports **Upsert**, **Update**, and **Insert** mode with the SAP ODATA API for Sales Orders.
### Field mapping
Hightouch supports syncing the Sales Order object and its nested fields.
Use Hightouch's object inline mapping to construct SAP S/4HANA's object values from your source.
### Record matching
In **Update and Upsert mode**, Hightouch requires a unique identifier in your model with item IDs from SAP S/4HANA. Rows without item IDs will fail to sync.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Segment
**URL:** https://hightouch.com/docs/destinations/segment
**Description:** Sync rich customer data from your data warehouse to hundreds of SaaS tools via Segment's large integration library
**Section:** Destinations
## Overview
Besides the ability to track first-party customer data, Segment offers robust integrations with the most popular SaaS tools. With our Segment integration, you can automatically sync enriched customer data from your data warehouse with Segment to send this data to downstream tools that Hightouch doesn't already integrate with.
## Supported syncing
|Sync Type|Description|Supported Sync Modes |API Reference|
|--|--|--|--|
|**Track \(Generic\) events**|Sends events of different names when a record is added, changed, or removed in the query results|Insert|[Track specs](https://segment.com/docs/connections/spec/track/)|
|**Track \(Segment Entered / Exited\) events**|Sends the event "Segment Entered" when a record is added and "Segment Existed" when a record is removed|Insert|[Track specs](https://segment.com/docs/connections/spec/track/)|
|**Identify events**|Sends an identify event with related properties of the user|Insert|[Identify specs](https://segment.com/docs/connections/spec/identify/)|
|**Group events**|Sends a group event to associate a user to a group|Insert|[Group specs](https://segment.com/docs/connections/spec/group/)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Segment
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Segment** and click **Continue**. You can then authenticate Hightouch to Segment using a Segment **Write Key**.
To retrieve the write key for your Segment destination in Hightouch:
1. Create a new HTTP API source in Segment using [these instructions](https://segment.com/docs/connections/sources/#create-a-source).
2. Navigate to **Source Settings > API Keys** and copy the **Write Key**.
## Sync configuration
Once you've set up your Segment destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Segment destination you want to sync to.
### User and anonymous IDs
You need to include send either the user or anonymous ID to Segment to identify the user associated with the event.
### Field mapping
You can sync columns from your source into user \(identify\) or event \(track\) properties.
### Sending initial results (Track events only)
For the two track event types, Hightouch lets you choose whether to:
- Send the initial set of results as track events as Entered / Added events
- Or, only send the changes between the results in the subsequent syncs as track events
Checking the **Send events for initial result set** checkbox results in a large amount of events being sent in the first sync.
Hightouch always send the initial results set when sending Identify and Group events.
### Event names and timestamps (Generic track events only)
For the generic track mode, there are further settings that are needed to determine which event name Hightouch sends to Segment for each added, changed, and removed record.
In the preceding screenshot, Hightouch sends Segment:
- A "User Signed Up" event when a record is added to the query results
- A "User Changed Subscription" event when a record is changed in the query results
- A "User Deleted Account" event when a record is removed in the query results
Hightouch allows you to send a timestamp that the event occurred for generic track events. This is useful for backfilling historical events.
Hightouch also lets you to send a message ID for each record to ensure that the event is deduplicated within Segment in the case that the same event erroneously gets sent twice.
## Tips and troubleshooting
### Common errors
#### Large payload issues
Segment returns a `400 Bad Request` error if your batch payload exceeds the limitations. You can find more information in [Segment's documentation](https://segment.com/docs/connections/sources/catalog/libraries/server/http-api/#errors). Adjust your batch configuration accordingly.
### Live debugger
### Sync alerts
---
## Twilio SendGrid
**URL:** https://hightouch.com/docs/destinations/sendgrid
**Description:** Run better email campaigns on SendGrid with up-to-date customer data from your data warehouse
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|-------------|-----------------------------------------------------------------------------|-------------------------------------|----------------------------------------------------------------------------------------------|
| **Contacts**| Sync data from any source to Contacts in SendGrid | Upsert | [Contacts docs](https://docs.sendgrid.com/api-reference/contacts/add-or-update-a-contact) |
| **Mail** | Trigger one-to-one transactional emails for each record in your data model | Triggered Sends (one row = one email) | [Mail Send API](https://docs.sendgrid.com/api-reference/mail-send/mail-send) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to SendGrid
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **SendGrid** and click **Continue**. You can then authenticate Hightouch to **SendGrid** by entering a SendGrid **API Key**.
To create an API Key in SendGrid, follow the steps outlined in the [SendGrid documentation](https://docs.sendgrid.com/ui/account-and-settings/api-keys#creating-an-api-key).
#### Permissions
The permissions the API key needs depend on what you plan on syncing.
| Sync Type | Required Permissions (Full Access) | Optional Permissions (Read Access) |
| ------------ | ---------------------------------- | -----------------------------------------------------------------------------------|
| **Contacts** | `Marketing` | - |
| **Mail** | `Mail Send` | `Template Engine` (for email template), `Suppressions` (for unsubscribe groups) |
## Sync configuration
Once you've set up your SendGrid destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the SendGrid destination you want to sync to.
### Syncing contacts
Upsert is the only available sync mode for the SendGrid destination. Use this sync mode to update existing contacts and insert new ones into SendGrid.
#### Record matching
To match rows from your model to contacts in SendGrid, you need to select the model column that contains your contacts' **email**.
SendGrid's `email` field is case-sensitive. Make sure the emails in your model follow a case convention, for example, all lowercase, to avoid duplicate contacts. For example, if your model has contact data for the email addresses `john@example.com` and `John@example.com`, this would create two separate contacts.
#### Field mapping
You can sync columns from your model to SendGrid default and custom fields.
## Tips and troubleshooting
### Common errors
#### Invalid list id
When you run a sync to SendGrid for the first time, Hightouch creates a new contact list. You can find this list on the **Contact Lists** page in your SendGrid workspace. If you delete this contact list, future sync runs will reject all rows with the `invalid list id` error.
To resolve this issue, create a new sync. This allows Hightouch to create a new contact list, which must exist for the sync to run successfully.
### Live debugger
### Sync alerts
---
## ServiceNow
**URL:** https://hightouch.com/docs/destinations/servicenow
**Description:** Improve your ITSM workflow by having your data in ServiceNow
**Section:** Destinations
## Overview
The ServiceNow Platform is a scalable cloud service that offers a range of applications, workflows and processes. Sharing common architecture and data, the platform incorporates a variety of ready-made suites catering for specific business areas.
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|**Standard Objects**|Sync data from any source to records in ServiceNow|Upsert, Update|
For more information about sync modes, check out the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Getting started
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select ServiceNow and then enter the name of your ServiceNow instance and credentials for an integration user.
Your integration user should have the following roles:
- `rest_service`
- `web_service_admin`
- `rest_api_explorer`
- Read access to the `sys_db_object` table
- Read access to the `sys_glide_object` table
- Read access to the `sys_dictionary` table
- Create/Write/Read access to the table being synced to
## Syncing data
Once you've set up your ServiceNow destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the ServiceNow destination you want to sync to.
### Record matching
You can match records from your source to your ServiceNow instance using any unique field in ServiceNow. This can be a custom field or an out-of-box field, as long as it's guaranteed to be unique. If the selected field isn't unique, the incorrect record may be updated.
If you're using the `sys_id` field, the value from the source must be a 32-character, lowercase, alphanumeric string.
### Field mapping
You can also sync columns from your source to any of the fields in the selected ServiceNow table.
## Tips and troubleshooting
### Maximum exection time exceeded
ServiceNow has some default request limits that can be configured in the `Transaction Quota Rules (sysrule_quota)` table.
If you're seeing `Transaction cancelled: maximum execution time exceeded` try decreasing the batch size in your sync configuration, or increasing the timeout duration of `REST Batch API request timeout` and `REST Table API request timeout` in the Transaction Quota Rules table.
### Live debugger
### Sync alerts
---
## Salesforce Marketing Cloud
**URL:** https://hightouch.com/docs/destinations/sfmc
**Description:** Personalize campaigns in Salesforce Marketing Cloud with rich customer data from your data warehouse
**Section:** Destinations
# Overview
Salesforce Marketing Cloud (SFMC) is a digital marketing platform that enables businesses to create personalized customer journeys across email, mobile, social, web, and more. This integration allows you to sync your data from Hightouch to SFMC to power personalized marketing campaigns, manage contact data, automate customer journeys, and trigger communications.
With the Hightouch SFMC integration, you can:
- Sync data to SFMC Data Extensions for segmentation and personalization
- Update contact information for better targeting
- Manage entry and exit from customer journeys based on your data
- Trigger email and SMS messages based on customer behaviors
This documentation will guide you through setting up the connection, configuring syncs, and troubleshooting common issues.
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------- | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Data Extensions** | Sync records to new or existing Data Extensions, using the SOAP API and optionally file uploads | Upsert, Mirror | [Data Extension docs](https://help.salesforce.com/s/articleView?id=sf.mc_co_salesforce_data_extensions.htm&language=en_US&type=5) |
| **Contacts** | Sync records to Marketing Cloud contacts | Upsert | [API Object docs](https://developer.salesforce.com/docs/marketing/marketing-cloud/guide/soap_web_service_objects.html) |
| **Journeys** | Trigger API events to enter and exit contacts from journeys as they're added to and removed from your query result | Add, Remove | [Contact entry docs](https://developer.salesforce.com/docs/marketing/marketing-cloud/guide/how-to-fire-an-event.html?q=entry) and [exit docs](https://developer.salesforce.com/docs/marketing/marketing-cloud/guide/contactExitRequest.html) |
| **Triggered send** | Send email and SMS messages to individual subscribers | Trigger | [Triggered send docs](https://developer.salesforce.com/docs/marketing/marketing-cloud/guide/creating_a_triggered_send_using_the_async_api.html) |
## Salesforce Marketing Cloud setup
Before you connect Hightouch to Salesforce Marketing Cloud (SFMC), you need to create an installed package in SFMC for Hightouch to use.
If you want to sync records to Data Extensions using [either mirror mode or upsert mode with file uploads](#syncing-data-extensions), you also need to [create an FTP user](#create-ftp-user).
### Create installed package in SFMC
1. Log in to your SFMC instance. Go to **Setup**.
2. In the left sidebar, go to **Apps** > **Installed Packages** and click **New**.
3. Name your package, for example, "Hightouch Integration." Optionally provide a description, and click **Save**.
4. Click **Add Component** and select **API Integration** as the component type. Click **Next**.
5. Select **Server-to-Server** as the integration type and click **Next**.
#### Permissions
These are the minimum scopes for syncing to Contacts, Journeys, and Data Extensions.
Carefully select the appropriate permissions based on your specific use case.
- **Automation > Journeys**: Read, Write, Execute, Activate/Stop/Pause/Resume/Send/Schedule
- **Contacts > List and subscribers**: Read, Write
- **Data > Data Extensions**: Read, Write
- **Provisioning > Accounts**: Read
If you want to sync to [Data Extensions](#syncing-data-extensions) using mirror mode or upsert mode with
[file uploads](#automatic-file-import-via-file-uploads), grant these additional permissions:
- **Data > File Locations**: Read, Write
- **Automation > Automations**: Read, Write, Execute
If you want to trigger sends, grant these additional permissions:
- **Channels > Email**: Read
- **Channels > SMS**: Read
If you would like to grant a custom subset of scopes for just the SFMC
features you want to use, see the section on [custom scopes](#custom-scopes).
Once you save your permissions, you should see a new API Integration component in your package.
With the API Integration created, copy the **Client ID**, **Client Secret**, and subdomain to enter into Hightouch.
The subdomain is the part of the **Authentication base URI** after `https://` and before `.auth.marketingcloudapis.com/`. For example, if your Authentication base URI is `https://mc6vgk-yfjd8879pq4v.auth.marketingcloudapis.com/`, your subdomain is `mc6vgk-yfjd8879pq4v`.
### Create FTP user
You only need to create a FTP user if want to sync [Data Extensions](#syncing-data-extensions) via file uploads.
Using File Upload mode can optimize sync performance, especially on high volume syncs (> 1
million records). Learn more about how file upload protocol works and its tradeoffs in the [file uploads
section](#automatic-file-import-via-file-uploads). You will need a SFMC FTP user even if you are using a blob storage file location.
If your SFMC instance has parent-child Business Units and you want to sync to
a child Business Unit using FTP, you must create the FTP user in the child
Business Unit. Creating the FTP user in the parent Business Unit will not work for syncing to child Business Units.
1. In your SFMC instance, go to **Setup** > **Administration** > **Data Management** > **FTP Accounts** and click **Create User**.
2. By default, the **FTP Username** is your [Marketing Cloud Member ID (MID)](https://trailhead.salesforce.com/content/learn/modules/marketing-cloud-developer-basics/learn-administration-basics), including the current parent account or child business unit. You can keep the default value.
3. Enter an **Email Address** for the new FTP user. This can be the email account you use to log in to SFMC or another one. You won't need this address to connect to Hightouch.
4. Enter a **Password** for the user. Password complexity requirements combine Marketing Cloud password policy and server-side FTP password requirements. These policies require a minimum of 12 characters and no reuse of the five most recent passwords. Reenter the password in **Repeat Password**.
5. The next screen prompts you to choose your authentication method.
- If you want to use a password, keep this selection.
- If you want to authenticate Hightouch to SFMC using an SSH key, select **SSH Key**. Follow [SFMC's instructions](https://help.salesforce.com/s/articleView?id=sf.mc_overview_key_create_ssh_for_ftp.htm&type=5) for how to add a key. Enter the public key into the SFMC and keep the private key for when you [connect to Hightouch](#connect-to-sfmc).
6. On the next screen, select permissions for your FTP user. At a minimum, provide the user **Upload** access to the **Import** directory.
7. Allowlisting IP addresses is optional. For additional security, you can enter [Hightouch's IP addresses](/security/networking#ip-addresses). Click **Next**.
8. Select a **Home Folder**. We recommend setting the home folder to the root directory. If you choose to specify a restricted scope, ensure that this FTP user has access to the **Import** directory.
9. Make sure to copy your FTP Username and either password or SSH private key for entry into Hightouch. Click **Save**.
## Connect to SFMC
You can now connect SFMC to Hightouch. Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button.
Select **Salesforce Marketing Cloud** and click **Continue**.
### Authentication
You can authenticate Hightouch to SFMC by entering the **Client ID**, **Client Secret**, and **Subdomain** you copied during [package creation](#create-installed-package-in-sfmc).
The subdomain is the part of the **Authentication base URI** after `https://` and before `.auth.marketingcloudapis.com/`. For example, if your Authentication base URI is `https://mc6vgk-y5h9sgf6c5t8vxs09pq4v.auth.marketingcloudapis.com/`, your subdomain is `mc6vgk-y5h9sgf6c5t8vxs09pq4v`.
### FTP credentials
This section is only required if you want to [sync records to Data Extensions using mirror mode or upsert mode with file uploads](#automatic-file-import-via-file-uploads).
Enter your FTP user credentials:
- **FTP Username**: typically a numeric Marketing Cloud MID
- **Password** _or_ **private key**
- To use a private key, you must have [uploaded the corresponding public key in SFMC](#create-ftp-user)
- If using a private key, you must enter the complete private key, starting with `-----BEGIN...PRIVATE KEY-----`.
- If your private key requires a passphrase, enter it as well
Incorrect FTP credentials will cause file upload syncs to fail. Double-check your credentials if you encounter connection issues.
### File location
You only need to set up a file location in SFMC if you are syncing Data
Extensions with file uploads.
Hightouch supports the following file location types:
- ExactTarget Enhanced FTP Import Directory (default)
- Relative Location under FTP Site
- Google Cloud Storage
- Amazon S3
#### ExactTarget Enhanced FTP Import Directory
For **ExactTarget Enhanced FTP**, if you have chosen a different External Key for this location, enter the External Key to ensure Hightouch is able to access the uploaded files.
#### Relative Location under FTP Site
For **Relative Location under FTP Site**, enter the Relative Folder Path and the External Key to ensure Hightouch is able to access the uploaded files.
If your FTP user has a Home Folder other than the root directory, enter the
file location relative to the FTP user's home folder in your Hightouch
destination configuration.
#### Google Cloud Storage
To use the Google Cloud Storage file location in SFMC you will need:
- Google Cloud Storage bucket
- Google Service Account with **read** access to your bucket. This will be used by SFMC to read from your bucket.
- Google Service Account with **write** access to your bucket. This will be used by Hightouch to upload files.
If you do not have a Google Service Account, you can follow the setup instructions in our [Google Cloud Platform (GCP) documentation](/security/gcp).
To set up a Google Cloud Storage file location in SFMC:
1. Go to **Setup > Data Management > File Locations** and create a new file location.
2. Configure your file location's name and external key.
3. Choose Google Cloud Storage for Location Type.
4. Enter your GCS bucket name.
5. If you would like for files to be uploaded to a specific path in your bucket, enter the path as the **Relative Location** in your file location.
6. Upload your Google Service Account credentials. Ensure that at a minimum this role has read access to your bucket.
In Hightouch, select the Cloud Credentials you'd like to use to write to your bucket. Ensure this account has write access to your bucket.
Enter your file location External Key, GCP project ID, bucket name, and relative path.
Test your connection to ensure you've successfully connected to SFMC's API, FTP, and your GCS bucket.
#### Amazon S3
To use the Amazon S3 file location in SFMC you will need:
- Amazon S3 bucket
- AWS Credentials with **read** access to your bucket. This will be used by SFMC to read from your bucket.
- AWS Credentials with **write** access to your bucket. This will be used by Hightouch to upload files.
If you do not have AWS Credentials, you can follow the setup instructions in our [AWS documentation](/security/aws).
To set up an AWS S3 file location in SFMC:
1. Go to **Setup > Data Management > File Locations** and create a new file location.
2. Configure your file location's name and external key.
3. Choose Amazon Simple Storage Service for Location Type.
4. Choose whether you'd like to authenticate with an Access Key or Access Key with IAM role and enter your AWS credentials. Ensure that at a minimum this account has read access to your bucket.
5. Enter your S3 bucket name.
6. If you would like for files to be uploaded to a specific path in your bucket, enter the path as the **AWS Relative Location** in your file location.
In Hightouch, select the Cloud Credentials you'd like to use to write to your bucket. Ensure this account has write access to your bucket.
Enter your file location External Key, AWS region, bucket name, and relative path.
Test your connection to ensure you've successfully connected to SFMC's API, FTP, and your S3 bucket.
### Custom scopes
This section is for advanced users who need specific permission scopes for their integration.
By default, Hightouch requests permission for Journeys, Contacts, and Data
Extensions. If you want to use a subset of these features, you can grant a
custom subset of scopes in your SFMC package. After you do this, enter a
space-separated list of scopes in the **Custom Scopes (advanced)** field in the
Hightouch destination configuration.
For example, if you want to sync to Data Extensions via the SOAP API, you
can grant the following scopes in your SFMC package:
- **Data > Data Extensions**: Read, Write
- **Provisioning > Accounts**: Read
Then, enter the following scopes in the Hightouch destination configuration:
```
accounts_read data_extensions_read data_extensions_write
```
See the [SFMC docs for a full list of
scopes](https://developer.salesforce.com/docs/marketing/marketing-cloud/references/mc_rest_rest_permission_scopes/rest-permissions-and-scopes.html).
The scopes listed in Hightouch can't exceed the scopes granted in your SFMC package. If you attempt to use scopes not granted in SFMC, your authentication will fail.
### Allowed Business Units
This section is relevant for SFMC instances with multiple Business Units.
By default, Hightouch can access all Business Units in your SFMC instance that
have the installed package enabled. If you want to restrict access to a subset
of Business Units, you can enter a space-separated list of Business Unit IDs in
the **Allowed Business Unit IDs (advanced)** field in the Hightouch destination
configuration.
You can find the Business Unit ID, or MID, by clicking on the account dropdown at
the top of the SFMC console.
## Sync configuration
Once you've set up your Salesforce Marketing Cloud destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Salesforce Marketing Cloud destination you want to sync to.
The following sections outline the different types of syncs you can configure with SFMC:
### Syncing Data Extensions
Hightouch supports syncing objects to existing Data Extensions and creating new ones.
You can select either **Upsert** or **Mirror** mode.
- In **Upsert** mode, you can choose whether to use the SOAP API or [file uploads](#automatic-file-import-via-file-uploads). Hightouch inserts new objects into the Data Extension and keeps all mapped object attributes up-to-date.
- In **Mirror** mode, you can't select between SOAP API or file uploads. Hightouch always uses file uploads, SOAP, and an Overwrite import automation in SFMC to overwrite the entire Data Extension with your model's query results.
#### Optional configurations
You can optionally choose which **Business Unit** and which **Data Folder** you would you like to sync data to. If you leave these configuration empty, Hightouch uses the defaults configured in SFMC.
#### Automatic Data Extension creation
You can optionally choose whether Hightouch should **automatically create a Data Extension**. If you enable automatic creation, Hightouch creates a Data Extension the first time the sync runs. It doesn't create new Data Extensions on subsequent runs. If you add columns to the sync config, Hightouch automatically adds them to the Data Extension.
When creating a new extension, you can optionally specify the Data Extension's name.
If you leave the name blank, Hightouch uses the model name.
The name also becomes the Data Extension's **customer key**.
You can include timestamp variables in the Data Extension name, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the Data Extension name:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
To change the name or customer key, you need to create a new sync. Hightouch doesn't modify the Data Extension's name or customer key after the first run.
You can also optionally enter a file path when creating a Data Extension, such as `path/to/dataExtension`. If you choose a file path that includes folders, Hightouch will only create new folders if they do not already exist, relative to the selected folder in the Data Folder option. This path can also include the variable values mentioned above, such as `{YYYY}/{MM}/{DD}/{model.name}`.
#### Automatic file import via file uploads
By default, Hightouch uses the SOAP API to sync Data Extension options in upsert mode. If you select to use File Upload mode, Hightouch uploads records via file transfer to SFMC, which may lead to better sync performance. While using File Upload mode is generally faster, it provides less visibility into individual records' sync successes and failures.
Generally, blob storage file locations (such as Google Cloud Storage and Amazon S3) are more performant file servers than SFMC's default option of ExactTarget Enhanced FTP server.
To use File Upload mode, ensure:
- your SFMC package has [the appropriate scopes](#create-a-server-to-server-package)
- you've [created an FTP user](#create-ftp-user) in SFMC
- you've [authenticated the FTP user](#connect-to-sfmc) in your destination configuration
Additionally, if you are using a blob storage file location, ensure:
- your SFMC file location has read access to your bucket
- the credentials you've provided Hightouch has write access to your bucket.
When using File Upload mode, Hightouch creates and runs an [import definition](https://help.salesforce.com/s/articleView?id=sf.mc_es_import_activity.htm&type=5) to load the data into the Data Extension. By default, Hightouch includes 100,000 rows in each file. You can configure this batch size in the Hightouch UI—the maximum value is 10 million rows per file. After each batch, Hightouch processes the SFMC results file to determine row errors.
SFMC provides more detailed error responses when using the SOAP API. If you
need to debug errors and the error messages from File Upload mode aren't
helpful, we recommend trying the SOAP API.
If you enable [deletion](#delete-behavior) in upsert mode, Hightouch removes
deleted rows in the Data Extension using the [SOAP
API](https://developer.salesforce.com/docs/marketing/marketing-cloud/guide/delete.htm),
since SFMC doesn't support deletions using FTP directly. If you want to
completely overwrite the Data Extension with the results of your sync, use
**Mirror** mode.
#### Record matching
You can match objects from your source to your Data Extension on your Data Extension's [primary key](https://help.salesforce.com/s/articleView?id=000388307&type=1).
Select the model column that contains these values.
Editing the Data Extension field used for record matching is only possible
before triggering the first sync run. Otherwise, making this change results in
an error. If you want to use a different field once the sync has already run,
you need to create a new sync.
#### Field mapping
You can map any model column to any field in your Data Extension.
Make sure that you map all **non-nullable fields** when syncing new objects to the Data Extension.
A non-nullable field is a field that is required to be populated with a value.
When creating or editing a record in SFMC, if a non-nullable field is left empty, the system won't allow the record to be saved and will return an error message.
You can check which fields are non-nullable fields in SFMC by inspecting your Data Extensions:
1. Open the Email Studio in your SFMC account.
2. Select the Data Extension you want to check.
3. Inspect the **Fields** section in the Data Extension to see a list of all fields.
4. Look for the **Nullable** column in the list. This column includes a checkbox for each field.
5. If a field doesn't have a check in the **Nullable** column, it means that the field is non-nullable and must have a value in it when a record is created or updated. Make sure you map all non-nullable fields.
In the example screenshot below, since `email` is the only field without a check in the Nullable column, it is the only non-nullable field.
In **Upsert** mode, you can sync records that don't include all non-nullable fields, but only if the object already exists in the Data Extension.
SFMC requires a minimum of one field mapping when sending data using
_SOAP_-mode. This can be a placeholder field.
#### Field creation
When creating a new Data Extension with Hightouch, you can create new fields and select their types while field mapping. First enter a field name, then select its type.
Text and Decimal field types allow for additional configuration.
Hightouch adds new fields to the Data Extension on each run but doesn't remove fields that leave the model's query results.
#### Sendable and testable Data Extensions
For auto-created Data Extensions, you can select whether the Data Extension is **sendable** and **testable**.
A sendable Data Extension is a type of Data Extension that can be used to send email messages or other types of communications to subscribers. Sendable Data Extensions are designed to contain subscriber data that can be used in email marketing campaigns, triggered sends, or other types of communications.
A testable Data Extension is used to send test emails or other test communications to a small group of recipients, usually internal stakeholders or quality assurance testers. Testable Data Extensions are designed to mimic the structure of a sendable Data Extension, but with a smaller set of test data.
To be designated as sendable, a Data Extension must meet certain requirements, including:
- It must contain an email address or mobile number field that is marked as the primary sendable field.
- It must contain at least one additional field that is used as a personalization field in the email message.
- It must be mapped to a profile attribute group, which is used to store additional subscriber data that is not included in the sendable Data Extension.
To make the Data Extension you are syncing via Hightouch sendable, select the field to use as the primary sendable field and the field to map to a subscriber field. The options available for the sendable field are [the fields you mapped](#field-mapping-2).
The length of the field chosen in send relationships is [limited to 254
characters](https://help.salesforce.com/s/articleView?id=sf.mc_rn_2024_spring_cab_de_restriction.htm&type=5).
Choose the `Text - 254 characters` field type when creating a sendable field
in Hightouch to enforce this limit.
You can only set a Data Extension to be testable once you select it to be sendable.
#### Data Retention Policy
You can set a data retention policy on your Data Extension through Hightouch. Hightouch will **not** edit existing data policies unless explicitly enabled.
You can choose what data should be deleted in your policy:
| Behavior | Description |
|-----------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|
| **All records and Data Extension**| All records and the Data Extension will be deleted at the end of your data retention period |
| **All records** | All records will be deleted at the end of your data retention period, but the Data Extension will not be deleted |
| **Individual records** | Records will be deleted from your Data Extension individually as they reach the Data Extension period. The Data Extension will not be deleted.|
You can choose the data retention period in days depending on how long you'd like to store your Data Extension and records.
If you choose to reset the retention policy upon each successful sync run, the data retention period will restart whenever data is successfully imported into your Data Extension.
Data retention policies cannot be removed through Hightouch. To remove an
existing data retention policy, navigate to your Data Extension in SFMC
Contact Builder and turn off the data retention policy.
#### Delete behavior
You can choose how to handle Data Extension when the corresponding rows are deleted in your source.
| Behavior | Description |
| --------------------- | --------------------------------- |
| **Do nothing** | Keep the object in SFMC |
| **Delete the record** | Remove the SFMC record completely |
#### Concurrency
When syncing to SFMC in File Upload mode, you can configure the concurrency used in the FTP upload to SFMC in Advanced options. Enter a value between 1 and 10.
Higher concurrency values will increase upload speeds, but can cause disruptions when simultaneously running multiple Hightouch syncs to SFMC. For this reason, the default concurrency is 1.
Concurrency options are not available for blob storage file locations.
#### File Retention
By default, Hightouch will delete the CSV files used to populate a Data Extension and the results file from the Import Definition automation from your SFMC FTP server. You can choose to keep files on your SFMC FTP server in Advanced options. Files on the SFMC FTP server will automatically be deleted by SFMC after 21 days.
Files are not automatically deleted from your blob storage file locations. We recommend setting a retention policy on your bucket to avoid storing stale CSV files.
#### Notifications
If you would like to receive an email notification when an Import Definition to your Data Extension succeeds, you can enter an email address in Advanced options.
In Mirror mode, one email will be sent for each sync run. In Upsert mode, one email will be sent for each batch completion. Depending on your batch size, this could be multiple emails per sync run.
#### Triggering automations
In Mirror mode, if you would like to trigger an automation after the completion of your Data Extension sync run, you can select **Yes** or **No** and enter either an **External key** or an **Automation ID**. If **Yes**, an automation will run even if there are failed rows in the sync. If the automation itself fails, the entire sync run is marked as failed.
### Syncing contacts
This integration supports upserting contacts into Marketing Cloud. In upsert mode, Hightouch inserts new users into SFMC and updates [mapped user attributes](#field-mapping) on existing contacts.
#### Record matching
Hightouch matches rows from your model to Marketing Cloud contacts by **Contact Key**. Select the model column that contains these values.
Contact Keys are different from Contact IDs and email addresses. Check out
[Salesforce's
article](https://trailhead.salesforce.com/content/learn/modules/marketing-cloud-contact-management/understand-contacts-and-contact-model-relationships)
to learn more.
#### Field mapping
You can sync any model columns to Marketing Cloud's contact attributes.
Ensure all the required fields in each attribute set is completed. For example, to add an email address, be sure to map both **Email Addresses.Email Address** and **Email Addresses.HTML Enabled** fields. Refer to the usage section of SFMC's [Creating Contacts](https://developer.salesforce.com/docs/atlas.en-us.noversion.mc-apis.meta/mc-apis/createContacts.htm) documentation for a list of attribute sets.
### Managing user journeys
Hightouch can manage the entry and exit of contacts in Marketing Cloud journeys based on your model's query results.
- When a record enters your results set, Hightouch adds a contact to the journey via the contact's [Contact Key](https://trailhead.salesforce.com/content/learn/modules/marketing-cloud-contact-management/understand-contacts-and-contact-model-relationships).
- When a record leaves your results set, Hightouch removes the contact from the journey via the contact's [Contact Key](https://trailhead.salesforce.com/content/learn/modules/marketing-cloud-contact-management/understand-contacts-and-contact-model-relationships).
Hightouch does not observe duplicate primary key errors. Adding a record with a primary key that already exists in the journey will be marked as successful. Similarly, removing a record with a primary key that was already removed from the journey will also be marked as successful.
#### Journey selection
Here is an example journey that you could manage with Hightouch:
Note that it's:
- A multistep journey
- Triggered by an entry event with an associated **Data Extension** and **event definition key**
Hightouch automatically looks for the event definition key if you select a journey that fits the preceding requirements.
#### User identifiers
Hightouch enters and exits contacts from the journey via **Contact Key**. Select the model column that contains these values.
Contact Keys are different from Contact IDs and email addresses. Check out
[Salesforce's
article](https://trailhead.salesforce.com/content/learn/modules/marketing-cloud-contact-management/understand-contacts-and-contact-model-relationships)
to learn more.
#### Field mapping
You can use field mapping to map any other columns in your query results to the Data Extension containing your journey's data.
The Data Extension requires the **email_address** field—be sure to map a model column to that attribute. You can do this in Hightouch by typing "email_address" as a **Field from Salesforce Marketing Cloud** in the mappings section and then selecting the appropriate model column.
### Triggered Sends
Hightouch can trigger emails to individual subscribers via the Marketing Cloud API. This is useful for sending personalized messages to specific users based on their behavior or attributes.
Select an existing Triggered Send definition, then map the model columns to the Triggered Send's attributes, including email addresses and custom attributes. Hightouch matches rows from your model to Marketing Cloud contacts by **Subscriber Key**.
## Tips and troubleshooting
### Common errors
#### Failed to create Data Extension
When syncing [Data Extensions](#syncing-data-extensions), an incomplete sync configuration can result in the `Failed to create Data Extension` error.
To resolve it, make sure to enter the field names and select the field types when [creating new fields](#field-creation).
#### Server was unable to read request. ---> There is an error in the XML document. ---> Instance validation error: 'DATA TYPE VALUE' is not a valid value for DataExtensionFieldType.
If you configure your sync to create a new Data Extension with the same name as a Data Extension that already exists, this error may occur. To resolve it, change the name of the Data Extension that you would like to create and select field types for each mapped column in the sync configuration.
#### The event data contains duplicate values for an existing primary key
The full error message looks something like this:
```
400 - {"message":"The event data contains duplicate value for an existing primary key. Please correct the event data and try again.","errorcode":30000,"documentation":""}
```
In this error message, the "primary key" refers to the primary/subscriber key of the related Data Extension. To avoid this error, ensure the column you selected doesn't contain duplicate values. You can read more about syncing Data Extensions in the dedicated [sync configuration section](#syncing-data-extensions).
{/* */}
#### The Custom Object field selected must have a type of Number or Long Number
{/* */}
If you choose to make the Data Extension sendable and set the subscriber relationship using the Subscriber ID, the field that you map to the Subscriber ID must be a number per SFMC's [documentation](https://help.salesforce.com/s/articleView?id=sf.mc_as_data_view_subscribers.htm&type=5), otherwise map this field to the Subscriber Key.
#### Could not create data extension fields: Error: Soap Error Key: "Adding a new Data Extension definition is not allowed when doing an update-only operation."
This error message can happen for multiple reasons; the data extension set in the sync configuration has been deleted from SFMC (by a user or process unrelated to Hightouch), or the Hightouch sync is set to [automatically create a new Data Extension](https://hightouch.com/docs/destinations/sfmc#automatic-data-extension-creation) and the name set in the configuration corresponds to an already existing Data Extension.
In order to fix this, you need to ensure that; your sync is sending data to an existing Data Extension, or if the sync is trying to create a new Data Extension then it should use a unique extension name that doesn't exist in your SFMC environment yet.
#### The import instance could not be validated for the following reason(s): FileNotFound: Soap Error
This message typically means that SFMC cannot access or find the file that was uploaded by Hightouch.
The disconnect can happen in a couple of places. Ensure that the FTP credentials entered into the destination configuration is correct. You can test the credentials by attempting to connect to the SFTP with the credentials outside of Hightouch to confirm this.
Next, ensure that the FTP user has access to the file location you have provided. If you have multiple business units, ensure the FTP user is created in either the parent business unit or in the same business unit as the data extension you are trying to write to.
The default file location is to SFMC's ExactTarget Enhanced FTP Import Directory. If your FTP user is created with a `Home Directory` in SFMC, this can cause the file to be incorrectly written to a nested directory instead of the `Import/` directory. Best practice is to set the FTP user's home directory to `/`, which is the default.
If you are writing to a blob-storage file location (either a AWS S3 bucket or GCS bucket), ensure the role that you have entered in your SFMC file location has correct read permissions.
### Live debugger
### Sync alerts
---
## SFTP
**URL:** https://hightouch.com/docs/destinations/sftp
**Description:** Transfer data at scale from your warehouse to your apps through SFTP
**Section:** Destinations
## Supported syncing
| Object Type | Description | Supported Sync Modes |
|--------------|-----------------------------------------------------------------------------|----------------------|
| Any data set | Sync data from a source to SFTP as CSV, JSON, NDJSON, XML, or Parquet files | Insert, All, Diff |
- **All**: All mode creates one file with all the rows in the query results, every time the sync runs.
- **Insert**: Insert mode creates one file with the rows that were added since the last sync.
- **Diff**: Diff mode creates three files, one for rows added, one for rows changed, and another for rows removed since the last sync.
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to SFTP
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **SFTP** and click **Continue**. You can then authenticate Hightouch to **SFTP** by entering the following required fields into Hightouch:
- **Hostname**
- **Port**
- Credentials (see the [supported authentication methods](#supported-authentication-methods) below)
The user needs to have access to the target directory and file with **write privileges**.
A `permission denied` error message during a sync indicates the user may not have write permissions for both the directory and the file.
### Supported authentication methods
We support three authentication methods:
- Username + password
- Username + SSH private key
- Username + SSH private key + passphrase
We support two formats for the SSH private key:
- OpenSSH
- RSA
Provide the full private key for the user, including the header and footer.
For example:
```bash
-----BEGIN OPENSSH PRIVATE KEY-----
-----END OPENSSH PRIVATE KEY-----
```
### PGP encryption
The SFTP destination supports PGP encryption. Given a **PGP Public Key**, all files transferred to the destination will be encrypted.
To use PGP encryption, provide your public key in the **PGP Public Key** text area including the header and footer in the following format:
```bash
-----BEGIN PGP PUBLIC KEY BLOCK-----
-----END PGP PUBLIC KEY BLOCK-----
```
The **PGP Private Key** is required to decrypt the files synced to your destination.
You don't need to insert this value in Hightouch.
## Sync configuration
Once you've set up your SFTP destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the SFTP destination you want to sync to.
### Select file format
Hightouch supports syncing CSV, JSON, NDJSON, XML, and Parquet files via SFTP.
### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch overwrites it.
If you are using an audience and would like to include the audience name, you will still use `{model.name}`.
### Set filename offset
By default, Hightouch uses the timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter `-86400` (`24 hours * 60 minutes * 60 seconds`). If you want the filename's data to be one hour after the sync takes place, you would enter `3600` (`60 minutes * 60 seconds`).
### CSV options
If you're syncing to a CSV file, you have additional configuration options:
- **Delimiter**: Your options are comma (`,`), semicolon (`;`), pipe (`|`), tilde (`~`), and tab
- (Optional) Whether to **include a CSV header row** in the exported files
- (Optional) Whether to **include a [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark)** in the exported files, the BOM is ``
### Columns to sync
You can export all columns exactly as your model returns them or choose to export specific ones.
If you need to rename or transform any column values you're syncing, you can use the [advanced mapper](/syncs/mapping-data#advanced-mapper) to do so. If you choose this option, Hightouch only syncs the fields you explicitly map.
The preceding example shows how to selectively export the `id`, `email` and `location`. These columns are mapped to new fields in the destination file as `user_id`, `user_email` and `user_location`. Hightouch exports these fields to the new fields in the file and ignores all other columns from your results.
`Boolean` values that appear as `True` and `False` in your model appear as `1` and blank respectively in the file created during the sync. If you'd like for these values to appear in another format in your downstream file, you can use [template mapping](/syncs/mapping-data#template-mapping) to achieve your desired format.
### Batch size
By default, and depending on your [sync mode](#supported-syncing), Hightouch sends one file for each export. (In diff mode, Hightouch sends three files.)
By enabling batching, you allow Hightouch to send multiple files if your model query results are large. This can help avoid file size errors.
### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
## Encoding
By default, Hightouch saves the file in ASCII encoding. If there are any non-ASCII characters in your results, the file will be saved with utf-8 encoding. If you've enabled [including a BOM](#csv-options), the file will be saved with utf-8 encoding with BOM.
## Tips and troubleshooting
The **Test connection** button located on the destination configuration page tries to connect to your configured server, but it can only detect if your credentials are completely invalid.
To fully test user permissions, you can create and run a one-row test sync to verify that your data is successfully synced to your SFTP server.
### Common errors
#### Renaming temporary file to target file failed
During an SFTP sync, Hightouch writes the data to a temporary file in your server and then renames it to the final filename as specified by your sync configuration.
This is an important step to avoid partial data writes and consumption by your downstream processes that can happen due to network issues.
Failure at this step indicates that the temporary file has been consumed and/or deleted by a downstream process that exists in the SFTP server before the sync has concluded.
Our suggestion is to adjust directory listeners to ignore temporary files, which are prefixed with a period `.`, or provide a different temporary file directory in your sync configuration.
Otherwise, this error can also occur if the [credentials you're using](#connect-to-sftp) don't have the necessary permissions to rename the temporary file or modify its directory.
#### Connection timed out/waiting for handshake
Usually, when this error occurs, it is likely that the IP addresses Hightouch uses are not allowlisted and therefore could be being blocked.
To help fix this, we recommend ensuring that our [IP Addresses](/security/networking#ip-addresses) are allowed within your SFTP's Firewall.
If you're unsure whether IPs are allowlisted, ask your network, IT team, or third party SFTP vendor to verify.
### Sync alerts
---
## Microsoft Sharepoint
**URL:** https://hightouch.com/docs/destinations/sharepoint
**Description:** Microsoft SharePoint is a powerful file storage solution that enables seamless document management and collaboration.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
-----------------|------------------------------------------------------------|----------------------
**Any data set** | Sync data from a source to SharePoint as JSON or CSV files | Insert, All, Diff
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Microsoft Sharepoint
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Microsoft Sharepoint** and click **Continue**. You can then authenticate Hightouch to **Microsoft Sharepoint**.
For the **Authentication method**, select **Log in to Sharepoint** and log into your Microsoft Sharepoint account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Microsoft Sharepoint destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Microsoft Sharepoint destination you want to sync to.
### Select file format
Hightouch supports syncing JSON and CSV files to SharePoint.
### Select a Site and Drive
In order to sync to SharePoint, you must select a site and a drive available to your account.
You can specify a folder within the site/drive combination, or create a new folder to write the file to.
### Enter filename
The filename field lets you specify the parent directory and the name of the file you want to use for your results.
You can include timestamp variables in the filename, surrounding each with `{}`.
Hightouch supports these timestamp variables:
- `YYYY`: Represents the full year in four digits.
- `YY`: The last two digits of the year.
- `MM`: Two-digit month format (01-12).
- `DD`: Two-digit day format (01-31).
- `HH`: Two-digit hour format in 24-hour clock (00-23).
- `mm`: Two-digit minute format (00-59).
- `ss`: Two-digit second format (00-59).
- `ms`: Three-digit millisecond format.
- `X`: Unix timestamp in seconds.
- `x`: Unix timestamp in milliseconds.
All dates and times are UTC.
For example, you could enter `upload/{YYYY}-{MM}-{DD}-{HH}-{mm}-result.json` to dynamically include the year, month, date, hour, and minute in each uploaded file. Hightouch would insert each file in the `upload` directory, which would need to already exist in your bucket.
You can also use other [variable values](/syncs/mapping-data#variable-values) to include sync metadata in the filename:
- `{model.id}`
- `{model.name}`
- `{sync.id}`
- `{sync.run.id}`
If a file already exists at the path you entered at the time of a sync, Hightouch will overwrite the file.
### Set filename offset
By default, Hightouch uses the UTC timestamp of the sync run to fill in timestamp variables. You can optionally include an offset in seconds. For example, if you want the filename's date to be 24 hours _before_ the sync takes place, enter '-86400' (`24 hours * 60 minutes * 60 seconds`).
#### Record matching
You can match rows from your model to any data set in Microsoft Sharepoint on any column in your model and any field in Microsoft Sharepoint. Ensure the data types of the model column and Microsoft Sharepoint field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync any data set fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default any data set fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Empty file results
You can select how Hightouch should handle empty results files. Empty result files can occur if your model's query results haven't changed since the last sync for Insert and Diff modes.
You can select whether to **skip empty files**. If you skip empty files, it means Hightouch won't export any files if your model's query results haven't changed since your last sync.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Shopify
**URL:** https://hightouch.com/docs/destinations/shopify
**Description:** Keep your storefront up-to-date with the latest customer and product info from your data warehouse
**Section:** Destinations
Shopify will be [deprecating their REST API](https://shopify.dev/docs/apps/build/graphql/migrate) and recommends their new [GraphQL API](https://shopify.dev/docs/api/admin-graphql). Hightouch supports using these new GraphQL endpoints for new syncs, and existing syncs can be updated to use the GraphQL endpoints.
## Overview
This destination lets you send customer, product, and fulfillment information from your data warehouse to Shopify so you never have to worry about stale inventory information or incorrect customer or product details.
## GraphQL vs REST API
Shopify will be deprecating their REST API in favor of their new GraphQL API. For more detailed information on this update, and how object types and inputs have changed, refer to this [Shopify resource](https://shopify.dev/docs/apps/build/graphql/migrate). Hightouch allows syncs to use either protocol for the time being, and you can change between the two protocols to see the differences between them. Most notably, the GraphQL endpoints use new object types **Product Variant** and **Product Option** that were previously coupled together for the REST API. They are now separate Object types to sync to based on existing **Products** in your shop.
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|**Objects**|Sync data from any source to customer, discount, inventory item, inventory level, and product objects|Varies depending on object|
|**Events**|Send fulfillment events to Shopify|Insert|
|**Actions**|Sync data from any source to any various Shopify actions|Insert|
## Connect to Shopify
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Shopify**.
To connect your Hightouch app to Shopify, you need to create a **custom app** in your Shopify account.
Shopify requires that custom apps made after January 1, 2026 be authenticated to using **Client credentials**. Custom apps made before January 1, 2026 can be authenticated to using an **Admin API access token**. If you are creating a new custom app, please use the Client credentials flow. If you have an existing custom app, you may use the Admin API access token flow.
You then need to provide your **Store name** and either the set of **Client credentials** or an **Admin API access token** of your custom app.
### Create Client credentials for your custom app
1. In your Shopify store's home page, click **Apps** on the sidebar.
2. Click on **Develop apps** and then select **Build apps in Dev Dashboard**.
3. Click **Create app** and go through the flow using the **Start from Dev Dashboard** option.
4. Add to the **Access > Scopes** section the scopes:
```
write_customers,
read_customers,
write_discounts,
read_discounts,
write_inventory,
read_inventory,
write_price_rules,
read_price_rules,
write_products,
read_products,
read_locations,
write_assigned_fulfillment_orders,
read_assigned_fulfillment_orders,
write_merchant_managed_fulfillment_orders,
read_merchant_managed_fulfillment_orders,
write_fulfillments,
read_fulfillments,
write_orders,
read_orders
```
5. Click **Release** and under the **Settings** tab you will find your **Client ID** and **Client secret**.
### Create an Admin API access token for your custom app
1. In your Shopify store's admin, click **Apps** in the left sidebar.
2. From the dropdown which appears, select **Apps and sales channel settings**.
3. Click **Develop apps for your store**.
4. On the next screen, click **Create an App** in the top right corner. Give it a descriptive name, for example, "Hightouch integration." Then click **Create app**.
5. Under **Configuration** tab, select **Configure** to configure the Admin API Integration.
6. Select the following **Admin API access scopes**:
- `write_customers`
- `read_customers`
- `write_discounts`
- `read_discounts`
- `write_inventory`
- `read_inventory`
- `write_price_rules`
- `read_price_rules`
- `write_products`
- `read_products`
- `read_locations`
- `write_assigned_fulfillment_orders`
- `read_assigned_fulfillment_orders`
- `write_merchant_managed_fulfillment_orders`
- `read_merchant_managed_fulfillment_orders`
- `write_fulfillments`
- `read_fulfillments`
- `write_orders`
- `read_orders`
7. Click **Save** in the bottom right of the screen.
8. In **API Credentials** tab, click **Install app**, and then **Install** in the modal that appears.
9. Your **Admin API Access Token** is now available. Copy and save your Admin API access token in a secure place.
## Sync configuration
Once you've set up your Shopify destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Shopify destination you want to sync to.
### Objects
Hightouch supports syncing to these Shopify objects with the following sync modes:
|Sync Type|Supported Sync Modes |
|--|--|
|**Customers**| Upsert, Update, and Insert |
|**Discounts**|Update and Insert|
|**Inventory Item**|Update only|
|**Inventory Level** (REST API only)|Upsert, Update, and Insert|
|**Products**|Upsert, Update, and Insert|
Check out the [sync modes docs](/syncs/types-and-modes#sync-modes) to learn more.
#### Record matching
Based on the Shopify object, database records can be matched from your source to your Shopify workspace by a unique property.
|Sync Type|Supported Identifiers |
|--|--|
|**Customers**| Email, ID, or phone |
|**Discounts**|ID|
|**Inventory Item**|Inventory item ID|
|**Inventory Level**|Inventory item ID|
|**Products**|Handle|
#### Field mapping
You can sync columns from your source to Shopify's default and custom fields.
You must provide a **Price Rule ID** from an existing **PriceRule** when syncing **Discounts**. Similarly, you must provide an **Inventory Item ID** from an existing **Product** when syncing either **Inventory Item** or **Inventory Level**.
#### Delete behavior
For **Products** (for the **REST API** protocol only), you can choose what Hightouch's behavior is when a record leaves the query result set.
| Behavior | Description |
|----------------------|----------------------------------------------|
| **Do nothing** | Keep the product in your shop |
| **Delete the Shopify product** | Remove the product from your shop |
### Events
Hightouch supports syncing to these Shopify events with the following sync modes:
|Sync Type|Supported Sync Modes |
|--|--|
|**Fulfillment event**|Insert only|
#### Field mapping
You need to include at least the **Fulfillment ID**, **Order ID**, and **Status** in your sync. Other field mappings are optional.
### Actions
Hightouch supports syncing to these Shopify events with the following sync modes:
|Sync Type|Supported Sync Modes |
|--|--|
|**Cancel fulfillment orders**|Insert only|
|**Mark fulfillment orders as incomplete**|Insert only|
|**Apply fulfillment hold on fulfillment orders with status `Open`**|Insert only|
|**Move fulfillment orders to another location**|Insert only|
|**Mark fulfillment orders as open**|Insert only|
|**Release the fulfillment hold on fulfillment orders**|Insert only|
|**Reschedule the time of scheduled fulfillment orders**|Insert only|
#### Field mapping
Each action requires you to include at least the **Fulfillment Order ID**. Some actions require additional fields:
|Action| Additional required fields |
|--|--|
|**Apply fulfillment hold on fulfillment orders with status `Open`**| Reason |
|**Move fulfillment orders to another location**|New Location ID|
|**Reschedule the time of scheduled fulfillment orders**|New Fulfill At|
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Singular
**URL:** https://hightouch.com/docs/destinations/singular
**Description:** Enrich your analytics in Singular with event data from your warehouse.
**Section:** Destinations
## Overview
This integration lets you send server-to-server events to Singular without writing a custom pipeline. Whether you're calculating cross-device attribution, analyzing ROI, or using Singular for fraud prevention, including event data helps you have a more complete analytics view.
## Supported syncing
| Type | Description | Supported Sync Modes |
| ---------- | ---------------------------------------------------- | -------------------- |
| **Events** | Hightouch supports syncing in-app events to Singular | Insert only |
## Getting started
Start by creating an instance of Singular destination in Hightouch. Upon creating your instance, you must enter a Singular API key for authentication.
### Authenticating with an API key
To get your API key, navigate to `Developer Tools` > `API Keys` and copy your API key into Hightouch.
## Syncing data
### Events
Any records added to [your source](/getting-started/concepts) are treated as new events and are sent to Singular when your sync runs.
`Platform`,`Package Name`, and `OS Version` fields are the minimum required fields to make the API call to Singular. For a richer analytics, we recommend that you provide all relevant fields in your use case.
The supported platforms are `iOS` and `Android`, case sensitive. If the
platform is `Android`, `andi` field is also required.
If you don't map the `ip` field, it will be defaulted with one from
Hightouch's servers.
You can use custom mappings to attach custom data relevant to your application.
Custom mappings are included as the [`e`
parameter](https://support.singular.net/hc/en-us/articles/360048588672-Server-to-Server-S2S-API-Endpoint-Reference#Optional_Parameters)
of the API request payload.
For more information about required fields, refer to [Singular's documentation](https://support.singular.net/hc/en-us/articles/360048588672#Event_Notification_Endpoint).
## Tips and troubleshooting
### Common errors
If you receive errors, ensure that your including all of Singular's required fields: `Platform`, `Package Name`, `OS Version` and `andi` if your `Platform` is `Android`.
### Live debugger
### Sync alerts
---
## Slack
**URL:** https://hightouch.com/docs/destinations/slack
**Description:** Sync your warehouse with Slack and push business metrics and user activity data to Slack in real-time
**Section:** Destinations
## Overview
With the Slack destination, Hightouch can serve as a notifier for various data-related events or changes.
This is a very flexible integration that you can use for many different use cases, such as:
- notifying your Customer Success teams when product usage for a key account suddenly drops
- notifying an Account Executive when product usage surpasses a certain threshold
- sending a daily summary of the number of sign-ups compared to the same day in the previous month
- sending a daily summary on the adoption of a core feature
## Supported syncing
| Sync type | Description |
| :---------------- | :--------------------------------------------------------------------------------------------------------------------------------------------- |
| **Message** | Sends individual messages for each row that was added, changed, or removed since the last sync. There will be no messages for the initial run. |
| **Batch Message** | Sends a formatted message (or multiple messages) of all the rows in the query results. |
| **Table** | Sends a formatted table of all rows in the query results as a single message. |
| **CSV** | Sends a CSV file of all rows in the query results attached to a single message. |
### Slack error alerting
To learn more about setting up Slack as your error alerting tool, see the [Alerting](/syncs/alerting) page.
## Connect to Slack
### Authenticating with OAuth
Use OAuth to connect Slack by navigating to **Destinations** > **Create New** > **Slack**.
Click **Connect** and log in with your desired Slack account.
Once you have selected your desired workspace, click **Allow** and you will see an **Authorization successful** screen in Hightouch.
Finish your setup by clicking the continue button and naming your destination.
### Adding the Hightouch bot to a channel
In order for Hightouch to message your channel, you need to ensure the Hightouch bot is added as an integration.
Follow the steps below to add the bot to your channel.
You can add the Hightouch bot both to public and private Slack channels.
Adding the Hightouch bot can be done by typing `/invite @hightouch` in the Slack composer. If that doesn't work you can also add the Hightouch bot by following these steps:
1. Go to your channel and click the dropdown button (**∨**) next to the channel name.
2. Click **Integrations** > **Add an App**.
3. Search for **Hightouch** and click **Add** next to the Hightouch app name.
4. The Hightouch bot should now appear in the channel settings.
## Sync configuration
Once you've set up your Slack destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data.
Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin.
Then, select the relevant model and the Slack destination you want to sync to.
### Syncing messages
Hightouch supports sending messages based on rows that have been [added, changed, or removed](/getting-started/concepts#change-data-capture) since the last sync run.
{" "}
This is only applicable when using the **Message** sync type
Each row is sent as an individual message and can be [templated with Liquid](#templating-messages).
You can also send messages to [different Slack channels](#channel-selection).
Hightouch doesn't send any messages on the initial sync run to prevent a flood of messages to your channel. Messages start appearing **after** the initial sync run. The same behavior applies to [full resyncs](/syncs/overview#resync-full-query) as well.
If you want avoid this behavior, modify your model query to return only one "test" row. Hightouch processes this row during the first sync run and doesn't send it as a message to Slack. Afterward, update your model query to include all rows from your source. Hightouch syncs all subsequent rows to Slack during the second sync run.
#### Channel selection
After adding the Hightouch bot to a Slack channel, you can select it from this dropdown menu or enter a channel ID manually.
You can also enable the **Use column** toggle to select a model column instead of a Slack channel name.
The model column selected in the dropdown menu must contain the correct Slack channel ID for every row in your model.
This also allows you to dynamically set the target channel, by inserting different Slack channel IDs for different rows in your model.
#### Private channels
Private channels are hidden from the dropdown. To send to a private channel you can manually enter the channel ID.
Check out the [Common
errors](#slack-channel-not-appearing-in-sync-configuration) section if you
don't see your Slack channel in this menu.
#### Added, changed, and removed messages
Hightouch allows you to send different messages based on the row type:
- when a row is **added** to your query results, Hightouch will send the _row added_ content.
- when a row is **changed** in your query results, Hightouch will send the _row changed_ content.
- when a row is **deleted** from your query results, Hightouch will send the _row deleted_ content.
This content can be [templated](#liquid-templating) and also supports the [Slack Block Kit](#slack-block-kit).
Check out the [Common errors](#parseerror-unexpected-token) section if your
column name contains spaces.
### Syncing batch messages
The batch message sync mode is similar to the normal [message](#syncing-messages) mode, except it sends all rows in the query result rather than only [added, changed, or removed rows](/getting-started/concepts#change-data-capture).
This sync mode is ideal if you want to write a template and have Hightouch push a message containing **all** rows to Slack, formatted as a single message.
This mode also allows you to specify how many rows are sent per message, which can range from 1 to 50.
With 1 row per message, it behaves similar to the [message](#syncing-messages) sync mode since it creates multiple messages in Slack.
Whereas 50 rows per message is similar to the [table](#syncing-tables) sync mode.
#### Channel selection
The batch message sync mode only supports sending your messages to a single channel, unlike the [message](#channel-selection) mode.
#### Private channels
Private channels are hidden from the dropdown. To send to a private channel you can manually enter the channel ID.
#### Header, body, and footer content
For each message, Hightouch allows setting the header, body, and footer content.
The header and footer are optional, whereas the message body is required.
The body is repeated for every row in your model.
This content can be [templated](#liquid-templating) and also supports the [Slack Block Kit](#slack-block-kit).
Check out the [Common errors](#parseerror-unexpected-token) section if your
column name contains spaces.
### Syncing tables
The table sync mode is similar to the [batch message](#syncing-batch-messages) mode.
It sends all of your query results as a table-formatted message to Slack.
This mode is only recommended for small datasets.
### Syncing CSV files
The CSV sync mode is similar to the [batch message](#syncing-batch-messages) and [table](#syncing-tables) modes.
It sends all of your query results as a CSV file attached to a Slack message.
### Templating messages
[Liquid templating](/syncs/mapping-data#liquid-reference) is supported for the [message](#syncing-messages) and [batch message](#syncing-batch-messages) sync modes.
This allows you to include model columns as part of the message body to insert dynamic values in your Slack messages.
Here is an example message making use of Liquid to include model columns in a message:
```text
User with name {{ row['first name'] }} and email {{ email }} has signed up
```
Check out our [Extended Liquid use cases](/syncs/mapping-data#extended-liquid-use-cases) section to learn how to format [dates](/syncs/mapping-data#date-function), [strings](/syncs/mapping-data#string-functions) or [currency amounts](/syncs/mapping-data#currency-amounts).
#### Tagging Slack users \(@...\)
Hightouch supports tagging users by either their **Slack User ID** or by email:
- **Tagging by User ID:** tag by user ID by writing `<@{{ userId }}>` where `userId` is the column that contains the user's Slack ID. If the `userId` is a static value, you can write something like `<@UL53466SY>`.
- **Tagging by Email:** tag by email by writing `<@{{ email | slack_user_lookup }}>` where email is the column that contains the user's email used in Slack. If the email is a static value, you can write something like `<@{{ "example@gmail.com" | slack_user_lookup }}>`.
#### Inserting URLs
You can insert URLs by highlighting a section of your template and pasting the URL.
Otherwise, you can build URLs directly by following this format: ``.
For example, a URL that points to a Salesforce organization's page could look like this: ``
We recommend using the second method when building dynamic URLs with [liquid templating](#templating-messages).
#### Unfurling links and blocking unfurl
By default, Slack automatically unfurls (displays a preview of) URL links in your messages.
To prevent automatic unfurling, surround the URL with backticks (`` ` ``), for example:
`` `https://www.hightouch.com/` ``
The [backtick](https://en.wikipedia.org/wiki/Backtick) is usually located
above the tab key on the left-hand side of the keyboard.
Blocking URL unfurling isn't possible when using the [Slack Block
Kit](#slack-block-kit). Please{" "}
{" "}
if you would like to request this feature.
#### Formatting text for app for Slack surfaces
Slack provides an [extensive guide](https://api.slack.com/reference/surfaces/formatting) to formatting surfaces in the app for Slack with instructions on how to:
- format text
- insert emojis
- mention users and channels
- format dates, quotes, and lists
{/* */}
### Slack Block Kit
{/* */}
To enable the Slack Block Kit, select **Slack Block Kit** for the content type.
Make sure to use [Slack's Block Kit Builder](https://app.slack.com/block-kit-builder) to draft your message content.
Each message content must be an array of Slack blocks, here is an example of a valid configuration.
The **Test Block Kit** button under the message content allows you to view your templates as a Slack message ([view an example](https://app.slack.com/block-kit-builder/T0215BZEKB2#%7B%22blocks%22:%5B%7B%22type%22:%22section%22,%22text%22:%7B%22type%22:%22mrkdwn%22,%22text%22:%22hello!%22%7D%7D,%7B%22type%22:%22section%22,%22text%22:%7B%22type%22:%22mrkdwn%22,%22text%22:%22email:%20%7B%7B%20email%20%7D%7D%22%7D%7D,%7B%22type%22:%22section%22,%22text%22:%7B%22type%22:%22mrkdwn%22,%22text%22:%22goodbye!%22%7D%7D%5D%7D)).
### Customizing bot appearance
Hightouch also allows you to customize how the Hightouch bot is displayed when sending messages to your Slack channel.
You can find these fields in your sync configuration:
| Name | Description |
| :----------- | :------------------------------------------------------------------------- |
| Display name | Display name of the Hightouch bot |
| Icon URL | URL of a 48x48 image that you wish to use as the bot icon |
| Emoji | Slack emoji to use instead of an icon image (for example, `:sweat_smile:`) |
Keep in mind that users in the channel will be able to see the real bot name and image by clicking on it.
## Tips and troubleshooting
### Common errors
#### Slack channel not appearing in sync configuration
If your Slack channel isn't appearing in the [channel selection menu](#channel-selection), make sure that you [added the Hightouch bot](#adding-the-hightouch-bot-to-a-channel) to the channel.
If this still doesn't work, try [reauthorizing your Slack destination](#authenticating-with-oauth) in Hightouch.
As a next troubleshooting step, remove the Hightouch app for Slack from your Slack workspace by visiting the [Hightouch app page](https://hightouchexternal.slack.com/apps/A010ZEEB44R-hightouch?tab=settings&next_id=0), opening the **Configuration** tab, and clicking **Remove App**.
Then [set up a new Slack destination](#authenticating-with-oauth) and [add the Hightouch bot](#adding-the-hightouch-bot-to-a-channel) to your Slack channel once more.
#### Invalid_blocks
Using `"plain_text"` instead of `"mrkdwn"` in your Slack Block Kit syntax is the most common cause of this error.
You can find examples of the correct syntax to use in the [Slack Block Kit section](#slack-block-kit) of this page.
#### Cannot read properties of undefined (reading 'length')
You may receive the `Cannot read properties of undefined (reading 'length')` error when syncing [batch messages](#syncing-batch-messages).
If you're using the [Slack Block Kit](#slack-block-kit) as the content type, your sync might return this error instead:
`Parsing failed with error SyntaxError: Unexpected end of JSON input`.
In either case, make sure to insert **body** content in the sync configuration.
Header and footer content is optional.
{/* */}
#### ParseError: unexpected token
{/* */}
The full error message may look like, `unexpected token at "name", line:1, col:13`. You may receive this error because a column header that is being referenced in your message contains a space. For example:
```text
First Name: {{ first name }}
```
To resolve this error you can try the following options:
- Rename your column so that the name does not contain spaces, for example, `firstName` or `first_name`.
An example will look like:
```text
First Name: {{ first_name }}
```
- If you cannot change the column's name then you can use this language of [liquid templating](/syncs/mapping-data#liquid-reference). An example will look like:
```text
First Name: {{ row['first name'] }}
```
### Live debugger
### Sync alerts
{" "}
---
## Smartsheet
**URL:** https://hightouch.com/docs/destinations/smartsheet
**Description:** Sync your warehouse data to your Smartsheet sheets
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-----------|-----------------------------------|------------------------|-----------------------------------------
**Sheets** | Sync data to sheets in Smartsheet | Upsert, Update, Insert | [Smartsheet API Reference](https://smartsheet.redoc.ly)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Smartsheet
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Smartsheet** and click **Continue**. You can then authenticate Hightouch to **Smartsheet** with either a Smartsheet API key or with OAuth.
### Authenticate with API key
To retrieve an API key, go to your Smarsheet dashboard.
Select **Account** > **Apps & Integrations** in the side nav.
In the **Personal Settings** form that appears, select **API Access**.
On the **API Access** tab, you can either generate a new token by clicking **Generate new access token** or use an existing one.
Enter the access token into **API Key** field in Hightouch and then click **Continue**.
Enter a descriptive name for your destination, and complete setup.
### Authenticate with OAuth
For the **Authentication method**, select **Log in to Smartsheet** and log into your Smartsheet account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Smartsheet destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Smartsheet destination you want to sync to.
### Syncing rows
Hightouch supports inserting, upserting, and updating data as rows into Smartsheet sheets.
Read through the [section below](#duplicate-rows-in-sheet) if you're using Upsert mode and duplicates are being created in your sheet.
#### Record matching
You can match rows from your model to rows in Smartsheet on any column in your model and any field in Smartsheet. Ensure the data types of the model column and Smartsheet field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
We recommend selecting your sheet's [primary column](https://help.smartsheet.com/articles/504645-primary-column) for record matching.
Read through the [section below](#duplicate-rows-in-sheet) if the values you're using for record matching start with a leading zero as that can cause duplicates.
In **Insert** mode, Smartsheet automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
You can sync columns from your model to columns within your selected Smartsheet sheet.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|------------------------------------------------------------
**Do nothing** | Keep the row in Smartsheet with all its synced fields
**Clear** | Clear all the mapped fields, but keep the row in Smartsheet
**Delete** | Delete the synced record from your Smartsheet sheet
## Tips and troubleshooting
### Common errors
#### Duplicate rows in sheet
As explained in the [Values starting with zero](https://help.smartsheet.com/learning-track/level-1-get-started/columns#:~:text=Values%20starting%20with%20zero) section, Smartsheet needs to add a leading apostrophe to any values that start with a leading zero. This process can cause duplicates when updating a sheet via API, so we recommend avoiding any leading zeros in your [record matching](#record-matching) values.
Furthermore, [Upsert mode](#supported-syncing) might create duplicates in your sheet if you run your sync too frequently (such as every 24 hours or more often). This is because it calls Smartsheet's [Search sheet](https://smartsheet.redoc.ly/tag/search/#operation/list-search-sheet) API endpoint, which might not return rows that were added or edited during the past 24 hours.
To avoid this, you can schedule your sync so it doesn't run as frequently or you can replace your Upsert mode sync with a two-part setup:
- for Update mode: one model, sync (set to Update mode), and Smartsheet sheet
- for Insert mode: another model, sync (set to Insert mode), and Smartsheet sheet
The data from the "Insert mode" sheet can then be synced to the "Update mode" sheet by a [copy rows](https://help.smartsheet.com/articles/2479626-automatically-copy-rows-between-sheets) Smartsheet workflow. If the workflow is set up correctly, this eliminates the risk of duplicate rows being created in your "Update mode" sheet.
{/* */}
#### errorCode: 1051
{/* */}
This error can happen if you try syncing data to [locked columns or rows](https://help.smartsheet.com/articles/522077-locking-unlocking-columns-and-rows).
### Live debugger
### Sync alerts
---
## Snapchat
**URL:** https://hightouch.com/docs/destinations/snapchat
**Description:** Empower your marketing team to run highly granular campaigns on Snapchat
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| --------------------- | --------------------------------------------------------- | -------------------- | ---------------------------------------------------------------------------------------------- |
| **Segments** | Create and keeps audience segments up-to-date in Snapchat | Add, Remove, Update | [Audience segment docs](https://marketingapi.snapchat.com/docs/#create-an-audience-segment) |
| **Conversion Events** | Create and send conversion events to Snapchat | Insert | [Conversions API docs](https://marketingapi.snapchat.com/docs/conversion.html#conversions-api) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Snapchat
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Snapchat** and click **Continue**. You can then authenticate Hightouch to **Snapchat**.
For the **Authentication method**, select **Log in to Snapchat** and log into your Snapchat account.
Then, authorize Hightouch to access your Snapchat account:
Once successful, you will be redirected back to Hightouch.
There, select the Snapchat business organization you want to sync data to.
Finally enter a descriptive name for your destination to complete setup.
## Sync configuration
Once you've set up your Snapchat destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Snapchat destination you want to sync to.
### Syncing segments
You can use Hightouch to create and keep audience segments in Snapchat up-to-date using [Snapchat's Ads API](https://marketingapi.snapchat.com/docs/).
Begin by selecting the Snapchat Ad Account your audience segments should belong to.
#### Select an existing audience or create a new one
You can create a new audience segment or use an existing one. When creating a new audience, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model. To use an existing audience, select the desired audience from the dropdown.
#### Audience member retention
You can specify a custom retention period that an audience segment should retain members.
Snapchat's default is 9,999 days.
You can only set custom values when first creating an audience segment.
If you want to set a new retention period, you need to create a new sync.
#### User identifiers
To identify which users to add or update in an audience, select model columns and the corresponding Snapchat fields. You can match on any of the following Snapchat fields:
- Email
- Phone
- Mobile ID
### Syncing conversion events
Version 2 will be deprecated in 2025. Hightouch will automatically migrate
syncs to Version 3 before then.
Snapchat's [Conversions API](https://marketingapi.snapchat.com/docs/conversion.html) requires the following event parameters:
- event name (named `event_type` in Snapchat's Conversions API)
- event conversion type (`event_conversion_type`)
- event tag (`event_tag`)
- timestamp
The sync configuration form ensures all these are set and provides some additional options.
For more information about Snapchat's Conversions API and it's prerequisites, check out [Snapchat's API help article](https://businesshelp.snapchat.com/s/article/api-apply?language=en_US).
#### Event name and conversion type
Providing an event name is required to send an event to the Conversions API. You can either provide a static value or select to **use a column** from your model.
You can select your event conversion type in the same way.
Snapchat's Conversions API accepts the following values:
{/* */}
| Name | Values |
| ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `event_type` | "PURCHASE," "SAVE," "START_CHECKOUT," "ADD_CART," "VIEW_CONTENT," "ADD_BILLING," "SIGN_UP," "SEARCH," "PAGE_VIEW," "SUBSCRIBE," "AD_CLICK," "AD_VIEW," "COMPLETE_TUTORIAL," "INVITE," "LOGIN," "SHARE," "RESERVE," "ACHIEVEMENT_UNLOCKED," "ADD_TO_WISHLIST," "SPENT_CREDITS," "RATE," "START_TRIAL," "LIST_VIEW," "APP_INSTALL," "APP_OPEN," "CUSTOM_EVENT_1," "CUSTOM_EVENT_2," "CUSTOM_EVENT_3," "CUSTOM_EVENT_4," "CUSTOM_EVENT_5" |
| `event_conversion_type` | "OFFLINE," "WEB," "MOBILE_APP" |
{/* */}
See the [Snapchat Marketing API documentation](https://marketingapi.snapchat.com/docs/conversion.html#parameters-for-event-type-platform) for more information about event parameters.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server.
If you select a column, it should be in UNIX timestamp, calculated in seconds or milliseconds.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
You must include the **event tag** field that Snapchat's Conversions API requires for each event.
Event tags are custom values you create, for example, "in-store," "weekend sales," "back-to-school campaign," etc.
Check out Snapchat's [Conversion parameters documentation](https://marketingapi.snapchat.com/docs/conversion.html#conversion-parameters) for more information about the parameters you can map.
**Version 3**
The new version of the conversion API takes many of the same parameters as the previous version but they are split into multiple mapping groups.
Checkout Snapchat's version 3 [Conversion parameters documentation](https://docs.snap.com/api/marketing-api/Conversions-API/Parameters) for more information about the parameters you can map.
### PII hashing
Snapchat requires identity information to be normalized and hashed using a SHA256 hash.
Enable PII hashing if you wish for Hightouch to normalize and hash these values for you.
If you turn off PII hashing, you must ensure your model data is hashed before sending it to Snapchat.
## Tips and troubleshooting
### Matched users count
Below only applies to the segment sync type.
### Common errors
### Live debugger
### Sync alerts
---
## Snowflake Iceberg
**URL:** https://hightouch.com/docs/destinations/snowflake-iceberg
**Description:** Hundreds of industry leaders use Hightouch to turn Snowflake into a marketing, sales, success and operational engine
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes |
|--------------|--------------------------------------------------------|------------------------|
| Any data set | Sync data from any source to a Snowflake Iceberg table | Upsert, Insert, Mirror |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
This destination is for Snowflake-managed Iceberg tables. If you want to sync to standard Snowflake tables, use the [Snowflake destination](/destinations/snowflake) instead.
## Prerequisites
Before setting up the Snowflake Iceberg destination, you must have an [external volume](https://docs.snowflake.com/en/user-guide/tables-iceberg-configure-external-volume) configured in Snowflake. Iceberg tables require an external volume that specifies where the table data and metadata are stored.
## Snowflake credential setup
To allow Hightouch access to Snowflake for Iceberg tables, it's best to create a user specifically provisioned with access to the required tables, schemas, and external volumes. You can also use a personal Snowflake login for your credentials, as long as it has the correct permissions, specifically:
- Add, update, and delete ([if applicable](#delete-behavior)) rows in your sync's Iceberg table
- View the `INFORMATION_SCHEMA.COLUMNS` table which is used for gathering metadata to set up the sync
- `USAGE` privilege on the external volume used for Iceberg tables
### Service account setup
You can use the following SQL template to create a service account with the necessary roles and permissions:
```sql
-- Edit the following variables
set ht_username='HIGHTOUCH_USER';
set ht_default_warehouse='';
set ht_database='';
set ht_default_namespace='';
set ht_default_role='HIGHTOUCH_ICEBERG_ROLE';
set ht_external_volume='';
set ht_comment='Used for Hightouch Iceberg integrations';
-- Set role for grants
USE ROLE ACCOUNTADMIN;
-- Create a role for Hightouch
CREATE ROLE IF NOT EXISTS identifier($ht_default_role)
COMMENT = $ht_comment;
-- Create Hightouch's user
CREATE USER IF NOT EXISTS identifier($ht_username)
TYPE=service
rsa_public_key=''
DEFAULT_WAREHOUSE=$ht_default_warehouse
DEFAULT_NAMESPACE=$ht_default_namespace
DEFAULT_ROLE=$ht_default_role
COMMENT=$ht_comment;
-- Grant permissions to the role
GRANT ROLE identifier($ht_default_role) TO ROLE SYSADMIN;
GRANT USAGE ON WAREHOUSE identifier($ht_default_warehouse) TO ROLE identifier($ht_default_role);
GRANT ROLE identifier($ht_default_role) TO USER identifier($ht_username);
GRANT USAGE ON DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT USAGE ON ALL SCHEMAS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON ALL TABLES IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON FUTURE TABLES IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON ALL VIEWS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
GRANT SELECT ON FUTURE VIEWS IN DATABASE identifier($ht_database) TO ROLE identifier($ht_default_role);
-- Iceberg-specific permissions
GRANT USAGE ON EXTERNAL VOLUME identifier($ht_external_volume) TO ROLE identifier($ht_default_role);
GRANT CREATE ICEBERG TABLE ON SCHEMA TO ROLE identifier($ht_default_role);
```
Once you've created a Snowflake service account, you're ready to set up the connection in Hightouch.
## Connect to Snowflake Iceberg
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Snowflake Iceberg** and click **Continue**. If you're not using a [tunnel](#ssh-tunneling), you can then authenticate Hightouch to Snowflake by entering the following fields:
- **Account identifier** - You can find this identifier at the beginning of your Snowflake URL, for example, (`https://ACCOUNT_IDENTIFIER.snowflakecomputing.com`).
- **Warehouse** - This specifies the warehouse to use when Hightouch executes queries in Snowflake.
- **Database** - This specifies the database to use when Hightouch executes queries in Snowflake.
**Account identifier** format may differ based on Snowflake account age. For example, older Snowflake accounts often have identifiers that look like `ACCOUNT_LOCATOR.CLOUD_REGION_ID.CLOUD`, whereas newer Snowflake accounts have identifiers that look like `ORGNAME-ACCOUNT_NAME`.
For more details, visit [Snowflake's account identifier docs](https://docs.snowflake.com/en/user-guide/admin-account-identifier.html).
### Iceberg configuration
- **External volume** - The Snowflake external volume that provides storage for Iceberg tables. This must be configured in your Snowflake account before setting up the destination.
- **Base location** (optional) - A prefix path within the external volume where Iceberg tables will be stored.
### Credentials
- **Username** - This can be your personal Snowflake login or a dedicated user for Hightouch. **Note**: Use the "Login name", which may [differ from the Snowflake username](https://docs.snowflake.com/en/user-guide/admin-user-management#using-classic-console).
- **Role** (optional) - Use this field to specify the role Hightouch should use when executing queries in Snowflake. If left blank, Hightouch uses the user's default role.
You have two options for finalizing your credentials:
- RSA key pair (recommended)
- Password
Snowflake is [deprecating password authentication](https://www.snowflake.com/en/blog/blocking-single-factor-password-authentification/) in favor of RSA key pair authentication or multi-factor authentication. Hightouch will continue to support password authentication for existing customers until they have migrated.
To ensure your credentials are correct, click **Test connection**. This confirms if Hightouch can connect to your database and verify access to the external volume.
### RSA authentication
Generate a [private key](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#step-1-generate-the-private-key) and [public key](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#step-2-generate-a-public-key) by running the following commands in your terminal:
```bash
$ openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8
$ openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub
```
**Save** the password you used for the key, you will need to enter it when creating the Snowflake Iceberg destination in Hightouch.
Then in Snowflake, execute an `ALTER USER` command to assign the public key to your Hightouch Snowflake user:
```bash
alter user ht_user set rsa_public_key='MIIBIjANBgkqh...';
```
Finally, in Hightouch under the **Authentication Method** section, drag and drop or upload the private key you just generated into the **Private key file** field. Enter the password for your private key in the box below here, unless you used an unencrypted key.
### SSH tunneling
Hightouch can connect directly to Snowflake over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your Snowflake instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
## Sync configuration
Once you've set up your Snowflake Iceberg destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Snowflake Iceberg destination you want to sync to.
### Table selection
Select the schema and Iceberg table you want to sync to. You can either select an existing Iceberg table or enable **Automatically create and update Iceberg table** to have Hightouch create a new Iceberg table using your configured external volume. If auto-create is enabled and the table already exists, Hightouch alters its definition to add new columns as needed.
### Sync mode
Hightouch supports **Upsert** mode using the [`MERGE INTO`](https://docs.snowflake.com/en/sql-reference/sql/merge) statement, with the option to [delete removed rows](#delete-behavior), **Insert** mode using the [`COPY INTO`](https://docs.snowflake.com/en/sql-reference/sql/copy-into-table) statement, and **Mirror** mode which will truncate the table and then insert all rows from the model.
**Mirror** mode will remove any existing data in the table before inserting all rows from the model
### Record matching
To match rows from your model to rows in the Iceberg table, Hightouch requires you to select a unique identifier in the table you are syncing to.
The Snowflake column you select must be set as a `UNIQUE`, `PRIMARY KEY` column within Snowflake.
You can see columns that fit this criteria as available options in records matching section.
If there are no fields in the dropdown, you must add a unique type column to your Iceberg table. Then, click the refresh icon to access the newly created column.
In **Insert** mode, Snowflake automatically generates an identifier for every new record synced, so there is no need to match an existing record.
### Column types
Iceberg tables have some data type limitations compared to standard Snowflake tables:
| Supported Types | Notes |
|------------------|----------------------------------------------------------|
| Text (VARCHAR) | Standard text/string data |
| Number | INTEGER, FLOAT, NUMBER |
| Boolean | TRUE/FALSE values |
| Date | DATE type |
| Timestamp (NTZ) | Timestamp without timezone |
| Timestamp (LTZ) | Timestamp with local timezone (recommended for Iceberg) |
| Time | TIME type |
**Semi-structured data types are not supported** by Iceberg tables. This includes `VARIANT`, `OBJECT`, and `ARRAY` types. If your source data contains these types, you must transform them before syncing.
`TIMESTAMP_LTZ` is the recommended timestamp type for Iceberg tables. It internally stores UTC time with a specified precision. All operations are performed in the current session's time zone, controlled by the [TIMEZONE](https://docs.snowflake.com/en/sql-reference/parameters.html#label-timezone) session parameter. Note that `TIMESTAMP_TZ` is not supported by Iceberg tables.
If you see type errors, it may be because your model is producing the wrong format. If so, use [typecasting](/models/data-types-casting) or [liquid templating](https://hightouch.com/docs/syncs/mapping-data#liquid-template-format) to resolve the issue.
### Field mapping
You can sync columns from your model to the columns in your Iceberg table. When auto-create is enabled, you can create new columns by mapping to column names that don't exist yet.
### Batch size
You can tune the number of rows Hightouch upserts or inserts per query based on your needs and database threshold. The default is 50,000 rows per batch.
If you want to increase the sync's speed, you can increase the batch size. Keep in mind that in upsert mode, Snowflake fails the entire batch of rows if it detects any erroneous row. If you suspect that you will have many bad rows, don't use a high batch size. To avoid locks, ensure you account for your database's capacity when increasing the batch size.
### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
|----------------|------------------------------------------------|
| **Do nothing** | Keep the row in the Iceberg table |
| **Delete row** | Remove the row from the Iceberg table entirely |
## Tips and troubleshooting
### External volume errors
If you see errors related to the external volume during connection testing, verify:
- The external volume exists in your Snowflake account
- Your user has `USAGE` privilege on the external volume
- The external volume name is spelled correctly in the destination configuration
### Common errors
### Live debugger
### Sync alerts
---
## Split
**URL:** https://hightouch.com/docs/destinations/split
**Description:** Sync data from your warehouse to Split.io for data-driven feature release and management, enhancing user experience through targeted deployment.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-------------|-------------------------------------------------------|----------------------|-------------------------------------------------------------------
**Users** | Sync data from any source to Split users | Update, Insert | [Users docs](https://docs.split.io/reference/users-overview)
**Segments** | Sync data from any source to new or existing segments | Add, Remove | [Segments docs](https://docs.split.io/reference/segments-overview)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
{/* */}
## Connect to Split
{/* */}
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Split** and click **Continue**. You can then authenticate Hightouch to **Split** by entering your Split **API key**.
To retrieve an API key in Split, click your user icon located at the top left corner of the screen. From the drop-down menu, select **Admin setting**. In the **Workspace settings** section, select **API keys**. You have the option to use an existing key or generate a new one by clicking on the **Actions** button at the top right corner and selecting **Create Admin API key**.
For more information, refer to [Split's API keys article](https://help.split.io/hc/en-us/articles/360019916211-API-keys).
## Sync configuration
Once you've set up your Split destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Split destination you want to sync to.
### Syncing users
When syncing users, you can choose to update or insert them. When inserting new users, they are created with a Pending status.
#### Record matching
You can match rows from your model to users in Split on any column in your model and any user field in Split. Ensure the data types of the model column and Split field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
You can map data from any of your model columns to fields in Split. If you are inserting users, you have the option to include email and groups. Additionally, you can update display name, email, disable 2FA, activate, and deactivate a user. Ensure the data types of your model columns match the data types of the fields you want to sync to.
### Syncing segments
Sync data from any source to new or existing segments. You need to specify the [workspace](https://docs.split.io/reference/workspaces-overview), [environment](https://docs.split.io/reference/environments-overview), and [traffic type](https://docs.split.io/reference/traffic-types-overview) to sync your segment. Segments are limited to 100,000 members and each update can only include 10,000 new members at a time.
#### Record matching
To match rows from your model to records in Split, you need to select the model column that contains the **identifier** field.
#### Create a segment
To create a new segment list, select **Create a new audience**. You can provide a name for this segment, or Hightouch defaults to the name of the model used to create the sync.
When you create a segment, it will be added to the chosen environment. If you enter a name that already exists, segment creation fails.
#### Use an existing segment
To use an existing segment list, select **Use existing segment**. You can select any of your existing Split segments from the drop-down select menu.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Spotify Ads Manager
**URL:** https://hightouch.com/docs/destinations/spotify
**Description:** Sync first-party audiences and conversion events to Spotify Ads Manager for better targeting, suppression, and measurement.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
----------------|--------------------------------------------------------|----------------------|----------------------------------------------------------------------------------------------------
**Audiences** | Sync data from any source to Spotify audiences | All | [Audiences docs](https://developer.spotify.com/documentation/ads-api/reference/v3.0/createAudience)
**Conversions** | Sync data from any source to Spotify conversion events | Insert | [Conversions docs](https://adshelp.spotify.com/s/article/Spotify-Conversions-API-US?language=en_US)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Spotify
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Spotify** and click **Continue**. You can then authenticate Hightouch to **Spotify**.
For the **Authentication method**, select **Log in to Spotify** and log into your Spotify account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Spotify destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Spotify destination you want to sync to.
### Syncing audiences
Sync data from any source to Spotify audiences.
#### Record matching
You can match rows from your model to audience members in Spotify using any of the supported identifiers. Ensure the data types of the model column and Spotify field you select match.
The supported identifiers are:
- Hashed email (`EMAIL_ID_SHA256`)
- Hashed device ID (`DEVICE_ID_SHA256`)
- Hashed phone number (`PHONE_NUMBER_SHA256`)
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync audience fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default audience fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
### Syncing conversions
Sync data from any source to Spotify conversion events.
#### Record matching
You can match conversion events to users in Spotify using any of the supported identifiers. Ensure the data types of the model column and Spotify field you select match.
The supported identifiers are:
- Hashed email (`hashed_emails`)
- Device ID (`device_id`)
- Hashed phone number (`hashed_phone_number`)
- IP address (`ip_address`)
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Sprig
**URL:** https://hightouch.com/docs/destinations/sprig
**Description:** Empower your team to run surveys with AI feedback.
**Section:** Destinations
## Overview
Sprig is a product experience platform that helps teams collect in-context user feedback through surveys, session replays, heatmaps, and AI-powered analysis. Use Hightouch to sync user attributes from your data warehouse to Sprig so you can target studies to the right audiences, filter responses by user properties, and keep your Sprig user profiles enriched with up-to-date data.
## Supported syncing
| Sync type | Description | Supported sync modes | API reference |
| --------- | ----------- | -------------------- | ------------- |
| [**Users**](#syncing-users) | Sync user profiles and attributes to Sprig | Upsert | [V2 Users API](https://docs.sprig.com/reference/post-user-v2-1) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
---
## Connect to Sprig
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Sprig** and click **Continue**.
### API key
To authenticate, you need a Sprig API key. Sprig uses the [Data Import API](https://docs.sprig.com/docs/apis) key for server-side integrations.
To find your API key in Sprig:
1. In the Sprig navigation pane, click **Integrations**.
2. Under **Sources**, click **Enrichment**.
3. Click **Data Import API**.
4. Copy the API key for the environment you want to sync to (Production or Development).
Paste your API key into the **API Key** field in Hightouch and click **Test connection** to verify the credentials.
Sprig provides separate API keys for Production and Development environments.
Make sure you use the key that matches the environment you want to sync data to.
Refer to the [Sprig API documentation](https://docs.sprig.com/docs/apis#find-your-api-keys) for details.
Configure any default sync alerts for this destination, then click **Continue** to finalize.
---
## Sync configuration
Once you've set up your Sprig destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Sprig destination you want to sync to.
### Syncing users
Sync user profiles and attributes to Sprig. Hightouch uses the [Sprig V2 Users API](https://docs.sprig.com/reference/post-user-v2-1) to upsert user records. When a matching user exists in Sprig, Hightouch updates their attributes. When no match is found, Hightouch creates a new user profile.
#### Record matching
To match rows from your model to users in Sprig, select a model column to map to the Sprig **User ID** field. This is the unique identifier Sprig uses to track individual users across your product.
The User ID should be a stable, unique identifier for each user, such as an internal user ID or database primary key. This must match the User ID you use in your [Sprig SDK installation](https://docs.sprig.com/docs/web-javascript) so that synced attributes are associated with the correct user profiles.
#### Field mapping
Hightouch lets you sync user attributes via [field mapping](/syncs/mapping-data). You can map data from any of your model columns to Sprig user attributes.
Sprig supports attributes with string, numeric, and boolean values. Attributes synced through Hightouch appear in Sprig and can be used to:
- **Filter study audiences**: Target surveys and feedback prompts to users with specific attribute values.
- **Filter study responses**: Drill into collected responses based on user attributes.
Any attribute key you sync that doesn't already exist in Sprig is created automatically. Refer to the [Sprig Attributes documentation](https://docs.sprig.com/docs/attributes) for more details on how attributes work.
Sprig does not support deleting individual attributes from a user profile.
If you sync an attribute in error, you can overwrite its value but you cannot remove the key entirely unless you delete the user's data.
---
## Tips and troubleshooting
### Rate limits
Enterprise Sprig customers have an API rate limit of 1,000 requests per second. If you're on a Starter or Free plan, API access may be limited. Refer to the [Sprig API documentation](https://docs.sprig.com/docs/apis#rate-limiting) for current rate limit details.
### Environment mismatch
Make sure the API key you use in Hightouch corresponds to the correct Sprig environment (Production or Development). Using a Development key will sync data to your Development environment, which won't affect your live studies or production user profiles.
### Common errors
| Error | Cause | Fix |
| ----- | ----- | --- |
| `401 Unauthorized` | The API key is invalid or has been revoked. | Generate a new API key in Sprig under **Integrations > Enrichment > Data Import API** and update the destination in Hightouch. |
| `429 Too Many Requests` | You've exceeded the Sprig API rate limit. | Reduce sync frequency or contact Sprig to increase your rate limit. |
### Live debugger
### Sync alerts
---
## SQL Server
**URL:** https://hightouch.com/docs/destinations/sqlserver
**Description:** Power internal tools, in-app experiences, and more
**Section:** Destinations
## Connect to SQL Server
### SSH tunneling
Hightouch can connect directly to SQL Server over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your SQL Server instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's [tunneling documentation](/security/ssh-tunneling).
The [connection string](#connection-string) method doesn't currently support SSH tunneling. Please {" "} if you're interested in this feature.
### Azure firewall setup
By default, Azure protects your SQL Server from any external IP address from connecting to your SQL Server. Follow these steps to add Hightouch's IP addresses to the firewall:
1. Log into your [Azure Console Dashboard](https://portal.azure.com/#home).
2. Click on **SQL databases** under **Services**.
3. Select the SQL database you want to update.
4. Select **Set server firewall** .
5. Under **Public network access**, select **Selected networks**.
6. Scroll to **Firewall rules** and click **Add a firewall rule**.
7. Allowlist the [Hightouch IP addresses](https://hightouch.com/docs/security/networking#ip-addresses) for your region.
8. Click the **Save** button on the bottom left.
#### Required permissions
Both setup options require:
- Server name \(IP address or Azure hostname\)
- Port number (1433 by default)
- Database name
- SQL Server Username
- SQL Server Password
Your credentials must be able to:
- Add/update/delete rows in your sync's table.
- View these tables:
(\*These are used for gathering metadata to set up the sync.)
- `INFORMATION_SCHEMA.COLUMNS`
- `INFORMATION_SCHEMA.TABLE_CONSTRAINTS`
- `INFORMATION_SCHEMA.CONSTRAINT_COLUMN_USAGE`
- `SYS.COLUMNS`
To ensure your credentials are correct, click **Test connection**. This will confirm
if Hightouch is able to properly connect to your database by running a simple `SELECT` query.
#### Password form
#### Connection string
Connection string must be in this format:
`Server=server;Database=database;User Id=username;Password=password;Encrypt=boolean`
## Sync configuration
Hightouch supports **Upsert** mode with the option to delete removed rows.
Select the schema and table you want to sync to. A common schema to use is the database's default, `dbo`.
### Record matching
Records can be matched from your source to your database table using any primary key or unique column. Hightouch only shows columns that are of the `PRIMARY KEY` or `UNIQUE` type in the dropdown.
**Note**: We do not display `IDENTITY` columns as those can't be edited or set
when inserting.
If there are no fields in the dropdown, you must add a primary key or unique constraint to your table in SQL Server. Then, click the refresh icon to access the newly created column.
### Field mapping
Select the fields from your source that you want to sync to SQL Server.
Make sure you account for column types and columns that are **non-nullable**.
If you upsert any row with the wrong column type or set a null value for a
non-nullable column, the entire batch with that row (up to 1k rows) will fail.
## Tips and troubleshooting
### Common errors
{/* */}
#### Cannot open server 'hightouch-test' request by login
{/* */}
If you see the following error message, you need to [update your firewall settings](#azure-firewall-setup) in your Azure Console.
```json
Cannot open server 'hightouch-test' request by login.
Client with IP address '54.196.30.169' is not allowed to access the server.
To enable access, use the Windows Azure Management Portal or run sp_set_firewall_rule
on the master database to create a firewall rule for this IP address or address range.
It may up take up to five minutes for this change to take effect.
```
#### "Invalid time" error
This error occurs when incorrectly formatted data is sent to a [SQL Server `time` field](https://learn.microsoft.com/en-us/sql/t-sql/data-types/time-transact-sql?view=sql-server-ver16).
To resolve this issue, you need to reformat your model column data. For example, if your model column contains values formatted like `hh:mm:ss`, you need to use the [template mapper](/syncs/mapping-data#template-mapping) to change the data's format to:
```
1970-01-01T{{ model.row['column_name'] }}Z
```
Be sure to replace `column_name` with the name of your time column.
SQL Server accepts these types of values and converts them to the `hh:mm:ss.nnnnnnn` format SQL Server expects.
### Live debugger
### Sync alerts
---
## StackAdapt
**URL:** https://hightouch.com/docs/destinations/stackadapt
**Description:** Syncing data from your warehouse to StackAdapt to enable data-driven advertising decisions and optimized targeting by leveraging comprehensive customer insights.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ---------------- | ---------------------------------------------------- | -------------------- | --------------------------------------------------------------------------------------------- |
| **CRM Segments** | Sync data from any source to StackAdapt CRM segments | Add | [Segments docs](https://docs.stackadapt.com/graphql/scenarios/crm-audience-creation-with-pii) |
| **Pixel API** | Sync data from any source to Pixel events | Add | [Pixel API docs](https://docs.stackadapt.com/pixel#http-method) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to StackAdapt
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **StackAdapt** and click **Continue**. You can then authenticate Hightouch to **StackAdapt** by entering the following StackAdapt tokens:
- **Pixel ID**: (Only necessary for using Pixel API). If you don't have a Pixel ID, create one in StackAdapt.
- **GraphQL API token**: (Only necessary for segment syncs). If you don't have an API token, please reach out to your account manager.
## Sync configuration
Once you've set up your StackAdapt destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the StackAdapt destination you want to sync to.
### Syncing segments
Sync data from any source to StackAdapt CRM Segments
#### Record matching
You can match rows from your model to segment in StackAdapt on any column in your model and any field in StackAdapt. Ensure the data types of the model column and StackAdapt field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
StackAdapt accepts the following fields: address, city, email, first name, last name, phone number, state, and zip code. For hashed PII data, you must select one of the following valid combinations of these identifiers:
- email
- first name, last name, email
- first name, last name, phone number
- first name, last name, zip code
If you do not provide at least one of the possible combinations, StackAdapt will be unable to find matches. You can learn more here: [StackAdapt CRM Audience Creation](https://support.stackadapt.com/hc/en-us/articles/360048698853-CRM-Audience-Creation-with-LiveRamp).
## Handling PII and hashing
You can sync data to StackAdapt in one of two formats: either as a list of strings containing hashed PII data or as a list of objects containing users' information.
Currently, StackAdapt only supports SHA_1 as the hashing algorithm. For more information, refer to the [StackAdapt documentation on hashed PII data](https://docs.stackadapt.com/graphql/scenarios/crm-audience-creation-with-pii#option-2-use-hashed-pii-data-and-specify-the-hashing-algorithm).
### Syncing pixel events
Sync data from any source to StackAdapt pixel events
#### Field mapping
Hightouch lets you map model columns to event properties. The required event properties are:
- IP address
- Page title
- User agent
- URL of target website
#### Event args
You can also map additional event data that Hightouch sends to StackAdapt via the args property which can be used for pixel rule matching.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Statsig
**URL:** https://hightouch.com/docs/destinations/statsig
**Description:** Sync to Statsig segments and events from your warehouse to analyze product performance and run experiments.
**Section:** Destinations
## Overview
Sync to segments and events from your warehouse to Statsig to analyze product performance and run experiments.
## Supported syncing
| Type | Description | Supported Sync Modes |
| ------------ | -------------------------------------------------- | -------------------- |
| **Segments** | Sync data from any source to segments in Statsig | Add, Remove |
| **Events** | Sync data from any source to log events in Statsig | Insert |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Statsig
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination**
button. Select **Statsig** and click **Continue**. You can then authenticate Hightouch to **Statsig** by entering your credentials.
Enter the following fields into Hightouch based on the types of syncs you plan on creating:
- **Console API key**: required for syncing segments
- **Client-SDK API key**: required for syncing events
You can find these API keys in the **Keys & Environments** section on your Statsig settings page. Check out Statsig's [Console API docs](https://docs.statsig.com/console-api/introduction) and [HTTP API docs](https://docs.statsig.com/http-api) for more information about how to create and access these keys.
## Sync configuration
Once you've set up your Statsig destination and have a [model](/getting-started/concepts#models) to pull
data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview
page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and
the Statsig destination you want to sync to.
### Segments
Sync data from any source to Statsig [segments](https://docs.statsig.com/console-api/segments). Hightouch only supports syncing to the ID list segment type.
#### Record matching
Hightouch matches rows from your model to Statsig segments by segment **id**. Select the model column that contains these values.
#### Create a segment
To create a new segment list, select **Create a new audience**. You can provide a name for this segment, or Hightouch defaults to the name of the model used to create the sync.
Then, select the **ID type** of the segment list you want Hightouch to create. Hightouch supports user ID and stable ID types.
You can change the ID type to a [custom ID](https://docs.statsig.com/guides/experiment-on-custom-id-types) on the segment's page in Statsig.
#### Use an existing segment
To use an existing segment list, select **Use existing segment**. You can select any of your existing Statsig segments from the drop-down select menu. All available options are of the **ID list** type.
### Events
Sync data from any source to [events](https://docs.statsig.com/guides/logging-events) in Statsig.
Statsig's [event explorer](https://docs.statsig.com/events-explorer) requires you to be on a [Pro Plan](https://www.statsig.com/pricing). You won't be able to view events you send via Hightoucht in the Statsig app without it.
#### Event name
Providing an event name is required to log an event in Statsig. You can either provide a static value or select to **use a column** from your model.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server.
Hightouch converts any `DateTime` values from your source to Unix timestamp as expected by Statsig.
#### Field mapping
You can sync columns from your source to different event and user attributes in the mappings section.
## Tips and troubleshooting
### Rate limits
Statsig has a default rate limit of 10 requests/second and 4500 request/15 minutes on Console API. If you're syncing segments and run into a rate limit error, [schedule](/syncs/schedule-sync-ui) your sync to run less frequently or .
### Common errors
### Live debugger
Hightouch provides complete visibility into the API calls made during each of your sync runs. We recommend reading our
[article on debugging tips and tricks](/syncs/debugger) to learn more.
### Sync alerts
---
## Stripe
**URL:** https://hightouch.com/docs/destinations/stripe
**Description:** Enrich Stripe with data from your warehouse to create, manage, and drive revenue analysis from your customers, products, prices, subscriptions, and invoices resources.
**Section:** Destinations
## Overview
By combining customer data with subscription data from Stripe, you can derive insights into the most common user actions that turn free accounts into paid ones. Additionally, you can figure out which accounts are at risk of churning when you see little to no activity in paid accounts, allowing you to proactively reach out to at-risk customers before the churn.
Check out our [playbook](https://hightouch.com/playbooks/automating-usage-based-billing) and [blog post](https://hightouch.com/blog/automated-billing) to learn how to automate usage-based billing to streamline your finance operations.
## Supported syncing
|Object Type|Description|Supported Sync Modes| API Reference|
|--|--|--|--|
|**Customers**|This object represents a customer of your business. It lets you create recurring charges and track payments that belong to the same customer. |Upsert, Update, Insert|[Customer endpoints](https://stripe.com/docs/api/customers)|
|**Products**|Products describe the specific goods or services you offer to your customers.|Upsert, Update, Insert|[Product endpoints](https://stripe.com/docs/api/products)|
|**Prices**|Prices define the unit cost, currency, and (optional) billing cycle for both recurring and one-time purchases of products. |Upsert, Update, Insert|[Price endpoints](https://stripe.com/docs/api/prices)|
|**Subscriptions**|Subscriptions allow you to charge a customer on a recurring basis.|Upsert, Update, Insert|[Subscription endpoints](https://stripe.com/docs/api/subscriptions)|
|**Invoices**|Invoices are statements of amounts owed by a customer, and are either generated one-off, or generated periodically from a subscription.|Upsert, Update, Insert|[Invoice endpoints](https://stripe.com/docs/api/invoices)|
|**Quotes**|A Quote is a way to model prices that you'd like to provide to a customer. Once accepted, it will automatically create an invoice, subscription or subscription schedule.|Upsert, Update, Insert|[Quote endpoints](https://stripe.com/docs/api/quotes)|
|**Invoice items**|Invoice Items represent the component lines of an invoice.|Upsert, Update, Insert|[Invoice item endpoints](https://stripe.com/docs/api/invoiceitems)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Stripe
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Stripe** and click **Continue**. You can then authenticate Hightouch to Stripe with an API key.
You can retrieve your an API key by following these steps:
1. Log in to [Stripe](https://dashboard.stripe.com/login).
2. Visit your [Account Dashboard Homepage](https://dashboard.stripe.com/dashboard).
3. Copy your **Secret key**.
If you don't see your Secret key on your dashboard homepage, you can also access it under **Developers**.
If you haven't created an API key yet, Stripe requires you to enter your business details on the [Stripe onboarding page](https://dashboard.stripe.com/account/onboarding).
## Sync configuration
Once you've set up your Stripe destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Stripe destination you want to sync to.
Your sync configuration options depend on the Stripe object you are syncing to:
- [Customers](#customers)
- [Products](#products)
- [Prices](#prices)
- [Subscriptions](#subscriptions)
- [Invoices](#invoices)
- [Invoice items](#invoice-items)
- [Quotes](#quotes)
### Customers
#### Record matching
In **Upsert** mode, you can match customer records from your source to your Stripe workspace on the **Customer Email** field.
In **Update** mode, you can match customer records from your source to your Stripe workspace on either **Customer ID** or **Customer Email**.
#### Field mapping
You can sync columns from your source to Stripe default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
Refer to the [inline mapping docs](/syncs/mapping-data#inline-mapping) to see an example of how to map nested objects like a customer's shipping address.
#### Delete behavior
The delete behavior defines how the sync behaves when the Hightouch sync engine recognizes that a record has been removed from your source. The Stripe integration only supports deletions in **Upsert** mode.
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the customer in Stripe |
| **Delete the Stripe record** | Remove the customer from Stripe |
### Products
#### Record matching
You can match rows in your model with products in Stripe on the following fields on the **Product ID** field.
#### Field mapping
You can sync columns from your source to Stripe default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
Refer to the [inline mapping docs](/syncs/mapping-data#inline-mapping) to see an example of how to map nested objects like a products's metadata.
#### Delete behavior
The delete behavior defines how the sync behaves when the Hightouch sync engine recognizes that a record has been removed from your source. The Stripe integration only supports deletions in **Upsert** mode.
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the product in Stripe |
| **Delete the Stripe record** | Remove the product from Stripe |
### Prices
#### Record matching
You can match rows in your model with prices in Stripe on the following fields on the **Price ID** field.
#### Field mapping
You can sync columns from your source to Stripe default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
Refer to the [inline mapping docs](/syncs/mapping-data#inline-mapping) to see an example of how to map nested objects like a price's `recurring` object.
#### Delete behavior
The Stripe integration doesn't suppport deleting prices.
### Subscriptions
#### Record matching
You can match rows in your model with subscriptions in Stripe on the following fields on the **Subscription ID** field.
#### Field mapping
You can sync columns from your source to Stripe default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
Refer to the [inline mapping docs](/syncs/mapping-data#inline-mapping) to see an example of how to map nested arrays like a subscription's items.
#### Associations
Hightouch can perform a reference look-up to associate a subscription to a customer. After selecting the **Customer ID** Stripe field to map, a reference input appears. Similar to the [**Customer** sync type](#customers), Hightouch can conduct a look-up using a **Customer Email**. Hightouch then sends the respective **Customer ID** in the request to Stripe.
If you select **Customer ID**, a look-up is not required.
#### Delete behavior
The delete behavior defines how the sync behaves when the Hightouch sync engine recognizes that a record has been removed from your source. The Stripe integration only supports deletions in **Upsert** mode.
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the price in Stripe |
| **Delete the Stripe record** | Remove the price from Stripe |
### Invoices
#### Record matching
You can match rows in your model with invoices in Stripe on the following fields on the **Invoice ID** field.
#### Field mapping
You can sync columns from your source to Stripe default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
Refer to the [inline mapping docs](/syncs/mapping-data#inline-mapping) to see an example of how to map nested objects like an invoice's `customer address` object.
#### Associations
Hightouch can perform a reference look-up to associate an invoice to a customer. After selecting the **Customer ID** Stripe field to map, a reference input appears. Similar to the [**Customer** sync type](#customers), Hightouch can conduct a look-up using a **Customer Email**. Hightouch then sends the respective **Customer ID** in the request to Stripe.
If you select **Customer ID**, a look-up is not required.
#### Delete behavior
The delete behavior defines how the sync behaves when the Hightouch sync engine recognizes that a record has been removed from your source. The Stripe integration only supports deletions in **Upsert** mode.
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the invoice in Stripe |
| **Delete the Stripe record** | Remove the invoice from Stripe |
### Invoice items
#### Record matching
You can match rows in your model with invoice items in Stripe on the following fields on the **Invoice Item ID** field.
#### Field mapping
You can sync columns from your source to Stripe default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
Refer to the [inline mapping docs](/syncs/mapping-data#inline-mapping) to see an example of how to map nested arrays like invoice items.
#### Associations
Hightouch can perform a reference look-up to associate an invoice item to a customer. After selecting the **Customer ID** Stripe field to map, a reference input appears. Similar to the [**Customer** sync type](#customers), Hightouch can conduct a look-up using a **Customer Email**. Hightouch then sends the respective **Customer ID** in the request to Stripe.
If you select **Customer ID**, a look-up is not required.
#### Delete behavior
The delete behavior defines how the sync behaves when the Hightouch sync engine recognizes that a record has been removed from your source. The Stripe integration only supports deletions in **Upsert** mode.
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the invoice item in Stripe |
| **Delete the Stripe record** | Remove the invoice item from Stripe.* |
*_Deletion is only supported if the associated invoice item is not attached to an invoice or if it's attached to a draft invoice, otherwise, Hightouch will do nothing to the existing invoice item in Stripe._
### Quotes
#### Record matching
You can match rows in your model with quotes in Stripe on the following fields on the **Quote ID** field.
#### Field mapping
You can sync columns from your source to Stripe default and custom fields. If you send data for a custom field that doesn't exist, Hightouch adds the field and automatically detects its type.
Refer to the [inline mapping docs](/syncs/mapping-data#inline-mapping) to see an example of how to map nested objects like a quote's `computed` object.
#### Delete behavior
The delete behavior defines how the sync behaves when the Hightouch sync engine recognizes that a record has been removed from your source. The Stripe integration only supports deletions in **Upsert** mode.
| Behavior | Description |
| -------------- | ---------------------------------------------------------------- |
| **Do nothing** | Keep the quote in Stripe |
| **Delete the Stripe record** | Remove the quote from Stripe |
## Tips and troubleshooting
### Common errors
#### Mismatched or missing data
Some Stripe object types have conditionally required fields based on other field values. You may receive Stripe API errors if the data you are syncing is incorrectly structured or if expected data is missing. We strongly recommend using [Stripe's API documentation](#supported-syncing) as reference when creating your Hightouch models and sync configurations.
If you run into any issues, . We're here to help.
### Live debugger
### Sync alerts
---
## PAR Retail Stuzo Open Commerce
**URL:** https://hightouch.com/docs/destinations/stuzo
**Description:** Stuzo helps Convenience & Fuel Retailers Know and Activate more customers and data in real-time, leading to more visits, more gallons, and bigger baskets.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes
-------------|-----------------------------------------------------------------|---------------------
**Segments** | Sync data from any source to PAR Retail segments | Add, Remove
**Events** | Sync data from any source to track loyalty points in PAR Retail | Insert
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to PAR Retail Open Commerce
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **PAR Retail Open Commerce** and click **Continue**. You can then authenticate Hightouch to **PAR Retail Open Commerce**.
Enter the following fields into Hightouch:
- **API key**: Assigned by Open Commerce and used to identify the communicating application
- **Client ID**: Shared by Open Commerce and used to generate an auth token
- **Client secret**: Shared by Open Commerce and used to generate an auth token
## Sync configuration
Once you've set up your PAR Retail Open Commerce destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the PAR Retail Open Commerce destination you want to sync to.
### Syncing segments
Sync data from any source to PAR Retail Open Commerce segments
#### Record matching
To match rows from your model to segments in PAR Retail Open Commerce, you need to select the model column that contains values that match the **Membership UUID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync segment fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default segment fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Split retries
PAR Retail Open Commerce counts all segments in a batch as rejected if the request contains a single invalid segment. To pinpoint which segments are getting rejected with which errors and reduce the number of valid segments that get retried, you can enable [split retries](/syncs/retries#split-retries).
### Syncing events
Sync data from any source to track loyalty points in PAR Retail Open Commerce
#### Record matching
You can match rows from your model to events in PAR Retail Open Commerce on any column in your model and any field in PAR Retail Open Commerce. Ensure the data types of the model column and PAR Retail Open Commerce field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## SugarCRM
**URL:** https://hightouch.com/docs/destinations/sugar-crm
**Description:** Feed the funnel and build the right pipeline when you choose the marketing automation platform built for marketing mavericks. Create campaigns, engage one-to-one at scale, predict needs, and understand what is working. With Market, you can let the platform do the work.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|-------------|-----------------------------------------------|----------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Objects** | Sync data from any source to SugarCRM objects | Upsert, Update | [Objects docs](https://support.sugarcrm.com/documentation/sugar_developer/sugar_developer_guide_14.1/integration/web_services/rest_api/endpoints/module_post/) [Bulk docs](https://support.sugarcrm.com/documentation/sugar_developer/sugar_developer_guide_14.1/integration/web_services/rest_api/endpoints/bulk_post/) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to SugarCRM
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **SugarCRM** and click **Continue**. You can then authenticate Hightouch to **SugarCRM**.
Enter the following fields into Hightouch:
- **URL** (Required): The base URL of your SugarCRM account
- **Username** (Required): The username of the user authenticating to the system.
- **Password** (Required): The plaintext password the user authenticating to the system.
## Sync configuration
Once you've set up your SugarCRM destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the SugarCRM destination you want to sync to.
### Syncing objects
Sync data from any source to SugarCRM objects. Currently supported objects are:
- **Contacts**
- **Accounts**
- **Opportunities**
#### Record matching
You can match rows from your model to objects in SugarCRM on any column in your model and any field in SugarCRM. Ensure the data types of the model column and SugarCRM field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync contact fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default contact fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
|----------------|---------------------------------------------------------------|
| **Do nothing** | Keep the contact in SugarCRM with all its synced fields |
| **Clear** | Clear all the mapped fields, but keep the contact in SugarCRM |
| **Delete** | Delete the synced object from SugarCRM |
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Taboola
**URL:** https://hightouch.com/docs/destinations/taboola
**Description:** Reach your audiences in their moment of next, when they finish consuming content and are most likely to convert.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| --------------- | --------------------------------------------------------- | -------------------- | -------------------------------------------------------------------------------------------- |
| **Audiences** | Sync data from any model to Taboola first-party audiences | Add, Remove | [Audiences docs](https://developers.taboola.com/backstage-api/reference/add-or-remove-users) |
| **Conversions** | Sync data from any model to Taboola conversions | Add | [Conversion docs](https://help.taboola.com/hc/en-us/article_attachments/7080790629271) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Taboola
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Taboola** and click **Continue**. You can then authenticate Hightouch to **Taboola** by entering the following required fields into Hightouch:
- **Account ID**: Taboola advertiser ID
- **Numeric Account ID**: Unique identifier for your Taboola account
- **Client ID**: Taboola Backstage API Client ID
- **Client Secret**: Taboola Backstage API Client Secret
Taboola provides the account ID, the client ID, and client secret to you during the [API on-boarding](https://developers.taboola.com/backstage-api/reference/welcome) process.
To get access to the Taboola Backstage API, reach out to Taboola's customer success team.
You can find your Taboola numeric account ID in the URL when signed in to Taboola. Example: `https://ads.taboola.com/campaigns?locale=en&accountId=`
## Sync configuration
Once you've set up your Taboola destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Taboola destination you want to sync to.
### Syncing audiences
Sync data from any model to Taboola [first-party audiences](https://developers.taboola.com/backstage-api/reference/onboarding-overview).
#### User identifiers
To identify which users to add or update in an audience, select model columns and the corresponding Taboola fields.
You can match on any of the following Taboola fields:
- **Email**
- **Device ID**
- **Taboola ID**
#### Handling PII and hashing
By default, Hightouch automatically hashes the **Email** fields before sending requests to Taboola.
You can disable this behavior in the sync configuration. If disabled, the data from the model should be appropriately normalized and hashed according to [Taboola's hashing requirements](https://developers.taboola.com/backstage-api/reference/user-id-object).
### Syncing conversions
Sync data from any model to Taboola [server-to-server conversions](https://help.taboola.com/hc/en-us/articles/115006850567-How-to-Track-Conversions-Using-Server-to-Server-Integration-S2S-).
#### Field mapping
Hightouch lets you sync conversion event properties via field mapping.
## Tips and troubleshooting
### Matched users count
Below only applies to the audiences sync type.
### Common errors
### Live debugger
### Sync alerts
---
## Talon.One
**URL:** https://hightouch.com/docs/destinations/talon-one
**Description:** Sync customer data to Talon.One audiences
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
|----------------------|------------------------------------------------------------------------|----------------------|------------------------------------------------------------------------------------|
| **Customer Profile** | Sync data from any source to a Talon.One audience as customer profiles | Upsert | [Customer Profile Audiences](https://docs.talon.one/integration-api#tag/Audiences) |
| **Events** | Sync data from any source to a Talon.One events | Insert | [Events](https://docs.talon.one/integration-api#tag/Events/operation/trackEventV2) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Talon.One
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Talon.One** and click **Continue**.
Enter your Talon.One base URL. To find your base URL, simply navigate to your Talon.One Campaign Manager home page and enter the URL.
This should be in the format `https://.talon.one/`.
You will need an Integration API key and a Management API key to successfully sync data from Hightouch.
If you do not have an Integration API key, follow the instructions [here](https://docs.talon.one/integration-api#section/Authentication/api_key_v1) to create one. If you have multiple Integration API keys, choose one to serve as your default.
If you do not have a Management API key, follow the instructions [here](https://docs.talon.one/management-api#section/Authentication/management_key) to create one. Enable access to the `v1/audiences`, `/v1/customer_search`, and `/v1/attributes`endpoint when creating your Management API key for the optimal experience. If you are connecting multiple applications, ensure this key has access to all of the applications.
Enter both of these API keys into Hightouch, and test that your connection is successful for both APIs.
You can connect multiple Talon.One applications in one Hightouch destination by entering multiple Integration API keys in the **Multiple application configuration** section. Enter an application name and corresponding Integration API key. When configuring your sync configuration, you can choose which application you'd like to sync to.
## Sync configuration
Once you've set up your Talon.One destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Talon.One destination you want to sync to.
### Syncing audiences
#### Audience selection
Select whether you'd like to create a new audience or sync to an existing audience. If you would like to create a new audience, enter the name of the audience in the sync configuration form.
#### Record matching
Talon.One uses `Profile Integration ID` to identify each customer profile. Choose which column in your model contains the Profile Integration ID.
#### Rule Engine
By default, adding customer profiles to Talon.One audiences does not trigger the Rule Engine. If you have defined [rules](https://docs.talon.one/docs/product/rules/overview) in Talon.One that dynamically apply effects to customers, enable this option.
This option is **less performant** than the default option, so you may observe slower sync speeds.
#### Custom attributes mapping
If you have created customer profile custom attributes, you can map to those attribute values in Hightouch. Choose which column in your model should be mapped to each attribute.
Customer profile custom attributes cannot be created in Hightouch. To create new custom attributes, follow the [instructions on Talon.One](https://docs.talon.one/docs/product/account/dev-tools/managing-attributes#creating-a-custom-attribute).
#### Delete behavior
When records leave your query result, they are removed from your audience by default. If you would not like to delete any records from your audience, you can choose to do nothing when records leave your query result.
### Syncing events
Sync data from any source to events in Talon.One.
#### Record matching
You can match rows from your model to events in Talon.One on any column in your model and the **Profile ID** field in Talone.One.
The Talon.One events API requires the **Profile ID** field, so you must map it to complete your configuration.
#### Attributes mapping
Hightouch lets you sync Talon.One event attributes that have already been created in your Talon.One account via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## TikTok
**URL:** https://hightouch.com/docs/destinations/tiktok
**Description:** Empower your marketing team by utilizing web events in TikTok
**Section:** Destinations
## Overview
With Hightouch and the TikTok marketing API, you can send data from your warehouse to events and custom audiences in TikTok.
Deliver relevant ads by utilizing data from various sources within your data warehouse.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| -------------------- | --------------------------------------------------------- | ---------------------- |
| **Web events** | Sync data from any source to TikTok's pixel events. | Insert |
| **Offline events** | Sync data from any source to TikTok's offline events. | Insert |
| **App events** | Sync data from any source to TikTok's app events. | Insert |
| **Custom audiences** | Create audience lists and keep them up-to-date in TikTok. | Add, All (file upload) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to TikTok
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **TikTok** and click **Continue**. You can then authenticate Hightouch to TikTok via OAuth.
Click **Log in to TikTok**, log in to your TikTok business account, and then click **Authorize**.
Once you've finished, Hightouch displays an **Authorization successful** message. To finish connecting, give your TikTok destination a descriptive name.
## Sync configuration
Once you've connected your TikTok destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the TikTok destination you want to sync to.
### Syncing web events
Syncing to TikTok's [web events](https://ads.tiktok.com/help/article?aid=10003669) lets you report different events from your source model.
#### Pixel code
You need to provide your TikTok pixel code to sync web events. To find the pixel code, navigate to your [TikTok Marketing Dashboard](https://ads.tiktok.com)
and go to **Assets menu** > **Manage Web Events**. Select the pixel you want to sync to and copy the ID.
Check out our [blog post](https://hightouch.com/blog/what-is-a-tiktok-pixel)
to learn how to implement a TikTok Pixel to help track conversions.
#### Event name
You can specify the event name or type, by a static value or based on a column in your model. The name must be one of these accepted values:
AddPaymentInfo, AddToCart, AddToWishlist, ClickButton, CompletePayment, CompleteRegistration, Contact, Download, InitiateCheckout,
PlaceAnOrder, Search, SubmitForm, Subscribe, or ViewContent.
#### Field mapping
You can send additional details on your web events such as the timestamp, event (object) properties, and context (customer) properties.
TikTok recommends that you provide as much context information as possible to increase probability of matching
web events with TikTok ads.
If mapping the `Click Id` field, TikTok recommends that you also send the
`Page url` field for best results.
#### Batch size
You can increase the TikTok events API batch size up to 1,000 per call. This can improve your
sync's speed performance. The default is one event per API call. Keep in mind that
TikTok fails an entire batch if it detects an erroneous row. If you suspect
that you could have erroring rows, don't use a high batch size.
#### Test event codes
Test event codes are special codes you can include in your event payload to simulate events. This allows you to test your API implementation without actually recording real event data.
You can optionally enter a test event code as part of your sync configuration. Navigate to **Events Manager** by clicking the **Assets** tab, **Event** and then **Manage Web Events**. You can find your test event code in the **Test Events** tab.
Make sure to remove the test event code if you're running a production sync.
#### PII hashing
TikTok requires customer identity information (email, external_id, and phone_number) to be hashed using a SHA256 hash.
By default, Hightouch automatically hashes these properties for you but you can choose to opt out of this and
hash the identity information yourself.
### Syncing offline events
Syncing to TikTok's [offline events](https://ads.tiktok.com/marketing_api/docs?id=1758427539356673) lets you
track and measure offline activity from people that see or interact with your ads.
Syncing offline events is an allowlist-only feature in TikTok. If you want
access, please contact your TikTok representative. If you initially configured
your TikTok destination before 06/06/23, you need to
[re-authenticate](#connect-to-tiktok) your connection to TikTok.
#### Event set ID
You need to select an event set ID to sync your offline events. You can create a new event set in your TikTok offline events manager dashboard.
#### Event name
You can specify the event name or type, by a static value or based on a column in your model. The name must be one of these accepted values:
AddPaymentInfo, AddToCart, AddToWishlist, ClickButton, CompletePayment, CompleteRegistration, Contact, Download, InitiateCheckout,
PlaceAnOrder, Search, SubmitForm, Subscribe, or ViewContent.
#### Field mapping
You need, at minimum, the event name and at least one user property, emails or phones. The emails and phones array values must include at least one value,
but you can include multiple elements. If you have single email or phone number as strings in a model column, Hightouch converts them into arrays with one element.
#### Batch size
You can increase the TikTok events API batch size up to 100 per call. This can improve your
sync's speed performance. The default is 100 events per API call. Keep in mind that
TikTok fails an entire batch if it detects an erroneous row. If you suspect
that you could have erroring rows, don't use a high batch size.
#### PII hashing
TikTok requires user identity information (emails and phone_numbers) to be hashed using a SHA256 hash.
By default, Hightouch automatically hashes these properties for you but you can choose to opt out of this and
hash the identity information yourself.
### Syncing app events
Syncing to TikTok's [app events](https://business-api.tiktok.com/portal/docs?id=1740857868662786) lets you
track mobile app activity like launching the app or in-app purchases.
Syncing app events is an allowlist-only feature in TikTok. If you want
access, please contact your TikTok representative. If you initially configured
your TikTok destination before 06/06/23, you need to
[re-authenticate](#connect-to-tiktok) your connection to TikTok.
#### TikTok app ID
You need to select an app ID to sync your app events. You can create a new app set in your TikTok ads manager dashboard.
To learn about how to generate a TikTok App ID, see [TikTok App ID](https://ads.tiktok.com/help/article/how-to-generate-a-tiktok-app-id).
#### Field mapping
You can send additional details on your app events such as the timestamp, event properties, and context properties.
TikTok recommends that you provide as much context information as possible to increase the probability of matching
app events with TikTok ads.
#### Batch size
You can increase the TikTok events API batch size up to 100 per call. This can improve your
sync's speed performance. The default is 100 events per API call. Keep in mind that
TikTok fails an entire batch if it detects an erroneous row. If you suspect
that you could have erroring rows, don't use a high batch size.
### Syncing custom audiences
Hightouch lets you create and maintain custom audiences via the [File upload API](https://ads.tiktok.com/marketing_api/docs?id=1739940501023746) or [Segment API](https://ads.tiktok.com/marketing_api/docs?id=1739940504185857).
You can read more on the different APIs and decide which is best for your use case in [TikTok's docs](https://ads.tiktok.com/marketing_api/docs?id=1739566528222210).
{/* */}
#### Segment API mode (Recommended)
{/* */}
TikTok lets you make real-time batch updates through their [Segment API](https://ads.tiktok.com/marketing_api/docs?id=1739940504185857). Hightouch sends data to the selected audience list
as data gets added in your source model. We recommend this integration type because the API is newer and the integration is maintained by our team.
#### User identifiers
To identify which users to add or update in an audience, select model columns and the corresponding Tiktok fields. You can match on any of the following hashed user fields:
- IDFA_SHA256
- IDFA_MD5
- GAID_SHA256
- GAID_MD5
- EMAIL_SHA256
- PHONE_SHA256
Hightouch automatically hashes the selected properties for you unless you specify otherwise.
{/* */}
#### File upload API mode (Deprecated)
{/* */}
While TikTok still supports this API, Hightouch has deprecated the
integration. We no longer add enhancements to the File upload API, and by end
of 2023, we will programmatically migrate existing syncs to the Segment API
mode and fully sunset this functionality.
TikTok lets you upload an entire audience through their [file upload API](https://ads.tiktok.com/marketing_api/docs?id=1739940567842818). Hightouch uploads the entire model query results for each sync run.
These results effectively replaces the previous file upload.
#### Record matching
The File upload mode lets you match on any one of these hashed user properties:
- IDFA_SHA256
- IDFA_MD5
- GAID_SHA256
- GAID_MD5
- EMAIL_SHA256
- or PHONE_SHA256
The property contains the name of what type of hash TikTok expects. Hightouch
automatically hashes the selected property for you unless you specify otherwise. Hightouch also creates the audience list,
if it doesn't exist yet, with the selected property's hash type.
#### Advertiser account
For both modes, you need to select the advertiser account you want to sync to.
Editing the advertiser account in the sync configuration is only possible
before triggering the first sync run. Otherwise, making this change results in
an error. If you want to use a different advertiser account once the sync has
already run, you need to create a new sync.
#### Custom audience list
Hightouch lets you use an existing audience in TikTok or create a new one for both modes. Note that if you're using Segment mode, you can't
change your audience list name once your sync has run, instead, you will have to create a new sync.
## Tips and troubleshooting
### Matched users count
Below only applies to the custom audiences sync type.
### Common errors
#### Custom audiences
If you want to switch your custom audience sync mode from [File upload API](#file-upload-api-mode-deprecated) to [Segment API](#segment-api-mode-recommended), you need to run a [full resync](/syncs/overview#resync-full-query) after saving the new configuration.
#### Error 40001: No permission to operate advertiser
This error can happen if the TikTok user credentials used to [connect to TikTok](#connect-to-tiktok) don't have the necessary permissions to send data to the [advertiser account](#advertiser-account) selected in the sync configuration. When this error happens, you might also see that the advertiser account dropdown menu in the sync configuration is empty.
To solve this, make sure that the credentials used to connect TikTok to Hightouch have the necessary permissions to send data to your advertiser accounts.
#### File upload API mode
Audiences with fewer than 1000 users are invalid and won't be available in TikTok. For more information, please refer to [TikTok's documentation](https://ads.tiktok.com/help/article?aid=9611).
TikTok allows one file-based audience upload per day per audience. See TikTok's [docs](https://ads.tiktok.com/marketing_api/docs?id=1708580518247426) for more information on their limits.
#### Web events
Most errors that occur with web event syncs are validation errors, like an invalid currency code or phone number. TikTok rejects any requests with invalid data, so Hightouch preemptively rejects these rows to make syncs run more efficiently. Please refer to TikTok's [documentation](https://ads.tiktok.com/marketing_api/docs?id=1740858565852225) to learn more about the expected phone number format and currency codes.
TikTok may update their currency code list.
### Live debugger
Hightouch provides complete visibility into the API calls made during each of your TikTok sync runs, _except for syncs in custom audience file upload mode_. In this mode, Hightouch syncs the full query results set, and therefore doesn't make a [change data capture computation](/getting-started/concepts#change-data-capture). As a result, you won't see any payload details in the [Live debugger](/syncs/debugger#inspecting-payloads).
For the web event and custom audience segment mode syncs, we recommend reading our [article on debugging tips and tricks](/syncs/debugger) to learn more.
### Sync alerts
---
## Topsort
**URL:** https://hightouch.com/docs/destinations/topsort
**Description:** Use Topsort to quickly turn your marketplace into a high-performing ad revenue engine with AI-driven, privacy-first infrastructure.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|--------------|-----------------------------------------------|----------------------|-----------------------------------------------------------------------------|
| **Segments** | Sync data from any source to Topsort segments | Add, Remove | [Segments docs](https://docs.topsort.com/en/api-reference/segments-service) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Topsort
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Topsort** and click **Continue**. You can then authenticate Hightouch to **Topsort**.
Enter the following fields into Hightouch:
- **API key**: To obtain your API key, refer to the Topsort [API documentation](https://docs.topsort.com/en/api-reference/authentication).
## Sync configuration
Once you've set up your Topsort destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Topsort destination you want to sync to.
### Syncing segments
Sync data from any source to Topsort segments.
#### Record matching
To match rows from your model to segments in Topsort, you need to select the model column that contains values that match the **User ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Totango
**URL:** https://hightouch.com/docs/destinations/totango
**Description:** Enrich your customer success platform with rich customer data from your warehouse to improve product adoption and reduce churn
**Section:** Destinations
## Setup
To configure your Totango destination you will need to provide the following information: Service ID and API Key.
### Service ID
To find your Service ID, you'll need to go to the upper right hand corner of the Totango dashboard, which is your profile picture. It will be listed at the bottom of the small context window that pops up.
Your service ID is listed as a string which we only need one part. In the screenshot above, the full service ID is 'SP-229775-0'. We will only need the middle part of the ID to authenticate with Totango. Using the example provided, you would only need to use '229775' when filling out the destination details on the Hightouch destination setup page.
### API key
To find your API Key, you'll need to go back to your profile picture.
When you click it, you will want to go to 'Edit profile'.
Once in your profile you'll want to go to the 'Integrations' page.
On the Integrations you will find your API Token Key. You will use this key when filling out the destination details on the Hightouch destination setup page.
## Supported modes
The Totango destination supports the upsert mode:
- **Upsert**: Upsert mode pushes new accounts or account users to Totango and updates fields that change in your warehouse.
---
## The Trade Desk
**URL:** https://hightouch.com/docs/destinations/tradedesk
**Description:** Sync audiences and conversion events to The Trade Desk using data from your warehouse for first-party optimization, third-party monetization, and retail media use-cases.
**Section:** Destinations
## Overview
Power your The Trade Desk platform with up-to-date data straight from your data warehouse. Sync first-party data or use highly customizable postback URLs to handle data your way. Keeping your data updated equips your team with the necessary tools they need to deliver a top of the line advertising experience.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------------------------- | ----------------------------------------------------------------------------------------------------- | -------------------- |
| **Postback events** | Sync data from any source to The Trade Desk via a postback URL | Insert |
| **Real-time conversion events** | Sync data from any source to The Trade Desk as real-time conversion events | Insert |
| **Offline conversion events** | Sync data from any source to The Trade Desk as offline conversion events | Insert |
| **CRM data segments** | Sync your Customer relationship management (CRM) data from any source to The Trade Desk data segments | Add, Remove |
| **CRM data conversions** | Sync query results as conversions to CRM Data segments | Insert |
| **First-party data segments** | Sync your first-party data from any source to The Trade Desk first-party data segments | Add |
| **Third-party data segments** | Sync your third-party data from any source to The Trade Desk third-party data segments | Add |
| **Event opt-out or deletion** | Opt-out or fully delete customer data from The Trade Desk | Insert |
## Getting started
You need to enter different credentials depending on whether you're syncing events, CRM data segments, or first-party data segments. If you plan to use multiple sync types, enter the necessary credentials for all of them. Entering all the required credentials lets you use the same destination setup for every Trade Desk sync.
Once you have entered your inputs, press the **Test Connection** button to check you have inputted valid credentials.
### Postback events credentials
For Hightouch to sync postback events, you need to enter a root postback URL.
### Conversion events credentials
For Hightouch to sync real-time conversion events, you need the following:
- Advertiser ID
### Offline conversion events credentials
For Hightouch to sync offline conversion events, you need the following:
- Advertiser ID
- Offline data provider ID
- Offline data provider secret key
If you would like to create offline tracking tags through Hightouch, an **API token** is also required.
### CRM data segment credentials
For Hightouch to sync data to your CRM data segments, you need the following:
- Advertiser ID
- Environment - select either your `Production` or `Sandbox` environment
- API token
You can create an API token by logging onto The Trade Desk platform using your API credentials. These API credentials can be provided by your account manager. Navigate to the **Manage API Tokens** option under the user profile menu at the top-right of The Trade Desk UI. Then either create a Long- or Short-Lived API token. Once you have copied the API token, navigate back to Hightouch and paste the API token.
### First-party data segment credentials
For Hightouch to sync data to your first-party data segments, you need the following:
- Advertiser ID
- Environment - select a server location of your choice
- Advertiser secret key
You can obtain your decoded advertiser secret key for uploading first-party data from your advertiser preferences page in The Trade Desk UI. If you can't find your secret key, contact your Account Manager. When you send data, the included secret key is validated against the secret key the platform has on file for the Advertiser ID.
### Third-party data segment credentials
For Hightouch to sync data to your third-party data segments, you need the following:
- Data provider ID
- Environment - select a server location of your choice
- Data provider secret key
You can obtain your decoded data provider secret key for uploading third-party data from your advertiser preferences page in The Trade Desk UI. If you can't find your secret key, contact your Account Manager. When you send data, the included secret key is validated against the secret key the platform has on file for the data provider ID.
### Event opt-out or deletion credentials
For Hightouch to sync opt-out or deletion requests, you need different credentials depending on the data type:
- **First-party data (Advertiser)**: Requires the Advertiser ID and Advertiser secret key from your first-party data credentials
- **Third-party data**: Requires the Data provider ID and Data provider secret key from your third-party data credentials
- **Merchant data**: Requires the Data provider ID and Data provider secret key from your third-party data credentials
When setting up for first-party data segments, upload your data to the
hostname closest to the location of the users targeted by your campaign. Using
the closest hostname reduces latency. After a short delay, Hightouch
replicates the data to other data centers globally. Please don't upload the
same data to more than one data center. Certain data types have exceptions
including how they replicate to countries with specific privacy regulations.
Refer to [The Trade Desk
docs](https://api.thetradedesk.com/v3/portal/data/doc/DataEnvironments) for
more details.
## Sync configuration
### Syncing postback events
Hightouch supports inserting impression data as events to The Trade Desk.
#### Field mapping
You can sync columns from your source to The Trade Desk event custom fields.
### Syncing real-time conversion events
Sync data from any source to The Trade Desk Ads as real-time conversion events.
#### Field mapping
Hightouch lets you sync event properties via field mappings.
Hightouch provides the ability to convert hashed emails and phone identifiers to UID2 identifiers during sync execution. UID2 values will be sent as the `adid` value on your event. See [ID Express](/destinations/tradedesk#id-express) for details.
#### Custom mappings
You can also map additional custom data using The Trade Desk's ten sequentially numbered custom dynamic properties to provide additional conversion metadata (`t1`-`t10`).
### ID Express
ID Express is available to all Customer Studio and Match Booster customers. If you do not have access to this feature, contact your rep to inquire about eligibility.
ID Express is a one-click solution that enables Hightouch customers to convert hashed emails and phone numbers into Unified ID 2.0 (UID2) identifiers for privacy-conscious audience targeting and conversion tracking through The Trade Desk.
#### Using ID Express
To use ID Express, map the appropriate email or phone number fields to the ID Express mappings in the configuration interface:
- `Hashed email → UID2`
- `Hashed phone → UID2`
Hightouch uses hashed identifiers when translating to UID2. If your email or phone number attributes are not hashed already, Hightouch automatically detects unhashed data and hashes them for you.
You can choose to map one or both fields to maximize match rates. You can also map other non-PII identifiers alongside these fields for additional reach, including Mobile Ad IDs (MAIDs) and The Trade Desk IDs (TTIDs).
#### How ID Express works
At sync time, Hightouch translates the hashed email and phone number values that you provide into UID2s before sending them to The Trade Desk. Hightouch then shares only the UID2 identifiers downstream with The Trade Desk, excluding the email and phone data.
If non-PII identifiers like MAIDs and TDIDs are also mapped, Hightouch recombines those identifiers with the UID2s and sends all of them to The Trade Desk to maximize match rates.
The Trade Desk then processes the UID2s and other IDs and makes them available as audiences and conversion events. Audiences can then be used to target and suppress devices across the open internet, and conversion events can be used to measure campaign results and optimize ad performance by providing better data to Kokai’s AI systems.
### Syncing offline conversion events
Sync data from any source to The Trade Desk as offline conversion events.
#### Offline conversion event types
Select which offline conversion event type you'd like to send to The Trade Desk:
- First party: these events are used in retargeting, campaign optimization, and measurement.
- Third party: send events to an advertising partner’s seat in The Trade Desk. You can set your conversion feed, CPM rate, and advertiser access for self-service monetization as well.
- Merchant: these events are used for retail use-cases, including SKU-level reporting for retailers and monetization by sharing conversion-based audience and measurement feeds
For **merchant** events, `Order ID` is a required field for each event. You also have the ability to include **line items** with each event.
For **third party** events, provide the advertiser ID you'd like to share these events with in the sync configuration. You can also set the CPM rate and media cost for these events in the sync configuration.
#### Offline tracking tags
An [offline tracking tag](https://partner.thetradedesk.com/v3/portal/data/doc/DataOfflineMeasurement#step1) is required for each offline conversion event in The Trade Desk. This offline tracking tag must be associated with the same offline data provider ID that is in the destination configuration.
Select whether you'd like to use the same tracking tag for all rows in your sync, or map the tracking tag ID from a column in your source.
You can optionally create a new offline tracking tag with the former option. Provide the name for your new offline tracking tag, and ensure that you have provided an API token in your destination configuration.
If you choose to create a new tracking tag or data rate through Hightouch, the first run of the sync will **fail**. This is intentional. The Trade Desk will not attribute your data to your segment until you assign a campaign to the created offline tracking tag.
After the first failed sync run, navigate to The Trade Desk UI to assign a campaign to the created tracking tag before attempting to run the sync again.
The data in your model will be synced to the segment on the next sync run.
The Trade Desk processes new data rates nightly. For best performance, we recommend waiting until the data rate processes before running the sync again.
#### Field mapping
At least one ID and the timestamp is required for each offline event. If you provide an `ImpressionId`, no other IDs are required.
Hightouch supports the following ID types for offline conversion events:
| ID Type | Data Type | Description |
|----------|------------|-------------|
| **TDID** | GUID | The Trade Desk 36-character GUID (including dashes) for this user. |
| **DAID** | GUID | The raw device ID for this user, sent in 36-character GUID format (including dashes). Use iOS IDFA or Android’s AAID. |
| **UID2** | String | The raw UID2 value, also known as UID2. This value is case-sensitive. Raw UID2s are generated and managed using UID2 APIs. |
| **UID2Token** | String | The encrypted UID2 token, also known as an advertising token. This token is case-sensitive. |
| **EUID** | String | The raw European Unified ID value (EUID), which offers user transparency and privacy controls designed for European regions. |
| **EUIDToken** | String | The encrypted EUID token, also known as an advertising token. |
| **RampID** | String | The 49-character or 70-character RampID (previously known as IdentityLink). Must be a RampID from LiveRamp mapped specifically for The Trade Desk. |
| **Hashed email → UID2** | String | A SHA-256–hashed email address that Hightouch automatically converts to a UID2 identifier during sync execution (via [ID Express](/destinations/tradedesk#id-express)). |
| **Hashed phone → UID2** | String | A SHA-256–hashed phone number that Hightouch automatically converts to a UID2 identifier during sync execution (via [ID Express)](/destinations/tradedesk#id-express). |
Hightouch provides the ability to convert hashed emails and phone identifiers to UID2 identifiers during sync execution. UID2 values will be sent as the `adid` value on your event. See [ID Express](/destinations/tradedesk#id-express) for details.
#### Custom mappings
You can also map additional custom data using The Trade Desk's ten sequentially numbered custom dynamic properties to provide additional conversion metadata (`t1`-`t10`).
### Syncing CRM data segments
Hightouch supports inserting and removing your data to your CRM data segments in The Trade Desk. CRM data is first-party data that includes all client information collected and shared across your organization.
#### User identifiers
To identify which users to add or update in an CRM data segment, select model columns and the corresponding Trade Desk fields. You can match on any of the following Trade Desk fields:
- Email
- Email (SHA256)
- Phone
- Phone (SHA256)
#### Creating a new segment
In your sync configuration, you can tell Hightouch to create a new data segment for you. You can optionally assign a custom name to your segment. If no name is provided, Hightouch defaults to using your model name.
You also need to select which region you would like to create your CRM data segment in. This is based on the origin of the PII.
#### Using an existing segment
Additionally, instead of creating a new data segment Hightouch can also sync users to existing segments that are already populated in your The Trade Desk account.
#### PII hashing
You can enable PII hashing if you wish for Hightouch to detect whether your data requires normalization and hashing based on the chosen record mapping. For example, if you select `Email (SHA256)` and the model column selected with the email data is unhashed, Hightouch hashes the value for you.
#### TTL
Data synced into The Trade Desk has a Time-To-Live (TTL) value. This is the amount of time the data remains active in the segment. The default recommended value by The Trade Desk is 90 days. If you wish to customize this interval, you can choose a column or enter the number of days (between 0 and 180) you want your data to live in the segment for.
Setting this value to 0 will mark synced rows for deletion. If all rows in your segment have a TTL of 0, your segment will be deleted, and subsequent sync runs may error.
#### Full Resync interval
This is the interval Hightouch will use to schedule a full resync of your data to The Trade Desk. The Trade Desk recommends a full data refresh every 30 days, however you can choose to retain hashed data for up to 90 days or process it once.
### Syncing CRM data conversions
Hightouch supports syncing up-to-date query results as conversions to CRM Data segments in The Trade Desk. This sync type is designed specifically for conversion tracking and differs from CRM data segments by requiring a timestamp for each conversion event and only supporting insert operations.
#### User identifiers
To identify which users to send conversion events for, select model columns and the corresponding Trade Desk fields. You can match on any of the following Trade Desk fields:
- Email
- Email (SHA256)
- Phone
- Phone (SHA256)
#### Creating a new conversion segment
In your sync configuration, you can tell Hightouch to create a new conversion segment for you. You can optionally assign a custom name to your segment. If no name is provided, Hightouch defaults to using your model name.
You also need to select which region you would like to create your CRM data segment in. This is based on the origin of the PII.
If you choose to create a new CRM conversions segment through Hightouch, the first run of the sync will **fail**. This is intentional. The Trade Desk will not attribute your data to your segment until you assign a campaign to the created tracking tag.
After the first failed sync run, navigate to The Trade Desk UI to assign a campaign to the created tracking tag before attempting to run the sync again.
The data in your model will be synced to the segment on the next sync run.
#### Using an existing conversion segment
Instead of creating a new conversion segment, Hightouch can also sync conversion events to existing conversion segments already populated in your The Trade Desk account.
#### PII hashing
You can enable PII hashing if you wish for Hightouch to detect whether your data requires normalization and hashing based on the chosen record mapping. For example, if you select `Email (SHA256)` and the model column selected with the email data is unhashed, Hightouch hashes the value for you.
#### Field mappings
CRM data conversions require a timestamp for each conversion event. You can also include additional conversion parameters:
| Field | Data Type | Description |
|-------|-----------|-------------|
| **Timestamp** | Timestamp | Required. The timestamp of the conversion event. |
| **Value** | Number | The monetary value of the conversion. |
| **Currency** | String | The currency code for the conversion value (e.g., USD, EUR). |
| **Country** | String | The country where the conversion occurred. |
| **Region** | String | The region/state where the conversion occurred. |
| **Metro** | Number | The metro code where the conversion occurred. |
| **City** | String | The city where the conversion occurred. |
| **Order ID** | String | A unique identifier for the order/conversion. |
| **Privacy type** | Enum | The privacy regulation type: GDPR or GPP. |
| **Is applicable** | Boolean | Whether the privacy settings apply to this conversion. |
| **Consent string** | String | The consent string for privacy compliance. |
#### Custom mappings
You can also map additional custom data using The Trade Desk's ten sequentially numbered custom dynamic properties to provide additional conversion metadata (`TD1`-`TD10`).
### First and third party data segments
Hightouch supports inserting your data to first and third party data segments in The Trade Desk.
#### Field mappings
You can sync columns from your model to The Trade Desk on the following fields:
- Trade Desk GUID: The Trade Desk 36-character GUID (including dashes) for a user.
- Raw Device ID: the raw device ID for this user, sent in 36-character GUID format, including dashes. Use iOS IDFA or Android's AAID.
- UID2: The 256-bit UID2 as a base64-encoded string.
- UID2Token: The encrypted UID2 token.
- EUID: The user's European Unified ID as a 44-character base64-encoded SHA-256 string. This ID is case-sensitive.
- EUIDToken String The encrypted EUID token.
- RampID: The 49-character or 70-character RampID, previously known as IdentityLink. This must be a RampID from LiveRamp that is mapped specifically for The Trade Desk. For details about mapping a RampID, see [LiveRamp documentation](https://sidecar.readme.io/docs/getting-started).
- ID5: Data providers holding these IDs should engage with ID5 directly to obtain the necessary decryption keys. For details see the [ID5 site](https://id5.io/). **NOTE: The ID5 ID must be in decrypted form.**
- netID: A 70-character base64-encoded string. For details see the [netID Developer Portal](https://developerzone.netid.dev/latest/devportal/).
- FirstId: A first-party cookie typically set by publishers in France. For details see the [First-id site](https://www.first-id.fr/).
- Hashed email → UID2: A SHA-256–hashed email address that Hightouch automatically converts into a Unified ID 2.0 (UID2) identifier during sync execution. If your email attribute is unhashed, Hightouch hashes it before translation.
- Hashed phone → UID2: A SHA-256–hashed phone number that Hightouch automatically converts into a Unified ID 2.0 (UID2) identifier during sync execution. If your phone attribute is unhashed, Hightouch hashes it before translation.
For more information on how Hightouch converts hashed identifiers to UID2, see [ID Express](/destinations/tradedesk#id-express).
#### Creating a new segment
In your sync configuration, you can tell Hightouch to create a new data segment for you. You can optionally assign a custom name to your segment. If no name is provided, Hightouch defaults to using your model name.
#### Using an existing segment
Instead of creating a new data segment Hightouch can also sync users to existing segments already populated in your The Trade Desk account.
#### Full resync interval
Data synced into The Trade Desk has a Time-To-Live (TTL) value. This is the amount of time the data remains active in the segment. The default value recommended value by The Trade Desk is 30 days. If you wish to customize this interval, enter the number of days you want your data to live in the segment for. After it expires, Hightouch triggers a full resync on your next sync run to keep your data refreshed in The Trade Desk.
#### Data taxonomy
When creating a new third party segment, you can create a corresponding element in your third-party data taxonomy in The TradeDesk.
To create a new data element, you will have to update your **destination** configuration with a long-lived API token.
Provide the following details when creating a new data element:
- Provider Element ID: the identifier for your segment. Hightouch will default to the **sync ID** by default.
- Parent Element ID: the identifier of the folder where this segment should live. Defaults to Root.
- Display name
- Description
You can optionally also provide a data rate for this segment to make it available to partners or advertisers.
You will need your brand ID. Contact your rep at The TradeDesk if you'd like to use this feature for your segment.
Hightouch will confirm the data segment and data rate batch are approved, but will **not** wait for processing to complete. We recommend checking the status of your segment and data rate batch in The TradeDesk UI to confirm the completion.
### Syncing event opt-out or deletion
Hightouch supports opt-out or full deletion of customer data from The Trade Desk. This sync type allows you to manage customer privacy requests by either opting out customer IDs from being used in campaigns or completely deleting their data from your seat in The Trade Desk.
#### Data types
Select which type of data you'd like to send opt-out or deletion requests for:
- **First-party data (Advertiser)**: Opt-out or delete customer data from first-party advertiser campaigns
- **Third-party data**: Opt-out or delete customer data from third-party data segments
- **Merchant data**: Opt-out or delete customer data from merchant campaigns
#### Required configuration
Depending on the data type you select, you'll need to provide additional configuration:
- **Third-party data**: `Brand ID` is required
- **Merchant data**: `Merchant ID` is required
- **First-party data**: Uses the `Advertiser ID` from your destination configuration
#### User identifiers
Configure how the columns in your query results should be mapped to ID fields for opt-out or deletion requests. You can map to the following Trade Desk ID fields:
- `Trade Desk GUID`
- `Raw Device ID`
- `UID2`
- `UID2Token`
- `EUID`
- `EUIDToken`
- `RampID`
- `ID5`
- `netID`
- `FirstId`
#### Opt-out vs. full deletion
By default, this sync type performs an opt-out operation, which prevents the customer's data from being used in campaigns while retaining the data in your datasets.
If you enable the **full delete** option, Hightouch will delete the customer's ID from your datasets entirely, including:
- Data segments
- REDS feeds
- All other associated data in The Trade Desk
Full deletion is irreversible. Once customer data is deleted, it cannot be recovered. Use this option carefully when processing deletion requests to comply with privacy regulations like GDPR or CCPA.
#### Sync mode
This sync type only supports insert mode. Each sync run will process the rows in your model as new opt-out or deletion requests.
## Tips and troubleshooting
### Matched users count
Below only applies to the CRM data segment sync type.
Hightouch retrieves the audience metadata from The Trade Desk in real-time. The matched user
count reflects the active IDs based on a **7-day** look back window where the ID came up for bidding.
### Common errors
### Live debugger
### Sync alerts
---
## Twilio
**URL:** https://hightouch.com/docs/destinations/twilio
**Description:** Send SMS updates to your customers in real time.
**Section:** Destinations
## Overview
With the Twilio SMS destination, Hightouch can serve as a notifier for various changes. This is a very flexible integration, but here are a few example use cases:
1. Send 2FA codes
2. Send usage notifications and alerts
3. Sending password reset links
## Sending messages \(added / changed / removed\)
Hightouch will not send any SMS messages on the initial run to prevent a flood
of messages to your channel. Messages will start after this initial sync.
Hightouch supports sending messages based on rows that have been added, changed, or removed since the last sync run. Each row is sent as an individual SMS message and templated with Liquid.
### Recipient and Sender selection
Hightouch supports both sending messages to specific static recipients, or a phone number in each row. Use the mapper to map rows or static numbers to the `To` and `From` fields.
#### Messaging Service SID
You can also enter or select a Twilio Messaging Service SID in the mapping field.
The Messaging Service is setup inside [Twilio](https://www.twilio.com/docs/messaging/services#send-a-message-with-a-messaging-service). It allows you to send messages from a phone number pool which is useful when you need larger message quotas.
### Phone number cleaning
Hightouch can attempt to clean phone numbers in your data to ensure they are formatted correctly in [E.164](http://en.wikipedia.org/wiki/E.164) format.
When cleaning phone numbers, you will need to provide a 2 letter [ISO code](https://en.wikipedia.org/wiki/ISO_3166-2) for the country the numbers belong to.
### Message content
#### Added, changed, and removed phone numbers
For each sms, Hightouch allows you template the content you would like to send in your message (required). The row content is repeated per row in the message and has access to the row values via templating.
- When a row is added in your query results, Hightouch will send the row added content.
- When a row is changed in your query results, Hightouch will send the row changed content.
- When a row is deleted in your query results, Hightouch will send the row deleted content.
This content can be [templated](#liquid-templating).
## Templating messages
### Liquid templating
For both the message modes, [Liquid templating ](https://shopify.github.io/liquid/)is supported. The most important implication of this is that columns from the added or removed row can be accessed as part of the message body. Here is an example message making use of Liquid to insert a column into the message.
```text
Welcome to the platform {{ row["name"] }}, here is your verification code {{ row["2fa_code"] }}.
```
---
## Unify
**URL:** https://hightouch.com/docs/destinations/unify
**Description:** Capture intent data, run AI agents, and engage every prospect with personalized outbound using Unify.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
------------|--------------------------------------------|----------------------|------------------------------------------------------------------------------
**Records** | Sync data from any source to Unify records | Upsert, Insert | [Records docs](https://unify-19-preview.mintlify.app/developers/introduction)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Unify
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Unify** and click **Continue**. You can then authenticate Hightouch to **Unify**.
Enter the following fields into Hightouch:
- **API key**: Generate an API key in Unify by navigating to Settings -> API Keys.
## Sync configuration
Once you've set up your Unify destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Unify destination you want to sync to.
### Syncing records
Sync data from any source to Unify records. Select an object type to sync those types of records.
#### Record matching
You can match rows from your model to object records in Unify on any column in your model to the object type's unique field in Unify. Ensure the data types of the model column and Unify field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, Unify automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync object record attributes via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the object's record attributes.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|----------------------------------------------------
**Do nothing** | Keep the record in Unify with all its synced fields
**Delete** | Delete the synced records from Unify
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Upland Waterfall
**URL:** https://hightouch.com/docs/destinations/upland-waterfall
**Description:** Elevate your customer experience and drive higher levels of engagement using personalised, targeted SMS/MMS mobile messaging with Waterfall.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
| ---------- | -----------------------------------------------------| ---------------------- | ------------------------------------------------------------------------------------------------------- |
| **Subscribers** | Sync data from any source to an Upland Waterfall list as subscribers | Upsert | [Subscriber Lists](https://apidocs.waterfall.com/#introduction)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Upland Waterfall
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Upland Waterfall** and click **Continue**.
Enter your API key from your Upland Waterfall account.
If you do not have an Upland Waterfall API key, contact your Upland Waterfall account manager.
You can optionally provide a default list ID. This will be the default list to add subscribers to for all syncs created with this destination.
## Sync configuration
Once you've set up your Upland Waterfall destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Upland Waterfall destination you want to sync to.
### Sync mode
- **Upsert**: pushes new objects to Upland Waterfall and updates mapped fields that change in your source.
### List selection
Select which list in your Upland Waterfall account you'd like to upsert subscribers to. If you configured a default list ID in your destination, selecting a list is not required in the sync configuration.
### Field mapping
Hightouch will fetch all metadata fields from Upland Waterfall as mapping options. You can map any field from your source to a metadata field with the inline mapper.
### Record matching
Upland Waterfall requires all subscribers to have a `msisdn`. You must select a column in your model to map to this field.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## User Interviews
**URL:** https://hightouch.com/docs/destinations/user-interviews
**Description:** Gain deeper understanding of your custom participants with User Interviews
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
-----------------|-----------------------------------------------------------|------------------------|--------------------------------------------------------------------------------------------------
**Participants** | Sync data from any source to User Interviews participants | Upsert, Update, Insert | [Participants endpoint](https://api-docs.userinterviews.com/reference/patch_api-participants-batches)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to User Interviews
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **User Interviews** and click **Continue**. You can then authenticate Hightouch to **User Interviews** by entering the following required fields into Hightouch:
- **API key**: The User Interviews API key to authenticate requests. See [obtaining an API key](https://api-docs.userinterviews.com/reference/authentication#obtaining-an-api-key) for information on how to get your API key.
- **Environment**: **Sandbox** or **Production**.
## Sync configuration
Once you've set up your User Interviews destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the User Interviews destination you want to sync to.
### Syncing participants
Update existing participants' fields and characeristics, insert new participants, or do both with upsert mode.
#### Record matching
To match rows from your model to participant in User Interviews, you need to select the model column that contains values that match the particpants' **email** field. Refer to the [record matching docs](/syncs/record-matching) for more information.
In **Insert** mode, User Interviews automatically generates an identifier for every new record synced, so there is no need to match an existing record.
#### Field mapping
Hightouch lets you sync participant fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default participant fields and characteristics.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
**Delete** is only available for **Upsert** mode and **Clear** is available for
**Upsert** and **Update** mode.
Behavior | Description
---------------|-------------------------------------------------------------------------
**Do nothing** | Keep the participant in User Interviews with all its synced fields
**Clear** | Clear all the mapped fields, but keep the participant in User Interviews
**Delete** | Delete the synced participants from User Interviews
#### Split retries
User Interviews counts all participants in a batch as rejected if the request contains a single invalid participant. To pinpoint which participants are getting rejected with which errors and reduce the number of valid participants that get retried, you can enable [split retries](/syncs/retries#split-retries).
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Userflow
**URL:** https://hightouch.com/docs/destinations/userflow
**Description:** Sync your freshest data to Userflow to drastically improve your user onboarding
**Section:** Destinations
## Setup
Upon selecting Userflow as your destination you will be prompted to
input your Userflow API key.
You can locate this API key within your Userflow account by navigating to _Settings > API_:
## Syncing
Hightouch supports syncing the following Userflow objects:
- `Users`
- `Groups`
### Sync modes
- **Upsert**: pushes new objects to Userflow _and_ updates fields that change in your warehouse.
- **Update**: only updates particular fields on existing objects in Userflow. It does _not_ add new objects.
### Record matching
Records can be matched from your source to your Userflow workspace by the **user ID** or **group ID**, depending on the object you are syncing.
### Field mapping
You may also sync columns from your source to Userflow's object properties. You have the option
of mapping to attributes that already exist on the Userflow object, or to create new attributes for current
and future use.
_\*NOTE: Aside from `groups` and `memberships` on `User` objects, all other fields will be associated with the `Group` or `User` object's `attributes` field within Userflow._
### A note on "groups" and "memberships"
The Userflow API allows you to create/update both groups and memberships through creating/updating
a user by including the appropriate `groups`/`memberships` as fields on the user object in the
request payload.
However, please be aware that their inclusion is not required and at most only one of either `groups`
or `memberships` can be set on a user per request. If your intention is only to sync group information,
it may be best to sync `Group` objects instead.
For more information on `groups` and `memberships`, including how they are properly formatted on a `User` object,
please consult the [Userflow API documentation](https://userflow.com/docs/api#create-or-update-a-user):
- Formatting `groups` for a `User`:
- Formatting `memberships` for a `User`:
### Pruning memberships
Hightouch also supports the ability to prune memberships of a user upon sync through the `prune_memberships`
property in the request payload. As per the Userflow API documentation:
> _"By default, the API will only update the memberships/groups that's included in the request. Existing memberships that aren't included will not be removed, unless you set `prune_memberships` to `true`. Only set `prune_memberships` to `true`, if the groups or memberships list is set and contains *all* the groups the user belongs to. When a membership is deleted, the group itself is left intact."_
When either `groups` or `memberships` are chosen to be synced to the destination, Hightouch will give you
the option to toggle membership pruning on or off.
_\*NOTE: By default, membership pruning is disabled._
---
## Vero Cloud
**URL:** https://hightouch.com/docs/destinations/vero
**Description:** Enable your sales team to notify users based on events in your warehouse with Vero Cloud.
**Section:** Destinations
## Setup
Grab your Vero Auth Token under [Settings > Project Settings > API Credentials](https://app.getvero.com/settings/project).
## Syncing
Hightouch supports syncing to the following Vero objects:
- Users
- Events
### Sync modes
Hightouch supports Update and Insert sync modes.
During Upsert mode, new objects will be inserted into Vero and all attributes will be kept up-to-date within Vero.
During Insert mode, existing objects will be ignored but new objects will be inserted into Vero.
### Upserting users
it's required to send a User ID to Vero to identify the user.
You may send any number of additional fields with the user.
### Inserting events
it's required to send a User ID and email to Vero to identify the user for the event.
You must also configure an Event Name to be used for the Sync.
You may send any number of additional fields with the event.
---
## Viant
**URL:** https://hightouch.com/docs/destinations/viant
**Description:** Viant is a pioneer in people-based digital advertising. Use AI to help advertisers plan, execute, and measure their digital advertising campaigns.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
--------------|----------------------------------------------|----------------------|-----------------------------------------------------------------------------
**Audiences** | Sync data from any source to Viant audiences | Add | [Audiences docs](https://docs.api.viantinc.com/reference/how-to-get-started)
**Conversion events** | Send online or offline conversion event data to Viant | Insert | [Conversions docs](https://docs.api.viantinc.com/reference/onboard-onlineoffline-conversion-data)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Viant
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Viant** and click **Continue**. You can then authenticate Hightouch to **Viant**.
For the **Authentication method**, select **Log in to Viant** and log into your Viant account. Once successful, you will be redirected back to Hightouch to enter a descriptive name for your destination and complete setup.
## Sync configuration
Once you've set up your Viant destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Viant destination you want to sync to.
### Syncing audiences
Sync data from any source to Viant audiences.
#### Record matching
You can match rows from your model to audiences in Viant on any column in your model and any field in Viant. Ensure the data types of the model column and Viant field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Accepted ID formats
Each Viant Audience Segment has a specified `dataType` that gets set when creating a new segment for the first time. If using an exsiting segment, the label will show what `dataType` the segment has. Viant supports the following `dataTypes`:
- `Email (sha_256)`
- `Address (sha_256)`
- `IP address`
- `Cookie`
- `Mobile ID`
If using `Email (sha_256)` or `Address (sha_256)`, Hightouch can auto apply hashing if selected. To use `Address (sha_256)`, please refer to Viant's docs on [preparing your user data](https://docs.api.viantinc.com/reference/prepaing-your-user-data) for formatting and standardizing your physical addresses before hashing.
#### Field mapping
Hightouch lets you sync audience fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default audience fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Split retries
Viant counts all audiences in a batch as rejected if the request contains a single invalid audience. To pinpoint which audiences are getting rejected with which errors and reduce the number of valid audiences that get retried, you can enable [split retries](/syncs/retries#split-retries).
### Syncing conversion events
Send online and offline conversion event data to Viant's DSP (Adelphic) platform.
#### Account configuration
| Field | Description |
|-------|-------------|
| **Account ID** (required) | Your Adelphic account ID |
| **Advertiser IDs** (optional) | List of advertiser IDs for conversion attribution |
#### User identifiers
At least one user identifier is required per conversion event. Viant supports the following identifier types:
- `Email (SHA256)` — `hashed_email`
- `Address (SHA256)` — `hashed_physical_address`
- `IP address`
- `Cookie`
- `Mobile ID` (IDFA/GAID)
If using `Email (SHA256)` or `Address (SHA256)`, Hightouch can auto-apply SHA256 hashing if selected. To use `Address (SHA256)`, refer to Viant's docs on [preparing your user data](https://docs.api.viantinc.com/reference/prepaing-your-user-data) for formatting and standardizing your physical addresses before hashing.
#### Field mapping
**Required fields:**
| Field | Description |
|-------|-------------|
| Timestamp | The timestamp of the conversion event |
| Transaction ID | Unique identifier for the transaction |
| Conversion event type | The type of conversion event (see supported types below) |
**Supported conversion event types:** `PageView`, `LandingPage`, `ItemView`, `AddToCart`, `InitiateCheckout`, `AddPaymentInfo`, `Purchase`, `Lead`
If you have any non-standard event you cannot map to the above list, you can pass a custom event name instead of above standard list as part of same conversion event type field.
**Optional fields:**
| Field | Description |
|-------|-------------|
| Amount | Monetary value of the conversion |
| Currency | Currency code (e.g., USD) |
| Store ID | Store or location identifier |
| Channel | `online` or `offline` |
| Purchased items | Array of item details (`itemId`, `productName`, `productCategory`, `price`, `quantity`) |
| Conversion location | Country, state, and/or city of the conversion |
#### Custom fields
You can include up to 10 custom key-value pairs for additional metadata. All custom field values are sent as strings.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Vistar Media
**URL:** https://hightouch.com/docs/destinations/vistar-media
**Description:** Empower your advertising team to run digital out-of-home (DOOH) campaigns with data-driven programmatic precision.
**Section:** Destinations
## Overview
Captivate your audience with digital out-of-home (DOOH) ads that are tailored to
the location, weather, and time of day based on the foot patterns of your
first-party audiences.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------- | ---------------------------------- | -------------------- |
| **Audiences** | Create and update custom audiences | All |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Vistar Media
This integration requires prior approval from the Vistar Media team and the
Hightouch team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your Vistar Media rep who will verify approval. Once we have confirmed,
we will unlock the integration for you to connect.
To connect, input the `Client Name` provided by your Vistar rep.
### Syncing custom audiences
You can input the ID of the audience you plan on delivering to Vistar along with
other properties.
#### User identifiers
To identify which users to add or update in an audience, select model columns
and the corresponding Vistar fields. You can match on any of the following user
fields:
- MAID
#### Delete behavior
The only supported sync mode is `all` mode, meaning that each sync run will refresh
the entire audience. Users removed from your audience will automatically be dropped
with each refresh - only present users will be in Vistar's audience.
## Tips and troubleshooting
### Refresh rate
Vistar Media has a max refresh rate of monthly. Though they recommend a one-and-done
sync run for each campaign. You should not have your Vistar syncs running more than
every 4 weeks (we recommend a manual schedule configuration).
### Matched device count
Match rates are not available to preview in the Hightouch platform. Contact your
Vistar rep for insights into match rates and audience reach.
Keep in mind that audience counts reflect the number of devices match, not
the number of users matched. Also keep in mind that matching occurs at the
household level, and that a single household may own multiple devices.
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Sync alerts
---
## Vitally
**URL:** https://hightouch.com/docs/destinations/vitally
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API reference |
|-------------------|-----------------------------------------------------------------------------------------------------|------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|
| REST Objects | Sync records to objects such as Accounts, Organizations or Users in Vitally using the REST API | Upsert, Update, Insert | [REST Object endpoints](https://docs.vitally.io/pushing-data-to-vitally/rest-api) |
| Analytics Objects | Sync records to objects such as Accounts, Organizations or Users in Vitally using the analytics API | Upsert | [Analytics object endpoints](https://docs.vitally.io/pushing-data-to-vitally/using-the-vitally-api/https-analytics-api/analytics-api-batch) |
| Events | Sync records as events to Vitally. | Upsert | [Events docs](https://docs.vitally.io/pushing-data-to-vitally/integrations/data-warehouses/configuring-sync/syncing-events) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Vitally
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select Vitally and click **Continue**. You can then authenticate Hightouch to by entering the following fields into Hightouch:
- **Region**: Either **EU** or **US**.
- **Analytics API Token:** This is required for syncing analytics objects and events.
- **REST API Token:** This is required for the REST endpoints.
Follow these instructions to generate tokens.
1. Log into your Vitally account, click the wheel icon on the bottom sidebar.
2. Under **Data Management**, select the **Integrations** option.
3. For the Analytics API Token, select the **Vitally Analytics API** card. Otherwise, skip to step 6 for the REST API Token.
4. Toggle on the integration.
5. Click the **Integrate via HTTPS API** tab and copy the **API KEY**. This is the Analytics **API Token** you need to enter in Hightouch.
6. For the REST API Token, select the **Vitally REST API** card.
7. Toggle on the integration and click to generate the API Key.
8. Copy the **Basic Auth Header** once it's generated to use in Hightouch. This is the **REST API Token** you need to enter in Hightouch.
Once you've entered the relevant token or tokens, click **Continue** and enter a descriptive name for your destination to complete setup.
## Sync configuration
Once you've set up your Vitally destination, have a [model](/getting-started/concepts#models) to pull data from, and have a clear idea of the data you want to sync, you can set up your sync. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Vitally destination you want to sync to.
### Syncing REST objects
Hightouch supports syncing to account, organization, and user objects in Vitally.
#### Record matching
To match rows from your model to objects in Vitally, you need to select the model column that contains values that match the [**External ID** field](https://docs.vitally.io/pushing-data-to-vitally/the-significance-of-external-id) in Vitally. When syncing users, you can also match on **Email**.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync object fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to any of an object's default fields.
Ensure the data type of your model column matches the data type of field you want to sync to in Vitally.
For example, a Vitally `text` field expects a model column with the `string` [data type](/models/data-types-casting#data-types).
When using **Upsert** or **Update** mode to sync users, if you [match on **Email**](#record-matching),
you need to map both the **External ID** **and** either **Account IDs** or **Organization IDs**.
Otherwise, Hightouch displays a `Required field` error and a `Must specify at least one of: accountIds or organizationIDs` error.
If you match users on **External ID**, you only need to provide either **Account IDs** or **Organization IDs**.
Otherwise, Hightouch displays a `Must specify at least one of: accountIds or organizationIDs` error.
Vitally accounts, organizations, and users objects contain an editable [**traits** attribute](https://docs.vitally.io/account-health-scores-and-metrics/traits) to store custom key/value pairs.
This is useful for storing information like a customer's plan, number of licenses, name, etc.
To properly sync traits to Vitally, you need to format the traits field as an `object` data type rather than a `string`.
You can create an `object` of all the fields you want to update, and then map them to traits.
An example traits attribute for an organization object could look like this:
```
{
"organizationId": "Id_value",
"organizationName": "Name_value"
}
```
You can use [template mapping](/syncs/mapping-data#template-mapping) to create objects like this in your sync configuration.
In the following example screenshot, the template uses this [Liquid](https://shopify.github.io/liquid/basics/introduction/) function
```
{{ | json_construct : 'organizationName', row['organizationName'] }}
```
to create an object where the key's name is `organizationName` and the value is the row's `organizationName` column.
For more information, refer to the [Vitally's API documentation](https://docs.vitally.io/pushing-data-to-vitally/rest-api/organizations).
You can sync columns from your model to existing traits in Vitally and/or create new ones by adding them directly to the traits object in your Hightouch sync.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| --- | --- |
| Do nothing | Keep the object in Vitally with all its mapped properties |
| Delete | Delete the object in Vitally and all its mapped properties |
### Syncing analytics objects
Hightouch supports syncing to account, organization, and user objects in Vitally with the analytics API.
#### Record matching
To match rows from your model to objects in Vitally, you need to select a model column that contains the corresponding ID for each supported object.
- **Account ID** for account objects
- **Organization ID** for organization objects
- **User ID** for user objects
#### Field mapping
Hightouch lets you sync analytics object fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default object fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
#### Traits mapping
Hightouch allows you to map traits for the analytics objects. Any mappings from this section will be synced as additional attributes in the traits parameter.
### Syncing events
Hightouch supports syncing product events in Vitally.
[Vitally's API](https://docs.vitally.io/pushing-data-to-vitally/integrations/data-warehouses/configuring-sync/syncing-events) requires the following event parameters:
- `event`: the name or type of event
- `timestamp`: the time the event was generated
- `message_id`: a unique ID for the event
And at least one of:
- `user_id`: the ID of the user that 'owns' the event
- `account_id`: the ID of the account that 'owns' the event
The sync configuration form ensures all these are set and provides some additional options.
#### Event name
Providing an event name is required to send an event to the Vitally's API. You can either provide a static value or select to **use a column** from your model. Hightouch syncs the static or column value as the `event` parameter the API requires.
#### Event timestamp
You can optionally select a column that contains timestamps of when events occurred. If this field is empty, Hightouch uses the time the event arrives at the server. Hightouch syncs the timestamp as the `timestamp` parameter the API requires.
#### Field mapping
Field mapping is where you select which event parameters you want to send to the Vitally's API.
The API requires you to map at least one of the following fields in order to save your sync.
- Account ID
- Organization ID
- User ID
You can also include data to sync as custom event fields in Vitally.
## Tips and troubleshooting
### Common errors
#### Must specify at least one of: accountIds, organizationIDs, user_id
You may receive an error like this in the Hightouch UI if you don't include all necessary values in your sync.
Consult the [field mapping](#field-mapping) section of your relevant sync type to learn more.
### Live debugger
### Sync alerts
---
## Vizio Ads
**URL:** https://hightouch.com/docs/destinations/vizio-ads
**Description:** Empower your advertising team to run targeted campaigns across Vizio Ad’s premium Smart TV inventory.
**Section:** Destinations
## Overview
Deliver more relevant ads across Vizio’s premium, brand-safe Smart TV ad inventory.
Reach addressable audiences by matching your first-party identifiers to Vizio IDs
using Match Booster.
## Supported syncing
| Sync Type | Description | Supported Sync Modes |
| ------------- | ---------------------------------- | -------------------- |
| **Audiences** | Create and update custom audiences | All |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Vizio Ads
This integration requires prior approval from the Vizio Ads team and the Hightouch
team before proceeding.
To gain approval, please contact the Hightouch support team and connect us directly
with your Vizio rep who will verify approval. Once we have confirmed, we will
unlock the integration for you to connect.
To connect, input the following provided by your Vizio rep:
- Access key
- Secret key
- Customer directory
### Syncing custom audiences
You can input the ID of the audience you plan on delivering to Vizio Ads.
#### User identifiers
To identify which users to add or update in an audience, select model columns
and the corresponding Vizio Ads fields. You can match on any of the following user
fields:
- IP address
#### Delete behavior
The only supported sync mode is `all` mode, meaning that each sync run will refresh
the entire audience. Users removed from your audience will automatically be dropped
with each refresh - only present users will be in Vizio Ads's audience.
## Tips and troubleshooting
### Refresh rate
Vizio Ads has a max refresh rate of daily. You should not have your Vizio Ads syncs
running more than daily.
### Matched device count
Match rates are not available to preview in the Hightouch platform. Contact your
Vizio rep for insights into match rates and audience reach.
Keep in mind that audience counts reflect the number of Vizio devices match, not
the number of users matched. Also keep in mind that matching occurs at the
household level, and that a single household may own multiple devices.
### Common errors
### Live debugger
Live debugger view is not support for this integration. Please reach out to us
if you see any issues.
### Sync alerts
---
## VWO Attribute Lists
**URL:** https://hightouch.com/docs/destinations/vwo
**Description:** Boost conversions with precise targeting and personalized experiences using VWO's powerful optimization tools.
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API Reference |
| ------------------- | ------------------------------------------------ | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Attribute Lists** | Sync data from any source to VWO Attribute Lists | Insert, Update, Upsert | [Attributes](https://help.vwo.com/hc/en-us/articles/8681465703705-Working-With-Attributes-in-VWO) and [Attribute Lists](https://help.vwo.com/hc/en-us/articles/900003106343-Targeting-Visitors-Using-Attributes-List-in-VWO) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to VWO
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **VWO** and click **Continue**. You can then authenticate Hightouch to **VWO**.
Enter your **Authorization Token** into Hightouch.
If you don't already have a token, you can generate one [here](https://app.vwo.com/#/developers/tokens).
## Sync configuration
Once you've set up your VWO destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the VWO destination you want to sync to.
### Syncing Attribute Lists
Sync data from any source to VWO Attribute Lists.
Currently, we do not enable the creation of non-existent attribute lists, or the ability to browse attribute lists.
Syncing to VWO Attribute Lists requires a knowledge of the Attribute List ID, which you should be able to find in the VWO console.
#### Column selection
Configuring your VWO sync is as simple as selecting the column in your source which you are associating with the selected Attribute List.
When run, Hightouch will sync this column from your source to your Attribute List, diffing changes and keeping your Attribute List up to date.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Webflow
**URL:** https://hightouch.com/docs/destinations/webflow
**Description:** Bring in the power of personalization and dynamic content to your web projects on Webflow using rich customer data from your data warehouse
**Section:** Destinations
## Supported syncing
| Sync Type | Description | Supported Sync Modes | API documentation |
| -------------- | --------------------------------------------------------- | -------------------- | ---------------------------------------------------------------------------- |
| **Collections**| Sync data from any source to Webflow CMS collection items | Upsert, Update |[Create Collection Item](https://developers.webflow.com/reference/create-item)|
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Webflow
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Webflow** and click **Continue**. You can then authenticate Hightouch to Webflow by entering your **API Token** from your Webflow project.
To generate an API token in Webflow:
1. Go to **Apps & Integrations** and scroll down to the **API Access** section.
2. Click **Generate API Token**.
3. Copy the generated token to your clipboard.
> For the most accurate instructions, see [Webflow’s API Token Documentation](https://developers.webflow.com/data/v2.0.0/reference/authentication).
## Sync configuration
Once you've set up your Webflow destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Webflow destination you want to sync to.
### Syncing collections
#### Record matching
To match source rows to Webflow collection items, you need to select a source column and corresponding Webflow field. You can match on any Webflow field that is set to [plain text](https://webflow.com/glossary/plain-text-field) type. Using the `Slug` field is recommended.
#### Field mapping
You can sync columns from your source to Webflow's collection properties.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
| Behavior | Description |
| ----------------- | -------------------------------------------- |
| **Do nothing** | Keep the record in Webflow |
| **Clear fields** | Keep the record, but clear the mapped fields |
| **Delete record** | Delete the record in Webflow |
**Clear fields** is only available in Update mode, whereas **Delete record** is only available in Upsert mode.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## WhatsApp
**URL:** https://hightouch.com/docs/destinations/whatsapp
**Description:** Message your customers when and where it matters most.
**Section:** Destinations
## Overview
Deliver critical customer journey communications in real-time to where it matters: your customers' pockets. Combine data from multiple sources to send promotional messages, special offers or order updates to customers who have actively opted into your company's communications via WhatsApp, initiating the first of many customer communication experiences.
## Supported syncing
|Message Type|Description|Supported Sync Modes | API Reference |
|--|--|--|--|
|**Message Templates**|Send a business-initiated message using an approved message template|Insert| [Message template reference docs](https://developers.facebook.com/docs/whatsapp/cloud-api/reference/messages#template-messages) |
## Connect to WhatsApp
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **WhatsApp** and click **Continue**. You can then authenticate Hightouch by entering the following fields:
- Your WhatsApp phone number ID
- Facebook business manager ID
- System User token.
The System User's token must be generated with the following scopes:
- `whatsapp_business_management`
- `whatsapp_business_messaging`
Before finalizing the destination creation, you need to agree to [WhatsApp's opt-in policy](https://developers.facebook.com/docs/whatsapp/overview/getting-opt-in).
## Sync configuration
Once you've set up your WhatsApp destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the WhatsApp destination you want to sync to.
### Message templates
WhatsApp message templates are specific message formats that businesses use to send out notifications or customer care messages to people that have opted in to notifications. Business-initiated messages-the types of messages Hightouch enables your team to send-must use a template approved by WhatsApp. If you don't see the template you would like to use in the sync configuration form, check the template's status in your [WhatsApp Template Manager](https://business.facebook.com/wa/manage/message-templates).
### Phone numbers
A column from your model must contain the user phone numbers to send a message to.
### Field mapping
If the selected template contains variables, each variable must be mapped to create a complete message.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Workday
**URL:** https://hightouch.com/docs/destinations/workday
**Description:** Sync enriched data from your warehouse to your Workday instance
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ----------------- | -------------------------------------------------- | -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Organizations** | Sync data from any source to Workday organizations | Upsert, Update | [Organization docs](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Add_Update_Organization.html) |
| **Workers** | Sync data from any source to Workday workers | Update | [Home contact information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Home_Contact_Information.html), [work contact information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Work_Contact_Information.html), and [personal information docs](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Personal_Information.html) |
| **Positions** | Sync data from any source to Workday positions | Update | [Position docs](https://community.workday.com/sites/default/files/file-hosting/productionapi/Staffing/v40.1/Edit_Position.html) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Workday
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Workday** and click **Continue**. You can then authenticate Hightouch to **Workday** by entering the following required fields into Hightouch:
- **Hostname**
- **Tenant ID**
- **Username**
- **Password**
The **hostname** is the hostname of the Workday SOAP endpoints you want to use.
You only need to enter the hostname, without `https://` nor particular endpoint URL.
For example, if you want to use this SOAP endpoint: `https://wd2-impl-services1.workday.com/ccx/service/your_company/Human_Resources/v40.1`,
enter `wd2-impl-services1.workday.com`.
If you don't have a hostname, enter `wd2-impl-services1.workday.com`.
You can find your **tenant ID** from your URL when you are logged into Workday.
For example, if the URL is `https://impl.workday.com/your_company/d/home.html`, then your tenant ID is `your_company`.
Workday recommends that you use Integrated System User (ISU) to when connecting to Workday via third-party services like Hightouch.
See [Workday's docs](https://doc.workday.com/reader/Z9lz_01hqDMDg6NSf7wCBQ/esBDCh5D66sgBhIxmQ5U5g) for more information on setting up an ISU.
## Sync configuration
Once you've set up your Workday destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Workday destination you want to sync to.
### Syncing organizations
This sync type adds new organizations and/or updates existing organizations with the supplied information.
A new effective-dated organization name is automatically created if any attributes used for organization name are different as of specified effective date.
To use this sync type, you need to have **Put** access for the **Manage: Organization Update Integration** domain.
#### Record matching
To match rows from your model to organization in Workday, you need to select a model column and corresponding Workday field.
- In **upsert** mode, you can match on either **Organization Reference ID** or **Integration ID Data**.
- In **update** mode, you must match on **Integration ID Data**.
If you are matching with **Integration ID Data**, you must also provide the **System ID** in the attributes mapping.
#### Field mapping
Hightouch lets you sync organization fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default organization fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
### Syncing workers
This sync type updates existing workers in Workday.
#### Record matching
To match rows from your model to worker in Workday, you need to select a model column and corresponding Workday field. You can match on any of the following Workday fields:
- **Universal_ID_Reference**
- **Person_Reference**
When matching with **Universal_ID_Reference**, you must also specify its **Reference Type** int the attributes mapping. Valid values are:
- `WID`
- `Universal_Identifier_ID`
When matching on **Person_Reference**, you must also specify its **Reference Type** in the attributes mapping.
See the [RoleObject](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Home_Contact_Information.html#RoleObjectIDType)'s `@type` for the full list of valid values.
#### Field mapping
Hightouch lets you use [field mapping](/syncs/mapping-data) to update the following kinds of worker information:
- [home contact information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Home_Contact_Information.html#Person_Contact_Information_DataType)
- [work contact information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Work_Contact_Information.html#Person_Contact_Information_DataType)
- [personal information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Personal_Information.html#Change_Personal_Information_DataType)
Field names for [home contact information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Home_Contact_Information.html#Person_Contact_Information_DataType) begin with _Home_,
field names for [work contact information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Work_Contact_Information.html#Person_Contact_Information_DataType) begin with _Work_, and field names for [personal information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Personal_Information.html#Change_Personal_Information_DataType) begin with _Personal Information_.
You can map data from any of your model columns to these fields. Ensure your model columns data types match the data types of the fields you want to sync to.
### Syncing positions
This sync type updates existing positions in Workday.
#### Record matching
To match rows from your model to position in Workday, you need to select a model column and corresponding Workday field. You can match on any of the following Workday fields:
- `WID`
- `Position_ID`
#### Field mapping
Hightouch lets you sync position fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default position fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
## Tips and troubleshooting
### Common errors
#### The task submitted is not authorized
This error means your Workday user doesn't have enough permission to perform the API operation configured in the Hightouch sync.
You can resolve the error by granting the user you [authenticated with](#connect-to-workday) with the permission required for the endpoint.
For example, if you get this error when you're trying to make changes with the [Change_Home_Contact_Information](https://community.workday.com/sites/default/files/file-hosting/productionapi/Human_Resources/v40.1/Change_Home_Contact_Information.html) endpoint, you should ensure your user has permission to use this API.
### Live debugger
### Sync alerts
---
## Workday Adaptive Planning
**URL:** https://hightouch.com/docs/destinations/workday-adaptive-planning
**Description:** Sync data from your warehouse to your Workday Adaptive Planning modeled sheets.
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
| ----------------- | ---------------------------------------- | -------------------- | -------------------------------------------------------------------------------------------------------------- |
| **Modeled sheets** | Import data to a selected modeled sheet | Insert, Mirror | [Modeled sheet docs](https://adaptiveplanning.doc.workday.com/r/DG7oXjCPB6kIw6Th6awNUg/~SS8TWR04JTiriCxM4rMBA) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Workday Adaptive Planning
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Workday Adaptive Planning** and click **Continue**. You can then authenticate Hightouch to **Workday Adaptive Planning** by entering the following required fields into Hightouch:
- Workday **Username**
- Workday **Password**
Once you've entered these fields, click **Continue**. Give your destination a descriptive name and **Finish** the destination connection setup.
## Sync configuration
Once you've set up your Workday Adaptive Planning destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Workday Adaptive Planning destination you want to sync to.
The Workday Adaptive Planning destination allows you to **Mirror** your data to a spreadsheet or insert new rows with **Insert** mode.
**Mirror** mode replaces the entire selected sheet's contents with the rows from your model.
Ensure that your sheet doesn't contain data that shouldn't be overwritten if you select this mode.
### Syncing modeled sheets
You can insert rows into or completely replace the selected modeled sheet with data from your model.
To begin, select the sheet you want to sync data to and its version.
#### Record matching
In **Insert** and **Mirror** mode, Workday Adaptive Planning automatically generates an
identifier for every new record synced, so there is no need to match an
existing record.
#### Field mapping
Hightouch lets you sync sheet fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to your modeled sheet fields.
Workday Adaptive Planning requires the following fields, so you must map them to complete your configuration:
- **Approval**
- **Asset Class**
- **Level**
- **Priority**
Ensure your model columns data types match the data types of the fields you want to sync to.
Workday Adaptive Planning uses numbers to represent boolean values. Use `0` for
`false` and `1` for `true`.
## Tips and troubleshooting
### Common errors
#### Data containing pipe symbols
Workday Adaptive Planning uses the pipe symbol (`|`) to separate the sheet's columns.
Make sure your data does not contain any pipe symbols since it will cause a mismatch
in the header and row data.
### Live debugger
### Sync alerts
---
## X Ads (formerly Twitter Ads)
**URL:** https://hightouch.com/docs/destinations/x
**Description:** Send conversion web events to X and run campaigns with custom audiences, retargeting, and lookalikes directly from your warehouse
**Section:** Destinations
## Overview
Build more robust campaigns on X with up-to-date customer data from your data warehouse.
## Getting started
### Connect to X
#### Add X as a destination in Hightouch
1. Log in to Hightouch or create a free Hightouch account
2. Click on **Destinations** in the left sidebar
3. Click on **Add destination** in the top right corner
4. Select **X** in the destination catalog and click **Continue** to proceed
5. Proceed with the OAuth steps until you are redirected to the Hightouch website.
6. Give your destination a name and unique slug (for example, X Production” and `x-production`)
7. Click **Finish** to create your X destination.
## Sync configuration
### Syncing custom audiences
This integration supports _Segment Mode_. New users in the model will be inserted into the X
Custom Audience. Users that are removed from the model will be removed from the
Custom Audience.
#### X account selection
In this section, select the X account you would like to sync your data to.
#### Audience selection
This integration supports syncing to an **existing audience** or **creating a new audience**.
To use an **existing segment**:
- Select `Use existing audience` from the radio buttons.
- Select the name of the audience from the dropdown menu.
To **create a new audience**:
- Select `Create a new audience` from the radio buttons.
- Specify a custom name for this audience in the input field. Otherwise Hightouch will use the name of the model as the audience's name.
#### User identifers
Hightouch allows you to sync columns from your source to the supported X fields.
X allows mapping to various user identifiers, such as Email, Device ID, X Handle, etc. X requires _all fields_ for users
to be both **normalized** and **hashed** using SHA256 (with the exception of Partner User ID).
You can achieve this in your model with a function
by using the `encode(digest(value, 'sha256'),'hex')` function in Postgres, or refer to the [Template Mappings](/destinations/X#template-mappings) Section in this documentation to learn about how Hightouch can handle hashing these records for you.
### Syncing conversion events
This integration supports syncing _Web Conversion Events_.
#### X account selection
In this section, select the X account you would like to sync your data to.
#### Pixel ID (Universal Website Tag) account section
If the **Pixel ID** exist in your model, toggle **Use Column**, and select the appropriate column from the dropdown menu.
To retrieve your Pixel ID:
- Navigate to your [Events Manager](https://business.x.com/en/help/campaign-measurement-and-analytics/conversion-tracking-for-websites) page.
- Copy the ID and paste it in the Hightouch sync form.
#### Event selection
Given the correct **Pixel ID**, Hightouch will filter for the eligible events that allow conversion events to be synced to.
Select the event you'd like to sync to from the dropdown menu. If the **ID** for these events exist in your model, toggle **Use Column**, and select the appropriate column from the dropdown menu.
#### Identifier section
When syncing conversion events, X expects one identifier at minimum, or you can provide both.
Select your choice for the sync from the radio buttons. This will render the fields in the mappings section, which you can map to from your model.
#### Field mapping
Hightouch allows you to sync columns from your source to the supported X fields.
If you wish to use the **Email Address** as an identifier, X expects it to be hashed using SHA256.
You can achieve this in your model with a function by using the encode(digest(value, 'sha256'),'hex') function in Postgres, or refer to the [Template Mappings](/destinations/X#template-mappings) section in this documentation to learn about how Hightouch can handle hashing these records for you.
### Template mappings
You can leverage Hightouch's Template Mappings feature to allow Hightouch to handle hashing certain fields for you automatically.
To use this feature:
- Select **Template** in the mappings window.
- Select the column you would like to hash from the **Variables** section.
- Select **sha256** from the **Functions** section.
- Click **Apply** to save.
To find out more about Template Mapping and other Mapping features, visit our [Mapping Data](/syncs/mapping-data) section.
## Tips and troubleshooting
Below only applies to the audiences sync type.
### Common errors
#### 400: INVALID_PARAMETER
The full error message looks something like this:
```json
{
"request": [
{
"params": {
"account_id": "01ab234cdef",
"custom_audience_id": "0a1b2"
},
"operation_type": "Update"
}
],
"operation_errors": [
[
{
"code": "INVALID_PARAMETER",
"message": "Tailored audience attribute value can not be empty.",
"parameter": ""
}
]
]
}
```
This error occurs if any of your [field mappings](#field-mapping) include `null` values.
X doesn't accepted hashed `null` values.
To resolve this, [remove rows containing `null` values](/models/sql-editor#exclude-rows-with-null-values) from your model's query results or enable the [**Don't sync null values** option](/syncs/mapping-data#dont-sync-null-values) in the [advanced mapper](/syncs/mapping-data#advanced-mapper).
### Live debugger
### Sync alerts
---
## Xandr
**URL:** https://hightouch.com/docs/destinations/xandr
**Description:** Enable your marketing team to target granular audiences and measure real-world outcomes
**Section:** Destinations
## Supported syncing
| Type | Description | Supported Sync Modes | API Reference |
|--------------|---------------------------------------------|----------------------|-----------------------------------------------------------------------------------------------------|
| **Segments** | Sync data from any source to Xandr segments | Add, Remove | [Segments docs](https://docs.xandr.com/bundle/xandr-api/page/uploading-segment-data-using-bss.html) |
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Xandr
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Xandr** and click **Continue**. You can then authenticate Hightouch to **Xandr**.
To get started with the Xandr API, you need to request access by contacting the [Xandr support team](https://help.xandr.com/s/login/). For more information, read the [Xandr API docs](https://docs.xandr.com/bundle/xandr-api/page/api-onboarding-process.html).
Before configuring your Xandr destination, you will need to follow [these steps](https://docs.xandr.com/bundle/xandr-api/page/token-based-api-authentication.html) to generate private and public keys. After generating your keys, register your public key with your Xandr account.
Enter the following required fields into Hightouch:
- **Username**: Enter your username.
- **Member ID**: Enter your member ID.
- **Public Key Name**: Enter the name of the public key.
- **Private Key**: Enter your private key
## Sync configuration
Once you've set up your Xandr destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Xandr destination you want to sync to.
### Syncing segments
Sync data from any source to Xandr segments.
#### Record matching
To match rows from your model to segment in Xandr, you need to select a model column and corresponding Xandr field. You can match on any of the following Xandr fields:
- **Xandr ID**
- **Extended ID**
- **External ID**
- **Device ID**
- **IFA**
#### Field mapping
Hightouch lets you sync segment fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default segment fields.
Ensure your model columns data types match the data types of the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Xero
**URL:** https://hightouch.com/docs/destinations/xero
**Description:** Gain financial visibility and empower your accounting team by keeping your financials up to date
**Section:** Destinations
## Overview
Give yourself confidence in the financial health of your business by integrating your Xero usage with Hightouch.
We automate your financial workflows and eliminate inconsistent data issues and manual data entry.
With this integration, you can deliver up-to-date data directly from your source of truth to allow your team the luxury to focus
on making financial decisions that benefit your business instead of wasting time finding, cleaning, and moving relevant data.
## Getting started
### Connect to Xero
**Authorize Xero with OAuth and create your destination.**
#### Add Xero as a destination in Hightouch
1. Log in to Hightouch or create a free Hightouch account
2. Click on **Destinations** in the left sidebar
3. Click on **Add destination** in the top right corner
4. Select **Xero** in the destination catalog and click **Continue** to proceed
5. Complete the authorization steps until you are redirected back to the Hightouch application
6. Give your destination a name and unique slug (for example, `Xero Production` and `Xero-production`)
7. Click **Finish** to create your Xero destination.
### Connect to your data source
Hightouch sits on top of your data warehouse and can read from Snowflake, Redshift, BigQuery, Databricks, Postgres, MySQL, and many other data sources. You can even fetch data from Google Sheets and Airtable. Hightouch can help activate your data wherever it resides.
We've written dedicated guides for each supported data source. Take your pick:
- Connect to [Airtable](/sources/airtable)
- Connect to [Amazon Athena](/sources/amazon-athena)
- Connect to [Amazon Redshift](/sources/amazon-redshift)
- Connect to [Amazon S3](/sources/s3)
- Connect to [ClickHouse](/sources/clickhouse)
- Connect to [Databricks](/sources/databricks)
- Connect to [Elasticsearch](/sources/elasticsearch)
- Connect to [Firebolt](/sources/firebolt)
- Connect to [Google BigQuery](/sources/google-bigquery)
- Connect to [Google Sheets](/sources/google-sheets)
- Connect to [Metabase](/sources/metabase)
- Connect to [MongoDB](/sources/mongodb)
- Connect to [MySQL](/sources/mysql)
- Connect to [Palantir Foundry](/sources/foundry)
- Connect to [PostgreSQL](/sources/postgresql)
- Connect to [Rockset](/sources/rockset)
- Connect to [SFTP](/sources/sftp)
- Connect to [Snowflake](/sources/snowflake)
- Connect to [SQL Server](/sources/sqlserver)
- Connect to [Trino](/sources/trino)
If you're missing the credentials necessary to connect to your data source,
try inviting your teammates to Hightouch. You can have unlimited collaborators
in your workspace.
#### Connect to a sample database instead
Waiting for credentials to your data warehouse? No problem. You can connect to one of our [sample datasets](/sources/sample-data) to get started right away. For the rest of this tutorial, we'll use the **B2B SaaS** sample source, which contains a `users` table. This sample source is available when you log in for the first time—no setup required.
Next, we'll write a SQL query to define which users should be synced to our Xero accounts resource. You can follow the same pattern to sync to other Xero resources.
### Create a model
In Hightouch, a model represents a query that filters or transforms the records in your data source. Models are used to determine exactly which records to sync between sources and destinations.
When you connect to a data warehouse, your models will probably be written in SQL. You can also import models from tools like dbt and Looker.
In this example, we'll keep it simple. Suppose you want to sync all your customers to Xero. To accomplish this, you'll want to create a model that looks like this:
```sql
SELECT * FROM users;
```
1. Click on **Models** in the left sidebar.
2. Click on **Add model** in the top right corner.
3. Select **Demo Database** as your source and click **Continue** to proceed.
4. Select **SQL Editor** as your modeling method.
5. Paste the SQL query from above into the editor on the left side of the page.
6. Click **Preview** to execute the query.
7. Click **Continue** to proceed.
8. Give your model a name and unique slug (for example, "All Users" and `all-users`).
9. Select **ID** as the primary key for your model.
10. Click **Finish** to create your model.
We just created a model using a SQL query that retrieves all our current users. Next, we'll configure our sync to add these users to Xero.
### Create a sync
1. Click on **Syncs** in the left sidebar.
2. Click on **Add sync** in the top right corner.
3. Select **All Users** as your model and click **Continue** to proceed.
4. Select **Xero** as your destination and click **Continue** to proceed.
## Sync configuration
### Object selection
Hightouch supports syncing to the following Xero resources:
- **Accounts**.
- **Contacts**.
- **Payments**.
- **Invoices**.
- **Items**.
### Accounts
#### Account type selection
- Select the type of accounts you would like to sync to your destination. To sync dynamic types, toggle 'use column' and select a column from your model that contains supported enum values by Xero.
Selecting an account type from the drop-down menu allows Hightouch to provide
you with the appropriate validations for each account type. These validations
will not exist if you choose to sync dynamic types from your model and will
require you to be familiar with what is required by Xero.
#### Sync modes
For **Bank** account type, our integration supports:
- **Insert** mode.
For all other account types, our integration supports:
- **Upsert**: Records that aren't found in Xero given the provided identifier will be inserted. Existing records will be updated.
- **Update**: Records that are found in Xero given the provided identifier will be updated.
- **Insert**: Records will be pushed to your destination.
#### Record matching
Records can be matched from your source to your Xero workspace by:
- **Account ID**
- **Name**
#### Field mapping
Hightouch allows you to sync columns from your source to the supported Xero fields.
#### Delete records
When the Hightouch sync engine recognizes that a record has been removed from your source, you can either:
In **Upsert** mode:
- Do nothing. An existing record in the destination would remain untouched.
- Delete the record in the destination. Only non-system accounts and accounts not used on transactions can be deleted.
### Contacts
#### Sync modes
This integration supports:
- **Upsert** mode: Records that aren't found in Xero given the provided identifier will be inserted. Existing records will be updated.
#### Record matching
Records can be matched from your source to your Xero workspace by:
- **Name**.
#### Field mapping
Hightouch allows you to sync columns from your source to the supported Xero fields.
### Payments
#### Sync modes
This integration supports:
- **Insert** mode: Records will be pushed to your destination.
#### Invoice record matching
Invoice records can be matched from your source to your Xero workspace by:
- **Invoice ID**
- **Invoice Number**
- **Credit Note ID**
- **Credit Note Number**
#### Account record matching
Account records can be matched from your source to your Xero workspace by:
**Account ID**
**Code**
#### Field mapping
Hightouch allows you to sync columns from your source to the supported Xero fields.
### Invoices
#### Invoice type selection
Select the type of invoices you would like to sync to your destination. To sync dynamic types, toggle 'use column' and select a column from your model that contains supported enum values by Xero.
#### Sync modes
This integration supports:
- **Upsert** mode: Records that aren't found in Xero given the provided identifier will be inserted. Existing records will be updated.
- **Insert** mode: Records that aren't found in Xero given the provided identifier will be inserted.
#### Record matching
In **Upsert** mode, records can be matched from your source to your Xero workspace by:
- **Invoice Number**
#### Field mapping
Hightouch allows you to sync columns from your source to the supported Xero fields.
#### Associations
Hightouch can perform a reference look-up for reference fields. Select an identifier to use to perform the look-up and use the respective contact to create the invoice object.
You can perform a look-up using:
- **Contact Number**
- **Email Address**
- **Name**
If the `Contact ID` exists in your model, you can skip the lookup step by selecting `Contact ID` from the drop-down menu.
### Items
#### Invoice type selection
Select the type of invoices you would like to sync to your destination. To sync dynamic types, toggle 'use column' and select a column from your model that contains supported enum values by Xero.
#### Sync modes
This integration supports:
- **Upsert** mode: Records that aren't found in Xero given the provided identifier will be inserted. Existing records will be updated.
- **Insert** mode: Records will be pushed to your destination.
#### Record matching
In **Upsert** mode, records can be matched from your source to your Xero workspace by:
- **Item Code**
#### Field mapping
Hightouch allows you to sync columns from your source to the supported Xero fields.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Xtremepush
**URL:** https://hightouch.com/docs/destinations/xtremepush
**Description:** Sync data to Xtremepush as user profiles, to trigger campaigns, or to maintain user list membership.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
----------------------|------------------------------------------------------------------------|----------------------|---------------------------------------------------------------------------------
**User profiles** | Sync data from any source to Xtremepush user profiles | Upsert | [User profiles docs](https://docs.xtremepush.com/reference/user-profile-import)
**User lists** | Sync data from any source to maintain Xtremepush user list memberships | Add, Remove | [User lists docs](https://docs.xtremepush.com/reference/list-import)
**Campaign triggers** | Trigger campaigns using source data | Insert | [Campaign triggers docs](https://docs.xtremepush.com/reference/campaign-execute)
**Events** | Sync data from any source to Xtremepush events | Insert | [Events docs](https://docs.xtremepush.com/reference/event-batch-hit)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Xtremepush
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Xtremepush** and click **Continue**. You can then authenticate Hightouch to **Xtremepush**.
Enter the following fields into Hightouch:
- **Base URL**
- **App token**
## Sync configuration
Once you've set up your Xtremepush destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Xtremepush destination you want to sync to.
### Syncing user profiles
Sync data from any source to Xtremepush user profiles.
#### Record matching
To match rows from your model to user profiles in Xtremepush, you need to select the model column that contains values that match the **User ID** field.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync user profile fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to default and custom user profile fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
#### Delete behavior
The [delete behavior](/syncs/overview#delete-behavior) you select dictates what to do when a row no longer appears in your model's query results. You have the following options:
Behavior | Description
---------------|---------------------------------------------------------------------
**Do nothing** | Keep the user profile in Xtremepush with all its synced fields
**Clear** | Clear all the mapped fields, but keep the user profile in Xtremepush
**Delete** | Delete the synced user profiles from Xtremepush
### Syncing user lists
Sync data from any source to maintain Xtremepush user list memberships.
#### Record matching
You can match rows from your model to user lists in Xtremepush on any column in your model and any field in Xtremepush. Ensure the data types of the model column and Xtremepush field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync user list fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default user list fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
### Syncing campaign triggers
Trigger campaigns using source data.
#### Record matching
You can match rows from your model to campaign triggers in Xtremepush on any column in your model and any field in Xtremepush. Ensure the data types of the model column and Xtremepush field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
### Syncing events
Sync data from any source to Xtremepush events.
#### Record matching
You can match rows from your model to events in Xtremepush on any column in your model and any field in Xtremepush. Ensure the data types of the model column and Xtremepush field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync event properties via field mapping.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Yahoo
**URL:** https://hightouch.com/docs/destinations/yahoo
**Description:** Ensure that your campaigns are efficiently reaching the right audiences for the best results with Yahoo DSP.
**Section:** Destinations
## Supported syncing
|Sync Type|Description|Supported Sync Modes |
|--|--|--|
|**Conversion events**|Sync data from any source to any events|Insert|
|**Segments** | Create and update [audiences](https://developer.yahooinc.com/dsp/api/docs/traffic/audience/about-audience.html) | Add |
## Connect to Yahoo
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Yahoo** and click **Continue**. You can then authenticate Hightouch to **Yahoo** by entering the required credentials.
### Conversion event credentials
For Hightouch to sync conversion events, Yahoo must authorize your account to use Hightouch with the Yahoo Conversion API. Reach out to the [Yahoo Account Team](https://help.yahoo.com/kb/account) to gain authorization.
### Segment credentials
For Hightouch to sync data to your segments, you need to enter:
- **Client ID**
- **Client Secret**
To retrieve these, open your [Yahoo DSP Dashboard](https://sso.admanagerplus.yahoo.com/app/dashboard/list). In the upper-right corner, select your name, then **My Account**.
Activate the API and agree to the terms.
A success message displays your credentials.
Copy the client ID and secret, and keep it in a safe place.
For more details, visit the [Yahoo DSP Authentication docs](https://developer.yahooinc.com/dsp/api/docs/authentication/vmdn-auth-overview.html?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAABxrNZzuVEB2pL8Fdf9cjsTku2w7ooTz2Mz7BCA_InEazpCygsScZNhzJVJ4SJypKZzOTik4_avcNoAAspLZaQ6nEuoKvVg8OUVL6VMTkOdeVAup900GJ8S_ql543N_0WRsyyQgpAr41kcwJpHGDYLIQjtfViMeWaVYod_mxkmhR).
## Sync configuration
Once you've set up your Yahoo destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Yahoo destination you want to sync to.
### Syncing events
When syncing events, Hightouch treats any records added to [your source](/getting-started/concepts#sources) as new events and sends them to Yahoo DSP when your sync runs.
#### Pixel ID
You must set a Pixel ID when syncing conversion events to Yahoo DSP. [Create a new Pixel ID](https://help.yahooinc.com/dsp-api/docs/standard-yahoo-conversion-api#create-a-pixel-id-yp-id-in-the-yahoo-dsp) in Yahoo DSP under the tracking tab for your advertiser in Yahoo DSP.
The Pixel ID used for CAPI should be unique and **only** used for CAPI.
#### Event timestamp
You can include a timestamp of when the event occurred. If this field is empty, Hightouch uses the time the event arrives at the server.
Hightouch converts any `DateTime` values to Epoch UTC milliseconds per Yahoo DSP's expectations.
#### Field mapping
Ensure that at least one of the `userData` fields is included in your mapped fields, otherwise Yahoo DSP will reject the event.
For more information about required and supported parameters, refer to [Yahoo's Conversion API documentation](https://help.yahooinc.com/dsp-api/docs/standard-yahoo-conversion-api#capi-payload).
### Syncing segments
The Yahoo DSP enables you to build audiences that are most responsive to your campaigns, are most likely to convert, or are otherwise advantageous.
Begin by selecting the Yahoo account you would like to sync to.
Then, select whether to create a new audience or use an existing one.
When creating a new audience, you can optionally enter a name; otherwise, Hightouch defaults to the name of the associated model. To use an existing audience, select the desired audience from the dropdown.
### Audience type selection
Yahoo DSP API allows you to create different types of audiences.
Hightouch supports email address, device ID, and IP address types.
You can only select one audience type per sync.
You can later combine the different audience types when building your campaigns in Yahoo.
#### User identifiers
To identify which users to add or update in an audience, select model columns and the corresponding Yahoo fields.
The available fields vary depending on your selected audience type.
Yahoo DSP's API doesn't support removing identifiers or audience members.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Yahoo DataX
**URL:** https://hightouch.com/docs/destinations/yahoo-datax
**Description:** Yahoo DataX enables partners to seamlessly and securely activate their data and enrich it with Yahoo's robust data assets for more powerful advertising.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
--------------|----------------------------------------------------|----------------------|--------------------------------------------------------------------------
**Audiences** | Sync data from any source to Yahoo DataX audiences | Add | [Audiences docs](https://help.yahooinc.com/datax/docs/user-audience-data)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Using the Yahoo DataX Hightouch integration
To use the Yahoo DataX Hightouch integration, contact your Yahoo representative. Once approved, your Hightouch account will be allowed to use the Yahoo DataX integration.
## Taxonomy
Hightouch manages your Yahoo audience structure (also known as taxonomy). Each workspace you connect to the Yahoo DataX destination creates it's own taxonomy that you can eventually see in Yahoo DSP. Yahoo DataX supports `Syndicated` and `Custom` audiences based on the type of audiences you want to sync.
`Syndicated` audiences can be accessed by all advertisers that you have whitelisted in the destination configuration. `Custom` audience access is granted at the audience level, providing more granular access control for privately listed audiences. Typically `Custom` audiences are only shared with one advertiser. However, you can share with multiple advertiser MDM IDs.
Learn more about Yahoo DataX's taxonomy [here](https://help.yahooinc.com/datax/docs/taxonomy-1).
## Segment Fees
Hightouch manages your Yahoo audience's segment fees. Each audience can have any of the four segment fee types that Yahoo DataX supports:
- `Display`
- `Video`
- `Connected TV`
- `Audio`
CPM values can range between $0 and $50 (both inclusive). If you do not set segment fees, they will by default be 0 in Yahoo DSP.
**Note:** Yahoo only updates audience fees once per day, at the end of each day. If you have updated audience fees but do not see the change in Yahoo DSP, you may need to wait a day.
Learn more about audience fees [here](https://help.yahooinc.com/datax/docs/fee).
## Connect to Yahoo DataX
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Yahoo DataX** and click **Continue**. You can then authenticate Hightouch to **Yahoo DataX**.
Enter the following fields into Hightouch:
- **Brand name**: This will be the name of your brand's top-level audience folder shown in Yahoo DSP. It will appear under `Hightouch > BRANDED > `
- (Optional) **Your MDM ID for internal access**: Enter your company’s own Master Data Management identifier (MDM ID) to grant your team access to view and use all of your `Syndicated` and `Custom` audiences within the Yahoo DSP. Contact your Yahoo representative for your MDM ID.
- (Optional) **Syndicated audience whitelist MDM IDs**: `Syndicated` audiences are accessible to all Yahoo DSP advertisers who have been whitelisted here. To whitelist advertisers and grant them access to your `Syndicated` audiences, enter their MDM IDs here. Contact your Yahoo representative for valid MDM ID values.
## Sync configuration
Once you've set up your Yahoo DataX destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Yahoo DataX destination you want to sync to.
### Syncing audiences
Sync data from any source to Yahoo DataX audiences.
#### Record matching
You can match rows from your model to audiences in Yahoo DataX on any column in your model and any field in Yahoo DataX. Ensure the data types of the model column and Yahoo DataX field you select match.
Refer to the [record matching docs](/syncs/record-matching) for more information.
#### Field mapping
Hightouch lets you sync audience fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default audience fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
Yahoo DataX allows the following identifier`urnType`s for your audiences. You may select any number of identifier types, and each identifier will be sent in the sync.
- `Email (sha_256)`
- `Phone (sha_256)` (referred to as `HASHEDID` by Yahoo)
- `ZIP4`
- `IDFA`
- `GPAVID`
After Hightouch finishes syncing, audience uploads typically takes 24-48 hours to reflect in Yahoo DSP.
## Tips and troubleshooting
### Rate limit recommendation
It is highly recommended that you stagger your Yahoo DataX audience syncs. Yahoo DataX has a rate limit of `100 req/hour`, and to avoid hitting rate limits, please stagger your sync schedules.
### Common errors
### Live debugger
### Sync alerts
---
## Zapier
**URL:** https://hightouch.com/docs/destinations/zapier
**Description:** Use Zapier as a destination if you are trying to quickly migrate system or if you need to connect a service we do not yet support.
**Section:** Destinations
## Overview
Zapier is best used for when there's a destination we do not yet support or if you want make a quick migration to using Hightouch from Zapier. Regardless, it's highly recommend you follow the steps to ensure a smooth setup as Zapier is quite unique compared to our other destinations
## Using Zapier
Due to limitations, we use Zapier's webhook trigger to send data as it's they easiest way set up for users.
#### Rate limits
Something to keep in mind, as you process all your rows of data is that Zapier limits the amount of times Hightouch can send data to the webhook. For example, Zapier **only accept 10,000 requests every 5 minutes**. It's important to keep this in mind when selecting which triggers you want to activate for sending your records, as each record triggered will count as 1 request sent.
[More Information On Zapier's Webhook Limit](https://zapier.com/help/troubleshoot/behavior/rate-limits-and-throttling-in-zapier#step-4)
## Setup
### Connect to Zapier
#### Creating Webhook URL in Zapier
1. Log in to Zapier
2. Click on **Create Zap** (skippable if editing zaps)
3. Click on **Trigger** tab and select the **Webhooks by Zapier** trigger
4. Click on **Trigger Event**, select **Catch Hook**, and then click **Continue**
5. Copy your custom webhook URL
6. Click **Continue**
it's important to not test the trigger until you are in the sync creation
step
#### Adding destination in HT
1. Log in to Hightouch or create a free Hightouch account
2. Click on **Destinations** in the left sidebar
3. Click on **Add destination** in the top right corner
4. Select **Zapier** in the destination catalog and click **Continue** to proceed
5. Paste your webhook URL from Zapier into the **Zapier Webhook Trigger URL ** field and click **Continue** to proceed
6. Give your destination a name and unique slug (for example, "Zapier (Google Sheet Zap)" and `zapier-googlesheet-zap`)
7. Click **Finish** to create your Zapier destination.
### Connect to your data source
Hightouch sits on top of your data warehouse and can read from Snowflake, Redshift, BigQuery, Databricks, Postgres, MySQL, and many other data sources. You can even fetch data from Google Sheets and Airtable. Hightouch can help activate your data wherever it resides.
We've written dedicated guides for each supported data source. Take your pick:
- Connect to [Airtable](/sources/airtable)
- Connect to [Amazon Athena](/sources/amazon-athena)
- Connect to [Amazon Redshift](/sources/amazon-redshift)
- Connect to [ClickHouse](/sources/clickhouse)
- Connect to [Databricks](/sources/databricks)
- Connect to [Elasticsearch](/sources/elasticsearch)
- Connect to [Firebolt](/sources/firebolt)
- Connect to [Google BigQuery](/sources/google-bigquery)
- Connect to [Google Sheets](/sources/google-sheets)
- Connect to [Looker](/sources/looker)
- Connect to [Metabase](/sources/metabase)
- Connect to [MySQL](/sources/mysql)
- Connect to [Palantir Foundry](/sources/foundry)
- Connect to [PostgreSQL](/sources/postgresql)
- Connect to [Rockset](/sources/rockset)
- Connect to [SFTP](/sources/sftp)
- Connect to [Snowflake](/sources/snowflake)
- Connect to [SQL Server](/sources/sqlserver)
- Connect to [Trino](/sources/trino)
If you're missing the credentials necessary to connect to your data source,
try inviting your teammates to Hightouch. You can have unlimited collaborators
in your workspace.
#### Connect to a sample database instead
Waiting for credentials to your data warehouse? No problem. You can connect to one of our [sample datasets](/sources/sample-data) to get started right away. For the rest of this tutorial, we'll use the **B2B SaaS** sample source, which contains a `users` table with names, emails, locations, and other user attributes. This sample source is available when you log in for the first time—no setup required.
### Create a model
In Hightouch, a model represents a query that filters or transforms the records in your data source. Models are used to determine exactly which records to sync between sources and destinations.
In this example, we'll keep it simple. Suppose you want to use Zapier to retrieve information for your customers located in the city of Nashville. To accomplish this, you'll want to create a model that looks like this:
1. Click on **Models** in the left sidebar
2. Click on **Add model** in the top right corner
3. Select **Demo Database** as your source and click **Continue** to proceed
4. Select **SQL Editor** as your modeling method
5. Paste following the SQL query into the editor on the left side of the page
```sql
SELECT * FROM users WHERE location = 'Nashville';
```
6. Click **Preview** to execute the query
7. Click **Continue** to proceed
8. Give your model a name and unique slug (for example, "Users in Nashville" and `users-in-nashville`)
9. Select **ID** as the primary key for your model
10. Click **Finish** to create your model.
### Create a sync
1. Click on **Syncs** in the left sidebar
2. Click on **Add sync** in the top right corner
3. Select **Users in Nashville** as your model and click **Continue** to proceed
4. Select **Zapier** as your destination and click **Continue** to proceed
5. Select whether you want to map your data or keep as is
6. Click on **Acknowledgement of testing** checkbox to en
7. Select **Create a new segment**
8. Give a name to your segment (for example, "My beta test")
9. Click **Continue** to proceed to the last step
10. Go back to the Zap you created, and click **Test Trigger**
11. Verify that a request was found in Zapier
12. Go back to Hightouch, and click **Continue** to proceed to the last step
13. Select **Interval** to tell your sync to run on a set interval
14. Configure your sync schedule to run every **5 minutes**
15. Click **Finish** to create your sync.
## Tips and troubleshooting
### Common errors
#### Couldn't find request
There are a few reasons you may encounter this error:
1. You tried to test the webhook trigger in Zapier, before you tested it in Hightouch.
2. You tested in Hightouch, however you may have tested it for a trigger you did not enable. For example, if you selected to trigger on **Record Changed**, you need to make sure run test as **Sync As Row Changed**
### Live debugger
### Sync alerts
---
## Zendesk
**URL:** https://hightouch.com/docs/destinations/zendesk
**Description:** Deliver better support by making user events and attributes available inside Zendesk
**Section:** Destinations
## Setup
Navigate to Admin Settings > Apps and integrations > Zendesk API and add a new API token for Hightouch. Make sure you enable _Token access_
Copy the API token and fill in the fields in Hightouch destination setup:
- **API token:** token from Zendesk
- **Email:** email used to login to Zendesk
- **Support URL:** URL for your Zendesk support site. make sure to include the `https://` (ex: `https://hightouch.zendesk.com`)
## Syncing
### Supported objects
This integration support syncing the following objects:
- `Users`
- `Organizations`
- `Tickets`
- `Custom Objects`
- `Legacy Custom Objects`
- `Organization Memberships`
Zendesk has [custom objects](https://support.zendesk.com/hc/en-us/articles/5914453843994-Understanding-custom-objects) and [legacy custom objects](https://support.zendesk.com/hc/en-us/articles/4408834725402). If your Zendesk account has any custom objects, they show up in the objects dropdown.
Custom objects have a set prefix: **(Custom)**. Legacy custom objects have a set prefix: **(Legacy custom)**
### Sync modes
- **Upsert**: pushes new objects to Zendesk and updates fields that change in your warehouse.
- **Update**: updates particular fields on existing objects in Zendesk. It doesn't add new objects.
- **Insert**: pushes new objects to Zendesk. It doesn't update any objects. (Only available for `Organization Memberships`)
### Record matching
Records can be matched from your source to your Zendesk objects by selecting a column in your results that matches the external ID, Zendesk ID, email \(_contact only_\), or custom fields of the Zendesk object.
#### Organization memberships
Organization memberships will behave differently. These objects will be matched by both the `user_id` _and_ `organization_id`.
This field should be a combination of your user and organization columns that you will use in the mappings section. For example,
if you are using the user's `email` and the organization's `external_id`, this field should be: `user_email:org_externalId`. Do not edit/change
either the user or organization field. Instead, you will need to remove the old row and create a new one to represent a new organization membership.
Your model should look similar to below:
### Field mapping
You can sync columns from your source to Zendesk's default fields, as well as custom fields for your users, organizations and tickets objects. For more info on how to add custom fields to users, refer to the [Zendesk docs](https://support.zendesk.com/hc/en-us/articles/203662066).
For custom objects, you will be able to sync to any properties from your defined schema.
### Relationship records
You can create associations between existing object records using a predetermined relationship type. The associations can be found in the field mappings. For more details, you can refer to the [Zendesk docs](https://developer.zendesk.com/api-reference/custom-data/custom-objects-api/relationships).
### Search batch size
Zendesk's search API can return up to 1000 results, but by default Hightouch only searches for 100 records
at a time. You can increase this amount up to 500 for a faster sync.
However, increasing the batch size
can increase the risk of receiving a ['414 - Long URI`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/414) error. Conversely, if you are experiencing these errors
with the default batch size setting, you can lower the batch size to avoid the errors.
---
## Zoho CRM
**URL:** https://hightouch.com/docs/destinations/zohocrm
**Description:** Push enriched data from your warehouse into Zoho CRM for your teams to act upon it
**Section:** Destinations
## Setup
After selecting Zoho CRM from our Destinations catalog, you will be prompted to
initiate an OAuth flow. Connect Zoho CRM via OAuth by granting permission to the Hightouch app.
## Syncing
Hightouch supports syncing to the following Zoho CRM Modules:
- `Leads`
- `Accounts`
- `Contacts`
- `Deals`
- `Campaigns`
- `Cases`
- `Solutions`
- `Products`
- `Vendors`
- `Price Books`
- `Quotes`
- `Sales Orders`
- `Purchase Orders`
- `Invoices`
- `Custom Modules`
### Sync modes
- **Upsert** - pushes new records to a module and all fields will be kept up-to-date within Zoho CRM.
### Record matching
Records can be matched from your source to your Zoho CRM workspace by using Zoho's _System-defined
Duplicate Check Fields_ or an _External ID_ you set for a module in your workspace settings.
To know more about each modules's _Sytem-defined Duplicate Check Fields_, refer to the _Sytem-defined Duplicate Check Fields_
section in the [Upsert Records API](https://www.zoho.com/crm/developer/docs/api/v2/upsert-records.html). Additionally, to learn
about creating an _External_ID_, refer to [this Zoho docs article](https://www.zoho.com/crm/developer/docs/api/v2/records-api-ext-id-overview.html).
### Field mapping
You can sync columns from your model to Zoho's modules. Ensure the module's _mandatory field_
is included in the sync and the model columns representing the module fields are of the correct data type.
To know more about each module's _mandatory fields_, refer to the _System-defined mandatory fields for
each module_ section in [this Zoho docs page](https://www.zoho.com/crm/developer/docs/api/v2/insert-records.html). Additionally,
to know more about module field types, refer to the _Sample Attributes_ section.
### Delete mode
You can choose what Hightouch's behavior is when records leave the query result set. The default is doing nothing, but you can also set Hightouch to delete the Zoho CRM record.
---
## Zoomd
**URL:** https://hightouch.com/docs/destinations/zoomd
**Description:** Target and suppress customers with mobile ads across 300M mobile devices using AI-driven insights, ensuring more effective and personalized campaigns.
**Section:** Destinations
## Supported syncing
Type | Description | Supported Sync Modes | API Reference
--------------|-----------------------------------------------|----------------------|-----------------------------------------------------------------------------------
**Audiences** | Sync data from any source to Zoomd audiences | Add, Remove | [Audiences docs](https://developers.kayzen.io/reference/audience-creation-via-api)
For more information about sync modes, refer to the [sync modes](/syncs/types-and-modes#sync-modes) docs.
## Connect to Zoomd
Go to the [**Destinations** overview page](https://app.hightouch.com/destinations) and click the **Add destination** button. Select **Zoomd** and click **Continue**. You can then authenticate Hightouch to **Zoomd**.
Enter the following fields into Hightouch:
- **API key**
## Sync configuration
Once you've set up your Zoomd destination and have a [model](/getting-started/concepts#models) to pull data from, you can set up your sync configuration to begin syncing data. Go to the [**Syncs** overview page](https://app.hightouch.com/syncs) and click the **Add sync** button to begin. Then, select the relevant model and the Zoomd destination you want to sync to.
### Syncing audiences
Sync data from any source to Zoomd audiences.
#### Record matching
To match rows from your model to audiences in Zoomd, you need to select a model column and corresponding Zoomd field. You can match on any of the following Zoomd fields:
- **IDFA**
- **GAID**
#### Field mapping
Hightouch lets you sync audience fields via [field mapping](/syncs/mapping-data).
You can map data from any of your model columns to the default audience fields.
Ensure your model's columns have the same data types as the fields you want to sync to.
## Tips and troubleshooting
### Common errors
### Live debugger
### Sync alerts
---
## Zuora
**URL:** https://hightouch.com/docs/destinations/zuora
**Description:** Sync business-critical data from your data warehouse to Zuora with Hightouch
**Section:** Destinations
## Setup
After selecting Zuora from our Destinations catalog, you will be prompted to enter your Zuora Base URL, Client ID, and Client Secret.
Use [this table](https://www.zuora.com/developer/api-reference/#section/Introduction/Access-to-the-API) from Zuora to select the correct Zuora URL, and follow
[these steps](https://knowledgecenter.zuora.com/Billing/Tenant_Management/A_Administrator_Settings/Manage_Users?_gl=1%2Ap9mz1o%2A_ga%2AMzI5ODc0MjYxLjE2NDY3NjQxMDI.%2A_ga_MY8CQ650DH%2AMTY1NDA0MTU1MC40My4xLjE2NTQwNDE2MTIuMA..&_ga=2.23071486.552820277.1653943857-329874261.1646764102#Create_an_OAuth_Client_for_a_User)
to create an OAuth Client in Zuora that will provide you with the required Client ID and Secret.
## Syncing
Hightouch supports syncing to the following Zuora objects:
- `Accounts`
- `Subscriptions`
- `Usage`
### Sync modes
Here are the possible modes for how to alter Outreach objects:
- **Upsert**: pushes new objects to Zuora and updates fields that change in your warehouse.
- Supported for `Accounts` and `Subscriptions`.
- **Update**: updates particular fields on existing objects in Zuora. It doesn't add new objects.
- Supported for `Accounts`, `Subscriptions`, and `Usage`.
- **Insert**: adds new objects in Zuora.
- Supported for `Usage`.
### Record matching
Records can be matched from your source to your Zuora workspace by the given fields in the dropdown.
The following are supported record ID's for each object:
- **`Accounts`**: `Account Number`, `Account ID`
- **`Subscription`**: `Subscription Number`, `Subscription ID`
- **`Usage`**: `Usage Record ID`
A record ID is required when performing `Upsert` or `Update` actions, but not
`Insert`.
### Field mapping
You can sync columns from your source to Zuora's object properties, including
custom fields.
#### Required fields
Each object / action pair will have their own required properties to be mapped. These required properties will be in their own mapping section during sync configuration. Be sure to scroll down to our [Zuora Object Notes](#object-notes) to see which fields are required, and their respective data types.
#### Optional and custom fields
Any optional built-in fields or custom fields sent to Zuora will be mapped in the custom mappings section of sync configuration. Scroll down to our [Zuora Object Notes](#object-notes) to see Zuora's optional built-in fields, and their respective data types.
#### Helpful tip:
Use our advanced mapper to fill in static values for numbers or string values that may be consistent across your sync, that may not have a column in your data warehouse. See the example below.
### Zuora object notes
#### Accounts
The following fields can be used for `Accounts` for each respective action.
{/* */}
`Upsert` *(click to view table)*
| Field Name | Notes | Required |
| ----------------------- | ------- | -------- |
| accountNumber | string |
| applyCredit | boolean |
| applyCreditBalance | boolean |
| autoPay | boolean |
| batch | string |
| billCycleDay | integer | yes |
| billToContact.firstName | string | yes |
| billToContact.lastName | string | yes |
| collect | boolean |
| communicationProfileId | string |
| creditMemoReasonCode | string |
| creditMemoTemplateId | string |
| crmId | string |
| currency | string | yes |
| debitMemoTemplateId | string |
| documentDate | string |
| invoice | boolean |
| invoiceCollect | boolean |
| invoiceTargetDate | string |
| invoiceTemplateId | string |
| name | string | yes |
| notes | string |
| parentId | string |
| paymentGateway | string |
| paymentTerm | string |
| runBilling | boolean |
| salesRep | string |
| sequenceSetId | string |
| soldToSameAsBillTo | boolean |
| tagging | string |
| targetDate | string |
| Class\_\_NS | string |
| CustomerType\_\_NS | string |
| Department\_\_NS | string |
| IntegrationId\_\_NS | string |
| IntegrationStatus\_\_NS | string |
| Location\_\_N | string |
| Subsidiary\_\_NS | string |
| SyncDate\_\_NS | string |
| SynctoNetSuite\_\_NS | string |
`Update` *(click to view table)*
| Field Name | Notes |
| ------------------------- | ------- |
| autoPay | boolean |
| batch | string |
| billCycleDay | integer |
| billToContact | Contact |
| communicationProfileId | string |
| creditMemoTemplateId | string |
| crmId | string |
| debitMemoTemplateId | string |
| defaultPaymentMethodId | string |
| invoiceDeliveryPrefsEmail | boolean |
| invoiceDeliveryPrefsPrint | boolean |
| invoiceTemplateId | string |
| name | string |
| notes | string |
| parentId | string |
| paymentGateway | string |
| paymentTerm | string |
| salesRep | string |
| sequenceSetId | string |
| soldToContact | Contact |
| tagging | object |
| Class\_\_NS | string |
| CustomerType\_\_NS | string |
| Department\_\_NS | string |
| IntegrationId\_\_NS | string |
| IntegrationStatus\_\_NS | string |
| Location\_\_NS | string |
| Subsidiary\_\_NS | string |
| SyncDate\_\_NS | string |
| SynctoNetSuite\_\_NS | string |
Notes:
- By default, `autoPay` is set to `false` for inserted `Accounts`, unless it's defined in the mappings for your sync.
- If you would like to set `autoPay` to true, you must include either a `paymentMethod`, `creditCard`, or `hmpCreditCardPaymentMethodId` in your mappings.
- If either the `workEmail` or `personalEmail` fields are included in mappings, the account's email delivery preference is automatically set to `true`.
- To see more information on any respective field, view the Zuora API docs for [Creating an account](https://www.zuora.com/developer/api-reference/#operation/POST_Account) or [Updating an account](https://www.zuora.com/developer/api-reference/#operation/PUT_Account) via the API.
#### Subscriptions
The following fields can be used for `Subscriptions` for each respective action.
`Upsert` *(click to view table)*
| Field Name | Notes | Required |
| -------------------------------------- | ------- | -------- |
| accountKey | string | yes |
| applyCredit | boolean |
| applyCreditBalance | boolean |
| autoRenew | boolean |
| collect | boolean |
| contractEffectiveDate | string | yes |
| creditMemoReasonCode | string |
| customerAcceptanceDate | string |
| documentDate | string |
| externallyManagedBy | string |
| gatewayId | string |
| initialTerm | integer |
| initialTermPeriodType | string |
| invoice | boolean |
| invoiceCollect | boolean |
| invoiceOwnerAccountKey | string |
| invoiceSeparately | boolean |
| invoiceTargetDate | string |
| notes | string |
| paymentMethodId | string |
| renewalSetting | string |
| renewalTerm | integer | yes |
| renewalTermPeriodType | string |
| runBilling | boolean |
| serviceActivationDate | string |
| subscriptionNumber | string |
| subscribetoRatePlans.productRatePlanId | string | yes |
| targetDate | string |
| termStartDate | string |
| termType | string | yes |
| CpqBundleJsonId\*\*QT | string |
| OpportunityCloseDate\*\*QT | string |
| OpportunityName\*\*QT | string |
| QuoteBusinessType\*\*QT | string |
| QuoteNumber\*\*QT | string |
| QuoteType\*\*QT | string |
| IntegrationId\*\*NS | string |
| IntegrationStatus\*\*NS | string |
| Project\*\*NS | string |
| SalesOrder\*\*NS | string |
| SyncDate\_\_NS | string |
`Update` *(click to view table)*
| Field Name | Notes |
| -------------------------------- | ---------------- |
| add | Array of objects |
| applicationOrder | Array of strings |
| applyCredit | boolean |
| applyCreditBalance | boolean |
| autoRenew | boolean |
| clearingExistingBillToContactId | boolean |
| clearingExistingPaymentTerm | boolean |
| collect | boolean |
| creditMemoReasonCode | string |
| currentTerm | integer |
| currentTermPeriodType | string |
| documentDate | string |
| externallyManagedBy | string |
| includeExistingDraftDocItems | boolean |
| includeExistingDraftInvoiceItems | boolean |
| invoice | boolean |
| invoiceCollect | boolean |
| invoiceSeparately | boolean |
| invoiceTargetDate | string |
| notes | string |
| preview | boolean |
| previewType | string |
| renewalSetting | string |
| renewalTerm | integer |
| renewalTermPeriodType | string |
| runBilling | boolean |
| targetDate | string |
| termStartDate | string |
| termType | string |
| CpqBundleJsonId\_\_QT | string |
| OpportunityCloseDate\_\_QT | string |
| OpportunityName\_\_QT | string |
| QuoteBusinessType\_\_QT | string |
| QuoteNumber\_\_QT | string |
| QuoteType\_\_QT | string |
| IntegrationId\_\_NS | string |
| IntegrationStatus\_\_NS | string |
| Project\_\_NS | string |
| SalesOrder\_\_NS | string |
| SyncDate\_\_NS | string |
{/* */}
Notes:
- Only the fields to be changed should be specified. If an empty field is submitted with this operation, the corresponding field in the account is emptied.
- To see more information on any respective field, view the Zuora API docs for [Creating a subscription](https://www.zuora.com/developer/api-reference/#operation/POST_Subscription) or [Updating a subscription](https://www.zuora.com/developer/api-reference/#operation/PUT_Subscription) via the API.
#### Usage
The following fields can be used for `Usage` for each respective action.
`Insert` *(click to view table)*
| Field Name | Type | Required |
| ------------------ | -------------------- | -------- |
| AccountId | string |
| AccountNumber | string |
| ChargeId | string |
| ChargeNumber | string |
| Description | string |
| EndDateTime | string \ |
| Quantity | double | yes |
| StartDateTime | string \ | yes |
| SubmissionDateTime | string \ |
| SubscriptionId | string |
| SubscriptionNumber | string |
| UOM | string | yes |
`Update` *(click to view table)*
| Field Name | Notes |
| ------------------ | -------------------- |
| EndDateTime | string \ |
| Quantity | double |
| RbeStatus | string |
| StartDateTime | string \ |
| SubmissionDateTime | string \ |
| UOM | string |
Notes
- Any field with a type of `string ` must be in the format `"YYYY-MM-DDTHH:mm:ss"`.
- To see more information on any respective field, view the Zuora API docs for [Creating a usage record](https://www.zuora.com/developer/api-reference/#operation/Object_POSTUsage) or [Updating a usage record](https://www.zuora.com/developer/api-reference/#operation/Object_PUTUsage) via the API.
---
## Build audiences with AI
**URL:** https://hightouch.com/docs/customer-studio/agents
**Description:** Create, refine, and understand audiences using natural language in Customer Studio.
**Section:** Customer Studio
The Agent is included with [Customer Studio](/customer-studio/overview) at no additional cost.
| | |
| ----------------- | --------------------------------------------------------------------------------- |
| **Audience** | Marketers, growth teams, and campaign managers |
| **Prerequisites** | [Customer Studio setup](https://hightouch.com/docs/customer-studio/quick-start) |
Use the Agent in Customer Studio to create, refine, and understand audiences with natural language. Describe the audience you want to build, ask questions about an existing audience, or request improvements. The Agent translates your request into audience filters, summaries, or insights.
---
### Learning objectives
In this article, you'll learn how to:
- [Build audiences using natural language prompts](#build-audiences-with-natural-language)
- [Summarize, analyze, and explore existing audiences](#understand-and-analyze-audiences)
- [Refine audience definitions through follow-up prompts](#refine-and-improve-audiences)
- [Improve results with more specific prompts and schema metadata](#tips-for-better-results)
- [Understand how Hightouch handles data when you use the Agent](#ai-and-data-privacy)
---
## Overview
The Agent is available when you build or edit an audience in Customer Studio. Select **Agent** to open a chat panel alongside the visual builder.
From the Agent panel, you can:
- Create audience definitions from natural language prompts
- Ask questions about an existing audience
- Generate summaries and insights
- Refine audience filters through follow-up prompts
- Explore available events, traits, columns, and properties
---
## Get started
### Open the Agent
1. Go to [**Customer Studio > Audiences**](https://app.hightouch.com/customer-studio/audiences).
2. Create a new audience from [scratch](/customer-studio/usage) or from a [template](/customer-studio/templates), or open an existing audience.
3. Open the **Definition** tab.
4. Select **Agent**.
### Quick actions
The Agent shows different quick actions depending on whether you're working with a new or existing audience. Select a quick action to start, or type your own prompt.
| Context | Sample quick actions |
| -------------------- | --------------------------------------------------------------------------------- |
| **New audience** | **Create audience**, **Start with a template** |
| **Existing audience**| **Generate insights**, **Suggest improvements**, **Summarize audience** |
---
## Build audiences with natural language
Describe the audience you want to build, and the Agent creates an audience definition with matching filters and conditions.
1. Open the Agent on a new or existing audience.
2. Enter a prompt that describes the audience you want to create. You can also attach images or files for additional context.
For example:
`Create an audience of customers who added items to their cart in the last 30 days but did not complete an order.`
3. Review the audience definition the Agent creates in the visual builder.
4. Check that the filters and conditions match your intent.
5. If the definition needs changes, send a follow-up prompt. The Agent keeps context from the conversation, so you can refine without starting over.
### Example prompts
| Goal | Example prompt |
| ---------------------------- | --------------------------------------------------------------------------------------------------------- |
| Create a segment from scratch | `Build an audience of high-value customers who purchased in the last 30 days but haven't opened an email recently.` |
| Target a behavioral funnel | `Create an audience of users who opened an email but didn't click in the last 7 days.` |
| Add a suppression condition | `Add a filter to exclude anyone who churned in the past 90 days.` |
| Discover available fields | `What columns can I use to filter by age?` |
| Explore event properties | `What properties are available for the Order Completed event?` |
---
## Understand and analyze audiences
Use the Agent to get plain-language explanations and data-backed analysis of an audience without writing queries or leaving the builder.
Select a quick action like **Summarize audience** or **Generate insights**, or enter your own prompt:
`Summarize this audience in 30 words or less for a business user.`
`Which conditions are excluding the most members from this audience?`
`Break down this audience by most recent order value.`
Agent responses can be verbose. For shorter output, include a word limit or format in your prompt.
---
## Refine and improve audiences
Use the Agent to update an existing audience definition through follow-up prompts. Describe the change you want, then review the updated definition in the visual builder.
You can also send a specific filter to the Agent from the visual builder. Select the **more options** menu on any filter and choose **Add filter to chat**.
### Apply or undo changes
When the Agent suggests changes to your audience definition:
1. Select **Apply these changes** in the Agent panel to update the visual builder.
2. Review the updated filters and conditions in the visual builder.
3. Select **Save audience** to keep the changes, or **Discard changes** to revert to the original definition.
If you've already applied changes, select **Undo** in the banner at the top of the builder to revert before saving.
---
## Tips for better results
### Enrich your schema metadata
The Agent uses your schema metadata — model names, column descriptions, and event labels — to understand your data. Clear, descriptive metadata is the single most effective way to improve Agent results.
### Be specific
Instead of `Find my best customers`, try:
`Find customers who completed at least 3 orders in the last 90 days with an average order value over $100.`
Include time windows, thresholds, and the specific events or filters you want. If you don't want the Agent to modify an existing filter, say so explicitly.
### Ask the Agent what's available
If you're not sure which fields to use, ask the Agent to explore your schema:
`What events and properties are available for audience filtering?`
This is especially useful when working with a data model you didn't set up yourself.
### Review before saving
Always review the audience definition in the visual builder before saving. The Agent can make mistakes, especially with complex logic, vague prompts, or schema fields with unclear names.
---
## AI and data privacy
### How data is processed
When you use the Agent, Hightouch sends relevant audience context to the AI model so it can generate a response. This context can include attributes, behavior signals, and schema metadata.
This data is:
- Not retained after the response is generated
- Not used to train or fine-tune models
- Governed by Hightouch's [AI Terms of Service Addendum](https://hightouch.com/ai-addendum)
### Chat privacy
Your conversations with the Agent are private to you. Other team members can't see your audience-specific chats.
---
## Audience overviews
**URL:** https://hightouch.com/docs/customer-studio/audience-overviews
**Description:** Audience overviews enable visual confirmation of audience construction and health.
**Section:** Customer Studio
| | |
| ----------------- | ---------------------------------------------------------------------- |
| **Audience** | Marketers |
| **Prerequisites** | A saved audience. (Optional) At least one dashboard |
_Use the Audience Overview as your audience’s homepage to track size, activity, and sync health, or attach a custom dashboard to surface the metrics most relevant to your goals._
The Audience Overview page is the landing page for all of your audiences. You’ll see a dashboard of charts that help you validate how an audience is built and monitor its health over time.
## What you’ll see
When you open an audience, the **Overview** tab shows a default dashboard with key charts:
* **Audience size**: Current total number of members
* **Audience size over time**: A line chart showing membership growth and change trends
* **Audience events**: Top events by members of the audience
* **Syncs**: Completed syncs over past 7 days
You will need to refresh to view chart information.
### Saved metrics
If your workspace has defined [metrics](https://hightouch.com/docs/campaign-intelligence/metrics) for your parent model (for example, purchases or revenue), these appear as key KPIs in the Audience Overview.
These appear at the end of the default dashboard. If you use a custom dashboard, these will not appear.
## Custom Dashboards
You can replace the default [dashboard](https://hightouch.com/docs/campaign-intelligence/dashboards) with one of your own:
1. Go to **Audience settings** (during creation or later by editing)
2. Under **Overview dashboard**, select an existing dashboard to attach
The attached dashboard automatically filters to your audience.
**Key points:**
* The same dashboard can be reused across multiple audiences
* Global dashboard filters are hidden automatically to avoid conflicts when rendered in the audience overview. The original dashboard’s filters remain intact.
## Features and behaviors
* **Time filters**: You can switch between 7, 30, 60, 90-day, or custom windows.
* **Caching**: Charts respect the same caching rules as dashboards, so repeated queries are optimized.
* **Sampling**: If an audience uses sampling, the Overview reflects those settings.
* **Permissions**: Anyone who can view details about an audience's members can also see its Overview. Only editors can change or attach dashboards.
* **AI description**: You can automatically generate a semantic summary of your audience.
---
## Snapshots
**URL:** https://hightouch.com/docs/customer-studio/audience-snapshots
**Description:** Audience snapshots give data marketers a record of membership history over time.
**Section:** Customer Studio
| | |
| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | **Platform admins** who want a historical record of audience membership to analyze when users entered or exited campaigns. |
| **Prerequisites** |
A source with [Lightning sync engine](https://hightouch.com/docs/syncs/lightning-sync-engine) enabled
Warehouse [source](https://hightouch.com/docs/getting-started/concepts#sources) with write permissions
|
*Audience snapshots give marketers a historical record of who was in an audience and when. This is useful for evaluating campaign reach, tracking changes in membership over time, and enabling downstream analysis or reporting.*
***
## Learning objectives
After reading this article, you'll be able to:
* Understand how audience snapshots work in Customer Studio
* Enable snapshotting on your source
* Access and interpret snapshot data in your warehouse
***
### Overview
After each sync of an audience, Customer Studio saves a **snapshot** of that audience's membership. These snapshots are written to your data warehouse and include:
* *Only* **audience member IDs** and no other customer data
* A **timestamp** for when each member entered or exited the audience
* The event type (`enter` or `exit`)
* An optional **split group ID**, if audience [splits](https://hightouch.com/docs/customer-studio/splits) are enabled
Audience snapshotting requires `WRITE` permissions to your warehouse. Snapshots are only available for sources using the [Lightning sync engine](https://hightouch.com/docs/syncs/lightning-sync-engine).
## Snapshot table schema
Snapshots are stored in a single shared table in the [**Hightouch Planner schema**](https://hightouch.com/docs/syncs/lightning-sync-engine#warehouse-schemas) in your warehouse: `hightouch_planner.audience_membership`
| Column | Type | Description |
| ---------------- | ----------- | --------------------------------------------------------------------------------------- |
| `ht_row_id` | `string` | The primary key of the user. Stringified if the original value is not a string. |
| `ht_audience_id` | `string` | The ID of the audience. |
| `ht_timestamp` | `timestamp` | When the user entered or exited the audience. |
| `ht_event_type` | `string` | Either `enter` or `exit`, depending on the change. |
| `ht_split_group` | `string` | The split group ID the user belongs to (if applicable). `null` if no split was applied. |
Each table row contains the audience member's ID, the timestamp of when the snapshot occurred, and either an `enter` or `exit` event type.
An audience's first snapshot records current membership in the form of `enter` events for every member.
Subsequent snapshots only contain changes to membership—either `enter` events for new members or `exit` events for members who aren't present in that snapshot but were in the previous one.
## Enable snapshots
To enable snapshots for a given source:
1. Navigate to your [**Source**](https://hightouch.com/docs/getting-started/concepts#sources)
2. Go to the `Sync log` tab.
3. Enable **Audience snapshotting** using the toggle.
Once enabled, snapshots will be automatically taken after each successful sync.
---
## Coupon assignment
**URL:** https://hightouch.com/docs/customer-studio/coupons
**Description:** Use coupon assignment to attach unique coupons (and related data) to users.
**Section:** Customer Studio
## Overview
Coupon assignment lets you attach per-user values (such as coupon codes or offer metadata) after defining an audience and before sending data to your destinations.
This is useful when:
- you need to assign **unique values per user** (e.g. single-use coupons)
- values come from a **shared dataset** (e.g. a coupon table)
- you shouldn't assign a coupon to a user that **isn't in your campaign**
---
## How it works
Coupon assignment happens inside an **audience-based Journey**. For each user, Hightouch assigns a value from a dataset and makes it available for downstream syncs.
---
## Example
- Parent model: **10,000 users**
- Campaign A audience: **500 users**
- Coupon pool: **500+ unique codes**
---
## Step 1: Create a coupon pool
Create a table in your warehouse that contains your coupon data.
Include:
- a coupon value column (e.g. `coupon_code`)
- optional metadata (e.g. `expiration_date`)
- a column filter on if you have cross-campaign coupons consolidated in one model (e.g. `campaign_id`)
You can set it up in your Customer Studio schema with a `coupons` tag.
---
## Step 2: Create an audience and Journey
Build an audience for your campaign (e.g. **Campaign A**) and create a Journey with:
> User is in Audience: Campaign A
Only users in this audience will receive a coupon.
Event based journeys are not supported.
---
## Step 3: Add a coupon assignment node
In the **Assign coupon** node:
- select your coupon dataset
- apply filters (e.g. `campaign_id = 10_dollars_off`)
- choose an "assignment" columns (e.g. `coupon_code`). This is the column that will be unique (1:1) to a user.
- choose any additional columns to assign (e.g. `expiration_date`, `coupon_threshold`). These do not need to be unique to the user.
- define variable names
For each user:
1. **Reuse existing assignment** (if present)
2. **Assign a new coupon** (if not)
3. **Store the result** in:
- Journey variables (for syncs)
- Journey logs
Assignments are deterministic per coupon journey node: the same user will receive the same values when entering the same journey node.
---
## Step 4: Sync assigned values
Use the assigned variables in a Sync node (e.g. to Braze or Salesforce). If you do not see the assigned variables in the Sync node, make sure to save the Journey first.
---
## Tracking
Assignments are recorded in Journey logs and can be used to:
- map coupons to users
- join with downstream usage or redemption data
---
## Behavior and constraints
- **Inventory required:** If coupons < audience size, the Journey fails before sending.
- **Consistent assignment**: Even if the underlying coupon pool changes, what coupon and coupon metadata a user is assigned stays the same if they visit the same coupon assignment node again. If you would like the user to be assigned a different value, use a different coupon assignment node.
---
## Data masking
**URL:** https://hightouch.com/docs/customer-studio/data-masking
**Description:** Data masking capabilities to help protect sensitive information, including personally identifiable information.
**Section:** Customer Studio
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Platform admins |
| **Prerequisites** | [Set up for platform admins and data teams →](https://hightouch.com/docs/customer-studio/quick-start#setup-for-platform-admins-and-data-teams) |
*Protect personally identifiable information by automatically detecting sensitive data, and manually reviewing and controlling access as needed.*
## Overview
Data masking automatically detects and allows you to take action towards protecting sensitive data in your datasets. Hightouch’s data masking feature helps you classify columns, enforce privacy controls, and maintain compliance while preserving data utility.
## Privacy levels
Data masking uses a multi-level privacy system:
**Approved**: Data cleared for unrestricted use.
**Blocked**: Data completely restricted from access and syncs.
**Redacted**: Data hidden in profile details, column suggestions, and analytics. Remains accessible for syncs and audience filters.
**Sync-only**: Data can sync to destinations but is not visible in the Hightouch app outside of syncs.
**Sync-blocked**: Data can be used in the Hightouch app but cannot be synced.
**Unreviewed**: Data not yet reviewed.
Unreviewed data continues to function normally until you assign a privacy level.
## Automatic PII detection
The system automatically flags potential personally identifiable information for review. Auto-flagging only applies to non-reviewed columns and will not override your existing classifications.
## Managing columns
### View and search columns
Access the data masking interface from the [**Governance**](https://app.hightouch.com/governance/) tab. The table view shows all columns sorted by most recently updated.
Use the search bar to filter columns by:
* Model
* Source
* Privacy level
* Flagged status
### Check auto-detected columns
Columns flagged as potential PII display a **warning badge**. You can filter for flagged columns, review them, and apply the correct privacy level.
### Set privacy level
You can set privacy levels individually or in bulk:
1. Select one or more columns.
2. Choose a privacy level from the dropdown:
1. **Blocked**: For highly sensitive data that must never sync downstream.
2. **Redacted**: For columns where data can be synced but values must stay hidden in the Hightouch app.
3. **Sync-only**: For columns where data should be synced to destinations but is not visible in the Hightouch app outside of syncs.
4. **Sync-blocked**: For columns where data can be used in the Hightouch app but cannot be synced.
5. **Approved**: For data cleared through your review process.
3. Apply the privacy level.
Once applied, the system removes auto-detection flags for those columns.
### Categorize columns
To improve management and filtering, assign categories to your columns. For example:
* Email
* Name
* Birthdate
* Address
* Phone
* IP address
## Permissions
Access to data masking is controlled by **schema permissions**. To edit privacy levels and PII categories, users must have `Configure schema` access on the source.
See [Source-level permissions](/workspace-management/roles#source) for more details.
---
## Destination rules
**URL:** https://hightouch.com/docs/customer-studio/destination-rules
**Description:** Destination rules allow organizations to filter out users from destinations they have not consented to during during syncing.
**Section:** Customer Studio
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Platform admins |
| **Prerequisites** | [Set up for platform admins and data teams →](https://hightouch.com/docs/customer-studio/quick-start#setup-for-platform-admins-and-data-teams) |
*Destination rules allow organizations to enforce row-level governance by filtering out users from being synced to destinations they haven’t consented to. This ensures that only eligible data reaches specific destinations based on compliance, privacy, or business rules.*
***
## Learning objectives
* After reading this article, you’ll be able to:
* Understand what destination rules are and when to use them
* Create and configure destination rules in the UI
* Preview the impact of a rule on an audience
* Apply rules automatically to syncs without modifying audience definitions
***
## Overview
Destination rules control which records can be synced to specific destinations. They apply additional filtering on top of the audience definition at **sync time**, without affecting the actual audience membership.
A common use case is enforcing **user consent**: for example, excluding users who haven’t opted in to email or advertising platforms.
Destination rules do not change your audience composition. They only affect **who gets synced** to a destination, not who qualifies for the audience.
## Set up destination rules
1. Go to **Customer Studio > Governance**, then click the [Destination rules tab](https://app.hightouch.com/governance/destination-rules)
2. Select a **parent model**
3. Click **Add destination rule**
3. Select a **destination** from the dropdown
4. Click **Add filter** to define which records are eligible to sync
The filter builder supports the same condition logic used in the [audience builder](https://hightouch.com/docs/customer-studio/usage), including nested `AND/OR` logic.
## Preview destination rules
You can preview the effect of a destination rule in two places:
* During **audience creation**
* From the audience’s **definition tab** (when editing an existing audience)
In both cases:
1. Use the `Preview a destination` dropdown to select a destination with an existing rule.
2. Hightouch applies the rule to your audience definition and updates the preview.
3. You’ll see how the rule impacts calculated size and membership before syncing.
This preview does **not** change the audience itself—it only simulates how the rule would affect syncing to the selected destination.
## Using destination rules in syncs
Create a sync with a parent model (proxied by the audience) and destination that has an existing rule.
Rows that don't meet a destination rule will be excluded at sync time. All syncs going forward will automatically have the rule applied.
Note that these excluded rows are still included in the audience definition, so they remain unaffected when using Customer Studio features that don't involve syncing.
---
## Experiments
**URL:** https://hightouch.com/docs/customer-studio/experiments
**Description:** Audience Experiments unlock seamless management of A/B and multivariate testing efforts by allowing users to create randomized test groups within audiences in Hightouch.
**Section:** Customer Studio
**Experiments** were previously called **Splits**.
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Marketers and analysts who want to compare the effectiveness of different campaign strategies using real audience data. |
| **Prerequisites** |
A saved [audience](https://hightouch.com/docs/customer-studio/usage)
At least one [sync](https://hightouch.com/docs/customer-studio/syncs) or campaign ready to launch
|
*Run controlled experiments directly in Customer Studio by creating randomized groups to test and measure the true impact of your marketing campaigns.*
***
## Learning Objectives
After reading this article, you will be able to:
- [Understand when and why to use audience experiments](#when-to-use-experiments)
- [Create A/B or multivariate experiment groups within a saved audience](#1-create-experiment-groups)
- [Sync each group to separate treatments, campaigns, or destinations](#1-create-experiment-groups)
- [Measure campaign effectiveness using Experiments in Campaign Intelligence](#2-measure-results)
- [Use stratified sampling to ensure balanced test and control groups](#stratified-sampling)
- [Access holdout group logs for detailed post-campaign reporting in your warehouse](#holdout-group-logs)
***
## Overview
Audience **Experiments** enable you to create randomized audience groups for accurate, controlled testing of campaign strategies. By assigning audience members to distinct test groups, you can compare outcomes between treatments, channels, or creatives and understand what’s truly driving performance.
You can also configure experiments directly on **A/B split nodes** within [Journeys](/customer-studio/journeys#ab-split).
---
### What you can measure
- **Incrementality**: Did your campaign actually cause the desired outcome?
- **A/B/n testing**: Compare creatives, messages, or channels.
- **Personalization impact**: Determine whether tailored experiences outperform generic ones.
- **Channel effectiveness**: Identify which channels drive more downstream conversions.
---
### When to use experiments
Use audience experiments when you want to move beyond assumptions like:
- **Correlation**: “Purchases increased—but was it because of the campaign?”
- **Attribution**: “A user converted after seeing my ad—but would they have converted anyway?”
Running a controlled experiment isolates the impact of your campaign by comparing outcomes between randomized groups.
---
## How audience experiments work
| Stage | What it means | Hightouch feature |
|-------------------------|----------------------------------------------|---------------------------|
| 1. Define your audience | Choose your target group | Build audiences |
| 2. Create experiment groups | Randomly assign audience members | Experiments |
| 3. Apply a treatment | Run a campaign for one or more groups | Syncs |
| 4. Measure results | Compare outcomes between groups | Experiments |
| 5. Interpret the impact | Evaluate lift and statistical significance | Experiment results chart |
---
## Example use cases
#### Email campaign holdout
- 50/50 experiment
- Group A receives promotional email
- Group B is held out
- Compare conversions or revenue per user
#### Ad creative A/B test
- Groups A and B synced to separate ad sets
- Compare CTR, CPC, or downstream conversion
#### Personalization test
- Group A receives personalized product recommendations
- Group B receives a generic message
- Compare engagement and purchases
#### Onboarding lifecycle experiment
- Test different onboarding flows
- Measure feature adoption or retention after 14 days
#### Channel comparison
- Group A receives email
- Group B receives paid ads
- Compare downstream LTV or sign-up quality
---
## Setup steps
### 1. Create experiment groups
1. Open your audience in [**Customer Studio**](https://app.hightouch.com/audiences).
2. Navigate to the **`Experiments`** tab.
3. Toggle **`Enable experiment groups`**.
4. Under **`Experiment groups`**, configure:
- Number of groups (e.g., 2 for A/B tests, 3+ for multivariate)
- Percentage distribution (e.g., 50/50)
- Group names (e.g., `Holdout`, `Treatment A`)
5. Optionally add more groups for A/B/n testing.
6. For each group, configure syncs to destinations.
7. Click **`View results`** to begin analysis.
---
### 2. Measure results
Once your audience has at least one sync involving experiment groups, Hightouch automatically generates a corresponding **Experiment**.
Clicking **`View results`** opens the Experiment overview.
You can also access Experiments anytime under:
**`Intelligence` → [`Experiments`](https://app.hightouch.com/experiments)`**
Use Experiments to:
- Compare lift across groups
- View statistical significance
- Analyze results using primary or secondary metrics
- Normalize results per member or against a baseline
Older *Splits measurement charts* have been deprecated. All measurement now occurs in **Experiments**.
[**Learn how to configure and interpret Experiments →**](/campaign-intelligence/experiments)
---
## Stratified Sampling
Simple randomization may not provide balanced groups when your audience contains diverse characteristics (e.g., geography, loyalty tier). **Stratified sampling** ensures proportional representation across experiment groups based on selected variables.
Stratified sampling is only available for one-time syncs.
### How to enable stratified sampling
1. Open your audience and go to the **`Experiments`** tab.
2. Under **`Advanced configuration`**, toggle **`Stratified sampling`**.
3. Select one or more columns to stratify by (e.g., `loyalty_tier`).
4. Save your changes.
---
## Holdout group logs
Holdout group logs are used primarily by **data teams** to perform row-level warehouse analysis. Marketers should rely on Experiment results in the Customer Studio UI.
Holdout group logs track which rows were **excluded** from a sync as part of a holdout (e.g., a 20% control group). This enables advanced SQL or BI analysis.
### How to enable holdout group logs
1. to enable the *Holdout group logs* feature flag. Requirements:
- [Lightning sync engine](https://hightouch.com/docs/syncs/lightning-sync-engine)
- A destination with **write access** to your warehouse
2. In your workspace:
- Go to **`Integrations → Sources`**
- Select a source and open the **`Sync Logs`** tab
- Enable **`Audience holdout group logs`**
When enabled, Hightouch logs excluded rows in the `hightouch_audit.audience_holdout` table.
### Table definition
| COLUMN | DESCRIPTION |
|---------------|-------------|
| `sync_id` | Sync ID |
| `sync_run_id` | Sync run ID |
| `model_id` | Model/audience ID |
| `timestamp` | Timestamp of sync |
| `row_id` | Primary key value from the model |
| `fields` | JSON snapshot of model data |
| `split_group` | Name of the experiment group |
---
## Audience insights
**URL:** https://hightouch.com/docs/customer-studio/insights
**Description:** Audience insights give marketers deeper understanding of the composition of their audiences.
**Section:** Customer Studio
| | |
| ----------------- | ---------------------------------------------------------------------- |
| **Audience** | Marketers and analytics/data engineers |
| **Prerequisites** | [Create audiences →](https://hightouch.com/docs/customer-studio/usage) |
_Understand how to use Insights in Customer Studio to explore metrics, analyze trends, and power downstream targeting and enrichment workflows._
---
## Learning Objectives
After reading this article, you will be able to:
- Calculate and interpret the size of an audience.
- Preview individual audience members before syncing.
- Explore audience overlaps to ensure clean campaign segmentation.
- Use audience breakdowns to understand the distribution of traits and metrics.
- Navigate and use the **Show insights** panel during audience definition.
- Explore members of your parent model outside of a specific audience.
---
## Overview
Understand and validate the size, composition, and characteristics of your audience using **Insights**. While defining or editing an audience in Customer Studio, you can view summary metrics, compare overlaps, and break down key traits inside the visual builder.
In addition to audience-level insights, Customer Studio includes [Campaign Intelligence](https://hightouch.com/docs/campaign-intelligence/overview)—a suite of tools for deeper analysis across campaigns, including [insights charts](https://hightouch.com/docs/campaign-intelligence/insights), [metrics](https://hightouch.com/docs/campaign-intelligence/metrics), [funnels](https://hightouch.com/docs/campaign-intelligence/funnels), and [split tests](https://hightouch.com/docs/campaign-intelligence/splits).
## Access audience insights
You can access insights while creating or editing an audience in two places:
- In the **audience creation flow**, during the `Define audience` step
- In the **audience editor**, under the `Definition` tab
Click the `Show insights` button in the top-right of the [filter builder](https://hightouch.com/docs/customer-studio/usage#step-2-add-filters) toolbar.
## Calculate audience size
To see how many users match your audience filters, click `Calculate size`. The audience size is shown at the top of the builder and is updated when filters change or a sync occurs.
You can also calculate the size of specific **filter groups** by clicking `Calculate size` within any top-level `AND` block. This helps you understand how each filter group contributes to—or reduces—overall audience membership.
When [schema labeling](https://hightouch.com/docs/customer-studio/schema#schema-labels) is enabled for a People ↔ Household or People ↔ Account relationship, you can view the size of both models.
For example, if your audience is based on a People model, you can also see how many Households or Accounts contain at least one person matching your filters.
If sampling is enabled on your data models, preview results in the audience
builder may be returned using sampled data for faster performance. [Learn more
about sampling.](https://hightouch.com/docs/customer-studio/sampling)
## Member preview
Click the size preview count to more closely inspect the list of members who currently meet your audience criteria and ensure you're building the correct audience. If you've chosen to [redact any columns](/customer-studio/schema#columns) in your model configuration, they are redacted here.
Clicking on a particular member will open the member details panel to give you an even deeper look at your audience members.
When [schema labeling](https://hightouch.com/docs/customer-studio/schema#schema-labels) is enabled for a People ↔ Household or People ↔ Account relationship, you can view details for both models and see the membership between them.
For example, if your audience is based on a People model, you can view the Households or Accounts that contain at least one person matching your filters.
Within each person, you can see which households or accounts they belong to, and vice versa.
## Audience overlaps
Audience Overlaps can only be set up between audiences that use the same
[parent
model](https://hightouch.com/docs/customer-studio/schema#1-define-the-parent-model).
With many audiences syncing simultaneously, it can be difficult to ensure that end users aren't being targeted by multiple campaigns.
**Audience overlaps** help you validate the composition of your audiences by letting you compare one audience against another. You can find audience overlaps within the `Show insights` panel when defining your audience.
Select a previously created audience from the dropdown to compare the audiences.
For more information, check out the [**Audience Overlaps** blog post](https://hightouch.com/blog/feature-release-audience-overlaps).
## Audience breakdowns
**Breakdowns** help quickly check your audience segmentation and ensure the audience meets your needs by enabling you to better understand the distribution of specific characteristics within audiences.
Breakdowns help you explore how an audience is distributed across key traits or metrics—for example, how many users fall into each region, loyalty tier, or spending bucket.
To create a breakdown:
1. Open the `Show insights` panel.
2. Scroll to the **Breakdowns** tab.
3. Choose a metric or property, such as:
- **Saved metrics** (custom metrics from your workspace)
- **Events** (e.g., Bookings)
- **Relations** (e.g., Preferred Hotel)
- **Properties** (e.g., Region, Language)
- **Traits** (custom trait columns)
You can also **group by** a user property (e.g., group `Total Bookings` by `Region`) and choose an **aggregation type** (e.g., count, average).
Breakdowns are shown as charts within the panel, helping you visually validate whether your audience reflects your desired segmentation.
### Numerical metrics
When you select a numerical property or a model-based metric (e.g. events), you can optionally select a non-numerical user property to group the metric by.
|  |  |
| ------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------- |
Except in the case of `Saved metrics`, if no `Group by` option is selected, the chart will group by the metric itself.
|  |  |
| ---------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
Depending on the type of numerical metric you select, you are able to choose additional options on what to view in your breakdown.
|  |  |
| --------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
### Further exploration
To explore a breakdown in more depth, you can open it in [Charts](https://hightouch.com/docs/campaign-intelligence/overview) to apply additional filters and groupings, explore different visualizations, and save the chart for future reference.
For more information, check out the [**Audience Breakdowns** blog post](https://hightouch.com/blog/audience-breakdowns).
## Profiles
In cases where you haven't built a specific audience, but need to explore members of your parent model to understand available attributes or lookup specific members to troubleshoot or understand behavior, use the Profiles tab.
After selecting a parent model, you can search members by Primary Key, or use the Filter option to locate users using any of the conditions available throughout Customer Studio.
Select a specific member to inspect their profile and dive into their attributes, recent events, key metrics, audience memberships and more.
## FAQ
#### Is our insight data saved by Hightouch?
**No**, Hightouch never saves your data.
#### Do Breakdowns and Overlaps reflect if an audience belongs to a priority list?
Yes-the insight tools take [audience priority](/customer-studio/priority-lists) into account.
---
## Journeys
**URL:** https://hightouch.com/docs/customer-studio/journeys
**Description:** Build automated, multi-step lifecycle flows using visual orchestration in Customer Studio.
**Section:** Customer Studio
| | |
| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers and operations teams who want to automate and personalize campaigns based on user actions, attributes, or timing. |
| **Prerequisites** |
A defined [data schema](https://hightouch.com/docs/customer-studio/schema) with a parent model
At least one [destination](https://hightouch.com/docs/getting-started/concepts#destinations) connected and [syncs](https://hightouch.com/docs/customer-studio/syncs) configured
|
*Journeys help you automate multi-step lifecycle flows such as welcome flows, winback sequences, lifecycle messaging, nurture tracks, and more.*
---
### Learning objectives
After reading this article, you’ll be able to:
- [Create and configure journeys](#create-a-journey)
- [Choose audience-based or event-based entry criteria](#step-2-configure-the-start-tile)
- [Build logic using delays, conditional branches, and experimentation](#step-4-configure-each-tile)
- [Trigger downstream syncs as users progress](#send-to-destination)
- [Configure evaluation schedules, exit rules, and alerting](#journey-settings)
- [Test journeys safely before launch](#test-a-journey)
- [Monitor performance and troubleshoot issues](#journey-performance)
- [Use templates to speed up journey creation](#journey-templates)
---
## Overview
Journeys let you design multi-step customer flows using a visual, tile-based canvas. Each tile represents a step in the lifecycle, such as waiting, branching, checking conditions, or triggering downstream actions.
You can drag tiles onto the canvas, connect them, and configure logic for each step.
---
## How journeys work
Journeys evaluate users on a **fixed schedule**. At each evaluation run, Hightouch:
1. Identifies users who qualify to enter
2. Moves active members forward through tiles
3. Triggers any configured `Send to destination` syncs
4. Applies exit criteria
5. Logs progress for reporting
Syncs inside a journey **ignore their own schedules**. They run **only when the journey evaluates**.
Journeys are made of tiles, each representing a logical step or branching point:
- **[Start](#step-2-configure-the-start-tile)** — defines who enters
- **[Time delay](#time-delay)** — pauses progression
- **[Hold until](#hold-until)** — waits for a condition
- **[Segment](#segment)** — branches based on user attributes or behaviors
- **[A/B split](#ab-split)** — randomly assigns users for testing
- **[Send to destination](#send-to-destination)** — triggers a downstream sync
- **[Context](#set-context-variables)** — sets a journey context variable
---
## Required setup
Journeys rely on the parent model defined in your [data schema](/customer-studio/schema).
Your **parent model must:**
1. Come from a **supported warehouse source**
2. Use the [Lightning sync engine](/syncs/lightning-sync-engine)
Journeys trigger downstream actions through [syncs](/syncs/overview) such as sending messages, updating user profiles, or adding members to advertising audiences. Make sure required [destinations](/syncs/create-your-first-sync#connect-a-destination) and syncs are connected before publishing a journey.
### Recommended sources
- Snowflake
- BigQuery
- Databricks
- Redshift
If you see the message "This source does not support journeys," contact the Hightouch team to confirm whether Journeys are enabled for your source.
Don’t see your source?{" "}
{" "}
to confirm compatibility.
---
## Create a journey
### Step 1: Open the Journeys list
1. Go to **Customer Studio → [Journeys](https://app.hightouch.com/journeys)**.
2. Click **`Add journey`**.
3. In the modal:
- Select a **parent model**.
- Enter a **journey name**.
- Add an optional **description**.
4. Click **`Create journey`**.
You will be taken to the journey canvas in **Draft** mode.
---
### Step 2: Configure the Start tile
The Start tile defines **who enters** the journey and **how often** they can enter.
1. Click the **Start** tile to open its configuration drawer.
2. Choose between two entry types:
- [**Enters audience**](#audience-based-entry-configuration) — users enter when they join an audience
- [**Performs events**](#event-based-entry-configuration) — users enter when they perform an event
---
#### Audience-based entry configuration
1. Select an **entry audience**.
2. (Optional) choose **exclusions**:
- Don’t exclude any members *(default)*
- Exclude members currently in **any** journey
- Exclude members currently in **specific** journeys
3. Choose a **re-entry** behavior:
- **Only one entry** *(default)*
- **Specific number of entries**
- **Unlimited entries**
You can optionally choose set a time delay to prevent journey members from re-entering too soon after their last exit.
4. Click **`Update`** to save your Start tile settings.
---
#### Event-based entry configuration
1. Select the **event** that should trigger entry.
**Only events with assigned primary keys can be used to enter a journey.** A warning appears if the event model lacks a primary key.
2. (Optional) enable **Include past events**:
- When enabled, you can specify a lookback window
- When disabled, only events occurring *after the journey starts* qualify
3. Choose **exclusions**:
- Don’t exclude any members *(default)*
- Exclude members in **any** journey
- Exclude members in **specific** journeys
4. Select a **re-entry** behavior:
- **Only one entry** *(default)*
- **Specific number of entries**
- **Unlimited entries**
You can optionally choose set a time delay to prevent journey members from re-entering too soon after their last exit.
5. Select a **simultaneous entry** behavior.
This setting only applies if you've selected "Specific number of entries" or "Unlimited entries" for re-entry behavior.
Note: If you've also set a re-entry time delay, that delay only applies when the user is not currently in the journey. Simultaneous entries can still happen if the user is already in journey.
Options:
- **Not allowed**
- **Specific number of simultaneous entries**
- **Unlimited simultaneous entries**
6. Click **`Update`** to save.
---
### Step 3: Add tiles to the canvas
Use the **Add to canvas** panel to add steps to your journey:
Available tiles:
- `Send to destination`
- `Time delay`
- `Hold until`
- `Segment`
- `A/B split`
- `Context`
Drag tiles onto the canvas or click the `+` below any tile.
#### Interacting with the canvas
- **Pan** by clicking on empty space and dragging or scrolling on a trackpad
- **Zoom** by using the +/- buttons or pinching on a trackpad
- **Create a tile** by dragging one onto the canvas from the library on the right
- **Move a tile** by clicking and dragging it around the canvas
- **Edit a tile’s configuration** by clicking on the tile
- **Delete a tile** by hovering over it and clicking the trash icon (note: the start tile cannot be deleted)
- **Duplicate a tile** by hovering over it and clicking the copy icon (note: the start tile cannot be duplicated)
- **Clean up** a messy canvas by clicking the clean up icon on the top left of the canvas
- **Select multiple tiles** by holding **Shift** on your keyboard while clicking and dragging around a set of tiles. Once they're selected, you can then move all of these tiles as a group, copy them, or delete them.
- **Expand and collapse tiles** using the compact mode and detail mode buttons
---
### Step 4: Configure each tile
Each tile on the canvas opens a configuration drawer where you define its logic.
---
#### Time delay
Pause each member for a fixed amount of time before advancing.
Members move forward at the **next journey evaluation** *after* the delay elapses.
Delay duration should be **equal to or longer than** the journey evaluation interval. For example, with an hourly evaluation schedule, a 15-minute delay advances on the next hourly run.
---
#### Hold until
Wait until a condition becomes true.
Examples of rule conditions:
- `Order completed` within last 7 days
- `loyalty_tier = Gold`
- `Purchase amount > 50`
Members move forward during the first evaluation where the condition is satisfied.
If the condition never becomes true, the member remains in the tile indefinitely or until they meet exit criteria.
---
#### Segment
Branch journey logic based on conditions.
Branches evaluate top-to-bottom. The first matching branch wins.
Segment tiles support branching using:
- Properties
- Traits
- Audience membership
- Event history
- Relationships
---
#### A/B split
Randomly assign members to branches for controlled experimentation.
Configure:
- Number of branches
- Percentage allocation per branch
- Group names (for example, `Treatment` and `Control`)
For clean experimental results, avoid changing split percentages after the journey goes live.
**Measurement configuration:**
You can attach an experiment to a split node to measure the impact of different branches on your key metrics. In the split node's details drawer, click **`Create experiment`** under **`Measurement configuration`** to get started.
Once created, configure your primary metric, optional secondary metrics, and measurement window. Results are accessible from the split node or under **`Intelligence`** → **`Experiments`**.
[**Learn more about configuring and interpreting experiments →**](/campaign-intelligence/experiments)
---
#### Send to destination
Trigger a sync to a downstream tool.
You can select an existing sync or create a new one from the tile.
Sync schedules are ignored inside journeys. **All syncs trigger only when the journey evaluates.**
---
#### Set context variables
Context variables are values that can be defined, updated, and synced as a user progresses through a journey. They help you track and act on information about how a user has moved through the journey, like if they passed a specific node.
Context variables are defined in [journey settings](https://hightouch.com/docs/customer-studio/journeys#variables), then are set to values within context tiles.
**Example:**
- Context variable `vip_user` is defined in the journey's settings.
- Segmentation tile segments on loyalty levels.
- The context tile that is on the relevant path sets `vip_user` = true. Another context tile on a different branch sets `vip_user` = false.
- The value of `vip_user` on that branch may now be synced to the destination.
- Save the journey to begin using the context variable.
After a journey is run, you can view the context variable's value in the journey member details panel.
---
## Edit a journey
Click **`Edit`** at the top right of the canvas.
---
## Journey settings
Enter **Edit mode** to access settings.
In edit mode, the **`Settings`** panel appears on the right.
Click **`Edit`** to update:
- Journey name and description
- Evaluation schedule
- Exit criteria
- Alerting recipients and triggers
---
### General
In the **`General`** tab, you can edit the journey's name and description.
---
### Evaluation schedule
In the **`Schedule`** tab, you can edit how frequently your journey evaluates.
Journeys run on a fixed cadence that controls entry, movement, and sync triggering.
#### How evaluation works
During each run, Hightouch:
1. Adds new members who qualify for the Start tile
2. Pushes each member forward until a stopping point
3. Fires all `Send to destination` syncs
4. Marks progress on all branches
#### Schedule types
| Type | Description | Examples |
|------|-------------|----------|
| **Interval** | Run on a repeating interval | Every 1 hour, every 30 minutes |
| **Custom recurrence** | Run at specific days/times | Weekdays at 9 AM |
#### Interval
Configure using the `Every` control.
#### Custom recurrence
Choose one or more day/time combinations.
#### First run timing
- **Interval schedules** run a few seconds after you click `Start`.
- **Custom recurrence** runs at the next matching time.
---
### Exit criteria
In the **`Exit criteria`** tab, you can remove members early based on conditions.
Examples:
- “Exit if user makes a purchase”
- “Exit if user unsubscribes”
- “Exit if user enters audience X”
Members leave the journey immediately once conditions match during evaluation.
---
### Variables
In the **variables** tab, you can define context variables to set within [context tiles](https://hightouch.com/docs/customer-studio/journeys#context-tile) throughout the journey.
---
### Alerting
In the **`Alerting`** tab, you can edit alerting rules.
Alerting notifies you when a journey run fails so you can investigate issues quickly. Journeys inherit default alert settings from your workspace, and you can override those settings at the journey level.
#### Recipients
Recipients determine where alerts are sent. Alerts can be delivered by email, Slack, PagerDuty, SMS, webhook, or any channel your workspace has configured.
To add a recipient, click **Add new recipient** to open the recipient modal.
When creating an email recipient, you can send a test message using **Send a test alert now** to confirm delivery before saving.
#### Triggers
Under **Triggers**, you can specify when alerts should fire.
Currently, journeys support alerts for:
- **Fatal errors** — Runs that fail during execution
If no trigger is configured, alerts will only appear in the Hightouch UI and will not be sent externally.
#### Global defaults (Integrations → Alerting)
Journey alerting inherits defaults from your workspace’s global alerting configuration, available under:
**Integrations → Alerting → Journeys**
From this page, you can:
- Configure default recipients for all journeys
- Set global triggers for fatal errors
- Manage shared alert channels (Slack, PagerDuty, email, etc.)
You can also manage your workspace’s full list of recipients under the **Recipients** tab and view destination-specific alerting under the **Syncs** tab.
#### Notification behavior
You receive **one alert per status change**, not per run. For example:
- A run fails → **one warning alert**
- The next run fails → no additional alert
- The next run succeeds → **one recovery alert**
- The next run fails → **one new warning alert**
To learn more about alerting across Hightouch, see
[**Sync alerting →**](/syncs/alerting#overview).
---
## Test a journey
Use **Run test** to simulate how members move through your journey before you publish it.
Tests help you verify logic, branching, and timing without sending data to any destinations.
All time delays are skipped during simulation.
---
### Step 1: Open the test modal
1. Open your journey.
2. Click **`Run test`** in the upper-right corner.
3. The **Configure a journey test** modal appears.
Test simulations do not trigger syncs or update downstream systems. They only show how members would progress based on your configuration.
---
### Step 2: Choose a start date and time
Set the **Start date & time** to define when the test begins.
- Choose **Now** to simulate new entries.
- Choose a **past date/time** to test event-based journeys using real historical events.
Events with timestamps **after** the selected start time are evaluated in the test.
Use a past start time when testing "Performs events" journeys to validate real-world event flow.
---
### Step 3: Select a test size
Choose how many members or events to include:
**Single event**
Simulates the path for one member.
**Set of events** *(default)*
Simulates all qualifying events and members since the chosen start time.
Useful for validating branch percentages and logic at scale.
---
### Step 4: Run the test
Click **`Run test`** to start the simulation.
The test displays:
- When members enter via the Start tile
- How they move through each tile
- Which branches they follow
- How many members reach each step
- Tiles that do not trigger due to unmet conditions
Time delays are skipped so members proceed immediately.
Running the test does not affect journey membership, logs, syncs, or evaluation cadence. It is a safe pre-launch validation step.
---
### (Optional) Step 5: Run a test sync
For audience-based Journeys, you can test how data is sent from a Journey’s "Send to destination tiles" before launch or while the Journey is live.
This allows you to validate that the correct users and data are synced without modifying your Journey logic.
#### Triggering a test sync
To trigger a test sync, you can:
- **Run a journey simulation:** Use members from the simulated journey path as the basis for the test sync. This ensures the selected users reach the sync tile.
- **Trigger from a sync tile:** Trigger a test from a specific sync tile by selecting members from the entry audience. The selected members are not guaranteed to reach the sync tile in a real journey run.
- **[Live] Use existing journey users:** Select users who have passed through the node in a live Journey.
#### Test data
Test syncs use real journey members as input. You can override key values (such as email) to ensure test data does not reach real recipients.
You can input data in one of the following ways:
- **Upload test users:** Upload a CSV of test data values. Use the template to identify required fields. Any fields left blank will use the original values from the selected journey member.
- **Select existing rows:** Search and select relevent journey members, or choose a random member.
#### Running a test
Once test data is configured, manually trigger the test run to send the selected users through the Journey.
#### Notes
- Some fields may appear redacted in the UI. If not edited, `` will be used when syncing.
- Test syncs do not modify your Journey configuration or underlying data. For live Journeys, test data does not enter or affect the live Journey state.
- Be sure to override recipient fields (such as email) to avoid sending to real recipients.
---
## Publish, pause, and resume a journey
### Publish
Click **`Start`** to activate the journey.
### Pause
**`Pause`** stops evaluations and sync triggering.
### Resume
Restart the journey without losing member progress.
---
### Turn off a journey
Click **`Turn off`** to stop all future evaluations for a journey.
You must choose how to handle current members:
- **Existing members should complete the journey first** (default)
All members currently in progress will complete the journey and no new entries will occur.
When you select this option, the journey enters **Draining** state.
- **Hard stop**
All members are removed on the next evaluation run and no new entries will occur.
#### Draining status
When a journey is turned off using **Existing members should complete the journey first**, it enters **Draining** state.
While draining, the journey:
- **Does not accept new entries**
- **Continues evaluating and advancing existing members**
- **Runs all configured syncs** for those members during each evaluation
- **Displays a Draining badge** in the Journeys list
- **Automatically transitions to `Off`** once no members remain in the flow
Draining is the safest option when retiring or replacing a journey because it prevents disruption to active messaging or downstream actions.
---
## Journey templates
Journey templates let you save and reuse common journey structures so teams can create consistent lifecycle flows without rebuilding them from scratch.
To learn how to create, manage, and apply templates, see [**Journey templates →**](/customer-studio/templates#journey-templates).
---
## Journey performance
Once your journey is running, the canvas updates to show how members move through each step. Line thickness reflects the volume of members flowing through a path. Hover over any line to see counts for that branch.
The **Performance** panel provides high-level metrics across the full history of the journey (unless you reset it):
- **In progress** — Members currently waiting in a tile
- **Entered** — Total members who have ever entered the journey
- **Met exit criteria** — Members removed due to global exit rules
- **Finished** — Members who reached the final step of the flow
#### Tile-level performance
Select any tile on the canvas to view metrics specific to that step:
- **Currently in tile** — Members waiting at this tile (commonly nonzero for Time delay and Hold until)
- **Entered tile** — Members who have ever reached this tile
- **Met exit criteria** — Members removed from this tile due to global exit rules
- **Advanced from tile** — Members who moved to the next step
For Segment, A/B split, and Hold until tiles, you can also view per-branch counts and percentages.
#### Member-level details
Select a count in the Performance panel to open the member drawer. This shows a sample of 100 random members who meet the criteria, with the option to search by primary key.
Select a member to view their complete attributes and a step-by-step visualization of their movement through the journey.
---
## Journey Logs
The Journeys system creates tables in your data warehouse to enable journey observation, reporting, and debugging.
Each journey has three individual tables created in the `HIGHTOUCH_PLANNER` schema, where `journey_id` in the naming scheme is the UUID of the journey with the dashes removed.
- `JOURNEY_LOG_`: this table logs the progress of all rows through a given journey.
This table is actually used as part of Journey execution, so while it’s safe
to read from it to do analysis, ***please do not insert, update, or delete
to/from it***, since this might cause the Journey to behave in unexpected
ways.
- `JOURNEY_METADATA_`: this table can be used to rebuild the journey graph and to access your node names for easier reporting.
- `JOURNEY_CONTEXT_LOG_`: this table is used to store the result of any journey [variables](https://hightouch.com/docs/customer-studio/journeys#variables) you sync.
Additionally, two global views are available in `hightouch_audit` that contain a comprehensive view of all of a workspace’s journeys in the warehouse, giving customers a single read-only point to monitor their journeys. These tables have the same schema as their underlying tables in addition to a reference to the source table that the row came from. We recommend that you use these global views rather than the individual `journey_log` and `journey_metadata` tables for your warehouse reporting.
- `JOURNEY_LOG_VIEW_`
- `JOURNEY_METADATA_VIEW_`
Locate your workspace slug under **Settings > Workspace > Workspace slug**.
All dashes in the slug should be replaced with an underscore. For example, if
your slug is `my-workspace`, use `my_workspace` in your query.
### Journey Log Table Schema
The `JOURNEY_LOG_` logs the progress of all rows through a given journey.
Each time a row enters the journey, moves from one node to another, or exits the journey, we create an entry in the log table. Note that since a row can enter a journey more than once, we also have a `ROW_INSTANCE_ID` column on the table, which is used to uniquely identify each time a row enters the journey. That instance ID will be consistent for that particular instance of the row as it moves through the journey.
Note that resetting a journey will clear the log table.
| Column | Type | Description |
| --------------- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| row_id | string | The primary key from the journey’s parent model that this row represents. |
| row_instance_id | string | A UUID to uniquely identify a row_id each time it enters the journey. |
| run_id | string | The ID of the journey run that executed this operation. This is an internal detail. |
| from_node_id | string | The ID of the node that this action originates with. For moves or exits, it’s the node that the row moved or exited from. For enters, it’s NULL. |
| to_node_id | string | The ID of the node that this action targets. For moves and enters, it’s the node the row moved into. For exits, it’s NULL. |
| timestamp | timestamp | The effective timestamp of this operation. Note that this does not always represent the actual time the operation occurred - for example, if a user moved from one node to another due to an event, this will be the timestamp of the event rather than the timestamp we actually ran the warehouse query. |
| event_type | string | What type of event this log line represents. One of `moved-to-node`, `entered-journey`, `exited-journey-by-criteria`, or `exited-journey`. |
### Journey Metadata Table Schema
The `JOURNEY_METADATA_` contains the positions of the nodes in a journey, and the customer-defined names of nodes.
Each time a change is saved to the Journey this table is updated.
| Column | Type | Description |
| ------------ | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| journey_id | text | The id of the journey. |
| journey_name | text | The customer-defined name of the journey. |
| node_name | text | The customer-defined name of the node. |
| node_type | text | The node type (entry-cohort, entry-event, sync, segments, segment-branch, time-delay, splits, split-branch, wait-until-event, wait-until-event-branch) |
| node_id | text | The ID of the journey node. |
| to_nodes | text[] | An array of the outbound node ids from the given node.
### Journey Context Table Schema
The `JOURNEY_CONTEXT_LOG_` logs the result of journey context [variables](https://hightouch.com/docs/customer-studio/journeys#variables) through a given journey.
| Column | Type | Description |
| --------------- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| row_id | string | The primary key from the journey’s parent model that this row represents. |
| row_instance_id | string | A UUID to uniquely identify a row_id each time it enters the journey. |
| run_id | string | The ID of the journey run that executed this operation. This is an internal detail. |
| node_id | string | The ID of the journey node. |
| entered_at | timestamp | When the row entered the "set a variable" node. |
| inserted_at | timestamp | Used for prioritizing when multiple context nodes assign the same variable. The most recent one is used. |
| [variable_name] | dynamic | There is one dynamic column per context variable. The type depends on the type of the context variable. |
### Global Journey Log View Schema
The `JOURNEY_LOG_VIEW_` view shows the progress of _all_ rows in _all_ journeys.
Each time the journey is added or removed from your workspace, this view is updated.
| Column | Type | Description |
| --------------- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| source_table | string | The name of the table this row came from: `journey_log_`. |
| row_id | string | The primary key from the journey’s parent model that this row represents. |
| row_instance_id | string | A UUID to uniquely identify a row_id each time it enters the journey. |
| run_id | string | The ID of the journey run that executed this operation. This is an internal detail. |
| from_node_id | string | The ID of the node that this action originates with. For moves or exits, it’s the node that the row moved or exited from. For enters, it’s NULL. |
| to_node_id | string | The ID of the node that this action targets. For moves and enters, it’s the node the row moved into. For exits, it’s NULL. |
| timestamp | timestamp | The effective timestamp of this operation. Note that this does not always represent the actual time the operation occurred - for example, if a user moved from one node to another due to an event, this will be the timestamp of the event rather than the timestamp we actually ran the warehouse query. |
| event_type | string | What type of event this log line represents. One of `moved-to-node`, `entered-journey`, `exited-journey-by-criteria`, or `exited-journey`. |
| journey_id | text | The id of the journey. |
### Global Journey Metadata View Schema
The `JOURNEY_METADATA_VIEW_` view contains the positions of the nodes across all journeys in the workspace and the customer-defined names of those nodes.
Each time the journey runs, this view is updated.
| Column | Type | Description |
| ------------ | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| source_table | string | The name of the table this row came from: `journey_log_`. |
| journey_id | text | The id of the journey. |
| journey_name | text | The customer-defined name of the journey. |
| node_name | text | The customer-defined name of the node. |
| node_type | text | The node type (entry-cohort, entry-event, sync, segments, segment-branch, time-delay, splits, split-branch, wait-until-event, wait-until-event-branch) |
| node_id | text | The ID of the journey node. |
| to_nodes | text[] | An array of the outbound node ids from the given node.
## Journey notes
Use notes to leave context about the goals of your journey, explain specific tile behavior, or call out intentional edge cases.
Notes help you and your teammates understand complex journeys faster, especially when collaborating or reviewing someone else’s work.
You can add notes in a few ways:
1. Drag notes onto tiles or anywhere on the canvas.
2. Add notes directly on the tile itself.
3. Open the tile drawer to add or edit a tile-specific note
Notes can also be added to journey templates. When a journey is created from a template, any notes are carried over, and the journey creator will appear as the author of those notes.
---
## Best practices
### Priority lists and entry audiences
If your entry audience is inside of a priority list, the priority will be respected when determining who should enter the journey. For example, if your entry audience, Audience B, is prioritized underneath Audience A, and user 1 meets the criteria for both audiences, the user will only fall into Audience A. This means they will not enter the journey until they drop out of Audience A (while remaining in Audience B).
---
### Sync execution inside journeys
#### Failed syncs and retries
If a record fails to get synced inside of a journey, Hightouch will attempt to retry that record upon subsequent runs.
- For cohort syncs, Hightouch will stop retrying when the record is removed from the sync
- For trigger syncs, Hightouch will continue to retry the record until it’s synced successfully
#### Sync sequencing
All `Send to destination` tiles trigger at the **same time**: the end of each journey evaluation.
If you need strict sequencing between two syncs, include a **Time delay** tile between them.
---
### Choosing sync configurations for journeys
Sync mode affects how users are added, updated, or removed from downstream tools. Use these guidelines to select the right configuration.
#### Trigger mode syncs
Best for triggered messages (email, SMS) where you want one event per user per step.
- Works best with **Insert** mode to ensure that members don't receive the same message multiple times.
- Users are never removed from the destination.
- Avoid **Upsert**, **Update**, **Mirror**, and **Diff** to prevent duplicate sends.
| Sync Mode | Trigger Mode behavior |
|----------|------------------------|
| **Insert** |
**Recommended**
Members who enter or re-enter a tile during a given journey run are synced to the destination
Changes to members who have previously synced to the destination are ignored
|
| Upsert |
Not Recommended: syncs are triggered for new and updated members, which can lead to duplicate triggered sends
Members who enter the tile will be synced to the destination
Changes to any member who has ever synced to the destination are synced to the destination
|
| Update |
Not Recommended: syncs are triggered for updated members, which can lead to duplicate triggered sends
Changes to any member who has ever synced to the destination are synced to the destination
New members entering the node will only sync to the destination if the member already exists in the destination
|
| Mirror |
Not Recommended
All members who have ever passed through the send to destination tile are synced every time the journey runs
|
| Diff |
Not Recommended
One file is sent for each operation. Any member who entered the node, has changed, or removed is synced to the destination.
|
#### Cohort mode syncs
Best for audiences, lists, or situations where a downstream destination should maintain a stable membership list.
- Works best with **Upsert** and **Mirror**.
- Removal behavior is determined by journey configuration.
- Avoid `deleteMode: do nothing` with cohort syncs, since removals may not be applied.
| Sync Mode | Cohort Mode Journeys | Cohort Removal Behavior |
|-----------|-----------------------|--------------------------|
| **Upsert** |
**Recommended**
New members entering the tile during a given journey run will sync to the destination
Changes to members already in the destination will sync
|
Members will be removed from the destination based on the journey configuration
|
| Update |
Not Recommended
Changes to members already in the destination will sync
New members entering the node will only sync to the destination if the member already exists in the destination
|
Members will be removed from the destination based on the journey configuration
Usually deleteMode for update syncs is set to clear, meaning the member remains in the destination but with all attributes cleared.
|
| Insert |
Not Recommended because removal configurations aren't respected
New members entering the node are synced to the destination
Members who are changed or removed are ignored
|
No removal will occur, regardless of journey settings
|
| **Mirror** |
**Recommended** when the members in the destination should exactly match the members in the journey
No diff is performed. All members who have ever passed through the send to destination tile and not met removal criteria are synced to the destination
|
Members will be removed from the destination based on the journey configuration
|
| Diff |
Not Recommended
One file is sent for each operation. Any member who entered the node, has changed, or removed is synced to the destination.
|
No removal will occur, regardless of journey settings
|
---
## Troubleshooting
If your journey run fails because of warehouse SQL issues (for example, invalid identifiers, unresolved columns, or missing objects), see [Resolve SQL compilation errors](/models/sql-compilation-errors).
### Users are not entering the journey
Common causes:
- Start tile entry type doesn't match how you're validating (audience vs event)
- Event entry is missing `Include past events` or has a lookback window that's too short
- Evaluation schedule is too infrequent for the behavior you expect
- Users are entering and then immediately meeting global exit criteria
What to check:
1. Confirm Start tile configuration (entry source, filters, exclusions, and re-entry settings).
2. For event entry, verify event model, event name, and lookback window.
3. Check the journey's evaluation schedule and last run time.
4. Validate source data freshness in your warehouse.
### Delays or Hold until behavior looks inconsistent
Journeys evaluate on a schedule, so delays and hold conditions are only checked on each run.
Common causes:
- Delay values are shorter than the schedule interval
- Multiple delay/hold tiles are chained, causing timing to "snap" to run cadence
Recommended fixes:
1. Align delay duration with schedule granularity (for example, hourly schedule with hour-level delays).
2. Increase journey frequency if you need tighter timing.
3. Avoid sub-interval delays (for example, 30-minute delay on a daily schedule).
### Users are not exiting when expected
Global exit criteria are evaluated on journey runs, not in real time.
Common causes:
- Exit action occurred after the last run and hasn't been evaluated yet
- Exit condition is configured with the wrong field or logic
- Data pipeline lag delays the exit signal in your warehouse
What to check:
1. Compare action timestamp (purchase/opt-out) to journey run history.
2. Review exit criteria definition in journey settings.
3. Confirm the exit-driving fields are updated before the next run.
### Users are re-entering unexpectedly
Common causes:
- Start tile re-entry is set to allow multiple entries
- Simultaneous entry settings allow overlapping instances
- Event-based entry conditions are repeatedly met
What to check:
1. Verify Start tile re-entry and simultaneous entry settings.
2. Confirm whether repeated qualifying events are expected.
3. If needed, add or increase re-entry delay.
### Tile counts don't match destination counts
Common causes:
- Destination applies additional filtering (consent, suppression, deduplication)
- Sync tile uses cohort mode with removal behavior that limits downstream membership
- Tile warning appears: `Users might never be synced`
If you see `Users might never be synced`, check whether the sync tile is:
- In cohort mode
- Configured to remove on exit
- The final tile in a branch
Recommended fixes:
1. Add a Time delay tile after the sync tile so users remain in the sync node for at least one run.
2. Adjust removal behavior (for example, remove after N days instead of immediate removal).
3. Review destination rules, subsets, and priority lists that may narrow final synced records.
### Simulation results differ from live runs
Simulation is best for logic validation, not end-to-end delivery behavior.
Why this happens:
- Simulation skips delays
- Simulation does not trigger destination syncs
- Simulation does not fully model cross-journey interactions or downstream platform rules
Recommended workflow:
1. Use simulation to validate branching and conditions.
2. Use a small live test cohort to verify real downstream behavior.
3. Verify production data freshness when comparing simulation to live outcomes.
If a journey behaves unexpectedly, first confirm Start tile logic, evaluation schedule, and exit criteria timing. Most troubleshooting issues come from schedule cadence and data freshness.
---
## What’s next?
Explore related Customer Studio features:
- [Create audiences →](/customer-studio/audiences)
- [Sync to destinations →](/customer-studio/syncs)
- [Use Experiments for A/B testing →](/customer-studio/experiments)
- [Add reusable traits →](/customer-studio/traits)
- [Analyze audiences in Insights →](/customer-studio/insights)
---
## OneTrust Snowflake Native App
**URL:** https://hightouch.com/docs/customer-studio/onetrust
**Description:** This guide explains how to use consent data collected and stored in OneTrust, and passed to Hightouch via the OneTrust Snowflake Native App. Once the app is configured, Hightouch can reference this data in subsets, destination rules, and sync behavior to enforce consent across downstream tools.
**Section:** Customer Studio
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Platform admins |
| **Prerequisites** | [Set up for platform admins and data teams →](https://hightouch.com/docs/customer-studio/quick-start#setup-for-platform-admins-and-data-teams) |
*This guide explains how to use consent data collected and stored in OneTrust, and passed to Hightouch via the OneTrust Snowflake Native App. Once the app is configured, Hightouch can reference this data in subsets, destination rules, and sync behavior to enforce consent across downstream tools.*
***
## Learning Objectives
After reading this article, you will be able to:
- Understand how the OneTrust Snowflake Native App makes consent data available in Hightouch.
- Connect OneTrust data stored in Snowflake to your Customer Studio schema.
- Filter audiences by consent status using Subsets.
- Enforce consent compliance across destinations with Destination Rules.
- Configure syncs to remove unconsented records automatically.
- Model various consent and preference types to support advanced use cases.
***
## Overview
If you’re using OneTrust [Universal Consent and Preference Management](https://www.onetrust.com/products/universal-consent-and-preference-management/) (UCPM) to manage customer consent and communication preferences, you can use the [OneTrust Snowflake Native App](https://www.onetrust.com/news/onetrust-accelerates-access-to-trusted-customer-data-with-snowflake-native-app-for-consent-management/) to automatically sync consent and preference data into your Snowflake warehouse. This keeps your data warehouse up-to-date in near real-time with the latest opt-ins, opt-outs, and preference changes. Snowflake’s data governance controls are used to mask rows and filter columns to ensure that specific roles can only access the data they are permissioned for.
Once centralized, consent becomes available in Hightouch, where it can be used to:
* Filter audiences with [Subsets](https://hightouch.com/docs/customer-studio/subsets)
* Block unconsented records with [Destination Rules](https://hightouch.com/docs/customer-studio/destination-rules)
* Remove records from downstream tools (Destinations) when consent is revoked
This workflow ensures that downstream tools only receive and act on data from users who have given the appropriate consent, automating compliance and improving campaign targeting.
## Setup
To start using OneTrust consent data in Hightouch:
1. [**Install and configure the OneTrust Snowflake Native App**](https://my.onetrust.com/s/article/UUID-2f18f280-ec76-c4c5-2419-73615c2ad4dd?language=en_US&topicId=0TO1Q000000ItSRWA0) in your Snowflake account.
* This streams real-time consent and preference updates from OneTrust into a Snowflake table in your warehouse.
* You can read OneTrusts complete documentation Universal Consent and Preferences Management [here](https://my.onetrust.com/s/article/UUID-9f6e9cab-3016-5dd0-342f-7f03287c358d?language=en_US)
2. [**Set up Snowflake as a Source**](https://hightouch.com/docs/sources/snowflake) **in Hightouch**
* Follow Snowflake source setup instructions to connect your warehouse and make the OneTrust data accessible to Hightouch.
3. [**Create a Parent Model**](https://hightouch.com/docs/customer-studio/schema#1-define-the-parent-model) **in Customer Studio Schema**
* Go to **Customer Studio → Schema**
* Click **Create parent model** and select your primary user table (e.g., `users`)
4. [**Add your consent table as a related model**](https://hightouch.com/docs/customer-studio/schema#2-add-related-models)
* Depending on your OneTrust setup, you may choose to perform some transformations on this table or create a new view before exposing it to Hightouch.
* Click on the **+** button next to the parent model
* Select the consent table (e.g., `DATASUBJECT_CONSENT_VIEW`)
* Define the join condition between your parent model and this consent table
* Choose the appropriate relationship type
* If you only have one “consent purpose” in OneTrust, you can use a 1:1 relationship
* If you have different “consent purposes,” you should choose 1:Many, as OneTrust stores different consent types in different rows
5. [**Select relevant consent columns**](https://hightouch.com/docs/customer-studio/schema#2-add-related-models)
* Once the tables are joined, select the consent fields you want to expose in your schema—such as `email_opt_in`, `sms_opt_in`, or `ad_sharing_opt_in`.
These fields will then be available throughout Hightouch to:
* Filter audiences using Subsets
* Enforce rules with Destination Rules
* Configure sync behavior based on consent status
## Enforcing consent in Hightouch
### 1. Filter audience segments by consent with Subsets
Use [Subsets](https://hightouch.com/docs/customer-studio/subsets) to define reusable filters based on consent preferences. You can apply subsets across audiences, journeys, and syncs to ensure only users who meet consent criteria are included.
**Example use case:**
Create a subset that includes only records where email\_opt\_in \= true. You can use the default `Email` subset in pre-initalized `Consent management` category.
Once created, this subset can be:
* **Applied to multiple audiences** to enforce consistent filtering across marketing use cases.
* **Set as mandatory** for specific user groups (e.g., your email marketing team), so they cannot build audiences without referencing the required consent field.
* Restricted to certain values, ensuring end users can only select approved options (e.g. forcing the selection of “Opted In” from a consent dropdown).
This helps prevent accidental inclusion of unconsented users in downstream tools.
### 2. Suppress unconsented records with Destination Rules
Use [Destination Rules](https://hightouch.com/docs/customer-studio/destination-rules) to define conditions under which Hightouch will block records from syncing to specific destinations.
**Example rule:**
Block all records where `sms_opt_in` \!= `true` from syncing to your SMS tool.
This acts as a fallback safeguard to prevent unintentional delivery of unconsented data.
### 3. Remove data when consent is revoked
When a user revokes consent, Hightouch can remove their data from downstream tools—but only if your model is filtered by consent **and** your sync is configured to support deletions.
#### Removal options
**1. Recurring syncs**
On the next scheduled run, Hightouch will remove any records that no longer match your model’s filter.
**Requirements:**
* Your model or audience includes a consent-based filter (e.g., `email_opt_in` = `TRUE`).
* Use a [sync mode](https://hightouch.com/docs/syncs/types-and-modes#sync-modes) that supports removals
* Enable record removal in the sync configuration
**2. Triggered deletions (real-time)**
Hightouch will automatically configure a sync as Triggered if your model and destination support real-time updates (e.g., using a CDC or event stream).
If your sync is a Triggered Sync:
* You’ll see the option:
“**Delete records when they no longer match the model**”
* Enable this to remove records from the destination as soon as a consent change is detected (e.g., `email_opt_in` becomes `FALSE`)
## Modeling consent and preference data
The OneTrust Snowflake Native App can sync a wide range of consent and preference fields into your warehouse—not just binary opt-ins. Modeling these fields effectively allows you to enforce consent logic across multiple tools using Hightouch.
When defining your [schema](https://hightouch.com/docs/customer-studio/define-data-schema), consider including:
* **Channel-specific consent fields**
**Example:** `email_opt_in`, `sms_opt_in`, `push_opt_in`
These allow you to enforce channel-level rules in Destination Rules or Subsets.
* **Communication frequency preferences**
**Example:** `email_frequency`= `weekly`
You can use these to suppress over-messaging by filtering for lower-frequency segments.
* **Content or category preferences**
**Example:** `category_preference`= `mens_clothing`
Supports targeting users based on stated interests.
* **Consent for data sharing**
**Example:** `ad_sharing_platform_opt_in`= `true`
Ensures records are only sent to third-party platforms (e.g., Meta, Google) when appropriate consent exists.
## Related documentation
**Use OneTrust data in Hightouch:**
* [Subsets](https://hightouch.com/docs/customer-studio/subsets)
* [Destination Rules](https://hightouch.com/docs/customer-studio/destination-rules)
**Modeling consent data:**
* [Define Data Schema](https://hightouch.com/docs/customer-studio/define-data-schema)
**Event-based consent management:**
* [Consent Manager for Events](https://hightouch.com/docs/events/consent/consent-manager)
* [OneTrust for Events](https://hightouch.com/docs/events/consent/onetrust)
**OneTrust documentation:**
* [Universal Consent and Preferences Management](https://my.onetrust.com/s/article/UUID-9f6e9cab-3016-5dd0-342f-7f03287c358d?language=en\_US)
---
## Customer Studio overview
**URL:** https://hightouch.com/docs/customer-studio/overview
**Description:** Hightouch's Customer Studio is a purpose-built no-code suite of features that enables anyone, regardless of their SQL experience, to activate data from directly from their warehouse.
**Section:** Customer Studio
| | |
| ------------ | -------------------------------------------------------- |
| **Audience** | Marketers, Platform admins, Data and analytics engineers |
_Customer Studio is a no-code audience builder where you define, activate, and manage audiences using data from your warehouse. It provides a visual interface for building audience logic and syncing users to tools like CRMs, ESPs, and ad platforms._
Customer Studio is available as an add-on to [Business tier
plans](https://hightouch.com/pricing).
---
### Learning objectives
After reading this article, you’ll understand:
- What Customer Studio is and how it fits into your workflow
- Who’s responsible for setup and day-to-day use
- Key features available in each stage of the workflow
- How to access Customer Studio in the Hightouch app
---
## Overview
**Customer Studio** is a **no-code audience builder** for marketers. It allows you to define and manage customer audiences using your organization’s data, without writing SQL or relying on engineering support.
You can apply **filters**, **traits**, and [**real-time events**](/real-time/real-time) to determine which users belong in an audience, then **sync that audience to tools** like ad platforms, CRMs, or email service providers (ESPs).
Because Customer Studio **connects directly to your data warehouse**, audiences remain current without manual list exports or updates. Data is not stored or copied unless explicitly configured.
[**Explore the interactive architecture diagram →**](https://architecture.hightouch.com/?from=docs&product=customer-studio&utm_source=hightouch-docs&utm_medium=referral&utm_campaign=architecture-explorer&utm_content=customer-studio)
See how data flows and where Customer Studio fits in your stack.
## Who uses it
| Role | Responsibilities | Learn more |
| -------------------------------- | ------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| **Platform admins / Data teams** | Set up sources and configure schema models, traits, and delivery controls | [Setup instructions →](https://hightouch.com/docs/customer-studio/quick-start#setup-for-platform-admins-and-data-teams) |
| **Marketers** | Build, sync, and manage audiences for campaigns | [How to use →](https://hightouch.com/docs/customer-studio/quick-start#how-marketers-use-customer-studio) |
## Example: Target a high-value loyalty segment
Let’s say you want to create a campaign for users who meet any of the following criteria:
- Placed an order in the last 2 weeks
- Are part of your loyalty program
- Have a lifetime value over $1,000
You can define this logic using filters in the audience builder, combine them with AND/OR conditions, and preview the matching users before activating the audience.
Once saved, you can sync this audience to multiple tools—such as Klaviyo for email or Meta Ads for paid media—without rebuilding the segment in each platform.
## Navigate to Customer Studio
To access Customer Studio in the Hightouch app:
1. Go to [**app.hightouch.com**](https://app.hightouch.com)
2. In the left sidebar, click [**Customer Studio**](https://app.hightouch.com/audiences)
## Core workflow
The Customer Studio workflow is centered around three core steps: setting up your data schema, building audiences, and syncing them to destinations. These steps form the foundation for targeting and activation.
### 1. Set up your schema
[**Data teams**](https://hightouch.com/docs/customer-studio/quick-start#setup-for-platform-admins-and-data-teams) connect relevant tables—such as `users`, `purchases`, `product views`, or `support tickets`—and define how these tables relate to each other. This creates the foundation that marketers use to filter, segment, and personalize audiences.
[Define data schema →](https://hightouch.com/docs/customer-studio/schema)
Before defining your schema, make sure at least one data
[source](https://hightouch.com/docs/getting-started/concepts#sources) is
connected to your Hightouch workspace.
### 2. Build your audience
[**Marketers**](https://hightouch.com/docs/customer-studio/quick-start#how-marketers-use-customer-studio) use the **visual audience builder** to define which users should be included in a campaign. You can combine filters, traits, and event conditions—then preview audience size, inspect matching users, and fine-tune logic before syncing.
You can also use the [**AI Agent**](/customer-studio/agents) to build, analyze, and refine audiences using natural language. Describe the segment you want in plain English, and the Agent translates your intent into audience logic automatically.
[Build audiences →](https://hightouch.com/docs/customer-studio/usage)
### 3. Sync to your tools
Once saved, **marketers** can sync audiences to destinations like CRMs, ad platforms, or email service providers. Choose a sync method that fits your use case:
- **Scheduled syncs** for batch delivery (e.g., daily Meta Custom Audiences)
- **Real-time syncs** for instant personalization (e.g., on-site or in-app experiences)
[Sync audiences →](https://hightouch.com/docs/customer-studio/syncs)
To sync your audience, at least one
[destination](https://hightouch.com/docs/getting-started/concepts#destinations)
must be connected to your workspace.
## Advanced personalization and management
After building and syncing your first audiences, you can layer on tools to support personalization, automation, testing, and compliance as needed.
### [**AI Agent →**](https://hightouch.com/docs/customer-studio/agents)
An AI-powered Agent built into the audience builder that lets you create, analyze, and refine audiences using natural language. Describe the segment you want, ask questions about audience composition, or iterate on definitions through conversation.
### [**Traits →**](https://hightouch.com/docs/customer-studio/traits)
Custom calculated fields created by you and your data team. Traits allow you to use advanced logic—like scores, tiers, or rollups—in audience filters without writing code.
### [**Insights →**](https://hightouch.com/docs/customer-studio/insights)
A side panel that helps you analyze audience composition. Compare overlap with other audiences or break down members by attributes like geography or behavior.
### [**Sampling →**](https://hightouch.com/docs/customer-studio/sampling)
A debugging and performance feature that limits audience previews to a small sample. Useful when working with large datasets or testing complex filters.
### [**Templates →**](https://hightouch.com/docs/customer-studio/templates)
Reusable audience logic that helps standardize targeting criteria across teams. Use templates to ensure consistency and speed up campaign setup.
### [**Snapshots →**](https://hightouch.com/docs/customer-studio/audience-snapshots)
A point-in-time export of an audience’s membership. Snapshots preserve historical lists for compliance, measurement, or backfills—independent of sync timing.
#### [**Journeys →**](https://hightouch.com/docs/customer-studio/journeys)
A visual builder for multi-step campaign logic. Use journeys to trigger actions based on real-time user behavior, apply branching rules, and automate follow-ups.
### [**Priority lists →**](https://hightouch.com/docs/customer-studio/priority-lists)
Tools for conflict resolution. Priority lists rank audiences so you can control which segment a user belongs to when they qualify for more than one.
### [**Splits →**](https://hightouch.com/docs/customer-studio/splits)
Divide an audience into randomized groups to run A/B or multivariate tests. Useful for comparing different campaign treatments or creatives.
### [**Destination rules →**](https://hightouch.com/docs/customer-studio/destination-rules)
Governance settings that apply conditions to when or how a destination receives data—for example, to prevent syncing users missing consent fields.
### [**Subsets →**](https://hightouch.com/docs/customer-studio/subsets)
Reusable filters that scope which records can be synced to certain destinations. Useful for limiting syncs by region, subscription status, or compliance flags.
### [**OneTrust Native Snowflake App →**](https://hightouch.com/docs/customer-studio/onetrust)
Connects Hightouch to your OneTrust consent management platform to automatically exclude opted-out users from audience syncs.
## Supported sources
Only the following sources support Customer Studio:
- Snowflake
- Databricks
- Google BigQuery
- Amazon Redshift\*
- Amazon Athena
- Azure Synapse
- MS SQL Server
- PostgreSQL
- Greenplum
- Microsoft Fabric
_\* Customer Studio may not work with [Amazon Redshift Spectrum](https://docs.aws.amazon.com/redshift/latest/dg/c-getting-started-using-spectrum.html) due to its [limitations around nested data](https://docs.aws.amazon.com/redshift/latest/dg/nested-data-restrictions.html). To mitigate these, you can try to query a [materialized view](https://docs.aws.amazon.com/redshift/latest/dg/materialized-view-overview.html) during your [parent model setup](/customer-studio/schema#1-define-the-parent-model)._
## What’s next?
- [Get started with the quick start guide →](https://hightouch.com/docs/customer-studio/quick-start)
- [Define your schema →](https://hightouch.com/docs/customer-studio/schema)
---
## Predictive traits
**URL:** https://hightouch.com/docs/customer-studio/predictive-traits
**Description:** Predictive traits use machine learning to forecast user behavior, giving you forward-looking signals like purchase propensity or churn risk.
**Section:** Customer Studio
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Marketers and analytics/data engineers |
| **Prerequisites** |
A defined schema with at least one parent model (for example, `Users`).
Event data that reflects the outcome you want to predict (for example, purchases, cart completions).
(Optional) Connected destinations for syncing audiences.
|
## Overview
When defining audiences, you sometimes want to act on signals about what customers are likely to do next, not just what they have already done.
**Predictive traits** extend Customer Studio traits with AI-powered scores that estimate the likelihood of a user performing a future action.
For example:
- **Purchase propensity (any purchase):** probability that a user will make a purchase within the next 7 days
- **Cart completion propensity:** probability that a user who started checkout will complete their purchase
Predictive traits are refreshed automatically on a schedule so your campaigns always use the latest predictions.
Predictive traits are built on the same machine learning models used in Hightouch [**AI Decisioning**](/ai-decisioning/overview). They are automatically trained on your historical event data and do not require custom ML expertise.
## Configure predictive inputs
Before you create a predictive trait, you must define **predictive inputs** in your schema. Predictive inputs are the user properties and event data that models will use for training and scoring.
- **User properties:** Stable attributes such as `plan_type`, `region`, `signup_source`, or `account_age`
- **Key events:** Behavioral actions such as `Viewed Product`, `Started Checkout`, or `Clicked Email`
Once saved, these inputs are reusable and can be applied to multiple predictive traits.
## Create a predictive trait
To create a predictive trait:
1. Go to **Customer Studio → Traits**.
2. Click **Create → New Trait**.
3. Configure **method**:
- Select a **parent model** (for example, `Users`).
- Under **Calculation method**, select **Prediction**.
- Click **continue**.
4. Configure **calculation**:
- Select the **predicted outcome event** (e.g., purchases at least once within 30 days)
- Choose **eligible users** (all users or a filtered subset).
- Set how often scores should be updated (daily is recommended).
- Choose **eligible users** (all users or a filtered subset).
- Set how often scores should be updated (daily is recommended).
- Click **continue**.
5. **Finalize**:
- **Name** your trait.
- Optionally, enter a **description**.
- Review the **summary** and click **Save**.
The system trains a model using your data and generates predicted scores for each user. Training typically takes several hours but will vary based on warehouse size and the amount of data being processed.
## View and use predictive traits
After training completes, predictive traits appear in the Traits list with type **Predictive**.
You can:
- **Preview results** in the trait details page
- **Use in audiences** by filtering on score thresholds or percentile ranges
- **Sync predictions** to destinations such as ad platforms, ESPs, or CRMs for activation
## Analyze predictions
The **Prediction analysis** tab shows model performance and conversion rates by percentile.
For example, in a purchase propensity model:
- The top 20% of users may convert at a significantly higher rate than average
- The bottom 80% may convert at a much lower baseline
This helps you evaluate model lift and choose effective thresholds for targeting.
Use percentile thresholds to build high-value segments. For example, create an audience of “Top 20% purchase propensity” users and sync them to ad channels for efficient spend.
## Use predictive traits in audiences
You can use predictive traits when defining audiences the same way you use other traits.
1. Go to **Customer Studio → Audiences** and click **Add audience**.
2. In the audience definition, select a trait.
3. Set the score range or percentile range you want to include (for example, 80–100%).
4. (Optional) Combine the predictive filter with other attributes, such as `country = US` or `plan_type = paid`.
Predictive traits can be layered with demographic or behavioral conditions, giving you highly targeted segments. For example:
- **High propensity to purchase + specific geography** (US, Canada)
- **Cart completion propensity + recent site activity** (visited in the last 7 days)
Predictive traits update on a schedule. Between updates, some people may complete the action you’re predicting (like making a purchase) but their score won’t change until the next update. To keep your audience focused on people who haven’t yet acted, **add a filter to exclude people who have already performed the predicted event** (e.g, `Purchases = 0` or `Purchases (in last 7 days) = 0`). This helps make sure your campaign only includes users who are still likely to take action.
## Sync predictive audiences
Once you define an audience with predictive traits, you can sync it to any connected destination the same way you sync other audiences.
For example, you might sync:
- **High purchase propensity** users to Google Ads or Meta Ads for paid acquisition
- **High cart completion propensity** users to an ESP for checkout reminder emails
When syncing, predictive scores are included alongside user attributes so downstream tools can use them directly.
After syncs run, you can monitor performance and health in the **Syncs > Overview** tab.
You don’t need to configure anything special for predictive traits when syncing--they behave like any other trait and can be mapped directly to destination fields.
## Related articles
- [Traits](https://hightouch.com/docs/customer-studio/traits)
- [Audiences](https://hightouch.com/docs/customer-studio/audiences)
- [Splits](https://hightouch.com/docs/customer-studio/splits)
- [AI Decisioning](https://hightouch.com/docs/ai-decisioning/overview)
---
## Priority Lists
**URL:** https://hightouch.com/docs/customer-studio/priority-lists
**Description:** Audience Priority Lists gives marketers control over the volume of marketing campaigns that a single user receives.
**Section:** Customer Studio
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Audience** | Marketers and analytics/data engineers |
| **Prerequisites** | [At least two audiences](https://hightouch.com/docs/customer-studio/usage) built using the same [parent model](https://hightouch.com/docs/customer-studio/schema#1-define-the-parent-model) |
*Control how many campaigns a single user receives by specifying audience priorities across your marketing programs.*
***
## Learning Objectives
After reading this article, you will be able to:
- Understand how Priority Lists manage overlapping audience membership.
- Create and manage a Priority List using the drag-and-drop editor.
- Ensure users are only included in their highest-priority campaign audience.
- Navigate to and modify an existing Priority List.
***
## Overview
Users often belong to multiple audiences based on buying behavior, demographics, or lifecycle stage. You may have campaigns targeting each of those audiences but you don’t want to message the same person multiple times.
**Priority Lists** let you control which audience takes precedence when there is overlap. When users qualify for multiple audiences in a Priority List, Hightouch only syncs them to the highest-priority audience. This ensures users receive only the most relevant messaging, prevents duplication, and improves campaign efficiency.
Priority Lists operate at sync time. Only one audience in a list will be active for a given user.
## Setup
Before setting up a Priority List, ensure you’ve created multiple audiences using the same [parent model](/customer-studio/schema#1-define-the-parent-model).
1. Go to **Customer Studio > [Priority lists](https://app.hightouch.com/priority-lists)**.
2. Click **Add priority list**.
3. Enter a **Name** and select a **Parent model**.
All audiences in the priority list must share this parent model.
3. Click **Add audiences** to open the audience picker.
You can only select audiences that are not already part of another priority list.
4. Reorder audiences by clicking and dragging them into the desired priority order.
The audience at the top has the highest priority.
5. Click **Save changes** to activate your Priority List.
You can return to this list anytime to modify the priority order or update which audiences are included.
## FAQ
#### Can an audience be in multiple priority lists?
No. One audience can only belong to one priority list.
#### Can a priority list have audiences created from different parent models?
No. A single priority list must only contain audiences that share a common **Parent Model**.
#### Do Insight tools, like Breakdowns and Overlaps, reflect if an audience belongs to a priority list?
Yes-[**Audience Insights**](/customer-studio/insights) take audience priority into account.
To learn more, read [the **Audience Insights** docs](/customer-studio/insights) or navigate to the [**Audiences**](https://app.hightouch.com/audiences) page and select an audience. The tools can be found under the **Overlap** and **Breakdown** tabs.
---
## Get started with Customer Studio
**URL:** https://hightouch.com/docs/customer-studio/quick-start
**Description:** This quick-start guide walks you through setup (for platform admins and data teams) and day-to-day usage (for marketers), with links to each key step.
**Section:** Customer Studio
| | |
| ----------------- | --------------------------------------------------------------------------------- |
| **Audience** | Platform admins and data teams, Marketers |
| **Prerequisites** | [Customer Studio overview →](https://hightouch.com/docs/customer-studio/overview) |
Customer Studio helps you build, activate, and orchestrate campaign-ready audiences directly from your warehouse.
This guide covers:
- [**Setup for platform admins and data teams →**](https://hightouch.com/docs/customer-studio/quick-start#setup-for-platform-admins-and-data-teams)
Configure your data schema, apply consent and governance rules, and enable features like sampling and holdout logs.
- [**Day-to-day usage for marketers →**](https://hightouch.com/docs/customer-studio/quick-start#how-marketers-use-customer-studio)
Use the visual interface to build audiences, sync to tools, analyze performance, and run campaigns.
---
## Setup for platform admins and data teams
Before marketers can build and sync audiences in Customer Studio, your workspace must be connected to the right data and destinations. This setup is typically done once per workspace.
### Connect sources and desintations
| Step | What you’ll do | Article |
| ------------------------------------------- | -------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- |
| **Connect your warehouse** | Ensure data sources are connected so models and traits can be defined. | [Sources →](https://hightouch.com/docs/getting-started/concepts#sources) |
| **Add destinations** | Set up tools where audiences will be delivered (e.g., CRMs, ESPs, ad platforms). | [Destinations →](https://hightouch.com/docs/getting-started/concepts#destinations) |
| **(Optional) Enable Lightning Sync Engine** | Improve sync throughput and speed for large or high-frequency audiences. | [Lightning Sync Engine →](https://hightouch.com/docs/syncs/lightning-sync-engine) |
Syncing audiences in Customer Studio requires both a source (data warehouse)
and at least one destination.
### Configure your data schema
| Step | What you’ll do | Article |
| --------------------- | ------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------- |
| **Define the schema** | Connect models for users, events, and related data so they’re available in the audience builder. | [Define data schema →](https://hightouch.com/docs/customer-studio/schema) |
---
### Optional setup
#### Enable observability and optimization
| Step | What you’ll do | Article |
| ----------------------------- | ----------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| **Enable sampling** | Speed up previews for large models by materializing a subset of data for audience and trait previews. | [Sampling →](https://hightouch.com/docs/customer-studio/sampling) |
| **Enable holdout group logs** | Track unsynced audience rows in your warehouse after each sync for auditing. | [Holdout group logs →](https://hightouch.com/docs/customer-studio/splits#holdout-group-logs) |
#### Manage governance and consent
| Step | What you’ll do | Article |
| --------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| **Apply destination rules** | Control sync behavior, redaction, and opt-outs. | [Destination rules →](https://hightouch.com/docs/customer-studio/destination-rules) |
| **Filter with subsets** | Narrow audience delivery on a per-destination basis. | [Subsets →](https://hightouch.com/docs/customer-studio/subsets) |
| **Manage consent** | Respect user consent preferences by syncing only users who have opted in. | [OneTrust Snowflake Native App →](https://hightouch.com/docs/customer-studio/onetrust) |
---
## How marketers use Customer Studio
Once the schema is set up, marketers can use the visual interface to build and activate audiences without SQL.
### Build audiences
Use filters, traits, and events to define who qualifies for a campaign.
| Step | What you’ll do | Article |
| -------------------- | --------------------------------------------------------------------- | --------------------------------------------------------------- |
| **Create audiences** | Apply filters and logic to build your target group. | [Audiences →](https://hightouch.com/docs/customer-studio/usage) |
| **Use traits** | Add calculated values like LTV, preferred channel, or last seen date. | [Traits →](https://hightouch.com/docs/customer-studio/traits) |
### Activate audiences
Send users to your tools on a schedule or in real time.
| Step | What you’ll do | Article |
| --------------------------- | ---------------------------------------------------------------------------- | -------------------------------------------------------------------- |
| **Sync audiences to tools** | Deliver audience data to destinations like Klaviyo, Meta, or Salesforce. | [Sync audiences →](https://hightouch.com/docs/customer-studio/syncs) |
| **Enable real-time syncs** | Send updates with sub-second latency for same-session personalization. | [Real-time syncs →](/real-time/real-time) |
### Analyze and validate
Check audience size, composition, and performance to ensure your segments behave as expected.
| Step | What you’ll do | Article |
| ---------------------------- | ------------------------------------------------------ | -------------------------------------------------------------------------------- |
| **Preview with insights** | Check size and breakdowns by trait or property. | [Insights →](https://hightouch.com/docs/customer-studio/insights) |
| **Spot-check with sampling** | Review a sample of audience members. | [Sampling →](https://hightouch.com/docs/customer-studio/sampling#using-sampling) |
| **Run experiments** | Use splits to test variations or apply control groups. | [Splits →](https://hightouch.com/docs/customer-studio/splits) |
| **Measure test results** | Use charts to evaluate audience splits. | [Splits charts →](https://hightouch.com/docs/campaign-intelligence/splits) |
### Orchestrate campaigns
Manage flows, priority, and message timing across audiences.
| Step | What you’ll do | Article |
| ---------------------------- | ------------------------------------------------------------- | ----------------------------------------------------------------------------- |
| **Create journeys** | Build multi-step campaigns with syncs, waits, and conditions. | [Journeys →](https://hightouch.com/docs/customer-studio/journeys) |
| **Manage campaign priority** | Use priority lists to handle audience overlaps. | [Priority lists →](https://hightouch.com/docs/customer-studio/priority-lists) |
### Manage and reuse audiences
Keep audiences consistent, auditable, and reusable.
| Step | What you’ll do | Article |
| --------------------- | ------------------------------------------------------------------------- | ---------------------------------------------------------------------------- |
| **Use templates** | Save standard audience or sync configurations for reuse. | [Templates →](https://hightouch.com/docs/customer-studio/templates) |
| **Capture snapshots** | Record audience membership at a point in time for auditing or comparison. | [Snapshots →](https://hightouch.com/docs/customer-studio/audience-snapshots) |
## What’s next?
- [Define your schema →](https://hightouch.com/docs/customer-studio/schema)
- [Build your first audience →](https://hightouch.com/docs/customer-studio/usage)
---
## Sampling
**URL:** https://hightouch.com/docs/customer-studio/sampling
**Description:** Sampling materializes a subset of a model's rows to make model previews faster.
**Section:** Customer Studio
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Marketers, Platform admins |
| **Prerequisites** |
A defined [data schema](/customer-studio/schema)
Warehouse [source](/getting-started/concepts#sources) with write permissions
|
*Sampling speeds up audience and trait previews in Customer Studio by materializing a small, representative subset of your schema models into your data warehouse. For large datasets, this optimization can significantly reduce query times and warehouse compute during campaign building.*
***
## What you'll learn
After reading this article, you'll be able to:
* Explain how sampling works and when to use it
* Enable and configure sampling for parent and related models
* Use sampled models for faster trait and audience previews
* Monitor and troubleshoot sampling jobs
***
## Overview
**Sampling** creates a reduced version of your schema models by materializing a subset of rows into dedicated tables in your data warehouse. This allows Hightouch to query smaller, faster subsets of data when generating previews in Customer Studio.
## How it works
Here’s how it works:
1. A configurable percentage of rows is sampled from your [**parent model**](/customer-studio/schema#1-define-the-parent-model).
2. For each sampled parent row, corresponding rows in [**related**](/customer-studio/schema#2-add-related-models) and [**event models**](https://hightouch.com/docs/customer-studio/schema#3-add-event-models) are also materialized.
3. The sampled tables are stored in the [**Hightouch Planner**](/syncs/lightning-sync-engine#warehouse-schemas) schema of your warehouse.
4. Sampling runs on a configurable schedule in the background.
This setup allows previews (like [traits](/customer-studio/traits) or [audiences](https://hightouch.com/docs/customer-studio/usage)) to query a representative subset of your data—ideal for faster iteration and testing.
Sampling requires write access to your warehouse and only works if the Lightning sync engine is enabled.
## Supported warehouses
Currently, sampling is supported on the following sources:
- [Amazon Redshift](/sources/amazon-redshift)
- [Databricks](/sources/databricks)
- [Fabric](/sources/fabric)
- [Google BigQuery](/sources/google-bigquery)
- [Microsoft SQL Server](/sources/sqlserver)
- [PostgreSQL](/sources/postgresql)
- [Snowflake](/sources/snowflake)
- [Trino](/sources/trino)
The sampled tables will be materialized in the [Hightouch Planner schema](/syncs/lightning-sync-engine#warehouse-schemas).
## Set up sampling
To enable sampling:
1. Go to a [**parent model**](/customer-studio/schema#1-define-the-parent-model) in your schema
2. Go to the `Sampling` tab
3. Toggle sampling **on**
Disabled | Enabled
:-------------------------:|:-------------------------:
 | 
You can then configure:
* **Sample size**: Determines the percentage of your parent model (and any related models) that will be sampled.
* **Sample frequency**: Determines how often the models get resampled. This schedule also applies to sampling for related models as well unless they're overridden.
* **Related models**: This table lists all of the related events and models. Tick the checkbox next to each related model you want to sample.
By default, each model will inherit the sampling schedule configured for the parent model. You can override the schedule if you wish.
***Last run***: This table lists information about the most recent run for each sampled model. If the sampling job has failed, hover over the status to see the error message.
## Related model
Sampling for [related models](/customer-studio/schema#2-add-related-models) is controlled through its parent model. However, you can check if the related model is being sampled by going to the Sampling tab:
## Using sampling
After sampling has been enabled and at least one run has successfully completed, you will be able to use the sampled model for certain preview features, such as the ones below.
* [**Audience**](/customer-studio/usage): A toggle will be present that allows you to use the sampled model. The result will also indicate that an approximation was returned.
* [**Trait**](/customer-studio/traits): A similar toggle will show up next to the preview button.
---
## Define data schema
**URL:** https://hightouch.com/docs/customer-studio/schema
**Description:** Configure parent, related, and event models so marketers can build audiences in Customer Studio.
**Section:** Customer Studio
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Data or analytics engineers, platform admins, technical marketers |
| **Prerequisites** |
A connected data source (e.g., Snowflake, BigQuery, Databricks)
Technical understanding of your data and its relationships
|
*Schema setup is a one-time configuration that determines what data marketers can use in the Customer Studio audience builder. It defines models, relationships, and events that power audience filtering.*
***
## What you'll learn
After reading this article, you'll know how to:
- Create parent, related, and event models
- Define relationships and join keys across datasets
- Configure optional schema features like merge columns and sampling
- Structure schema so marketers can build audiences more easily
---
## Overview
Before marketers can build audiences visually, data teams configure the schema in Customer Studio.
In Customer Studio, you’ll define:
- [**Parent model**](/customer-studio/schema#1-define-the-parent-model): The base entity audiences are built from (for example, `Users`)
- [**Related models**](/customer-studio/schema#2-add-related-models): Additional context joined to the parent (for example, `Households`)
- [**Event models**](/customer-studio/schema#3-add-event-models): Timestamped behaviors (for example, `Product Viewed`)
Once configured, marketers can build audiences using warehouse data.
---
### What is a data schema?
Imagine you run an online plant store. You want to target customers in the Midwest who recently bought a succulent.
To build this audience, Hightouch needs:
- Who your customers are (a `Users` table)
- What they’ve done (a `Purchases` table)
Together, these models and relationships form your **data schema**—a map of what marketers can filter on (for example, `purchased succulents in the last 30 days`) when building audiences.
---
### Coordinate with business teams
Before building your schema, align with marketers or campaign owners on:
- Which entities they’ll target (for example, Users, Accounts)
- Which attributes or behaviors they want to filter on
- Whether real-time or batch syncs are part of their workflow
- How frequently audiences need to update
Use this input to decide:
- Which tables or views to expose
- Which relationships to define
- How to name models and columns clearly for business users
---
### Schema design best practices
- Use clear, marketer-friendly names (avoid internal abbreviations)
- Keep the schema focused on fields that support targeting and analysis
- Start small (one parent model + a few related/event models), then expand over time
- Validate join keys early to avoid confusing audience results
---
## Schema setup
### 1. Define the parent model
The parent model is the core dataset for audience building--most commonly a `Users` table, but it could also be `Accounts`, `Households`, or `Devices`.
**Requirements:**
- One row per entity (for example, one row per user)
- A stable, unique **primary key** (for example, `user_id`)
- Fields useful for filtering (for example, `email`, `region`, `created_at`)
#### Create a parent model
1. Go to **Customer Studio → [Schema](https://app.hightouch.com/schema-v2/view)**.
2. Click **Create parent model**.
3. Select a modeling method:
- Table selector (default)
- SQL query
- dbt model
- dbt Cloud model
- Looker model
- Sigma model
4. Preview the results, then click **Continue**.
5. Configure the model:
| Field | Description | Example |
| ----- | ----------- | ------- |
| **Name** | Display name in Customer Studio | `Users` |
| **Description** | Context for your team | `Represents end users of the platform` |
| [**Primary key**](/getting-started/concepts#unique-primary-key-requirement) | Unique ID column | `user_id` |
| **Primary label** | Display name in audience previews | `full_name` or `email` |
| **Secondary label** | Additional preview context | `signup_date` |
**Column suggestions** are enabled by default to help marketers discover useful filters.
6. Click **Create parent model**.
---
### 2. Add related models
Related models join to the parent via a foreign key and provide additional filtering context—like purchases, subscriptions, or devices.
**Examples:**
- Households linked to People by `household_id`
- Subscriptions linked to Companies by `company_id`
#### Create a related model
1. Go to **Customer Studio → [Schema](https://app.hightouch.com/schema-v2/view)**.
2. Click the **+** icon next to your parent model.
3. Select **Create related model**.
4. Choose your table or modeling method and preview results.
5. Configure the relationship:
| Field | Description | Example |
| ----- | ----------- | ------- |
| **Name** | Related model name | `Households` |
| **Description** | Context for marketers | `User purchase records` |
| **Relationship** | Cardinality (`1:many`, etc.) | `1:many` |
| **Foreign key (join key)** | Column joining models | `user_id` |
To match on multiple columns (for example, `product_id` and `location_id`), enable **multiple join keys** in the relationship configuration.
The primary key uniquely identifies rows. The join key connects tables.
6. (Optional) If you want to add columns from the parent model onto the related model, turn on [**Merge columns**](#merge-columns).
7. Click **Create related model**.
---
### 3. Add event models
Event models represent timestamped behaviors like page views, logins, or checkouts. These power both real-time and historical audience logic.
**Requirements:**
- At least one **timestamp column** (for example, `created_at`)
- A **foreign key** linking to the parent model (for example, `user_id`)
#### Create an event model
1. Go to **Customer Studio → [Schema](https://app.hightouch.com/schema-v2/view)**.
2. Click the **+** icon next to your parent model.
3. Select **Create related event**.
4. Choose your table or modeling method and preview results.
5. Click **Continue**.
6. Configure the event:
| Field | Description | Example |
| ----- | ----------- | ------- |
| **Name** | Event name | `Product Viewed` |
| **Description** | Context for marketers | `Tracks user product views` |
| **Timestamp column** | When the event occurred | `created_at` |
| **Event type** | Category (`Generic`, `Checkout`) | `Generic` |
| **Primary key** | For referencing or triggering Journeys | `event_id` |
| **Relationship** | Cardinality to parent | `1:many` |
| **Foreign key (join key)** | Column joining event with parent | `user_id` |
The primary key uniquely identifies a row. The join key links it to another model. Often, the **primary key in the parent model** becomes the **join key in a related model**, but they serve distinct purposes.
7. (Optional) If you want to add columns from the parent model onto the event model, turn on [**Merge columns**](#merge-columns).
8. Click **Create event**.
If all events live in a single table, you can create multiple event models by filtering by event type (for example, `Page View` and `Add to Cart`).
---
## Manage schema
If your team uses Git for version control, you can manage your Customer Studio schema—including parent models, related models, event models, and relationships—using [Schema Git Sync](/extensions/configure-schema-git-sync#customer-studio-schema). Note that some UI-configured settings are not synced. See [Git Sync limitations for Customer Studio schema](/extensions/git-sync#git-sync-limitations-for-customer-studio-schema) for details.
### View and edit your schema
To manage models in your schema:
1. Go to **Customer Studio → [Schema](https://app.hightouch.com/schema-v2/view)**.
2. Select a model to open its details page.
3. Use the tabs across the top to configure the model.
Available tabs depend on the model type:
- [`Columns`](#columns): enable fields, set aliases, redact values, and manage suggestions
- [`Relationships`](#relationships): create and edit relationships to other models
- [`Match Booster`](#match-booster): enhance match rates using additional identifiers (if enabled)
- [`Sampling`](#sampling): configure sampling to speed up previews
- [`Activity`](#activity): review changes and see what depends on the model
- [`Configuration`](#configuration): manage core model settings like primary key, labels, and suggestion refresh interval
---
### Delete a model
1. Open the model you want to remove.
2. Click the three-dot menu, then select **Delete**.
Deleting a model removes it from the audience builder. Any audiences using it will break and must be updated.
---
## Columns
Use the `Columns` tab to control which fields appear in Customer Studio and how marketers interact with them.
1. Open the model you want to update.
2. Go to the `Columns` tab.
### Enable or exclude columns
Use the toggle next to each column to control whether it appears in Customer Studio.
Disable columns when:
- the field isn’t useful for filtering (for example, internal IDs)
- the field adds noise for marketers
- you don’t want it available in audience filters
### Rename columns using aliases
Aliases control how columns appear in the audience builder.
For example:
- `geo_region` → `Region`
- `created_at` → `Signup date`
To set an alias, hover over the column and click the pencil icon.
### Redact column values in previews
Redaction hides values in places like audience previews and profile exploration, while keeping the column available for filtering.
This is useful for sensitive fields like:
- email addresses
- phone numbers
- internal identifiers
### Enable filter value suggestions
Suggestions help marketers choose common values from a dropdown when filtering (for example, brands like `Nike` or `Adidas`).
You can also control how often suggestions refresh from the model’s [`Configuration`](#configuration) tab.
### Refresh columns from your source
If you add or remove columns in your warehouse, you can refresh the columns available in the model.
1. Open the model.
2. Click the three-dot menu.
3. Select **Refresh columns available in source**.
---
## Relationships
Use the `Relationships` tab to define how models connect in your schema. These relationships power cross-model filtering in the audience builder.
1. Open the model you want to update.
2. Go to the `Relationships` tab.
### Relationship types
Relationships define how rows connect across models:
| Relationship | Use when... | Example |
| :-- | :-- | :-- |
| `1:many` | One parent maps to many related records | One user has many purchases |
| `1:1` | One parent maps to one related record | A company has one active subscription |
| `many:1` | Many records map to one shared record | Many users belong to one household |
Use `1:many` for most behavioral data (for example, purchases, page views). Use `many:1` when referencing shared entities like plans or organizations.
### Add a relationship
1. Click **Add relationship**.
2. Choose the relationship type (for example, `1:many` or `many:1`).
3. Select the model you want to connect to.
4. Choose join keys for each model.
5. Click **Save**.
### Edit an existing relationship
To update a relationship:
1. Go to the `Relationships` tab.
2. Select the relationship you want to edit (for example, `Purchase History`).
3. Update join keys, the connected model, or cardinality.
4. Click **Save changes**.
### Match on multiple join keys
Multiple join keys let you match models on **two or more columns**. This is helpful when you need compound keys (for example, `product_id` and `location_id`).
#### Enable multiple join keys
1. Open the model and go to the `Relationships` tab.
2. Select the relationship you want to edit.
3. Toggle **Multiple join keys** ON.
4. Select the additional join key columns.
5. Click **Save changes**.
### Merge columns
Merge columns let you surface fields from a parent model into a related or event model, so marketers can filter on those fields without switching models.
This is useful when:
- the related/event model doesn’t include marketer-friendly fields like region or plan name
- you want to reduce duplication across traits or audiences
- marketers need both behavior (events) and customer attributes in the same workflow
Merged columns appear as **read-only** copies in the target model and are filterable like native fields.
#### Merge column eligibility
Whether you can merge columns depends on the relationship’s cardinality:
| Relationship type | Merge allowed? | Merge direction | UI toggle appears? |
| ----------------- | -------------- | --------------- | ------------------ |
| `1:1` | ✅ Yes | Either direction | ✅ Yes |
| `1:many` | ❌ No | N/A | ❌ No |
| `many:1` | ✅ Yes | From parent → related | ✅ Yes *(on related)* |
**Examples:**
- **Users → Purchases**
- Merge `geo_region` from Users so marketers can filter Purchases by customer location.
- **Products → Product Viewed events**
- Merge `brand` or `category` from Products so marketers can build audiences based on product attributes.
#### Enable merge columns
1. Open the model and go to the `Relationships` tab.
2. Select the relationship you want to edit.
3. Under **Merge columns**, toggle **Merge columns** ON.
4. Choose the columns you want to merge, then click **Save changes**.
If you don't see the merge columns toggle, check to see that your model has a cardinality of `1:1` or `many:1`.
### Through relationships
Through relationships let you connect two models through an intermediate model (a linking table). This is useful for many-to-many scenarios.
**Example:** Users → Memberships → Subscriptions
- Users join to Memberships on `user_id`
- Memberships join to Subscriptions on `subscription_id`
This enables filters like “users with an active subscription” without duplicating logic.
#### Set up a through relationship
Through relationships are configured in the `Relationships` tab of your parent model.
1. Create the two direct relationships:
- Users → Memberships (`1:many`)
- Memberships → Subscriptions (`many:1`)
2. Select your parent model, then open the `Relationships` tab.
3. Click **Add through relationship**.
4. Under **Access**, select the model you want to access (for example, `Products`).
5. In the through dropdown, choose the indirect path that connects your parent model to the target model (for example, `Purchases` → `Products`).
6. Click **Save**.
---
## Match Booster
Match Booster enhances match rates by enriching your schema with additional identifiers from Hightouch’s identity graph.
#### Enable Match Booster
1. Open your **parent model**.
2. Go to the `Match Booster` tab.
3. Toggle **Enable Match Booster**.
4. Select one or more identifier columns (for example, email or phone).
5. Click **Initialize Match Booster** to start enrichment.
Learn more: [Match Booster overview →](/match-booster/overview)
---
## Sampling
Sampling creates a smaller subset of your model data to improve performance when previewing audiences.
#### Enable sampling
1. Open your model.
2. Go to the `Sampling` tab.
3. Turn on sampling and configure the settings.
4. Click **Save & run sampling**.
Learn more: [Sampling →](/customer-studio/sampling)
---
## Activity
Use the `Activity` tab to review recent changes and understand how a model is being used.
You can track:
- recent updates to the model definition
- audiences, traits, or syncs that reference the model
- usage history for debugging and cleanup
---
## Configuration
Use the `Configuration` tab to manage core model settings such as primary key and preview labels.
Common settings include:
- **Primary key**
- **Primary label**
- **Secondary label**
- **Column suggestion refresh interval**
### Column suggestion refresh interval
Hightouch can automatically refresh filter suggestions so dropdown values stay up to date as your warehouse data changes.
1. Open your model.
2. Go to the `Configuration` tab.
3. Under **Column suggestion refresh interval**, choose how often suggestions should update.
This setting controls how frequently Hightouch updates **suggested dropdown values** in the audience builder.
It does **not** affect:
- live audience results
- sync behavior
- warehouse queries (these always use your source-of-truth data)
Refreshing columns updates the fields and **dropdown values** shown in filters. It does not change your warehouse data or audience results.
---
## What’s next?
After defining your schema, choose the next step based on your role.
#### If you're managing governance and delivery
Set up rules that control how data flows to destinations and how consent is enforced:
- [**Destination rules** →](/customer-studio/destination-rules)
Define sync behavior, redaction policies, and delivery restrictions per destination.
- [**Subsets** →](/customer-studio/subsets)
Apply destination-specific filters (for example, opt-outs) without changing core audience logic.
- [**OneTrust Snowflake Native App** →](/customer-studio/onetrust)
Enforce consent policies using Snowflake-native integration with OneTrust.
#### If you're a marketer
Once the schema is in place, you can begin building audiences:
- [**Audiences** →](/customer-studio/usage)
Use filters, traits, and events to define dynamic segments directly from your data.
---
## Configure Schema Git Sync
**URL:** https://hightouch.com/docs/customer-studio/schema-setup
**Description:** Connect your Customer Studio schema to a Git repository for version control, consistency, and collaboration across environments.
**Section:** Customer Studio
| | |
|----------------------|-----------------------------------------------------------------|
| **Audience** | IT, DevOps, Data Engineering, or Analytics |
| **Prerequisites** |
Access to a Hightouch workspace
A Git repository (GitHub, GitLab, etc.) for schema storage
[Git Sync](https://hightouch.com/docs/extensions/git-sync) enabled in your workspace
Appropriate permissions for Git and Hightouch
|
_**Schema Git Sync** extends [Git Sync](/extensions/git-sync#customer-studio-schema), Hightouch’s integration for version-controlling resources in Git, to include **Customer Studio schema models, events, and relationships**. Use it to keep schemas consistent across environments (for example, staging and production) and to version, review, and recover changes directly from your Git repository._
## Overview
**Schema Git Sync** connects your Customer Studio schema—parent, related, and event models, plus their relationships—to a Git repository.
Whenever you update your schema in Hightouch, the changes are automatically written to YAML files in Git, and vice versa.
This gives your team all the benefits of **version control** for your customer data model:
- Every schema change is captured as a Git commit.
- You can roll back to earlier versions if needed.
- Teams can review and approve schema updates using pull requests.
- Environments like *staging* and *production* can stay perfectly aligned.
Think of it as turning your schema into **infrastructure as code**: you define your data model in Hightouch, but track, share, and deploy it safely through Git.
For authentication, repository setup, and permissions, see the [main Git Sync guide →](https://hightouch.com/docs/extensions/git-sync#setup).
---
## Setup overview
Schema Git Sync setup involves these main steps:
1. [Set up and enable Git Sync in your workspaces](#step-1--set-up-workspaces)
2. [Create your Git repository](#step-2-create-your-git-repository)
3. [Connect Git to each workspace](#step-3-connect-git-to-each-workspace)
4. [Configure branches and paths](#step-4-configure-file-paths-branches)
5. [Build your schema in Customer Studio](#step-5-create-your-schema)
6. [Sync schema changes to Git](#step-6-sync-schema-to-git)
7. [Validate and troubleshoot syncs](#validation-checklist)
For repository authentication and credential setup, see the [main Git Sync guide →](/extensions/git-sync#setup).
---
## Step 1 – Set up workspaces
Git Sync is behind a feature flag.
Ask your Hightouch account team to enable it in each workspace you plan to sync.
Create or identify the Hightouch workspaces you’ll connect to Git.
Each workspace typically represents an environment (for example, *staging* and *production*).
Clone your data source into each workspace to maintain identical schemas across environments.
---
## Step 2 – Set up your Git repository
Create a dedicated Git repository for your schema files.
Hightouch uses this repo to store YAML definitions for all models, events, and relationships.
**Automatically generated repository structure:**
```bash
/syncs/ # YAML files for all syncs
/models/ # YAML files for all models
/schema/ # Customer Studio schema files
├── event-models/ # Event model definitions
├── related-models/ # Related model definitions
├── parent-models/ # Parent model definitions
├── model-relationships/ # Relationship model definitions
manifest-*.yaml # Auto-generated source/destination listings
```
For detailed repo setup and authentication, follow the [Git Sync configuration steps](/extensions/git-sync#configure-git-sync).
---
## Step 3: Connect Git to each workspace
Configure your Git repository connection from Hightouch.
1. In Hightouch, go to **Integrations → [Extensions](https://app.hightouch.com/extensions)** and select **Version control with Git**.
2. Under **Overview**, click **Configure extension**.
3. Under **Git credentials**, click **Set up** and select your Git service (e.g., GitHub App).
4. Follow the [**Git Sync Authentication instructions**](https://hightouch.com/docs/extensions/git-sync#authenticate-to-git) to authenticate and grant Hightouch permission to your repository.
Once connected, Hightouch displays your sync status and automatically detects inbound and outbound changes.
Branches do not need to exist beforehand. Hightouch creates them automatically during setup.
---
## Step 4: Configure branches (optional)
### Use the same branch (recommended)
For most teams, point all workspaces at the same branch (for example, `main`). Both workspaces read from and write to the same YAML files, and [alias files](/extensions/git-sync#aliases) handle any differences in source or destination IDs. This is the simplest setup and works well when all workspaces share the same schema and parent model structure.
### Use separate branches
Only use separate branches if your environments intentionally diverge — for example, if your staging workspace includes additional parent models or experimental relationships that should not reach production yet.
**Example mapping (for divergent environments):**
| Environment | Workspace | Git Branch |
| ----------- | ---------------------- | ------------------- |
| Staging | `hightouch-staging` | `schema-staging` |
| Production | `hightouch-production` | `schema-production` |
Promote changes by merging from your staging branch into production via pull request. Use [CI checks](/extensions/git-sync#validating-changes) to validate changes before they reach production.
For a full walkthrough of connecting staging and production workspaces to the same repo, including alias files and branch strategies, see [Connect staging and production workspaces →](/extensions/git-sync#connect-staging-and-production-workspaces).
For more about branch and path setup, see [Configure Git Sync →](/extensions/git-sync#configure-git-sync).
---
## Step 5: Create your schema
Build your schema models in Customer Studio.
**Define:**
- **Parent models** (e.g., `Users` table)
- **Related models** (e.g., `Purchases`, `Subscriptions`)
- **Event models** (e.g., `Page Views`, `Clicks`)
- **Relationships** between models
Each model and relationship automatically becomes a YAML file in your Git repository under `/schema/`.
**Example workflow:**
1. Create a `Users` parent model.
2. Add a related model such as `Purchases`.
3. Define a relationship between them (e.g., “Users → Purchases”).
4. Save — Hightouch commits each definition as YAML in Git.
Follow step-by-step instructions to define your schema in Customer Studio: [Define data schema →](https://hightouch.com/docs/customer-studio/schema)
---
## Step 6 – Sync your schema to Git
Once Git Sync is enabled and YAML generation is turned on, Hightouch automatically performs an **initial full sync** of your schema. This first sync creates YAML files for your models, events, relationships, alias files, and manifest files in your Git repository.
After the initial setup, any changes you make in Customer Studio—such as creating or editing models or relationships—are automatically committed to Git within about 30 seconds.
You can monitor progress in the **Runs** tab of the Git Sync extension or by checking your Git commit history.
### How syncing works
- **Initial sync:** Runs automatically when Git Sync is first enabled.
- **Automatic updates:** Every schema change in Customer Studio commits automatically to Git within about 30 seconds.
- **Two-phase commits:** Hightouch commits models first, then relationships, to prevent dependency errors.You don’t need to manage this sequencing manually; Hightouch handles it automatically during both manual and automatic syncs.
- **Full resyncs:** You can trigger a full resync manually if you change repository or branch settings, or to troubleshoot issues.
Large initial syncs may take 10–20 minutes depending on schema size.
### What your YAML files look like
Once synced, each schema component appears as a YAML file in your repository under `/schema/`. The structure of each file depends on the model type.
For full YAML field references, see the [Customer Studio Schema section of the Git Sync guide](/extensions/git-sync#customer-studio-schema). Key things to know:
- **Parent models** follow the [standard model schema](/extensions/git-sync#model-schema), with optional `primaryLabel` and `secondaryLabel` fields.
- **Related models** follow the [standard model schema](/extensions/git-sync#model-schema).
- **Event models** follow the [standard model schema](/extensions/git-sync#model-schema), with a required `eventTimestampColumn` field.
- **Relationships** have a more complex schema depending on whether they are direct or [through relationships](/extensions/git-sync#model-relationship-schemas).
The easiest way to learn the YAML schema for any model type is to create it in the Hightouch UI first and inspect the resulting file in your Git repository.
Field names in YAML files are case-sensitive. If a column name in your warehouse changes casing (for example, `user_id` to `User_Id`), Git Sync treats these as different fields and may cause unexpected behavior.
---
## Step 7 - Two-phase commit process
Schema Git Sync automatically commits schema changes in two stages to prevent dependency errors.
| Phase | Description |
|--------|-------------|
| **1 – Models** | Hightouch commits model definitions first to register columns and sources. |
| **2 – Relationships** | Relationships commit after models exist, ensuring valid references. |
This process happens automatically during both manual and automatic syncs.
---
## Validation Checklist
Before your first sync:
**Feature setup**
- Git Sync feature flag is enabled for each workspace.
- Each workspace includes an alias file (`aliases-{workspace-slug}.yaml`).
- Branches or paths are unique only if environments diverge (for example, different parent models).
**Schema setup**
- All workspaces use identical model slugs if they share the same schema YAMLs.
- Source and destination slugs can differ; Hightouch manages these automatically through [alias files](#resource-slug-consistency-and-aliases).
- Data sources are cloned consistently across environments.
If these prerequisites aren’t met, your sync may fail with `error serializing resource`. If model slugs don’t match across workspaces, Hightouch can’t link shared schema YAMLs correctly. Alias files handle differences for sources and destinations, but not for models.
---
## Resource slug consistency and aliases
Hightouch uses **resource slugs**—unique identifiers for each model, source, or destination—to link schema YAML files in Git with the corresponding resources in each workspace connected to Git Sync.
To maintain consistency across environments:
- **Model** and **event contract** slugs must be identical across workspaces sharing the same repository.
- **Source** and **destination** slugs can differ between environments because Hightouch automatically handles those differences through **alias files**.
Alias files act as a translation layer between environments. When your staging and production workspaces use different data sources or destinations, these files ensure the same schema YAMLs continue to work correctly in both.
### How alias files work
Each workspace connected to Git Sync automatically gets its own alias file in your Git repository:
```yaml
aliases-{workspace-slug}.yaml
```
You don’t need to create these files manually—Hightouch automatically generates an empty alias file if one doesn’t exist.
If your workspaces use different data sources or destinations, (for example, `snowflake-dev` in staging vs. `snowflake-prod` in production), you can edit these files to define the correct mappings for each environment.
**Example structure:**
```yaml
sources:
main-database: snowflake-dev
destinations:
crm-platform: salesforce-sandbox
```
- **Keys** (for example, `main-database`) are shared alias names referenced in your schema YAMLs.
- **Values** (for example, `snowflake-dev`) are the actual resource slugs in that specific workspace.
This lets all environments share the same schema YAMLs while Hightouch automatically resolves the right connections in each workspace.
#### Example: staging vs. production
```yaml
# aliases-staging.yaml
sources:
main-database: snowflake-dev
destinations:
crm-platform: salesforce-sandbox
# aliases-production.yaml
sources:
main-database: snowflake-prod
destinations:
crm-platform: salesforce-prod
```
Both workspaces use the same schema YAMLs, while each resolves to its own data sources and destinations.
---
## Troubleshooting
If a sync fails with an error like:
```go
error serializing resource
```
#### Meaning
Git Sync tried to serialize (save) a schema component to YAML but couldn’t find a matching resource in the workspace.
#### Common causes
- Mismatched model or source slugs between environments
- Missing or incorrect alias mappings
- Workspace connected to the wrong Git branch
- Schema synced before alias setup
#### Fix checklist
1. Verify model and source slugs match across environments.
2. Confirm alias files map correctly to workspace resource IDs.
3. Ensure each workspace is using its correct Git branch.
4. Trigger a Full resync after correcting slugs or aliases.
For more help, see the [Git Sync troubleshooting guide](https://hightouch.com/docs/extensions/git-sync#faq) or contact Hightouch Support.
---
## Best practices
- Plan workspace and branch structure before syncing.
- Test schema changes in staging before merging to production.
- Use pull requests to review schema changes before deployment.
- Keep model and event contract (event schema) slugs identical across all workspaces sharing the same repository.
- Periodically audit alias files and manifests after cloning or renaming resources.
- Document your branch-to-workspace mapping for future maintainers.
For schema modeling guidance, see [Schema Management in Customer Studio](/customer-studio/schema).
---
## Related resources
- [Git Sync Overview](https://hightouch.com/docs/extensions/git-sync)
- [Schema management in Customer Studio](/customer-studio/schema)
- [Sync monitoring](https://hightouch.com/docs/customer-studio/syncs#monitor-syncs#monitor-syncs)
- [Warehouse sync logs](https://hightouch.com/docs/syncs/warehouse-sync-logs)
---
## Subsets
**URL:** https://hightouch.com/docs/customer-studio/subsets
**Description:** Subsets allow organizations to segment their total addressable audience data based on its attributes and then grant employees access to that data based on the segment.
**Section:** Customer Studio
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Platform admins |
| **Prerequisites** | [Set up for platform admins and data teams →](https://hightouch.com/docs/customer-studio/quick-start#setup-for-platform-admins-and-data-teams) |
*Subsets restrict which rows of customer data users can access in Customer Studio. Admins define subset filters based on attributes like region, brand, or consent type, then assign access to specific teammates. All audiences built by those teammates are automatically scoped to the rows they’re permitted to see.*
***
## Learning objectives
After reading this article, you’ll be able to:
- Understand how subsets restrict data access in Customer Studio
- Create and configure subset categories and subsets
- Assign users to specific subsets
- Apply subsets in the audience builder and enforce base filters
***
## Overview
Subsets allow organizations to segment their total addressable audience data
based on its attributes and then grant employees access to that data based on
the segment. For example, some organizations may have a Global Customers table
but only want employees to be able to access data in Hightouch from the region
in which they work. By creating a subset for each region, companies can assign
specific employees to each region and limit the data each employee can view and
activate.
### Common use cases for subsets
- **Consent**: Some companies may collect consent or opt-ins for different marketing use
cases. Companies can use subsets to create different filters for each
consent type. For example, lifecycle marketers may be assigned to an “Email
Opt-Ins” subset, while performance marketers may be assigned to a subset for
“Targetable Customers.”
- **Region**: In multinational corporations, subsets can help scope employee data access
to data within a particular geography or region.
- **Brand**: For companies with multiple business units, brands, or product lines,
subsets can be applied so that employees can only create audiences with data
scoped to their specific business unit, brand, or product.
### Subset categories vs. subsets
A parent model can be segmented with subset categories and subsets. In the above
example, region is the subset category and each region (North America, EMEA,
APAC) is the subset.
When building an audience, you can select more than one subset category and more
than one subset within each subset category. A user must be part of any subset
within a subset category, and satisfy all subset categories to be part of the
audience.
## Set up subsets
### 1. Navigate to the Governance page
Navigate to **Customer Studio > Governance**, then open the [**Subset categories** tab](https://app.hightouch.com/governance/subset-categories).
### 2. Select a parent model
Select your **parent model** at the top.
### 3. Create a subset category
Click **Add subset category**, name the category (e.g., `Region`, `Consent`), and optionally add a description.
Once created, you'll see the category appear in the list.
## 4. Set category requirements (optional)
Click into the new category and toggle **Required** if you want users to be required to select a subset from this category when creating an audience.
## 5. Add subsets to the category
Click **Add subset** to define specific filters for that category (e.g., `Australia`, `Canada`, `Email Opt-In`, `Targetable Customers`).
1. Name your subset
2. Click **Add filter** to define who is included
3. Select which users should have access
To create your subset, you can mix and match different conditions to segment
your parent model to a specific filter. Similar to building an audience, the
builder provides five types of filters you can use
with nested Boolean (`AND` / `OR`) logic.
- Property conditions
- Related model conditions
- Event conditions
- Audience conditions
- Trait conditions
For help using the builder UI, see [Add filters →](https://hightouch.com/docs/customer-studio/usage#step-2-add-filters).
## 5. Grant access to users
Once your filter conditions have been created, select which users should have
access to this subset.
## 6. Save subset
Click **Create subset**
## Use subsets in audience creation
When creating or editing an audience, users will see any required categories and can choose one or more subsets they have access to. Subset filters will be applied as base filters on top of the audience definition.
If a user does not have access to any subsets in a required category, they won't be able to build audiences using that parent model.
## Related articles
- [Create audiences →](https://hightouch.com/docs/customer-studio/usage)
- [Destination rules →](https://hightouch.com/docs/customer-studio/destination-rules)
---
## Sync audiences
**URL:** https://hightouch.com/docs/customer-studio/syncs
**Description:** Sync audiences to downstream destinations using Hightouch's sync templates.
**Section:** Customer Studio
| | |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers, Platform admins |
| **Prerequisites** |
A saved [audience](https://hightouch.com/docs/customer-studio/usage)
At least one [destination](https://hightouch.com/docs/getting-started/concepts#destinations)
|
_Once you’ve [built and saved an audience](https://hightouch.com/docs/customer-studio/usage), you can sync it to destinations—like CRMs, ad platforms, or email service providers. Hightouch keeps those destinations up to date by automatically adding or removing users as the audience changes._
---
## What you'll learn
After reading this article, you’ll know how to:
- Sync a saved audience to a connected destination
- Configure sync frequency, identifier fields, and removal settings
- Monitor and manage sync status
- Apply sync templates to save time
---
## Overview
Once you've [created an audience](/customer-studio/usage), the last step is to sync it to your desired destination(s).
For example, say you've created an audience of users who've added a particular product type to their cart without purchasing it. You likely want to sync this audience to ad platforms like Meta or Google so you can target them with personalized ads about the product in their cart.
Sync configuration for an audience is the same as any other sync configuration. You need to [map fields](/syncs/mapping-data), [schedule](/syncs/schedule-sync-ui) the sync, and [set up alerts](/syncs/alerting) as desired.
Most syncs run on a set schedule and send the full audience to your destination (e.g., hourly CRM updates or daily data warehouse exports). For event-driven campaigns—like retargeting users after a checkout or suppressing ads after a purchase—Hightouch also supports real-time syncs that deliver users the moment they enter or exit an audience.
{" "}
To learn more about syncing audiences in **real time**, see [Real-time audiences](/real-time/real-time).{" "}
## Add a sync to an audience
To create a sync for an existing audience:
1. Go to **Customer Studio → [Audiences](https://app.hightouch.com/audiences)**
2. Open a saved audience and click **Add sync**
3. Select a **destination**
Refer to your chosen [Destination docs](https://hightouch.com/docs/destinations/acoustic) for specific configuration details.
For more information on creating syncs, check out the [sync configuration docs](https://hightouch.com/docs/syncs/overview).
## Monitor syncs
After setup, you can monitor and manage your syncs from the Syncs tab of each audience.
Each sync shows:
- **Destination name**
- **Sync status** (e.g., active, paused, failed, warning)
- **Last run time**
- **Sync frequency**
- **Matched users** — an approximate count of users successfully matched in the destination
- **Last updated date**
- A **button to manually run the sync**
You can pause, resume, or edit any sync at any time.
## Use sync templates
If your workspace includes [**sync templates**](https://hightouch.com/docs/customer-studio/templates#sync-templates), you can apply one when creating a new sync. Templates help standardize common configurations across destinations or teams.
Templates may pre-fill values like:
- Destination
- Identifier field
- Field mappings
- List name
- Removal behavior
{" "}
Learn how to create and apply [Sync templates](https://hightouch.com/docs/customer-studio/templates#create-a-sync-template).{" "}
## Related articles
- [Create audiences →](https://hightouch.com/docs/customer-studio/usage)
- [Real-time audiences →](/real-time/real-time)
- [Syncs overview →](https://hightouch.com/docs/syncs/overview)
---
## Templates
**URL:** https://hightouch.com/docs/customer-studio/templates
**Description:** Templates let you reuse common logic across audiences, syncs, and journeys.
**Section:** Customer Studio
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Platform admins, Marketers |
| **Prerequisites** |
A saved [audience](https://hightouch.com/docs/customer-studio/usage)
A [configured sync](https://hightouch.com/docs/customer-studio/syncs)
Or a [saved journey](https://hightouch.com/docs/customer-studio/journeys)
|
*Templates let you reuse common logic across audiences, syncs, and journeys--helping you reduce manual setup—especially for recurring targeting strategies or multi-team workflows.*
***
### Learning objectives
After reading this article, you'll know how to:
* Create audience, sync, and journey templates
* Apply templates to new audiences, syncs, or journeys
* Standardize workflows across teams
---
## Types of Templates
Templates are reusable components in Customer Studio. There are three types:
| Template type | What it stores | Where it’s used |
| ----- | ----- | ----- |
| [**Audience templates**](http://hightouch.com/docs/customer-studio/templates#audience-templates) | Filter conditions, traits, and event logic | Audience builder |
| [**Sync templates**](http://hightouch.com/docs/customer-studio/templates#sync-templates) | Destination, identifiers, sync frequency, and behavior | Sync setup |
| [**Journey templates**](http://hightouch.com/docs/customer-studio/templates#journey-templates) | Workflow steps and logic | Journey builders |
Templates are workspace-wide and can be created by platform admins or users with appropriate permissions.
---
## **Use cases**
| Use case | Example |
| ----- | ----- |
| **Standardized targeting** | Create an audience template for “High-value cart abandoners” used across channels |
| **Faster setup** | Prefill sync fields for destinations with consistent matching settings |
| **Cross-team coordination** | Let marketers safely reuse logic built by analysts or engineers |
---
## Audience templates
### Create an audience template
1. Go to **Customer Studio → Templates → [Audience templates](https://app.hightouch.com/templates/audience-templates)**
2. Click **Add audience template**
3. Select a parent model
4. Define filters, traits, or events like you would when [building a normal audience](https://hightouch.com/docs/customer-studio/usage)
* Leave blank fields where customization is expected—users will be prompted to fill these when applying the template
5. Name the template and optionally add a description
6. Click **Finish**
### Apply an audience template
To use an audience template:
1. Go to **Customer Studio → [Audiences](https://app.hightouch.com/audiences)**
2. Click **Add Audience** and select a parent model
3. Open the 3-dot menu and click **Add a template** (or select a template beneath **Start with an audience template**)
4. Select a template with a matching parent model
5. Fill in any required blanks (highlighted in blue)
6. Save and finish editing as needed
You can apply more than one audience template to the same audience. Filter logic from each will be combined.
---
## Sync templates
When managing many audiences, it can become tedious to recreate the same sync configuration each time. For example, you may want to sync several [Lookalike audiences](https://hightouch.com/blog/driving-paid-media-roas-through-audience-activation#lookalike-audiences) to Meta. Hightouch offers a tool called **sync templates** to streamline your audience sync setup.
Sync templates let you define sync configuration settings _once_ for a particular destination. You create the sync template and then use it to add syncs for each audience requiring the same configuration settings and schedule. You can also use multiple templates for an audience if you're sending it to several destinations.
### Create a sync template
1. Go to **Customer Studio → Templates → [Sync templates](https://app.hightouch.com/templates/sync-templates)**
2. Click **Add sync template**
3. Choose a **parent model**
4. Select a **destination** and configure the sync settings.
Follow individual [destination configuration instructions →](http://hightouch.com/docs/destinations/acoustic)
5. Click **Save**
You can now use the sync template for multiple audience syncs.
### Apply a sync template during audience creation
1. Build a new audience and proceed through:
* **Select parent model**
* **Define audience**
* **Configure audience details**
2. On the **Sync to destination (optional)** step, select one or more matching sync templates
3. Click **Finish**, or **Save audience without syncs** to skip for now
### Apply a sync template to an existing audience
Audiences can only use sync templates derived from the same parent
model as the audience.
1. Go to **Customer Studio → [Audiences](https://app.hightouch.com/audiences)**
2. Select an audience
3. Click **Add** and choose **Sync** from the dropdown
4. In the modal, select one or more **Sync templates**
5. Click **Add syncs**
The sync inherits configuration and scheduling from the template. If the template changes, those changes propagate to all derived syncs.
### Require sync templates for audience activation
Workspace admins can optionally enforce the use of sync templates for audience-based activation.
To enable this:
1. Go to **Settings → Workspace → Workspace**.
2. Turn on **Require sync templates for audience activation**.
3. Click **Save changes**.
When this setting is enabled:
- Users can **only** add syncs to audiences using **sync templates**.
- Users **cannot detach** audience syncs from their templates.
- If no sync templates exist for a workspace, users won’t be able to add new audience syncs until at least one template is created.
This setting is useful when you want to standardize identifiers, enforce consistent mapping/consent handling, and prevent one‑off audience sync configurations.
### Apply a sync template in journeys
1. Build your journey and add a **Send to destination** tile to your canvas.
2. When configuring your sync, select a template from the dropdown
3. Fill in any of the fields that were left blank in the template, or make edits to fields that are unlocked.
4. Save your sync and journey.
This sync inherits configuration from the template. If the template changes, those changes will propogate to the journey syncs.
Journey Setting Configurations (Trigger or Cohort mode) on the template do **not** propogate to the syncs using the template. If you make a change to this setting, you'll need to update the syncs individually.
### Unlock sync template fields
You can configure templates to leave certain fields “unlocked,” allowing users to customize them per sync.
Supported destinations include:
- [Adobe Campaign Classic](/destinations/adobe-campaign-classic)
- [Amazon Ads](/destinations/amazon-ads)
- [Braze](/destinations/braze)
- [Braze Cohorts](/destinations/braze-cohorts)
- [Criteo](/destinations/criteo)
- [Display & Video 360](/destinations/display-video)
- [Meta Custom Audiences](/destinations/facebook)
- [Google Ads](/destinations/google)
- [Google Ad Manager 360](/destinations/gam360)
- [Google Partner Audiences](/destinations/google-partner-audiences)
- [Google Sheets](/destinations/google-sheets)
- [HTTP Request](/destinations/http-request)
- [Iterable](/destinations/iterable)
- [LinkedIn](/destinations/linkedin)
- [Marigold Engage](/destinations/marigold-engage)
- [Marigold Engage Plus](/destinations/marigold-engage-plus)
- [Magnite](/destinations/magnite)
- [Marketo](/destinations/marketo)
- [Microsoft Bing Ads](/destinations/bingads)
- [Pinterest Ads](/destinations/pinterest-ads)
- [Reddit Ads](/destinations/reddit-ads)
- [S3](/destinations/s3)
- [Salesforce Marketing Cloud](/destinations/sfmc)
- [Snapchat](/destinations/snapchat)
- [Talon.One](/destinations/talon-one)
- [TikTok](/destinations/tiktok)
- [The Trade Desk](/destinations/tradedesk)
- [Upland Waterfall](/destinations/upland-waterfall)
- [X Ads](/destinations/x)
- [Yahoo](/destinations/yahoo)
- [Yahoo DataX](/destinations/yahoo-datax)
**To unlock a field:**
1. Open the sync template configuration
2. Toggle **Lock this field** off
Some fields are dependent on others. Hover over the toggle to see dependencies.
Once the template is attached to an audience, you can override unlocked fields.
You can update these overrides at any time from the sync configuration.
---
## Journey templates
### Create a journey template
1. Go to **Customer Studio → Templates → [Journey templates]([https://app.hightouch.com/templates/journey-templates)**
2. Click **Add a journey template**
4. Select a parent model
5. Enter a template name and optionally add a description
6. Click **Create template**
### Use a journey template
1. Go to **Customer Studio → [Audiences](https://app.hightouch.com/audiences)**
2. Select an audience
3. Click **Add** and choose **Journey** from the dropdown
4. In the modal, enter a **name** and optional **description**
5. Click **continue**
You’ll be taken to the journey builder with the selected template pre-applied.
---
## Best practices
* Use templates for commonly used audiences, especially suppression groups or VIP cohorts
* Leave placeholders in audience templates where campaign owners should customize filters
* Use sync templates to enforce consistent identifiers and opt-out handling across destinations
* Name templates clearly by purpose (e.g., “Klaviyo cart abandon sync” or “Loyalty email segment”)
---
## Related articles
* [Create an audience →](https://hightouch.com/docs/customer-studio/usage)
* [Set up a sync →](https://hightouch.com/docs/customer-studio/syncs)
* [Build a journey →](https://hightouch.com/docs/customer-studio/journeys)
---
## Traits
**URL:** https://hightouch.com/docs/customer-studio/traits
**Description:** Hightouch Traits enable users of any technical background to create and sync computed user attributes to various business applications.
**Section:** Customer Studio
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Marketers and analytics/data engineers |
| **Prerequisites** |
A defined schema with at least one parent model.
(Optional) Connected destinations for syncing audiences.
|
## Overview
When defining audiences, you often need computed or calculated attributes rather than raw data.
For example, you might want to create an audience based on **Lifetime Value (LTV)** even though your `Users` table doesn’t have this column. In that case, you can compute it by summing all purchases made by a user.
**Traits** in Hightouch let users of any technical background:
- [Create computed fields](/customer-studio/traits#create-a-trait)
- Use them to [define audiences](#create-audiences-with-traits)
- [Sync](#enrich-audiences-with-traits) them to business applications. Only parent model traits can be synced.
You can think of traits as reusable computed fields that power personalization, segmentation, and campaign automation.
---
## Trait types
| **Trait Type** | **Description** | **Example Usage** |
|------------------------|---------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------|
| [**Aggregation**](http://hightouch.com/docs/customer-studio/traits#aggregation-traits-sum-average) | Calculates the `sum` or `average` of a numeric field across a related or event model. | Total amount a user has spent across all purchases (`order_value`) |
| [**Count**](http://hightouch.com/docs/customer-studio/traits#count-traits) | Counts the number of records that match optional filters. | Number of emails a user opened in the past 30 days |
| [**Occurrence**](http://hightouch.com/docs/customer-studio/traits#occurrence-traits-first-last-most-frequent) | Returns the first, last, most frequent, or least frequent value from a column in related data. | Most frequently viewed product category or last brand interacted with |
| [**List**](http://hightouch.com/docs/customer-studio/traits#list-traits) | Returns an array of values or objects from related or event records, with optional sorting, slicing, and filters. | List of SKUs a user has browsed or added to cart in the past 7 days |
| [**SQL Aggregation**](http://hightouch.com/docs/customer-studio/traits#sql-aggregation-traits) | Uses SQL to calculate a value across related records. Supports advanced logic like JSON or arrays. | JSON object of purchased products with price and quantity, for use in dynamic email personalization |
| [**SQL Formula**](http://hightouch.com/docs/customer-studio/traits#sql-formula-traits) | Computes logic using SQL on model-level fields without joins. Ideal for string manipulation or flags. | Boolean trait for whether a user is in a loyalty program (`membership_tier = 'Gold'`) |
Traits can be defined on any type of model, though in most uses cases they will be defined on the parent model.
---
## Create a trait
You can create a trait in one of three ways:
**Option 1:**
1. Go to **Customer Studio → [Traits](https://app.hightouch.com/traits)**
2. Click the **Create** dropdown and select **New trait**
**Option 2:**
Select **Start with a template** to use a predefined calculation
**Option 3:**
Create a custom trait while building an audience using the Custom traits filter group.
---
## Configure traits
### Step 1: Method
In the **New trait** flow, the first step is choosing the logic type.
1. Select a model. Typically this will be your parent model, but you can also create traits on a related or event model.
2. Select the calculation method.
**Available calculation methods:**
* **Aggregation:** Sum or average across rows
* **Count:** Total number of matching events
* **Occurrence:** First, last, most frequent, or least frequent value
* **List:** Return an array of values or objects
* **SQL Aggregation:** SQL for grouping/array logic
* **SQL Formula:** SQL expressions based on parent properties
---
### Step 2: Calculation
In Step 2, you'll configure the logic for your selected calculation method. The fields shown depend on the method you chose in Step 1 (e.g., Aggregation, Count, Occurrence, SQL, etc.).
For most methods, you’ll need to:
* **Select a related or event model**
Choose the dataset you're calculating across (e.g., Purchases, Sessions, Page Views)
* **Add optional filters**
Filter the input records before applying your calculation (e.g., REVENUE > 500, event_type = "checkout")
* **Complete [method-specific settings](https://hightouch.com/docs/customer-studio/traits#method-specific-settings)**
Filters can include field-based logic (e.g., REVENUE > 500) and will be applied before aggregation.
Use the **Preview results** button to validate before moving to the final step.
---
### Method specific settings
#### Aggregation traits (Sum, Average)
**Use case:** Compute the sum or average of a numeric column across related or event data.
You’ll configure:
- **Related or event model** (e.g. `Purchases`)
- **Filter conditions** (optional)
- **Aggregation type**: `Sum` or `Average`
- **Column to aggregate** (e.g. `REVENUE`)
#### Count traits
**Use case:** Count the number of records (e.g. purchases, sessions, clicks) that meet a condition.
You’ll configure:
- **Related or event model**
- **Optional filters**
- **Count by**: Select a column (optional — defaults to row count)
#### Occurrence traits (First, Last, Most frequent)
**Use case:** Return the first, last, or most frequent value of a column from a related model.
You’ll configure:
- **Related or event model**
- **Optional filters**
- **Occurrence type**: `First / Min`, `Last / Max`, `Most frequent`, or `Least frequent`
- **Trait value**: Column to return (e.g. `product_name`)
- **Order by**: Determines ordering for first/last
#### List traits
Contact your Customer Success team to enable returning list of objects. By default, only list of values are supported.
**Use case:** Return an array of values or objects for use in downstream tools (e.g. product IDs, brand names).
You can configure:
- **Column to list**
- Select one column (e.g. `product_id`) to return a list of values
- Select multiple columns to return a list of objects. The object key is the column alias; change the alias to control the output key name in the object.
- **Optional filters** to control which rows are included
- **Deduplication toggle**: Check to skip duplicate values
- **Order by** and **Sort direction**
- **Optional slice controls**: Optionally, choose which part of the list to return (e.g. first 3)
**Examples:**
- Value list: `["product_1", "product_2", "product_3"]`
- Object list: `[{"id": "abc", "expiration": "2024-10-01"}, {"id": "xyz", "expiration": "2024-11-15"}]`
> **Note:** For value lists, if using "Deduplication", "Order by" must match the selected column. For object lists, "Order by" and "Deduplication" cannot be used together.
#### SQL Aggregation traits
**Use case:** Build advanced calculations using SQL across related models.
You’ll configure:
- **SQL expression**, using `{{ column "" }}` syntax
- **Default value** for null or missing results
- **Property type** (e.g. `String`, `Number`, `Boolean`)
#### SQL Formula traits
**Use case:** Compute logic based on fields from the trait model itself using SQL. Does not join to related models.
You’ll configure:
- **Formula expression** (no `{{ }}` syntax—just reference columns directly)
- **Property type**
> **Note:** In SQL Formula traits, use direct field names (e.g. `ltv`, `email_verified`) instead of `{{ column }}` syntax.
---
### Step 3: Finalize
1. Give your trait a clear name and optional description. This name will appear when filtering in the audience builder.
2. Review the summary shown in plain language.
3. Click **Create trait** to finish.
The new trait is now associated with the trait model and can be used in audiences or, if the trait model is a parent model, synced to destinations.
---
## View and manage traits
After creation, traits appear in the **Traits table**, where you can:
* View name, the model the trait belongs to, calculation type, and the model used to calculate the trait
* Edit or archive traits
Use the tabs to switch between:
* **Traits:** All saved traits
* **Templates:** Reusable blueprints
* **Archived:** Deactivated traits
---
## Trait configuration and usage
Click into any trait to view:
* The **Configuration** tab: shows logic summary and allows edits
* The **Usage** tab: lists audiences where this trait is used
---
## Trait templates
Trait templates let you define reusable logic that can be used as a base for other traits. You can create templates once and generate new traits from them, saving time and ensuring consistency across teams.
### Create a new template
To create a template:
1. Go to **Customer Studio → [Traits](https://app.hightouch.com/traits)**
2. Click the **Create** dropdown and select **New template**
The flow for creating a template is the same as creating a standard trait:
* Choose a trait model
* Select a calculation method
* Configure filters and logic
* Finalize the template by naming and saving it
Once saved, the template appears under the **Templates** tab.
Updating a trait template will also update all traits created from it—unless those traits have overridden the base logic. For example, if you change a template’s aggregation type from `Sum` to `Count`, any traits built directly from that template will adopt the new `Count` method automatically.
### Create a trait from a template
Once you’ve created a template, you can use it to quickly generate new traits with consistent logic.
To create a trait from a template:
1. Go to **Customer Studio → [Traits](https://app.hightouch.com/traits)**
2. Click the **Create** dropdown and select **Start with a template**
3. In the modal, select a template and click **Choose template**.
3. In the **Choose a template modal**, select your desired template from the list.
* You’ll see a summary of the calculation logic, including parent and related models.
* Click **Preview results** to validate the output.
* Click **Choose this template** to continue.
4. On the next screen, you'll see the template logic pre-populated.
* (Optional) Add additional filter conditions.
* Enter a **Name** and optional **Description**.
* Click **Preview results** to validate trait output.
* Click **Save** to finalize.
Once saved, this trait will be available in the Traits tab and can be used for audience filters and destination syncs.
Changes made to the base template will automatically update all traits created from it—unless local filters or logic were modified when the trait was saved.
---
## Create audiences with traits
Once you've created a trait, you can use it for filtering within your audiences. Traits filters are under **Traits** in the filters dropdown.
There are three kinds of traits you can use in your audience filter:
- Trait templates
- Traits
- Custom traits
### Use trait templates in the audience
1. Click **Add filter**
2. Navigate to **Traits** and select a trait template.
For example, if you created an LTV trait, you can use it to filter for "All users that have spent more than $100."
3. [Optional] You can also add filters to apply before Hightouch calculates trait values. For example, you can restrict the LTV calculation to purchases where the `Brand` was Nike directly in the audience builder.
4. [Optional] You can save the trait filter as a new [trait enrichment](#enrich-audiences-with-traits). To do so, click the actions button and "Save trait enrichment"
### Use traits in the audience
1. Click **Add filter**
2. Navigate to **Custom traits** and select a trait
3. [Optional] You can edit the trait configuration by clicking **Edit**. Updating these filters will not affect the original trait.
4. [Optional] If you edit the trait and want to save it as a new trait, click the actions button and "Save trait."
### Create a custom trait within the audience
Maybe you don't have a suitable trait or trait template to act as a filter. Or, maybe you just want to experiment with a trait filter without necessarily creating one for the parent model.
You can create a **custom trait** inline within the audience query builder.
1. Click **Add filter**
2. Navigate to **Custom traits** and click **Create a custom trait**
3. Select a related model
4. Select the configuration for your trait
5. [Optional] If you decide that this custom trait is useful enough to persist it for the entire parent model, click the actions button and "Save trait."
Doing so converts it into a new trait that can be synced.
---
## Enrich audiences with traits
Traits defined on the parent model are automatically available for syncing to destinations. Alternatively, you can create **trait enrichments** which are traits that are specific to an audience.
Trait enrichments are based on a parent trait which are the trait templates you've defined on the parent model.
1. Go to the **Traits** tab of the audience you want to add enrichments to.
2. Click **Create trait**.
3. Enter a **Name** and select the **Parent trait**.
4. [Optional] Add conditions to apply on top of the parent trait.
Once created, these enrichment traits appear in the Audience preview and sync configuration like any other field.
---
## Event trait enrichments
You can also create traits from events in the audience builder. These automatically become [trait enrichments](#enrich-audiences-with-traits), meaning you can view them in the preview and sync their values. One common use case for event traits is to capture information about abandoned carts.
1. To create a new event trait enrichment, click **Add trait** on an event condition in the audience builder.
You can only access the **Add trait** option from an event filter once you've
saved the audience.
2. Give the event trait enrichment a descriptive **Name**.
3. Select the aggregation you want to use, for example, **Last**.
4. Select the information you want to capture about the event. For example, you may want to capture the `Product_ID`, `SKU`, or `Brand` of the final product a user added to their cart before abandoning it.
---
## SQL Aggregation traits
Custom SQL traits provide a powerful escape hatch for custom aggregations that the [default aggregations](#trait-types) don't cover. To build a SQL trait, [create a trait](#create-a-trait) and follow these additional instructions:
1. Select **SQL** as the **Aggregation** type.
2. Enter a raw SQL aggregation. For example, to create a `SUM` trait with just SQL, you would enter `SUM({{ column "price" }})`.
**Column references** must use the following syntax `{{column ""}}`. Column names are case-sensitive.
3. Enter a default value for when the SQL returns no rows.
4. Define the **Property type** of the SQL aggregation result.
### JSON aggregations
JSON aggregations are a helpful way to aggregate the raw data in a related model. For example, imagine you want to get a list of all product IDs users have purchased. You can do this with the following SQL trait:
```
ARRAY_AGG(DISTINCT {{column "product_id"}})
```
### CASE statements
`CASE` statements are a useful way to create friendly options for aggregations. For example, imagine you want to bucket customers into `low`, `medium`, and `high` LTV. You can do this with the following SQL trait:
```
CASE
WHEN SUM( {{column "price"}} ) > 100 THEN 'high'
WHEN SUM( {{column "price"}} ) > 50 THEN 'medium'
ELSE 'low'
END
```
---
## SQL Formula traits
Formula traits allow you to reference the columns, merged columns, and other traits belonging to the trait model to build inline transformations. Unlike Custom SQL Traits, there is no aggregation against a single related model. Instead, you can reference columns from a multitude of sources.
This unlocks a powerful and flexible way to compose columns all within Hightouch.
1. Select **Formula** as the **Aggregation** type.
2. Enter raw SQL. To reference a column or trait, type the name for the autocompletion dropdown to show up. Then, select it to populate the editor as shown above.
Here are a few examples of SQL queries that can be done using formula traits:
- Combine multiple columns into one: `CONCAT({{"first_name"}}, ' ', {{"last_name"}})`.
- Use a string column to create a boolean flag: `CASE WHEN {{"merged.pets.animal"}} LIKE 'dog' THEN true ELSE false END`
- Reference other traits: `CASE WHEN {{"trait.lifetime_total_value"}} > 1000 THEN 'high' ELSE 'low' END`.
**Column references** in formula traits use special syntax. Please use the SQL editor's autocompletion feature to write the column references.
References to non-existent columns or traits will be shown with a red pill.
3. Define the **Property type** of the SQL aggregation result.
## Trait value suggestions
**Most Frequent**, **First**, **Last**, [**Custom SQL**](#custom-sql-traits), and [**Formula**](#formula-traits) traits that result in a string type support trait value suggestions. When enabled this provides **suggested values** for the visual audience builder. For example, if you're creating an audience you want to filter on a "Most Recent City" trait, turning on suggestions populates a dropdown with values—San Francisco, Los Angeles, etc.—found in the dataset.
### Enabling suggestions
Trait value suggestions can be enabled when creating or viewing a trait.
## Traits on event or related models
You can define traits on related or event models (e.g. Purchase, Session) as well as the parent model (e.g. User). This enables traits that compute within the scope of a related row and can be referenced in audience filters.
**Example use cases**
* Per purchase, how many items over 1K do I have? Filter for users with purchases that include 3 items over $1000.
* I only have a timestamp colmn on my event. How many events occurred on the weekend vs the weekday? Filter for users with purchases that occured on a weekend.
**Key notes**
* Traits "live" on the event or related model: the event or related model is the trait model, and the trait is calculated relative to it.
* In non-formula traits you can aggregate across models related to trait models. In formula traits, you can use the trait model's columns directly in the formula.
* You can’t sync these traits directly. They can only be filtered.
* Preview rows show the join key and, for events, timestamp and primary key if available.
---
## Create audiences
**URL:** https://hightouch.com/docs/customer-studio/usage
**Description:** Build and activate customer segments using Hightouch's visual audience builder.
**Section:** Customer Studio
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Marketers |
| **Prerequisites** | Completed [schema setup](/customer-studio/schema) by your data team |
*Audiences define the group of users or accounts you want to target in your campaigns. Using the visual audience builder, marketers can filter, preview, and activate audiences without writing SQL.*
---
### Learning objectives
After reading this article, you’ll know how to:
* Create a new audience in Customer Studio
* Apply filters to define and refine audience membership
* Preview and validate audience results
* Limit and control audience size
* Analyze and reuse audiences
* Sync audiences to destinations
---
## Overview
The **audience builder** lets you visually define and activate segments of users or accounts. It’s designed for marketers to build precise audiences using data from your warehouse, preview the results, and activate them directly in campaigns.
You can filter audiences based on properties, events, relationships, and traits, then preview and activate them directly in your marketing and advertising tools.
Want to skip manual filter configuration? Use the [**AI Agent**](/customer-studio/agents) to build and analyze audiences using natural language. Describe the segment you need, and the Agent creates the audience logic for you.
Once created, audiences can be:
- Analyzed in [Insights](/customer-studio/insights)
- Used in [Journeys](/customer-studio/journeys)
- [Split](/customer-studio/splits) for experiments
- [Synced](/customer-studio/syncs) to destinations
Audiences are built from [models](/customer-studio/schema#schema-setup) configured by your data team. These models define the source data you’ll use to segment users, accounts, or other entities.
---
## Audience types (batch vs real-time)
Hightouch supports both **batch** and **real-time** audience evaluation.
| Type | Description | Example use |
|-------|-------------|-------------|
| **Batch** | Evaluated on a schedule or manually refreshed. | Weekly “High-LTV Customers” campaign |
| **Real-time** | Evaluated continuously as new events arrive. | “Users who searched flights in the last 10 minutes” |
Use **batch audiences** when you need stable segments that update on a predictable cadence, such as lifecycle or nurture campaigns.
Use [**real-time audiences**](/real-time/real-time) when you want to respond instantly to user behavior like triggering an ad, notification, or personalization within seconds of an event.
---
## Create and refine audiences (core workflow)
### Step 1: Create a new audience
1. Go to **Customer Studio → [Audiences](https://app.hightouch.com/audiences)**.
2. Click **`Add Audience`**.
3. Select a [**Parent Model**](../schema#1-define-the-parent-model) to segment (for example, `Users`, `Accounts`, or `Devices`).
4. Name your audience (for example, *High-Value Customers*) and add an optional description.
---
### Step 2: Apply filters
Start from an [audience template](/customer-studio/templates) or create filters from scratch.
| Filter type | Use it to filter by… | Example filters |
|--------------|----------------------|------------------|
| **Properties** | Columns on the parent model | `email contains "@gmail.com"` |
| **Relations** | Linked records (purchases, accounts) | `has Purchase where amount > 100` |
| **Events** | Actions users performed | `Product Viewed within last 30 days` |
| **Audiences** | Membership in another audience | `is in VIP Members` |
| **Journeys** | Participation in a journey | `currently in Abandoned Cart Journey` |
| **Traits** | Custom or computed fields | `customer_score > 80` |
If you find yourself recreating the same logic across multiple audiences, convert that logic into a reusable [trait](/customer-studio/traits). Traits help maintain consistent segmentation.
#### Combine filters with AND/OR logic
Use **AND**, **OR**, and nested groups to express complex logic.
**Example:**
Include users who made a purchase **or** added to cart, **and** belong to the Gold tier.
| Group | Condition |
|--------|------------|
| (A) | `has Purchase where amount > 100` OR `added to cart within last 7 days` |
| (B) | `loyalty_tier equals "Gold"` |
| Audience logic | (A) AND (B) |
Unexpectedly small results often indicate a one-to-many relationship mismatch or overlapping event time windows. Check your schema relationships and event time filters.
---
### Step 3: Preview and validate your audience
After defining filters, preview your audience to confirm that your logic returns the expected members.
- Click **`Calculate size`** to estimate audience size.
- Click the member count (for example, `110 members`) to preview sample records.
Use [Sampling](/customer-studio/sampling) to test filters and preview large datasets without querying your entire warehouse.
#### Limit audience size
Limit audience size to focus on your highest-value members or to run controlled experiments on an audience subset.
1. Click the **`Limit audience size`** icon.
2. Choose **Top** or **Bottom**, specify size, and select a sort column (for example, `purchase_amount` or `last_purchase_date`).
You can also limit by **percentile** (for example, top 25th percentile of spend). The percentile option appears only for numeric columns without value suggestions enabled.
#### Preview with destination rules
If your workspace uses [destination rules](/customer-studio/destination-rules), you can select a destination to preview how the audience will sync before saving.
---
### Step 4: Analyze your audience
Use **[Insights](/customer-studio/insights)** to analyze composition and overlap.
- Click **`Show Insights`** to open the panel.
- **Overlap** compares shared members with another audience.
- **Breakdown** displays distribution by trait, geography, or behavior.
---
### Step 5: Save and reuse audiences
Click **`Save audience`** to finalize your setup.
#### Reuse audiences with templates
To create repeatable logic, apply an **[audience template](/customer-studio/templates)** directly in the builder.
Templates let you standardize approved audience definitions for faster setup.
#### Clone audiences
To duplicate an existing audience, open the overflow menu and click **`Clone`**.
You can modify filters, rename the copy, and choose which syncs to duplicate.
---
### Step 6: Sync to destinations
After saving, activate your audience by syncing to tools such as CRMs, ESPs, or ad platforms.
Learn more in [Sync audiences →](/customer-studio/syncs)
Run a test sync to a sandbox destination before enabling production syncs. This helps validate column mappings and audience behavior without affecting live campaigns.
---
## Advanced filtering and configuration
Use these optional tools to create more sophisticated or large-scale audiences.
### Filter using related / event models
If your data includes relationships (for example, `Users → Orders`), you can build filters across connected models. This lets you target users based on behaviors or attributes in other tables.
**Example**
"Target users who placed an order worth more than $100."
If related models or expected fields are missing, your data team may need to adjust [schema relationships](/customer-studio/schema#relationships-and-cardinality) or [merge columns](/customer-studio/schema#merge-columns).
---
### Filter using nested related / event models
#### Overview
Nested filtering lets you filter audiences using data across multiple 1:many relationships (e.g. `User → Purchases → Items`). This allows you to express logic like:
"Target users who have a purchase in the past 30 days where that purchase includes an item in the 'Skincare' category."
Previously, filters could only natively look one relationship deep. Now you can traverse **up to three levels** of related or event models to capture richer, real-world relationships.
#### When to Use It
Use nested filtering when you need to build conditions on data that is in a downstream event or related model. Nested filtering is especially useful when you want to combine conditions within sub-models with conditions on the current model.
#### Example
**Schema:** `User → Purchases → Items`
**Goal:** Find users who have made a purchase in the last 30 days that included a 'Toy' item.
#### How It Works
Each nested block evaluates conditions within its own level.
* The outer block ( `Has ≥1 CompletedCheckout`) evaluates purchases belonging to the user.
* The inner block (`PurchaseItems include item\_type \= 'Toy'`) evaluates items belonging to each purchase.
In this example, a user qualifies if **at least one** purchase matches the nested condition. You can also use "Number of" filters at both levels (e.g., "≥3 purchases where for each such purchase, ≥2 items match").
This is different from universal logic (e.g., finding users where all purchases included a 'Toy'). Universal logic queries can be performed via [traits](https://hightouch.com/docs/customer-studio/traits).
#### Limitations
* Only supports traversal up to **three levels deep** from the parent model.
Nested filtering is an advanced feature intended for users needing to access multiple levels of models. Contact your Customer Success team to enable nested filtering for your workspace.
---
### Filter nested JSON data
Some columns contain nested JSON objects or arrays (type `unknown JSON`).
You can filter inside these structures using **dot-path notation** to access specific fields.
For example:
- `booking.product.geo.city` filters by a nested property inside an object.
- `booking.0` filters by the first item in an array.
This is useful when event data or object payloads store multiple attributes within a single column.
JSON filtering is supported on **parent**, **event**, and **related** models.
**Example JSON**
```json
{"inventory":{"on_order":50,"stock":750,"warehouse":"WHB"}, "dimensions":[9, 8, 7]}
{"inventory":{"on_order":10,"stock":120,"warehouse":"WHA"}, "dimensions":[5, 4, 3]}
```
If the column is on a parent model:
If the column is on an event model:
**How to filter**
Use dot-path notation to specify the exact field you want to filter on. Hightouch supports both object and array traversal for nested JSON fields.
By default, JSON columns are treated as type unknown with the exists operator. You can manually set the data type—string, number, or boolean—to unlock the operators that make sense for that field.
All standard operators supported in the audience builder (for example, equals, contains, greater than, exists) are available, but only if they’re compatible with the selected type.
**Preview**
Depending on your dataset size and how recent your events are (if filtering event data), you may use the [member preview](https://hightouch.com/docs/customer-studio/usage#validate-audience-size-and-results) to confirm that your JSON paths and filters return the expected results.
JSON filtering is an advanced feature intended for users who understand their data structure. If you’re unsure which JSON fields to target, ask your data team for help or contact your Customer Success team to enable JSON column filtering for your workspace.
---
### Drag and drop values
For certain [textual](#textual-operators) and [numerical](#numerical-operators) operators, you can upload a list of values directly into your filter using **drag and drop** instead of entering them manually.
This feature saves time when working with long lists, such as customer IDs, organization IDs, or email addresses, so you don’t have to copy and paste each value.
You can drag and drop `.txt` or `.csv` files containing **comma-separated values (CSV)** directly into the filter’s value field.
**Example use case**
- You have a CSV file with hundreds of organization IDs.
- Instead of entering each ID manually, drag and drop the file into the input box of a property filter.
- Hightouch automatically uploads and parses the list of values for you.
**Supported operators**
| **Data type** | **Supported operators** |
|----------------|--------------------------|
| **Textual** | contains · does not contain · equals · does not equal |
| **Numerical** | equals · does not equal |
---
### Operator reference
Each filter condition in the audience builder uses a set of **comparison operators** to determine whether records meet your criteria.
The available operators depend on the **data type** (text, number, Boolean, or timestamp).
Use this table to confirm which operators are available for each data type.
| **Data type** | **Supported operators** |
|----------------|-------------------------|
| **Textual** | equals · does not equal · contains · does not contain · starts with · ends with · exists · does not exist |
| **Numerical** | equals · does not equal · greater than · less than · greater than or equal to · less than or equal to · exists · does not exist |
| **Boolean** | equals · does not equal · exists · does not exist |
| **Timestamp** | equals · before · after · within · not within · between · anniversary · exists · does not exist |
The **anniversary** operator checks whether a date falls on an anniversary of the current day—useful for birthdays, renewals, or membership milestones.
#### Textual operators
When filtering on textual data (for example, product brand, location, or email domain), you can use:
- equals
- does not equal
- contains
- does not contain
- starts with
- ends with
- exists
- does not exist
**Behavior details**
The **contains** operator performs a fuzzy match—it translates to the SQL `LIKE` operator with the value wrapped in `% %`.
The **equals** operator looks for an exact match (also using `LIKE`, but without surrounding wildcards).
#### Numerical operators
When filtering on numerical data (for example, price, score, or lifetime value), you can use:
- equals
- does not equal
- greater than
- less than
- greater than or equal to
- less than or equal to
- exists
- does not exist
You can use the [drag and drop](#drag-and-drop-values) feature with the following operators: **equals** and **does not equal.**
When using **greater than**, **less than**, **greater than or equal to**, or **less than or equal to**, you can enter:
- an **absolute numerical value** (for example, `purchase_amount > 100`), or
- a **percentile** (for example, “top 50th percentile of lifetime value”).
To use the **percentile** option, make sure the column has **value suggestions turned off** in the [schema configuration](/customer-studio/schema#columns).
Percentile filtering is helpful when audience size, rather than a fixed number, is your targeting driver (for example, “top 10% of users by spend”).
#### Timestamp operators
When filtering on timestamps (for example, purchase date or last activity), you can use:
- equals
- before
- after
- within
- not within
- between
- anniversary
- exists
- does not exist
You can choose between **rolling** or **calendar** time periods for relative filters.
**Examples:**
- To find users active in the last 60 days → use “within previous 2 months” (rolling window).
- To find users active during a specific range → use “within previous 2 calendar months.”
If today is June 3, this filters for dates between April 1 and May 31.
If a column contains timezone data, that timezone is used when calculating date and time values. If not, Hightouch uses the warehouse’s default timezone for both stored values and current time calculations.
Operators vary by data type. If you don’t see an operator available for a specific column, check the column’s configured data type in your [schema setup](/customer-studio/schema#columns). For example, `string` types support text-based filters, while `number` types unlock percentile and inequality operators.
---
### Configure null-value behavior
When building filters, you may encounter `null` data (for example, missing country values).
Admins can control how nulls behave for **“not”** operators in [Workspace Settings](https://app.hightouch.com/settings/workspace).
**Options:**
- **Treat null values like any other value (default):**
`Country does not equal USA` → includes `Country = null`
- **Use special behavior for null values:**
`Country does not equal USA` → excludes `Country = null`
---
## Experimentation and governance
### Split audiences for experiments
Divide an audience into random or rule-based groups for testing.
| Split type | Description |
|-------------|-------------|
| **A/B Split** | Divide members randomly between two syncs. |
| **Multi-branch Split** | Route members to multiple destinations. |
Each profile is assigned once, based on its primary key.
Changing split percentages can reassign users—use a [snapshot](/customer-studio/audience-snapshots) to preserve consistency.
---
### How audiences stay up to date
Audiences update automatically as your source data changes so campaigns always reach the right people.
| Behavior | Description |
|-----------|-------------|
| **Automatic updates** | New data or events trigger re-evaluation. |
| **Manual refresh** | Trigger a full recalculation from the overflow menu. |
| **Snapshots** | Freeze membership at a point in time to measure lift or maintain fixed test groups. |
---
### Segment by households or accounts
In some workspaces, audience data is organized around more than one entity type, such as: **People**, **Households**, or **Accounts**. When you enable [schema labeling](/customer-studio/schema#schema-labels), the audience builder automatically reflects these relationships.
This allows you to create audiences that consider shared characteristics or behaviors across related entities, rather than limiting segmentation to individuals.
With schema labels applied, filters adjust automatically so you can build audiences using data from connected entities. For instance, you can define a **People** audience that includes or excludes members based on traits of their associated **Household** or **Account**.
**Examples:**
- **Household segmentation:** Include people in households that meet a shared condition (for example, at least one member identifies as female or holds an active subscription).
- **Account segmentation:** Include people whose associated account shows recent activity or engagement (for example, a recent purchase or contract renewal).
Schema labeling enables you to build audiences using both individual and group-level data.
This supports coordinated targeting and measurement when multiple related entities—like people within households or employees within accounts—affect campaign outcomes.
---
## Example use cases
### Abandoned checkout retargeting
**Goal:** Retarget users who started checkout but didn’t complete.
**Filters**
- Event: `Checkout started` within last 24 hours
- Event: `Order completed` not performed in same window
- Trait: `LTV > 200`
- Property: `Email is not null`
**Sync to**
- Meta Custom Audiences
- Klaviyo List
Combine this audience with a [subset](/customer-studio/subsets) or [destination rule](/customer-studio/destination-rules) to exclude opt-outs.
---
### Real-time travel personalization
**Goal:** Personalize offers for users who searched flights or hotels in the last 24 hours.
**Filters**
- Event: `search` where `category = flight OR hotel`
- Event: `search_time` performed within 24 hours
- Trait: `loyalty_status = Gold`
Enable **real-time evaluation**, split 80/20 for testing, and sync to ad platforms.
Learn more in [Same-session audiences →](/real-time/real-time)
---
## Best practices
| Goal | Recommendation |
|------|----------------|
| **Ensure consistent segmentation** | Build complex logic once as reusable [traits](/customer-studio/traits). |
| **Avoid audience drift** | Use [snapshots](/customer-studio/audience-snapshots) to lock membership for control groups. |
| **Simplify experimentation** | Keep [split](/customer-studio/splits) audiences clearly named (for example, `retargeting_a` and `retargeting_b`). |
| **Test before activation** | Use [sampling](/customer-studio/sampling) to preview results on smaller datasets. |
---
## Troubleshooting
| Issue | Likely cause | Resolution |
|--------|---------------|------------|
| Audience size changes unexpectedly | Real-time evaluation enabled | Use snapshots to lock membership |
| Filters return no results | Wrong relationship or null handling | Check schema configuration and null settings |
| Members appear in both split groups | Split percentages edited after launch | Avoid editing live splits |
| Cannot assign sync | Missing permissions | Contact your workspace admin |
If you see warehouse SQL errors (for example, unresolved columns, invalid identifiers, or object authorization failures), use [Resolve SQL compilation errors](/models/sql-compilation-errors).
### Common issues and detailed resolutions
**Filter value suggestions not appearing**
Ensure you’ve [enabled column suggestions](/customer-studio/schema#columns).
If suggestions are enabled but missing, refresh them in the filter UI.
**Error: column "…" does not exist in _ht_with_column_casts**
This occurs when a model column was deleted but cached in Hightouch.
Preview and re-save the model configuration to refresh cached columns.
**Error: [UNRESOLVED_COLUMN.WITH_SUGGESTION] … cannot be resolved**
This error means a filter or audience condition references a column that is not present in the model output (for example, `segment_value`), often due to a renamed/removed column or stale filter configuration.
To resolve it:
1. Open the parent model and confirm the referenced column exists in the selected columns.
2. Preview and re-save the model to refresh available columns in Customer Studio.
3. Update audience filters that reference the missing column, or add the column back to the model.
4. Re-run audience preview to confirm the error is gone.
If audience sizes or syncs change unexpectedly, review your schema relationships and sync schedules regularly to ensure alignment.
---
## What’s next?
After creating audiences, you can activate and measure them using these related features:
* [Sync audiences to destinations →](/customer-studio/syncs)
* [Build and analyze audiences with the AI Agent →](/customer-studio/agents)
* [Add traits for filtering →](/customer-studio/traits)
* [Orchestrate campaigns with journeys →](/customer-studio/journeys)
* [Reuse audiences with templates →](/customer-studio/templates)
---
## Agent alerting
**URL:** https://hightouch.com/docs/ai-decisioning/agent-alerting
**Description:** Set up automated notifications for agent run failures and recommendation volume drops
**Section:** AI Decisioning
| | |
| ----------------- | ------------------------------------------------------------------------------------------------ |
| **Audience** | Marketers, data teams |
| **Prerequisites** |
A running [agent](/ai-decisioning/agents)
|
*Agent alerting notifies you about engine run failures, long-running or stalled agents, and unexpected drops in recommendation volume before they impact your campaigns.*
---
### What you'll learn
After reading this article, you'll know how to:
- [Understand alert triggers](#alert-triggers)
- [Configure alert severity](#alert-severity)
- [Choose notification channels](#notification-channels)
- [Set up alerting on an agent](#set-up-alerting)
---
## Overview
AI Decisioning agents support automated alerting for engine runs — the daily runs that generate and schedule AID messages for your users. The alerting system for AID agents follows the same model used for [syncs](/syncs/alerting) and journeys in Hightouch.
Every agent comes pre-configured with a default run failure monitor. No setup is required for monitoring to begin.
To access alerting, open an agent and go to the **Alerting** tab.
---
## Alert triggers
There are four alert triggers available for AID agents:
**Agent run failure:** Triggers when one or more consecutive engine runs fail. Each engine run is responsible for generating and scheduling the AID messages sent to your users, so a failure means messages may not be produced or delivered for that cycle. The alert fires regardless of which pipeline stage caused the failure and links directly to the specific failed run, taking you to the **View details** page on the [Health tab](/ai-decisioning/agent-health#view-details).
From View details, you can see each pipeline stage and its status, which stage the run failed at, and an error message explaining what went wrong. For more information, see [Agent health → View details](/ai-decisioning/agent-health#view-details).
**Agent run duration:** Triggers when an agent has been running for longer than a threshold you define (in minutes). Use this to detect runs that are taking unusually long, which may indicate a warehouse performance issue or a bottleneck in the pipeline.
**Agent idle time:** Triggers when no new agent run has started within a specified number of minutes. Use this to detect situations where your agent has stopped running entirely, which could mean a scheduling issue or an upstream dependency failure.
**Low number of recommendations generated:** Triggers when the number of interactions (messages) generated in an engine run falls below a threshold you define. Use this to catch unexpected drops in decision volume that may not surface as an outright failure but could signal an upstream data issue or audience eligibility problem.
---
## Alert severity
Each trigger can be configured with a severity level:
- **Critical:** For high-urgency issues that require immediate attention, such as a run failure.
- **Warning:** For lower-urgency signals, such as a drop in recommendation volume that may warrant investigation but is not blocking.
- **Custom:** Set separate thresholds for Critical and Warning severity on the same trigger. For example, fire a Warning if fewer than 500 interactions are generated and a Critical if fewer than 100.
---
## Notification channels
Alerts can be routed to the following destination types: Slack, Email, PagerDuty, SMS, and Webhook.
Each recipient can be scoped to receive only Critical alerts or all alerts.
---
## Set up alerting
1. Go to any AID agent and select the **Alerting** tab.
2. Review the pre-configured **Agent run failure** monitor. Adjust the severity or switch to a Custom threshold if needed.
3. Click the **+** button under **Recipients** to add a notification channel (Slack, Email, PagerDuty, SMS, or Webhook).
4. Optionally, configure additional triggers such as **Agent run duration**, **Agent idle time**, or **Low number of recommendations generated** with your preferred thresholds and severity.
5. Select **Save changes**. Alerts fire on the next qualifying run.
---
## What's next
- [Agent health: Monitor pipeline runs and delivery rates →](/ai-decisioning/agent-health)
- [Insights: Measure agent and message performance →](/ai-decisioning/insights)
- [Agents: Configure goals, audiences, and scheduling →](/ai-decisioning/agents)
---
## Agent health
**URL:** https://hightouch.com/docs/ai-decisioning/agent-health
**Description:** Monitor agent pipeline runs, channel delivery rates, and outcome data freshness
**Section:** AI Decisioning
| | |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers, data teams |
| **Prerequisites** |
A running [agent](/ai-decisioning/agents)
[Send event tracking](/ai-decisioning/configuration#send-event-tracking) configured for channel delivery metrics
|
*The Health tab gives you real-time visibility into your agent's pipeline runs, channel delivery rates, and the freshness of the outcome data powering optimization.*
---
### What you'll learn
After reading this article, you'll know how to:
- [Monitor message generation runs](#message-generation)
- [Diagnose pipeline failures](#view-details)
- [Track channel delivery success rates](#channel-send-success)
- [Troubleshoot low delivery rates](#troubleshoot-low-delivery-rates)
- [Check outcome data freshness](#outcome-health)
---
## Overview
The `Health` tab provides operational visibility into your agent from a single view. No additional setup is required — the tab automatically pulls data from your existing agent configuration, runs, and event models.
To access it, open an agent and go to the **Health** tab.
---
## Message generation
The **Message generation** section shows the current and historical state of your agent's daily runs — the runs that schedule and produce AID messages.
**Current run:** If a generation is in progress, you can see when it started, how long it has been running, and its current pipeline step.
**Last run:** Shows when the most recent generation completed, how long it took, and how many recommendations were produced.
**Historical runs:** A table of recent runs with start time, duration, run status, and recommendation count.
**Failed runs:** If a run encountered an error, it is flagged in red. Click **View details** to see per-stage durations and actionable error messages.
### View details
Click **View details** on any run to open a per-stage pipeline breakdown. This view is useful for diagnosing run failures, identifying bottlenecks, and understanding where time is spent across the generation process.
Each run includes the following pipeline phases:
- **Cache Interactions** — Loads recent interaction data
- **Assign User Groups** — Assigns users to treatment or holdout groups
- **Pull Warehouse** — Retrieves data from your warehouse
- **Generate Interactions** — Produces message recommendations for eligible users
- **Write Warehouse** — Writes results back to your warehouse
- **Compact Interactions** — Cleans up interaction records
- **Trigger Analytics** — Updates reporting and analytics data
Each phase shows its duration and status (`Completed`, `Pending`, or `Failed`). Stages that haven't started yet show as `Pending`.
When a run fails, the error message includes guidance on what went wrong and what you can do to fix it. Errors fall into two categories:
- **External issues (actionable by you):** Problems originating from your data or configuration. For example, a column may have been dropped or renamed in your warehouse while your parent model, user feature matrix, audience, or event goals still reference it. Update the affected configuration and re-run the agent.
- **Internal issues:** Problems on Hightouch's side. Hightouch receives internal alerting for all agent run failures and proactively investigates. You can also reach out to your Hightouch contact for support.
If a run fails at **Pull Warehouse**, check that your data warehouse is running and accessible, and verify that your connection credentials in Hightouch are correct.
A common cause of run failures is a column being dropped or renamed in your warehouse while your audience, feature matrix, event models, or parent model still reference it. If you see a missing column error, check each of those configurations for stale references, update them to reflect the current schema, and re-run the agent.
---
## Channel send success
The **Channel send success** section shows how successfully your agent's messages are being delivered to users, broken down by channel (Email, SMS, Push, and others).
- **Delivery success rate:** The percentage of scheduled messages confirmed sent by your ESP, based on the send event model configured for each channel.
- **Raw counts on hover:** Hover over the percentage to see exact sent versus scheduled message counts (for example, "34,189,852 sent / 36,293,161 scheduled").
- **Destination:** Shows which integration destination is handling delivery for that channel.
- **Time filter:** Use the dropdown in the top-right corner to filter delivery success by **All time**, **Last 7 days**, **Last 30 days**, or **Last 90 days**.
If send success is not configured for a channel, click **Open send success insights** or go to [Insights → Send success](/ai-decisioning/insights) to configure it.
Send success tracking requires send events to be linked to your channels. See [Send event tracking](/ai-decisioning/configuration#send-event-tracking) for setup instructions.
### Troubleshoot low delivery rates
If you see unexpected delivery numbers, here are the most common causes:
- **0% with a warning ("Send event model may not be set up correctly"):** The send event model for that channel likely needs to be reviewed. Confirm that the event model tracking send confirmations from your ESP is correctly configured and scoped to the relevant business units or campaign identifiers for your AID campaigns.
- **Below 50% (highlighted in red):** This threshold flags channels that may have a misconfigured event model or a delivery issue worth investigating. This is particularly common during initial setup, when send tracking is first being validated.
---
## Outcome health
The **Outcome health** section tracks the freshness of the event data powering each of your agent's configured outcomes (goals). This helps you catch reverse ETL pipeline issues before they affect agent optimization.
- **Recent timestamp:** Outcome event data is flowing correctly.
- **Stale or missing timestamp ("--"):** Event data has not been loaded recently. Investigate whether the reverse ETL pipeline feeding that outcome has stalled or broken.
Each outcome's "Last data loaded" timestamp reflects when event data was most recently ingested, not when the agent last ran. If you see a stale timestamp, check your reverse ETL sync for that outcome before the next agent run.
---
## Troubleshooting
| Issue | Likely cause | Resolution |
|---|---|---|
| Channel send success shows 0% with a warning | The send event model for that channel is not configured correctly. | Confirm the event model tracking send confirmations from your ESP is correctly configured and scoped to the relevant business units or campaign identifiers for your AID campaigns. |
| Outcome health shows a stale or missing timestamp | The reverse ETL pipeline feeding that outcome has stalled or broken. | Check the reverse ETL sync for the affected outcome and resolve any pipeline issues before the next agent run. |
| Pipeline run fails at Pull Warehouse | Your data warehouse is unreachable or credentials are invalid. | Verify that your warehouse is running and accessible, and confirm your connection credentials in Hightouch are correct. |
| Pipeline run fails with a missing column error | A column was dropped or renamed in your warehouse while your parent model, audience, feature matrix, or event models still reference it. | Update the affected configurations to reflect the current schema, then re-run the agent. |
---
## What's next
- [Alerting: Get notified about run failures and volume drops →](/ai-decisioning/agent-alerting)
- [Insights: Measure agent and message performance →](/ai-decisioning/insights)
- [Agents: Configure goals, audiences, and scheduling →](/ai-decisioning/agents)
---
## Content Analysis & Suggestions
**URL:** https://hightouch.com/docs/ai-decisioning/content-analysis
**Description:** Analyze message structure, similarity, and performance signals to understand why content works and how to improve it.
**Section:** AI Decisioning
| | |
|----------------------|------------------------------------------------------------------------------------------------------|
| **Audience** | Marketers |
| **Prerequisites** |
Access to [AI Decisioning](/ai-decisioning/overview)
[Tags](/ai-decisioning/tags) configured for at least one message or variant.
|
*Content Analysis & Suggestions helps you understand how your messages relate to each other, why certain content performs, and where you can improve your creative library. It builds on your tag framework and provides insights to guide ongoing content planning.*
---
### Learning objectives
After reading this article, you’ll know how to:
* [Find Content Analysis and Content Suggestions](#where-to-access-content-analysis--suggestions)
* [Review message uniqueness before launch](#review-message-uniqueness-before-launch)
* [Interpret weekly Content Suggestions](#interpret-weekly-content-suggestions)
* [Ask follow-up questions using Hightouch Agents](#ask-follow-up-questions-with-hightouch-agents)
---
## Overview
Content Analysis & Suggestions helps you understand *why* some messages perform better than others. Instead of only seeing which message won, it highlights the elements—such as tone, structure, or value proposition—that influenced performance.
Inside each message, **Content Analysis** shows whether your content is unique before activating an agent. After your messages start running, the weekly **Content Suggestions** report in [**Insights**](/ai-decisioning/insights) helps you decide when to add new content, refresh existing messages, or retire content that’s no longer effective.
Together, these tools help you understand what’s working, avoid unintended duplication, and plan what to create next.
---
## Before you start
Complete these steps before using Content Analysis & Suggestions:
1. **Create a message.**
Build or update message content inside an agent.
See: [Add and manage messages →](/ai-decisioning/agents#add-and-manage-messages)
2. **Add or refine tags.**
Apply message and variant tags to describe key attributes such as tone, value proposition, or offer.
See: [AID Tags →](/ai-decisioning/tags)
### Content Analysis depends on your tags
Content Analysis uses your message and variant tags to understand tone, structure, and strategic intent.
Consistent tagging helps the system:
- Identify similar messages
- Detect overlap or redundancy
- Surface content gaps
- Provide more accurate recommendations
---
## How to use Content Analysis & Suggestions
Follow this workflow to evaluate and improve your content over time:
1. **Review Content Analysis before launch.**
In the message’s `Content` tab, check whether the message is unique and not unintentionally duplicating other messages in your library.
2. **Activate your agent.**
When your content looks distinct, enable the agent so your messages begin generating performance data.
3. **Review weekly Content Suggestions.**
In `Insights → Summary`, check for declining engagement, content gaps, or recommended improvements.
4. **Use Agents to explore insights.**
From the Summary tab, ask follow-up questions—such as “Why is this message similar to others?” or
“Which tones performed best this week?”—to understand results in plain language.
5. **Update or create new content.**
Refresh, retire, or expand content based on the insights and recommendations you discover.
---
## Review message uniqueness before launch
Content Analysis summarizes the creative attributes of a message and shows how it relates to other messages in your content library.
### View Content Analysis inside a message
Navigate to:
[`AI Decisioning → Agents`](https://app.hightouch.com/ai/flows/) `→ Select an agent → Messages → Select a message → Content`
You'll see:
### **Tag summary**
All manual and AI-generated tags applied to the message.
### **Similar messages**
Messages with similar tone, structure, framing, value proposition, or language patterns.
Expand each item to see why the messages match.
### **Differentiation indicators**
Signals that help you determine whether your content is:
- **Distinct** — introduces new strategic or creative value
- **Similar** — overlaps with existing content
- **Highly similar** — potentially redundant or interchangeable
Use Content Analysis before activating an agent to confirm each message adds something unique.
---
## Interpret weekly Content Suggestions
Weekly suggestions appear in the [**Summary**](https://hightouch.com/docs/ai-decisioning/insights#1-summary) view of Insights:
[`AI Decisioning → Insights → Summary`](https://app.hightouch.com/ai/insights/summary/)
You can review this report during weekly planning to decide what to adjust or create next.
The report includes:
### **Performance highlights**
Overall trends across themes, tones, or formats.
### **Content gaps**
Areas where your content library lacks variety.
### **Content to refresh**
Messages that are declining in engagement.
### **Recommended actions**
Suggestions such as:
- Refreshing visuals or copy
- Adding a new variant
- Retiring a low-differentiation message
- Expanding topics that perform well
Content Suggestions refresh automatically each week based on updated performance data.
---
## Ask follow-up questions with Hightouch Agents
From the Summary tab, select:
**Ask a follow-up question** or **Start a new chat about this report**
[**Agents**](/agents/overview) help you explore insights in plain language.
You can ask:
- “Why is this message similar to others?”
- “Which motivators performed best last month?”
- “Where are our biggest content gaps?”
- “What themes should we prioritize next?”
Learn more [**Agents overview →**](/agents/overview)
Use Agents to validate ideas or explore trends as you build your next set of content variants.
---
## Troubleshooting
| Issue | Likely cause | Resolution |
|--------|--------------|------------|
| Similar messages not appearing | Message is new or missing tags | Add or refine [tags](/ai-decisioning/tags) |
| Weekly report missing | Summary not configured | Contact support |
| Recommendations feel too general | Limited content variety or inconsistent tag usage | Refine tags or add more content |
| Fatigue insights unavailable | Not enough message sends | Allow more time for performance data to accumulate |
Content Suggestions rely on recent performance. Newly created messages may need time before insights appear.
---
## What’s next?
* [AID Tags →](/ai-decisioning/tags)
* [QA guide →](/ai-decisioning/qa-guide)
* [Hightouch Agents overview →](/agents/overview)
* [AID Insights →](/ai-decisioning/insights)
* [AID Agents →](/ai-decisioning/agents)
---
## Messages
**URL:** https://hightouch.com/docs/ai-decisioning/messages
**Description:** Create, manage, and optimize message variants that AI Decisioning evaluates and improves over time.
**Section:** AI Decisioning
| | |
|----------------------|------------------------------------------------------------------------------------------------------|
| **Audience** | Marketers, business users |
| **Prerequisites** |
At least one [Agent](/ai-decisioning/agents)
A connected destination (Braze, Iterable, or Salesforce Marketing Cloud)
|
*Add Messages to optimize content from your destination templates by defining variants and letting agents control which version is sent, when, and to whom.*
---
## What you’ll learn
In this article, you’ll learn how to:
- [Add messages from your destination](#add-a-message)
- [Understand base vs optimized content](#message-content)
- [Create and manage message variants](#add-and-manage-variants)
- [Control variant eligibility with rules and tags](#variant-level-controls)
- [Configure message-level settings](#message-configuration)
- [Manage messages at scale with CSV uploads and bulk actions](#manage-messages-at-scale)
## Overview
**Messages** define the creative variations an [agent](/ai-decisioning/agents) can send. Each message is sourced from a destination (such as Braze or Iterable) and can include multiple variants for experimentation.
Agents continuously evaluate messages to determine which variant, timing, and channel perform best for each user.
---
## Add a message
1. Open an agent and go to the `Messages` tab.
2. Click `Add messages`, then select **Add messages individually**.
3. In the **Add messages** window:
1. Select a **channel and destination**.
- A **channel** defines how a message is delivered (for example, email or push), while the **destination** defines where the content is managed (such as Braze or Iterable).
2. Select the destination-specific identifier:
- **Braze**: Choose a campaign
- **Iterable**: Choose a template
- **Salesforce Marketing Cloud**: Choose a triggered send or message
- **Custom (S3)**: Enter a resource identifier
3. Enter a **message name** to identify the message in Hightouch.
4. Click `Add messages`.
The message is added to the agent and appears in the Messages list.
Use message names that describe the intent or use case (for example, `Cross-sell – accessories email`) to make messages easier to find and analyze later.
---
### View and manage messages
The `Messages` page provides a centralized view of message configuration and performance for an agent.
From this view, you can:
- Review **message status** (`Enabled` or `Draft`)
- See the number of **variants** per message
- View applied **tags**
- Search messages by **name**
- Filter messages by **status**
Select a message to access three views:
| Tab | Purpose |
|-----|--------|
| `Configuration` | Review and manage message settings including tags, offers, goals, collections, and advanced delivery guardrails (send limits, timeline, delivery windows, eligibility, and campaign attribution). |
| `Usage` | See how often the message is eligible to send and how it is being evaluated by the agent. |
| `Performance` | Review delivery and outcome metrics to understand how the message contributes to agent goals. |
Use these views together to confirm setup and understand how messages perform over time.
---
## Message content
Each message includes two parts: **base content** and **optimized content**.
### Base content
**Base content** is the underlying message content defined in your marketing destination.
Examples include:
- An email built in Iterable, Braze, or Salesforce Marketing Cloud
- A push or in-app message defined in your destination
- Destination-managed layout, variables, or logic
When you add a message, you select the destination and the campaign, template, or send to use. That destination-managed content becomes the base content for the message.
AI Decisioning does not modify your base content. It uses your content as the foundation for optimized content and variants.
---
### Optimized content
**Optimized content** includes the specific fields within a message that agents can vary and learn from over time.
Common optimized content types include:
- Subject
- Preheader
- Call to action (CTA)
- Other supported message fields
Each optimized content type can include multiple variants, which are evaluated against your agent’s goals.
---
## Variants
A **variant** is one version of an optimized content type (for example, a subject line option).
Variants are the individual options that agents compare and learn from over time.
Variants differ from traditional A/B tests:
- Traffic allocation adapts over time
- There is no fixed test window
- No manual winner selection is required
---
## Add and manage variants
Variants are added inside an individual message, in the `Content` tab.
1. Open an agent and go to the `Messages` tab.
2. Select a message.
3. Open the `Content` tab.
4. Expand an optimized content type (for example, `Subject`).
5. Click `Add variant`.
6. Enter the variant content.
You can add multiple variants to the same optimized content type.
Start with a small number of clear variants that differ meaningfully in tone, framing, or offer.
---
## Variant-level controls
Each variant includes controls that determine when it is eligible to send.
To manage variant-level settings, open a message and go to the `Content` tab. Under **Optimized content**, click `Edit tags and rules`.
From here, you can add or update:
- Tags
- Audience rules
- Date rules
- Send limits
---
### Audience rules
Audience rules limit a variant to specific users.
Examples:
- Show a loyalty-focused subject line only to returning customers
- Restrict a discount variant to lapsed users
Audience rules apply at the **variant level**, not the message level.
---
### Date rules
Date rules control when a variant is active.
Use date rules to:
- Run seasonal or time-bound messaging
- Automatically retire expired content
- Stage future variants ahead of launch
Expired variants remain visible but are no longer eligible to send.
---
### Send limits
Send limits control how often the same variant can be sent to a single user.
Available options include:
- **No limit**
- **Static limit** (for example, once per user)
- **Time window** (for example, once every 30 days)
Send limits apply to variants individually, not to the entire message.
---
### Tags
Tags describe the intent or attributes of a variant, such as:
- Tone (`friendly`, `urgent`)
- Offer type (`discount`, `free gift`)
- Value proposition (`convenience`, `exclusivity`)
Tags help agents group and analyze content and are used in [Content Analysis](/ai-decisioning/content-analysis) and [Insights](/ai-decisioning/insights).
See [Tags →](/ai-decisioning/tags) for guidance on tagging strategy.
AI Decisioning only compares variants that are eligible at the time of send. Audience rules, date rules, and send limits determine which variants are considered.
---
## Manage messages at scale
AI Decisioning supports two bulk workflows for managing messages:
- [**CSV upload**](#bulk-upload-and-export-messages-csv) for content and metadata updates
- [**Bulk actions**](#apply-message-guardrails-in-bulk) for applying message guardrails
Use CSV upload for structured content changes and bulk actions for delivery controls.
---
### Bulk upload and export messages (CSV)
Use bulk CSV upload to add or update messages in an agent at scale. This workflow is best for structured, repeatable updates that are easier to manage in a spreadsheet.
#### What you can manage with CSV upload
Using a CSV file, you can:
- Add new messages to an agent
- Update existing messages
- Add or modify:
- Message content
- Variables and variable options
- Message- and variant-level tags
#### What CSV upload does not support
CSV upload does not support message guardrails, including:
- Eligibility filters
- Send limits
- Campaign attribution
Apply these settings in the product UI using [bulk actions](#apply-message-guardrails-in-bulk).
#### Upload messages with CSV
1. Go to `AI Decisioning → Agents` and select an agent.
2. Open the `Messages` tab.
3. Click `Add messages`, then select `Bulk upload with CSV`.
4. Download the provided CSV template.
5. Edit the template with your updates.
6. Upload the completed CSV to apply changes.
You must use the provided CSV template. Uploads will fail if required columns are missing or renamed.
---
### Apply message guardrails in bulk
Use bulk actions to apply message-level guardrails to multiple messages at once.
#### Supported bulk guardrails
You can apply:
- Eligibility filters
- Send limits
- Campaign attribution values
#### Apply guardrails in bulk
1. Go to `AI Decisioning → Agents` and select an agent.
2. Open the `Messages` tab.
3. Select one or more messages.
4. Click `Actions`.
5. Choose one of the following:
- `Add eligibility filters`
- `Add send limits`
- `Add campaign attribution`
6. Configure the settings and apply them.
Bulk actions update message configuration only. They do not change message content, variants, or tags.
---
## Message configuration
The `Configuration` tab contains message-level settings that control tagging, offer attribution, goal visibility, collections, and advanced delivery guardrails.
To access these settings, open an agent, go to the `Messages` tab, select a message, and open the `Configuration` tab.
---
### Tags
Tags are key-value pairs that describe a message's intent or attributes, such as `motivator = efficiency` or `value_proposition = convenience`.
Tags can be added manually or generated automatically using the **Generate tags with AI** button, which analyzes the message content and suggests relevant tags.
Message-level tags apply to the message as a whole and are separate from [variant-level tags](#tags-1). Both are used in [Content Analysis](/ai-decisioning/content-analysis) and [Insights](/ai-decisioning/insights).
See [Tags →](/ai-decisioning/tags) for guidance on tagging strategy.
---
### Offers
Attach an offer to have the agent factor the message's cost and value into its decisions. Messages with higher expected values (value minus cost, weighted by conversion likelihood) are more likely to be selected. Each message can have at most one attached offer, and not all messages require one.
To attach an offer, click `Add offer` in the **Offer** section and configure the following:
- **Name**: A short, human-readable name (for example, `$10 off first order`).
- **Description** *(optional)*: Additional context about the offer.
- **External ID** *(optional)*: A stable identifier in your source system. Required for CSV upserts to match existing offers.
- **Tags** *(optional)*: Key-value attributes to help organize and categorize offers.
- **Offer type**: **Static** offers apply the same cost to every user. **Column** offers read per-user values from warehouse columns.
- **Cost value**: A fixed cost associated with each conversion (for example, $10 off).
- **Cost multiplier**: A discount rate applied to each conversion's value (for example, `0.2` = 20% discount).
Exactly one of cost value or cost multiplier must be non-zero.
---
### Goals
The **Goals** section displays the goals inherited from the parent agent, along with a performance rating for each goal (for example, `Best`, `Very good`, `Good`, `Worst`). These ratings reflect how well the message contributes to each goal relative to other messages in the agent.
Goals are configured at the agent level and cannot be edited from the message Configuration tab. See [Agents →](/ai-decisioning/agents) for details on goal configuration.
---
### Collections
[Collections](/ai-decisioning/collections) let you attach item recommendations to a message. For example, you can include product recommendations from a "In-stock Products" collection alongside the message content.
To configure collections, select one or more available collections and set how many items to recommend per message (for example, `Recommend 4 item(s) per message`).
Collections are optional. See [Collections →](/ai-decisioning/collections) for details.
---
### Advanced settings
The **Advanced** section contains delivery guardrails that control how, when, and to whom a message is sent. Expand the **Advanced** section to access these settings.
#### Send limits
Send limits control the maximum number of times a message can be sent to the same user.
Available options include:
- **No limit**: The message can be sent without restriction.
- **Static**: A fixed cap (for example, send at most 3 times per user).
- **Time window**: A rolling limit (for example, once every 30 days).
- **Minimum other messages in between**: Requires a set number of other messages to be sent before this message can be sent again.
Message-level send limits apply to all variants of the message. They are separate from [variant-level send limits](#send-limits), which apply to individual variants.
#### Timeline
Set a start and end date for a message. Use timeline settings to stage messages ahead of a launch or automatically stop a message after a campaign ends.
If no dates are set, the message is eligible to send as long as it is enabled.
#### Delivery windows
Delivery windows let you restrict when a message can be sent by specifying allowed days and time slots. This gives you direct control over message timing without relying on complex channel or schedule configurations.
To set a delivery window:
1. Toggle on **Delivery windows**.
2. Select the days and time blocks during which the message is allowed to send.
Time blocks that are not selected are blocked, meaning the agent will not send the message during those windows.
Use delivery windows to send promotional emails only during morning hours or restrict campaigns to specific days, such as midweek-only sends.
#### Eligibility
Eligibility filters control which users can receive the message. The base eligibility filter is inherited from the agent and cannot be edited at the message level. You can add additional filters to further restrict the audience.
Click `Calculate size` to estimate how many users match the current eligibility criteria.
#### Campaign attribution
Add additional campaign IDs for outcome attribution purposes. This lets you associate message performance with specific campaigns in your destination, even if the message is used across multiple campaigns.
For a step-by-step checklist to verify all Configuration tab settings before launch, see the [QA guide →](/ai-decisioning/qa-guide).
---
## How Messages connect to Content Analysis
As your agent runs, message and variant data feeds into **Content analysis and suggestions**, where you can:
- Review content similarity before launch
- Identify overlapping or redundant variants
- Understand which themes or tones perform best
- Decide when to add, refresh, or retire content
See [Content analysis and suggestions →](/ai-decisioning/content-analysis) for guidance.
---
## What’s next
- [Agents →](/ai-decisioning/agents)
- [Tags →](/ai-decisioning/tags)
- [Content analysis and suggestions →](/ai-decisioning/content-analysis)
- [QA guide: Validate messages before launch →](/ai-decisioning/qa-guide)
- [Insights →](/ai-decisioning/insights)
---
## QA guide
**URL:** https://hightouch.com/docs/ai-decisioning/qa-guide
**Description:** Step-by-step guide to QA email, SMS, and push messages in AI Decisioning — send test emails with specific variants, verify personalization, and confirm targeting before launch.
**Section:** AI Decisioning
| | |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Marketers, campaign managers |
| **Prerequisites** |
At least one [agent](/ai-decisioning/agents) with [messages](/ai-decisioning/messages) configured
A connected destination ([Braze](/ai-decisioning/braze), [Iterable](/ai-decisioning/iterable), or [Salesforce Marketing Cloud](/ai-decisioning/sfmc))
|
*Use this checklist to QA your messages before launching an AI Decisioning agent.*
---
## Why QA in AI Decisioning
Your team has likely already QA'd creatives before adding them to AI Decisioning. This guide covers the additional QA you should run inside AID — specifically for things AID introduces on top of your existing creative workflow:
- **Variants** — AID can serve different subject lines, images, CTAs, and more. You need to confirm every option renders correctly.
- **Payload fields** — AID passes user data (like first name or product recommendations) to your messaging platform. You need to confirm these populate in the rendered message.
- **Targeting and send rules** — AID applies audience eligibility, send limits, and timing at the message, variant-type, and individual-variant levels. You need to confirm these match your campaign intent.
- **UTM parameters** — AID constructs links with tracking parameters. You need to confirm URLs resolve correctly and UTMs are intact.
Run through this checklist whenever you launch a new agent, add or update messages, or change targeting rules.
---
## How to preview and test variants
A single message in AID can have multiple **variant types** (for example, subject line, image, CTA). Each variant type can have multiple **options**. To QA your message, you preview and test each option individually.
For each variant option:
1. **Preview the variant** — In your message’s **Content** tab, expand the variant type you want to test (for example, **subject**) and click the **eye icon** next to the option you want to preview. The **"Now previewing"** banner confirms the option is selected.
2. **Send a test** — Click **Send test**. In the dialog that appears, enter the identifier for your test user (for example, an email address or user ID) and click **Send test**. The test message uses exactly the variant options you’ve selected.
3. **Check the test message** — Open the test email, SMS, or push notification and verify the content renders as expected. Use the [channel-specific checklists](#email-qa) below to know what to look for.
4. **Repeat for each variant option** until you’ve tested every option across all variant types.
5. After testing all variants, switch to the **Configuration** tab to review message-level settings like send limits, timing, and audience eligibility. See the [channel-specific checklists](#email-qa) below for the full list of configuration checks.
"Now previewing" is not just a visual preview. It determines what gets sent in your test message.
---
## Email QA
Preview and test each variant option in the **Content** tab, then confirm your send rules in the **Configuration** tab.
Go to: `AI Decisioning → Agents → [Your Agent] → Messages → [Your Message]`
### Content tab
If your message uses variants, [preview and test each option](#how-to-preview-and-test-variants) as you work through the checks below.
- [ ] [Sender and metadata](#sender-and-metadata)
- [ ] [Subject line and preheader](#subject-line-and-preheader)
- [ ] [Body content](#body-content)
- [ ] [Links and CTAs](#links-and-ctas)
- [ ] [Footer](#footer)
- [ ] [Variant-type send limits](#variant-type-send-limits)
- [ ] [Individual variant eligibility](#individual-variant-eligibility)
#### Sender and metadata
* Confirm the **sender email address** matches what recipients should see in their inbox.
#### Subject line and preheader
* Confirm the subject line and preheader render correctly for each variant.
* Confirm personalization tokens (first name, dynamic fields, etc.) populate with real user data — not placeholder text or blanks.
#### Body content
* Confirm the content matches your intended messaging and layout.
* Confirm images render correctly and include alt text.
* Confirm dynamic or conditional content behaves as expected for different audience segments.
#### Links and CTAs
AID constructs the final URL including UTM parameters, so check the full link — not just the display text.
* Confirm all links and buttons lead to the correct destination.
* Confirm UTM parameters and tracking links are intact.
#### Footer
* Confirm the footer displays correctly.
* Confirm unsubscribe, privacy, and legal links work.
#### Variant-type send limits
These let you set send limits per variant type. For example, you can cap how often a specific subject line is sent to the same user. To access these, click **Send limit** at the top of a variant type section.
* Confirm send limits are set correctly for each variant type.
#### Individual variant eligibility
These let you restrict specific variant options to certain audience segments. For example, you can limit a "VIP exclusive" subject line to loyalty-tier users. To access these, hover over the **audience icon** to the right of a variant option and click **Add eligibility filters**.
* Confirm audience eligibility filters are correct for each variant option.
### Configuration tab
Expand the **Advanced** section.
- [ ] [Message-level settings](#message-level-settings)
- [ ] [Message status](#message-status)
#### Message-level settings
These apply to all variants of this message.
* **Send limit**: How often this message can be sent to the same user. Confirm this matches your campaign cadence (for example, 1 send every 30 days). See [Send limits](/ai-decisioning/messages#send-limits-1).
* **Timeline**: Start and end dates for the message. Confirm these are set or intentionally left open. See [Timeline](/ai-decisioning/messages#timeline).
* **Delivery windows**: If enabled, confirm the allowed days and time blocks match your intended send schedule. See [Delivery windows](/ai-decisioning/messages#delivery-windows).
* **Audience eligibility**: The base audience filter inherited from your agent. You can’t edit it here, but confirm it’s correct. See [Eligibility](/ai-decisioning/messages#eligibility).
* **Campaign attribution**: If set, confirm the campaign IDs are correct for outcome tracking. See [Campaign attribution](/ai-decisioning/messages#campaign-attribution).
#### Message status
Confirm the message is set to **Enabled** before your agent goes live.
You can send a **test** even if a message is **disabled**.
However, once AID is live, agents will **NOT** send a message unless it is **enabled**.
---
## SMS QA
SMS has tighter constraints than email — character limits, carrier filtering, and compliance requirements. Preview and test each variant option in the **Content** tab, then confirm your send rules in the **Configuration** tab.
Go to: `AI Decisioning → Agents → [Your Agent] → Messages → [Your Message]`
### Content tab
If your message uses variants, [preview and test each option](#how-to-preview-and-test-variants) as you work through the checks below.
- [ ] [Sender](#sender)
- [ ] [Title and body content](#title-and-body-content)
- [ ] [Links and actions](#links-and-actions)
- [ ] [Compliance](#compliance)
- [ ] [Variant-type send limits and eligibility](#variant-type-send-limits-and-eligibility)
#### Sender
* Confirm the **sender ID or phone number** appears as expected.
#### Title and body content
* Confirm the text is accurate, correctly formatted, and not truncated. SMS messages over 160 characters split into multiple segments, which can affect readability and cost.
* Confirm personalization tokens populate with real user data.
* Confirm dynamic content renders correctly for different audience segments.
#### Links and actions
* Confirm all links work and track correctly.
* Confirm shortened URLs or deep links open as expected.
#### Compliance
* Confirm opt-out instructions are included in every variant (for example, "Reply STOP to unsubscribe").
* If your messages are subject to TCPA or other regulations, confirm required disclosures are present.
#### Variant-type send limits and eligibility
If your message has variants, confirm send limits and audience eligibility filters are set correctly at the variant level. See the [email QA section](#variant-type-send-limits) for details on where to find these settings.
### Configuration tab
Expand the **Advanced** section.
* Confirm message-level send limits, timing, delivery windows, audience eligibility, and campaign attribution are correct. See [Message configuration →](/ai-decisioning/messages#message-configuration) for details on each setting.
* Confirm the message is set to **Enabled** before your agent goes live.
You can send a **test** even if a message is **disabled**.
However, once AID is live, agents will **NOT** send a message unless it is **enabled**.
---
## Push notification QA
Push notifications have tighter display constraints than email — titles and body text can be truncated differently on iOS and Android. Preview and test each variant option in the **Content** tab, then confirm your send rules in the **Configuration** tab.
Go to: `AI Decisioning → Agents → [Your Agent] → Messages → [Your Message]`
### Content tab
If your message uses variants, [preview and test each option](#how-to-preview-and-test-variants) as you work through the checks below.
- [ ] [Sender and app info](#sender-and-app-info)
- [ ] [Notification content](#notification-content)
- [ ] [Links and actions](#links-and-actions-1)
- [ ] [Variant-type send limits](#variant-type-send-limits-1)
- [ ] [Individual variant eligibility](#individual-variant-eligibility-1)
#### Sender and app info
* Confirm the app name and sender ID are correct.
* Confirm icons or media display correctly.
#### Notification content
* Confirm the title and body display as expected.
* Confirm personalization tokens populate with real user data.
* If possible, test on both iOS and Android. iOS typically truncates titles around 40 characters and bodies around 110 characters. Android varies by device.
#### Links and actions
Deep links open specific screens in your app (for example, a product page or cart). If misconfigured, the user may land on the home screen instead.
* Test all deep links by tapping the notification on a test device and confirming you reach the correct screen.
* If you have fallback URLs for users without the app, test those as well.
#### Variant-type send limits
These let you set send limits per variant type. For example, you can cap how often a specific notification title is sent to the same user.
* Confirm send limits are set correctly for each variant type.
#### Individual variant eligibility
These let you restrict specific variant options to certain audience segments. For example, you can limit a promotional notification to users who have the latest app version.
* Confirm audience eligibility filters are correct for each variant option.
### Configuration tab
Expand the **Advanced** section.
- [ ] [Message-level settings](#message-level-settings-1)
- [ ] [Message status](#message-status-1)
#### Message-level settings
These apply to all variants of this message.
* Confirm send limits, timing, delivery windows, audience eligibility, and campaign attribution are correct. See [Message configuration →](/ai-decisioning/messages#message-configuration) for details on each setting.
#### Message status
Confirm the message is set to **Enabled** before your agent goes live.
You can send a **test** even if a message is **disabled**.
However, once AID is live, agents will **NOT** send a message unless it is **enabled**.
---
## Troubleshooting
If something isn't working during QA, check here for the most common issues.
Go to: `AI Decisioning → Agents → [Your Agent] → Messages → [Your Message]`
### Message test not received
If your test send doesn’t arrive:
1. **Confirm the message is enabled.**
2. **Check the identifier.**
* Confirm you entered the correct identifier (for example, email, user ID, or subscriber key).
* It must match both your channel configuration and the value shown in the test send.
3. **Confirm the identifier exists in your destination.**
* The user must already exist in your email or messaging platform.
4. **Select a user in preview (SFMC).**
* Go to **Settings → Select user** before sending a test.
### Personalization not working
If personalization appears blank or incorrect:
1. **Check your channel payload configuration.**
* Confirm the fields you’re using are included in the channel payload (these are the fields sent with each message).
2. **Check your user data model.**
* Fields like `first_name` and `last_name` must exist on your main user model (AID parent model).
* If they’re on a related model, you can either:
* Merge the model into the main user model (so all fields are available in one place), or
* Add the fields through **Configuration → Additional user columns** (to include specific fields without merging everything)
### Variants not showing
If your test email shows the same content regardless of which variant you select, the destination template is likely missing the required code snippet.
* Confirm the code snippet is added to your destination template.
* See the variable setup guide for your destination: [Braze](/ai-decisioning/braze#use-variables-in-braze), [Iterable](/ai-decisioning/iterable#5-use-variables-in-iterable-templates), or [Salesforce Marketing Cloud](/ai-decisioning/sfmc#use-variables-in-sfmc).
This is a one-time setup step. Once the snippet is in your template, it applies to all messages and variants.
### Variant images not displaying
If images appear broken or don't load in your test email:
* Confirm the code snippet is added to your destination template.
* Confirm all image URLs are publicly accessible — open each URL in an incognito browser window to verify.
---
## Next steps
Once you've completed QA, confirm your [agent](/ai-decisioning/agents) is enabled and ready to launch.
---
## Get started with AI Decisioning
**URL:** https://hightouch.com/docs/ai-decisioning/quick-start
**Description:** This quick-start guide walks you through initial setup (for admins and data teams) and day-to-day usage (for marketers), with links to each key step.
**Section:** AI Decisioning
| | |
|-------------------|-----------------------------------------------------------------------------------|
| **Audience** | Platform admins and data teams, marketers |
| **Prerequisites** | [AI Decisioning overview →](/ai-decisioning/overview) |
**AI Decisioning (AID)** helps teams automate and improve campaign decisions at the individual user level. You define **agents** that decide who to target and what to optimize for, then add **messages** so agents can test and deliver different versions of your content. Over time, agents learn which message, timing, and channel perform best for each user.
This guide covers:
- [**Technical setup (admins and data teams) →**](#technical-setup-admins-and-data-teams)
Configure AI Decisioning settings, prepare data for decisioning, review data handling, and connect destinations.
- [**Day-to-day usage for marketers →**](#define-agents)
Define agents, add and optimize messages, validate delivery, and measure performance.
---
## Technical setup (admins and data teams)
Before marketers can create agents or messages, your workspace must be configured with the right data models and connected destinations.
Most of this setup is completed once per workspace.
### Configure workspace and data
| Step | What you’ll do | Article |
|------|----------------|----------|
| **Configure settings** | Define global eligibility rules, time zones, and channel defaults. | [Configuration →](/ai-decisioning/configuration) |
| **Prepare data for AID** | Define audience models, structure event data, and prepare tables for decisioning. | [Prepare data for AID →](/ai-decisioning/prepare-data-for-aid) |
### Connect destinations
| Step | What you’ll do | Article |
|------|----------------|----------|
| **Connect destinations** | Connect your ESP or messaging platform so AID can deliver messages. | [Braze](/ai-decisioning/braze), [Iterable](/ai-decisioning/iterable), or [Salesforce Marketing Cloud](/ai-decisioning/sfmc) |
After technical setup is complete, marketers can manage agents and messages directly from [**AI Decisioning → Agents**](https://app.hightouch.com/ai/flows).
---
## Define agents
Agents define *who* to target and *what* to optimize for. Each agent combines an audience, goals, and decisioning rules into a single adaptive campaign.
### Create and configure an agent
| Step | What you’ll do | Article |
|------|----------------|----------|
| **Create an agent** | Define the target audience and campaign goals. | [Agents →](/ai-decisioning/agents) |
| **Configure scheduling** | Set send frequency, quiet hours, and blackout dates. | [Agents → Configure scheduling](/ai-decisioning/agents#step-3-optional-configure-scheduling) |
| **Enable Smart Suppression (optional)** | Reduce low-impact sends across the entire agent. | [Smart Suppression →](/ai-decisioning/smart-suppression) |
| **Add collections (optional)** | Define dynamic sets of products or content the agent can recommend. | [Collections →](/ai-decisioning/collections) |
Most teams start with a single agent focused on one goal (for example, reactivation or purchase) and expand as performance data accumulates.
---
## Add and optimize messages
Once an agent is defined, add messages from your destination and define variants the agent can evaluate and learn from.
### Create messages and variants
| Step | What you’ll do | Article |
|------|----------------|----------|
| **Add messages** | Start from destination templates and connect them to your agent. | [Messages → Add a message](/ai-decisioning/messages#add-a-message) |
| **Create variants** | Define multiple versions of fields like subject lines or CTAs. | [Messages → Add and manage variants](/ai-decisioning/messages#add-and-manage-variants) |
| **Apply tags and rules** | Control variant eligibility and classify creative attributes. | [Tags →](/ai-decisioning/tags) |
| **Review content quality** | Identify overlap, gaps, and opportunities to improve creative. | [Content analysis and suggestions →](/ai-decisioning/content-analysis) |
Messages are sourced from your destination. AI Decisioning does not change your base content—it evaluates variants and controls which version is sent.
---
## Measure and learn
After your agent is running, validate delivery and review performance to refine your strategy over time.
### Validate delivery and analyze results
| Step | What you’ll do | Article |
|------|----------------|----------|
| **QA messages** | Run through channel-specific checklists before launching. | [QA guide →](/ai-decisioning/qa-guide) |
| **Inspect delivery** | Preview sends, confirm eligibility, and troubleshoot issues. | [Inspector →](/ai-decisioning/inspector) |
| **Analyze performance** | Compare results across agents, messages, variants, and audiences. | [Insights →](/ai-decisioning/insights) |
Review Insights regularly to adjust goals, suppression thresholds, and creative as agents continue learning.
---
## Salesforce Marketing Cloud integration with AI Decisioning
**URL:** https://hightouch.com/docs/ai-decisioning/SFMC
**Description:** Learn how to connect Salesforce Marketing Cloud (SFMC) to AI Decisioning in Hightouch to send personalized, real-time messages across email, SMS, push, and in-app channels.
**Section:** AI Decisioning
| | |
|----------------------|------------------------------------------------------------------------------------------------------|
| **Audience** | Technical marketers, marketing and lifecycle teams |
| **Prerequisites** | [Prepare data for AID →](/ai-decisioning/configuration#prepare-data-for-aid) |
*Connect Salesforce Marketing Cloud to AI Decisioning so agents can deliver personalized emails, SMS, push notifications, and in-app messages through your existing SFMC Triggered Sends and Journeys.*
---
## What you'll learn
In this article, you'll learn how to:
- [Set up SFMC as a destination](#1-set-up-sfmc-as-a-destination)
- [Add channels in AI Decisioning](#2-add-channels-in-ai-decisioning)
- [Add messages from SFMC](#3-add-messages)
- [Configure SFMC Triggered Sends and Journeys for AID](#4-configure-sfmc-triggered-sends-and-journeys-for-aid)
- [Use dynamic variables in SFMC templates](#use-variables-in-sfmc)
---
## Overview
Hightouch's AI Decisioning (AID) helps you deliver real-time, personalized messages based on live customer context and predictions. By connecting AID with Salesforce Marketing Cloud (SFMC), you can:
- Automatically trigger messages from AID Agents via SFMC Triggered Sends or Journeys
- Personalize SFMC templates using dynamic variables from Hightouch—no custom code required
- Centralize targeting and decision logic in Hightouch, while using SFMC to handle message delivery
## 1. Set Up SFMC as a Destination
Already configured SFMC? Even if you've previously set SFMC up as a destination, you'll still need to apply the specific permissions and scopes outlined below.
### Why these permissions matter
To send messages through SFMC using AID, Hightouch needs access to certain channels, content, and user data. These permissions allow Hightouch to:
- Trigger messages via Triggered Send and Journey Builder
- Read and update subscriber data
- Access and personalize email, SMS, push, and in-app templates
### Connect SFMC and grant permissions
**In SFMC:**
1. [Create an installed package in SFMC](/destinations/sfmc#create-installed-package-in-sfmc) until you reach **Set Server-to-Server Properties**.
2. Under **Channels**, enable the permissions for each message type you want Hightouch to use (read, write, send, publish, or post).
3. Click **Save**.
**In Hightouch:**
4. Continue setting up the SFMC destination until you reach the [Custom Scopes](/destinations/sfmc#custom-scopes) step.
5. Under **Advanced permissions > Custom Scopes**, paste in the following:
`email_read email_write email_send accounts_read data_extensions_read data_extensions_write journeys_read journeys_write journeys_aspr journeys_execute list_and_subscribers_write list_and_subscribers_read push_read push_write push_send`
6. Click **Test Connection** to verify that setup is complete.
## 2. Add Channels in AI Decisioning
Once SFMC is set up as a destination, you can add **channels**. Channels define how Hightouch sends messages to SFMC—via Triggered Sends or Journeys.
### Set up a channel
1. Go to `AI Decisioning → Configuration → Channels`.
2. Click `Add channel`.
3. Choose a channel type: **Email**, **SMS**, **Push**, or **In-App**.
4. Select **Salesforce Marketing Cloud** as the destination.
5. Select a [sync type](#sync-type-comparison):
- **Triggered Send** – sends a 1:1 message in real time
- **Journey entry** – sends as part of a journey workflow
- **Data Extension** *(advanced)* – batch sends managed within SFMC
6. Choose an **Identifier** that matches the Subscriber Key field in SFMC (for example, email or user ID).
7. Click `Add channel`, then `Save changes`.
### Sync type comparison
The **Sync Type** determines how Hightouch sends messages through SFMC:
| Channel | Triggered Send | Journey | Data Extension |
|---------|:--------------:|:-------:|:--------------:|
| Email | ✅ | ✅ | ✅ |
| SMS | ❌ | ✅ | ✅ |
| Push | ✅ | ✅ | ✅ |
| In-App | ❌ | ✅ | ✅ |
| Sync type | Best for | Setup notes |
|------------------------|---------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Triggered Send** | Real-time, event-based sends tied to a single user action | Requires send classification: use **Commercial** for marketing content (users can opt out) or **Transactional** for system notifications (users can't opt out). |
| **Journey entry** | Automated, multi-step campaigns (for example, onboarding, retargeting) | Select the **Synced user columns** (for example, email, name, product interest) you want to sync. These fields are available inside the journey's Data Extension for personalization or decision splits. |
| **Data Extension** *(advanced)* | Batch sends, scheduled campaigns, and advanced setups where you need full control over delivery logic | A [Data Extension](https://help.salesforce.com/s/articleView?id=mktg.mc_es_data_extension_data_relationships_classic.htm&type=5) is a table in SFMC that stores subscriber or message data. When you choose this sync type, Hightouch writes data directly to a Data Extension, and you manage the send logic (timing, audience filters, scheduling) within SFMC. This gives you the most flexibility but requires more setup on the SFMC side. Choose **Insert** to append new rows or **Mirror** to keep the Data Extension in sync with your warehouse. |
---
## 3. Add Messages
After you add channels in AI Decisioning, you can add **messages** to your agents. Each message connects an agent's decision to a specific template in SFMC.
For full details on message configuration, variants, and tags, see [Messages →](/ai-decisioning/messages).
### Add a message from SFMC
1. Go to `AI Decisioning → Agents` and select your agent.
2. Open the `Messages` tab.
3. Click `Add messages`, then select **Add messages individually**.
4. Select your configured SFMC **channel and destination** tile (for example, **Email - Salesforce Marketing Cloud**).
5. Select the appropriate **template** or [event identifier](https://help.salesforce.com/s/articleView?id=mktg.mc_jb_create_journey_specific_entry_event.htm&type=5) from SFMC.
6. Enter a **message name** and click `Add messages`.
You can repeat this process for any channel and sync type. Continue the setup process in the channel-specific sections below:
- [Send emails via Triggered Send](#send-emails-via-triggered-send)
- [Send emails via Journey](#send-emails-via-journey)
- [Send emails via Data Extension](#send-emails-via-data-extension)
- [Send SMS via Journey](#send-sms-text-messages-via-journey)
- [Send SMS via Data Extension](#send-sms-text-messages-via-data-extension)
- [Send push (outbound) via Triggered Send](#send-push-outbound-notifications-via-triggered-send)
- [Send push (app inbox) via Triggered Send](#send-push-app-inbox-notifications-via-triggered-send)
- [Send push via Journey](#send-push-notifications-via-journey)
- [Send push via Data Extension](#send-push-notifications-via-data-extension)
- [Send in-app via Journey](#send-in-app-notifications-via-journey)
- [Send in-app via Data Extension](#send-in-app-notifications-via-data-extension)
### Test a message
1. Open the message and go to the `Content` tab.
2. Under **Optimized content**, confirm your variants are configured. If your message uses [dynamic variables](#use-variables-in-sfmc), set values for testing.
3. In the **Preview** panel, verify the rendered message looks correct.
4. Click `Send test` to send the message to a test user.
---
## 4. Configure SFMC Triggered Sends and Journeys for AID
Before adding a message in Hightouch, you need to configure the corresponding Triggered Send or Journey in SFMC.
### Send emails via Triggered Send
1. [Add a channel](#set-up-a-channel) with the sync type set to **Triggered Send**.
2. [Add a message](#add-a-message-from-sfmc) from your SFMC email template.
3. Close the window that opens and return to the **Messages** tab.
4. Click the link next to **Base content** to go to SFMC.
5. This will prompt you to log in to SFMC and should open the **Email Studio > Interactions** page.
6. Click **Triggered Sends**.
7. Select the send name that corresponds with your newly created message.
8. Click **Start/Restart**.
9. Return to **Hightouch** and open your message.
10. In the Content tab, [set the variables](#use-variables-in-sfmc) and click **Send Test**.
### Send emails via Journey
1. [Add a channel](#set-up-a-channel) with the sync type set to **Journey entry**.
2. [Add a message](#add-a-message-from-sfmc) from your SFMC email template.
3. In the Content tab, [set the variables](#use-variables-in-sfmc) and click **Send Test**.
### Send emails via Data Extension
1. [Add a channel](#set-up-a-channel) with the sync type set to **Data Extension**.
2. Choose **Insert** (append new rows) or **Mirror** (keep in sync with your warehouse).
3. [Add a message](#add-a-message-from-sfmc) from your SFMC email template.
4. In SFMC, configure your send logic (automation, query activity, or triggered automation) to read from the Data Extension that Hightouch populates.
### Send SMS text messages via Journey
1. [Add a channel](#set-up-a-channel) with the sync type set to **Journey entry**.
2. [Add a message](#add-a-message-from-sfmc) from your SFMC SMS template.
3. In the **Content** tab, [set the variables](#use-variables-in-sfmc) and click **Send Test**.
### Send SMS text messages via Data Extension
1. [Add a channel](#set-up-a-channel) with the sync type set to **Data Extension**.
2. Choose **Insert** or **Mirror** mode.
3. [Add a message](#add-a-message-from-sfmc) from your SFMC SMS template.
4. In SFMC, configure your send logic to read from the Data Extension that Hightouch populates.
### Send push (outbound) notifications via Triggered Send
#### Enable the push channel
1. [Add a channel](#set-up-a-channel) with the sync type set to **Triggered Send**.
#### Configure your push template in SFMC
**In SFMC:**
2. Go to **Mobile Studio > MobilePush**.
3. Click **Create Message**.
4. Select the **Outbound** template and click **Next**.
5. Name your message and select your app.
6. Select **API Triggered** as the Send Method.
7. Select **Alert** as the Push Method and click **Next**.
You can only trigger Alert Only or Inbox Only messages, not Alert + Inbox.
8. Create the template for your push alert.
9. **Save** your message.
10. Click **Send** in the top right-hand corner.
11. An activation window should pop up with an API key. Copy this API key to make it easier to find the template in Hightouch in a later step.
#### Add a message in Hightouch
**In Hightouch:**
12. [Add a message](#add-a-message-from-sfmc) from your SFMC push template.
#### Preview content
Hightouch automatically creates the variables for push notification title, subtitle, and body. The values you supply here are used directly in the push notification without any need for code snippets in the template. If these are left blank, Hightouch uses the default values in the template.
Hightouch uses the override capabilities in the API. If even one variable has a value supplied, the default values in the template for **all** variables get overridden. Make sure to supply at least the default value for all variables, even if you only want AID to vary one.
#### Send tests
13. To test push notifications, paste the contact key you use in your SFMC contacts.
14. Personalization strings can be supplied directly in the values for the variables without any need for code snippets in the template.
### Send push (app inbox) notifications via Triggered Send
#### Enable the push channel
1. [Add a channel](#set-up-a-channel) with the sync type set to **Triggered Send**.
#### Configure your push template in SFMC
**In SFMC:**
2. Go to **Mobile Studio > MobilePush**.
3. Click **Create Message**.
4. Select the **Inbox** template and click **Next**.
5. Name your message and select your app.
6. Select **API Triggered** as the Send Method.
7. Select **Alert** as the Push Method and click **Next**.
You can only trigger Alert Only or Inbox Only messages, not Alert + Inbox.
8. Create the template for your push alert.
9. **Save** your message.
10. Click **Send** in the top right-hand corner.
11. An activation window should pop up with an API key. Copy this API key to make it easier to find the template in Hightouch in a later step.
#### Add a message in Hightouch
**In Hightouch:**
12. [Add a message](#add-a-message-from-sfmc) from your SFMC push template.
### Send push notifications via Journey
1. [Add a channel](#set-up-a-channel) with the sync type set to **Journey entry**.
2. [Add a message](#add-a-message-from-sfmc) from your SFMC push template.
3. In the **Content** tab, [set the variables](#use-variables-in-sfmc) and click **Send Test**.
### Send push notifications via Data Extension
1. [Add a channel](#set-up-a-channel) with the sync type set to **Data Extension**.
2. Choose **Insert** or **Mirror** mode.
3. [Add a message](#add-a-message-from-sfmc) from your SFMC push template.
4. In SFMC, configure your send logic to read from the Data Extension that Hightouch populates.
### Send in-app notifications via Journey
#### Enable the in-app channel and add a message
1. [Add a channel](#set-up-a-channel) with the sync type set to **Journey entry**.
2. [Add a message](#add-a-message-from-sfmc) from your SFMC in-app template.
#### Link your message to an SFMC Journey
In these next steps, you will link your newly created message to an associated Journey in SFMC.
**In Hightouch:**
3. Go to the **Content** tab within your message to add the variables you would like Hightouch to control.
**In SFMC:**
4. Go to **Journey Builder > Journey Builder**.
5. Click **Create New Journey**.
6. Select **Multi-Step Journey**.
7. Select **API Event** as the entry source. Drag it onto the Journey Builder.
8. Click the grey **API Event** icon in the Journey Builder. This will open a list of API Events.
9. Select the message name you chose when adding the message in Hightouch.
Hightouch automatically creates the required Data Extension with the event so you don't have to set that up.
10. Click **Done**.
11. Next, drag **In-App Message** to the Journey Builder.
12. Click the grey **In-App Message** icon in the Journey Builder.
13. **Select** (or create) your desired in-app message template.
14. In the **In-App Message Activity Summary**, edit the required settings for **Message Configuration** and **Display Trigger**. Optionally, configure **Display Options** and **Advanced Options** as desired. See [SFMC documentation](https://help.salesforce.com/s/articleView?id=mktg.mc_jb_configure_inapp_in_journey_builder.htm&type=5) for more details.
15. Optional: If you would like Hightouch to control aspects of the In-App Message, follow the instructions for [using variables in SFMC templates](#use-variables-in-sfmc).
- Paste the variable code snippets from Hightouch into SFMC:
- Select a message in **Message Definition**
- Click **Edit Selected Message**
16. Paste the copied code snippets from Hightouch into the **Content** tab that opens.
17. **Save** your changes.
18. Optional: To continuously test sends, click the **Gear icon > Journey Settings** and select re-entry any time.
19. **Activate** your journey.
**In Hightouch:**
20. Return to **Hightouch** and refresh the Content tab within your message.
21. Preview your content to ensure everything is accurate.
22. You can now **Send test** to subscribers.
### Send in-app notifications via Data Extension
1. [Add a channel](#set-up-a-channel) with the sync type set to **Data Extension**.
2. Choose **Insert** or **Mirror** mode.
3. [Add a message](#add-a-message-from-sfmc) from your SFMC in-app template.
4. In SFMC, configure your send logic to read from the Data Extension that Hightouch populates.
---
## Use Variables in SFMC
Hightouch variables allow you to personalize SFMC messages using live warehouse data, any variable you've defined within the message such as subject line variations, or different content blocks.
### Access variables
1. Open a message in Hightouch and go to the `Content` tab.
2. Under **Optimized content**, expand a content type (for example, **subject** or **body**) and add your variants.
3. Click the `...` menu on a content type and select `Copy code snippet` to get the variable code.
4. Paste the code into your SFMC template. These snippets render with real values for each user when the message is sent.
### Important notes
- **Triggered Sends**: Variables are fully supported.
- **Journeys**: Variables are available via Data Extension fields synced by Hightouch.
---
## Troubleshooting
| Issue | Likely cause | Resolution |
|-------|-------------|------------|
| Message fails to send | API errors or misconfigured destination | Check Hightouch logs for API errors. |
| Test user not receiving messages | Identifier mismatch | Verify the identifier matches a valid Subscriber Key in SFMC. |
| Triggered Send not firing | Send not started | Confirm the Triggered Send is in **Running** status in Email Studio. |
| Journey sends not triggering | Journey not activated or entry event misconfigured | Confirm the Journey is activated and the API entry event is configured. |
| Variables not rendering | Incorrect syntax or missing field mapping | Double-check variable syntax from `Copy code snippet` and field mapping. |
---
## What's next
- [Messages: Create and manage message variants →](/ai-decisioning/messages)
- [QA guide: Validate messages before launch →](/ai-decisioning/qa-guide)
- [Agents →](/ai-decisioning/agents)
- [Insights →](/ai-decisioning/insights)
---
## AID Tags
**URL:** https://hightouch.com/docs/ai-decisioning/tags
**Description:** Plan, apply, analyze, and manage message tags to compare creative performance across AI Decisioning agents.
**Section:** AI Decisioning
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Marketers (with support from data or platform teams) |
| **Prerequisites** |
Access to [AI Decisioning](/ai-decisioning/overview)
At least one agent, message, or variant created
|
*Tags help you organize messages and variants, compare creative performance, and apply consistent structure across your AI Decisioning workspace.*
---
## What you'll learn
After reading this article, you'll know how to:
* [Plan a tagging framework](#plan-your-tagging-framework) before you start
* [Add tags](#tag-messages-and-variants) for messages and variants
* [Analyze performance](#analyze-tag-performance) using tags in Insights
* [Standardize, govern, and optimize tags](#manage-and-govern-your-tags) across your workspace
---
## Overview
Tags in **AI Decisioning** classify and group [message variants](/ai-decisioning/messages#add-and-manage-variants) so you can compare how different creative strategies perform across your agents.
Each tag consists of a **key** and a **value**. The key defines a category, such as `tone` or `product`, and the value describes a specific attribute, such as `playful` or `sunglasses`. Together, these pairs describe both what your message says and how it says it.
| Tag type | Description | Example use |
|-----------|--------------|-------------|
| **Message tags** | Describe the overall goal, topic, or focus of a message | `Summer Promo`, `Loyalty Push`, `Flight Offers` |
| **Variant tags** | Describe specific creative differences within a message | `Emotional`, `Discount CTA`, `Travel Deals` |
Once applied, tags let you break down performance in [Insights](/ai-decisioning/insights) to identify which themes, tones, offers, or structures drive engagement. Tags are most useful when multiple messages or variants share the same concepts, because that overlap makes it easier to compare results and identify patterns.
Consistent tagging also helps AI Decisioning learn from past performance, improving future recommendations and experiments.
---
## Plan your tagging framework
Before you start tagging, decide what you want to learn from your campaigns — such as which **offers**, **value propositions**, or **tones** drive engagement. Planning your framework upfront helps you tag consistently from the start.
### Decide what to tag
Your tags should capture:
- **What the content says** — for example, product, offer, or audience
- **How it says it** — for example, tone, urgency, or format
**Guidelines**
- Keep tag names short and consistent
- Use one value per tag type, such as one `Tone` or one `Offer`
- Avoid duplicate concepts, such as `Incentive` and `Offer`
- Don't recreate system fields like `channel`, `message`, or `subject line`
- Create tags for any dimension you want to measure or control, such as business unit or product line
Tags describe **content attributes**, not user traits. It's okay if they overlap. For example, a message might use the tag `Career = Marketer` while a user profile also includes a `career` trait.
### Choose message tags vs. variant tags
Use **message-level tags** for broad attributes that apply to the entire message, such as product, campaign theme, or business unit. Use **variant-level tags** for finer creative differences, such as subject line tone, CTA style, or emoji usage.
### Start small, then expand
| Tag type | Recommended range | Notes |
|-----------|------------------|-------|
| **Keys** | 3–8 | Focus on the main dimensions you want to measure, such as tone, offer, or motivator. |
| **Values** | 3–6 per key | Each value should apply to multiple messages. Avoid long lists of rarely used values. |
As you add new tags, Insights automatically updates results for past agents that use them.
### Best practices
| Goal | Recommendation |
|------|----------------|
| **Stay consistent** | Use a standard naming pattern, such as `Tone`, `Offer`, or `Audience`. |
| **Avoid duplicates** | Regularly review tags in the **Global Tag Manager** to remove similar values. |
| **Be specific** | Use clear labels like `CTA: Learn More` instead of generic names like `Email Test`. |
| **Align with goals** | Create tags that reflect what you want to measure, such as conversion or engagement. |
| **Use AI to assist** | Generate tags quickly, then refine them for accuracy. |
| **Review performance** | Use [Insights](/ai-decisioning/insights) to identify which tags drive results. |
Keep a shared reference document for your team's approved tags, such as `Promo Type`, `Tone`, and `Audience`, to maintain consistency across agents.
### Example tag structures
#### Financial company example
| Category | Key | Example values |
|-----------|-----|----------------|
| **Product** | Product type | Checking, Savings, Credit Card, Mortgage |
| **Product** | Feature | No ATM Fees, Auto Transfers, 2% Cash Back |
| **Expression & Style** | Tone | Playful, Urgent, Reassuring, Empowering |
| **Expression & Style** | Motivator | Save Money, Earn Rewards, Gain Control, Simplify Life |
| **Expression & Style** | Narrative | Problem–Solution, Benefit-Focused, Storytelling |
| **Incentive** | Value prop | Low Fees, High Yield, Personalized Offers |
| **Incentive** | Offer | 10% Off, $100 Bonus, Free Trial |
| **Incentive** | CTA | Learn More, Apply Now, Get Started |
| **Creative Structure** | Message length | Short, Medium, Long |
| **Creative Structure** | Urgency | Low, Medium, High |
#### Streaming service example
| Category | Key | Example values |
|-----------|-----|----------------|
| **Product** | Subscription plan | Basic, Standard, Premium, Bundle |
| **Product** | Feature | Ad-Free, Offline Download, 4K, Multi-Screen |
| **Expression & Style** | Tone | Inspirational, Humorous, Reassuring, Playful |
| **Expression & Style** | Motivator | Entertainment, Discovery, Relaxation, Family Fun |
| **Expression & Style** | Genre | Action, Comedy, Drama, Kids, Documentary |
| **Incentive** | Value prop | Free Trial, Exclusive Content, Personalized Recommendations |
| **Incentive** | Offer | 1 Month Free, 20% Off Annual Plan, Bundle Deal |
| **Incentive** | CTA | Watch Now, Start Free Trial, Subscribe Today |
| **Creative Structure** | Message length | Short, Medium, Long |
| **Creative Structure** | Emoji usage | None, Light, Heavy |
| **Business Unit** | Region | US, GB, FR, DE, AUS, NZ, JP |
Use these examples as a starting point. Choose the categories that fit your business, then expand over time.
---
## Tag messages and variants
Once you've planned your tag structure, you can start applying tags while building or reviewing agents. Use the Agent view to tag messages in context, or the [Global Tag Manager](#manage-tags-in-global-tag-manager) to manage tags across your workspace.
| Method | Best for |
|---------|-----------|
| **From an Agent** | Adding or editing tags while building or reviewing a specific message |
| **From Global Tag Manager** | Creating, standardizing, or bulk-editing tags across multiple agents |
### Message tags
**Message tags** classify the overall purpose or focus of a message. You can add them directly from the message configuration view inside an agent.
1. Go to `AI Decisioning → Agents → Select an Agent`.
2. In the `Messages` tab, choose a message.
3. Open the `Configuration` tab.
4. Under `Tags`, use the dropdowns to select existing tags or click `+ Add tag` to create new ones.
5. Click `Save changes` to apply.
**Examples**
- `Product: Bluelight`
- `Motivator: Wellness`
- `Incentive Type: Free gift`
- `Value Proposition: Convenience`
You can also generate message tags with AI by clicking `Generate tags with AI` under the tag list.
---
### Variant tags
**Variant tags** help you compare creative differences within a single message, such as tone, offer, structure, or subject line style.
1. Go to `AI Decisioning → Agents → Select an Agent`.
2. In the `Messages` tab, choose a message.
3. Open the `Content` tab.
4. Under `Optimized content`, click `Edit tags and rules`.
5. Hover over the tag icon next to a variant and click `+ Add content tags`.
6. In the `Edit content tags` modal, use the dropdown to select existing tag values or type to create new ones.
7. Click `Apply` to save your changes.
**Examples**
- `Length: Short`
- `Tone: Savings focused`
- `Urgency: Normal`
- `Has emoji: Yes`
Add variant tags while testing subject lines, images, or CTAs. These tags connect performance differences directly to specific creative choices in [Insights](/ai-decisioning/insights).
---
## Analyze tag performance
Use tags to understand how creative themes perform across agents and messages.
### Insights
Once your messages and variants are tagged, you can measure how each tag performs across agents in **Insights**.
1. Go to `AI Decisioning → Insights`.
2. Select the `Creative performance` view.
3. In the `Break down by` dropdown, choose a tag key such as `incentive_type`, `tone`, or `motivator`.
If your workspace uses these tag keys consistently, they'll appear as available breakdown fields.
The table shows how each tag value performs across your agents and messages, including:
- **Sends** — total number of messages delivered for each tag value
- **Key conversion events** — such as purchases, sign-ups, or other tracked actions
- **Engagement metrics** — including click-through rate (CTR), open rate, or conversion rate
**Example**
Breaking down by `incentive_type` reveals which offers drive the strongest results. For example, `fixed_discount` may outperform `free_shipping` or `free_gift` in conversion rate, helping you understand which incentive types resonate most across audiences.
#### Drill-down views
You can also **drill down** into a specific message or variant to explore deeper insights:
- **Personalization insights** — Compare how engagement differs by user attributes such as device type, region, or affinity
- **Timing insights** — Identify when messages perform best by day or time of day
Use the **Insights** tab to evaluate your content strategy at scale. Tag-driven breakdowns connect creative attributes — such as incentive type, tone, or motivator — to measurable performance outcomes.
---
### Content analysis
Each message also includes **content-level insights** to help you understand relationships between creatives.
In the message's `Content` tab, scroll to **Content analysis** to view:
- **Tags** — Applied tags such as `Motivator: Efficiency` or `Value Proposition: Convenience`
- **Similar messages** — Messages with similar structure, tone, or intent
You can expand each comparison to see why messages are related. This helps you identify patterns in your creative library and understand how messages may influence model learning.
**Content analysis** focuses on creative similarity and reuse, while the **Insights** tab focuses on quantitative performance and impact. Use both to understand what works and why.
---
## Manage and govern your tags
As your tagging system grows, use these tools to keep tags consistent, apply them efficiently, and extend their use beyond analysis.
### Manage tags in Global Tag Manager
The Global Tag Manager (`AI Decisioning → Tags`) provides a centralized view of all tags in your workspace. Changes made here apply globally to every agent that uses the tag key.
| Action | Description |
|--------|-------------|
| **View tags** | Review all tag keys, such as `product`, `tone`, or `incentive_type`, and their allowed values. |
| **Create tags** | Define new tag keys and allowed values for messages or variants. |
| **Edit tag values** | Rename or remove values to keep naming consistent. *(Tag keys can't be renamed — delete and recreate if needed.)* |
| **Add tag instructions** | Specify when a tag or value should be applied to guide consistent tagging and improve AI suggestions. |
| **Bulk apply tags with AI** | Use AI to classify many messages at once for a selected tag key. |
| **Check consistency** | Confirm agents use the same key–value pairs, such as `tone = playful` or `product = bluelight`. |
---
### Add tag instructions
In the Global Tag Manager, you can add **tag-level** and **value-level instructions** that define when the tag — and each of its values — should be applied.
Clear instructions improve consistency across teams and help AI apply tags more accurately.
#### What to include in tag instructions
Effective tag instructions answer one question:
**When should this tag or value be used?**
Good instructions help people tag consistently and give AI the context it needs to distinguish between similar values.
#### Writing effective instructions
When writing tag or value instructions:
- Describe **when to apply** the tag or value
- Explain **how it differs** from other values
- Avoid generic definitions or restating the tag name
- Use short, directive language
#### Example: tag-level instructions
**Tag:** `incentive_type`
**Good instruction**
Use this tag to classify the primary incentive offered in the message. Select the value that best represents the main benefit emphasized to the user.
**Weak instruction**
Indicates the type of incentive.
#### Example: value-level instructions
Value-level instructions are most useful when values are similar or easily confused, messages include multiple incentives or concepts, or you use AI-assisted tagging.
**Tag:** `incentive_type`
- **`discount`**
Use when the message emphasizes a price reduction, percentage off, or limited-time deal.
- **`free_gift`**
Use when the message highlights a free item added to a purchase, regardless of purchase amount.
- **`free_shipping`**
Use when the message emphasizes waived shipping costs as the primary incentive.
---
### Use AI to suggest and apply tags
AI can help you tag faster, whether you're working on a single message or tagging across your workspace.
#### Generate tags for individual messages or variants
When editing a message or variant, click `Generate tags with AI` to get suggestions based on your content.
1. Click `Generate tags with AI`.
2. Review the AI-generated tags.
3. Edit or remove any that don't fit your naming conventions.
**Example suggestions**
- Message: *"Earn double points on every booking this weekend"* → AI suggests `Loyalty`, `Limited Time Offer`
- Variant: *"Book now to unlock rewards faster"* → AI suggests `Urgent CTA`, `Rewards Focused`
AI-suggested tags do not apply automatically. Always review suggestions before saving to ensure accuracy and consistency.
#### Bulk-apply tags in the Global Tag Manager
For tagging many messages at once, hover over a tag key and click the **magic wand** button to apply tags with AI in the Global Tag Manager.
---
### Set send frequency caps using tags
You can also use tags to apply **send frequency caps** more granularly. This helps you control how often users receive messages within specific categories, such as by business unit or product line.
For example, if your agent includes messages across multiple business units, you can tag each message with a `Business Unit` value such as `Insurance` or `Banking`, then set separate frequency caps for each one. This helps prevent users from receiving too many messages from the same category, even when the overall agent-level cap has not been reached.
You can also set send frequency caps at the **channel level** (email, SMS, push, and so on). You do not need to tag whether a message is email, push, or SMS because AI Decisioning identifies the channel automatically.
---
## Troubleshooting
| Issue | Likely cause | Resolution |
|--------|---------------|------------|
| Tags not appearing in Insights | Agents not synced or tags missing | Check that each message or variant includes at least one tag and that Insights is enabled. |
| Duplicate tags | Inconsistent naming, such as `Loyalty` vs. `Loyalty Campaign` | Delete duplicates in **Global Tag Manager**. |
| Unclear AI suggestions | Inconsistent message naming or unclear tag definitions | Review your tag list, instructions, and message copy for clearer context. |
Tag visibility in Insights depends on your workspace's sync schedule. If you recently applied tags, wait for the next sync cycle before results appear.
---
## What's next?
After tagging your AID messages and variants, explore related features:
* [Analyze results in Insights →](/ai-decisioning/insights)
* [Inspect message performance →](/ai-decisioning/inspector)
* [Create Agents to automate decisions →](/ai-decisioning/agents)
* [Set up Collections for grouped experimentation →](/ai-decisioning/collections)
---
## Master Booster Frequently Asked Questions
**URL:** https://hightouch.com/docs/match-booster/faqs
**Description:** Increase match rates for your audiences and conversion events in advertising platforms.
**Section:** Match Booster
### What format should source identifiers use?
Match Booster only supports hashed (SHA256 or MD5) identifiers for emails and phone numbers. Hashed values need to be lowercased and trimmed before hashing.
Other PII identifiers need to be in a raw, unhashed format. Hightouch then normalizes the identifiers—for example, by lowercasing and removing spaces to ensure it matches data in our identity graph.
### How should I store multiple identifiers of the same type for one user?
The best way to model multiple identifiers of the same type is to consolidate all values in a single row, and to capture multiple entries as an array.
If you have just one value per identifier, a string is appropriate.
For example:
| First Name | Email | Phone | uuid |
| ---------- | --------------------------------------- | ----------------------------------- | ---- |
| Tyler | `“tyler@work.com”` | `[”555-123-4567” , “999-555-1234”]` | 1 |
| Amina | `[”amina@work.com”, “amina@gmail.com”]` | `”555-567-8309”` | 2 |
Hightouch formats the data to match the requirements of ad platform APIs.
### When does enrichment take place? Does Hightouch store a copy of my data?
After you configure match boosting for a model, Hightouch runs a comparison of your selected identifiers against our identity graph and looks for exact matches.
Hightouch uses a hashed version of your model's primary key to keep track of and access matched rows.
_Hightouch never stores any of your proprietary data._
### How long does model enrichment take?
Once you've configured your boosted model and selected which identifiers to use for enrichment, you can expect enrichment to take a couple of hours and up to one day.
This is true for both the first and subsequent enrichments.
### How often do my boosted models refresh?
You can choose how frequently your boosted models are refreshed. By default, after the first enrichment cycle, Hightouch refreshes boosted models every 14 days.
### How can I know the last time my boosted model has been refreshed?
You can find the following information in the **Match boosting** tab of any boosted model's configuration:
- Enrichment Schedule
- Last enrichment
- Next enrichment
### Is there a way to manually refresh my boosting?
Yes. Click **Save and run enrichment** from the **Match boosting** tab in model configuration.
### Why do my match rates appear lower with Match Booster?
Upon the initial sync to an advertising platform, match rates can take up to 48 hours to stabilize depending on the platform.
Allow enough time for results to calculate.
### Why is Snapchat reporting greater than 100% match?
Snapchat is unconventional in the way it reports match rate.
If you send multiple identifiers for the same user, Snapchat initially reports _separate matches_, which can lead to a greater than 100% match rate.
Later, Snapchat deduplicates and resolves multiple matches down to a single user.
Check back in a day or two to see the stabilized match rate.
---
## Match Booster Implementation
**URL:** https://hightouch.com/docs/match-booster/implementation
**Description:** Configure Match Booster to enrich your data with additional identifiers and improve match rates in advertising platforms.
**Section:** Match Booster
Use this guide to configure Match Booster for the first time, choose supported match keys, and send enriched identifiers to supported destinations.
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Marketers, Platform Admins *(implementation)*, Data teams *(confirm identifiers exist in the model and are formatted correctly)* |
| **Prerequisites** | A [model](/models/creating-models) or Customer Studio [parent model](/customer-studio/schema) configured in Hightouch and at least one [supported destination](/match-booster/overview#supported-destinations) |
## Overview
Setting up Match Booster involves two steps:
1. **Model-level setup**
[Enable Match Booster on a model or parent model](#configure-match-booster-one-time-setup-per-model)
Configure enrichment, select match keys, and run the initial enrichment.
2. **Sync-level configuration**
[Enable Match Booster in a sync](#enable-match-booster-in-a-sync)
Turn on Match Booster and send enriched identifiers to a supported destination.
After setup, you can review performance and delivery over time:
- [Monitor match rates](#monitor-match-rates)
---
## Where Match Booster lives in Hightouch
You can enable Match Booster in two places in Hightouch.
- If you're building **audiences in Customer Studio**, you can enable Match Booster on the **parent model**.
- If you're writing a **SQL query or accessing a table directly**, you can enable Match Booster on the **model**.
| Option | Best for | Where to configure |
| --- | --- | --- |
| Enable on a model | Syncs built directly from a model in Activation | `Activation → Models →` select a model → `Match Booster` |
| Enable on a Customer Studio audience (parent model) | Customer Studio audiences (built from the schema) | `Customer Studio → Schema →` select a **parent model** → `Match Booster` |
---
## Configure Match Booster (one-time setup per model)
### Enable Match Booster on the model
1. Open the model or parent model where you want to enable Match Booster:
- `Activation → Models →` *select a model* → `Match Booster`
- `Customer Studio → Schema →` *select a parent model* → `Match Booster`
2. Toggle `Enable Match Booster` on.
This unlocks additional configuration options.
---
### Select expansion options
Configure optional expansion behavior before mapping match keys.
- **Household Expansion**: Enable if you want to target, suppress, or measure at the household level instead of just the individual.
- **CTV Matching**: Enable if you run ads on CTV platforms (e.g., Roku, LG, Samsung, Vizio) to match against CTV IDs and IPs.
- **LG Ad IDs (optional)**: Enable if you specifically want LG Ad ID matching via Hightouch’s LG integration.
---
### Map match keys (identifiers)
Under **Which identifiers should be used to link your model to Hightouch’s identity graph?**, map your match keys by selecting:
- a warehouse **Column name**
- an **Identifier type** (for example, `Email`, `Phone`, or `Mobile ID`)
1. Select the identifiers you want to match into the MatchBooster identity graph, such as:
- Email (raw or hashed — Hightouch hashes raw values automatically)
- Phone number (raw or hashed)
- Additional identifiers like first name, last name, and address (for name/address matching)
- Device IDs, IP addresses, or mobile ad IDs (for MatchBooster Anonymous use cases)
2. Ensure required combinations are provided for name/address matching (e.g., name + address + ZIP).
Click the **magic wand** icon to automatically map match keys.
For name and address matching, you must provide **first name**, **last name**, and at minimum the **first address line** and **postal code**. Including city and state improves accuracy.
---
### Match keys reference guide
Match Booster uses supported **match keys** to match your records into Hightouch’s identity graph. The match keys you provide determine:
- which records can be matched
- what additional identifiers Match Booster can append to your model
- what enriched identifiers you can map into your destinations
Match keys are selected in the `Match Booster` tab by mapping a warehouse **Column name** to an **Identifier type**.
#### Known users (MB Known)
Use this option when you already have user-level identifiers like email, phone number, MAIDs, or postal address information.
**Supported inputs:**
| Match key | Supported input format | Notes |
| --- | --- | --- |
| Email | Plain text, MD5, SHA256 | Email can be unhashed or hashed |
| Phone | Plain text | Phone must be provided as plain text |
| MAIDs | Plain text | Mobile advertising IDs |
| Full postal address | Plain text | Includes first name, last name, street 1, street 2, city, state, postal code |
Match Booster can **append** the following enriched identifiers to your model:
| Enrichment appended | Notes |
| --- | --- |
| Email | Hashed |
| Phone | Hashed |
| MAIDs | |
| IP Address (CTV) | Format can vary |
| First name | |
| Last name | |
| City | |
| State | |
| Postal code | |
#### Anonymous users (MB Anonymous)
Use this option when you don’t have known user identifiers, but you do have IP address data.
**Supported inputs:**
| Match key | Supported input format | Notes |
| --- | --- | --- |
| IP address | Plain text | Used for anonymous matching |
Match Booster can **append** the following enriched identifiers to your model:
| Enrichment appended | Notes |
| --- | --- |
| Email | Hashed |
| Mobile Ad ID | |
#### Conversion events (MB Conversions)
Use this option when you’re syncing conversion records and want to increase matchability downstream.
**Supported inputs:**
| Match key | Supported input format | Notes |
| --- | --- | --- |
| Email | Plain text, MD5, SHA256 | Email can be unhashed or hashed |
| Phone | Plain text | Phone must be provided as plain text |
Match Booster can **append** the following enriched identifiers to your model:
| Enrichment appended | Notes |
| --- | --- |
| Email | Hashed |
| Phone | Hashed |
| First name | |
| Last name | |
| City | |
| State | |
| Postal code | |
---
### Set enrichment schedule
Choose how frequently Hightouch should enrich your data against the identity graph.
Enrichment creates a secure mapping between your hashed primary keys and the third-party identity graph without storing raw PII.
---
### Add enrichment filters (on parent models only)
Use enrichment filters to define a subset of your parent model to be enriched. For example:
- limit enrichment to specific regions
- enforce consent rules by enriching only consented users
---
### Run initial enrichment
Click `Initialize Match Booster` to run Match Booster for the first time.
The initial run performs a full enrichment and may take longer for large datasets. Subsequent runs only enrich new or updated records.
Initialization is a one-time step that can take **up to 72 hours**.
Select `Email me when enrichment is complete` to avoid monitoring progress manually.
---
## Enable Match Booster in a sync
After enrichment is enabled, you can configure your sync to send enriched identifiers to [supported destinations](/match-booster/overview#supported-destinations).
1. Open a sync and go to the `Configuration` tab. (Alternatively, create a new sync.)
2. In the field mapping section, toggle `Enable Match Booster for higher audience reach ✨` on.
3. Choose a **targeting level**:
- **Person**: target individuals
- **Household**: target households (requires Household Expansion)
- **CTV**: target consumers on connected TV devices
4. Review which identifiers will be sent for matching.
- Hightouch automatically selects the best available enriched identifiers for each destination. Some destinations allow you to remove specific identifiers if needed.
You do not need to manually map enriched Match Booster fields. Hightouch automatically sends the appropriate identifiers.
If you just enabled Match Booster for the first time, wait for the initial enrichment run to complete before running your sync.
---
## Monitor match rates
Hightouch surfaces match rates for syncs to paid advertising destinations, whether or not Match Booster is enabled.
To view match rates for a sync:
1. Go to `Activation` → `Syncs`.
2. Select the sync you want to review.
3. Open the `Overview` tab.
4. In the right-hand panel, select `Matched users` → `See details`.
Clicking the match rate opens a details view that shows:
- external status of the sync
- matched users
- match percentage
- links to your audiences in the ad platform
---
## Matching into the Match Booster identity graph
**URL:** https://hightouch.com/docs/match-booster/matching-options
**Description:** Match Booster takes your first-party identifiers across multiple data types and resolves them to people and devices in its identity graph to improve audience reach and match rates across ad platforms.
**Section:** Match Booster
## Overview
Match Booster takes your first-party identifiers across multiple data types and resolves them to people and devices in its identity graph to improve audience reach and match rates across ad platforms.
## Supported match types
### Consumer data
Match Booster supports matching profiles to targetable people and devices across ad platforms. You may be collecting emails or phone numbers from consumers, yet those identifiers have low match rates in ad platforms. Audience boosting of consumer data entails enriching your first-party data with additional emails, phones, device IDs, and cookie IDs to increase match rates.
### Work data
Match Booster supports matching B2B employees to consumer profiles and devices. You may be collecting work emails from prospects or have purchased a list of prospects from a third party. However, audience match rates of those emails are low. Audience boosting of work data entails enriching your first-party data with additional consumer emails, phones, device IDs, and cookie IDs to increase match rates.
### Address data
Match Booster supports matching your first-party name and address data to our graph of person-level identifiers that most ad platforms can match on, including hashed emails, phone numbers, and device IDs.
### Anonymous web visitor data (IP address)
Match Booster supports matching anonymous visitor IP addresses into known consumers and devices using reverse IP-lookup technology. By providing a first-party IP address, Hightouch will filter the data to ensure it’s coming from a residential IP address. If it is, Hightouch’s identity graph will match that IP address to the people and devices associated with that household and sync those profiles to downstream ad platforms for targeting and measurement.
---
## Match Booster Offerings
**URL:** https://hightouch.com/docs/match-booster/offerings
**Description:** Increase match rates for your audiences and conversion events in advertising platforms.
**Section:** Match Booster
*Learn about the offerings under the **Match Booster** product line.*
## Audience boosting
Match Booster supports boosting the size of your first-party audiences across walled gardens, DSPs, SSPs, CTV platforms, and publisher direct destinations.
### Overview
Match Booster supports boosting the size of your first-party audiences by matching more of your first-party customer records to users across ad platforms. Hightouch can match audiences across a number of different record types, including consumers, B2B buyers, households, and devices.
### Use cases
* **Custom Targeting**: increase audience size to avoid over-bidding on a small segment
* **Suppression**: better suppress ads on existing customers or low-likelihood audiences
* **Lookalikes**: increase precision of seeds for lookalike audiences
* **Bid Optimization**: increase the accuracy of audiences used to modify bid strategies for new vs returning customers
* **Offer Optimization**: increases the accuracy of the qualified audience for special offers, pricing, and discount offers, like loyalty members who qualify for discounted hotel rates on search platforms
---
## Conversion event boosting
### Overview
Match Booster supports boosting the match rate of online and offline conversion events that you send to ad platforms, improving attribution accuracy and conversion optimization.
Note: Conversion event boosting does not cause the same event to be sent into the ad platform multiple times.
### Use cases
* **Attribution and measurement**: match more conversions to ad impressions and clicks, improving the accuracy of attribution and incrementality measurement
* **Budget allocation**: improved attribution enables ad platforms to make better budget allocation decisions across ads, ad groups, and campaigns
* **Conversion optimization**: improve ROAS by feeding better training data to the ad platform’s AI systems responsible for predictive targeting, dynamic creative optimization, and automated bidding
---
## Anonymous visitor boosting
### Overview
Match Booster supports boosting of anonymous visitors via IP addresses. You may be collecting user events and their associated IP addresses, even if the user does not provide personally identifiable information like emails and phone numbers. Hightouch’s anonymous identity graph can identify the most common emails and device IDs associated with an IP addresss using a reverse IP-lookup, and send those identifiers to ad platforms to improve the size of your web retargeting audiences.
Note: Hightouch rigorously cleans observed linkages between IP addresses, HEMs, and MAIDs to eliminate public locations.
### Use cases
* Web retargeting: retargeting anonymous visitors to your web properties
---
## CTV device boosting
### Overview
Match Booster supports boosting audiences to them across Smart TV devices like Roku Advertising, LG Ad Solutions, Samsung Ads, and Vizio Ads. Most connected TV (CTV) devices rely on CTV IDs and IP Address matching to target viewers with ads. Hightouch’s identity graph translates your first-party identifiers into these identifiers to increase the number of people and households you can target for your CTV direct campaigns.
Note: Hightouch rigorously cleans observed linkages between IP addresses, HEMs, and MAIDs to eliminate public locations.
### Use cases
* **User acquisition**: acquire new app installs and subscribers through personalized ads targeting and more effective suppression
* **Subscription upsell**: upsell customers from a free to paid tier on your CTV application
* **Tune-In**: drive consumers to your app to tune-in to specific shows, movies, or live events by displaying native ads on the Smart TV homescreen and explore pages
---
## Match Booster Overview
**URL:** https://hightouch.com/docs/match-booster/overview
**Description:** Increase match rates for your audiences and conversion events in advertising platforms.
**Section:** Match Booster
Hightouch **Match Booster** is a suite of built-in data onboarding products that enhance the match rates of your audiences, conversion events, and measurement feeds across [various paid advertising platforms](#supported-destinations).
The following are common use cases for Match Booster:
- **Custom audience targeting**: increase the reach of your targeting and retargeting campaigns
- **Suppression**: ensure your existing customers are excluded from your campaigns
- **Conversion optimization**: improve attribution accuracy and train the ad platform’s AI-powered targeting, bidding, and budgeting algorithms
- **Anonymous targeting**: target anonymous visitors on your digital properties
- **CTV targeting**: target consumers on their Smart TVs to drive signups and tune-in
Match Booster enhances your audiences, conversion events, and measurement feeds with additional identifiers from Hightouch’s licensed third-party identity graph. These additional identifiers help ad platforms better match their limited identity data with your first-party data. Ultimately, this enables better targeting, increased ROAS, and more accurate measurement.
[**Explore the interactive architecture diagram →**](https://architecture.hightouch.com/?from=docs&product=match-booster&utm_source=hightouch-docs&utm_medium=referral&utm_campaign=architecture-explorer&utm_content=match-booster)
See how data flows and where Match Booster fits in your stack.
---
## Modern Warehouse-Native Onboarding
**Match Booster** is built on a modern, warehouse-native architecture that eliminates the need for data replication. Your data is never persisted on Hightouch infrastructure, besides a hash of the primary key of your model.
Hightouch achieves this by using data sharing to match our identity graph to your first-party data and maintaining a pseudonymized mapping of our third-party identity graph on our premises for future activation. When data is finally activated, the third-party data from our identity graph is then queried and appended to your first-party data at sync time. This means you never have to store regulated third-party data in your infrastructure, and Hightouch never has to store your first-party data on our premises.
**Warehouse-Native Benefits**
- **Security**: since your first-party data is never stored with an intermediary, you significantly reduce security and privacy risks.
- **Direct data access**: Match Booster queries your warehouse directly. This means whenever a new data point is added to your first-party data, you get immediate access. No extra pipelines are needed.
- **Faster refreshes**: Hightouch infrastructure supports enrichment and syncing on an hourly and daily basis. This means you don't need to wait weeks for your data to be onboarded.
- **Simpler workflow**: since Hightouch is a simple enrichment and pass-through layer, you avoid all the headaches of maintaining segments in another tool.
---
## Zero-Fee Activation
Unlike many legacy data onboarding providers, Hightouch does not charge additional revenue share fees, CPM fees, or % of media fees for activating data to ad platforms or marketplaces.
This means that you can sync audiences to SSPs, DSPs, and publisher direct destinations without incurring hidden fees that eat into your working media budget.
For customers using Hightouch for monetization use-cases, like retail media networks, it means that you can list audiences on DSP marketplaces and SSPs without giving up a portion of your revenue to a middleman identity provider.
---
## How does Match Booster work?
Match Booster increases match rates by enriching first-party data with two types of additional identifiers sourced from third-party vendors.
1. **Offline data** such as email addresses and phone numbers
2. **Online data** such as Mobile Device IDs
Hightouch uses the additional identifiers to build a custom [identity graph](https://blog.hubspot.com/service/identity-graph) for each [model with Match Booster enabled](#model-configuration). You can then sync those identifiers to the [supported destinations](#supported-destinations) as audiences or conversion events, alongside your own first-party data.
In this architecture, your data is not stored at rest outside of your own infrastructure. Additional identifiers are appended in-flight during a sync to the destination.
---
## Data vendors and data validation
**Data Vendors**
Hightouch sources third-party data from leading brand-name providers and cutting-edge emerging startups. Hightouch stores a full copy of third-party identity graphs on our premises so that your data does not need to leave your environment.
The list of data vendors that Hightouch works with will be made available in any contractual agreement.
**Data Cleaning**
Hightouch rigorously cleans data provided from third-party vendors. These processes include filtering out abnormally large profiles, observing the frequency and recency of observed device linkages, prioritizing verified data (like emails), and using additional data points (like IP addresses, names) to filter out outliers.
---
## Regional Coverage
Match Booster currently maintains an identity graph in the following regions & countries:
**North America (NAMER)**
- United States of America (US)
Hightouch is consistently evaluating support for other regions. For inquiries or to request support for a new region, please contact your account representative.
---
## Enrichment identifiers
Hightouch supports the following enrichment identifiers to match on:
- Email
- Hashed Email (SHA256 or MD5)
- Phone
- Hashed Phone (SHA256 or MD5)
- Mobile Device ID
- IP Address
- Full Name + Postal Address
- First Name
- Last Name
- Street
- Street 2
- City
- State
- Postal Code
---
## Supported destinations
You can use Match Booster on syncs to the following destinations:
**Custom Audiences**
- [Amazon Ads DSP](/destinations/amazon-ads)
- [Criteo](/destinations/criteo)
- [Digital Turbine](/destinations/digital-turbine)
- [Dynata](/destinations/dynata)
- [Meta Custom Audiences](/destinations/facebook)
- [Google Ads](/destinations/google)
- [Google Display & Video 360](/destinations/display-video)
- [InMobi](/destinations/inmobi)
- [iSpot.tv](/destinations/ispot-tv)
- [LiveIntent](/destinations/liveintent)
- [LG Ad Solutions](/destinations/lg-ad-solutions)
- [LinkedIn Ads](/destinations/linkedin)
- [Microsoft Bing Ads Audiences](/destinations/bingads)
- [Pinterest Ads](/destinations/pinterest-ads)
- [Reddit Ads](/destinations/reddit-ads)
- [Rokt](/destinations/rokt)
- [Roku Ads](/destinations/roku-ads)
- [Samsung Ads](/destinations/samsung-ads)
- [Snapchat](/destinations/snapchat)
- [StackAdapt](/destinations/stackadapt)
- [Taboola](/destinations/taboola)
- [TikTok](/destinations/tiktok)
- [The Trade Desk](/destinations/tradedesk)
- [X](/destinations/x)
- [Vistar Media](/destinations/vistar-media)
- [Vizio Ads](/destinations/vizio-ads)
- [Yahoo DSP](/destinations/yahoo)
**Conversion Events**
- [Meta Conversions](/destinations/meta-conversions)
- [Google Ads](/destinations/google)
- [TikTok](/destinations/tiktok)
The catalog of supported destinations is continually growing. Please if there are additional ad platforms you're interested in using Match Booster on.
---
## Targeting levels
**URL:** https://hightouch.com/docs/match-booster/targeting-levels
**Description:** Match Booster supports targeting at the person, device, and household level.
**Section:** Match Booster
## Supported targeting levels
With Match Booster, you can target audiences and track conversions at the **person level** or **household level**. Targeting levels are defined per sync (for example, an audience sync or a conversion feed).
Person-level matching is Hightouch’s default targeting level. Household-level targeting must be enabled in the model’s Match Booster configuration before it can be applied to a sync.
Targeting levels cannot be adjusted after the sync is enabled. If you want to change targeting levels for an audience, create a new sync with the same configuration and adjust the targeting level before activating it.
Consider using Templates to create separate configurations for person-level and household-level audiences so that users can easily sync campaigns for both types.
### Person-level matching
Person-level matching links your customer and device records to a single individual, improving match rates while maintaining a high level of precision.
Person-level matching works by finding deterministic linkages between your first-party identifiers and people in Match Booster’s third-party identity graph. When matching to devices, Match Booster can be configured to isolate the most likely user of a device or broaden the scope by including shared devices in the match criteria (for example, a shared iPad or tablet).
### Household-level matching
Household-level matching uses Household Expansion to match a single record to a residential address in the identity graph, and then expands targeting to reach all of the people and devices in that household.
### Common use cases
Household Expansion is especially effective for brands and media networks whose products are purchased or used by an entire household.
- **Retail marketing and retail media networks** – Although retailers are improving their ability to match in-store transactions to individual buyers through loyalty programs, those memberships are often shared across family members. Household Expansion helps retailers target, suppress, and monetize the entire household based on past purchase behavior.
- **Streaming TV** –m Household members typically share a single streaming TV login, even when they have separate profiles within the application. Household Expansion helps media companies target, suppress, measure, and monetize audiences more effectively by ensuring every member of the household is being matched.
- **Automotive** – Automotive sales are often family decisions due to the high price and shared use. Household Expansion allows businesses to target, suppress, measure, and monetize audiences based on purchase behavior across all members of the household.
- **Telecom and utilities** – Businesses providing household services, including utilities, HVAC, home appliances, and telecommunications (phone and internet), typically sell to a household unit. Using Household Expansion, they can match a single customer record to the entire household for more effective suppression of existing customers, targeting of high-intent leads, or measurement across multiple household touchpoints.
---
## Consent Manager
**URL:** https://hightouch.com/docs/events/consent/consent-manager
**Description:** Hightouch Events Consent Manager UI
**Section:** Hightouch Events
Hightouch Events offers the Consent Manager UI for managing user consent and tracking preferences. Users can opt in/out of specific categories of data collection, and those preferences are synced to your configured destinations.
## Prerequisites
In order to use Consent Manager UI, you must use the Browser (Javascript) SDK snippet on your site.
Note: `e.load('WRITE_KEY')` is removed from the snippet since the Consent Manager will load the Browser SDK for you.
```html
```
## Installation
The Consent Manager is available via [NPM](https://npmjs.org/@ht-sdks/consent-manager). It can be installed on any site using a `
```
```jsx
return (
)
}
```
## API
For a full list of features and options, see the [README](https://github.com/ht-sdks/consent-manager).
---
## Hightouch Events + OneTrust
**URL:** https://hightouch.com/docs/events/consent/onetrust
**Description:** Hightouch Events with OneTrust consent integration
**Section:** Hightouch Events
Hightouch Events offers a pre-built integration for managing user consent with OneTrust.
## Prerequisites
Make sure to setup your consent categories in OneTrust before integrating with Hightouch Events.
The integration expects the [OneTrust Banner SDK](https://my.onetrust.com/articles/en_US/Knowledge/UUID-66bcaaf1-c7ca-5f32-6760-c75a1337c226) to be available in order to interact with OneTrust. This library derives the group IDs that are active for the current user from the `window.OneTrustActiveGroups` object provided by the OneTrust SDK ([more info](https://my.onetrust.com/articles/en_US/Knowledge/UUID-66bcaaf1-c7ca-5f32-6760-c75a1337c226)).
## Installation
The Hightouch Events + OneTrust integration is available via [NPM](https://npmjs.org/@ht-sdks/events-sdk-js-consent-wrapper-onetrust). It can also be installed using a `
```
## API
For a full list of features and options, see the [README](https://github.com/ht-sdks/events-sdk-js-mono/tree/master/packages/consent/consent-wrapper-onetrust).
---
## Manage contracts
**URL:** https://hightouch.com/docs/events/contracts/management
**Description:** Define, validate, and version event schemas using contracts to ensure reliable event data.
**Section:** Hightouch Events
| Audience | How you’ll use this article |
| --------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| **Marketing teams** | Understand how contracts protect downstream tools from incomplete or inconsistent event data. |
| **Data teams** | Define required event properties, manage schema versions, and control how undeclared fields are handled. |
| **Engineering teams** | Implement JSON Schema validation, configure enforcement rules, and manage context field validation. |
## Overview
Hightouch Contracts helps marketing and data teams plan, manage, and enforce the data they track through event collection.
Contracts have two components::
- A **contract event** defines the name, type, description, and validation schema for a single event (for example, `Order Completed`).
- A **contract** is a collection of contract events that can be applied to one or more event sources (for example, `E-commerce Events`).
Each contract event belongs to a single contract. A contract can be applied to multiple event sources.
---
## Create a contract
To create a new contract:
1. Go to **Event collection → Contracts**.
2. Click **New contract**.
3. Complete the setup flow:
### 1. Name & description
Provide a contract name and optional description.
### 2. Select sources
Choose the event sources this contract should monitor.
Hightouch will validate events coming from these sources against the contract schema.
### 3. Enforcement
Configure how Hightouch handles events that don’t match the contract.
#### Undeclared event types
When an event type is sent that is not defined in the contract:
- **Allow** — Sync the event with a warning.
- **Block** — Prevent the event from syncing.
After saving, you can begin adding contract events.
---
## Add a contract event
To define an event within a contract:
1. Open your contract.
2. Click **Add event**.
3. Configure the event in the **Configuration** section:
- **Type** (for example, `Track`)
- **Name** — must exactly match the event name sent in your code (including capitalization and spacing).
---
### 1. Define the event schema
When creating a contract event, the most important step is defining its **validation schema**.
Under **Schema**, define the structure of the event payload.
You can:
- Use the visual schema builder
- Toggle **Edit as JSON** to directly modify the schema
- Create multiple schema versions
Hightouch uses [JSON Schema Draft 7](https://json-schema.org/draft-07/json-schema-release-notes). This allows for highly complex validation of event payloads, including [validating field values with regular expressions](https://json-schema.org/understanding-json-schema/reference/regular_expressions).
#### Define event properties
Under **Properties**, you can:
- Add fields using **Add property**
- Set field types (for example, `String`, `Number`, `Array`)
- Mark fields as **Required**
- Define nested objects or array element types
These rules determine whether an event is considered valid.
#### Configure context validation
Events include a `context` object, which contains metadata such as device information, page details, or SDK-provided fields.
Under **Context**, you can define validation rules for context fields.
#### Include built-in context fields
Enable **Include built-in context fields** to automatically allow [standard SDK fields](/event-spec#automatically-captured-fields) (like device info, page data, IP address, etc.) without having to define them manually.
If you define a context field that shares a name with a built-in one, your definition takes precedence.
---
### 2. Manage schema versions
Contracts support **multiple versions of the same event schema**.
Versioning is useful when it is difficult to migrate your codebase to use the latest payload. For example, when an iOS app takes a while to be updated across all devices.
After creating your first event, you can create additional versions of the
schema. The first version is the "default" version, which applies to events that
do not specify a version in the payload. It is recommended that your events
follow an incrementing pattern (for example, v1, v2, v3), although not strictly
required.
In the **Schema** section, you can:
- Create a new version using the **+** button
- Switch between versions using the version dropdown (for example, `v2`, `default`)
- Edit versions independently
When invoking an event, you can specify the schema version via:
```
client.track('Order Completed', {
...
}, {
context: {
htevents: {
schemaVersion: "v2"
}
}
});
```
---
### 3. Enforcement behavior for fields
Contracts define what happens when an event contains:
- **Undeclared properties** (fields not defined in the schema)
- **Invalid fields** (missing required fields or incorrect types)
#### Undeclared fields
For undeclared fields within an event, you can choose:
- **Allow** — Sync the event with a warning.
- **Filter** — Sync the event but remove undeclared fields.
- **Block** — Prevent the event from syncing.
#### Invalid fields
For invalid fields, you can choose:
- **Allow** — Sync the event with a warning.
- **Block** — Prevent the event from syncing.
These settings apply to both event properties and context fields.
---
## Handle violations
**URL:** https://hightouch.com/docs/events/contracts/violations
**Description:** Understand the types of contract violations and configure how Hightouch enforces them.
**Section:** Hightouch Events
## Overview
A **violation** occurs when an event sent through an event source does not satisfy the contract applied to that source.
Violations help you identify incomplete, inconsistent, or unexpected event data before it impacts downstream tools.
Events can cause 3 different types of violations:
- An **undeclared event** violation occurs when a specific event is not
specified in the contract
- e.g. `Button Clicked` event is sent to Hightouch, but it is not in the
E-commerce Events contract
- An **undeclared field** violation occurs when a field in an event is not
specified in the validation schema
- e.g. `user_name` field is sent to Hightouch in the `Order Completed`
payload, but it is not specified in the validation schema
- A **schema validation** violation occurs when a field does not pass the schema
validation defined by JSON Schema rules
- e.g. `userId` field is passed in as a `string` when the schema violation
defines the field as type `number`
All violations are written to a separate `event_violations` table in your
destination, regardless of how violation enforcement is configured.
This table has the schema:
- **payload**: JSONB
- **event_name**: string (derived from payload)
- **event_type**: string (derived from payload)
- **schema_version**: string (derived from payload)
- **has_undeclared_schema**: boolean
- **has_undeclared_fields**: boolean
- **has_schema_violated**: boolean
- **violations**: string[]
### **Violation enforcement**
Hightouch allows you to configure how to handle violations at both the contract
and event level. Depending on the cleanliness of your current event tracking, you may want
to allow violations to be written to your main event tables.
This table represents the different ways Hightouch can handle a violation. The
bolded actions represent the default behavior.
| Schema validation | Undeclared field | Undeclared event |
| :---------------: | :--------------: | :--------------: |
| **Allow** | **Allow** | **Allow** |
| Block event | Block event | Block event |
| | Omit fields | |
Configure how to handle undeclared events at the contract level.
Schema violations and undeclared fields can be configured at an event-level on the event schema page.
---
## Understand event types and payload structure
**URL:** https://hightouch.com/docs/events/event-spec
**Description:** Learn how Hightouch Events structures identify, track, page, screen, and group events—including required fields, identity behavior, and example payloads.
**Section:** Hightouch Events
| Audience | How you’ll use this article |
|--------|-----------------------------|
| **Marketing teams** | Understand which event types exist and what they represent. |
| **Data teams** | Validate event payloads and understand how events map to tables. |
| **Engineering teams** | Implement SDKs and ensure correct identity and schema usage. |
## Overview
Hightouch Events provides a standardized way to collect **user traits** and **user behavior** across your applications, while allowing teams to control event structure and identity behavior.
This article defines the **event tracking specification** used by Hightouch:
- Supported event types
- Required and optional fields
- Identity behavior across anonymous and known users
- Shared payload structure
- Example JSON payloads
Use this guide when implementing SDKs, reviewing event data, or validating incoming events.
---
## Supported event types
Hightouch supports the following event types:
| Event type | Purpose |
|----------|---------|
| [`identify`](#identify-events) | Identify a known user and set user-level traits |
| [`track`](#track-events) | Record an action a user performed |
| [`page`](#page-events) | Record a browser page view |
| [`screen`](#screen-events) | Record a mobile app screen view |
| [`group`](#group-events) | Associate a user with a group or account |
Each event type builds on a shared base structure and adds its own required fields.
---
## Shared event structure
All event types share a common base structure. This ensures events can be validated, governed, and activated consistently across destinations.
### Required fields
- `type` — The event type (`identify`, `track`, `page`, `screen`, `group`)
- `messageId` — A unique identifier for the event
- At least one identifier: `anonymousId` or `userId`
- `originalTimestamp` — When the event occurred
- `sentAt` — When the event was sent by the SDK or API
### Optional fields
- `context` — Automatically collected metadata (device, OS, SDK version, etc.)
- `integrations` — Destination-specific flags
- `traits` or `properties` — Metadata attached to the event:
- **Traits** describe the user (for example, email or plan)
- **Properties** describe the action or context (for example, price or product ID)
---
### Identify events
**Identify events** capture data about *who* a user is. They are used to:
- Associate anonymous activity with a known user
- Set or update user-level traits
- Link events across devices and sessions
For example, you might use Identify events to power follow-up email campaigns, segment users by location or plan, or link anonymous activity to authenticated behavior.
Users often generate events *before* they log in or sign up. During this time, events are associated with an `anonymousId`.
When you later send an Identify event that includes both a `userId` and the same `anonymousId`, subsequent events from that device include both identifiers. You can then join earlier anonymous-only events to the known user in your warehouse models using the shared `anonymousId`.
---
#### Components of Identify events
- **`anonymousId`**
A device- or browser-level identifier generated by the SDK to associate events from the same user before they are identified. This value is included automatically in other event types (e.g., Track events).
An `anonymousId` typically persists across multiple sessions on the same device or browser.
- **`userId`**
A stable, persistent identifier for a known user (for example, it's common to use an ID generated by your database).
Avoid using mutable values such as email addresses.
- **`traits`**
Optional user attributes, such as email, name, or age.
#### Example Payload for Identify Event
```json
{
"type": "identify",
"messageId": "111e984d-c93c-444c-b29c-f499a117c500",
"traits": {
"email": "kevin@hightouch.io"
},
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"originalTimestamp": "2023-09-22T18:05:47.063Z",
"sentAt": "2023-09-22T18:05:47.064Z"
}
```
---
### Track events
**Track events** capture data about *what* a user did, such as completing a purchase or clicking a button.
Additional details about the action are included in the `properties` object. For example, for a `Purchase Complete` event, you may want to include
the `product ids` of the purchased products.
When events are stored in the warehouse, the `event` field is used to generate [table
names](./warehouse-schema#per-event-tables) in the warehouse. Additionally, the
`properties` field is autoconverted into
[columns](./warehouse-schema#per-event-tables) in the warehouse.
#### Example Payload for Track Event
```json
{
"type": "track",
"messageId": "7b1e2872-4ebb-4779-b9ad-e7aa1a8eab82",
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"event": "Purchase completed",
"properties": {
"cart_total": 24.99
},
"originalTimestamp": "2023-09-22T18:12:00.584Z",
"sentAt": "2023-09-22T18:12:00.585Z"
}
```
### Page events
**Page events** track the webpages users interact with. In most situations, you
should automatically call the page load event when a page is loaded.
For single-page applications, trigger page events on route changes.
#### Example Payload for Page Event
```json
{
"type": "page",
"messageId": "bb9fcbbd-9e19-4bfd-a726-7b1e7e44f8ba",
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"properties": {
"title": "Home"
},
"originalTimestamp": "2023-09-22T18:05:47.061Z",
"sentAt": "2023-09-22T18:05:47.063Z"
}
```
### **Screen events**
**Screen events** are the equivalent of page views for _mobile apps_. Like
page events, you should automatically call the screen event when a mobile
screen view is loaded.
#### Example Payload for Screen Event
```json
{
"type": "screen",
"messageId": "bb9fcbbd-9e19-4bfd-a726-7b1e7e44f8ba",
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"properties": {
"title": "Home"
},
"originalTimestamp": "2023-09-22T18:05:47.061Z",
"sentAt": "2023-09-22T18:05:47.063Z"
}
```
### **Group events**
Group events let you associate users with a larger group. For example, a user
may be part of an organization, or family.
The `traits` field is optional, and can include information on traits on the
group. For example, if linking a user to a company, you may include traits on
the company such as the company size or location.
#### Example Payload for Group Event
```json
{
"type": "group",
"messageId": "d87596da-5f1e-40f4-a28a-7099290548b7",
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"groupId": "456",
"traits": { "company_name": "Hightouch" },
"originalTimestamp": "2023-09-22T18:46:23.098Z",
"sentAt": "2023-09-22T18:46:23.099Z"
}
```
### Automatically captured fields
Hightouch SDKs automatically populate common context fields about the device, environment, and session. The table below shows which fields are captured by each SDK.
| CONTEXT FIELD | Javascript | iOS | Android | Description |
| --- | --- | --- | --- | --- |
| app.name | | ✅ | ✅ | Name of the application. |
| app.version | | ✅ | ✅ | Version of the application. |
| app.build | | ✅ | ✅ | Build of the application. |
| campaign.name | ✅ | | | Name of the campaign. |
| campaign.source | ✅ | | | Source of the campaign. |
| campaign.medium | ✅ | | | Medium of the campaign. |
| campaign.term | ✅ | | | Term of the campaign. |
| campaign.content | ✅ | | | Content of the campaign. |
| device.type | | ✅ | ✅ | Type of the device. |
| device.id | | ✅ | ✅ | ID of the device. |
| device.advertisingId | | ✅ | ✅ | Advertising ID of the device. |
| device.adTrackingEnabled | | ✅ | ✅ | If ad tracking is enabled on the device. |
| device.manufacturer | | ✅ | ✅ | Manufacturer of the device. |
| device.model | | ✅ | ✅ | Model of the device. |
| device.name | | ✅ | ✅ | Name of the device. |
| library.name | ✅ | ✅ | ✅ | Name of the library. |
| library.version | ✅ | ✅ | ✅ | Version of the library. |
| ip | ✅ | ✅ | ✅ | Current user’s IP address. |
| locale | ✅ | ✅ | ✅ | Locale string of the user. |
| network.bluetooth | | | ✅ | Bluetooth information. |
| network.carrier | | ✅ | ✅ | Carrier information about the network connection. |
| network.cellular | | ✅ | ✅ | Cellular information about the network connection. |
| network.wifi | | ✅ | ✅ | WiFi information about the network connection. |
| os.name | | ✅ | ✅ | Name of the operating system. |
| os.version | | ✅ | ✅ | Version of the operating system. |
| page.path | ✅ | | | Path of the current page in the browser. |
| page.referrer | ✅ | | | Referrer of the current page in the browser. |
| page.search | ✅ | | | Search of the current page in the browser. |
| page.title | ✅ | | | Title of the current page in the browser. |
| page.url | ✅ | | | URL of the current page in the browser. |
| screen.density | | | ✅ | Density of the device’s screen. |
| screen.height | | ✅ | ✅ | Height of the device’s screen. |
| screen.width | | ✅ | ✅ | Width of the device’s screen. |
| traits | | ✅ | ✅ | Traits of the user. |
| userAgent | ✅ | | ✅ | User agent of the device making the request. |
| userAgentData | ✅ | | | Information about the user’s timezone. |
| timezone | | ✅ | ✅ | Information about the user’s timezone. |
- IP Addresses are captured by Hightouch's servers when a message is received for client-side events and is not captured by Hightouch's libraries.
- userAgentData is only collected if the Client Hints API is available on the browser.
---
## Event streaming
**URL:** https://hightouch.com/docs/events/event-streaming
**Description:** Stream events from Hightouch to downstream tools in real time for activation, personalization, and time-sensitive workflows.
**Section:** Hightouch Events
| Audience | How you’ll use this article |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| **Marketing teams** | Understand the end-to-end event streaming workflow and how event data powers real-time activation, personalization, and experimentation. |
| **Data teams** | Learn how streamed events are filtered, mapped, validated, and delivered to downstream tools alongside warehouse-based workflows. |
| **Engineering teams** | Follow the recommended implementation path, from source setup and validation to configuring event streaming syncs and monitoring delivery. |
## What you’ll learn
By the end of this article, you’ll be able to:
1. [Decide when to use event streaming](#when-to-use-event-streaming)
2. [Understand how event streaming works](#how-event-streaming-works)
3. [Set up an event streaming sync](#set-up-event-streaming)
4. [Configure event types and filters](#configure-streamed-events)
5. [Map event and user data to a destination](#field-mappings)
6. [Monitor delivery and troubleshoot errors](#errors)
7. [Set up alerting](#alerting)
---
## Overview
**Event streaming** lets you send events collected by Hightouch directly to destinations in real-time, without going through the warehouse. Events stream **in seconds**, which is useful for latency-sensitive use cases where timing matters such as: triggering messages, updating consent, or powering real-time personalization.
Event streaming is a type of [**Event Sync**](/syncs/types-and-modes#event-syncs) that takes events from an Event Source and syncs them to an Event Destination in real-time.
---
## When to use event streaming
Use event streaming when your workflow depends on **something that just happened**.
Examples:
- **Post-purchase workflow** to send upsell offers to users immediately following a purchase
- **Ad impression measurements** so you can immediately pause ads that surpass their budget
- **Other time-sensitive emails** or messages sent to users after significant events like a demo request, new user sign up, and missed call follow-up
- **Update users’ consent** or contact status immediately after a change
### When to use reverse ETL instead
Reverse ETL is a better fit when timing is less critical and **context matters more than immediacy**.
Use reverse ETL when you need:
- **Warehouse-enriched or aggregated data**
(joins across tables, LTV, days since last purchase, ML scores)
- **Audiences built from historical behavior**
(for example, users who purchased 3+ times in the last 90 days)
- **Ongoing monitoring and governance at scale**
(shared models and metrics used across many workflows)
For example, reverse ETL works better for:
- Conversion events that benefit from complete identity and revenue context
- Audience membership that should change over time, not from a single event
- Features or scores computed from many events (churn risk, power users, high-value cohorts)
→ Learn more about [Reverse ETL syncs](/syncs/types-and-modes).
---
## How event streaming works
With event streaming, events flow through Hightouch and are forwarded to destinations in near real time.
At a high level, each streamed event follows this path:
1. The event is received from an **event source** (SDK, HTTP API, streaming source, or webhook).
2. (Optional) **Contracts** validate the event’s structure, naming, and required fields.
3. (Optional) An **Event Function** transforms or enriches the payload.
4. **Filters** decide whether the event should be forwarded for a given event type.
5. **Field mappings** shape event and user data to match the destination schema.
6. The event is delivered to the **destination**, with automatic retries on failures.
This flow happens **per event**, rather than on a schedule. For warehouse-based, batch processing, use event storage and reverse ETL instead of event streaming.
---
### Optional validation and transformation
You can optionally add additional controls to this flow:
- **Data contracts**
Validate event structure, naming, and required fields before events are delivered.
→ See [Managing contracts](/events/contracts/management)
- **Event functions**
Modify or enrich events in-flight before they are sent to the destination.
→ See [Functions overview](/events/functions/overview)
These steps are optional and are commonly introduced as your event implementation grows.
---
## Set up event streaming
### Prerequisites
Before you begin, make sure you have:
- An **event source** that’s already set up and receiving events
- A **destination** where you want to forward those events
---
### Step 1: Create a new event sync
1. Go to **`Event Collection → Event Syncs`**
2. Click **Add sync**
---
### Step 2: Select an event source
Choose the source that emits the events you want to stream.
If you don’t see the expected source, confirm that events are being received:
1. Go to **`Event Collection → Event Sources`**
2. Select the source
3. Open the **Debugger** tab
4. Trigger the event in your app or site and verify it appears in the debugger
---
### Step 3: Select an event destination
Event syncs can either **store events** or **forward events**, depending on your use case:
- **Event storage** destinations write events to your warehouse
- **Event forwarding** destinations send events to downstream tools in real time
Select a destination under **Event forwarding**.
Use the **Docs** link in the UI to review destination-specific requirements.
---
### Step 4: Name and create the sync
Enter a clear, descriptive name for the sync, then create it.
Once created, you’ll be taken to the sync’s detail page.
---
## Configure streamed events
Event configuration happens **inside the sync**, not during creation.
**Set in:**
`Event Collection → Event Syncs → [Select sync] → Configuration`
---
### Event types
Each destination supports one or more **standard event types** (for example, a purchase event) and a **custom event type**.
We recommend always configuring the **custom event type** so you don’t unintentionally drop events.
---
### Referencing fields
In [**event filters**](#event-filters) and [**field mappings**](#field-mappings), you can reference fields on the event payload to filter on and match to fields in the destination object. Keep in mind:
- Reference nested fields using dot notation, for example `properties.price` or `event`.
- If a destination expects a specific data type, use **Liquid templates** to cast or format values, for example `{{ row['properties'].price | cast : 'number'}}`.
- If a field name contains a period (`.`) or backslash (``), escape it with a backslash.
- You do not need to escape spaces.
- The `event` property refers to the event’s name for track events.
- The `type` property refers to the SDK and API event type (`track`, `identify`, `page`, `screen`, `group`).
Use the **Debugger** in your event source to inspect real payloads while designing filters and mappings.
---
### Event filters
Event filters determine which incoming events are sent as each event type.
For example, when configuring a purchase event, you might filter for only purchase-related events.
---
### Field mappings
Field mappings control **what data is sent to the destination** and **where it appears**.
Mappings are configured per event type and grouped by purpose in the UI.
**Set in:**
`Event Collection → Event Syncs → [Select sync] → Configuration → [Select event type]`
---
### Map required event fields
This section maps core event data to fields **required by the destination**.
Required fields (such as items, total, currency, or timestamp) must be mapped before the sync can run successfully. If required fields are missing or mis-typed, the destination may reject events.
---
### Sync additional user fields (optional)
This section lets you send **user-level data** along with each event, such as email or subscription status.
Use this when:
- The destination updates user profiles based on events
- You want events to enrich user data in real time
---
### Sync custom event fields (optional)
This section controls which **event-specific details** are included beyond required fields.
For custom events, we recommend sending the **entire `properties` object** so downstream tools receive full context.
---
### Choosing source values
When mapping fields or creating filters, you choose **where each value comes from**.
You can use:
- **Field** – Pull a value directly from the event payload (for example, `properties.total`)
- **Static value** – Send the same value for every event
- **Template** – Format or transform values
- **Create an array** – Combine one or more fields into an array
These options let you adapt your data to match what the destination expects without changing how events are tracked.
---
### (Advanced) Edit JSON mode
Within a sync's **`Configuration`** tab, you also have the option to modify the underlying sync configuration directly. This mode is intended for **advanced use cases** where the standard UI does not provide enough flexibility.
Use Edit JSON mode when you need to:
- Make bulk or highly repetitive field mappings
- Reference fields or structures not exposed in the UI
- Apply destination-specific configuration that can’t be expressed visually
Most users should configure event streaming using the event filters and field mappings in the UI.
→ See [**Referencing fields**](#referencing-guide).
---
## Errors
Each event sync includes built-in visibility and controls:
- **Overview**: event volume and delivery status
- **Errors**: failed and retried events
- **Alerting**: notifications for failures or latency issues
Failed events are retried automatically before being archived.
When troubleshooting, check:
- Filters (events may be excluded before mapping)
- Required mappings (missing required fields)
- Destination credentials and rate limits
- Any Event Functions attached to the sync
---
## Alerting
Alerting helps you proactively monitor event streaming health.
You can configure alerts to notify your team when:
- Event delivery fails or error rates increase
- Retry thresholds are exceeded
- Latency increases beyond acceptable limits
Alerts are optional but recommended for production workflows.
---
## Supported destinations
Any destination in the Hightouch catalog that supports **event forwarding** can be used with event streaming. Check each destination’s documentation for:
- Supported event types
- Required fields
- Rate limits and identity requirements
---
## Migrating from Segment or RudderStack
**URL:** https://hightouch.com/docs/events/faq/migrating-from-segment-or-rudderstack
**Description:** How to migrate to Hightouch Events from another event collection solution.
**Section:** Hightouch Events
Migrating from Segment or RudderStack is straightforward because the tracking
libraries have the same interface. This document describes step-by-step how to migrate.
### Prepare
First, prepare for your migration by evaluating how your events are currently used:
- What applications do you collect events from?
- What tools do you send your events to?
- What data pipelines depend on the events in your warehouse? For example, analytics and activation.
The query logs from your warehouse are a great way to find what queries depend on your event tables.
### Instrument Your Applications
Next, you need to add tracking code to your applications to collect the same information as your old event collection tool.
Hightouch Events was designed to work with the most common tracking interfaces, so reinstrumenting only requires importing the Hightouch SDK and invoking the Hightouch SDK alongside your old tool.
Not only does Hightouch accept the same tracking options, but it also _autocollects_ the same properties as other event tools. Additionally, Hightouch automatically migrates **anonymous ids** from your users' devices, so anonymous sessions will seamlessly transfer over.
We recommend updating your application code to expose an **analytics wrapper** that invokes your old event tool **and** Hightouch in the same call (kudos if you designed your events code like this from day one!). This wrapper can add a `migrationId` to both versions of your events so that they can be rejoined in the warehouse when validating your migration.
```jsx
function track(eventName, properties) {
const context = { migrationId: getMigrationId() };
oldTool.track(eventName, properties, context);
htevents.track(eventName, properties, context);
}
function getMigrationId(): string {
const timestamp = Date.now();
const randomness = Math.floor(Math.random() * 1000);
return `${timestamp}-${randomness}`;
}
```
Additionally, you need to decide how you want the data to appear in the warehouse. To simplify the migration you may choose to write the data in the same format as your old tool, or you can customize the schema if it simplifies your data pipelines.
Either way, we recommend writing the Hightouch events into a separate schema in your warehouse. This makes it easier to isolate the events while you test, and to compare them with your old data.
### Validate Your Data
Now that you have some test data in your warehouse, you can validate it to confirm that your new tracking can power all your existing use cases. For example, you may realize that there's a backend service that you forgot to migrate over.
We recommend checking the data using some (or all) of the following techniques:
1. Does the Hightouch data have the same **schema** as your existing data? Depending on your current setup, you can check this by directly comparing the tables and schemas in the warehouse, or by just running your existing models against the new schemas.
2. Does Hightouch collect the same data values?
- Does Hightouch collect approximately the same **volume** of data? (Note that you shouldn't expect a 100% match. A variety of expected scenarios like deployment rollout, ad blockers, and network errors can cause events to appear in one tool but not another.)
- JOIN events between Hightouch and your old events tool using your `migrationId` to compare the column values
3. Are your business reports and dashboards the same when built off the Hightouch events?
### Unify Your Data
Once you're confident in the data you're collecting with Hightouch, you will need to combine it with historical data from your old events tool. This can be done by UNIONing the data together, and deduping events that were collected in both tools.
We recommend deduping by picking a timestamp before which you will just use events from your old events provider, and after which you will use Hightouch's events.
For additional deduping to account for events that are collected at slightly different times, you may deduplicate based on the migration ID as well. The migration ID is particularly useful for mobile apps to evaluate when enough of your users have upgraded to the Hightouch-instrumented version of your app.
---
## First party tracking
**URL:** https://hightouch.com/docs/events/first-party-tracking
**Description:** Run Hightouch Events as a first party tracking solution
**Section:** Hightouch Events
## Proxying
You can run Hightouch Events on a custom domain by setting up a proxy. When using a proxy, clients send their event data to your domain rather than directly to Hightouch. This lets you have more control over your data.
### Prerequisites
You'll need a certificate for whatever domain you plan to use for the proxy.
This guide uses [Amazon Cloudfront](https://aws.amazon.com/cloudfront), but any CDN or reverse proxy will work.
### Proxying the JavaScript SDK
Out of the box, the JavaScript SDK is served from cdn.hightouch-events.com. But you can serve the JavaScript SDK from your own domain instead.
First, create a [Cloudfront distribution](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-web-creating-console.html) with the following settings:
| Setting | Value |
| ------------------------------ | -------------------------------------------------------------- |
| Origin domain | `cdn.hightouch-events.com` |
| Origin path | Leave this blank |
| Name | Give your distribution a unique name |
| Enable Origin Shield | `No` |
| Compress objects automatically | `Yes` |
| Viewer protocol policy | `HTTPS only` |
| Allowed HTTP methods | `GET, HEAD` |
| Restrict viewer access | `No` |
| Cache key and origin requests | `Cache policy and origin request policy` |
| Cache policy | `CachingOptimized` |
| Origin request policy | `AllViewerExceptHostHeader` |
| Response headers policy | None |
| Function associations | No associations |
| Alternate domain name (CNAME) | The domain you want to expose to clients |
| Custom SSL certificate | Select a certificate that corresponds to the CNAME you entered |
| Supported HTTP versions | Enable `HTTP/2` and `HTTP/3` |
| Default root object | Leave this blank |
| Standard logging | `Off` |
| IPv6 | `On` |
Then, point your clients to the CNAME you configured. If using the JavaScript snippet, you can do this by replacing `cdn.hightouch-events.com` in the body of the snippet with your CNAME.
### Proxying events
By default, events are sent to `hightouch-events.com`. But you can configure Hightouch to send events to your own domain instead.
First, create a [Cloudfront distribution](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-web-creating-console.html) with the following settings:
| Setting | Value |
| ------------------------------ | -------------------------------------------------------------- |
| Origin domain | `us-east-1.hightouch-events.com` |
| Origin path | Leave this blank |
| Name | Give your distribution a unique name |
| Enable Origin Shield | `No` |
| Compress objects automatically | `Yes` |
| Viewer protocol policy | `HTTPS only` |
| Allowed HTTP methods | `GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE` |
| Restrict viewer access | `No` |
| Cache key and origin requests | `Cache policy and origin request policy` |
| Cache policy | `CachingDisabled` |
| Origin request policy | `AllViewer` |
| Response headers policy | None |
| Function associations | No associations |
| Alternate domain name (CNAME) | The domain you want to expose to clients |
| Custom SSL certificate | Select a certificate that corresponds to the CNAME you entered |
| Supported HTTP versions | Enable `HTTP/2` and `HTTP/3` |
| Default root object | Leave this blank |
| Standard logging | `Off` |
| IPv6 | `On` |
Then, set your SDK's `apiHost` setting to the CNAME you configured. For example, if you are using the Browser SDK:
```js
HtEventsBrowser.load(
{ writeKey: "WRITE_KEY" },
{ apiHost: "events.example.com" }
);
```
---
## Intelligent Tracking Prevention
_Intelligent Tracking Prevention_ (ITP) is an automatic feature of the Safari web browser, designed to limit user tracking. More specifically, the program is designed to prevent analytics libraries from using client-side Javascript to correlate user sessions. ITP configures the Safari browser in the following ways:
1. Third-party cookies are blocked by default.
2. First-party cookies (and localStorage) have a default expiry of 7 days.
3. A user revisiting a website will restart the 7 day expiry.
Unfortunately, most analytics libraries persist session information by:
1. Storing an identifier in a Cookie.
2. Storing an identifier in localStorage.
A legacy analytics installation, one not working to mitigate ITP, will collect less accurate session data for Safari users. For example, a single user visiting a website on both 01/01/2023 and 01/15/2023 will look like two different users. After Safari deletes the first cookie on 01/08/2023, the web SDK will be forced to make a new cookie and a new identifier for the session occurring on 01/15/2023. Downstream, your warehouse will likely see these sessions as two different users. This simple inaccuracy may impact otherwise useful metrics generated from analytics tracking.
### Mitigating ITP
When properly configured, Hightouch Events can store session identifiers outside of ITP's 7 day expiry. It does this by using first party, HTTPOnly cookies. These are not subject to Safari's 7 day expiry. These types of cookies are generally used for logging into a website and are less likely to be limited, even when dealing with anonymous users.
These "server cookies" must follow several rules put in place by ITP:
1. The server providing the HTTPOnly cookie must be on the same domain as the website.
2. If the server is on a subdomain of the website, its IP address must match the IP address that served the main HTML document.
In practice, this means that ITP mitigation requires customers to host some sort of web service responsible for returning HTTPOnly cookies to the browser. The service does not need to be complex; Hightouch has numerous example servers. The most important part of this **HTTPCookieService** is that it must be a first party service: routing a subdomain via DNS will not suffice.
In order for the HTTPCookieService to serve first party, HTTPOnly cookies, you'll need **_one of_** the following:
A **webserver** that handles both your HTML documents and your API. This would mean building the HTTPCookieService functionality into your regular webserver. As an example, you might already have a Java Spring server for handling regular website requests.
A **reverse proxy** that can forward path-dependent requests to different servers. This requires proxying your domain to both an HTTPCookieService and your regular webserver. As an example, you might already have something like NGINX that proxies HTML requests to one place and API requests to another.
A **CDN** that can run programmatic logic when matching certain path-dependent requests. If you already have a CDN, you may be able to add custom handlers without standing up new servers. This can be achieved with a variety of technologies like Amazon's Cloudfront, Lambda@Edge, or API Gateway. Cloudflare Workers can also work.
### Client SDK setup
Once your HTTPCookieService is reachable on your domain, you may configure your Hightouch Events SDK to make use of it when generating Cookies:
```Javascript
const htevents = HtEventsBrowser.load(
{ writeKey: ''},
{
apiHost: "us-east-1.hightouch-events.com", // HtEvents API remains the same
httpCookieServiceOptions: {
clearUrl: '/ht/clear', // route hosted on *your* domain and infra
renewUrl: '/ht/renew', // route hosted on *your* domain and infra
}
},
)
htevents.identify('hello world')
document.body?.addEventListener('click', () => {
htevents.track('documentBodyClick')
})
```
### Server setup
The Events SDK expects to interact with a customer's HTTPCookieService that implements a specific spec for two routes. You can name the endpoints whatever you want, as long as you configure the client SDK to call those endpoints.
### An API for creating server and browser cookies
This route should look for the following **browser** cookies (from Events SDK):
- `request.headers.get("Cookie")["htjs_anonymous_id"]`
- `request.headers.get("Cookie")["htjs_user_id"]`
This route should return these values as **server** cookies:
- `response.cookie("htjs_anonymous_id_srvr", anonVal, {httpOnly:true, ...})`
- `response.cookie("htjs_user_id_srvr", userIdVal, {httpOnly:true, ...})`
If there are no browser cookies found, return any server cookies as **browser** cookies:
- `anonVal = request.headers.get("Cookie")["htjs_anonymous_id_srvr"]`
- `userIdVal = request.headers.get("Cookie")["htjs_user_id_srvr"]`
- `response.cookie("htjs_anonymous_id", anonVal, ...)`
- `response.cookie("htjs_user_id", userIdVal, ...)`
### An API for clearing server cookies
This route should look for **server** cookies and clean them:
- `res.cookie("htjs_anonymous_id_srvr", "", {maxAge: 0, httpOnly:true, ...});`
- `res.cookie("htjs_user_id_srvr", "", {maxAge: 0, httpOnly:true, ...});`
### API spec
The spec of the actual `request` and `response` payloads are kept intentionally vague. The spec should fit a variety of server environments.
The Events SDK only requires that the server: A) handles cookies and B) returns a `200` status code.
### Server examples
- [Express.js and NGINX](https://github.com/ht-sdks/events-sdk-js-mono/blob/master/packages/browser/src/core/http-cookies/server-examples/node-express-js.md)
- [Next.js and Vercel](https://github.com/ht-sdks/events-sdk-js-mono/blob/master/packages/browser/src/core/http-cookies/server-examples/node-next-js.md)
- [AWS Lambda and API Gateway](https://github.com/ht-sdks/events-sdk-js-mono/blob/master/packages/browser/src/core/http-cookies/server-examples/node-aws-lambda.md)
### More information
- Safari: https://webkit.org/blog/9521/intelligent-tracking-prevention-2-3/
---
## Getting started
**URL:** https://hightouch.com/docs/events/functions/getting-started
**Section:** Hightouch Events
Functions give you the power to create custom transformations on
[Hightouch Events](https://hightouch.com/docs/events/overview). By writing
Javascript code, you can easily modify event properties, validate fields, or
enrich your event data!
### Creating a function
To create a new function, navigate to **Event Collection > Functions** and
click **Create function**.
When you create a function, you’ll start with a default function definition,
which you can edit to add custom transformation logic.
Some thing to keep in mind:
- All code must have a `transformEvents` function definition, which receives a
batch of events, and returns a batch of transformed events
- Output events must conform to the core [Hightouch Event spec](https://hightouch.com/docs/events/event-spec)
- Event batches can include multiple event types (so make sure to check
`event.type` if you need to handle event types differently)
- Batch sizes are variable based on sync volume and event size
- Functions time out after 5 seconds of execution time
- Events can be filtered out by omitting them from the function return
### Testing
Test the function works as expected by using the test panel in the function
code editor view.
Simply modify the input in the test panel, and hit **Test**. Results will be
displayed in the output panel, along with any `console` logs. If the function
errors during execution, the error and stack trace will be displayed.
### Save & connect
To save a function, hit **Save changes** at the bottom of the code editor.
Upon save, if the function is connected to any event syncs, the new function
code will immediately be used.
To connect a function to a sync, use the **Connected syncs** tab. Here you can
easily connect or disconnect a function from a sync. When a function is connected
to a sync, events in the target event sync will immediately begin to be transformed by
the function.
Note: a function can be connected to multiple syncs, but a sync can only have
one function connected to it at a time.
### Runtime and dependencies
Functions are executed in a Node.js v20 runtime environment.
Additional available modules:
- [Crypto](https://developer.mozilla.org/en-US/docs/Web/API/Crypto) is available as `crypto`
- [Fetch_API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) is available as `fetch`
### Debugging
Errors from functions will be displayed in the event sync error logs, along
with sample event payloads that received that error.
### Other notes
- Functions are executed after contract validation (if a contract is defined
for the respective event source). This means events blocked by contracts
are not able to be transformed using functions.
- Use `Promise.all` to run async code in parallel, reducing the chance of
hitting a timeout
- If a function errors, the entire batch is failed and will be retried
based on the sync retry configuration
---
## Handling webhooks
**URL:** https://hightouch.com/docs/events/functions/handling-webhooks
**Section:** Hightouch Events
A common use case for functions is cleaning up webhook data from other tools
so that they may be ingested into your warehouse, or forwarded to other tools.
These webhooks usually contain event data that are helpful for building a full
profile of your users. For example, Shopify has webhooks for when users check
out, and Iterable has webhooks for when users open emails.
To set this up, you can create a [Webhook Source](https://hightouch.com/docs/events/sources/webhook)
and add it to your tool that emits events. You can then process the raw webhook
data using Functions.
For example, your code might look something like:
```js
async function transformEvents(events) {
return events.map((event) => ({
...event,
type: "track",
event: event.event,
messageId: event.messageId,
properties: {
...event.properties.body.dataFields,
email: event.properties.body.email,
},
anonymousId: event.properties.body.userId
? null
: event.properties.body.email,
userId: event.properties.body.userId,
}));
}
```
---
## Functions overview
**URL:** https://hightouch.com/docs/events/functions/overview
**Section:** Hightouch Events
Functions give you the power to create custom transformations on
[Hightouch Events](https://hightouch.com/docs/events/overview). By writing
Javascript code, you can easily modify event properties, validate fields, or
enrich your event data!
### Features
- Modify, validate, enrich, or filter events with custom logic before syncing
to destinations
- Write transformations in simple, modern JavaScript
- Code, test, and connect to syncs all within the Hightouch app
- Powerful interface to transform batches of events at once
### Use cases
- **Enrich event data.** Call external APIs to add additional data to your
event payload. For example, you can add geolocation fields based on IP addresses.
- **Data cleaning and normalization.** Encrypt sensitive information, normalize
complicated data types, or completely remove PII from event payloads.
- **Data validation.** Write advanced validation logic to ensure your event
data conforms to spec.
- **General transformations.** Completely transform your event before sending
it to a downstream destination.
### Getting started
It’s easy to write functions to transform events in any Hightouch Event sync!
Create a function, test the code, and connect to a sync in just minutes.
[Read our guide](https://hightouch.com/docs/events/functions/getting-started) to get started!
---
## Get started with Hightouch Events
**URL:** https://hightouch.com/docs/events/get-started
**Description:** Follow a step-by-step guide to plan, collect, validate, store, and activate event data using Hightouch Events.
**Section:** Hightouch Events
| Audience | How you’ll use this article |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| **Marketing teams** | Understand the end-to-end event workflow and how event data powers real-time activation, personalization, and experimentation. |
| **Data teams** | Learn how events are planned, validated, stored in the warehouse, governed, and made available for downstream use. |
| **Engineering teams** | Follow the recommended implementation path, from instrumentation and validation to streaming and migration. |
# Get started with Hightouch Events
This guide walks you through the end-to-end process of implementing Hightouch Events: from **planning what to track** to **validating live event data** and **activating it in downstream tools**. The steps below reflect the natural order most teams follow and link out to deeper documentation where needed.
Use this page as your starting point, then dive into each linked section as you implement.
---
## Implementation overview
The table below outlines the recommended order for implementing Hightouch Events. Each step maps directly to a section in the Events documentation and reflects the structure you’ll see in the left navigation.
| Step | What you’ll do | Docs |
| ------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------- |
| **[Define events to track](#1-plan-events-to-track)** | Define the events you want to collect, including event names, types, and required properties. Align on naming conventions and identity strategy before writing code. | [📋 Event tracking spec](/events/event-spec) |
| **[Connect event sources](#2-connect-event-sources-to-start-collecting-events)** | Install SDKs or connect supported sources to start sending events to Hightouch. Use the debugger to confirm events are received with the expected payloads. | [📋 Connect event sources](/events/sources/overview) |
| **[Write events to your warehouse](#3-write-events-to-your-warehouse)** | Write event data to your warehouse so it’s queryable for analytics, modeling, and downstream use cases. | [📋 Write events to your warehouse](/events/warehouses/warehouse) |
| **[Stream events to destinations](#4-stream-events-to-destinations)** | Forward events in real time to downstream tools for activation, personalization, and operational workflows. | [📋 Event streaming](/events/event-streaming) |
| **[(Advanced) Govern event data](#5-advanced-govern-event-data)** | Define and enforce rules for event structure, naming, and formats to keep data consistent as your implementation scales. | [📋 Managing contracts](/events/contracts/management) |
| **[(Advanced) Transform events](#6-advanced-transform-events)** | Transform or enrich events in-flight before they’re written to your warehouse or streamed to destinations. | [📋 Functions overview](/events/functions/overview) |
| **[(Advanced) Set up first-party tracking](#7-advanced-set-up-first-party-tracking)** | Proxy events through your own domain to improve reliability and mitigate browser restrictions like Safari ITP. | [📋 First-party tracking](/events/first-party-tracking) |
| **[Validate your events](#8-validate-your-events)** | Review end-to-end event flow, inspect live data, and (if migrating) compare results against your previous system. | [📋 Migration overview](/events/migration-guide/overview) |
If you are migrating to Hightouch Events see [**Migrate to Hightouch Events**](#migrate-to-hightouch-events).
---
## Step-by-step guidance
### 1. Plan events to track
Start by deciding **which user actions and events you want to capture** and how they should be represented. This step often involves collaboration between marketing, data/analytics teams, and platform admins.
You should define:
* Event names and when each event fires
* Which fields are user traits vs event properties
* Required vs optional properties
* How users are identified across devices and sessions
This planning work becomes the foundation for every step that follows.
→ See [**Event tracking spec**](/events/event-spec)
---
### 2. Connect event sources to start collecting events
Once you’ve defined your event spec, instrument your applications and services to send events to Hightouch.
Hightouch supports multiple ingestion methods, including browser and mobile SDKs, server-side SDKs, HTTP APIs, and streaming platforms like Kafka.
At this stage, you should:
* Install and initialize the appropriate SDKs or sources
* Start sending `identify` and `track` calls
* Use the debugger to confirm events are arriving as expected
#### Supported SDKs
Hightouch provides several SDKs (software development kits) that you can deploy to your app or site in order to start collecting events. There are separate SDKs for:
- [Browser (Javascript)](./sdks/browser)
- [Node.js](./sdks/nodejs)
- [Ruby](./sdks/ruby)
- [Go](./sdks/go)
- [PHP](./sdks/php)
- [Java](./sdks/java)
- [Python](./sdks/python)
- [iOS](./sdks/ios)
- [Android](./sdks/android)
Additionally, you can collect events directly from [Kafka](./sources/kafka), [PubSub](./sources/google-pubsub), and [Webhooks](./sources/webhook) and inspect events in a web-based debugger to ensure you're capturing events from these different sources (including from SDKs) correctly.
#### Recommended regions
* `aws-us-east-1` (US East)
* `aws-ap-south-1` (Asia Pacific – Mumbai)
* `aws-eu-west-1` (Europe – Ireland)
For setup in other regions, please and we’ll help you get started.
→ See [**Event sources overview**](/events/sources/overview)
---
### 3. Write events to your warehouse
After events are collected, you can write them to your data warehouse. This makes event data available for querying, analytics, and downstream modeling.
In this step, you’ll:
* Connect your warehouse destination
* Review how event tables and schemas are structured
* Configure error handling and schema behavior
→ See [**Write events to your warehouse**](/events/warehouses/warehouse)
---
### 4. Stream events to destinations
Once events are flowing reliably, you can stream selected events to downstream tools such as marketing platforms, analytics systems, or internal services.
This is useful when you want to:
- Trigger actions immediately after a user event
- Send conversion or interaction events to external platforms
- Power same-session or near–real-time workflows
Event forwarding applies filters, mappings, and governance rules before events are delivered.
→ See [**Event streaming**](/events/event-streaming)
---
### 5. (Advanced) Govern event data
As your event implementation grows, data contracts help you maintain consistency and prevent breaking changes.
Contracts let you:
* Enforce required fields and types
* Validate naming conventions
* Detect and respond to violations automatically
→ See [**Managing contracts**](/events/contracts/management) and [**Handling violations**](/events/contracts/violations)
---
### 6. (Advanced) Transform events
Event functions allow you to transform event data in-flight without re-instrumenting your sources.
Common use cases include:
* Renaming events or properties
* Enriching events with derived values
* Filtering or normalizing payloads
→ See [**Functions**](/events/functions/overview)
---
### 7. (Advanced) Set up first-party tracking
First-party tracking routes events through your own domain, improving delivery reliability and helping mitigate browser limitations such as Safari ITP.
This step is optional, but recommended for teams with strict reliability or privacy requirements.
→ See [**First-party tracking**](/events/first-party-tracking)
---
### 8. Validate your events
Before fully relying on your implementation, validate the entire event pipeline.
You should:
* Inspect live events in the debugger and event catalog
* Confirm data matches your tracking spec
---
## Migrate to Hightouch Events
If you’re migrating from another event tracking solution, migration typically happens **in parallel** with the steps above:
* You’ll define events and instrument sources while still running your existing system
* You’ll validate data side-by-side before switching downstream tools
The migration guide walks through this process in detail:
→ [**How to migrate to Hightouch Events**](/events/migration-guide/overview)
---
## Step 3: Defining data contracts for data governance
**URL:** https://hightouch.com/docs/events/migration-guide/defining-data-contracts
**Description:** Explore the importance of establishing data contracts early in the migration process to ensure consistent data quality and maintain governance standards.
**Section:** Hightouch Events
Establishing robust data contracts early in your migration process sets up your data pipeline to maintain data quality, consistency, and governance. Hightouch provides a powerful tool for defining and managing these contracts through [data contracts](https://hightouch.com/docs/events/contracts/management).
Hightouch includes contracts with Events at no additional cost. While it's a paid, premium feature in some event tracking platforms, we think data governance is crucial for collecting and activating high-fidelity data, and we want to enable everyone using Hightouch Events to build for success.
## Understanding data contracts
Hightouch's contracts consists of two main components:
1. **Contract Events**: These define individual events, including their name, type, description, and validation schema.
2. **Contracts**: A set of contract events that can be applied to an event source.
Contracts can be used for multiple sources. This structure allows for flexible and granular control over your event data.
## Creating data contracts in Hightouch
For each event you want to validate, follow the [instructions](https://hightouch.com/docs/events/contracts/management) to set up event contracts.
Hightouch contracts use [JSON Schema Draft 7](https://json-schema.org/), and contracts can be created with our graphical editor or by editing JSON directly in the app.
## Event versioning
Hightouch data contracts support event versioning, which is crucial when migrating from another platform or when you expect to have multiple versions of your application available, as is common with mobile applications.
See [these docs](https://hightouch.com/docs/events/contracts/management#event-versioning) for help in implementing additional versions of a contract event after initial setup.
## Best practices for data contracts
Many customers are concerned with losing out on data by implementing data contracts too early in setting up event collection, or are worried about devoting time to creating contracts when there's other work to be done downstream.
1. **Start with core events**: Begin by defining contracts for your most critical events, such as those used for key business metrics, or for events that you expect to monitor more closely due to likelihood of errors.
2. **Implement blocking gradually**: Data contracts enable you to allow, filter, or block events. You can define contracts, and allow those events through with a warning until you've fine-tuned the contracts and built confidence. At that point, you can shift to filtering or blocking the event.
3. **Collaborate**: Involve stakeholders from different teams to ensure the contracts meet various needs across the organization, and so that all stakeholders understand the value of data governance.
By leveraging data contracts, you're setting a strong foundation for data quality and governance in your new event tracking setup. These contracts will help maintain consistency, catch errors early, and provide clear documentation for all data consumers in your organization.
In the next section, we'll cover how to [validate your migration](validating-your-migration) to ensure that your new Hightouch Events setup is capturing data correctly and consistently with your previous system.
---
## Step 2: Implementing Hightouch Events SDKs
**URL:** https://hightouch.com/docs/events/migration-guide/implementing-hightouch-events
**Description:** Discover the process of switching to Hightouch SDKs, including implementing an analytics wrapper function to support seamless validation during migration.
**Section:** Hightouch Events
Hightouch Events provides a range of SDKs across web, mobile, and server-side languages. This step covers the technical implementation of switching from your current event tracking SDK (such as Segment's or Rudderstack's) to Hightouch Events.
Hightouch Events is backward compatible with common tracking APIs like those of Segment and Rudderstack, making the initial code changes straightforward. We'll cover examples below for a couple of languages.
To facilitate a side-by-side comparison and ensure a seamless transition, we
recommend implementing a wrapper function that sends events to both your
current platform and Hightouch. You'll see this in practice in the Javascript
example below.
## Implementing the Browser SDK (Javascript)
Use the [Browser SDK](https://hightouch.com/docs/events/sdks/browser#installation) to track events in webpages.
1. Install the Browser SDK by loading the Javascript through a `script` tag with the following snippet or through `npm`.
Through a `script` tag:
```html
```
Through `npm`:
Install with `npm install @ht-sdks/events-sdk-js-browser`.
Import and initialize the SDK:
```jsx
{ writeKey: "WRITE_KEY" },
{ apiHost: "us-east-1.hightouch-events.com" }
);
```
In both cases, replace your `WRITE_KEY` with the write key from your source.
## Implementing standard tracking calls, such as track, page, and identify
During this step of the migration, you'll update your instances of event method calls to also call the same Hightouch SDK method.
The example below shows an implementation of a wrapper function that calls the `track` method of a prior tool and Hightouch Events. Both track calls take the same parameters, and include a `migrationId` property that enables validation of data during the migration process.
```jsx
function track(eventName, properties) {
const context = { migrationId: getMigrationId() };
analytics.track(eventName, properties, context);
htevents.track(eventName, properties, context);
}
function getMigrationId(): string {
const timestamp = Date.now();
const randomness = Math.floor(Math.random() * 1000);
return `${timestamp}-${randomness}`;
}
```
This wrapper function does the following:
- Generates a unique `migrationId` for each event
- Sends the event to your current tracking system
- Sends the same event to Hightouch Events
- Includes the `migrationId` with both events for later comparison and deduplication
## Handling identify and page view events
Create similar wrapper functions for other common event types based on their parameters and type signatures. For example, for a `identify` method in the Browser SDK,
```jsx
function identifyUser(userId, traits) {
const context = { migrationId: getMigrationId() };
analytics.identify(userId, traits, context);
htevents.identify(userId, traits, context);
}
```
## Migrating server-side events
Follow a similar pattern for server-side event tracking. Below, we show an example of using the [Python SDK](https://hightouch.com/docs/events/sdks/python) with a wrapper function.
```python
analytics.write_key('YOUR_SEGMENT_WRITE_KEY')
htevents.write_key('YOUR_HIGHTOUCH_WRITE_KEY')
def track_event(user_id, event_name, properties):
migration_id = str(uuid.uuid4())
properties_with_id = {**properties, 'migrationId': migration_id}
# Track with Segment
analytics.track(user_id, event_name, properties_with_id)
# Track with Hightouch
htevents.track(user_id, event_name, properties_with_id)
```
## Update analytics code across your repositories
If your codebase historically directly calls Segment's (or another provider's) library to trigger events, use your preferred method for making bulk code changes across your repository or repositories, whether it's using `grep` / `find` and `sed`, find and replace across a workspace in your editor, or other tools to replace track, identify, group, page, and screen calls with the wrapper functions.
Make sure that you don't just update the analytics code with the wrapper function(s), but also update library imports across files that call the new wrapper function.
If you've already implemented wrapper functions and have modified them using the preceding migration examples, you should be all set.
## Testing the implementation
After implementing the wrapper functions:
1. Test thoroughly in a development or staging environment.
2. Verify that events are being received by both your current system and Hightouch Events.
3. Check that the `migrationId` is correctly included in events from both systems.
## Anonymous IDs and automatically collected information
Hightouch Events’ Browser, iOS, and Android SDKs automatically collects a range of information. Read [the docs](https://hightouch.com/docs/events/event-spec#automatically-captured-fields) for the full list of fields.
Hightouch Events also automatically migrates anonymous IDs that were previously assigned by Segment or Rudderstack. If a user had an anonymous ID assigned via one of those providers, we'll continue to use the same anonymous ID in Hightouch Events so that user sessions are not interrupted or fragmented by the migration.
## Optional: Gradual rollout
Depending on the number of your event sources, your event volume, and whether any power critical downstream services, you can consider implementing a gradual rollout strategy. This can help build the migration team's confidence in Hightouch Events and make the migration process more manageable by constraining event volume and sources in the first migration efforts.
1. Start using Hightouch Events with a small percentage of your user base or a specific segment. Some organizations migrate a portion of traffic to start with, while others migrate sources related to a particular platform (such as mobile apps) before migrating additional platforms.
2. Monitor for any issues or discrepancies.
3. Gradually increase the percentage of users or sources on the new system.
By following these steps, you'll have successfully implemented the Hightouch SDK alongside your existing event tracking system. This dual-tracking approach allows for thorough validation and comparison in the next phases of the migration process.
[Event streaming](https://hightouch.com/docs/events/event-streaming)
destinations differ in process. Since streaming events bypass the warehouse
and go straight to their destinations, to avoid sending large volumes of
duplicate events to your streaming destinations you won't want to have events
streaming from your old tool and Hightouch Events at the same time. Instead
you'll set up and turn on your Hightouch streaming destination and turn off
your prior tool's streaming destination at the same time.
In the next section, we'll explore how to [define data contracts](defining-data-contracts) to ensure robust data governance throughout your migration and beyond.
---
## How to migrate to Hightouch Events
**URL:** https://hightouch.com/docs/events/migration-guide/overview
**Description:** Learn how to migrate to Hightouch Events from another event collection solution.
**Section:** Hightouch Events
The key to an effective and on-time migration between two event tracking systems is thorough planning and detail-oriented execution.
This guide provides a roadmap for migrating from your existing event tracking solution (such as Segment or Rudderstack) to Hightouch Events based on our experience helping customers migrate. Following this roadmap can minimize disruption to your existing data workflows while ensuring a smooth transition.
## What you'll learn
In this guide, we'll cover:
1. **[Preparing for migration](preparing-for-migration)**: Scope your migration, decide on your data model and tracking plan, and estimate the time and resources required.
2. **[Implementing Hightouch Events SDKs](implementing-hightouch-events)**: Switch to Hightouch SDKs with an analytics wrapper function to support validation.
3. **[Defining data contracts for data governance](defining-data-contracts)**: Establish data contracts early in the process to ensure data quality and consistency.
4. **[Validating your migration](validating-your-migration)**: Verify your setup and ensure data quality matches or exceeds your previous platform.
5. **[Unifying historical and new data](unifying-data)**: Deduplicate and unify your historical data with new events collected by Hightouch.
6. **[Setting up monitoring and alerts](setting-up-alerts)**: Set up monitoring, observability, and advanced alerting for long-term success.
7. **[Updating downstream applications and destinations](updating-downstream-applications)**: Configure or set up the applications that consume your event data.
By following this guide, you can leverage Hightouch Events' warehouse-centric architecture, flexible schema options, and data governance features while maintaining continuity with your existing data processes.
---
## Step 1: Preparing for migration
**URL:** https://hightouch.com/docs/events/migration-guide/preparing-for-migration
**Description:** Learn how to scope your migration, plan your data model and tracking strategy, and estimate the resources needed for a successful transition to Hightouch Events.
**Section:** Hightouch Events
Organizational size and technical complexity can impact each of these steps, but during a migration process, you'll typically:
1. **Form a migration team and communication plan**: Gather a cross-functional team, including data and analytics engineers, analysts, and stakeholders that will carry out the migration and verify data quality. Ensure that you inform affected teams of upcoming changes and provide a channel for concerns or questions.
2. **Determine the scope of your migration**: Decide whether you're modifying your data model during the migration or keeping it the same, whether you're merging Hightouch Event data with historical data, and whether you want to replace existing event tables or create new ones.
3. **Map downstream destinations**: Document any data destinations, dashboards, reports, or other consumers of event data that may need to be updated or validated as part of the migration.
4. **Establish a timeline**: Set realistic deadlines for each migration phase based on available personnel.
## Determining the scope of migration
### Data model and tracking plan
Decide on your approach to migration. We think there are two main options:
1. **Simple migration**: Keep your existing data model and event structure as much as possible. This makes validation and updating downstream applications and destinations more straightforward, and may require little or no data wrangling to fit new event data into the existing data model.
2. **Complex migration**: Address issues or concerns in your data model during the migration process. Changes during a migration process could also include re-instrumenting events for a new tracking plan or integrating new tooling.
While these changes can have positive impacts on long-term data quality, each change increases the complexity of the migration and should be evaluated for return on the increased implementation time.
For most organizations, **we strongly recommend a simple migration**. Minimizing changes increases the likelihood of timely completion and cuts down on the roadblocks that can develop through substantial changes in the data model.
If data quality is a significant issue, a migration to Hightouch Events can be a good occasion to build for long-term data quality and usability.
### Historical data handling
During planning, you should decide where you want the new event data to appear in your warehouse and whether you intend to union the historical and new data.
Regardless of your choice on unioning, we recommend writing the new Hightouch Event data into a separate schema in your warehouse. This makes it easier to isolate the events while you test and validate your migration, comparing them to historical data. We'll discuss where and how to unify historical and new data in [Step 5](unifying-data) of the guide.
### Mapping Sources and Downstream Destinations
During planning, identify:
- **Sources**: All the applications you collect events from that you want to migrate to Hightouch Events.
- **Destinations**: All the places that should receive or consume event data, including your warehouse, BI and analysis tools, and marketing and advertising platforms.
While most organizations send their events into their warehouse, then reverse ETL (rETL) data to other tools, Hightouch Events also supports [streaming events](https://hightouch.com/docs/events/event-streaming) directly to other destinations.
It's worthwhile to create a table or other structure to map each source to its destinations and downstream consumers along with the relevant transfer method (rETL or streaming).
Once your team has decided on any changes to the data model and mapped data pipelines from sources to destinations, you can plan for any updates that will be needed for each downstream consumer or tool.
## Estimating time to migrate
Once you've broken down the migration according to key phases on the roadmap, you can estimate effort based on personnel, number of events, and number of sources and destinations.
Our estimates below—and designated personnel—are based on our experience
helping customers migrate to Hightouch Events. Every organization is
different, though, and migration time can vary significantly based on
organizational and technical factors.
For organizations with 10 to 30 event sources, we often see 4 to 6 calendar weeks for a migration. Organizations with 70 to 100 sources can migrate over 8 to 12 weeks.
1. **Planning and preparation**: 1 week effort for a **data engineer** and **data analyst** for research, planning, and documenting the plan, though this scales with changes to the data model and tracking plan.
2. **SDK implementation**: 1 week (or less) for an engineer to implement, test, and deploy new SDK code and initial data contracts.
3. **Configuring destinations**: 1 week (or less) for a data analyst to configure, test, and do initial monitoring.
4. **Validating and unioning data**: 1 week for a data analyst, though this scales with number and complexity of events and destinations and how you're unioning historical data.
5. **Updating downstream applications and deprecating your prior event collection tool**: 1 week or less for an engineer and a data analyst to update downstream data consumers, replace analytics code, re-validate data, and sunset your prior event collection tool.
Remember, these are rough estimates. Your actual timeline may vary based on the complexity of your current setup, the scope of changes you're implementing, and the number of dedicated personnel you have working on the migration.
By thoroughly preparing and planning your migration, you'll set a strong foundation for the technical implementation phases that follow. In the next section, we'll dive into the process of [switching SDKs](implementing-hightouch-events) to start collecting data with Hightouch Events.
---
## Step 6: Setting up for continued success with monitoring and alerts
**URL:** https://hightouch.com/docs/events/migration-guide/setting-up-alerts
**Description:** Discover how to implement robust monitoring, observability, and advanced alerting systems for long-term success with Hightouch Events.
**Section:** Hightouch Events
Robust monitoring and alerting systems enable your organization to identify, diagnose, and address any issues that may arise in your event tracking setup.
Hightouch has three primary features to support engineering and growth teams in managing event collection and data activation:
1. Per-source volume graph
2. In-app live debugger
3. Per-sync configurable alerts
We'll look at a few examples of how you can leverage these features.
## Monitoring event volume
Teams can get a quick overview of event volume over time with Hightouch's built-in visualization. Each event source page shows a graph of event volume at 1, 3, and 7 day time periods.
This visual representation helps you quickly identify unusual spikes or drops in event volume, which could indicate issues with data collection or unexpected changes in user behavior.
While human checks are great, most organizations will want automated alerts to notify the engineering or analytics team of noticeable drops in event volume.
Events uses Hightouch's configurable alerting system in conjunction with event-specific triggers to send messages by email, SMS, Slack, or PageDuty. You can configure an alert so that a team member or Slack channel is notified when event traffic drops below a particular level, enabling your team to investigate the occurrence quickly.
## Detecting event violations
Hightouch also supports alerting based on your event contracts. Contracts can send invalid events to a violations table in your warehouse. That sync to the warehouse can send an alert when violations exceed a threshold in a given time period for particular [types of violations](https://hightouch.com/docs/events/contracts/violations).
To understand a particular set of violations, you can use the live debugger associated with each event source. The debugger enables you to inspect individual events and their payloads to understand schema and contract violations. This event-level view complements macro-level monitoring to provide teams the information they need to remediate errors.
## Alerting on downstream sync issues
Hightouch supports row and sync-level alerting for reverse ETL syncs, and treats event syncs the same. You can set up alerts to notify your team when there is an elevated row rejection rate, such as a particular percentage of rows failing during a sync to your data warehouse. You can also be notified when there is a series of consecutive syncs with fatal errors. Both types of alerts enable an engineering and analytics team to dive in and address sync inconsistencies or errors quickly.
These specific alert triggers are a beta feature. Please for access.
## Regular reviews and maintenance
While Hightouch's automated monitoring and alerting features do much of the heavy lifting, regular human oversight is still crucial. Monthly and quarterly reviews of Hightouch's data quality are important, and teams need to update contracts to account for new events or properties in their tracking plans.
Leveraging Hightouch's built-in monitoring and alerting capabilities sets your organization up for long-term success with your event data. These tools help maintain data quality and provide valuable insights into your data collection processes and user behavior patterns.
In the next, and final, section, we'll [update downstream applications and destinations](updating-downstream-applications) that consume event data.
---
## Step 5: Unifying historical and new data
**URL:** https://hightouch.com/docs/events/migration-guide/unifying-data
**Description:** Understand methods for deduplicating and unifying your historical data with new events collected by Hightouch, ensuring continuity in your analytics.
**Section:** Hightouch Events
After successfully setting up Hightouch Events and getting event data flowing, you'll almost certainly want to integrate your historical data with the newly collected events. While it's possible that an organization wants a fresh start to its event data or has completely rewritten its data model, if you're migrating from an existing solution like Segment or Rudderstack and have downstream applications, you'll want to unify your historical and new data to ensure continuity in your analytics and a clean transition for your data consumers.
## Planning your unification strategy
Before you begin, consider the following:
1. **Time frame**: Based on your use cases, decide how far back you need to unify data (for example, 1 year, 2 years, or all historical data). Your decided time frame affects processing time and storage requirements and ensures you have enough historical context for your analyses.
2. **Data volume**: Assess the volume of historical data you need to unify. This could impact processing time and storage costs and may influence your choice of unification method.
3. **Schema differences**: Identify any differences between your old and new event schemas if you haven't in a prior step. If there are changes, this will inform any necessary data transformations during the unification process.
4. **Storage considerations**: Determine where the unified data will be stored (for example, an existing or new table in your data warehouse). Your decision can affect query performance, how you update downstream destinations, and may have cost implications.
There are three main options for where to store your unified data: in the previous provider's events tables, in the Hightouch tables, or in an intermediary materialized table. Any of these options would work, but **we recommend joining the prior events tables into the Hightouch tables to avoid recurring unioning costs from an intermediate table and for ease of use (Hightouch will write events into the source of truth tables).**
## Unioning historical and new data
To ensure continuity in your analytics and a smooth transition for data consumers, we'll union the historical Segment data with the new Hightouch Events data.
Your process will depend on where you've decided to store the unioned data and any needed data transformations. We've sketched in SQL here for inserting 2 years of historical data from a prior provider (in this case Segment) into the Hightouch tables with minimal transformation.
In this particular case, we're only going to insert events from Segment that don't have a `migrationId`, since any event in Segment with a `migrationId` should already be in the Hightouch tables.
```sql
-- Insert historical data from Segment to Hightouch identifies table
INSERT INTO HIGHTOUCH.identifies (
id, anonymous_id, user_id, timestamp, email, name, migrationId
)
SELECT
s.id, s.anonymous_id, s.user_id, s.timestamp, s.email, s.name, s.migrationId
FROM SEGMENT.identifies s
WHERE s.migrationId IS NULL
AND s.timestamp >= DATEADD(day, -730, CURRENT_DATE()); -- Adjust time frame as needed
```
Modify this for table names and to include any traits or other properties you need. Repeat similar INSERT statements for `tracks`, `pages`, and other relevant tables, mapping the columns appropriately.
If there are differences between Segment and Hightouch schemas, you may need to transform the data during insertion. If you are migrating from Segment, for example, the schemas are quite similar and should only require basic SQL transformations, if any.
## Validating the unified dataset
We've previously validated that Hightouch Events is collecting an equivalent volume of events with matching values. For peace of mind, after unifying the de-duped data, you can perform some basic validation steps:
1. **Check event counts**: Ensure the total event count makes sense based on the historical data and new Hightouch events.
```sql
SELECT
DATE_TRUNC('day', timestamp) as date,
COUNT(*) as event_count,
COUNT(DISTINCT user_id) as user_count
FROM HIGHTOUCH.tracks
GROUP BY 1
ORDER BY 1;
```
2. **Verify data continuity**: Look for any unexpected drops or spikes in event volume, especially around the migration period.
3. **Compare with Segment data**: Run a comparison query to ensure all historical data was properly migrated.
4. **Sample data**: Manually review a sample of events from different periods to ensure data format and content consistency.
By following these steps, you'll create a unified dataset in your Hightouch tables that combines your historical Segment data with new events from Hightouch Events. This approach ensures data continuity and provides a single source of truth for all your event data.
Remember to adjust the SQL queries as needed to match your specific schema and table names. Also, consider running these operations in batches if you're dealing with a large volume of historical data.
In the next section, we'll [set up alerts and monitoring](setting-up-alerts) for long-term success of your events infrastructure.
---
## Step 7: Updating downstream applications and destinations
**URL:** https://hightouch.com/docs/events/migration-guide/updating-downstream-applications
**Description:** Learn the process of configuring and setting up applications that consume your event data, ensuring a smooth transition for all data-dependent systems.
**Section:** Hightouch Events
At this point, you've instrumented your event collection with Hightouch Events, validated data quality, and unified your new data with historical data. The final step of migration is updating your downstream applications and destinations to ensure your entire data ecosystem is aligned with your new event tracking infrastructure.
It is possible to run this step in parallel with earlier validation to shorten the migration, but you may need to repeat some steps depending on any changes made during validation or unioning with historical data.
[Event streaming](https://hightouch.com/docs/events/event-streaming)
destinations differ in process. Since streaming events bypass the warehouse
and go straight to their destinations, to avoid sending large volumes of
duplicate events to your streaming destinations you won't want to have events
streaming from your old tool and Hightouch Events at the same time. Instead
you'll set up and turn on your Hightouch streaming destination and turn off
your prior tool's streaming destination at the same time.
## Re-check your inventory of downstream applications
In your planning, you likely created an initial list of applications consuming event data. Take this opportunity to review and update that list in light of the migration so far.
Make sure your list includes any systems, tools, and processes that consume your event data, including:
- Analytics platforms (for example, Google Analytics, Amplitude)
- Marketing and advertising tools (for example, Meta Ads, Google Ads)
- Customer engagement platforms (for example, Braze, [Customer.io](http://customer.io/))
- Business intelligence tools (for example, Tableau, Looker)
- Custom internal dashboards and applications
Rank your downstream systems based on their criticality to your business operations to plan a phased approach to the updates if necessary.
## Update data sources
For each downstream system:
1. **Identify current data source**: Determine how the system is currently receiving event data (for example, reverse ETL, streaming, or direct SDKs).
2. **Update configurations or set up new syncs**: Depending on the downstream tool, you might:
- Create new or update existing Hightouch syncs
- Set up or modify streaming destinations in Hightouch
- Modify configurations in the downstream tool to point to the unified events data.
3. **Verify data flow**: Confirm that events are flowing correctly into the updated system.
## Adapt to schema changes
If your migration to Hightouch Events involved changes to your event schema:
1. **Document changes**: Create a clear mapping between your old and new event schemas.
2. **Update data models**: Modify any data models or transformations in your downstream systems to align with the new schema.
3. **Revise queries**: Update SQL queries, API calls, or other data retrieval methods to reflect the new event structure.
## Validate metrics and reports
For each updated system:
1. **Reconcile key metrics**: Compare critical metrics between the old and new data sources to ensure consistency.
2. **Update dashboards**: Revise any dashboards or standard reports to use the new data source and schema.
3. **Cross-check historical data**: Verify that historical data is accurately represented in updated reports.
## Handle data discrepancies
If you encounter differences between your old and new systems:
1. **Investigate root causes**: Determine if differences are due to improved data collection, schema changes, or potential issues.
2. **Document explanations**: For any persistent differences, document the reasons and communicate these to stakeholders.
3. **Adjust baselines**: If necessary, establish new baselines for key metrics based on the new data from Hightouch Events.
## Manage the transition period
If you're executing your migration in a linear fashion, you'll carry out this work as a whole after validating and confirming your new event data and cutting off your prior event collection tool.
However, if you're gradually shifting traffic over to Hightouch Events, you might decide to start transitioning downstream applications to use the new data at the same time and run in parallel with your prior tool and historical. If you do so, have a clear date planned for switching all downstream applications over to Hightouch events data and communicate changes to stakeholders as you transition.
## Enabling users across your organization
As users across your organization interact with the new event data, gather feedback and refine your event collection and data activation setup to enable users to leverage your event data effectively.
By methodically updating your downstream applications and destinations, you ensure that the benefits of migrating to Hightouch Events propagate throughout your entire data ecosystem.
## Turning off your previous provider
Once you're confident in your Hightouch Events setup, you can begin sunsetting your previous provider. We recommend the process below.
1. **Gradual cutover**: Incrementally increase the proportion of traffic sent only to Hightouch Events.
2. **Monitor closely**: Watch for any unexpected changes in data patterns or downstream system behavior.
3. **Final switch**: When ready, update your tracking code to send events only to Hightouch Events. With a wrapper function, this is as easy as a one line removal.
4. **Decommission old system**: Follow your previous provider's instructions for account closure and data handling.
This final step completes your migration journey, setting you up for improved data quality, more efficient analytics, and better-informed decision-making across your organization.
Remember, this is not just a one-time process. Regularly review and optimize your event data flow to continually derive maximum value from your Hightouch Events implementation.
---
## Step 4: Validating your migration
**URL:** https://hightouch.com/docs/events/migration-guide/validating-your-migration
**Description:** Learn techniques for verifying your Hightouch Events setup and ensuring that data quality matches or exceeds your previous event tracking platform.
**Section:** Hightouch Events
After implementing Hightouch Events alongside your existing system, it's crucial to validate that your new setup captures data correctly and consistently and can power your existing use cases.
There are two main steps in validation:
1. Verify that event data is flowing from Hightouch Events with the correct properties and data types.
2. Check that the data from Hightouch Events is comparable to the data from your existing provider in volume and in values, such as user IDs and event properties.
## Verifying your setup
1. **Check event reception**:
- Confirm that events are being received by both your current system and Hightouch Events.
- Use the Hightouch debugger to view incoming events in real-time.
2. **Verify event structure**:
- Ensure all expected properties are present in the Hightouch events.
- Check that data types are correct (for example, numbers aren't being sent as strings).
3. **Test all event types**:
- Manually trigger each type of event (page views, user identifications, custom events) in your application.
- Verify that they appear correctly in both systems.
## Checking data quality
Compare the data between your current platform and Hightouch Events to make sure that Hightouch Events is instrumented correctly and sending the data you expect. We recommend checking both event volume and values.
If you're maintaining your data model during your migration or only making minor changes, validating data values will be more straightforward. Significant changes to the model model during migration will make validation more complex and time-consuming.
### Compare event volume
Let's assume that you're migrating from Segment to Hightouch Events, and have data flowing from Segment into tables per Segment's schema and from Hightouch Events into tables per Hightouch's schema. We'll also assume events from during the migration have a `migrationId` assigned through an analytics wrapper function, described in Step 2.
We could use the following query—or something adjusted to your warehouse and setup—to look at the count of `identifies` events within the last 7 days. While we expect to see roughly the same count of events, there can be some variation—we'll cover why differences can occur later in this step of the guide.
```sql
WITH segment_identifies AS (
SELECT DATE(timestamp) AS event_date, migrationId, COUNT(*) AS segment_count
FROM SEGMENT.identifies
WHERE timestamp >= DATEADD(day, -7, CURRENT_DATE())
GROUP BY DATE(timestamp), migrationId
),
hightouch_identifies AS (
SELECT DATE(timestamp) AS event_date, migrationId, COUNT(*) AS hightouch_count
FROM HIGHTOUCH.identifies
WHERE timestamp >= DATEADD(day, -7, CURRENT_DATE())
GROUP BY DATE(timestamp), migrationId
)
SELECT
COALESCE(s.event_date, h.event_date) AS event_date,
s.migrationId,
s.segment_count,
h.hightouch_count,
s.segment_count - h.hightouch_count AS count_difference,
CASE
WHEN s.segment_count = h.hightouch_count THEN 'Match'
ELSE 'Mismatch'
END AS comparison_result
FROM segment_identifies s
FULL OUTER JOIN hightouch_identifies h ON s.event_date = h.event_date AND s.migrationId = h.migrationId
WHERE s.migrationId IS NOT NULL OR h.migrationId IS NOT NULL
ORDER BY event_date, migrationId;
```
### Compare property values
We also need to validate that Hightouch Events is collecting the same values as your prior provider.
The query below looks at a selection of properties relevant to `identifies` calls and compares between the two platforms over a 7-day period.
You can modify this query to examine different properties, use a different time window, or look at data in a narrower set of dates.
```sql
WITH segment_data AS (
SELECT
id AS segment_id,
migrationId,
anonymous_id,
user_id,
timestamp,
email,
name
FROM SEGMENT.identifies
WHERE timestamp >= DATEADD(day, -7, CURRENT_DATE())
),
hightouch_data AS (
SELECT
id AS hightouch_id,
migrationId,
anonymous_id,
user_id,
timestamp,
email,
name
FROM HIGHTOUCH.identifies
WHERE timestamp >= DATEADD(day, -7, CURRENT_DATE())
)
SELECT
s.migrationId,
COALESCE(s.user_id, h.user_id) AS user_id,
s.segment_id,
h.hightouch_id,
s.anonymous_id AS segment_anonymous_id,
h.anonymous_id AS hightouch_anonymous_id,
s.timestamp AS segment_timestamp,
h.timestamp AS hightouch_timestamp,
CASE WHEN s.email = h.email THEN 'Match' ELSE 'Mismatch' END AS email_comparison,
CASE WHEN s.name = h.name THEN 'Match' ELSE 'Mismatch' END AS name_comparison,
s.email AS segment_email,
h.email AS hightouch_email,
s.name AS segment_name,
h.name AS hightouch_name
FROM segment_data s
FULL OUTER JOIN hightouch_data h ON s.migrationId = h.migrationId
WHERE
s.email != h.email
OR s.name != h.name
OR s.anonymous_id != h.anonymous_id
OR s.migrationId IS NULL
OR h.migrationId IS NULL
ORDER BY s.timestamp DESC
LIMIT 100; -- Limit to 100 rows for a manageable sample
```
## What differences to expect between your old provider and Hightouch Events
Event volume should be approximately the same, but
some discrepancies between systems are normal. Deployment rollout, ad blockers, and network errors could all cause events to appear in one tool but not another, leading to differences in volume.
1. **Timing differences**: Events may be processed in slightly different orders or with small time variations.
2. **Dropped events**: Network issues might cause events to be lost in one system but not the other.
3. **Duplicate events**: Some events might be sent twice in edge cases (for example, page reloads).
4. **Blocking behavior**: Other providers might not block events as strictly as Hightouch does when running type checks.
Hightouch Events should collect the same values as your previous event collection provider, though there may be minor variations in [automatically collected fields](https://hightouch.com/docs/events/event-spec).
In the next section, we'll explore how to [unify your historical data with new Hightouch data](unifying-data).
---
## Events overview
**URL:** https://hightouch.com/docs/events/overview
**Description:** Learn what Hightouch Events is used for, how it works, and how event data flows through Hightouch.
**Section:** Hightouch Events
| Audience | How you’ll use this article |
|--------|-----------------------------|
| **Marketing** | Understand what event data is used for and how it supports activation, personalization, and experimentation. |
| **Data teams** | Learn how events are collected, structured, stored, and reused across analytics and activation workflows. |
| **Engineering teams** | Understand the event model, ingestion paths, and how Events fits into your data architecture. |
## Overview
Hightouch **Events** collects customer behavior from websites, mobile apps, and backend systems and **centralizes that data in your data warehouse**. You can then reuse the same events across analytics, activation, and real-time workflows.
Events represent meaningful actions, such as viewing a product, completing a purchase, or updating an account. Each event includes properties and identity information, and Events applies consistent definitions and identity rules so the data stays reliable as it moves to downstream tools.
This article explains what Events are, what you’ll use them for, and how they fit into Hightouch.
If you’re ready to start implementing tracking, see
→ [**Get started with Hightouch Events**](/events/get-started)
---
## Use cases
- **Analytics**
Analyze customer behavior, measure campaign performance, and identify trends and opportunities.
- **Marketing personalization**
Power personalized messaging and experiences based on recent and historical behavior.
- **Operational workflows**
Trigger actions such as sending emails, updating CRM records, or routing events to internal systems.
- **AI Decisioning and machine learning**
Train and evaluate models for use cases such as churn prediction, product recommendations, propensity scoring, and fraud detection.
---
## Core building blocks
Hightouch Events has three core building blocks that make up collection to activation.
### Event sources
Where events come from.
Examples include browser SDKs, mobile SDKs, server-side SDKs, HTTP APIs, and streaming platforms.
### Event destinations
Where events go.
Destinations can store events in your warehouse or forward events to downstream tools.
### Event syncs
How events move between sources and destinations.
Each sync defines which events are sent, how they’re filtered, and how data is mapped.
---
## What you can do after events are collected
Once events are flowing into Hightouch, you can:
- **Store events in your warehouse** for analytics and modeling
- **Stream events to destinations** in real time for time-sensitive workflows
- **Build audiences and traits** using event data in Customer Studio
- **Apply governance rules** with data contracts
- **Transform events** in-flight using event functions
- **Use events in AI Decisioning** as real-time signals
---
## How Events fits into the rest of Hightouch
Hightouch Events integrates with other parts of the platform:
- **Customer Studio**: build audiences, traits, and journeys using event data
- **Reverse ETL**: sync modeled event data from the warehouse to downstream tools
- **Identity Resolution**: apply consistent identity rules across event and warehouse data
- **AI Decisioning**: use events as real-time inputs for decisioning and personalization
---
## Migrating to Hightouch Events
Use Hightouch Events when you need more control and flexibility over how event data is collected, governed, and activated. Benefits include:
- **Flexible event schemas**
Define and evolve event schemas without being constrained by predefined models.
- **Composable architecture**
Run event collection on your own data infrastructure, with modular components that integrate with existing tools and workflows.
- **Simple, consistent tracking APIs**
Instrument events using a straightforward API that focuses on what to track, while keeping downstream behavior configurable.
- **Warehouse-first data flow**
Centralize event data in the warehouse so it can be reused across analytics, activation, and machine learning use cases.
- **Enterprise governance and observability**
Apply data contracts, version control, and monitoring to maintain quality and visibility as event volume grows.
- **Migration-friendly implementation**
Keep existing tracking APIs (for example, Segment- or RudderStack-style calls) and run systems in parallel during migration.
Hightouch Events supports both real-time activation and warehouse-based analysis from the same underlying event data.
If you are migrating, see the [**Migration guide**](/events/migration-guide/overview).
---
## What to do next
If you’re new to Events, start with the step-by-step implementation guide:
→ [**Get started with Hightouch Events**](/events/get-started)
→ [**Migrate to Hightouch Events**](/events/migration-guide/overview).
---
## Android SDK
**URL:** https://hightouch.com/docs/events/sdks/android
**Description:** Hightouch's Android SDK for collecting events
**Section:** Hightouch Events
The Android SDK makes it easy to track events from Java and Kotlin Android applications.
## Installation
The Android SDK is hosted on [Jitpack](https://jitpack.io/#ht-sdks/events-sdk-android/).
You may install it via Gradle by adding it to your `settings.gradle` and `build.gradle`.
```
// settings.gradle
allprojects {
repositories {
maven { url 'https://jitpack.io' }
}
}
```
```
// build.gradle
dependencies {
implementation 'com.github.ht-sdks.events-sdk-android:analytics:0.+'
}
```
## Initialization
You should initialize the SDK in a central location and store it in a variable
for use across your application. For example, you may want to initialize it in
your application's `onCreate` callback. After initialization, you can use the
SDK anywhere you import the `Analytics` class via the singleton instance.
**Kotlin example**:
```
val analytics = Analytics.Builder(context, "YOUR_WRITE_KEY")
.defaultApiHost("us-east-1.hightouch-events.com/v1")
.trackApplicationLifecycleEvents()
.build()
Analytics.setSingletonInstance(analytics)
```
**Java example**:
```
Analytics analytics = new Analytics.Builder(context, "YOUR_WRITE_KEY")
.defaultApiHost("us-east-1.hightouch-events.com/v1")
.trackApplicationLifecycleEvents()
.build();
Analytics.setSingletonInstance(analytics);
```
**Configuration options**:
| Option | Type | Description |
| --------------------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------- |
| `defaultApiHost` | String | The API host to send the tracked events to. |
| `flushQueueSize` | int | The number of events to queue before sending events to the Hightouch API. Defaults to 20. |
| `flushInterval` | int | The maximum amount of time (in seconds) to store events locally before forwarding to Hightouch. Defaults to 30 seconds. |
| `trackApplicationLifecycleEvents` | n/a | Automatically track application lifecycle events (like opening the application). |
| `recordScreenViews` | n/a | Automatically track screen views. |
| `trackDeepLinks` | n/a | Automatically track deep links. |
### Context
By default, Hightouch automatically collects contextual metadata with each event, such as the device operating system. This context is forwarded with each event.
#### Modify global context
To customize the default context sent with every event, use the `getAnalyticsContext()` method. This allows you to add or remove metadata that will be included with all subsequent events.
For example, to set UTM campaign parameters globally:
```
AnalyticsContext context = Analytics.with(context).getAnalyticsContext();
context.putValue("custom_key", "custom_value");
```
This applies the campaign data to every future event in the session.
For more information, see:
- [AnalyticsContext.java](https://github.com/ht-sdks/events-sdk-android/blob/main/analytics/src/main/java/com/hightouch/analytics/AnalyticsContext.java)
- [Android SDK README – Context](https://github.com/ht-sdks/events-sdk-android#context)
#### Override context per event
You can override the context for an individual `track()` or `identify()` call by passing an `Options` object as the third argument. This is useful when you want to apply custom context — like UTM campaign parameters — to a single event.
```
ValueMap campaign = new ValueMap()
.putValue("utm_source", "adwords")
.putValue("utm_medium", "cpc")
.putValue("utm_campaign", "product_launch");
ValueMap contextMap = new ValueMap().putValue("campaign", campaign);
Options options = new Options().setContext(contextMap);
Analytics.with(context).track("Product Viewed", null, options);
```
This will attach campaign context only to the specified event.
For implementation details, see:
- [Options.java](https://github.com/ht-sdks/events-sdk-android/blob/main/analytics/src/main/java/com/hightouch/analytics/Options.java)
- [Android SDK README – Context](https://github.com/ht-sdks/events-sdk-android#context)
## API
### Identify
The `identify` method sends an `identify` event. It accepts user id and trait
information for the current user -- if you only have partial information, you
can just supply a single parameter.
**Java method signature**:
```
Analytics.with(context).identify("userId", new Traits().putEmail("email"), null);
```
**Kotlin method signature**:
```
Analytics.with(context).identify("userId", Traits().putEmail("email"), null);
```
**Method parameters**:
| Parameter | Type | Description |
| --------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------- |
| `userId` | String | The user's persistent ID |
| `traits` | Traits (optional) | Additional traits about the user, such as `email` and `name`. |
| `options` | Options (optional) | Overrides to the event's context may be passed through the `options` object. This includes setting custom fields like [campaign data](#override-context-per-event). *Note that this overrides the default context.* |
### Track
The `track` method sends a `track` event. The event name is required, and
additional properties that describe the event may be supplied.
**Java method signature**:
```
Analytics.with(context).track("eventName", new Properties().putValue("custom", "property"), null);
```
**Kotlin method signature**:
```
Analytics.with(context).track("eventName", Properties().putValue("custom", "property"), null);
```
**Method parameters**:
| Parameter | Type | Description |
| ------------ | --------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `name` | String | The name of the event. |
| `properties` | Properties (optional) | Additional properties about the event, such as `product_id`. |
| `options` | Options (optional) | Overrides to the event's context may be passed through the `options` object. Note that this overrides the default context. |
### Screen
The `screen` method sends a `screen` event. The screen title is required, and
additional information about the screen's category and properties may be
supplied.
**Java method signature**:
```
Analytics.with(context).screen("category", "name", new Properties().putValue("custom", "property"), null);
```
**Kotlin method signature**:
```
Analytics.with(context).screen("category", "name", Properties().putValue("custom", "property"), null);
```
**Method parameters**:
| Parameter | Type | Description |
| ------------ | ----------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `category` | String (optional if name is supplied) | The screen's category. For example "Docs" |
| `title` | String (optional if category is supplied) | The screen's name. For example "Getting started" |
| `properties` | Properties (optional) | Additional properties about the event, such as `from_link`. |
| `options` | Options (optional) | Overrides to the event's context may be passed through the `options` object. Note that this overrides the default context. |
### Group
The `group` method sends a `group` event.
The id identifying the user's group is required. Additional traits on the group
can be provided.
**Java method signature**:
```
Analytics.with(context).group("groupId", new Traits().putValue("custom", "property"), null);
```
**Kotlin method signature**:
```
Analytics.with(context).group("groupId", Traits().putValue("custom", "property"), null);
```
**Method parameters**:
| Parameter | Type | Description |
| --------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------- |
| `groupId` | String | The id for the group. |
| `traits` | Traits (optional) | Additional traits about the group, such as `company_name`. |
| `options` | Options (optional) | Overrides to the event's context may be passed through the `options` object. Note that this overrides the default context. |
### Flush
The Android SDK buffers events locally before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and makes the tracking
non-blocking.
To force the local buffer to be sent to Hightouch immediately call the `flush`
method.
**Method signature**:
```
Analytics.with(context).flush()
```
### Reset
The `reset` method resets the `identify` and `group` for the local session.
The `reset` method should be called when users log out. This way, if the user
logs back in with another account, the userIds and traits from the different
sessions remain separate.
**Method signature**:
```
Analytics.with(context).reset()
```
### Opt out
The `optOut` method disabled event tracking. When called, events are not sent
to Hightouch's servers.
Note that the opt out persists across device reboots. To opt back in to
tracking, pass `false` to `optOut`.
**Method signature**:
```
Analytics.with(context).optOut(true)
```
---
## Browser (Javascript) SDK
**URL:** https://hightouch.com/docs/events/sdks/browser
**Description:** Hightouch's browser Javascript SDK for collecting events
**Section:** Hightouch Events
The Browser SDK makes it easy to track events in webpages.
## Installation
The Browser SDK can be installed by loading the Javascript via a `script` tag, or directly bundled via a `npm` package.
To install using a `script` tag, add the following snippet to your site, replacing WRITE_KEY with the write key from your Event Source:
```
```
To install using `npm`:
1. Install the SDK by running `npm install @ht-sdks/events-sdk-js-browser`
2. Import and initialize the SDK with the following:
```jsx
{ writeKey: "WRITE_KEY" },
{ apiHost: "us-east-1.hightouch-events.com" }
);
```
## API
### Identify
The `identify` method is used to send an `identify` event.
If `identify` is called multiple times for the same user, the `traits` are
merged together.
**Method signature**:
```
htevents.identify(userId, [traits], [context], [callback])
```
**Method parameters**:
| Parameter | Type | Description |
| ---------- | -------- | ---------------------------------------------------------------------------------------------------------------- |
| `userId` | String | The user's persistent ID. |
| `traits` | Object | Additional traits about the user, such as `email` and `name`. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `callback` | Function | A callback that's invoked after the event is sent. |
### Track
The `track` method is used to send a `track` event.
**Method signature**:
```
htevents.track(event, [properties], [context], [callback])
```
**Method parameters**:
| Parameter | Type | Description |
| ------------ | -------- | ---------------------------------------------------------------------------------------------------------------- |
| `event` | String | The event name (for example "Checked Out"). |
| `properties` | Object | Additional properties about the event, such as `product_id`. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `callback` | Function | A callback that's invoked after the event is sent. |
### Page
The `page` method is used to send a `page` event.
**Method signature**:
```
htevents.page(category, name, [properties], [context], [callback])
```
**Method parameters**:
| Parameter | Type | Description |
| ------------ | -------- | ------------------------------------------------------------------------------------------------------------------------ |
| `category` | String | The page's category. For example, "Docs". |
| `name` | String | The page's name. For example, "Getting started". |
| `properties` | Object | Additional properties about the event, such as `url`. By default, the SDK autocollects properties such as page referrer. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `callback` | Function | A callback that's invoked after the event is sent. |
### Group
The `group` method sends a `group` event.
**Method signature**:
```
htevents.group(groupId, [traits], [context], [callback])
```
**Method parameters**:
| Parameter | Type | Description |
| ---------- | -------- | ---------------------------------------------------------------------------------------------------------------- |
| `groupId` | String | The id for the group. |
| `traits` | Object | Additional traits about the group, such as `company_name`. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `callback` | Function | A callback that's invoked after the event is sent. |
### Reset
The `reset` method resets the `identify` and `group` for the local session.
Specifically, it resets the `anonymousId`, `userId`, `groupId`, and `traits`.
The `reset` method should be called when users log out. This way, if the user
logs back in with another account, the userIds and traits from the different
sessions remain separate.
**Method signature**:
```
htevents.reset()
```
## Single page applications
While Single Page Apps (SPAs) are great for many reasons, they do require some extra consideration in order to set up client-side tracking than with a traditional webpage.
By default, the Hightouch htevents.js library doesn't generate or store the referrer value. Instead, the referrer value you see in the payload is the value returned by `document.referrer` directly from the browser, and the URL value is the canonical URL on the page.
When a user navigates between pages on an SPA website, there won't be a referrer because there is no concept of a new page since it's all a single page load. This means that the referrer will always be the same as it was on the first page call where someone was first directed to your site. However, in order to circumvent this, you can manually set the referrer and URL in your Hightouch calls by updating the context object.
For example, a Page call with the referrer and URL manually set looks like this:
```js
htevents.page({
referrer: 'https://hightouch.com/',
url: 'https://hightouch.com/pricing/?ref=nav'
})
```
A Track call with these fields manually updated looks like this:
```js
htevents.track('Example Event', {}, {page: {
referrer: 'https://hightouch.com/',
url: 'https://hightouch.com/pricing/?ref=nav'
}})
```
### Tracking emulated page views
Your application should update the URL in the address bar to emulate traditional webpage navigation. Full page requests aren't made in most of these instances since the resources are loaded on initial page load. This means that the Page call in the traditional htevents.js snippet won't fire again as a user navigates around your site.
You should still place the snippet in the head of your site, but you should remove the Page call and fire it whenever you're emulating a page load. Hightouch recommends that you call [Page](/events/sdks/browser#page) from the same block of logic that updates the view and URL path like below:
```js
// The new view has been called to render
htevents.page("Home")
```
To track more than the page field, pass those fields in as additional properties. Hightouch recommends that you use variables to set information about page properties, rather than hard-coding. In most SPA frameworks, you can automate this by attaching the Page call to the routing service.
## QueryString API
The Browser SDK can trigger `track` and `identify` events based on the URL query string. You can use this when tracking email click-throughs, social media clicks, and digital advertising.
**Query parameters**:
| Parameter | Description | Triggers |
| --------------------- | --------------------------------------- | ------------------------------------------------------------------------------------------ |
| `ajs_uid` | The `userId` for the `identify` event. | This triggers an `identify` event. |
| `ajs_event` | The `name` for the `track` event. | This triggers a `track` event. |
| `ajs_aid` | The `anonymousId` for the user. | This sets the `anonymousId`, by itself does not trigger any event. |
| `ajs_prop_` | A property to set on the `track` event. | This sets a property on the resulting `track` event, by itself does not trigger any event. |
| `ajs_trait_` | A trait to set on the `identity` event. | This sets a trait on the resulting `identify` event, by itself does not trigger any event. |
### Example
The following URL:
```
https://hightouch.com/?ajs_uid=123&ajs_event=Clicked%20Email&ajs_aid=abc123&ajs_prop_emailCampaign=First+Touch&ajs_trait_name=John
```
would result in these calls:
```js
htevents.identify('123', { name: 'John' });
htevents.track('Clicked Email', { 'emailCampaign': 'First Touch' });
htevents.user().anonymousId('abc123');
```
### Configuration
The `useQueryString` option allows you to control the behavior of the query parameters. For example, you can entirely disable query string processing by setting `useQueryString` to `false`:
```js
htevents.load('WRITE_KEY', {
useQueryString: false
})
```
You can also keep query string processing on, but enforce validation rules. For example:
```js
htevents.load('WRITE_KEY', {
useQueryString: {
// set a pattern for anonymous id
aid: /([A-Z]{10})/,
// set a pattern for user id
uid: /([A-Z]{6})/
}
})
```
## Cross Domain Tracking
Using the [QueryString API](#querystring-api) you can easily track users across different domains. For example, if you have the following sites:
- Site 1: `www.landing-page.com`
- Site 2: `www.product-page.com`
To track a user who starts on Site 1 and navigates to Site 2, you just need to pass the `userId` from Site 1 to Site 2 using the `ajs_uid` query parameter.
```
https://www.product-page.com/?ajs_uid=123
```
This will automatically trigger an `identify` event on Site 2 with `userId: 123`.
If you want to track anonymous users across domains, you can access the `anonymousId` on Site 1 using:
```js
htevents.user().anonymousId()
```
You can then pass the `anonymousId` from Site 1 to Site 2 using the `ajs_aid` query parameter. Note that the `ajs_aid` parameter does not automatically trigger an `identify` event.
## Autocollected fields
The browser SDK autocollects the following fields within the `context` of each event:
| Field | Description |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| `page.path` | The current page's sanitized path, i.e. with search parameters and the site domain removed. |
| `page.referrer` | The referrer to the current page. An empty string denotes a direct referrer. |
| `page.search` | The search parameter of the current page, if any |
| `page.title` | The current page's title |
| `page.url` | The current page's full URL |
| `sessionId` | An identifier for the visitor's current session. |
| `userAgent` | The raw user agent string |
| `userAgentData` | Parsed information on the user agent. This includes whether the visitor's device is a mobile device, and the platform of the device. |
| `locale` | The browser's locale |
| `library` | Information about the SDK version used to send the event. |
| `timezone` | The visitor's local timezone. |
## Session tracking
A session is a group of "uninterrupted" events that were performed by the same visitor.
The session resets once the session's timeout has expired, and automatically refreshes when a new event is sent.
For example, if the visitor views a couple pages, and then returns to your site after a day, the next day's events will be considered a new session.
The browser SDK automatically tracks a `sessionId` field to track what events occurred in the same session. If two events occured in the same session, they
will have the same session ID.
The session timeout can be adjusted from the default of **30 minutes** when the SDK is initialized:
```js
e.load('YOUR_WRITE_KEY', {
apiHost: 'us-east-1.hightouch-events.com',
user: {
sessions: {
autoTrack:true,
// Set the timeout to 60 minutes.
timeout: 60*60*1000,
}
}
})
```
## Campaign Attribution (UTM)
Hightouch automatically captures UTM parameters that you pass in URLs. You can also manually pass UTMs into the `context.campaign` field.
UTM parameters are only used when linking to your site from outside of your domain. When a visitor arrives to your site using a link containing UTM parameters, Hightouch's Browser SDK will automatically parse the URL query string and store UTM parameters within the `context.campaign` object as outlined here. These parameters do not persist to subsequent calls unless you pass them explicitly.
**UTM parameters**:
| Parameter | Mapping | Description |
| -------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `utm_campaign` | `campaign.name` | This is the name of your campaign. All marketing activities that support this campaign need to have the same `utm_campaign` so that downstream analysis to measure performance for this specific campaign can be done off this primary key. |
| `utm_medium` | `campaign.medium` | How the traffic is coming to your site. Is it through email, a display ad, or an online forum? This ensures downstream analysis can easily see which channel performs the best.
eg. `email`, `paid-display`, `paid-social`, `organic-social` |
| `utm_source` | `campaign.source` | Where the traffic is specifically coming from. You can be specific here. This ensures downstream analysis can measure which specific source brings the most conversions.
eg. `x`, `facebook`, `adroll` |
| `utm_content` | `campaign.content` | For multiple calls to action on a single page, `utm_content` indicates which one. For example, on a website, there may be three different display ads. While the link on each display ad will have the same `utm_campaign`, `utm_medium`, and `utm_source`, the `utm_content` will be different.
eg. `banner`, `left-side`, `bottom-side` |
| `utm_term` | `campaign.term` | This is the parameter suggested for paid search to identify keywords for your ad. If you’re using Google Adwords and have enabled “autotagging”, then you don’t need to worry about this. Otherwise, you can manually pass the keywords from your search terms through this parameter so that you can see which keywords convert the most. Note that this parameter is reserved explicitly for search.
eg. `toast`, `butter`, `jam` |
If you’d like UTM parameters to persist in subsequent calls, you’ll need to manually set those fields in `context.campaign`. For example:
```js
htevents.page("My Page", {}, {
campaign: {
name: "TPS Innovation Newsletter",
source: "Newsletter",
medium: "email",
term: "tps reports",
content: "image link"
},
});
```
## Cookies and LocalStorage
Hightouch stores cookies to track information about website visitors across multiple visits.
- `htjs_user_id`: ID tracking known or identified users as defined [here](/events/event-spec#identify-events#identify-events).
- `htjs_anonymous_id`: ID tracking anonymous website visitors as defined [here](/events/event-spec#identify-events#identify-events).
- `htjs_group_id`: ID tracking the group a user is associated with as defined [here](/events/event-spec#identify-events#group-events).
- `htjs_sesh`: Information about the user's current session as defined [here](/events/sdks/browser#session-tracking).
In addition to tracking this information in cookies, we use localStorage to track this as well. In localStorage, we also track:
- `htjs_user_traits`: Traits defined about the user being tracked
- `htjs_group_properties`: Properties defined about the group being tracked
For more information on measures to increase persistence of cookies, please refer to our guide on [proxying and server-side cookies](/events/first-party-tracking).
---
## C# SDK
**URL:** https://hightouch.com/docs/events/sdks/csharp
**Description:** Hightouch's C# SDK for collecting events
**Section:** Hightouch Events
The C# SDK makes it easy to track events from all C#/.NET applications.
## Installation
The C# SDK can be installed from [NuGet](https://www.nuget.org/packages/Hightouch.Events.CSharp):
```sh
dotnet add package Hightouch.Events.CSharp
```
To initialize the C# SDK in your application, create a new `Analytics` instance:
```c#
using Hightouch.Events;
using Hightouch.Events.Serialization;
var configuration = new Configuration("WRITE_KEY",
apiHost: "https://us-east-1.hightouch-events.com",
flushAt: 1);
var analytics = new Analytics(configuration);
```
## API
### Identify
The `analytics.Identify` method sends an [`identify`](../event-spec#identify-events) event.
**Example usage**:
```c#
analytics.Identify("user-123", new JsonObject {
["username"] = "MisterWhiskers",
["email"] = "hello@test.com",
["plan"] = "premium"
});
```
### Track
The `analytics.Track` method sends a [`track`](../event-spec#track-events) event.
**Example usage**:
```c#
analytics.Track("View Product", new JsonObject {
["productId"] = 123,
["productName"] = "Striped trousers"
});
```
### Page
The `analytics.Page` method sends a [`page`](../event-spec#page-events) event.
**Example usage**:
```c#
analytics.Page("PageName", new JsonObject {
["productSlug"] = "example-product-123"
});
```
### Screen
The `analytics.Screen` method sends a [`screen`](../event-spec#screen-events) event.
**Example usage**:
```c#
analytics.Screen("ScreenName", new JsonObject {
["productSlug"] = "example-product-123"
});
```
### Group
The `analytics.Group` method sends a [`group`](../event-spec#group-events) event.
**Example usage**:
```c#
analytics.Group("user-123", new JsonObject {
["username"] = "MisterWhiskers",
["email"] = "hello@test.com",
["plan"] = "premium"
});
```
### Flush
The C# SDK buffers events locally, either in-memory or on disk, before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and makes the tracking non-blocking.
To force the local buffer to be sent to Hightouch immediately call the `Flush()`
method. `Flush()` should be called when shutting down your app to make sure no events are lost.
**Example usage**:
```c#
analytics.Flush()
```
---
## Flutter SDK
**URL:** https://hightouch.com/docs/events/sdks/flutter
**Description:** Hightouch's Flutter SDK for collecting events
**Section:** Hightouch Events
The Flutter SDK makes it easy to track events from all types of Flutter applications.
## Installation
The Flutter SDK is hosted on [pub.dev](https://pub.dev/packages/hightouch_events).
You may install the SDK by adding it to your `pubspec.yaml` file:
```sh
flutter pub add hightouch_events
```
## Initialization
You should initialize the SDK in the top-level app state and store it in a variable
for use across your application. For example, you may want to initialize it in
your app delegate.
**Example**:
```
final analytics = createClient(Configuration("WRITE_KEY"));
```
**Configuration options**:
| Option | Type | Description |
| --------------------------------- | ------ | ---------------------------------------------------------------------------------------------------------------- |
| `writeKey` | String | Your Hightouch write key. |
| `apiHost` | String | The API host to send the tracked events to. |
| `trackApplicationLifecycleEvents` | Bool | Whether to automatically track application lifecycle events (like opening the application). Defaults to `false`. |
## API
### Identify
The `analytics.identify` method sends an [`identify`](../event-spec#identify-events) event.
**Example usage**:
```
analytics.identify(userId: "user-123", userTraits: UserTraits(email: "user@example.com"));
```
### Track
The `analytics.track` method sends a [`track`](../event-spec#track-events) event.
**Example usage**:
```
analytics.track("View Product", properties: {"productId": 123});
```
### Screen
The `analytics.screen` method sends a [`screen`](../event-spec#screen-events) event.
**Example usage**:
```
analytics.screen("Home");
```
### Group
The `analytics.group` method sends a [`group`](../event-spec#group-events) event.
**Example usage**:
```
analytics.group("group-123", groupTraits: GroupTraits(name: "Group 123"));
```
### Flush
The Flutter SDK buffers events locally before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and makes the tracking
non-blocking.
To force the local buffer to be sent to Hightouch immediately call the `flush()`
method.
### Reset
The `reset` method resets the `identify` and `group` for the local session.
Specifically, it resets the `anonymousId`, `userId`, `referrer`, and `traits`.
The `reset` method should be called when users log out. This way, if the user
logs back in with another account, the userIds and traits from the different
sessions remain separate.
---
## Go SDK
**URL:** https://hightouch.com/docs/events/sdks/go
**Description:** Hightouch's Go SDK for collecting events
**Section:** Hightouch Events
The Go SDK makes it easy to track events from Go applications.
## Installation
The Go SDK can be installed from [GitHub](https://github.com/ht-sdks/events-sdk-go):
```
go get github.com/ht-sdks/events-sdk-go
```
To initialize the Go SDK in your application, create a new `Client` instance:
```
package main
"github.com/ht-sdks/events-sdk-go"
)
func main() {
client, _ := htevents.NewWithConfig("WRITE_KEY", htevents.Config{
Endpoint: "https://us-east-1.hightouch-events.com"
})
defer client.Close()
}
```
## API
### Identify
The `htevents.Identify` message sends an `identify` event.
Unlike the [Browser SDK](./browser), it does **not** persist the user ID and
traits locally, so user IDs must be explicitly added to other events. This is
because server side events are usually servicing many different users at once.
**Example usage**:
```
client.Enqueue(htevents.Identify{
UserId: "123",
Traits: htevents.Traits{
"location": "San Francisco",
},
})
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `UserId` | string | The user's persistent ID |
| `AnonymousId` | string | The user's anonymous ID |
| `Traits` | Traits | Additional traits about the user, such as `email` and `name`. |
| `Context` | Context | Overrides to values in the event `Context`. By default, `Context` contains information autocollected by the SDK. |
| `Timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Track
The `htevents.Track` message sends a `track` event.
**Example usage**:
```
client.Enqueue(htevents.Track{
Event: "Order completed",
UserId: "123",
Properties: htevents.Properties{
"total": 29.99,
},
})
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `UserId` | string | The user's persistent ID |
| `AnonymousId` | string | The user's anonymous ID |
| `Event` | string | The name of the event. |
| `Properties` | Properties | Additional properties about the event, such as `product_id`. |
| `Context` | Context | Overrides to values in the event `Context`. By default, `Context` contains information autocollected by the SDK. |
| `Timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Page
The `htevents.Page` message sends a `page` event.
**Example usage**:
```
client.Enqueue(htevents.Page{
Name: "Getting started",
UserId: "123",
Properties: htevents.Properties{
"total": 29.99,
},
})
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `UserId` | string | The user's persistent ID |
| `AnonymousId` | string | The user's anonymous ID |
| `Name` | string | The page's name. For example "Getting started" |
| `Properties` | Properties | Additional properties about the event, such as `url`. |
| `Context` | Context | Overrides to values in the event `Context`. By default, `Context` contains information autocollected by the SDK. |
| `Timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Screen
The `htevents.Screen` message sends a `screen` event.
**Example usage**:
```
client.Enqueue(htevents.Screen{
Name: "Getting started",
UserId: "123",
Properties: htevents.Properties{
"total": 29.99,
},
})
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `UserId` | string | The user's persistent ID |
| `AnonymousId` | string | The user's anonymous ID |
| `Name` | string | The page's name. For example "Getting started" |
| `Properties` | Properties | Additional properties about the event, such as `url`. |
| `Context` | Context | Overrides to values in the event `Context`. By default, `Context` contains information autocollected by the SDK. |
| `Timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Group
The `htevents.Group` message sends a `group` event.
**Example usage**:
```
client.Enqueue(htevents.Group{
GroupId: "G-1",
UserId: "123",
Traits: htevents.Traits{
"company_location": "San Francisco",
},
})
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `UserId` | string | The user's persistent ID |
| `AnonymousId` | string | The user's anonymous ID |
| `GroupId` | string | The id for the group. |
| `Traits` | Traits | Additional traits about the group, such as `company_name`. |
| `Context` | Context | Overrides to values in the event `Context`. By default, `Context` contains information autocollected by the SDK. |
| `Timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Close
The Go SDK buffers events locally before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and makes the tracking
non-blocking.
To force the local buffer to be sent to Hightouch immediately call the `Close`
method. `Close` should be called when shutting down your app to make sure no events are lost.
**Example usage**:
```
client.Close()
```
---
## HTTP API
**URL:** https://hightouch.com/docs/events/sdks/http
**Description:** Hightouch's HTTP API endpoint for collecting events
**Section:** Hightouch Events
## HTTP API Source
Hightouch's HTTP Tracking API enables you to record data from any website or application.
## Headers
### Authentication
To authenticate Hightouch's tracking API, send your projects Write Key along with a request. Authentication uses HTTP Basic Auth.
To do this, you should include an Authorization header that includes `Basic` followed by your base64 encoded write key. This looks like this:
`Authorization: Basic YmJjYmU434YTNmRlNWRkYjM3MDUzMDRlMzN2JjM2NiY2RjYWJmZTA5YTA3ZGYxWNkNjQ5Zg==`
## Content-Type
Ensure your content-type header is set to 'application/json'
## Identify
**Identify events** capture data about _who_ your users are. This user context is vital for understanding and targeting your users. For example, you might use the collected identifiers in followup email campaigns, to segment your users by location, or to link anonymous and authenticated events.
_Example `identify` call:_
```
POST https://us-east-1.hightouch-events.com/v1/identify
```
```
{
"type": "identify",
"messageId": "111e984d-c93c-444c-b29c-f499a117c500",
"traits": {
"email": "kevin@hightouch.io"
},
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"originalTimestamp": "2023-09-22T18:05:47.063Z",
"sentAt": "2023-09-22T18:05:47.064Z"
}
```
## Track
**Track events** capture data on _what_ the user is doing. Track events can capture a broad range of actions, such as clicking a button, completing a purchase, or creating a new account.
_Example `track` call_
```
POST https://us-east-1.hightouch-events.com/v1/track
```
```
{
"type": "track",
"messageId": "7b1e2872-4ebb-4779-b9ad-e7aa1a8eab82",
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"event": "Purchase completed",
"properties": {
"cart_total": 24.99
},
"originalTimestamp": "2023-09-22T18:12:00.584Z",
"sentAt": "2023-09-22T18:12:00.585Z"
}
```
## Page events
**Page events** track the webpages users interact with. In most situations, you should automatically call the page load event when a page is loaded.
For single page apps, you should simulate page events in your routing layer.
_Example `page` call_
```
POST https://us-east-1.hightouch-events.com/v1/page
```
```
{
"type": "page",
"messageId": "bb9fcbbd-9e19-4bfd-a726-7b1e7e44f8ba",
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"properties": {
"title": "Home"
},
"originalTimestamp": "2023-09-22T18:05:47.061Z",
"sentAt": "2023-09-22T18:05:47.063Z"
}
```
## Screen events
**Screen events** are the equivalent of page views for _mobile apps_. Like page events, you should automatically call the screen event when a mobile screen view is loaded.
_Example `screen` call_
```
POST https://us-east-1.hightouch-events.com/v1/screen
```
```
{
"type": "screen",
"messageId": "bb9fcbbd-9e19-4bfd-a726-7b1e7e44f8ba",
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"properties": {
"title": "Home"
},
"originalTimestamp": "2023-09-22T18:05:47.061Z",
"sentAt": "2023-09-22T18:05:47.063Z"
}
```
## Group events
Group events let you associate users with a larger group. For example, a user may be part of an organization, or family.
The `traits` field is optional, and can include information on traits on the group. For example, if linking a user to a company, you may include traits on the company such as the company size or location.
_Example `group` call_
```
POST https://us-east-1.hightouch-events.com/v1/group
```
```
{
"type": "group",
"messageId": "d87596da-5f1e-40f4-a28a-7099290548b7",
"anonymousId": "ce576487-95f9-4262-9315-2c7942b061c8",
"userId": "123",
"groupId": "456",
"traits": { "company_name": "Hightouch" },
"originalTimestamp": "2023-09-22T18:46:23.098Z",
"sentAt": "2023-09-22T18:46:23.099Z"
}
```
---
## iOS SDK
**URL:** https://hightouch.com/docs/events/sdks/ios
**Description:** Hightouch's iOS SDK for collecting events
**Section:** Hightouch Events
The iOS SDK makes it easy to track events from Swift and Objective-C applications.
## Installation
The iOS SDK is hosted on GitHub at
`git@github.com:ht-sdks/events-sdk-swift.git` under the name
`events-sdk-swift`.
You may install the SDK by adding it to your `package.swift` file, or adding it
as a Package Dependency via Xcode.
## Initialization
You should initialize the SDK in a central location and store it in a variable
for use across your application. For example, you may want to initialize it in
your app delegate.
**Swift example**:
```
let configuration = Configuration(writeKey: "YOUR_WRITE_KEY")
.apiHost("us-east-1.hightouch-events.com/v1")
analytics = Analytics(configuration: configuration)
```
**Objective-C example**:
```
@import Hightouch
HTConfiguration *config = [[HTConfiguration alloc] initWithWriteKey:@"YOUR_WRITE_KEY"];
config.apiHost = "us-east-1.hightouch-events.com/v1";
analytics = [[HTAnalytics alloc] initWithConfiguration: config];
```
**Configuration options**:
| Option | Type | Description |
| --------------------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------- |
| `writeKey` | String | Your Highotuch write key. |
| `apiHost` | String | The API host to send the tracked events to. |
| `flushAt` | Int | The number of events to queue before sending events to the Hightouch API. Defaults to 20. |
| `flushInterval` | Int | The maximum amount of time (in seconds) to store events locally before forwarding to Hightouch. Defaults to 30 seconds. |
| `trackApplicationLifecycleEvents` | Bool | Whether to automatically track application lifecycle events (like opening the application). Defaults to `true`. |
## API
### Identify
The `identify` method sends an `identify` event. It accepts user id and trait
information for the current user -- if you only have partial information, you
can just supply a single parameter.
**Method signature**:
```
func identify(userId: String, traits: T)
func identify(traits: T)
func identify(userId: String)
```
**Method parameters**:
| Parameter | Type | Description |
| --------- | ------- | ------------------------------------------------------------- |
| `userId` | String | The user's persistent ID |
| `traits` | Codable | Additional traits about the user, such as `email` and `name`. |
### Track
The `track` method sends a `track` event. The event name is required, and
additional properties that describe the event may be supplied.
**Method signature**:
```
func track(name: String)
func track(name: String, properties: P?)
```
**Method parameters**:
| Parameter | Type | Description |
| ------------ | ------- | ------------------------------------------------------------ |
| `name` | String | The name of the event. |
| `properties` | Codable | Additional properties about the event, such as `product_id`. |
### Screen
The `screen` method sends a `screen` event. The screen title is required, and
additional information about the screen's category and properties may be
supplied.
**Method signature**:
```
func screen(title: String, category: String? = nil)
func screen(title: String, category: String? = nil, properties: P?)
```
**Method parameters**:
| Parameter | Type | Description |
| ------------ | ------- | ----------------------------------------------------------- |
| `title` | String | The screen's name. For example "Getting started" |
| `category` | String | The screen's category. For example "Docs" |
| `properties` | Codable | Additional properties about the event, such as `from_link`. |
### Group
The `group` method sends a `group` event.
The id identifying the user's group is required. Additional traits on the group
can be provided.
**Method signature**:
```
func group(groupId: String)
func group(groupId: String, traits: T?)
```
**Method parameters**:
| Parameter | Type | Description |
| --------- | ------- | ---------------------------------------------------------- |
| `groupId` | String | The id for the group. |
| `traits` | Codable | Additional traits about the group, such as `company_name`. |
### Flush
The iOS SDK buffers events locally before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and makes the tracking
non-blocking.
To force the local buffer to be sent to Hightouch immediately call the `flush`
method.
**Method signature**:
```
func flush()
```
### Reset
The `reset` method resets the `identify` and `group` for the local session.
Specifically, it resets the `anonymousId`, `userId`, `referrer`, and `traits`.
The `reset` method should be called when users log out. This way, if the user
logs back in with another account, the userIds and traits from the different
sessions remain separate.
**Method signature**:
```
func reset()
```
---
## Java SDK
**URL:** https://hightouch.com/docs/events/sdks/java
**Description:** Hightouch's Java SDK for collecting events
**Section:** Hightouch Events
The Java SDK makes it easy to track events from Java applications.
## Installation
The Java SDK is available via
[JitPack](https://jitpack.io/#ht-sdks/events-sdk-java/).
MavenGradle
```
jitpack.iohttps://jitpack.iocom.github.ht-sdks.events-sdk-javaanalyticsLATEST
```
```
allprojects {
repositories {
...
maven { url 'https://jitpack.io' }
}
}
dependencies {
...
compile 'com.github.ht-sdks.events-sdk-java:analytics:+'
}
```
To initialize the Java SDK in your application, create an `Analytics` instance:
```
public class Main {
public static void main(String... args) throws Exception {
final Analytics analytics =
Analytics.builder(WRITE_KEY)
.endpoint(API_ENDPOINT)
.build();
}
}
```
## API
### Identify
The `IdentifyMessage` sends an `identify` event.
Unlike the [Browser SDK](./browser), it does **not** persist the user ID and
traits locally, so user IDs must be explicitly added to other events. This is
because server side events are usually servicing many different users at once.
**Example usage**:
```
analytics.enqueue(IdentifyMessage.builder()
.userId("123")
.traits(ImmutableMap.builder()
.put("name", "John Doe")
.put("email", "test@example.com")
.build()
)
);
```
**Method parameters**:
| Parameter | Type | Description |
|---------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `traits` | Map | Additional traits about the user, such as `email` and `name`. |
| `context` | Map | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Track
The `TrackMessage` sends a `track` event.
**Example usage**:
```
analytics.enqueue(TrackMessage.builder("Item Purchased")
.userId("123")
.properties(ImmutableMap.builder()
.put("revenue", 39.95)
.put("shipping", "2-day")
.build()
)
);
```
**Method parameters**:
| Parameter | Type | Description |
|---------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `event` | String | The name of the event. |
| `properties` | Map | Additional properties about the event, such as `product_id`. |
| `context` | Map | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Page
The `PageMessage` sends a `page` event.
**Example usage**:
```
analytics.enqueue(PageMessage.builder("Schedule")
.userId("123")
.properties(ImmutableMap.builder()
.put("category", "Sports")
.put("path", "/sports/schedule")
.build()
)
);
```
**Method parameters**:
| Parameter | Type | Description |
|---------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `name` | String | The page's name. For example "Getting started" |
| `properties` | Map | Additional properties about the event, such as `url`. |
| `context` | Map | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Screen
The `ScreenMessage` sends a `screen` event.
**Example usage**:
```
analytics.enqueue(ScreenMessage.builder("Schedule")
.userId("123")
.properties(ImmutableMap.builder()
.put("category", "Sports")
.put("path", "/sports/schedule")
.build()
)
);
```
**Method parameters**:
| Parameter | Type | Description |
|---------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `name` | String | The screen's name. For example "Getting started" |
| `category` | String | The screen's category. For example "Docs" |
| `properties` | Map | Additional properties about the event, such as `url`. |
| `context` | Map | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Group
The `GroupMessage` sends a `group` event.
**Example usage**:
```
analytics.enqueue(GroupMessage.builder("group123")
.userId("123")
.traits(ImmutableMap.builder()
.put("name", "Hightouch")
.put("size", 50)
.build()
)
);
```
**Method parameters**:
| Parameter | Type | Description |
|---------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `groupId` | String | The id for the group. |
| `traits` | Map | Additional traits about the group, such as `company_name`. |
| `context` | Map | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Flush
The Java SDK buffers events locally before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and makes the tracking
non-blocking.
To force the local buffer to be sent to Hightouch immediately call the `flush`
method. `flush` should be called when shutting down your Java app to
make sure no events are lost.
**Example usage**:
```
analytics.flush()
```
---
## Node.js SDK
**URL:** https://hightouch.com/docs/events/sdks/nodejs
**Description:** Hightouch's Node.js SDK for collecting events
**Section:** Hightouch Events
The Node.js SDK makes it easy to track events from Node.js applications.
## Installation
The Node.js SDK is available via NPM.
To install the Node.js SDK, run:
```
npm install @ht-sdks/events-sdk-js-node
```
Initialize the SDK exactly once and export it:
```
// initialize once
writeKey: "HT_WRITE_KEY",
host: "https://us-east-1.hightouch-events.com",
});
```
Then, import it into the rest of your app, as needed:
```
// import your already initialized instance into the rest of your app
// example usage in an express like framework
app.post('/login', (req, res) => {
htevents.identify({
userId: req.body.userId,
previousId: req.body.previousId
})
res.sendStatus(200)
})
```
The `HtEvents` constructor accepts the following parameters:
| Parameter | Type | Description |
| ---------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `writeKey` | String | Key that corresponds to your Hightouch source |
| `host` | String | The base URL of the API. Default: "https://us-east-1.hightouch-events.com" |
| `path` | String | The API path route. Default: "/v1/batch" |
| `maxRetries` | Number | The maximum number of attempts when flushing a batch. Default: 3 |
| `maxEventsInBatch` | Number | The maximum number of messages to enqueue before flushing. Default: 15 |
| `flushInterval` | Number | The maximum amount of time (in milliseconds) to store events locally before forwarding to Hightouch. Defaults to 10000. |
| `httpRequestTimeout` | Number | The maximum number of milliseconds to wait for an http request. Default: 10000 |
| `disable` | Boolean | Disable the analytics library. All calls will be a noop. Default: false |
## API
### Identify
The `identify` method sends an `identify` event.
Unlike the [Browser SDK](./browser), it does **not** persist the user ID and
traits locally, so user IDs must be explicitly added to other events. This is
because server side events are usually servicing many different users at once.
**Example usage**:
```
htevents.identify({
userId: "123",
traits: {
location: "San Francisco",
}
})
```
The first argument to the method is an object representing the identify event to be sent. The second argument is an optional callback that's invoked after the event is sent.
```
htevents.identify(message, [callback])
```
The message object may include the following fields:
| Parameter | Type | Description |
| ------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------- |
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `traits` | Object | Additional traits about the user, such as `email` and `name`. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to the past. By default, this is autoset to the current date. |
### Track
The `track` method sends a `track` event.
**Example usage**:
```
htevents.track({
userId: "123",
event: "Order completed",
properties: {
total: 29.99,
}
})
```
The first argument to the method is an object representing the track event to
be sent. The second argument is an optional callback that's invoked after the
event is sent.
```
htevents.track(message, [callback])
```
The message object may include the following fields:
| Parameter | Type | Description |
| ------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `event` | String | The name of the event. |
| `properties` | Object | Additional properties about the event, such as `product_id`. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Page
The `page` method sends a `page` event.
**Example usage**:
```
htevents.page({
userId: "123",
category: "Docs",
name: "Getting started",
})
```
The first argument to the method is an object representing the page event to
be sent. The second argument is an optional callback that's invoked after the
event is sent.
```
htevents.page(message, [callback])
```
The message object may include the following fields:
| Parameter | Type | Description |
| ------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `category` | String | The page's category. For example "Docs" |
| `name` | String | The page's name. For example "Getting started" |
| `properties` | Object | Additional properties about the event, such as `url`. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Screen
The `screen` method sends a `screen` event.
**Example usage**:
```
htevents.screen({
userId: "123",
category: "Docs",
name: "Getting started",
})
```
The first argument to the method is an object representing the screen event to
be sent. The second argument is an optional callback that's invoked after the
event is sent.
```
htevents.screen(message, [callback])
```
The message object may include the following fields:
| Parameter | Type | Description |
| ------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `category` | String | The screen's category. For example "Docs" |
| `name` | String | The screen's name. For example "Getting started" |
| `properties` | Object | Additional properties about the screen. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Group
The `group` method sends a `group` event.
**Example usage**:
```
htevents.group({
userId: "123",
groupId: "456",
traits: {
company_location: "San Francisco",
},
})
```
The first argument to the method is an object representing the group event to
be sent. The second argument is an optional callback that's invoked after the
event is sent.
```
htevents.group(message, [callback])
```
The message object may include the following fields:
| Parameter | Type | Description |
| ------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | String | The user's persistent ID |
| `anonymousId` | String | The user's anonymous ID. This is automatically set if both `userId` and `anonymousId` are omitted. |
| `groupId` | String | The id for the group. |
| `traits` | Object | Additional traits about the group, such as `company_name`. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
---
## PHP SDK
**URL:** https://hightouch.com/docs/events/sdks/php
**Description:** Hightouch's PHP SDK for collecting events
**Section:** Hightouch Events
The PHP SDK makes it easy to track events from PHP applications.
## Installation
The PHP SDK can be installed via [Composer](https://packagist.org/packages/ht-sdks/events-sdk-php):
```sh
composer require ht-sdks/events-sdk-php
```
To initialize the PHP SDK in your application:
```php
use Hightouch\Hightouch;
Hightouch::init('WRITE_KEY', [
'host' => 'https://us-east-1.hightouch-events.com',
]);
```
## API
### Identify
The `identify` method sends an `identify` event.
Unlike the [Browser SDK](./browser), it does **not** persist the user ID and
traits locally, so user IDs must be explicitly added to other events. This is
because server side events are usually servicing many different users at once.
**Example usage**:
```php
Hightouch::identify([
'userId' => '123',
'traits' => [
'location' => 'San Francisco',
],
]);
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | string | The user's persistent ID |
| `anonymousId` | string | The user's anonymous ID |
| `traits` | array | Additional traits about the user, such as `email` and `name`. |
| `context` | array | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | int | The UNIX timestamp (seconds) of the message. When backfilling, this can be a time in the past. By default it uses `time()`. |
### Track
The `track` method sends a `track` event.
**Example usage**:
```php
Hightouch::track([
'userId' => '123',
'event' => 'Order Completed',
'properties' => [
'total' => 29.99,
],
]);
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | string | The user's persistent ID |
| `anonymousId` | string | The user's anonymous ID |
| `event` | string | The name of the event. |
| `properties` | array | Additional properties about the event, such as `product_id`. |
| `context` | array | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | int | The UNIX timestamp (seconds) of the message. When backfilling, this can be a time in the past. By default it uses `time()`. |
### Page
The `page` method sends a `page` event.
**Example usage**:
```php
Hightouch::page([
'anonymousId' => 'anonymous-id',
'name' => 'events-sdk-php',
'category' => 'docs',
'properties' => [
'path' => '/docs/libraries/php/',
],
]);
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | string | The user's persistent ID |
| `anonymousId` | string | The user's anonymous ID |
| `name` | string | The page's name. For example "Getting started" |
| `properties` | array | Additional properties about the event, such as `url`. |
| `context` | array | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | int | The UNIX timestamp (seconds) of the message. When backfilling, this can be a time in the past. By default it uses `time()`. |
### Screen
The `screen` method sends a `screen` event.
**Example usage**:
```php
Hightouch::screen([
'userId' => '123',
'name' => 'Getting started',
'properties' => [
'path' => '/getting-started',
],
]);
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | string | The user's persistent ID |
| `anonymousId` | string | The user's anonymous ID |
| `name` | string | The page's name. For example "Getting started" |
| `properties` | array | Additional properties about the event, such as `url`. |
| `context` | array | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | int | The UNIX timestamp (seconds) of the message. When backfilling, this can be a time in the past. By default it uses `time()`. |
### Group
The `group` method sends a `group` event.
**Example usage**:
```php
Hightouch::group([
'userId' => '123',
'groupId' => 'G-1',
'traits' => [
'company_location' => 'San Francisco',
],
]);
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `userId` | string | The user's persistent ID |
| `anonymousId` | string | The user's anonymous ID |
| `groupId` | string | The id for the group. |
| `traits` | array | Additional traits about the group, such as `company_name`. |
| `context` | array | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | int | The UNIX timestamp (seconds) of the message. When backfilling, this can be a time in the past. By default it uses `time()`. |
### Flush
The PHP SDK buffers events locally before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and improves performance.
To force the local buffer to be sent to Hightouch immediately call the `flush`
method. `flush` should be called when shutting down your app to make sure no events are lost.
**Example usage**:
```php
Hightouch::flush();
```
---
## Python SDK
**URL:** https://hightouch.com/docs/events/sdks/python
**Description:** Hightouch's Python SDK for collecting events
**Section:** Hightouch Events
The Python SDK makes it easy to track events from Python applications.
## Installation
The Python SDK can be installed via
[pypi.org](https://pypi.org/project/events-sdk-python/).
```
pip install events-sdk-python
```
To initialize the Python SDK in your application, import `htevents` and set the `write_key`:
```
htevents.write_key = 'WRITE_KEY'
```
## API
### Identify
The `identify` method sends an `identify` event.
Unlike the [Browser SDK](./browser), it does **not** persist the user ID and
traits locally, so user IDs must be explicitly added to other events. This is
because server side events are usually servicing many different users at once.
**Example usage**:
```
htevents.identify('user123', {
'email': 'test@example.com',
'name': 'John Doe',
})
```
**Method parameters**:
| Parameter | Type | Description |
|----------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `user_id` | str | The user's persistent ID |
| `anonymous_id` | str | The user's anonymous ID |
| `traits` | dict | Additional traits about the user, such as `email` and `name`. |
| `context` | dict | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | datetime | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Track
The `track` method sends a `track` event.
**Example usage**:
```
htevents.track('user123', 'Order completed', {
'product_id': '123',
})
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `user_id` | str | The user's persistent ID |
| `anonymous_id` | str | The user's anonymous ID |
| `event` | str | The name of the event. |
| `properties` | dict | Additional properties about the event, such as `product_id`. |
| `context` | dict | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | datetime | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Page
The `page` method sends a `page` event.
**Example usage**:
```
htevents.page('user123', 'Docs', 'Python', {
'url': 'http://example.com/docs/python'
})
```
**Method parameters**:
| Parameter | Type | Description |
|----------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `user_id` | str | The user's persistent ID |
| `anonymous_id` | str | The user's anonymous ID |
| `category` | str | The screen's category. For example "Docs" |
| `name` | str | The page's name. For example "Getting started" |
| `properties` | dict | Additional properties about the event, such as `url`. |
| `context` | dict | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | datetime | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Screen
The `screen` method sends a `screen` event.
**Example usage**:
```
htevents.screen('user123', 'Docs', 'Python', {
'url': 'http://example.com/docs/python'
})
```
**Method parameters**:
| Parameter | Type | Description |
|----------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `user_id` | str | The user's persistent ID |
| `anonymous_id` | str | The user's anonymous ID |
| `category` | str | The screen's category. For example "Docs" |
| `name` | str | The screen's name. For example "Getting started" |
| `properties` | dict | Additional properties about the event, such as `url`. |
| `context` | dict | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | datetime | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Group
The `group` method sends a `group` event.
**Example usage**:
```
htevents.group('user123', 'group123', {
'company_name': 'Initech',
})
```
**Method parameters**:
| Parameter | Type | Description |
|----------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `user_id` | str | The user's persistent ID |
| `anonymous_id` | str | The user's anonymous ID |
| `group_id` | str | The id for the group. |
| `traits` | dict | Additional traits about the group, such as `company_name`. |
| `context` | dict | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | datetime | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Flush
The Python SDK buffers events locally before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and makes the tracking
non-blocking.
To force the local buffer to be sent to Hightouch immediately call the `flush`
method. `flush` should be called when shutting down your Python app to
make sure no events are lost.
**Example usage**:
```
htevents.flush()
```
---
## React Native SDK
**URL:** https://hightouch.com/docs/events/sdks/react-native
**Description:** Hightouch's React Native SDK for collecting events
**Section:** Hightouch Events
The React Native SDK makes it easy to track events in React Native
applications. The SDK supports Both iOS and Android applications.
## Installation
The React Native SDK is distributed via `npm`.
Install the required dependencies into your React Native project via `npm`:
```
npm install --save \
@ht-sdks/events-sdk-react-native \
@ht-sdks/sovran-react-native \
react-native-get-random-values \
@react-native-async-storage/async-storage
```
### iOS
On iOS, you will also need to install the native modules by running:
```
npx pod-install
```
### Android
The SDK requires additional permissions on Android in order to populate the
device context for events.
Add these permissions to your app by updating your `AndroidManifest.xml`:
```
```
## SDK Initialization
Once the dependencies are installed, you can import the tracking client into
your application.
```
const htClient = createClient({
writeKey: 'API_KEY'
});
```
The client can either be passed around explicitly throughout your app, or
exposed through a [React context
hook](https://react.dev/learn/passing-data-deeply-with-context). To pass the
context through a Context, use `AnalyticsProvider` and `useAnalytics`.
```
// The Child component can retrieve the htClient by calling `useAnalytics()`.
const App = () => (
);
```
The client may be configured with the following options:
| Parameter | Type | Description |
| ------------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `writeKey` | String | The write key used to identify the source in Hightouch. |
| `collectDeviceId` | Boolean | Whether to autocollect a device ID via the Android DRM API. The ID is stored in `context.device.id` on all events. This is only supported on Android, and is disabled by default. |
| `debug` | Boolean | Whether to generate debug logs. Defaults to `true`. |
| `logger` | Logger | Custom logger instance to redirect internal logging from the library. |
| `flushAt` | Number | Maximum number of events to accumulate before sending to the Hightouch API. Defaults to `20`. |
| `flushInterval` | Number | Maximum number of seconds to hold events before sending to the Hightouch API. Defaults to `30`. |
| `flushPolicies` | Array | Optional advanced customization for how events are buffered before sending to the Hightouch API. |
| `maxBatchSize` | Number | The maximum number of events to send in one HTTP request to the Hightouch API. You may want to adjust this if your events are extremely large. Defaults to `1000`. |
| `trackAppLifecycleEvents` | Boolean | Whether to automatically track lifecycle events like `Application Installed`. Defaults to `false`. |
| `trackDeepLinks` | Boolean | Whether to automatically track deep links. Only supported for Android. Defaults to `false`. |
| `proxy` | String | Used to override the URL used for sending events to Hightouch. This should contain the full path, e.g. `https://us-east-1.hightouch-events.com/v1/batch` |
## Manual tracking API
### Identify
The `identify` method is used to send an `identify` event.
If `identify` is called multiple times for the same user, the `traits` are
merged together.
**Method signature**:
```
htClient.identify(userId, [traits])
```
**Method parameters**:
| Parameter | Type | Description |
| --------- | ------ | ------------------------------------------------------------- |
| `userId` | String | The user's persistent ID. |
| `traits` | Object | Additional traits about the user, such as `email` and `name`. |
### Track
The `track` method is used to send a `track` event.
**Method signature**:
```
htClient.track(event, [properties])
```
**Method parameters**:
| Parameter | Type | Description |
| ------------ | ------ | ------------------------------------------------------------ |
| `event` | String | The event name (for example "Checked Out"). |
| `properties` | Object | Additional properties about the event, such as `product_id`. |
### Screen
The `screen` method is used to send a `screen` event.
**Method signature**:
```
htClient.screen(name, [properties])
```
**Method parameters**:
| Parameter | Type | Description |
| ------------ | ------ | -------------------------------------------------- |
| `name` | String | The screen's name. For example, "Getting started". |
| `properties` | Object | Additional properties about the event. |
### Group
The `group` method sends a `group` event.
**Method signature**:
```
htClient.group(groupId, [traits])
```
**Method parameters**:
| Parameter | Type | Description |
| --------- | ------ | ---------------------------------------------------------- |
| `groupId` | String | The id for the group. |
| `traits` | Object | Additional traits about the group, such as `company_name`. |
### Reset
The `reset` method resets the `identify` calls for the local session.
Specifically, it resets the `anonymousId`, `userId`, and `traits`.
The `reset` method should be called when users log out. This way, if the user
logs back in with another account, the userIds and traits from the different
sessions remain separate.
**Method signature**:
```
htClient.reset()
```
## Out of the box events
### Lifecycle events
When `trackAppLifecycleEvents` is enabled, Hightouch automatically tracks the following events:
- `Application Installed` -- Emitted when the app is first opened after a new install.
- `Application Updated` -- Emitted when the app is first opened after upgrading from a previous version.
- `Application Opened` -- Emitted whenever the app is opened (including when it's resuming from the background).
- `Application Backgrounded` -- Emitted whenever the app is backgrounded.
## Advertising identifiers
To collect IDFAs and AAIDs for advertising identification, you may use the
`@ht-sdks/events-sdk-react-native-plugin-advertising-id` and
`@ht-sdks/events-sdk-react-native-plugin-idfa` packages.
These plugins load native modules that pull the local advertising ID.
### Android AAID
Once setup, the AAID will be stored in `context.device.advertisingId`
1. Add the Android AAID plugin to your project.
```
npm install @ht-sdks/events-sdk-react-native-plugin-advertising-id
```
2. Add the plugin to your code by adding the following code after initializing your client.
```
htClient.add({plugin: new AdvertisingIdPlugin()});
```
3. Update your Android application manifest to include your [AdMob app ID](https://developers.google.com/admob/android/quick-start).
```
```
### iOS IDFA
Once setup, the IDFA will be stored in `context.device.advertisingId`
1. Add the iOS IDFA plugin to your project.
```
npm install @ht-sdks/events-sdk-react-native-plugin-idfa
```
2. Add the plugin to your code by adding the following code after initializing your client.
```
htClient.add({plugin: new IdfaPlugin()});
```
3. Update your `Info.plist` to set
[NSUserTrackingUsageDescription](https://developer.apple.com/documentation/bundleresources/information_property_list/nsusertrackingusagedescription).
This description will be displayed when prompting users for permission to
track their IDFA.
---
## Ruby SDK
**URL:** https://hightouch.com/docs/events/sdks/ruby
**Description:** Hightouch's Ruby SDK for collecting events
**Section:** Hightouch Events
The Ruby SDK makes it easy to track events from Ruby applications.
## Installation
The Ruby SDK can be installed via
[RubyGems.org](https://rubygems.org/gems/events-sdk-ruby/). You may add the SDK
to your Gemfile if you're using Bundler, or install it directly:
```
gem install events-sdk-ruby
```
To initialize the Ruby SDK in your application, create a new Analytics instance:
```
require 'hightouch'
analytics = Hightouch::Analytics.new(
write_key: WRITE_KEY,
host: 'us-east-1.hightouch-events.com'
)
```
## API
### Identify
The `identify` method sends an `identify` event.
Unlike the [Browser SDK](./browser), it does **not** persist the user ID and
traits locally, so user IDs must be explicitly added to other events. This is
because server side events are usually servicing many different users at once.
**Example usage**:
```
analytics.identify(
user_id: "123",
traits: {
location: "San Francisco",
}
)
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `user_id` | String | The user's persistent ID |
| `anonymous_id` | String | The user's anonymous ID |
| `traits` | Hash | Additional traits about the user, such as `email` and `name`. |
| `context` | Hash | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Track
The `track` method sends a `track` event.
**Example usage**:
```
analytics.track(
user_id: "123",
event: "Order completed",
properties: {
total: 29.99,
}
)
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `user_id` | String | The user's persistent ID |
| `anonymous_id` | String | The user's anonymous ID |
| `event` | String | The name of the event. |
| `properties` | Hash | Additional properties about the event, such as `product_id`. |
| `context` | Hash | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Page
The `page` method sends a `page` event.
**Example usage**:
```
analytics.page(
user_id: "123",
category: "Docs",
name: "Getting started",
)
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `user_id` | String | The user's persistent ID |
| `anonymous_id` | String | The user's anonymous ID |
| `name` | String | The page's name. For example "Getting started" |
| `properties` | Hash | Additional properties about the event, such as `url`. |
| `context` | Hash | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Screen
The `screen` method sends a `screen` event.
**Example usage**:
```
analytics.screen(
user_id: "123",
category: "Docs",
name: "Getting started",
)
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `user_id` | String | The user's persistent ID |
| `anonymous_id` | String | The user's anonymous ID |
| `name` | String | The screen's name. For example "Getting started" |
| `category` | String | The screen's category. For example "Docs" |
| `properties` | Hash | Additional properties about the event, such as `url`. |
| `context` | Hash | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Time | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Group
The `group` method sends a `group` event.
**Example usage**:
```
analytics.group(
user_id: "123",
group_id: "456",
traits: {
company_location: "San Francisco",
},
)
```
**Method parameters**:
| Parameter | Type | Description |
| -------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| `user_id` | String | The user's persistent ID |
| `anonymous_id` | String | The user's anonymous ID |
| `groupId` | String | The id for the group. |
| `traits` | Object | Additional traits about the group, such as `company_name`. |
| `context` | Object | Overrides to values in the event `context`. By default, `context` contains information autocollected by the SDK. |
| `timestamp` | Date | The date of the message. When backfilling data, this can be set to a date in the past. By default, this is autoset to the current date. |
### Flush
The Ruby SDK buffers events locally before sending them to Hightouch's servers.
This minimizes the number of requests made by the SDK and makes the tracking
non-blocking.
To force the local buffer to be sent to Hightouch immediately call the `flush`
method. `flush` should be called when shutting down your Ruby app to
make sure no events are lost.
**Example usage**:
```
analytics.flush()
```
---
## Google PubSub
**URL:** https://hightouch.com/docs/events/sources/google-pubsub
**Description:** Hightouch and Google PubSub integration
**Section:** Hightouch Events
## Overview
Hightouch supports streaming events from Google Pub/Sub in realtime.
Simply configure your Google Pub/Sub source in the Hightouch UI, and Hightouch will automatically extract, transform, and load your data into Hightouch Events. Your data can then be synced to your warehouse or streamed to our catalog of realtime destinations.
## Setup
To get started, [**create an event source**](https://app.hightouch.com/events/sources/new), select **Google Pub/Sub** and follow the steps below.
### Configure your source
Configure the following fields in Hightouch:
- **Credentials**: GCP credentials for connecting to Pub/Sub.
- **Subscription**: The full name of the Pub/Sub subscription to consume events from.
- **Project ID**: The Google Cloud Project ID.
- **Regional API Endpoint**: [List of endpoints](https://cloud.google.com/Pub/Sub/docs/reference/service_apis_overview#Pub/Sub_endpoints)
### Grant permissions
Hightouch believes in the principle of least privilege. We ask for no more permissions than necessary. For the Google Pub/Sub event source, Hightouch requires `roles/Pub/Sub.subscriber` and `roles/Pub/Sub.viewer`. Ensure you run the command in your Google Cloud Console to grant Hightouch's service account the correct permissions.
### Message format
Hightouch Events supports PubSub messages encoded using Avro and Protobuf as long as a [PubSub Schema](https://cloud.google.com/pubsub/docs/schemas) is configured and associated with the topic. Otherwise, messages need to be encoded as JSON.
## Event Structure
All events are ingested into Hightouch as `track` events. The Pub/Sub message `id`, `data`, and `attributes` are exposed as custom `properties` on the `track` event. Metadata such as `subscription` are also exposed in the `context.Pub/Sub` object.
For example, the following Pub/Sub message:
```json
{
"id": "123",
"data": {
"user_id": "user_123",
"amount": 123.45
},
"attributes": {
"a1": "abc"
}
}
```
Will be ingested into Hightouch as:
```json
{
"type": "track",
"event": "Google Pub/Sub Event",
"properties": {
"id": "123",
"data": {
"user_id": "user_123",
"amount": 123.45
},
"attributes": {
"a1": "abc"
}
},
"context": {
"Pub/Sub": {
"subscription": "projects/my-project/subscriptions/my-subscription"
}
},
"timestamp": "2024-07-29T14:52:37.648Z"
}
```
### Field mappings
You can configure how Hightouch transforms messages from your Pub/Sub topic to Hightouch [`track` events](/events/event-spec#track-events). Mappings can be configured for the following fields:
- `event`
- `userId`
- `anonymousId`
- `messageId`
- `timestamp`
You can also transform your events using Functions in order to standardize the schema or enrich its contents.
### Schema Enforcement
[Event Contracts](/events/contracts/management) can be attached to Google Pub/Sub sources just like any other Event Source.
## Tips and troubleshooting
---
## Kafka (Apache, Confluent, & Redpanda)
**URL:** https://hightouch.com/docs/events/sources/kafka
**Description:** Hightouch integration with Apache Kafka, Confluent Cloud, and Redpanda
**Section:** Hightouch Events
## Overview
Hightouch supports streaming events from **Apache Kafka**, **Confluent Cloud**, and **Redpanda** clusters in real time.
All three are **Kafka-compatible platforms**:
- **Apache Kafka**: the open-source distributed event streaming platform.
- **Confluent Cloud**: a fully managed Kafka-as-a-service offering with enterprise features.
- **Redpanda**: a high-performance, Kafka API–compatible platform designed as a drop-in replacement.
Since they all speak the **Kafka protocol**, setup and event ingestion in Hightouch are nearly identical.
Once connected, Hightouch automatically extracts, transforms, and loads your event data into Hightouch Events. Your data can then be synced to your warehouse or streamed to our catalog of real-time destinations.
## Setup
To get started, [**create an event source**](https://app.hightouch.com/events/sources/new) and select either:
- **Kafka** for Apache Kafka
- **Confluent** for Confluent Cloud
- **Redpanda** (for Redpanda clusters)
Both sources use the same configuration flow.
### Choose connection type
### Configure your source
The required fields are the same across Apache Kafka, Confluent, and Redpanda:
- **Broker URL**:
- Kafka: hostname:port for your brokers.
- Confluent: provided in your Confluent cluster settings.
- Redpanda: hostname:port for your Redpanda brokers.
- **Topic**: The topic to consume events from. Messages must be JSON-formatted.
- **Authentication**:
- Kafka: Multiple auth methods supported.
- Confluent: Use SASL/SSL credentials from Confluent Cloud.
- Redpanda: Uses Kafka-compatible auth methods configured in your cluster.
### Message format
Hightouch Events requires Kafka messages be encoded as JSON.
## Event Structure
All events from Kafka, Confluent, and Redpanda are ingested into Hightouch as `track` events.
- Message `key`, `value`, and `headers` → `properties`
- Metadata such as `topic`, `partition`, and `offset` → `context.kafka`
For example, the following Kafka message:
```json
{
"partition": 1,
"offset": 123,
"timestamp": 1721659050648,
"key": "k1",
"value": {
"user_id": "user_123",
"amount": 123.45
},
"headers": [
{
"key": "h1",
"value": "abc"
}
]
}
```
Will be ingested into Hightouch as:
```json
{
"type": "track",
"event": "Kafka Event",
"properties": {
"key": "k1",
"value": {
"user_id": "user_123",
"amount": 123.45
},
"headers": {
"h1": "abc"
}
},
"context": {
"kafka": {
"topic": "my-topic",
"partition": 1,
"offset": "123"
}
},
"timestamp": "2024-07-22T14:52:37.648Z"
}
```
### Field mappings
You can configure how Hightouch transforms messages from your Kafka topic to Hightouch [`track` events](/events/event-spec#track-events). Mappings can be configured for the following fields:
- `event`
- `userId`
- `anonymousId`
- `messageId`
- `timestamp`
You can also use Functions to standardize schemas or enrich event data.
### Schema Enforcement
[Event Contracts](/events/contracts/management) can be attached to Kafka, Confluent, and Redpanda sources just like any other Event Source.
## Tips and troubleshooting
---
## Event sources overview
**URL:** https://hightouch.com/docs/events/sources/overview
**Description:** Monitor, validate, and govern events as they enter Hightouch.
**Section:** Hightouch Events
| Audience | How you’ll use this article |
| --------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| **Marketing teams** | Understand where events come from, how they’re monitored, and how event health impacts real-time activation and personalization. |
| **Data teams** | Learn how incoming events are validated, inspected, governed, and made reliable for downstream use. |
| **Engineering teams** | Monitor event volume, debug payloads, validate schemas, and configure alerts to catch issues early. |
## What you’ll learn
By the end of this article, you’ll be able to:
1. [Add or configure an event source](#add-or-configure-an-event-source)
2. [Monitor event volume and health](#monitoring-and-overview)
3. [Explore events in the event catalog](#event-catalog)
4. [Set up alerting for event behavior](#alerting)
5. [Inspect events using the debugger](#debugger)
6. [Configure SDK for event source](#setup)
---
## Overview
An **event source** represents a system that sends events into Hightouch, such as a web app, mobile app, backend service, or data pipeline.
Event sources are the starting point for all event-based workflows in Hightouch, including event storage, event streaming, and downstream activation.
---
#### Supported SDKs
Hightouch provides several SDKs (software development kits) that you can deploy to your app or site in order to start collecting events. There are separate SDKs for:
- [Browser (Javascript)](/events/sdks/browser)
- [Node.js](/events/sdks/nodejs)
- [Ruby](/events/sdks/ruby)
- [Go](/events/sdks/go)
- [PHP](/events/sdks/php)
- [Java](/events/sdks/java)
- [Python](/events/sdks/python)
- [iOS](/events/sdks/ios)
- [Android](/events/sdks/android)
Additionally, you can collect events directly from [Kafka](/events/sources/kafka), [PubSub](/events/sources/google-pubsub), and [Webhooks](/events/sources/webhook) and inspect events in a web-based debugger to ensure you're capturing events from these different sources (including from SDKs) correctly.
Please note that there is a 96 kilobytes size limit for each event that's collected.
#### Recommended regions
* `aws-us-east-1` (US East)
* `aws-ap-south-1` (Asia Pacific – Mumbai)
* `aws-eu-west-1` (Europe – Ireland)
For setup in other regions, please and we’ll help you get started.
---
## Add or configure an event source
To add or manage an event source:
1. Go to [https://app.hightouch.com/](https://app.hightouch.com/)
2. Navigate to **`Event Collection → Event sources`**
3. Click **Add source** to create a new event source, or
4. Select an existing source from the list to begin monitoring, debugging, or configuration
---
## Monitoring and overview
The **Overview** tab provides a high-level view of event activity for a single source.
From this view, you can:
* Monitor **total event volume** over time
* Identify gaps, drops, or spikes in traffic
* See whether any **alerts** are currently active
Use the time range controls (`24h`, `3d`, `7d`) to analyze recent changes or longer-term trends.
This view is typically used to answer questions like:
* *Is this source actively sending events?*
* *Did volume change after a release or deployment?*
* *Are any alerts firing that require investigation?*
---
## Event catalog
The **Event catalog** lists all event types received from the source, along with their recent volume and configuration status.
For each event, you can:
* View the **event name and type** (for example, `Track`, `Identify`, or `Group`)
* Monitor **event volume** over time
* Configure **data contracts** to validate schema and required fields
* Set up **alerts** for volume-based changes
New event types appear in the catalog after they are received for the first time. This may take a few minutes after initial instrumentation.
Use the event catalog to:
* Understand which events are actively being sent
* Identify high- or low-volume events
* Decide where schema validation or alerting is needed
---
## Alerting
Alerting helps you detect unexpected changes in event behavior.
Alerts are configured at the **event level** from the **Event catalog**.
**Navigate to alerting:**
1. Go to **`Event Collection → Event sources`**
2. Select an event source
3. Open the **`Event catalog`** tab
4. Find the event you want to monitor
5. Click **Configure** in the **Alerts** column
This opens the event’s **Alerting** panel, where you can review alert history, define triggers, and configure recipients.
From this panel, you can configure alerts to notify your team when:
* Event volume exceeds a high threshold
* Event volume drops below a minimum threshold
Alerts can be sent to external notification channels, such as email or messaging tools, so teams can respond quickly to issues.
Alerting is commonly used to:
* Detect broken or stalled event pipelines
* Catch unintended increases in event volume
* Monitor critical events that power downstream workflows
---
## Debugger
The **Debugger** lets you inspect individual events as they arrive from the source.
From the debugger, you can:
* View recently received events and timestamps
* Search events by name, type, or payload content
* Inspect the **full JSON payload** for a selected event
* Pause or clear the event stream during debugging
The debugger is especially useful during:
* Initial SDK or API setup
* Instrumentation changes
* Troubleshooting missing or malformed events
If no events appear, confirm that the SDK or integration is installed correctly and that the source is actively sending data.
---
## Setup
Use the **Setup** tab to install and configure the SDK or integration for this event source. This view provides source-specific installation instructions, including required keys, endpoints, and example code.
---
## Next steps
* [**Connect event sources**](/events/get-started/#2-connect-event-sources-to-start-collecting-events) to start collecting events
---
## Error handling
**URL:** https://hightouch.com/docs/events/warehouses/error-handling
**Description:** Details on the Hightouch handles events that fail to sync
**Section:** Hightouch Events
Sometimes, events are unable to be uploaded to the warehouse. This can happen
for a variety of reasons; for example, the warehouse credentials may be
invalid, or your event schema may have evolved to have incompatible types.
Regardless, Hightouch ensures that data is never dropped.
## Viewing sync errors
Sync errors can be found under the sync run page.
The run summary page provides a quick summary of the types of errors that are
keeping your data from being synced.
For each error, you can see how many events it's affecting, as well as
individual examples of bad events.
## Retrying data
Invalid data is automatically retried for at least 48 hours. If the data issue
has still not been resolved, the invalid events are stored so that they can be
replayed at a later date.
Replaying events is done via the "Archived Failures" screen.
### Fixing column types
To change an existing column type, you can create a temporary version of the
column with the new format.
For example, the following query may used on Snowflake to change the `price`
column on the `product_ordered` table from an `INT` to a `FLOAT`:
```
BEGIN;
ALTER TABLE product_ordered ADD COLUMN price_float FLOAT;
UPDATE product_ordered SET price_float=price;
ALTER TABLE product_ordered DROP COLUMN price;
ALTER TABLE product_ordered RENAME COLUMN price_float TO price;
COMMIT;
```
---
## Warehouse schema
**URL:** https://hightouch.com/docs/events/warehouses/schema
**Description:** Details on the warehouse schema created by Hightouch Events
**Section:** Hightouch Events
Hightouch automatically creates and manages tables for all event types. New
tracked fields are automatically imported into the warehouse without additional
data engineering work.
All the tables for an event source are written into the same schema in the
warehouse. By default, this schema name is autogenerated based on the event
source. However, the destination schema can be configured in the destination
settings.
## Tables
### Identifies
All `identify` events are written into the warehouse in the `identifies` table.
| Column | Description |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `id` | The `messageId` from the event |
| `anonymous_id` | The anonymous id from the event |
| `user_id` | The user id from the event |
| `` | The user traits provided in the `identify` request. Each key in the `traits` object is converted into a separate column. |
| `context_` | Additional context about the request. Often autocollected by the tracking SDKs. |
| `timestamp` | The clock-skew adjusted timestamp this event occurred. In most cases, this column should be used for timestamp analysis. |
| `received_at` | When Hightouch received the event. Used for calculating `timestamp`. |
| `sent_at` | When the tracking SDK sent the event. Used for calculating `timestamp`. |
| `original_timestamp` | The client-side timestamp for when the tracking SDK sent the event. Used for calculating `timestamp`. |
### Tracks
All `track` events are written into the warehouse in the `tracks` table.
| Column | Description |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `id` | The `messageId` from the event |
| `anonymous_id` | The anonymous id from the event |
| `user_id` | The user id from the event |
| `context_` | Additional context about the request. Often autocollected by the tracking SDKs. |
| `event_text` | The event name. |
| `event` | The snake-cased version of the event name. This matches the table name autocreated by Hightouch. |
| `timestamp` | The clock-skew adjusted timestamp this event occurred. In most cases, this column should be used for timestamp analysis. |
| `received_at` | When Hightouch received the event. Used for calculating `timestamp`. |
| `sent_at` | When the tracking SDK sent the event. Used for calculating `timestamp`. |
| `original_timestamp` | The client-side timestamp for when the tracking SDK sent the event. Used for calculating `timestamp`. |
#### Per-Event tables
By default, Hightouch creates a separate table for each Track
event type. When these per-event tables are enabled, the event properties are
written as separate columns within the table.
When per-event tables are turned off, the events `properties` column is added
to the `tracks` table so that custom properties are still accessible.
### Pages
All `page` events are written into the warehouse in the `pages` table.
| Column | Description |
| -------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `id` | The `messageId` from the event |
| `anonymous_id` | The anonymous id from the event |
| `user_id` | The user id from the event |
| `context_` | Additional context about the request. Often autocollected by the tracking SDKs. |
| `` | The page properties provided in the `page` request. Each key in the `properties` object is converted into a separate column. |
| `timestamp` | The clock-skew adjusted timestamp this event occurred. In most cases, this column should be used for timestamp analysis. |
| `received_at` | When Hightouch received the event. Used for calculating `timestamp`. |
| `sent_at` | When the tracking SDK sent the event. Used for calculating `timestamp`. |
| `original_timestamp` | The client-side timestamp for when the tracking SDK sent the event. Used for calculating `timestamp`. |
### Screens
All `screen` events are written into the warehouse in the `screens` table.
| Column | Description |
| -------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `id` | The `messageId` from the event |
| `anonymous_id` | The anonymous id from the event |
| `user_id` | The user id from the event |
| `context_` | Additional context about the request. Often autocollected by the tracking SDKs. |
| `` | The screen properties provided in the `screen` request. Each key in the `properties` object is converted into a separate column. |
| `timestamp` | The clock-skew adjusted timestamp this event occurred. In most cases, this column should be used for timestamp analysis. |
| `received_at` | When Hightouch received the event. Used for calculating `timestamp`. |
| `sent_at` | When the tracking SDK sent the event. Used for calculating `timestamp`. |
| `original_timestamp` | The client-side timestamp for when the tracking SDK sent the event. Used for calculating `timestamp`. |
### Groups
All `group` events are written into the warehouse in the `groups` table.
| Column | Description |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `id` | The `messageId` from the event |
| `anonymous_id` | The anonymous id from the event |
| `user_id` | The user id from the event |
| `group_id` | The group id from the event |
| `context_` | Additional context about the request. Often autocollected by the tracking SDKs. |
| `` | The group traits provided in the `group` request. Each key in the `traits` object is converted into a separate column. |
| `timestamp` | The clock-skew adjusted timestamp this event occurred. In most cases, this column should be used for timestamp analysis. |
| `received_at` | When Hightouch received the event. Used for calculating `timestamp`. |
| `sent_at` | When the tracking SDK sent the event. Used for calculating `timestamp`. |
| `original_timestamp` | The client-side timestamp for when the tracking SDK sent the event. Used for calculating `timestamp`. |
## FAQ
### How are the warehouse column types determined?
Hightouch determines what types to use for new columns based on the event
values. Event values are categorized into one of the following types:
- String
- Integer
- Float
- Timestamp
- Boolean
Timestamps must be sent as ISO-8601 string.
### What happens if my event types change?
When possible, Hightouch attempts to convert the values in the event payload
into the existing types in the warehouse. For example, if an event contains a
Integer value for a field, but the current type in the warehouse is a String,
the Integer is automatically converted.
If a conversion can't be safely done without losing data, Hightouch doesn't
upload the event, and [retries it for 48 hours](./error-handling). This way, if
you fix the type in the warehouse, Hightouch will automatically sync the event.
### How are complex objects handled?
When tracking events with Objects as custom properties, Hightouch flattens
nested columns into a flat schema.
For example, the following properties:
```json
{
"company": {
"name": "Hightouch",
"location": "San Francisco"
}
}
```
Would get imported into the warehouse as:
| company_name | company_location |
| ------------ | ---------------- |
| Hightouch | San Francisco |
---
## Write events to your warehouse
**URL:** https://hightouch.com/docs/events/warehouses/warehouse
**Description:** Configure an event storage destination so Hightouch can persist event data to your warehouse.
**Section:** Hightouch Events
| Audience | How you’ll use this article |
|--------|-----------------------------|
| **Marketing teams** | Understand where event data lives and how it becomes available for activation and analytics. |
| **Data teams** | Configure warehouse storage, schema behavior, and table structure for event data. |
| **Engineering teams** | Set up secure warehouse connections and validate end-to-end event persistence. |
## Overview
To write event data collected by Hightouch Events to your warehouse, you must configure an **event storage destination**.
An event storage destination defines:
- **Where** events are written (Snowflake, BigQuery, Redshift, etc.)
- **How** event tables and schemas are created
- **How** event properties and context fields are stored
Once configured, Hightouch automatically creates and manages event tables in your warehouse as new events and fields are received.
Event storage is **optional but strongly recommended** if you want to use events for analytics, modeling, or warehouse-based activation.
You can also use Hightouch Events in a [**streaming‑only** mode](/event-streaming) to forward events directly to downstream tools without writing them to the warehouse.
---
## Supported event storage destinations
Hightouch supports the following event storage destinations:
- Snowflake
- BigQuery
- Redshift
- Databricks
- PostgreSQL
- Amazon S3
Each destination supports the same core event schema, with destination-specific configuration options.
---
## Step-by-step: Set up an event storage destination
### 1. Add a destination
1. In the left navigation, go to **`Event Collection → Event destinations`**
2. Click **`Add destination`**
---
### 2. Select a destination type
On the **Select a destination** step:
1. Under **Event storage**, choose your warehouse (for example, **Snowflake**)
2. Click the destination tile to continue
---
### 3. Connect the destination
On the **Connect destination** step, enter the required warehouse credentials.
For example, for Snowflake, you’ll configure:
- **Account identifier**
- **Warehouse**
- **Database**
- **Authentication method** (RSA key recommended)
You’ll also upload credentials and validate that Hightouch can connect successfully.
For more detailed destination **credential/permission setup** see:
The destination docs linked below cover both credential setup and **model-based sync configuration**.
For **event destinations**, only the credential/permission setup applies. Skip sections like **Sync configuration**, **Add sync**, and any model mapping or sync mode instructions.
- [Snowflake](/destinations/snowflake#connect-to-snowflake)
- [BigQuery](/destinations/google-bigquery#connect-to-google-adh-via-bigquery)
- [Databricks](/destinations/databricks#connect-to-databricks)
- [PostgreSQ](/destinations/postgres#connect-to-postgresql)
- [Amazon S3](/destinations/s3#connect-to-amazon-s3)
Click **`Continue`** once the connection details are complete.
---
### 4. Finalize the destination
On the **Finalize** step:
1. Review the destination details
2. Click **`Finish`**
Once finalized, the destination becomes available for event storage.
---
## What happens after setup
After your event storage destination is created and used in an event sync:
- Hightouch begins writing incoming events to your warehouse
- Schemas and tables are created automatically (see: [**Warehouse schema**](/events/warehouses/schema))
- New event fields are added as columns as they appear
You can manage this behavior using the destination’s **Configuration** and **Schema settings** tabs.
---
## Configuration after initial setup
Navigate to:
**`Event Collection → Event destinations → [Select destination]`**
### Configuration tab
The **Configuration** tab contains the core warehouse connection settings, including:
- Account and authentication details
- Warehouse, database, and role configuration
- Credentials and key management
Use this tab if you need to:
- Rotate credentials
- Update authentication methods
- Change the warehouse or database used for writes
---
### Schema settings tab
The **Schema settings** tab controls how Hightouch writes event data into your warehouse.
From here, you can configure:
- **Schema name**
Customize the schema Hightouch uses. By default, this is derived from the event source name.
- **Per-event tables**
When enabled, Hightouch creates a separate table for each `track` event type.
When disabled, all track events are written to a shared `tracks` table.
- **Convert properties and context fields to columns**
When enabled, event properties and context fields are written as individual columns.
When disabled, they are stored as JSON.
- **Write Page and Screen events to the Tracks table**
Useful if you want all user interactions in a single table.
These settings apply to all events written to this destination.
---
## Validating event storage
After setup, you should:
1. Send test events from an event source
2. Confirm tables and schemas appear in your warehouse
3. Verify event fields match your tracking spec
You can also inspect events in the debugger before they’re written to storage.
→ See [**Warehouse schema**](/events/warehouses/schema) for a full breakdown of tables and columns.
---
## Next steps
- Understand how events are written to your warehouse: [Warehouse schema](/events/warehouses/schema)
- Review retries, failures, and archived events: [Error handling](/events/warehouses/error-handling)
- Forward events to downstream tools: [Event streaming](/events/event-streaming)
---
## Trigger syncs with Airflow
**URL:** https://hightouch.com/docs/extensions/airflow
**Description:** Learn how to trigger syncs with Hightouch's Airflow Operator.
**Section:** Extensions
## Overview
Hightouch provides an [Airflow Operator](https://airflow.apache.org/docs/apache-airflow/stable/concepts.html#operators) for triggering syncs via Airflow. The [Git repository](https://github.com/hightouchio/airflow-provider-hightouch) contains the latest code as well as [example DAGs](https://github.com/hightouchio/airflow-provider-hightouch#examples) to help you get started.
## Setup
### Create an API key
The first step is to create a Hightouch API key in your Hightouch workspace settings.
### Add the Airflow connection
1. In Airflow, create a new connection by going to **Admin > Connections** and clicking the **+** button to add a new connection.
2. Enter `hightouch_default` as the **Conn ID**, select **HTTP** as the **Conn Type**, and set the host to `https://api.hightouch.com`. Optionally, enter a **Description**.
3. Enter the **API key** you created in the **Password** field.
### Set sync schedule type
### Install the Airflow package
While the details will vary for your particular Airflow installation, the simplest way to install our package is through pip.
```bash
pip install airflow-provider-hightouch
```
### Add the HightouchTriggerSyncOperator to your DAG
The last step is to add the operator to a DAG and enter the sync ID of the sync you want to trigger. When the task is run, Airflow sends a call to the Hightouch API to trigger a run which completes asynchronously.
Alternatively, you can choose to send a synchronous request and await the results of the sync before completing the task.
For a full description of the operator parameters, [view the source code in the GitHub repository](https://github.com/hightouchio/airflow-provider-hightouch/blob/main/airflow_provider_hightouch/operators/hightouch.py).
```python
from airflow_provider_hightouch.operators.hightouch \
import HightouchTriggerSyncOperator
with DAG(....) as dag:
...
# This task submits the request but doesn't wait for completion of the sync before completing.
my_async_task = HightouchTriggerSyncOperator(
task_id="run_my_sync", sync_id=1234
)
# This task waits for the sync status to complete or error before completing.
# Warnings can be treated as errors or successes depending on your use case.
my_sync_task = HightouchTriggerSyncOperator(
task_id="run_sync", sync_id=1234, synchronous=True, error_on_warning=True
)
```
---
## Connections Overview
**URL:** https://hightouch.com/docs/extensions/connections
**Description:** Learn how to use Connections in Hightouch to import data from third-party applications into your data warehouse.
**Section:** Extensions
| | |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Data engineers, analysts, and technical marketers |
| **Prerequisites** |
Access to a supported data warehouse (e.g., Snowflake, BigQuery, Databricks, or Postgres)
Admin or API access to the external application you plan to connect
|
_Use Connections to bring data from external tools--such as Salesforce, Klaviyo, or Braze--into your data warehouse for analysis and activation._
---
## Learning Objectives
After reading this article, you will be able to:
- Understand when and why to use Connections.
- Create and configure a new Connection in Hightouch.
- Select which tables and columns to import using the schema picker.
- Schedule or manually run Connection syncs.
- Manage and monitor existing Connections from your workspace.
---
## Overview
Connections let you bring data from third-party applications, such as Salesforce, Klaviyo, or Braze, into your data warehouse.
While [destinations](/destinations/overview) send data _out_ of Hightouch to external tools, **Connections** bring data _in_. This allows you to centralize and analyze information from multiple systems within your warehouse, supporting downstream workflows in Hightouch.
You can use Connections to:
- Import data from marketing, sales, and engagement platforms
- Enrich warehouse data with external attributes and activity
- Support analysis, segmentation, and activation across Customer Studio and other Hightouch features
Each Connection uses a secure API or OAuth authentication flow and provides schema controls so you can decide which data to include and how it’s stored.
---
## Prerequisites
Before creating a Connection, make sure you have:
- **Access permissions** for both Hightouch and your target application (for example, admin-level or API key access)
- **Warehouse permissions** to write tables or manage schemas
- **API credentials** (if required by your integration). Some tools, like Klaviyo or Iterable, require API keys while others authenticate through OAuth.
---
## Create a Connection
### Access Connections
To create or manage Connections, go to **Data Management → Connections** in your Hightouch workspace or visit [app.hightouch.com/connections](https://app.hightouch.com/connections).
From this page, you can view existing connections or click **Add connection** to start a new one.
---
### 1. Select a data source
Choose the external application you want to import data from.
---
### 2. Select a warehouse
Choose the warehouse where you want to store your imported data.
---
### 3. Configure connection
Authenticate your selected tool using OAuth or API credentials. Read documentation for your specific [destination](/destinations/overview) for detailed instructions.
After entering your credentials, click **Test connection** to verify that Hightouch can connect successfully before continuing.
---
### 4. Manage schema
After connecting, Hightouch will detect the schema from your external application.
Use the **visual schema picker** to select which tables and columns to import into your warehouse.
---
### 5. Finalize and schedule
Before completing setup, decide how schema changes should be handled over time:
- **Sync all:** Include all new schemas, tables, and columns automatically
- **Sync new columns only:** Add new columns to existing tables but skip new tables or schemas
- **Do not sync changes:** Keep your schema fixed and ignore new data structures
Then, choose how and when your Connection should sync:
- **Manual:** Run the sync manually in the app or through the API
- **Interval:** Schedule syncs to run at a set interval (for example, every hour)
- **Custom recurrence or Cron expression:** Define your own schedule
- **dbt Cloud or Fivetran:** Trigger syncs automatically after jobs finish in these tools
When finished, click **Finish** to complete setup.
For integration-specific setup instructions, such as authentication details,
required permissions, or field mappings, see the relevant [Destination
documentation](/destinations/overview) for your tool.
---
## Manage and Edit Connections
After creating a Connection, you can manage it from the **Connections** page in your workspace.
From here, you can:
- View recent connection runs and their status
- Edit how schema changes are handled
- Update selected tables and columns when new fields become available
- Modify your sync schedule
Hightouch tracks connection activity and run results automatically, so you can monitor data freshness and resolve issues directly from the UI.
---
## Related articles
- Learn more about integration setup in [Destination documentation](/destinations/overview)
- Visit [Alerting](/syncs/alerting) to configure sync alerts for your Connections
- Review the [Syncs Overview](/syncs/overview) to understand how incoming data supports downstream activation
---
## Trigger syncs with Dagster
**URL:** https://hightouch.com/docs/extensions/dagster
**Description:** Learn how to trigger syncs and gain visibility into sync execution trends with Hightouch's Dagster extension.
**Section:** Extensions
## Overview
Hightouch provides a Dagster extension to trigger syncs and gain visibility into sync execution trends. The [Git repository](https://github.com/hightouchio/dagster-hightouch) contains the latest code.
## Setup
### Create an API key
The first step is to create a Hightouch API key in your Hightouch workspace settings.
### Install library
The simplest way to install the Hightouch Dagster extension is by using `pip`.
```bash
pip install dagster-hightouch
```
### Configure library
To configure the library, provide your Hightouch API key as a resource in the `resources.py` file:
```python
# resources.py
from dagster_hightouch.resources import ht_resource as hightouch_resource
ht_resource = hightouch_resource.configured(
{"api_key": "555555-4444-3333-2222-1111111111"},
)
```
### Set sync schedule type
### Schedule syncs
You can schedule syncs by using the `hightouch_sync_op` function with the sync IDs of the syncs you want to schedule.
This example creates a job to trigger two syncs—one for Salesforce account objects and one for organization objects—every thirty minutes.
```python
# The hightouch_sync_op calls the Hightouch API to trigger a sync, and then polls the API until the sync either completes or returns an error.
from dagster import ScheduleDefinition, get_dagster_logger, job
from dagster_hightouch.ops import hightouch_sync_op
from .resources import ht_resource
# Set sync IDs as constants
SFDC_ACCOUNTS_SYNC_ID = "23620"
SFDC_ORGS_SYNC_ID = "39619"
# Define configured sync ops
run_ht_sync_accounts = hightouch_sync_op.configured(
{"sync_id": SFDC_ACCOUNTS_SYNC_ID}, name="hightouch_sfdc_accounts"
)
run_ht_sync_orgs = hightouch_sync_op.configured(
{"sync_id": SFDC_ORGS_SYNC_ID}, name="hightouch_sfdc_organizations"
)
# Create a job with the defined resources, specifying the dependencies
@job(
resource_defs={
"hightouch": ht_resource,
}
)
def ht_sfdc_job():
ht_orgs = run_ht_sync_orgs(start_after=ht_contacts)
run_ht_sync_accounts(start_after=ht_orgs)
# Schedule it to run as often as you like with a cron expression
# This example shows every 30 min
every_30_schedule = ScheduleDefinition(job=ht_sfdc_job, cron_schedule="*/30 * * * *")
```
To learn more, check out the [blog post](https://hightouch.com/blog/introducing-the-hightouch-dagster-integration) about the extension.
---
## Datadog
**URL:** https://hightouch.com/docs/extensions/datadog
**Description:** The Datadog extension allows you to receive real-time metrics and events about your syncs
**Section:** Extensions
## Overview
Hightouch integrates with Datadog to help you gain greater observability into your syncs and monitor their health. Once you've authorized access to Datadog in Hightouch, we send real-time sync metrics and events to your Datadog account. You can then aggregate these statistics to gain insight into the health of your syncs.
## Setup
You need your [Datadog site](https://docs.datadoghq.com/getting_started/site/) and [API key](https://docs.datadoghq.com/account_management/api-app-keys/) to set up the extension. You may want to create a new API key for Hightouch. Make sure that the API key creator in Datadog has the necessary write permissions for [custom metrics](https://docs.datadoghq.com/metrics/custom_metrics/) (`Metric Tags Write`, `Host Tags Write`, `Metrics Metadata Write`) and [events](https://docs.datadoghq.com/events/) (`Event Config Write`).
Make sure to use the **API Key** itself and not the Key ID. Also ensure you use an API Key and not an Application Key to avoid the `Datadog configuration is invalid` error. You can find more information about the difference between these keys in [Datadog's documentation](https://docs.datadoghq.com/account_management/api-app-keys/).
1. Go to the [**Monitoring** section](https://app.hightouch.com/extensions/monitoring) of the **Extensions** page and select **Configure extension**.
2. Select your Datadog [**site**](https://docs.datadoghq.com/getting_started/site/) and enter your **API Key**.
3. Click **Save**.
Once you've saved your Datadog credentials in Hightouch, your syncs automatically start emitting metrics and events to Datadog.
Hightouch metrics and events don't immediately show up in Datadog, but only once sync runs are triggered.
## Emitted metrics and events
Hightouch emits both [metrics](https://docs.datadoghq.com/metrics/types/) and [events](https://docs.datadoghq.com/events/). Metrics are best suited for dashboards and statistical aggregations, while events are best for filtering. Don't hesitate to if there's another metric or event you're interested in using.
### Metrics
#### `hightouch.sync.row_processed`
This metric is a [`COUNT`](https://docs.datadoghq.com/metrics/types/?tab=count) type metric and is incremented for each row processed in the sync.
It provides the following tags:
- `sync` - The sync slug in Hightouch.
- `workspace` - The slug of the Hightouch workspace that contains the sync.
- `op` (added/changed/removed) - The operation performed: whether the row was added, changed, or removed in the query.
- `success` (true/false) - Whether the row was successfully synced to the destination.
#### `hightouch.sync.complete`
This metric is a [`COUNT`](https://docs.datadoghq.com/metrics/types/?tab=count) type metric and is incremented each time the sync completes.
It supports the following tags:
- `sync` - The sync slug in Hightouch.
- `workspace` - The slug of the Hightouch workspace that contains the sync.
- `status` (success/failed/warning/cancelled) - The overall status of the sync run.
#### `hightouch.sync.query_time`
This metric tracks your syncs' querying times.
It supports the following tags:
- `sync` - The sync slug in Hightouch.
- `workspace` - The slug of the Hightouch workspace that contains the sync.
#### `hightouch.sync.execution_time`
This metric tracks your syncs' execution times.
It supports the following tags:
- `sync` - The sync slug in Hightouch.
- `workspace` - The slug of the Hightouch workspace that contains the sync.
#### `hightouch.sync.total_time`
This metric tracks your syncs' total run times.
It supports the following tags:
- `sync` - The sync slug in Hightouch.
- `workspace` - The slug of the Hightouch workspace that contains the sync.
### Events
#### `hightouch.sync.complete`
Hightouch emits this event each time a sync completes. It's identical to the metric of the same name.
It supports the following tags:
- `sync` - The sync slug in Hightouch.
- `workspace` - The slug of the Hightouch workspace that contains the sync.
- `status` (success/failed/warning/canceled) - The overall status of the sync run.
## Use cases
With just a few metrics, you can build complex dashboards and monitors. For example, you can create dashboards that target specific syncs that are critical to your operations and report the total number of operations or top syncs that have warnings or failures.
You can [import the JSON](https://docs.datadoghq.com/dashboards/#copy-import-or-export-dashboard-json) for a basic dashboard using this [sample code](https://gist.github.com/PedramNavid/71f73c98d98373053674f19b8239b06c) to get started.
### Completed sync status
One common use case is to build a basic dashboard that tracks completed syncs by status over time.
The preceding chart uses the [`hightouch.sync.complete` metric](#hightouchsynccomplete) and sums the counts by `status`. You can filter further by including only statuses that are `status:success` or `NOT status:success`.
### Anomaly detection
Anomaly detection can help flag significant changes, like many users suddenly being deleted or created. You can use the Datadog extension for anomaly detection in added, changed, or removed rows.
#### Anomaly detection setup
1. In Datadog, create a [metric monitor](https://docs.datadoghq.com/monitors/create/types/metric/) with the **Anomaly Detection** method.
2. Define the metric by selecting the [`hightouch.sync.row_processed`](##hightouchsyncrow_processed) metric. Specify the `op_type`, for example, `removed`, and `sync` name. Select the metric aggregation as `sum by` and enter `sync`.
3. In [**Alert Conditions**](https://docs.datadoghq.com/monitors/create/types/metric/?tab=threshold#set-alert-conditions), set the **Alert threshold** and **Warning threshold** to values that make sense for your use case.
4. (Optional) You can inject the sync name into the monitor's subject or message body using `{{sync.name}}`.
### Other use cases
You're not limited to the preceding examples. Here are some other use cases to consider:
- Metric monitors for when the number of failed rows exceeds a threshold
- Metric monitors for when a critical sync hasn't run over a specified time period
---
## dbt Cloud - Trigger syncs and sync models
**URL:** https://hightouch.com/docs/extensions/dbt-cloud
**Description:** The dbt Cloud extension lets you both schedule Hightouch syncs to run as soon as a dbt Cloud job completes, and to sync models from dbt into Hightouch. Relying on dbt Cloud for sync scheduling ensures that Hightouch performs syncs when fresh data is available.
**Section:** Extensions
This extension requires a paid Hightouch plan
dbt requires you to have a paid plan to use this extension
The dbt Cloud extension allows you to:
- Schedule Hightouch syncs to run as soon as a dbt Cloud job completes. Relying on dbt Cloud for sync scheduling ensures that Hightouch performs syncs when fresh data is available.
- Sync models from dbt into Hightouch
Extension setup requires connecting your dbt account to Hightouch.
## Connect to dbt Cloud
Open the [dbt Cloud configuration page](https://app.hightouch.com/extensions/dbt-cloud/configuration) in your workspace and enter the **dbt Cloud URL**. Depending on your setup, you need to insert:
- for regular dbt Cloud setups, a [default access URL](https://docs.getdbt.com/docs/cloud/about-cloud/access-regions-ip-addresses)
- for accounts with cell-based account prefixes, you can find the custom [unique access URL](https://docs.getdbt.com/docs/cloud/about-cloud/access-regions-ip-addresses#api-access-urls) in dbt on the **Account settings** page (below the **Account information** pane)
These are the most commonly used default access URLs:
- `https://cloud.getdbt.com`, for the US
- `https://emea.getdbt.com`, for EMEA
Then navigate to dbt Cloud and generate a [service account token](https://docs.getdbt.com/docs/dbt-cloud-apis/service-tokens) or a [personal access token (PAT)](https://docs.getdbt.com/docs/dbt-cloud-apis/user-tokens). In Hightouch, enter your token in the **dbt Cloud API token** field in the **Configuration** tab of the dbt Cloud extension. Click **Connect** to complete your connection configuration.
A **Member-level or higher** [service account token](https://docs.getdbt.com/docs/dbt-cloud-apis/service-tokens#team-plans-using-service-account-tokens) is required.
Read-only tokens are no longer supported because the dbt Cloud integration uses webhooks, which require elevated permissions.
If you previously used a read-only dbt Cloud token, you must update to a **Member** token (or higher). Without this change, Hightouch syncs triggered by dbt Cloud will fail.
To update your dbt Cloud API token in Hightouch, edit your credentials within the extension and set a new API token. This won't reset the schedule configuration of syncs that use the [dbt Cloud scheduling](#configure-your-sync-schedule) method, nor are any synced models removed.
## Configure sync schedules
When creating or modifying a sync you can select **dbt Cloud** as a schedule type.
1. Go the **Schedule** tab of overview page of the sync to schedule. Select **dbt Cloud** as the schedule type.
2. Select the appropriate dbt **Account** and **Job**.
3. Click **Save**.
Your sync is now scheduled to run whenever your chosen dbt Cloud jobs complete.
The sync won't get triggered if your dbt Cloud job fails.
## dbt Cloud model sync
To sync models from dbt Cloud to Hightouch, the extension requires a **Member-level or higher** service account token as described
in the preceding section. A webhook ID should display next to your account status.
### Connecting a source
On the [extension configuration page](https://app.hightouch.com/extensions/dbt-cloud/configuration) there is a list of sources from your workspace within Hightouch that can be
connected to dbt Cloud. Choose one that you would like to configure and click "Connect to dbt."
Configure the dbt Cloud account and job from the dropdowns, then choose "Create Configuration"
Select a dbt job that has a "Compile" step (`compile`, `build`, `run`) or no model artifacts get produced.
### Syncing models
Once configured, the "dbt to Hightouch" status shows "Waiting to be synced." If your job in dbt Cloud has previous
runs you can click the "refresh" button to manually sync models from the job's result. Otherwise, run the job within dbt
Cloud and the result should trigger the update in Hightouch.
After a successful run, dbt Cloud models are available within Hightouch.
A dbt Cloud job can fail and still produce model artifacts which sync to Hightouch.
### dbt Cloud model sync advantages
Along with the [advantages of using dbt models](/extensions/dbt-models#dbt-model-advantages) that dbt Core brings,
dbt Cloud brings more helpful features:
- dbt Mesh: reference models in other projects (`{{ ref('customers', 'contact_info') }}`)
- "Versionless" (now called "Latest") - always runs your compile job with the latest dbt version
- Private dependencies
- Database adapters that Hightouch doesn't support for dbt Core (such as Fabric)
---
## Create models using dbt models and analyses
**URL:** https://hightouch.com/docs/extensions/dbt-models
**Description:** Connect Hightouch to your dbt project's Git repository and build Hightouch models using existing dbt models and analyses.
**Section:** Extensions
Now you can sync models from [dbt Cloud](/extensions/dbt-cloud).
This extension requires a paid Hightouch plan
Hightouch can communicate with two distinct Git repository types. The dbt extension interacts with Git repositories that contain dbt projects. These are separate from the Git repository used for [Git Sync](/extensions/git-sync) to version control your Hightouch settings.
If you've invested in building models with [dbt](https://www.getdbt.com/), you likely want to use those models as the source of truth for your downstream data needs. At Hightouch, we use dbt to build most of our models for analytics and reporting.
Our dbt integration lets you define Hightouch models using your dbt models and [analyses](https://docs.getdbt.com/docs/build/analyses).
## Supported dbt versions
Hightouch supports dbt Core 1.10 and newer. You can set the version on the dbt setup page. If you need support for a version not listed, .
If you update your dbt version, return to Hightouch to update your dbt version selection. Once you've made the update, it may take up to 15 minutes for the changes to take effect.
## Connect to your Git repository
Navigate to the [dbt extension configuration](https://app.hightouch.com/extensions/dbt-models/configuration) page and enter your Git credentials under **Credentials**.
If you're using GitHub, it's best to use [Hightouch's GitHub App](https://github.com/apps/hightouch-connect/) for the most up-to-date feature support.
To set it up, choose **GitHub App** as the **Git Service**, and then select **Configure GitHub App**.
This prompts you to install the Hightouch Connect app in the relevant GitHub repository.
For other Git services, you can choose to use either **SSH** or **HTTP** protocol. SSH protocol requires you provide an **SSH private key** and HTTP requires a **username** and **token**.
See [GitHub documentation](https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/creating-a-personal-access-token) or [GitLab documentation](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) for instructions on creating personal access tokens.
## Connecting a source
You can use dbt models and analyses from your Git repository for any [source](https://app.hightouch.com/sources) in Hightouch. On the [extensions configuration page](https://app.hightouch.com/extensions/dbt-models/configuration), there is a list of sources from your workspace within Hightouch that can be connected to dbt models. Choose one that you would like to configure and click "Connect to dbt."
## Configure dbt model selector
When connecting dbt sources for the first time or making changes to your dbt model sync configuration, you usually only need to click **Save** and not **Full Resync** for Hightouch to create or update your configuration.
The **Full Resync** action is only necessary when making significant changes in your dbt Git repository. It restarts the dbt compilation from a fresh state and can take a significant amount of time. After clicking this button, wait for the updated dbt models to compile before making any changes to your configuration.
The default schema is the schema where dbt [materializes](https://docs.getdbt.com/docs/build/materializations) your tables to unless otherwise specified in project file. Some common examples are `public`, `production`, `dbt_production`.
Hightouch lets you optionally specify [dbt selectors](https://docs.getdbt.com/reference/node-selection/syntax#examples) like `tag:hightouch`. Otherwise, we use `*` and sync _all_ dbt models to Hightouch.
You can also specify which version of dbt you want to use. By default, Hightouch uses the most recent version available in the drop-down menu.
Hightouch's dbt integration automatically looks for your `dbt_project.yml` file in the repository directory. If your project file is named something else, for example, `dbt_project.yaml`, or you have multiple `dbt_project.yml` files, you can specify a relative path to your project file, for example, `./hightouch/dbt/dbt_project.yaml`.
After saving your settings, it may take a one to two minutes for Hightouch to sync your dbt models and for the sync status to turn green.
## Select a model
Once you've configured the dbt model selector, you can use dbt models and analyses in the Hightouch [model creation process](/models/creating-models). Select the dbt model tab in the query explorer to view all your available dbt models and analyses.
The compiled SQL shown in the preview is what's ran against your source. We do not leverage the output table written by dbt.
### Model column descriptions
Hightouch pulls in column descriptions from dbt.
You can see them in the **Columns** tab of a model's overview page.
## Unsupported features
Hightouch doesn't support the following features:
- Private dependency packages
- Pre- and post-hooks (limited support, this may work depending on the hook)
- [Python models](https://docs.getdbt.com/docs/build/python-models)
Hightouch compiles the project using `dbt compile` to resolve your data warehouse's schema and generate compiled SQL for preview purposes. When running the model in Hightouch, we query the materialized table directly.
## dbt model advantages
When deciding between Hightouch's [table selector](https://hightouch.com/docs/models/table-selector) and our [dbt model selector](https://hightouch.com/docs/extensions/dbt-models), the advantages of leveraging dbt models might not be immediately clear, since many dbt models are often materialized as tables and views in a [Source](https://hightouch.com/docs/getting-started/concepts#sources).
However, there are a few key benefits of building Hightouch Models using the dbt model selector:
- It provides a clean, information-rich modal with metadata about the underlying model, including its unique ID, description, and repository link.
- It allows you to automatically pull in [column descriptions](https://hightouch.com/docs/extensions/dbt-models#model-column-descriptions) into Hightouch for data governance/organization.
- It allows you to leverage our [dbt observability layer](https://hightouch.com/docs/extensions/dbt-observability), which includes dbt exposures and CI checks.
- Not all dbt resources materialize as tables/views in the warehouse. For example, dbt analyses will not form tables/views at all.
## dbt observability layer
Hightouch lets you [validate dbt model changes](/extensions/dbt-observability#dbt-ci-checks) as they relate to Hightouch syncs and observe [how Hightouch syncs are using your dbt models](/extensions/dbt-observability#exposure-syncs). Check out our [dbt observability layer documentation](/extensions/dbt-observability) for more information.
## Tips and troubleshooting
Hightouch compiles the project using [`dbt compile`](https://docs.getdbt.com/reference/commands/compile) to resolve your data warehouse's schema and generate compiled SQL for your model. If you encounter error messages on your dbt extension configuration page, try running [`dbt compile`](https://docs.getdbt.com/reference/commands/compile) locally on your dbt project to see if the errors still occur.
### Credential errors
If your Git credentials are out of date, you won't be able to connect your dbt models and analyses.
We recommend using [Hightouch's GitHub app](#connecting-to-your-git-repository) for your credentials if you're using GitHub.
### Duplicated mapping key error
You may receive a `duplicated mapping key` error under your dbt model sync status if there are duplicate keys in your `dbt_project.yml` file.
To resolve the error, you can either remove the duplicates or rename duplicated keys.
### The YAML property file is missing a version tag
This dbt compilation error can occur if you use the same Git repository for [Git Sync](/extensions/git-sync) and [dbt models sync](#configure-dbt-model-selector).
To solve this, make sure that you're using two different Git repositories for these extensions.
Otherwise, if you would like to use the same repository, you can specify a custom path in the [Git Sync configuration](/extensions/git-sync#configure-git-sync).
---
## dbt observability layer
**URL:** https://hightouch.com/docs/extensions/dbt-observability
**Description:** See which Hightouch syncs are using your dbt models from your dbt docs and lineage graph.
**Section:** Extensions
Hightouch provides two separate features to monitor your dbt models as they relate to Hightouch syncs:
- [dbt CI checks](#dbt-ci-checks)
- [dbt exposure sync](#dbt-exposure-sync)
You can use either one or both of these features together. Both require that you've [set up dbt-based models](/extensions/dbt-models) in Hightouch.
## dbt CI checks
The dbt CI checks feature doesn't currently work with [Customer Studio](/customer-studio/overview) models.
You can use Hightouch's **dbt CI checks** to validate changes in your dbt project Git repository. This feature checks your GitHub pull requests to ensure they don't break any existing, [enabled](/syncs/overview#individual-sync-overview-page) dbt-based syncs in Hightouch.
To enable this, toggle on **GitHub Pull Request checks** from your dbt connected source's [configuration page](https://app.hightouch.com/extensions/dbt-models/configuration) and then select **Save**. You need to do this for every connected source you want to enable CI checking on.
Once enabled, Hightouch runs a CI check on any pull request made to the dbt project's Git repository. If a breaking change is detected—for example, deleting, renaming, or moving a model used in a sync—the check fails.
Clicking into the details of a failed CI check shows which sync the breaking change effects.
The CI check passes if the pull request doesn't affect any [enabled](/syncs/overview#individual-sync-overview-page) Hightouch syncs.
Though dbt CI checks work similarly to [version control with Git
Sync](/extensions/git-sync#validating-changes), these are separate features
and aren't dependent on one another.
## dbt exposure sync
Hightouch exposure sync lets you see which Hightouch syncs are using your dbt models via your dbt docs and lineage graph.
Exposure sync works by defining and updating exposures in your chosen dbt repository anytime a Hightouch sync that uses a dbt model is added or modified.
### Turn on exposure sync
You can enable exposure sync by navigating to **dbt Models** on the [**Extensions page**](https://app.hightouch.com/extensions) and selecting your **dbt Connected Source**. From there, activate the **dbt Exposures Sync** toggle and click **Save**.
You should see exposures starting to sync within a few minutes.
In your dbt repository, you should see a new file called `hightouch_sync_exposures.yml`. This file updates every time a Hightouch sync that uses a dbt model is added, modified or disabled.
Exposures will be created for syncs using audiences created with Customer Studio where the [parent model](/customer-studio/schema#1-define-the-parent-model) is a dbt model. If an audience references [related or event models](/customer-studio/schema#related-models-and-events) that are built with dbt models, this information won't be included in the exposure file. If you'd like to request this functionality, please .
## Tips and troubleshooting
### Git credentials
If your Git credentials are out of date, you won't be able to connect your dbt models and analyses. We recommend using [Hightouch's GitHub app](https://github.com/apps/hightouch-connect/) for your credentials if you're using GitHub.
The Hightouch GitHub app must have commit access to your chosen dbt branch to create exposures. If Hightouch is unable to push to your branch you may have to update your branch protection rules in your dbt GitHub repository.
### dbt build errors
The Hightouch exposures file is intended to stop you from breaking Hightouch syncs as a result of deleted dbt models. If you need to delete a model defined in the Hightouch exposures file, you can remove the Hightouch exposures file at any point—Hightouch will automatically recreate it.
### Your local changes to the following files would be overwritten by checkout
This error generally occurs if the **exposure sync branch** in your linked Git repository has diverged from your main branch.
Once you update your exposure branch so that it's up-to-date with your main one, it should resolve the error. You can find more information on how you can resolve this type of error in this [Stack Overflow article](https://stackoverflow.com/questions/22424142/error-your-local-changes-to-the-following-files-would-be-overwritten-by-checkou).
### Remote rejected, protected branch hook declined
You may receive this error if you have [branch protections](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches) on the **branch** you selected for you [exposure sync configuration](#turn-on-exposure-sync).
To avoid this, you can select a branch to use for exposure syncs specifically.
Then, you can open a pull request from the dbt exposure sync branch to your main dbt branch.
This allows the exposure sync with Hightouch to operate smoothly while also allowing branch protection rules to be applied to the main branch.
You can read more about a similar error in [this section](/extensions/git-sync#protected-branch-update-failed) of the Git Sync page.
---
## Trigger syncs with Fivetran
**URL:** https://hightouch.com/docs/extensions/fivetran
**Description:** The Fivetran extension lets you schedule Hightouch syncs to run as soon as a Fivetran job completes. Relying on Fivetran dbt transformations and connectors for sync scheduling ensures that Hightouch performs syncs when fresh data is available.
**Section:** Extensions
The Fivetran extension lets you schedule Hightouch syncs to run as soon as a Fivetran job completes. Relying on Fivetran dbt transformations and connectors for sync scheduling ensures that Hightouch performs syncs when fresh data is available.
Extension setup requires two steps:
- Connecting your Fivetran account to Hightouch
- Configuring your Hightouch syncs to trigger with Fivetran
## Connect to Fivetran
Navigate to your Fivetran dashboard and [retrieve your API Key and Secret](https://fivetran.com/docs/rest-api/getting-started#gettingstarted).
The API Key and Secret must belong to a user with the [Account Administrator role](https://fivetran.com/docs/getting-started/fivetran-dashboard/account-management/role-based-access-control#rolesinourrbacmodel) or you'll receive [an error](#forbidden-access-denied-error) when trying to save your configuration.
In Hightouch, enter your credentials in the [Fivetran configuration page](https://app.hightouch.com/extensions/fivetran/configuration) and click **Connect**.
Behind the scenes, Hightouch adds a new [signed](https://fivetran.com/docs/rest-api/webhooks#signing) webhook to your Fivetran account.
## Configure your sync schedule
Now, when creating or modifying a sync you can select **Fivetran** as a schedule type.
1. Go the **Schedule** tab of overview page of the sync you want to schedule. Select **Fivetran** as the schedule type.
2. Select the appropriate **Fivetran workspace**.
3. Select one or more Fivetran connectors or Fivetran-managed dbt Core transformations as the **Fivetran jobs** to trigger the sync.
3. Click **Save**.
Your sync should is now scheduled to run whenever you chosen Fivetran jobs complete.
## Tips and troubleshooting
### Forbidden access denied error
Ensure your Fivetran account has the [Account Administrator role](https://fivetran.com/docs/getting-started/fivetran-dashboard/account-management/role-based-access-control#rolesinourrbacmodel).
Otherwise, you might encounter this error when saving your Fivetran configuration in Hightouch:
`Forbidden access denied. Required permission: ACCOUNT_SECURITY_MANAGE`
---
## Git Sync
**URL:** https://hightouch.com/docs/extensions/git-sync
**Description:** Version-control supported Hightouch resources in a Git repository.
**Section:** Extensions
{/* */}
Git Sync connects your Hightouch workspace to a Git repository so you can version-control supported Hightouch resources.
With Git Sync, you can:
- Create and edit multiple Hightouch models and syncs in code
- Review changes through Git commits and file diffs
- Roll resources back to a previous version
- Keep staging and production workspaces aligned through shared YAML files
Git Sync is bidirectional:
- Changes made in the Hightouch UI are written to YAML files in Git. These changes are marked as `OUTBOUND`.
- Changes made to YAML files in Git are applied back to Hightouch. These changes are marked as `INBOUND`.
Each update to a supported resource creates a commit in your configured repository.
Git Sync currently supports models (including Customer Studio schema models), syncs, and event contracts. Git Sync does not currently support Customer Studio audiences or syncs created from those audiences.
Inbound changes made in your Git Sync repository don't require approval from
another team member, even if your workspace uses [approval
flows](/workspace-management/approval-flows).
Git Sync supports most Git providers, including GitHub, Bitbucket, GitLab, and private [on-premise](#tunneling) repositories. In most cases, changes sync between Hightouch and Git in about one minute.
Hightouch can connect to Git repositories for two different purposes.
Git Sync uses a Git repository to version-control Hightouch resources. This is separate from Git repositories used to [leverage dbt models](/extensions/dbt-models).
---
## Setup
To set up Git Sync, complete these steps:
1. Create or choose a Git repository.
2. Authenticate Hightouch with your Git provider.
3. Configure Git Sync in Hightouch.
### Create or choose a Git repository
Use a Git repository to store the YAML files that represent your supported Hightouch resources.
If you are creating a new repository, GitHub repositories are the easiest to start with because they can use the Hightouch GitHub app.
If you create an empty repository, make an initial commit, such as adding a README file. This creates the branch Hightouch needs to connect to.
### Authenticate to Git
After you have a repository, configure your Git credentials in Hightouch.
1. Go to the [Git Sync configuration page](https://app.hightouch.com/extensions/git-sync/configuration).
2. Under **Configuration > Git credentials**, select **Set up**.
3. Choose your Git provider and authentication method.
If you use GitHub, we recommend using the [Hightouch GitHub app](https://github.com/apps/hightouch-connect/) for the most up-to-date feature support.
Regardless of provider, grant Hightouch access only to the repository or repositories you want to use with Git Sync.
#### Authenticate to a GitHub Enterprise repository
If you use GitHub Enterprise with a self-hosted server:
1. Under **Git service**, select **Other**.
2. For **Username**, enter your full GitHub Enterprise username. For example, `hightouch123@github.hightouch.com`.
3. For **Token**, enter a [GitHub access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) tied to that username.
4. Make sure the token has access to the repository or repositories Hightouch should use.
#### Authenticate to an AWS CodeCommit repository
If you use AWS CodeCommit, you must use SSH to authenticate.
1. Configure your AWS CodeCommit repository for SSH by following [AWS's setup instructions](https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-ssh-unixes.html).
2. In Hightouch, select **Other** as the **Git service**.
3. Select **SSH** as the **Protocol**.
4. Under **SSH Private key**, enter the private key you generated when configuring AWS CodeCommit.
5. Select **Save**.
6. For **Repository**, enter the full SSH URL, including the AWS SSH key ID.
The AWS SSH key ID is different from the general access keys in your AWS account.
You can copy your URL from these [instructions in AWS's docs](https://docs.aws.amazon.com/codecommit/latest/userguide/how-to-connect.html).
### Configure Git Sync
After you authenticate to Git, enable Git Sync for your workspace.
1. Go to [Integrations > Extensions](https://app.hightouch.com/extensions/git-sync/configuration).
2. Open the **Version control with Git** extension.
3. On the **Configuration** tab, enable **Git Sync**.
4. Select the **Repository** and **Branch** Hightouch should read from and write to.
5. Optionally, enter a **Path** where Hightouch should create folders for Git Sync files.
6. Select **Save**.
If you enter a path, Hightouch creates the `models`, `syncs`, and other supported folders inside that subfolder. If you don't enter a path, Hightouch saves Git Sync files in the root of your repository.
After you save the configuration, Hightouch creates the configured folder structure if needed. Git Sync then starts syncing supported resources between Hightouch and Git.
To start the syncing process immediately, select **Full resync**.
This pushes all supported workspace resources to Git, then pulls changes from Git back to your workspace. Because this happens in that order, a full resync can overwrite Git changes that have not synced to Hightouch yet.
For more information, see [What does the Full resync button do?](#what-does-the-full-resync-button-do).
Git Sync does not support deletions from Git.
If you delete a YAML file in Git, the corresponding model or sync remains in Hightouch. The file is not recreated in Git until the next change is made in Hightouch or you trigger a [full resync](#what-does-the-full-resync-button-do).
---
## Use Git Sync with branch protection
If your repository uses branch protection, you may need to let the Hightouch Connect app bypass specific branch rules. The Hightouch Connect app can only push to a protected branch if you add it to the bypass list under GitHub rulesets.
To configure this in GitHub:
1. Go to **Settings > Rules > Rulesets**.
2. Add the Hightouch Connect app to the **Bypass list**.
3. Configure the rest of the ruleset to enforce your branch protection requirements.
GitHub supports branch protection in both **Settings > Branches** and **Settings > Rules > Rulesets**.
The Hightouch Connect app only works with branch protection when you add it to the **Bypass list** under **Rulesets** and define the rest of your branch protection rules there. It does not work as a force-push app under **Branches**.
---
## Connect staging and production workspaces
You can use Git Sync to keep staging and production workspaces aligned through the same Git repository. When multiple workspaces connect to the same repository, [alias files](#aliases) handle environment-specific source and destination IDs.
### Use the same branch
For most teams, the simplest setup is to point both workspaces to the same repository and branch, such as `main`.
1. In each workspace, go to the [Git Sync configuration page](https://app.hightouch.com/extensions/git-sync/configuration).
2. Select the same repository and branch.
3. Share model and sync YAML files between both workspaces.
4. Create [alias files](#aliases) for each workspace to map shared aliases to the correct source and destination IDs.
For example, both workspaces can reference `aliased-postgres` in their model files. The staging alias file maps `aliased-postgres` to `staging-postgres`, while the production alias file maps it to `production-postgres`.
For a full staging-to-production example, see [Aliases](#aliases).
### Use separate branches
For a more controlled promotion workflow, use a separate branch for each environment.
1. Point your staging workspace to a development branch, such as `develop`.
2. Point your production workspace to your production branch, such as `main`.
3. Test and iterate in staging. Changes sync to the development branch.
4. When ready, open a pull request to merge the development branch into the production branch.
5. After the pull request is merged, Git Sync picks up the changes in your production workspace.
Use [CI checks](#validating-changes) to validate changes before they reach production.
### Understand what syncs across environments
Git Sync can version-control and replicate the following resources across environments:
- Models
- Syncs
- Event contracts
- Customer Studio schema models, including parent, related, and event models
- Customer Studio schema relationships
Git Sync does not replicate everything. Configure the following manually in each workspace:
- Customer Studio traits, audiences, and syncs from audiences
- Workspace-level settings, such as approval flows, spaces, and environments
- Column-level schema settings configured in the UI, such as redaction, column suggestions, and display configuration
For Customer Studio schema setup instructions, see [Configure Schema Git Sync](/customer-studio/schema-setup).
---
## Schemas
Git Sync represents supported Hightouch resources as YAML files in your Git repository.
Field names in YAML files are case-sensitive. For example, a field named
`Email` is not the same as `email`. This applies to all field references,
including source column names, primary keys, and field mappings. If you rename
a column in your warehouse changing only the casing, Git Sync treats it as a
different field, which can cause Git Sync errors or unexpected behavior.
---
### Model schema
Models use the following schema:
```yaml
name: > the model name/slug. unique per model
source: > the name/slug of the source. the slug is visible on the source page
type: > one of: raw_sql, table, or dbt_model
dbtModel: > null or the name of the dbt model to select, written as `model..`
rawSql: > null or the sql to use, for example, select * from albums
tableName: > null or the table name to use
isSchema: > used internally by Hightouch, this should always be false
primaryKey: > primary key column for this model, for example, album_id
```
---
### Sync schemas
Sync schemas vary by destination. Because each destination has different configuration options, we recommend creating syncs in the Hightouch UI first. You can then use the generated YAML file as a starting point for updates or new syncs. The YAML filename is the sync name.
```yaml
model: > model name/slug for this sync.
destination: > destination name/slug for this sync
config:
> destination dependent configuration for this sync.
schedule: > type of schedule to use. the format here varies based on the
schedule selected, we recommend using the UI as a starting point.
schedulePaused: > boolean, where this sync is paused, or enabled.
```
Based on the boolean value assigned to `schedulePaused`, your sync will either
be [enabled or disabled](/syncs/overview#individual-sync-overview-page).
Avoid modifying YAML files while a sync is actively running. Pause the sync
in the Hightouch UI before committing structural changes to prevent unexpected
behavior.
---
### Customer Studio Schema
Git Sync supports version-controlling your Customer Studio schema—including parent models, related models, event models, and relationships. For full setup instructions, see [Configure Schema Git Sync](/customer-studio/schema-setup).
Support for Customer Studio schema models is currently a beta feature. Please if you're interested in this feature.
#### Parent model schema
[Parent models](/customer-studio/schema#1-define-the-parent-model) live in the `schema/parent-models` directory. They have the same schema as [regular models](#model-schema), with the following additions:
```yaml
primaryLabel: "primary_label_column" # Optional
secondaryLabel: "secondary_label_column" # Optional
```
#### Related model schema
[Related models](/customer-studio/schema#2-add-related-models) live in the `schema/related-models` directory. They have the same schema as [regular models](#model-schema).
#### Event model schema
[Event models](/customer-studio/schema#3-add-event-models) live in the `schema/event-models` directory. They have the same schema as [regular models](#model-schema), with the following addition:
```yaml
eventTimestampColumn: "column_name" # Required
```
#### Model relationship schemas
[Model relationships](/customer-studio/schema#relationships) live in the `schema/model-relationships` directory.
The schema for model relationships depends on whether the relationship is direct or through other relationships.
Direct relationships look like this:
```yaml
from:
relationshipName: "" # Relationship name for from -> to
slug: "model_slug"
mergeColumns: false # or true
to:
relationshipName: "" # Relationship name for to -> from
slug: "model_slug"
mergeColumns: false
cardinality: "1:1" # One of: "1:1", "1:many", "many:1"
mapping:
- fromColumn: ""
toColumn: ""
```
Through relationships look like this:
```yaml
name: "relationship name"
from:
slug: "model_slug"
to:
slug: "model_slug"
through:
# List of relationship slugs that constitute this through relationship.
- relationship_slug_1
- relationship_slug_2
```
---
### Git Sync limitations for Customer Studio schema
Git Sync currently supports version-controlling model definitions and relationship structure. However, certain schema configurations set in the Hightouch UI are not synced across environments.
For example, Git Sync does **not** copy:
- Column-level settings (for example, redaction, column suggestions, or display configuration set in the UI)
- Merge column selections configured in the UI
- Column aliases
If you migrate a schema to another environment using Git Sync, you may need to reconfigure these settings manually after deployment.
Field names are also case-sensitive in YAML. If a column name changes only in casing (for example, `user_id` to `User_Id`), Git Sync treats these as different fields. Make sure column references in your YAML files match the exact casing used in your warehouse.
→ For full setup instructions, see [Configure Schema Git Sync](/customer-studio/schema-setup).
---
### Event Contract Schema
#### Event Contracts
[Event Contracts](/events/contracts/management) have a simple, consistent schema:
```yaml
name: > name of the event contract
description: > optional: description for the contract
onUndeclaredSchema: > optional: one of "ALLOW_EVENT" (default) or "BLOCK_EVENT"
```
The name of the file is the slug of the event contract, which is derived from the name of the contract.
#### Events
[Event Schemas](/events/contracts/management#event-definition) are YAML files that define the allowed format of events sent to Hightouch.
Each file represents a single version of a single event within a contract.
The files are located in the `events/contracts` directory, within a subdirectory named after the contract.
The filename is the slug of the event, which is derived from the name or type, the contract slug, and the version.
The schema for an event is as follows:
```yaml
type: > one of "track", "identify", "group", "page", "screen", or "alias"
name: > for track events, the event name; otherwise should not be present
version: > optional: the version of the event; defaults to "default"
onSchemaViolation: > optional: one of "ALLOW_EVENT" (default) or "BLOCK_EVENT"
onUndeclaredFields: > optional: one of "ALLOW_EVENT" (default), "BLOCK_EVENT", or "OMIT_FIELDS"
includeBuiltInContext: > optional: one of "true" (default), or "false"
schema: |
Schema object for validating the event, adhering to the JSON Schema spec
Usually the root of this schema is an object which contains a nested object named properties
```
In the following example, we have a Sample Event that contains a single optional "id" field:
```yaml
# events/contracts/web/sample-event-web-default.yaml
name: Sample Event
onSchemaViolation: ALLOW_EVENT
onUndeclaredFields: ALLOW_EVENT
schema:
type: object # The root of our schema is an object
properties: # Objects contain a "properties" element in the JSON Schema spec
properties: # We define a single child named properties
type: object # Which is an object
properties: # Finally, the properties our child object contains
id: # Which is only an optional id field
type: string # Which is of type string
type: track
version: default
```
This corresponds to the following JSON in the in-app JSON Editor:
```json
{
"type": "object",
"properties": {
"properties": {
"type": "object",
"properties": {
"id": {
"type": "string"
}
}
}
}
}
```
Because event schemas may get complex, using Hightouch [CI checks](#validating-changes) is a great way to validate your changes
before merging them into your repository.
---
### Manifests
Hightouch creates a manifest file at the root of your Git repository for each connected workspace. **Manifest files** list all sources and destinations in a workspace, making it easier for you to find their IDs to add to sync and model files or reference in [aliases](#aliases).
Manifest filenames follow this format: `manifest-.yaml`. A manifest file follows this format:
```yaml
# READ ONLY FILE
# Changes made here will be ignored. Please use the Hightouch dashboard to create and edit sources and destinations.
sources:
production-postgres:
name: Production postgres
type: postgres
snowflake-243-tenant:
name: Snowflake EU instance
type: snowflake
destinations:
production-salesforce:
name: Production Salesforce
type: salesforce
```
Sources and destinations are listed by their ID. In the example file above, the Postgres source's ID is `production-postgres` and the Salesforce destination's ID is `production-salesforce`.
Manifest files are **read-only**. Git Sync overwrites any changes made to
manifest files. Use the Hightouch dashboard to create and edit sources and
destinations.
---
### Aliases
Managing multiple Hightouch workspaces from the same Git repository can be difficult because source and destination IDs can differ between workspaces. **Alias files** let you create custom IDs for sources and destinations in a given workspace allowing model and sync files to work across different workspaces.
Hightouch creates an empty alias file in your Git repository for each connected workspace. Alias filenames follow this format: `aliases-.yaml`.
You can then add sources and destinations to the alias file following this format:
```yaml
# e.g. aliases-hightouch-production-workspace.yaml
sources:
my-source-alias: my-actual-source-id
othersourcealias: another-workspace-source
destinations:
my-destination-alias: existing-destination
```
Each key is an alias and each value should be an actual source or destination ID. You can find source and destination IDs in the [manifest file](#manifests) in the root of your Git repository.
Aliases should be unique across sources and destinations in the same
workspace. In other words, you can't use the same alias for multiple resources
in the same alias file.
#### Example alias usage
Consider a workspace, `acme-staging`, with a model `stocking-list.yaml` file like this:
```yaml
name: Stocking list
source: staging-postgres
type: table
tableName: public.workspaces
isSchema: false
primaryKey: id
```
Another workspace called `acme-production` is connected to the same repository but can't use this model during Git Sync. That's because the `acme-production` workspace doesn't have a source with the ID `staging-postgres`. Instead, `acme-production` uses a Postgres source with the ID `production-postgres`.
You can use aliases so that both workspaces can use the same model `stocking-list.yaml` file.
1. Add an alias to the `acme-staging` workspace alias file:
```yaml
# aliases-acme-staging.yaml
sources:
aliased-postgres: staging-postgres
```
This creates the custom ID `aliased-postgres` that points to the source with the ID `staging-postgres`.
2. Add an alias to the `acme-production` workspace alias file:
```yaml
# aliases-acme-production.yaml
sources:
aliased-postgres: production-postgres
```
This creates the custom ID `aliased-postgres` that points to the source with the ID `production-postgres`. For the next step, it's important that both alias files use the same alias name. Since these aliases exist in different workspaces, they are still unique within their respective workspaces.
3. Update the `source` value of `stocking-list.yaml` to `aliased-postgres`.
```diff
name: Stocking list
- source: staging-postgres
+ source: aliased-postgres
type: table
tableName: public.workspaces
isSchema: false
primaryKey: id
```
Now, during Git Sync, both workspaces resolve `aliased-postgres` to their respective Postgres slugs and Git Sync completes for both staging and production workspaces.
---
## Validate changes
You can use Hightouch CI checks to validate changes in your Git Sync repository before merging them.
To enable CI checks, turn on **GitHub Pull Request checks** in your Git Sync [configuration page](https://app.hightouch.com/extensions/git-sync/configuration).
After CI checks are enabled, Hightouch runs a check on each pull request made to the connected Git repository. If a breaking change is detected, such as deleting a model used by a sync, the CI check fails. Open the failed check details to see which sync is affected.
If the pull request does not affect any Hightouch syncs, the CI check passes.
---
## Review change history
Git Sync gives you a changelog through commit history. You can also use
[audit logs](/workspace-management/audit-logs) to review changes in Hightouch.
You can use your Git repository to see who created or modified supported resources and when.
To show the correct user for Hightouch-created commits, the user's Hightouch email must match their GitHub email.
In GitHub, open the repository's **Commits** tab or file history to review changes.
Open a commit to see the exact line-by-line diff.
---
## Use tunneling for private Git repositories
If your Git repository is hosted on an on-premise server without public internet access, use [tunneling](/security/ssh-tunneling) to connect Hightouch to your Git server.
1. Go to the [**Tunnels** tab](https://app.hightouch.com/settings/tunnels) on the **Settings** page.
2. Create a tunnel or reverse tunnel that connects to your Git server. For more information, see the [tunneling documentation](/security/ssh-tunneling).
3. Go to the [Git Sync configuration page](https://app.hightouch.com/extensions/git-sync/configuration).
4. Under **Configuration > Git credentials**, select **Manage**.
5. Add your Git credential and select the correct tunnel.
We recommend using the SSH protocol and `ssh_privatekey` to connect to private Git repositories.
Basic authentication can have issues forwarding authentication headers through an SSH tunnel and HTTP redirects. For more information about SSH authentication, see the [GitHub SSH documentation](https://docs.github.com/en/authentication/connecting-to-github-with-ssh).
Hightouch does not support `ssh_privatekey` with passphrases.
---
## Next steps
After Git Sync is configured, you can use Git workflows to manage supported Hightouch resources. For example, you can:
- Create a new model by copying an existing model file and updating the name and query
- Create a new model and sync in the same pull request
- Use [CI checks](#validate-changes) to validate changes before merging them
- Use [aliases](#aliases) to share model and sync files across staging and production workspaces
---
## FAQ
### What does the Full resync button do?
Selecting **Full resync** runs an outbound sync for all supported resources in the workspace. This pushes all supported resources from Hightouch to Git, then pulls changes from Git back to your workspace.
If your Git repository has changes that have not synced to Hightouch yet, **Full resync** can overwrite those changes.
{/* */}
### Recreate a deleted YAML file
Git Sync does not support deletions from Git. If you delete a YAML file in Git, the corresponding model or sync remains in Hightouch. The file is not recreated in Git until the next change is made in Hightouch or you trigger a [full resync](#what-does-the-full-resync-button-do).
To recreate a deleted YAML file, select **Full resync**.
You can also recreate the file manually by adding a YAML file with the same filename and [schema](#schemas) as the original file. Hightouch recognizes that the file refers to an existing model or sync and associates it with the existing resource.
### Protected branch update failed
If you selected a protected branch, Git Sync setup might fail with this error:
```
Protected branch update failed - Changes must be made through a pull request
```
To resolve the error, use one of these options:
- Select a branch that is not protected, then open pull requests in GitHub to merge updates into the protected branch.
- Follow the instructions in [Use Git Sync with branch protection](#use-git-sync-with-branch-protection) to let the Hightouch Connect app work with your protected branch.
---
## Create models using Looker Looks
**URL:** https://hightouch.com/docs/extensions/looker-models
**Description:** Connect Hightouch to your Looker instance and build Hightouch models using existing Looks.
**Section:** Extensions
If you've already invested in modeling business data in [Looker](https://www.looker.com/), you may want to use your Looks as the basis for [Hightouch models](/getting-started/concepts#models).
When you import a model from Looker, Hightouch converts your Look into a SQL query and runs it directly against your data warehouse. As you publish changes in Looker, your Hightouch model updates to reflect the latest version of your Look.
This extension uses Looker as a *modeling method*, not a *data source*. In
other words, Looker provides the SQL query, not the query results. Before
enabling this extension, you must first [create a
source](/syncs/create-your-first-sync#connect-a-source) in Hightouch that
connects to the same data warehouse used by Looker.
## Supported sources
You can create models based on Looker Looks for the following sources:
- [Amazon Redshift](/sources/amazon-redshift)
- [Amazon Athena](/sources/amazon-athena)
- [Google BigQuery](/sources/google-bigquery)
- [Greenplum Database](/sources/greenplum)
- [PostgreSQL](/sources/postgresql)
- [Snowflake](/sources/snowflake)
## Connect to Looker
In Hightouch, go to the [**Extensions** page](https://app.hightouch.com/extensions) and select [**Looker**](https://app.hightouch.com/extensions/looker/configuration).
Enter the following fields:
- (Optional) **Tunnel** - If you've previously set up an [SSH tunnel](/security/ssh-tunneling) and want to use it to connect to Looker, you can select it here.
- **URL** - Hightouch uses this URL to connect to your Looker instance. The format is often `https://.looker.com`. Refer to [Looker's documentation](https://docs.looker.com/reference/api-and-integration/api-getting-started#looker_api_path_and_port) if you have a non-standard Looker setup.
- **Client ID** - You can find this in your [Looker API 4 credentials](https://docs.looker.com/reference/api-and-integration/api-auth).
- **Client secret** - You can find this in your [Looker API 4 credentials](https://docs.looker.com/reference/api-and-integration/api-auth).
If you need to create credentials, refer to Looker's documentation on [authentication with an SDK](https://docs.looker.com/reference/api-and-integration/api-auth#authentication_with_an_sdk).
## Select a Look
After configuring your Looker credentials, you can create a model using a [source that supports Looker](#supported-sources). Select your source and then choose **Import Model from Looker** as your modeling method.
On the next screen, you can select which Look you want to base your model on.
## Tips and troubleshooting
If you encounter any error messages during Look-based model setup, confirm the following about your credentials:
### Permissions
You must be a Looker admin to retrieve your Looker connection URL. Check [Looker's documentation on roles, permissions, and access](https://docs.looker.com/admin-options/settings/roles) for more information.
### Looker URL
Check you're using the correct URL by consulting [Looker's API connection troubleshooting guide](https://docs.looker.com/reference/api-and-integration/api-troubleshooting#endpoint_unreachable).
### Common error: 404 Looker is unavailable
When the Looker extension can’t reach your Looker instance (or the API credentials are invalid/expired), Looker may return an HTML 404 page instead of a JSON API error. In Hightouch this can appear as:
> Look not found. Has it been deployed to production?
> Looker Not Found (404) Looker is unavailable.
> If you typed in a URL, double-check the spelling.
> This may also be due to a temporary condition such as an outage, scheduled maintenance or upgrade.
If you see this for all Looker-based models:
- Check the **Looker API 4 client ID/secret** in **Extensions → Looker** (rotate if expired).
- Confirm the **Looker URL** is correct.
- Verify your Looker instance is reachable and not under maintenance.
---
## Trigger syncs with Mage
**URL:** https://hightouch.com/docs/extensions/mage
**Description:** Learn how to trigger syncs with Mage.ai.
**Section:** Extensions
## Overview
Mage provides a native `HightouchClient` to trigger Hightouch syncs. Check out the [Mage's integration documentation](https://docs.mage.ai/integrations/hightouch) for usage details.
## Setup
To use Mage to trigger syncs, you need a Hightouch API key.
### Create an API key
The first step is to create a Hightouch API key in your Hightouch workspace settings.
### Set sync schedule type
## Usage
You can then trigger a sync with Mage's `HightouchClient`:
```python
from mage_ai.services.hightouch.hightouch import HightouchClient
client = HightouchClient(config=dict(api_key='api_key'))
syncs = client.list_syncs() # Get all the syncs
sync_id = syncs['data'][0]['id'] # Get sync id
client.sync_and_poll(sync_id) # Trigger sync and poll status
```
If successful, the output looks like this:
```bash
Polling Hightouch Sync [id]. Current status: querying. 0% completed.
Polling Hightouch Sync [id]. Current status: success. 100% completed.
Sync request status: success. Polling complete
```
---
## PagerDuty
**URL:** https://hightouch.com/docs/extensions/pagerduty
**Description:** The PagerDuty extension lets Hightouch notify your on-call staff about failed queries and syncs.
**Section:** Extensions
The PagerDuty extension lets Hightouch notify your on-call staff about failed queries and syncs.
Once you've authorized access to PagerDuty, you must [set up alerts for each sync](/syncs/alerting#sync-alert-setup) that you want to send alerts through PagerDuty on.
## Setup
1. Create a PagerDuty Events API V2 extension and copy its API Key:
- In PagerDuty, go to the **Integrations** tab in the **Services** page. Select **+ Add another Integration**.
- Select **Events API V2** integration and click **Add**.
- Back in the **Integrations** tab, copy the **Integration Key**.
2. In Hightouch, go to the [**Extensions** page](https://app.hightouch.com/extensions) and select [**Alerting**](https://app.hightouch.com/extensions/alerting). Open the [**Configuration** tab](https://app.hightouch.com/extensions/alerting/configuration).
3. Click **Set up** under PagerDuty and paste the **PagerDuty API Key** you previously copied.
4. In the [**Configuration** tab](https://app.hightouch.com/extensions/alerting/configuration), click **Add alert**.
5. In the modal that appears, enter a descriptive name for your alert, select **PagerDuty** as the **Platform** and select the **Credentials** you just entered.
6. Set the **Default Behaviors**, and optionally enter a **Minimum Alert Interval**.
- The **Default Behaviors** can be **fatal error** and/or **row error**.
- A **fatal error** is an error communicating with either a source or a destination. it's 'fatal' because the sync can't even run.
- A **row error** means a problem with a particular row or rows. The sync works, but one or more rows failed.
- Setting the **Minimum Alert Interval** lets you receive alerts at spaced intervals—once every hour, for instance. If you don't set this, you receive alerts every time your selected error occurs. You must enter the Minimum Alert Interval in minutes.
7. Once you've entered your alert preferences, click **Create**.
8. [Set up alerts for each sync](/syncs/alerting#sync-alert-setup) that you want to send alerts through PagerDuty on.
Now when a sync attempt fails, Hightouch sends an alert to your PagerDuty service.
---
## Trigger syncs with Prefect
**URL:** https://hightouch.com/docs/extensions/prefect
**Description:** Learn how to trigger syncs with Prefect's Hightouch Tasks.
**Section:** Extensions
## Overview
Prefect provides native **tasks** to interact with Hightouch. Check out the [Git repository](https://github.com/PrefectHQ/prefect-hightouch) for examples and [Prefect's API reference](https://docs-v1.prefect.io/api/latest/tasks/hightouch.html#hightouchrunsync) for usage details.
## Setup
To use Prefect's tasks you need:
- A Hightouch API key
- The sync IDs of the syncs you want to trigger using Prefect
### Create an API key
The first step is to create a Hightouch API key in your Hightouch workspace settings.
### Set sync schedule type
## Usage
Refer to Prefect's [Hightouch Git repository](https://github.com/PrefectHQ/prefect-hightouch) for the most up-to-date examples and instructions.
---
## Create models using Sigma
**URL:** https://hightouch.com/docs/extensions/sigma
**Description:** Connect Hightouch to your Sigma instance and build Hightouch models using existing workbooks.
**Section:** Extensions
If you've already invested in modeling business data in [Sigma](https://www.sigmacomputing.com/), you may want to use your Sigma workbooks as the basis for [Hightouch models](/getting-started/concepts#models).
When you import a model from Sigma, Hightouch converts your workbook element into a SQL query and runs it directly against your data warehouse. As you publish changes in Sigma, your Hightouch model updates to reflect the latest version of your Sigma workbook.
This extension uses Sigma as a *modeling method*, not a *data source*. In
other words, Sigma provides the SQL query, not the query results. Before
enabling this extension, you must first [create a
source](/syncs/create-your-first-sync#connect-a-source) in Hightouch that
connects to the same data warehouse used by Sigma.
## Supported sources
You can create models based on Sigma workbooks for the following sources:
- [Amazon Redshift](/sources/amazon-redshift)
- [Databricks](/sources/databricks)
- [Greenplum Database](/sources/greenplum)
- [Google BigQuery](/sources/google-bigquery)
- [PostgreSQL](/sources/postgresql)
- [Snowflake](/sources/snowflake)
## Connect to Sigma
In Hightouch, go to the [**Sigma configuration page**](https://app.hightouch.com/extensions/sigma/configuration) in **Extensions**.
Enter the following fields:
- **Sigma cloud** - Follow [these instructions](https://help.sigmacomputing.com/hc/en-us/articles/360037430573-Technical-Specifications#h_01EJF3BT46HVZ8YMNFQ149SD0A) to find your Sigma cloud.
- **Client ID** - Follow [these instructions](https://help.sigmacomputing.com/hc/en-us/articles/4408555307027-Get-an-API-Token-and-Client-ID) to find your client ID.
- **Client secret** - Follow [these instructions](https://help.sigmacomputing.com/hc/en-us/articles/4408555307027-Get-an-API-Token-and-Client-ID) to create a secret tied to your client ID. Make sure to create an **API token** rather than **Embed secret**.
## Build models
After configuring your Sigma credentials, you can create a model using a [source that supports Sigma](#supported-sources). Select your source and then choose **Import Model from Sigma** as your modeling method.
On the next screen, you can select which Sigma **Workbook**, **Page**, and **Element** to build your Hightouch model off of. Click **View SQL** to see the query that Hightouch executes on your data warehouse. Make sure to click **Preview** to preview your results before finishing your model setup.
## Tips and troubleshooting
### Worksbooks aren't visible in Hightouch
Hightouch only displays [published](https://help.sigmacomputing.com/hc/en-us/articles/1500010372322-Workbook-Lifecycle-Explore-Draft-and-Publish#h_01F6JP3G04NC4APKVE1DQ7ESH8) Sigma workbooks, to which the credentials [you entered into Hightouch](#connect-to-sigma) have access.
If you don't see all the workbooks you expect in Hightouch, make sure:
- The workbook is published (not in draft mode).
- The user whose credentials you used to [connect to Sigma](#connect-to-sigma) has at least **view** access to the workbook; if they don't, share it with them and give them at least view access.
---
## Snowflake Intelligence
**URL:** https://hightouch.com/docs/extensions/snowflake-intelligence
**Description:** Use Snowflake Intelligence to understand your Hightouch models
**Section:** Extensions
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Data teams, marketers, and admins connecting Snowflake Intelligence with Hightouch |
| **Prerequisites** |
A Snowflake account with required privileges (e.g., ACCOUNTADMIN or equivalent role to create Network Rules, Secrets, and Integrations).
A Hightouch API key.
At least one model configured in Hightouch.
|
*Use Snowflake Intelligence to explore your Hightouch models and audiences using natural-language questions powered by your warehouse.*
---
## Overview
[**Snowflake Intelligence**](https://www.snowflake.com/en/developers/guides/getting-started-with-snowflake-intelligence/) is Snowflake’s Enterprise Intelligence Agent, conversational AI assistant that helps you explore warehouse data and connected systems using natural language.
By integrating Snowflake Intelligence with Hightouch, you can ask questions about your Customer Studio models, customer segments, activation campaigns, and audience performance directly from Snowflake.
This guide explains how to let Snowflake Intelligence fetch **Hightouch model metadata** (such as model name, description, and query type) using the Hightouch [**Models API**](https://hightouch.com/docs/api-reference#operation/ListModel).
---
## Learning objectives
After reading this article, you’ll know how to:
- Understand how Snowflake Intelligence uses Hightouch model metadata
- Configure Snowflake to call the Hightouch Models API
- Test the connection from Snowflake
- Ask questions in Snowflake Intelligence about your Hightouch models and audiences
---
## How Snowflake Intelligence works with Hightouch
Snowflake Intelligence can call external APIs using Snowflake’s **External Access Integrations (EAIs)**.
When you authorize Snowflake to call the Hightouch Models API, Snowflake Intelligence can:
- Retrieve a list of your Hightouch models
- Identify which models power visual audiences
- Read model descriptions and query types
- Reference this metadata when answering user questions
Hightouch continues to remain the system of record for model definitions—Snowflake only receives the metadata needed to answer questions.
Snowflake Intelligence does *not* modify, create, or run Hightouch models.
It simply enriches your warehouse-level conversations with Hightouch model context.
---
## Example questions you can ask
Once connected, Snowflake Intelligence can answer questions such as:
- "What audiences have we built that target churned customers?"
- "Which segments am I sending to my email platform?"
- “Which of my audiences was updated most recently?”
---
## Example setup: Calling the Hightouch Models API from Snowflake Intelligence
This example shows how to configure Snowflake Intelligence to call the Hightouch Models API.
You can extend this pattern to connect Snowflake Intelligence with other Hightouch APIs like our Destinations API for activation insights or Events API for real-time customer behavior, creating a comprehensive conversational interface.
### 1. Use a Snowflake role with integration privileges
```sql
USE ROLE ACCOUNTADMIN;
USE DATABASE DEMO_DB;
USE SCHEMA PUBLIC;
```
---
### 2. Create a network rule
Allows Snowflake to connect to the Hightouch API.
```sql
CREATE OR REPLACE NETWORK RULE hightouch_rule
MODE = egress
TYPE = host_port
VALUE_LIST = ('api.hightouch.com:443');
```
---
### 3. Create a secret with your Hightouch API key
```sql
CREATE OR REPLACE SECRET hightouch_api_key
TYPE = generic_string
SECRET_STRING = '';
```
**Note:** Replace `` with your actual Hightouch API key. Create an API key in your [workspace settings →](https://app.hightouch.com/settings/api-keys).
---
### 4. Create the External Access Integration
```sql
CREATE OR REPLACE EXTERNAL ACCESS INTEGRATION hightouch_eai
ALLOWED_NETWORK_RULES = (hightouch_rule)
ALLOWED_AUTHENTICATION_SECRETS = (hightouch_api_key)
ENABLED = TRUE;
```
---
### 5. Create a stored procedure to call the Hightouch Models API
```sql
CREATE OR REPLACE PROCEDURE CALL_HIGHTOUCH_API()
RETURNS STRING
LANGUAGE PYTHON
RUNTIME_VERSION = '3.10'
PACKAGES = ('requests','snowflake-snowpark-python')
EXTERNAL_ACCESS_INTEGRATIONS = (HIGHTOUCH_EAI)
SECRETS = ('cred' = HIGHTOUCH_API_KEY)
HANDLER = 'main'
AS
$$
def main():
"""
Calls the Hightouch models endpoint
"""
api_key = _snowflake.get_generic_secret_string('cred')
url = "https://api.hightouch.com/api/v1/models"
headers = {
"Authorization": f"Bearer {api_key}",
}
try:
resp = requests.get(url, headers=headers, timeout=30)
resp.raise_for_status()
return json.dumps(resp.json())
except Exception as e:
return f"Error: {str(e)}"
$$;
```
---
### 6. Test your connection
```sql
CALL CALL_HIGHTOUCH_API();
```
You should receive a JSON list of your Hightouch models.
If you receive an empty response, verify that the API key’s user has access to Models in Hightouch.
---
## Enable Snowflake Intelligence to use the procedure
Create a Snowflake Intelligence agent that can call the stored procedure.
```sql
CREATE OR REPLACE AGENT hightouch_agent
WITH PROFILE='{ "display_name": "Hightouch Agent" }'
COMMENT=$$ Enables users to interact with the Hightouch API $$
FROM SPECIFICATION
$$
{
"models": { "orchestration": "auto" },
"orchestration": {},
"instructions": {
"orchestration": "This is a simple agent for general knowledge with access to the Hightouch API"
},
"tools": [
{
"tool_spec": {
"type": "generic",
"name": "hightouch_models",
"description": "{\n \"name\": \"hightouch_models\",\n \"description\": \"Fetch Hightouch models. If the user asks about audiences, they are referring to models with query_type = 'visual'\",\n \"parameters\": { \"type\": \"object\", \"properties\": {}, \"required\": [] },\n \"example_payload\": {},\n \"stored_proc_call\": \"CALL CALL_HIGHTOUCH_API();\"\n }",
"input_schema": { "type": "object", "properties": {}, "required": [] }
}
}
],
"tool_resources": {
"hightouch_models": {
"execution_environment": { "type": "warehouse", "warehouse": "" },
"identifier": "DEMO_DB.PUBLIC.CALL_HIGHTOUCH_API",
"name": "CALL_HIGHTOUCH_API",
"type": "procedure"
}
}
}
$$;
```
---
## Troubleshooting
| Issue | Likely cause | Resolution |
| ----------------------------------------- | -------------------------------------------------------- | -------------------------------------------------------- |
| Empty API response | API key lacks required permissions | Use an admin-level API key or adjust permissions |
| Network rule error | Incorrect host or missing port | Ensure `api.hightouch.com:443` is configured |
| “EAI not found” | Integration wasn’t created in the active database/schema | Recreate using the same database/schema as the procedure |
| Snowflake agent cannot call the procedure | Missing tool spec or stored procedure reference | Recreate the agent using the example spec |
---
## What’s next
- [Explore your Hightouch models →](/models/overview)
- [Learn more about the Hightouch Models API →](/api-reference#operation/ListModel)
- [Review Snowflake’s guide to Snowflake Intelligence →](https://www.snowflake.com/en/developers/guides/getting-started-with-snowflake-intelligence/)
---
## AI and agent access
**URL:** https://hightouch.com/docs/developer-tools/agent-resources
**Description:** Give AI tools access to Hightouch documentation for more accurate answers and integrations.
**Section:** Developer tools
## Overview
AI tools can use Hightouch documentation to answer questions about the platform and help you build integrations. We publish our docs in formats that tools like Claude, ChatGPT, Cursor, and GitHub Copilot can read directly, so they have accurate and current information to work with.
## Documentation for AI tools
We publish plain-text versions of our docs that AI tools can read without parsing web page markup. Both files are regenerated on every deployment, so they always reflect the latest content.
[**llms.txt**](https://hightouch.com/docs/llms.txt) is an index of every public page, organized by product area. Use this when you want an AI tool to know what documentation is available.
[**llms-full.txt**](https://hightouch.com/docs/llms-full.txt) contains the full text of every public page in a single file. Use this when you want an AI tool to have complete context about Hightouch.
To use either file, paste the URL into your AI tool:
```
https://hightouch.com/docs/llms.txt
https://hightouch.com/docs/llms-full.txt
```
## Skills
[Agent skills](https://agentskills.io/) are instructions that help AI tools produce more accurate results for a specific product. Hightouch publishes a [skill file](https://hightouch.com/docs/skill.md) covering the REST API, CLI, Events SDK, sync configuration, and core platform concepts. When an AI tool has access to this file, it can work with the right endpoints, authentication patterns, and parameters for Hightouch.
## Additional resources
- [API overview](/developer-tools/api-guide) — REST API authentication and usage
- [API reference](/api-reference) — Full API endpoint documentation
- [CLI overview](/developer-tools/cli-guide) — Command line interface for Hightouch
- [Events SDK](/events/overview) — Track user events for real-time activation
---
## API overview
**URL:** https://hightouch.com/docs/developer-tools/api-guide
**Description:** Hightouch exposes a REST style API that allows users to interact with resources like syncs, models, sources and destinations.
**Section:** Developer tools
## Overview
Hightouch exposes a REST API that lets users interact with resources like syncs, models, sources, and destinations.
Though the REST API is open to all Hightouch users, we aim to build features so you don't need to rely on it directly. For example, [Git Sync](/extensions/git-sync) lets you manage workspace resources programmatically.
If you're looking to create a custom destination, it's best to use the [HTTP Request destination](/destinations/http-request) as your base. If you find you still need to use the REST API directly, about your use case.
To start using the API, you must first create and retrieve an API key from your workspace.
## Required permissions
To use Hightouch programmatically through the API, you must use an API key created by an Admin user of your Hightouch workspace. See [Role Based Access](https://hightouch.com/docs/workspace-management/rbac#default-roles) for more information.
## Create an API key
Log in to Hightouch as an Admin user of your Hightouch workspace, then follow these steps:
Once you have an API key, you can follow the [API Reference documentation](/api-reference) to start using the API.
## Revoke an API key
Your API key provides read/write access to sensitive Hightouch resources and should be kept secure. If you believe your API key may have been compromised, revoke it immediately.
Log in to Hightouch as an Admin user of your Hightouch workspace, then follow these steps:
1. From the [**API keys** tab](https://app.hightouch.com/settings/api-keys) on the **Settings** page, select the API key or keys you want to revoke.
2. Select **Delete selected**.
3. In the modal that appears, select **Delete**.
## Hightouch API Rate Limits
The Hightouch API rate limit is 200 requests per 10 seconds per workspace.
## Tips and troubleshooting
If you encounter issues using the Hightouch API, check our [status page](https://status.hightouch.io/) or for assistance.
### Common Errors
The following error indicates an issue with your API key. Check that the creator of the API key is an Admin user of your Hightouch workspace.
```json
{
"message": "Authentication error",
"details": "No user found"
}
```
---
## CLI overview
**URL:** https://hightouch.com/docs/developer-tools/cli-guide
**Description:** Hightouch provides an intuitive Command Line Interface (CLI) to give users access to resources in the terminal or automation scripts.
**Section:** Developer tools
## Overview
Hightouch provides an intuitive Command Line Interface (CLI) to give users access to resources in the terminal or automation scripts.
You can use the CLI to access the definition of your resources like syncs and models or trigger a sync run from your terminal or automation scripts.
## Quick start
### Install CLI
To install from shell scripts, run:
```shell
# Install the latest version
$ curl -sLf https://getcli.hightouch.io | sh -
# Install specific version
$ curl -sLf https://getcli.hightouch.io | INSTALL_HT_VERSION=v0.1.0 sh -
```
### Login with API key
Next, you need to create a Hightouch API key in your Hightouch workspace settings.
Then you can login with this API key:
```shell
$ ht login
Hightouch API key [************************************]:
```
You can also use your API key with ENV variables.
```shell
HIGHTOUCH_APIKEY=${yourkey} ht COMMANDS
```
## CLI reference
Once you've installed and logged in, you can follow the [CLI Reference documentation](/developer-tools/cli-reference) to start using the CLI.
## Issues
If you have any problems using CLI, please file an issue through [GitHub](https://github.com/hightouchio/cli/issues), or contact us through Slack or.
---
## CLI Reference
**URL:** https://hightouch.com/docs/developer-tools/cli-reference
**Section:** Developer tools
- [`ht destinations`](#ht-destinations)
- [`ht inspect RESOURCE`](#ht-inspect-resource)
- [`ht login`](#ht-login)
- [`ht models`](#ht-models)
- [`ht runs`](#ht-runs)
- [`ht sources`](#ht-sources)
- [`ht syncs`](#ht-syncs)
- [`ht trigger SYNC`](#ht-trigger-sync)
## `ht destinations`
List all destinations in current workspace
```
USAGE
$ ht destinations
DESCRIPTION
List all destinations in current workspace
ALIASES
$ ht destination
EXAMPLES
$ ht destinations
```
_See code: [src/commands/destinations.ts](https://github.com/hightouchio/cli/blob/v0.1.0/src/commands/destinations.ts)_
## `ht inspect RESOURCE`
Inspect resources in the current workspace
```
USAGE
$ ht inspect [RESOURCE] -f
ARGUMENTS
RESOURCE Resource ID
FLAGS
-f, --format= (required) [default: json] Output format(yaml/json)
DESCRIPTION
Inspect resources in the current workspace
EXAMPLES
$ ht inspect model/run-user
```
_See code: [src/commands/inspect.ts](https://github.com/hightouchio/cli/blob/v0.1.0/src/commands/inspect.ts)_
## `ht login`
Login Hightouch with API key
```
USAGE
$ ht login
DESCRIPTION
Login Hightouch with API key
EXAMPLES
$ ht login
```
_See code: [src/commands/login.ts](https://github.com/hightouchio/cli/blob/v0.1.0/src/commands/login.ts)_
## `ht models`
List all models in current workspace
```
USAGE
$ ht models
DESCRIPTION
List all models in current workspace
ALIASES
$ ht model
EXAMPLES
$ ht models
```
_See code: [src/commands/models.ts](https://github.com/hightouchio/cli/blob/v0.1.0/src/commands/models.ts)_
## `ht runs`
List all syncs run for a given sync
```
USAGE
$ ht runs -s [-l ]
FLAGS
-l, --limit= [default: 5] Limit the number of runs to output
-s, --sync= (required) Specify sync name to list runs
DESCRIPTION
List all syncs run for a given sync
ALIASES
$ ht run
EXAMPLES
$ ht runs
```
_See code: [src/commands/runs.ts](https://github.com/hightouchio/cli/blob/v0.1.0/src/commands/runs.ts)_
## `ht sources`
List all sources in current workspace
```
USAGE
$ ht sources
DESCRIPTION
List all sources in current workspace
ALIASES
$ ht source
EXAMPLES
$ ht sources
```
_See code: [src/commands/sources.ts](https://github.com/hightouchio/cli/blob/v0.1.0/src/commands/sources.ts)_
## `ht syncs`
List all syncs in current workspace
```
USAGE
$ ht syncs
DESCRIPTION
List all syncs in current workspace
ALIASES
$ ht sync
EXAMPLES
$ ht syncs
```
_See code: [src/commands/syncs.ts](https://github.com/hightouchio/cli/blob/v0.1.0/src/commands/syncs.ts)_
## `ht trigger SYNC`
Trigger sync to run
```
USAGE
$ ht trigger [SYNC] [-f] [-q]
ARGUMENTS
SYNC Sync ID
FLAGS
-f, --full-sync Whether to run a full sync
-q, --quiet Whether to show run progress
DESCRIPTION
Trigger sync to run
EXAMPLES
$ ht trigger --sync test123
```
_See code: [src/commands/trigger.ts](https://github.com/hightouchio/cli/blob/v0.1.0/src/commands/trigger.ts)_
---
## Warehouse tables reference
**URL:** https://hightouch.com/docs/developer-tools/warehouse-tables
**Description:** A reference for all the tables and schemas Hightouch creates in your data warehouse, with usage guidance and stability expectations.
**Section:** Developer tools
Hightouch creates and manages tables in your data warehouse to power change data capture, sync logging, audience snapshots, journeys, identity resolution, and more. This page is a single reference for all of these tables: which schema they belong to, which feature creates them, and how you should (and shouldn't) use them.
## Usage guidance
Hightouch writes these tables to your warehouse for observability, debugging, and lightweight reporting. Before building on top of them, keep the following in mind.
### Recommended uses
- **Ad-hoc debugging and auditing.** Query sync logs to investigate failed rows, inspect audience membership changes, or trace a user's journey path.
- **Lightweight reporting.** Build dashboards on top of sync run metadata, audience snapshots, or identity resolution outputs to monitor Hightouch activity.
- **Analytics and activation with Identity Resolution outputs.** The `_resolved`, `_resolved_identifiers`, and `_golden_records` tables are explicitly designed for downstream use. You can query them directly, register them as dbt sources, or sync from them.
### Not recommended
- **Wiring tables into production workflows.** Don't build business-critical pipelines that depend on these tables without . Table schemas may change as Hightouch evolves, and we don't guarantee schema stability for all tables.
- **Writing to Hightouch-managed tables.** Never insert, update, or delete rows in the `hightouch_planner` schema (especially `JOURNEY_LOG` tables and `_plan`/`_rejections` tables). These tables are actively used during sync and journey execution — modifying them can cause unexpected behavior.
- **Depending on internal state tables.** Tables prefixed with `IDR_` or `IDR_BACKUP_` are internal to Identity Resolution. Don't query or reference them in pipelines — they can change without notice.
Documenting a table here doesn't mean Hightouch is committing to a stable schema contract. If you want to build durable integrations on top of these tables, so we can help you find the right approach.
---
## Schemas overview
Hightouch uses up to three schemas in your warehouse, depending on which features you enable:
| Schema | Purpose | Created when |
| --- | --- | --- |
| `hightouch_planner` | Change data capture, audience snapshots, journey logs, Identity Resolution outputs, AI Decisioning tables | [Lightning sync engine](/syncs/lightning-sync-engine) is enabled |
| `hightouch_audit` | Warehouse Sync Logs, holdout group logs, journey views, data extraction reports | Lightning sync engine is enabled; individual features populate specific tables |
| Custom (per Events source) | Event data ([Hightouch Events](/events/warehouses/schema)); schema name is configurable | An Events source writes to a warehouse destination |
---
## hightouch_planner schema
### Lightning sync engine (CDC)
These tables are created automatically when the [Lightning sync engine](/syncs/lightning-sync-engine) is enabled. They're **required** for syncs using the Lightning engine.
| Table | Key columns | Description |
| --- | --- | --- |
| `*_plan` | — | Stores a model's query results for a sync run, used to compute row-level diffs. Table names change with every run. |
| `*_rejections` | — | Stores rows rejected during a sync run. Table names change with every run. |
Hightouch only keeps the two most recent pairs of `_plan` and `_rejections` tables per sync. Because table names change with every run, Hightouch requires write access to the entire schema.
Don't delete tables from `hightouch_planner`. Removing CDC tables breaks change data capture and requires a [full resync](/syncs/overview#resync-full-query) to recover.
For full details, see [Lightning sync engine — Warehouse schemas](/syncs/lightning-sync-engine#warehouse-schemas).
### Audience snapshots
**Optional** — must be enabled on the source's **Sync Logs** tab. See [Audience snapshots](/customer-studio/audience-snapshots).
| Table | Key columns | Description |
| --- | --- | --- |
| `audience_membership` | `ht_row_id`, `ht_audience_id`, `ht_timestamp`, `ht_event_type`, `ht_split_group` | Tracks audience membership changes over time. Each row records an `enter` or `exit` event for a member. |
### Journey tables
Created **automatically** when a journey is active. See [Journeys — Journey Logs](/customer-studio/journeys#journey-logs).
| Table | Key columns | Description |
| --- | --- | --- |
| `JOURNEY_LOG_` | `row_id`, `row_instance_id`, `from_node_id`, `to_node_id`, `event_type`, `timestamp` | Logs all row movement through the journey: entries, transitions, and exits. |
| `JOURNEY_METADATA_` | `journey_id`, `journey_name`, `node_name`, `node_type`, `node_id` | The journey graph structure and customer-defined node names. Updated on save. |
| `JOURNEY_CONTEXT_LOG_` | `row_id`, `row_instance_id`, `node_id`, `entered_at`, `[variable_name]` | Results of journey context [variables](/customer-studio/journeys#variables) for each row. |
Don't write to `JOURNEY_LOG` tables. They're actively used during journey execution — inserting, updating, or deleting rows can cause the journey to behave unexpectedly.
The `journey_id` in table names is the journey's UUID with dashes removed.
### Identity Resolution outputs
Created **automatically** when an identity graph runs. See [Lookup table usage](/identity-resolution/usage).
| Table | Key columns | Description |
| --- | --- | --- |
| `_resolved` | `ht_id`, `source`, `primary_key`, `latest_timestamp` | Maps every input row to a resolved identity. |
| `_resolved_identifiers` | `ht_id`, `identifier`, `value`, `first_timestamp`, `last_timestamp`, `count` | All identifier values associated with each resolved identity. |
| `_unresolved` | — | Input rows that couldn't be processed, typically due to duplicate primary keys. |
| `_golden_records` | `ht_id`, plus one column per configured field | One row per identity with canonical field values. Only created when [Golden Record](/identity-resolution/golden-record) is enabled. |
These four output tables **are designed for your use**. You can safely query them, reference them in dbt, or build views on top of them. For long-term stability, we recommend creating a view or derived table rather than transforming them in place.
### Identity Resolution internal state
| Table | Description |
| --- | --- |
| `IDR_*` / `IDR_BACKUP_*` | Internal tables for incremental processing and graph consistency. |
Don't query or modify these tables. They're internal implementation details and can change without notice.
### AI Decisioning
Created **automatically** per agent when using Hightouch-assigned groups. See [AI Decisioning — Group assignment](/ai-decisioning/group-assignment#experiment-groups-table-in-the-warehouse).
| Table | Key columns | Description |
| --- | --- | --- |
| `de_user_experiment_groups_` | `user_id`, `user_hash`, `experiment`, `first_seen_at`, `last_seen_at` | Tracks which experiment bucket (`treatment`, `holdout`, `customer_managed`) each user belongs to over time. |
---
## hightouch_audit schema
### Warehouse Sync Logs
**Optional** — must be enabled per source or per sync. See [Warehouse Sync Logs](/syncs/warehouse-sync-logs).
| Table | Key columns | Description |
| --- | --- | --- |
| `sync_changelog` | `sync_id`, `row_id`, `op_type`, `status`, `failure_reason`, `fields` | A log of every operation across all sync runs. Each synced row gets one entry per run. |
| `sync_snapshot` | `sync_id`, `row_id`, `op_type`, `status`, `failure_reason`, `fields` | The latest status of each row across all syncs. Replaced after each run. |
| `sync_runs` | `sync_id`, `sync_run_id`, `model_name`, `started_at`, `finished_at`, `status` | Metadata for each sync run, including row counts for planned, attempted, succeeded, and failed operations. |
### Holdout group logs
**Optional** — requires a feature flag and must be enabled on the source's **Sync Logs** tab. See [Experiments — Holdout group logs](/customer-studio/experiments#holdout-group-logs).
| Table | Key columns | Description |
| --- | --- | --- |
| `audience_holdout` | `sync_id`, `sync_run_id`, `model_id`, `row_id`, `fields`, `split_group` | Logs rows excluded from a sync due to holdout group membership, for post-campaign analysis. |
### Journey views
Created **automatically** when journeys are active. These are read-only views that combine all per-journey tables into a single workspace-level view.
| View | Description |
| --- | --- |
| `JOURNEY_LOG_VIEW_` | Combines all `JOURNEY_LOG` tables. Includes a `source_table` column to identify the originating journey. |
| `JOURNEY_METADATA_VIEW_` | Combines all `JOURNEY_METADATA` tables. |
Find your workspace slug under **Settings > Workspace > Workspace slug**. Replace dashes with underscores in the slug when querying (for example, `my-workspace` becomes `my_workspace`).
### Data extraction
Created **automatically** when data extraction is enabled for a destination. See [Data extraction](/syncs/data-extraction).
| Table | Key columns | Description |
| --- | --- | --- |
| `{destination-type}_{destination-id}_audiences` | Varies by ad platform | One table per enabled destination. Schema mirrors the ad platform's API response. |
| `external_audiences_metadata_v1` (view) | `ad_account_id`, `audience_id`, `audience_name`, `audience_size`, `ht_fetched_at` | A combined, standardized view across all destination tables. |
---
## Events schema
[Hightouch Events](/events/warehouses/schema) writes event data into a **configurable schema** — not `hightouch_planner` or `hightouch_audit`. The schema name defaults to an auto-generated value based on the event source but can be configured in destination settings. Tables are created **automatically** when an Events source writes to a warehouse destination.
| Table | Key columns | Description |
| --- | --- | --- |
| `identifies` | `id`, `anonymous_id`, `user_id`, ``, `timestamp` | All `identify` events. Trait keys become separate columns. |
| `tracks` | `id`, `anonymous_id`, `user_id`, `event`, `event_text`, `timestamp` | All `track` events. |
| Per-event tables | Varies | A separate table for each track event type, with event properties as columns. Can be disabled. |
| `pages` | `id`, `anonymous_id`, `user_id`, ``, `timestamp` | All `page` events. |
| `screens` | `id`, `anonymous_id`, `user_id`, ``, `timestamp` | All `screen` events. |
| `groups` | `id`, `anonymous_id`, `user_id`, `group_id`, ``, `timestamp` | All `group` events. |
For the full column-level schema, see [Events warehouse schema](/events/warehouses/schema).
---
## Approval flows
**URL:** https://hightouch.com/docs/workspace-management/approval-flows
**Description:** Require approval before changes to models and syncs are published.
**Section:** Workspace management
Approval flows let you stage new resources ([models](/getting-started/concepts#models) and [syncs](/getting-started/concepts#syncs)) and changes to existing resources in **draft mode** until an approver reviews them.
Approval flows apply only to **models** and **syncs**. Other resources, such as [Audiences](/customer-studio/audiences) and [Journeys](/customer-studio/journeys), are published immediately.
## Getting started
To enable approval flows:
1. Navigate to [**Settings**](https://app.hightouch.com/settings) in the right sidebar.
2. Open the **Workspace** tab.
3. Enable **Require approvals**.
4. Select **Save**.
After you enable approval flows, any role with the `Draft models & syncs` permission can save changes as drafts or submit them for approval — not just the built-in **Workspace draft editor** role.
Whether a user must get approval depends on their role:
- Users with draft-only roles must submit changes for approval before publishing.
- Users with publishing permissions can submit changes for approval or use **Publish now** to publish directly.
- Users with the `Approve models & syncs` permission can review, approve, decline, or withdraw approval requests.
### Require approval for a user
To require a user's model and sync changes to go through approval, assign them a role that enforces drafts.
#### Use the Workspace draft editor role
1. Go to **Settings > Members**.
2. Find the user.
3. Change their role to **Workspace draft editor**.
Users with this role can create and edit models and syncs, but their changes must be approved before they are published.
#### Create a custom draft role
1. Go to **Settings > Roles**.
2. [Create a custom role](/workspace-management/rbac#custom-roles).
3. Open the **Sources** or **Destinations** tab.
4. Select `Draft models & syncs` for the relevant resources.
Users with this permission can draft changes but do not see the **Publish now** option.
### Who can publish directly
Some roles can bypass approval and publish directly. For example, users with the **Admin** or **Workspace editor** role can save drafts, submit approval requests, or select **Publish now**.
Any role without the `Draft models & syncs` restriction can self-publish.
The **Workspace draft editor** role is the built-in role whose changes must be approved. It is not the only role that can create drafts. Any role with the `Draft models & syncs` permission can create and edit drafts. Roles that can publish directly can also save drafts voluntarily but still have access to **Publish now**.
If you use approval flows, we recommend the following role structure:
- **Admin** — Can approve changes and manage the workspace.
- **Workspace editor** — Can approve changes and publish directly.
- **Workspace draft editor** — Can create and edit models and syncs, but changes require approval before publishing.
For more advanced setups, [create custom roles](/workspace-management/rbac#custom-roles) that include or exclude the `Approve models & syncs` permission on model and sync resources.
## Create new resources
When you create a new model or sync, the available actions depend on your role:
- **Publish now** publishes the resource immediately. Only available to roles that can bypass approval.
- **Submit approval request** sends the draft to one or more approvers for review. You can include a comment.
- **Save as draft** saves the resource without notifying anyone. You can keep editing and submit it for approval later.
If your role requires approval, you do not see **Publish now**. You can only submit the resource for approval or save it as a draft.
If a sync uses a model that is still in draft mode, both the model and the sync must be published before the sync can run.
## Edit existing resources
Editing existing models and syncs uses the same approval options as [creating new resources](#create-new-resources).
The following changes require approval:
| Resource | Changes requiring approval |
| -------- | --------------------------------------------- |
| Model | Query, configuration, and column type changes |
| Sync | Configuration and schedule changes |
If a sync uses a model with unpublished changes, the sync refers to the draft version of the model instead of the published version. Both the model and the sync must be published before the sync can run.
## Approve requests
[Approvers](#who-can-approve-requests) can view approval requests by selecting **View request** on the corresponding resource.
When reviewing a request, approvers can choose one of the following actions:
- **Approve & publish** publishes the new resource or drafted changes.
- **Delete draft** deletes the draft. New resources or drafted changes are discarded and cannot be recovered.
- **Withdraw request** removes the approval request but keeps the draft. Users can continue editing and submit it again later.
Any user with approval permission and access to the resource can approve the request, even if they were not selected as the reviewer.
For sync changes, the approval request shows a comparison of changes to the sync configuration. For model changes, the approval request shows changes to the model query.
### Who can approve requests
The `Approve models & syncs` [permission](/workspace-management/rbac#actions) controls who can approve requests. By default, all roles [except **Workspace draft editor** and **Workspace viewer**](/workspace-management/rbac#using-roles-with-approval-flows) include this permission:
- **Admin**
- **Workspace editor**
- **Model + sync editor**
- **Sync editor**
- **Source admin**
- **Destination admin**
- **Audience editor**
To grant or remove approval permissions for a [custom role](/workspace-management/rbac#custom-roles), add or remove the `Approve models & syncs` permission for the relevant sources or destinations. Users with this permission can use **Approve & publish**, **Delete draft**, and **Withdraw request** when reviewing a draft.
---
## Audit logs
**URL:** https://hightouch.com/docs/workspace-management/audit-logs
**Description:** Hightouch offers audit logs so workspace admins can see what actions users took in-app and when.
**Section:** Workspace management
Hightouch offers audit logs so workspace admins can see what actions users took in-app and when. For example, suppose someone notices that a sync has sent incorrect data to a destination. A workspace admin can use audit logs to review changes made to the sync before the incorrect data appeared. The admin can correct the issue, and—if necessary—implement [controls](/workspace-management/overview#access-management) to avoid future incidents.
## View audit logs
Workspace admins and those [with the appropriate access](#who-can-view-audit-logs) can view logs by going to the [**Audit log** tab](https://app.hightouch.com/settings/audit-log) on the **Settings** page. The logs consist of a table of actions beginning with the most recent. Each row represents one action.
Each action includes:
- The **Date** and time, in the viewer's timezone, the action took place
- The **Action** name, for example, "Sync updated"
- The **User** who performed the action
- The **Resource type** of the resource the action was performed on
- The **Resource name** of the resource the action was performed on
Clicking on an action displays the action's full details in JSON format in the right panel. Added lines are highlighted in green, while removed lines are highlighted in red.
## Filter audit logs
You can filter logs by:
- the user who performed the action
- the resource type the action was performed on
- the resource name the action was performed on
- a range of dates
You can select multiple users and resource types to filter by simultaneously.
## FAQ
### Who can view audit logs?
Anyone with [`read`](/workspace-management/rbac#actions)-access to the [`workspace`](/workspace-management/rbac#resources) resource can view audit logs.
[Out-of-the-box roles](/workspace-management/rbac#default-roles) with these permissions include Admins, Destination Admins, and Source Admins. If you want to create a [custom role](/workspace-management/rbac#custom-roles) with the ability to view audit logs, it should include this policy:
```JSON
{
"effect": "allow",
"actions": "read",
"resource": [
"workspace"
]
}
```
### How far back do audit logs go?
Audit logs track up to 90 days of activity.
### What do audit logs include?
Audit logs track user-driven activity vs. programmatic changes. They work by taking a JSON snapshot of the changed resource for every user-driven change. Hightouch uses these snapshots for a before and after comparison for any changes.
A snapshot may include programmatic fields such as `last_run_at`, but Hightouch doesn't append a new audit log every time a field like `last_run_at` is updated since sync runs can be configured to run at regular intervals.
The exception to this is activity driven by [Git Sync](/extensions/git-sync). For example, if a user changes a configuration via [Git Sync](/extensions/git-sync), Hightouch treats that as a user-driven change, and it appears in the audit logs.
For a complete list of tracked actions, see the following [list](#resource-types-and-tracked-actions).
## Resource types and tracked actions
Audit logs show creation, update, and deletion type actions for the following resource types:
| Resource type | Tracked actions |
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Alert** | Creations, updates, or deletions of [alerts](/syncs/alerting) you configure to inform you of syncing issues |
| **API key** | Creations and deletions of API keys _Note: Audit logs don't include API Key usage_ |
| **Approval request** | Actions taken during [approval flows](/workspace-management/approval-flows), including sending a request to review a draft or approving a draft |
| **Audience** | Creations, updates, or deletions of [audience](/customer-studio/overview) configurations |
| **Audience schema** | Creations, updates, or deletions of [parent models, relationships, and event models](/customer-studio/schema) |
| **Cloud credential** | Creations, updates, or deletions of cloud credentials, including changing your [external storage configuration](/security/storage) |
| **Draft** | Creations, updates, or deletions of drafts used in [approval flows](/workspace-management/approval-flows) |
| **Git credential** | Creations, updates, or deletions of Git credentials used for [Git Sync](/extensions/git-sync) |
| **Git Sync configuration** | Creations, updates, or deletions of [Git Sync](/extensions/git-sync) configurations; for example, changing your tracked Git branch |
| **Invite** | Creations and deletions of workspace invitations |
| **Membership** | Creations, updates, or deletions of workspace members, including changing a user's role |
| **Membership request** | Creations and deletions of workspace membership requests |
| **Model** | Creations, updates, or deletions of general model configurations, including updating the model query or primary key |
| **Model column** | Creations, updates, or deletions of model columns |
| **Model preview** | Actions taken while previewing a model, including a user clicking “Preview” while editing a model or running a SQL query against a source during model setup |
| **Model trait definition** | Creations, updates, or deletions of audience traits |
| **Role** | Creations, updates, or deletions of [custom roles](/workspace-management/rbac#custom-roles) |
| **Session** | User sign-ins and sign-outs |
| **Source** | Creations, updates, or deletions of source configurations, including updating credentials or adding labels |
| **Sync** | Creations, updates, or deletions of sync configurations, including adding labels, scheduling, and turning a sync on or off |
| **Sync run** | Actions taken while viewing a sync; for example, a user clicking “Run” _Note: Audit logs don't include scheduled sync runs nor API-triggered sync runs_ |
---
## Environments and deployments
**URL:** https://hightouch.com/docs/workspace-management/environments
**Description:** Hightouch supports change management in-app via separate workspaces for different environments, for example, production, staging, development, and sandbox.
**Section:** Workspace management
**Not sure whether you need environments or a separate workspace?** Environments are the right choice when your teams share the same sources, destinations, and user roster but need a safe promotion path (dev → staging → prod). If you need hard isolation — separate sources, separate users, nothing shared — you need a separate workspace instead. For a side-by-side comparison, see [Do you need another workspace?](/workspace-management/workspaces#do-you-need-another-workspace)
Maintaining separate [environments](https://en.wikipedia.org/wiki/Deployment_environment) for different stages of the release process is a common practice across technical domains.
Separating environments minimizes the risk of accidental changes or disruptions to the user-facing experience.
For example, environments can help you avoid overwriting CRM data you didn't mean to, activating a marketing campaign prematurely, or incurring unexpected data warehouse charges.
Generally, production environments are for live, user-facing applications, staging for pre-release testing, and development for experimentation.
You can set up whatever environments your organization requires in Hightouch.
## How do environments work?
Each Hightouch [workspace](/workspace-management/overview#core-concepts) in your organization acts as a separate environment.
You can link resources between workspaces, which allows you to **deploy** [models and syncs](/getting-started/concepts) from one environment to another.
Each workspace can have its own set of [alerts](/syncs/alerting) and can use its own extensions.
Environments integrate seamlessly with other governance features, such as [role-based access controls](/workspace-management/rbac) (RBAC) and [approval flows](/workspace-management/approval-flows).
Workspace [permissions](/workspace-management/rbac) and [drafts](/workspace-management/approval-flows) are enforced across workspaces when deploying changes.
You can set up your organization so that specific team members have access to certain resources.
For example, some team members, such as your data team, may have permission to set up sources and models.
Others, such as the marketing team, can deploy or draft sync deployments.
### Separate resources for separate environments
Each workspace has its own set of resources, including [sources](/getting-started/concepts#sources), [models](/getting-started/concepts#models), [destinations](/getting-started/concepts#destinations), and [syncs](/getting-started/concepts#syncs).
Hightouch allows you to link resources across workspaces.
For example, you can link a Snowflake playground in your "dev" workspace to a production Snowflake instance in your "prod" workspace,
a Salesforce sandbox to your production Salesforce instance, etc.
Environments allow you to **deploy** changes to models and syncs.
When you deploy a model or sync, Hightouch copies its configuration to another workspace, updating or creating resources where necessary.
Before performing the copy, Hightouch checks that all [underlying resources](#resource-dependencies) are appropriately configured.
For example, you may be experimenting with changes to a Salesforce sync [mapping](/syncs/mapping-data) in your development environment.
When you're happy with your change, you can deploy it to your production environment.
Before copying your changes to a production sync, Hightouch checks that the underlying model and destination have linked counterparts in production.
Keeping resources separate between environments lets you create as many as you want in a development environment without cluttering other environments you may want to keep pristine.
And since changes don't automatically deploy to linked resources,
you can safely delete or update resources without worrying about knock-on effects in other environments.
Each resource also has its own history, which you can inspect, for example, by examining its activity.
### Resource dependencies
While any resources of the same type can be linked, for example, source to source and sync to sync, **you can only deploy models and syncs**.
To deploy models and syncs, you must link their underlying sources and destinations to their relevant counterparts.
Environments apply to **Reverse ETL models and syncs only**. Customer Studio resources — including audiences, traits, parent models, and schema models — **cannot be deployed between environments**. Each environment must be configured and maintained independently for Customer Studio workflows. For Customer Studio change management, use [Approval Flows](/workspace-management/approval-flows) instead.
When you [deploy](#deploy-changes) models or syncs, Hightouch checks that the underlying resources match linked counterparts in the target environment.
For example, if you link one **source** to another, you can deploy **models** built off the linked source.
To deploy **syncs**, the underlying linked **models** _and_ **destinations** need to match in both workspaces.
## Environment usage
To deploy changes, you first need to link resources.
You can either link existing sources and destinations on their configuration pages,
or link them as part of the [deployment workflow](#deploy-changes).
You only need to link underlying resources once to allow deployments.
Removing links between resources doesn't impact syncs or models you previously deployed.
### Link resources
To link resources, you need [`create`
permissions](/workspace-management/rbac#actions) for the relevant resource
type (source or destination) in all workspaces you want to create the link
between. For example, if you want to link two sources, you need [`create`
permissions](/workspace-management/rbac#actions) for sources in both
workspaces you want to link them in.
You can link existing sources and destinations to their counterparts in other workspaces.
You can only link models and syncs during the deployment workflow.
When you [deploy](#deploy-changes) a resource for the first time, you create and simultaneously link the resource to a counterpart in the target workspace.
During deployments, Hightouch ensures you've linked the necessary [underlying resources](#resource-dependencies).
If you haven't, you can link them directly from the deployment wizard.
#### Link sources and destinations
To link existing sources and destinations, follow these steps:
1. Go to the resource's overview page and open the **Linking** tab.
2. Select the relevant resource for each workspace you want to link and then click **Link**.
Hightouch only displays eligible resources for linking. To be eligible: - the
resources must be of the same type; for example, they both must be Snowflake -
the user creating the link must be a member of both workspaces - both
workspaces must belong to the same organization
#### Link syncs and models
The only way to link syncs to other syncs and models to other models is by [deploying them](#deploy-changes).
Once deployment creates a new sync or model, it's automatically linked for future deployments.
You cannot deploy to existing syncs or models. If you want to link existing syncs or models from one workspace to another,
delete the development or testing version and recreate it by deploying it from your production environment.
Don't hesitate to if you have any questions or concerns.
### Deploy changes
To deploy changes, you need [permission to at least create
drafts](/workspace-management/rbac#using-roles-with-approval-flows) for the
relevant resource type (model or sync) in the target workspace to which you
want to deploy changes. For example, if you want to deploy a sync, you need
permission to create sync drafts in the workspace you want to deploy the sync
to.
To deploy syncs or models, follow these instructions:
1. From a model or sync's configuration page, click the deploy icon to the left of the **Run** button and select **Deploy**.
2. Select the environment you want to deploy the change to. The first time you deploy a particular resource, Hightouch creates a new resource in the target environment. If you've deployed the resource before, the deployment updates the previously created and linked resource.
3. If the relevant [underlying resources](#resource-dependencies) aren't already linked, follow the steps in the modal to link them.
4. The deployment wizard shows what has changed: added, removed, and updated configurations. Review these to ensure they're the changes you want to deploy.
5. For sync deployments, you can choose whether to deploy its [schedule](/syncs/schedule-sync-ui#schedule-types) as well as the other configuration changes.
6. Click **Deploy**.
7. **Syncs created via deployment are disabled by default.** If you want to enable a deployed sync, make sure to do so from its [overview page](/syncs/overview#individual-sync-overview-page) once you've deployed it.
You can't deploy
[drafted](/workspace-management/approval-flows#editing-existing-resources)
changes on a configuration. You first need to get approval and/or publish
changes to a resource before deploying the changes.
#### Deployment validations
Before deploying changes, Hightouch validates dependencies.
Hightouch blocks deployments until you've resolved all dependency issues.
For example, if you try to deploy a [dbt-based](/extensions/dbt-models) model from your staging to production workspace,
but the [dbt extension](/extensions/dbt-models) isn't set up in production, Hightouch blocks the deployment.
Once you've resolved the issue, for example, by configuring a missing extension, you can retry the deployment.
Deployments will clear or replace any existing [drafts or approval requests](/workspace-management/approval-flows) on a sync or model in a target workspace. You will be warned if a deployment will replace a draft during the validation step.
## FAQ
### What permissions do you need to use environments?
To link resources, you need [`create` permissions](/workspace-management/rbac#actions) for the relevant resource type in both workspaces you want to create the link between.
To deploy changes, you need [permission to create or update drafts](/workspace-management/rbac#using-roles-with-approval-flows) for the relevant resource type in the target workspace you want to deploy changes to.
### Can you link any resource to another resource of the same type?
You can link sources to other sources, models to other models, destinations to other destinations, and syncs to other syncs.
**Customer Studio resources — including audiences, traits, parent models, and schema models — cannot be linked or deployed between environments.** Each environment must be set up and maintained independently for Customer Studio. For Customer Studio change management, use [Approval Flows](/workspace-management/approval-flows) instead.
Please if this is something you're interested in.
### How do you link existing models and syncs?
The only way to link syncs to other syncs and models to other models is by [deploying them](#deploy-changes).
Once deployment creates a new sync or model, it's automatically linked for future deployments.
If you want to link existing syncs or models from one workspace to another,
delete the development or testing version and recreate it by deploying it from your production environment.
Don't hesitate to if you have any questions or concerns.
### Can you use environments to copy resources?
You can use [deployments](#deploy-changes) to copy existing models and syncs but not sources and destinations.
To copy an existing model or sync to a different workspace, follow the instructions for [deploying changes](#deploy-changes).
### Can you only deploy changes to production environments?
Deploying changes has no directionality.
You can deploy changes from a testing workspace to a production workspace and vice-versa, as long as the [validations](#deployment-validations) pass.
### Can you use environments and deployments with Git Sync?
If you already use [Git Sync](/extensions/git-sync) to manage your syncs and destinations across workspaces, deploying a sync or model to your target workspace would create its corresponding `yaml` file in the configured Git Sync repository.
Beyond updating `yaml` files, environments and deployments don't interact with your Git Sync repositories.
You should use _either_ Git Sync _or_ the in-app environments and deployments feature for your environment management.
### Can you use environments and deployments with Hightouch Event Collection?
At the moment, it's not possible to create deployments of event resources like event syncs, event sources or event destinations and so it isn't currently possible to separate your Event Collection resources into staging and production environments.
---
## Folders, filters, and labels
**URL:** https://hightouch.com/docs/workspace-management/folders-filters
**Description:** Hightouch provides folders, filters, and labels to organize your workspace.
**Section:** Workspace management
Hightouch offers folders, filters, and labels to help you organize and navigate your workspace, whether you're managing a handful of syncs or thousands. These tools allow you to:
- Create dedicated spaces for specific teams
- Organize marketing campaigns
- Group resources by project, department, or any other dimension
## Folders
You can show or hide the folder sidebar by clicking the arrow next to the main app sidebar.
Folders allow you to group your resources like syncs, models, and audiences. These resources can be placed into folders, and you can nest folders without limits.
Only users with the **Workspace admin** role can modify the folder structure
inside a workspace. Other roles can only add or remove syncs from existing
folders.
### Creating folders
To create a folder, click **New folder** and enter a name for your folder.
### Editing and removing folders
To edit or remove a folder, click the three dots next to the folder name and select an action.
By default, removing a folder won’t delete its contents. The resources will simply become unfiled but still accessible. If you want to delete the folder and its resources together, you’ll need to choose the last option from the dropdown menu.
### Moving resources to folders
1. Select the resources you want to move from the syncs, models, or audiences list.
2. Click **Actions**.
3. Click **Move to folder**.
You can move the resources into an existing folder or create a new one during this process.
### Folder hierarchy
Hightouch enforces a shared folder hierarchy across resources. Models and their attached syncs share the same folder structure, as do audiences and their attached syncs.
When you move a sync, Hightouch automatically moves its associated model or audience to the same folder. This helps keep your folder structure organized and consistent across resource types.
## Filters
Filters allow you to quickly narrow down your list of resources.
Hightouch supports different filters depending on the resource type.
### Sync filters
- **Progress**: whether the sync is currently running
- **Destination**: the business tool where the data is being sent
- **Source**: the origin of the model associated with the sync
- **Label**: any custom key/value labels assigned to the sync
- **Health**: whether the sync is healthy (as determined by its alert triggers)
- **Creator**: the user who originally created the sync
- **Match Booster**: indicates whether identifiers are boosted during the sync process
### Model filters
- **Query type**: how the model is defined (e.g., SQL editor, table selector, dbt, etc.)
- **Source type**: the type of source where the data originates (e.g., Snowflake, Databricks, etc.)
- **Created by**: the user who initially created the model
- **Labels**: any custom key/value labels assigned to the model
- **Duplicate primary keys**: indicates if the model contains any duplicate primary keys
- **Match Boosted**: indicates whether the model's identifiers have been boosted
### Audience filters
- **Parent model**: the parent model from which the audience is derived
- **Source**: the source from which the audience's parent model is derived
- **Created by**: the user who initially created the audience
- **Labels**: any custom key/value labels assigned to the audience
- **Duplicate primary keys**: indicates if the parent model contains any duplicate primary keys
- **Subsets**: the subset applied to the parent model when creating the audience
### Applying filters
Multiple filters can be applied simultaneously to refine your results. Hightouch displays only resources that match all the selected filters. For example, if you apply `Health: Unhealthy` and `Destination: Salesforce`, only syncs that are unhealthy and connected to Salesforce will be shown.
If you're inside a folder, filters apply only to the resources within that folder. To filter across all resources, switch to **All syncs**, **All models**, or **All audiences**.
### Saving filters
You can save a set of filters for reuse by creating a **Saved filter**. After selecting your filters, click **Save as** to store them for future use.
Name your saved filter and select its visibility: private (visible only to you) or shared with the entire workspace.
## Labels
Labels in Hightouch are structured as key/value pairs and offer a flexible way to categorize and manage resources. Unlike folders, which group resources in a strict hierarchy, labels provide a more dynamic approach by allowing you to tag resources with multiple labels. This is especially useful when you want to organize resources across different dimensions that don't fit neatly into a folder structure.
For example, while you might use folders to organize resources by team or department, labels are ideal for tracking specific characteristics like campaign type (e.g., `sms`, `email`, `push`) or brand (e.g., `Brand A`, `Brand B`). Labels allow for more granular filtering and flexible organization across all resources, regardless of their folder placement.
### Applying labels
Select the resources you want to label by checking their boxes. Click the **Actions** button at the top of the page and choose **Add labels** from the dropdown.
In the modal, enter your key/value pairs (e.g., `brand = ACME`). Labels can contain letters, numbers, spaces, underscores, and dashes. Then, click **Save changes**.
### Filtering by label
On any top-level resource page, you can expand the **Labels** section in the filter sidebar and include or exclude specific key/value pairs.
---
## Governance best practices
**URL:** https://hightouch.com/docs/workspace-management/governance
**Description:** Learn when to use governance controls like roles, subsets, Spaces, destination rules, approval flows, and environments to manage access, collaboration, and activation safely.
**Section:** Workspace management
## Overview
**Customer Data Platforms (CDPs)** are often used by many teams across an organization, including:
- Marketers building audiences and launching campaigns
- Data teams managing models, permissions, and warehouse access
- Administrators overseeing workspace setup, security, and compliance
- Regional, brand, or business-unit teams working from shared customer data
- Legal, privacy, or operations teams reviewing how data is used
As more teams begin using the same customer data, governance becomes important.
Without clear governance controls, teams can end up with too much access, send data to the wrong destinations, override each other's work, or launch changes without enough review.
Hightouch helps you prevent that by adding governance controls at the **activation layer**.
Because Hightouch runs directly on top of your warehouse, **your warehouse remains the foundation for data access and security**. Hightouch inherits the controls already defined there — if a table, column, or dataset is restricted in your warehouse (e.g. via BigQuery IAM policies, Snowflake role grants, Databricks Unity Catalog, or Redshift permissions), Hightouch respects that access automatically. Hightouch then adds controls for how teams work with that data in the product:
- Who can access a workspace
- What data they can use
- What work they can see
- What can be sent to downstream tools
- Which changes require approval
This guide explains the main governance controls available in Hightouch, why they matter, and when to use them.
[**Explore the interactive architecture diagram →**](https://architecture.hightouch.com/?from=docs&product=governance&utm_source=hightouch-docs&utm_medium=referral&utm_campaign=architecture-explorer&utm_content=governance)
See how governance controls fit into the Hightouch architecture.
---
## How composable governance differs from traditional governance
In traditional customer data platforms, customer data is typically copied into the platform's own infrastructure. That means governance often happens inside a separate system with its own permissions, storage layer, and rules.
Hightouch works differently.
Hightouch connects directly to your existing warehouse and inherits the controls already defined there. If a table, column, or dataset is restricted in your warehouse, Hightouch respects that access. Hightouch then adds additional controls for activation workflows, so teams can use customer data without bypassing the governance your organization already has in place.
In practice, this means **governance in Hightouch is composable**:
- Your warehouse controls the underlying data access
- Hightouch controls how teams activate and distribute that data
- You can apply only the controls you need based on how your organization works
For most teams, this is easier to manage than creating separate systems or duplicating customer data just to enforce access rules.
---
## What governance helps you manage
Hightouch governance controls help answer a few practical questions:
- Who should be able to access this workspace?
- What data should each team be allowed to work with?
- Which audiences and campaign assets should teams be able to see?
- What data should be allowed to leave the platform?
- Which changes should require review before going live?
Each control in Hightouch helps with a different part of that process.
---
## 1. Control who can access a workspace
### Organizations and workspaces
Before configuring workspaces, it helps to understand the two levels of structure in Hightouch: [**Organizations**](/workspace-management/overview#organizations) and [**Workspaces**](/workspace-management/overview#workspaces).
A **Hightouch Organization** is the parent container representing your company as a whole. It is the top-level entity used for user management, billing, and cross-workspace administration. Within an Organization, you create one or more **Workspaces** — each a separate environment with its own sources, destinations, models, syncs, and users.
Two pre-built organization-level groups are available by default:
- **Organization Admin** — full access across all workspaces and org-level settings, including managing SSO configuration, billing, creating new workspaces, inviting users, and creating new user groups
- **Organization Viewer** — read-only access across the organization
Workspace-level roles and groups are separate from org-level groups. A user can be an Org Admin without being a Workspace Admin, and vice versa. In practice, Org Admins are typically members of your IT or data platform team, while day-to-day workspace management is delegated to Workspace Admins within each workspace.
### SSO and SCIM
For teams managing access at scale, Hightouch supports **SAML SSO** through identity providers like Okta and Microsoft Entra ID, plus automatic user provisioning via **SCIM**.
- **SAML SSO** automatically creates Hightouch users on their first login
- **SCIM** synchronizes user account changes from your identity provider to Hightouch automatically — including provisioning, deprovisioning, and group assignments
You configure SSO at the Organization level, then map groups in your identity provider to user groups already defined in Hightouch. When someone joins a team in Okta, they automatically receive the right workspace access and role permissions without manual provisioning.
See:
- [SSO setup guide](/workspace-management/sso)
### Workspace structure
The next governance decision is whether teams should work in the same workspace or in separate ones.
A **workspace** is a separate container with its own sources, destinations, models, syncs, and users. Separate workspaces are useful when you need *complete isolation*, such as:
- Regional or data residency requirements
- Strict separation between staging and production
- Business units that should not share data or configuration
For many organizations, though, separate workspaces are not the best starting point.
If teams share the same underlying customer data, keeping them in one workspace is often easier to manage. It reduces duplicate setup and allows teams to work from the same source of truth while still applying internal governance controls.
Use workspace-level structure when you need hard separation. Otherwise, start with a shared workspace and add more targeted controls inside it.
Create separate workspaces only when required for geo-specific data processing, separate infrastructure, or compliance. For most enterprise customers, a single production workspace is sufficient.
See:
- [Workspaces](/workspace-management/workspaces) — includes a [side-by-side comparison](/workspace-management/workspaces#do-you-need-another-workspace) of workspaces vs. environments
- [Core concepts](/workspace-management/overview)
- [Environments](/workspace-management/environments)
### When to use this
Use separate workspaces when you need complete operational or regulatory isolation.
Use a shared workspace when teams should collaborate from the same models and data, but still need scoped access and control.
---
## 2. Control what data teams can work with
Once users are in a workspace, the next question is: what data should they actually be able to use?
This is where access controls become important for day-to-day work.
For example:
- A regional team may only need access to customers in its market
- A brand team may need to work only with its own customer segment
- Marketers may need to build audiences without seeing raw PII
- Some users may need to review data, while others need to create and launch workflows
Hightouch supports this through **user groups and roles**, **subsets**, and **data masking**.
### User groups and roles
Access control in Hightouch is managed at the **user group** level, not on individual users. A **user group** is a collection of users who share the same permissions. Each group is assigned a role per workspace, which determines what actions members can take.
There are two types of user groups:
- **Pre-built groups** — Organization Admin and Organization Viewer, available by default
- **Custom groups** — created by org admins to reflect different teams, regions, brands, or any other structure that impacts permissions
A user can belong to multiple groups and inherits the combined permissions of all their group memberships. If a user is in both a Viewer group and an Editor group for the same workspace, they receive the higher level of access — Editor.
**Roles** define what actions a group can perform within a workspace. Pre-built roles apply across all sources and destinations:
- **Admin** — full grants, typically assigned to data or IT teams
- **Editor** — can create and configure resources, but cannot delete or update existing source and destination configurations
- **Viewer** — read-only access across the workspace
- **Draft Contributor** — can create and propose changes, but cannot publish; all changes go through an approval flow
**Custom roles** are scoped to specific sources and destinations rather than applying globally. Permissions for dependent resources — models, syncs, and audiences — are inherited from their associated source and destination. Custom role permissions are configured across four dimensions:
- **General** — manage who can create new integrations
- **Sources** — manage rights per source (including schema management, which also governs access to traits, sync templates, and governance features like destination rules and subsets)
- **Destinations** — manage rights per destination
- **Customer Studio** — manage rights per parent model
This means you can create a "Paid Media Marketing" role that has access to Google Ads, Meta, and TikTok destinations but cannot see or touch Salesforce CRM or Braze configurations — enforced at the platform level, not by policy document.
Start with pre-built roles during onboarding. Switch to custom roles to scope access to specific sources and destinations as your usage scales.
See:
- [User groups](/workspace-management/user-management)
- [Roles](/workspace-management/roles)
### Subsets
**Subsets** limit which records a team can access at the row level.
This is useful when multiple teams work in one workspace but should only be able to use part of the overall customer data, such as:
- One region
- One brand
- One business unit
- One consented customer segment
Subsets are enforced at the query level — there is no way to bypass them through the UI or API. Users in the audience builder automatically see only the data their subset allows.
Common subset patterns include consent enforcement (filtering out opted-out customers globally), regional partitions, brand-level segments, and custom suppression lists for legal hold or VIP exclusions.
Subsets are configured under **Governance > Subset Categories** in Customer Studio.
See:
- [Subsets](/customer-studio/subsets)
### Data masking
Data masking helps teams work with customer data without exposing sensitive values in previews, profile views, or record inspection.
Three masking levels are available:
- **Approved** — data is visible and available for use in syncs and filters
- **Redacted** — data is hidden in the UI (profile details, column suggestions, analytics) but remains accessible for syncs and audience filters
- **Blocked** — data is completely restricted from access and syncs
This is useful when users need to build audiences from behavioral or account data, but should not see raw identifiers or sensitive fields.
See:
- [Data masking](/customer-studio/data-masking)
### When to use this
Use these controls when multiple teams share a workspace, but not everyone should have the same level of access to the same data.
---
## 3. Control which campaign assets teams can see
The previous section covered who can access the platform and what underlying data they can query. This section is about something different: which audiences, syncs, and campaign assets different teams can see once they're inside the workspace.
Even if two teams have identical data access, they may have no reason to see each other's audiences, syncs, or journeys. Without that separation, large workspaces become noisy, harder to navigate, and easier to change by mistake.
### Spaces
This is where **Spaces** help.
Spaces let you partition a single workspace into isolated sub-workspaces for different regions, brands, teams, or business units. Within a Space, teams work from the same underlying models, sources, and destinations — but their audiences and syncs are scoped to that Space and are not visible to other teams.
Key behaviors:
- Models, sources, destinations, and schema are **shared** across all Spaces in a workspace
- Audiences and syncs are **scoped** to a specific Space
- Only users with explicit access to a Space can create, view, or edit its resources
- Workspace admins retain cross-Space visibility for governance and troubleshooting
- Access is managed via **user groups**, not individual users
In practical terms, Spaces help answer questions like:
- Which audiences should this team see?
- Which syncs belong to this region?
- How do we keep one team from accidentally editing another team's work?
When enabled, Spaces can be accessed and configured under **Settings → Workspaces → Spaces**.
Spaces is currently in early access and must be enabled for your workspace by your Hightouch team. Contact your Hightouch representative to request access.
See:
- [Spaces](/workspace-management/spaces)
### When to use this
Use Spaces when teams should share the same workspace and data foundation, but should not all see the same campaign assets or operational work.
---
## 4. Control what data can leave Hightouch
One of the most important governance decisions is what happens when customer data is sent to downstream tools or [destinations](/destinations/overview).
Even if the right team built the right audience, you may still need to control:
- Which records can be synced to a destination
- Whether consent rules are enforced
- Which fields are allowed to be mapped
- Whether syncs follow an approved setup pattern
Hightouch supports this through **Destination Rules**, **Data Masking**, **Audience Templates**, and **Sync Templates**.
### Destination Rules
**Destination Rules** automatically filter which records are eligible to sync to a specific destination.
They are evaluated at sync time — they do not affect the underlying audience definition. Think of them as a channel-specific filter that Hightouch applies automatically every time an audience syncs to a given destination.
They are especially useful for enforcing requirements like:
- Email consent
- SMS consent
- Advertising consent
- Regional policy restrictions
- Minimum audience thresholds
- Frequency capping
This helps move consent enforcement out of individual campaign setup and into a reusable governance control that applies regardless of who built the audience or which Space it belongs to.
Destination Rules are configured under **Governance > Destination Rules** in Customer Studio.
See:
- [Destination Rules](/customer-studio/destination-rules)
### Data Masking
**Data Masking** can also be used at the destination layer to ensure that raw sensitive values — such as email addresses or phone numbers — are not mapped to destinations that only need hashed identifiers or anonymized fields.
See:
- [Data masking](/customer-studio/data-masking)
### Audience Templates
**Audience Templates** let admins define reusable audience structures that users can quickly adapt with minimal changes.
Admins set up an audience as usual but leave certain fields blank — those become the fields users fill in when creating an audience from the template. This ensures that baseline segmentation logic, suppression rules, and consent filters are baked into every audience from the start. Marketers can customize within the template's parameters but cannot bypass the foundational logic.
You can also mandate that all new audiences must start from an approved template, ensuring governance guardrails are always in place.
Audience Templates are configured under **Templates > Audience Templates** in Customer Studio.
See:
- [Audience Templates](/customer-studio/templates#audience-templates)
### Sync Templates
**Sync Templates** give teams an approved starting point for destination configuration.
Instead of setting up every sync from scratch, admins can define approved mappings, modes, and field-level permissions ahead of time. For specific fields, admins can use the **editable toggle** to allow users to customize only those fields when creating a sync from a template.
You can also enforce the use of sync templates entirely by enabling the **Require sync templates** toggle in Settings.
This helps teams move faster while reducing setup mistakes and configuration drift.
Sync Templates are configured under **Templates > Sync Templates** in Customer Studio.
See:
- [Sync Templates](/customer-studio/templates#sync-templates)
### When to use this
Use destination-level controls when different downstream tools have different requirements, or when your team wants to enforce safe and repeatable activation patterns.
---
## 5. Organize and navigate workspace resources
As workspaces grow, teams need ways to keep resources organized and easy to find. Hightouch provides three tools for this: **Folders**, **Labels**, and **Filters**.
### Folders
Folders let you organize resources like syncs, models, and audiences into a shared hierarchy. You can group resources in a strict hierarchy or organize by team or department. Folder hierarchy is shared across resource types — moving a sync also moves its associated model to the same folder.
Only workspace admins can modify the folder structure. Other roles can add or remove resources from existing folders.
### Labels
Labels are key/value pairs that let you tag resources across multiple dimensions that don't fit into a folder structure. For example:
- Track campaign type: `type:email`, `type:sms`, `type:push`
- Track brand or region: `brand:brand-a`, `region:eu`
Multiple labels can be applied to a single resource. You can then filter by label combinations in the sidebar.
### Filters
Filters let you narrow the resource list by attributes like sync progress, destination, source, health status, and more. Once you've set up a useful filter combination, you can save it for future use.
See:
- [Folders and labels](/workspace-management/folders-filters)
### When to use this
Use these tools in any workspace with multiple teams or a large number of resources. They don't enforce access — they improve discoverability and reduce noise.
---
## 6. Control which changes require review
Some changes are low risk. Others should not go live without review.
For example:
- Launching a new audience to a paid media platform
- Changing sync mappings for a production destination
- Updating logic that affects a large customer segment
- Allowing less technical users to propose changes without publishing directly
Hightouch supports this through **Approval Flows**, and in some cases through **Environments** for staging and production workflows.
### Approval Flows
Approval Flows let designated users create or edit resources in draft mode, then require review before those changes go live.
To set this up:
1. Enable the **Require Approvals** toggle in **Settings > Workspace**
2. Assign the **Workspace Draft Contributor** role to users who should go through the approval process
3. By default, all roles except Workspace Draft Contributor and Workspace Viewer can grant approval — if you need to grant approval permissions to other roles, build a custom role
Approval Flows pair well with Audience Templates and Sync Templates. When both are in place, admins can require that all new audiences start from an approved template — ensuring that baseline segmentation logic, suppression rules, and consent filters are baked in. Marketers can customize within the template's parameters but cannot bypass the foundational logic.
This is useful when you want marketers or operators to move quickly, but still want an admin, data team, or compliance owner to review higher-impact changes first.
See:
- [Approval flows](/workspace-management/approval-flows)
### Environments
Environments apply to **models and syncs only** ([rETL workflows](/getting-started/concepts)). Customer Studio resources — such as audiences, destination rules, and sync templates — cannot be deployed between environments. For Customer Studio change management, use **Approval Flows** instead.
Environments help teams separate testing from production and move changes more intentionally between them. Each workspace acts as a separate environment. You can link workspaces and deploy models and syncs from one environment to another without recreating them from scratch.
Before a deployment executes, Hightouch validates that all underlying resources (sources, destinations, models) match their production counterparts — preventing mismatches that could cause data integrity issues.
See:
- [Environments](/workspace-management/environments)
### Audit visibility
Governance also depends on being able to understand what changed, when, and by whom.
Audit logs are available under **Settings > Audit log** and track up to 90 days of in-app activity. Each log entry records the user, action, resource, and a before/after JSON diff. For longer retention or custom dashboarding, audit data can also be written back to your warehouse via [Warehouse Sync Logs](/syncs/warehouse-sync-logs).
See:
- [Audit logs](/workspace-management/audit-logs)
### When to use this
Use review and change-management controls when multiple teams can affect production workflows, or when your organization needs more formal approval before activation.
---
## Choosing the right governance controls
Not every team needs every governance feature from day one.
A practical way to roll out governance is to start with the controls that match your organization's complexity.
### Smaller teams
Start with:
- Workspace structure
- User groups
- Roles
### Growing teams
Add:
- Subsets
- Spaces (if early access is available)
- Basic destination controls
### Larger or more regulated teams
Consider:
- Custom groups and scoped roles by team, region, or brand
- Subsets by region, brand, or business unit
- Spaces
- Destination Rules
- Audience Templates
- Sync Templates
- Approval Flows
- Environments (for rETL models and syncs only)
- Audit logs
The goal is not to add as many controls as possible. The goal is to give teams the access they need while reducing unnecessary risk.
---
## Recommended governance patterns
### If teams share customer data but work independently
Use:
- One shared workspace
- Roles for action-based access
- Subsets for data access
- Spaces for team-level visibility (if early access is enabled)
### If different regions or entities must be fully separate
Use:
- Separate workspaces
- Separate infrastructure or regional connections as needed
### If consent rules differ by channel
Use:
- Destination Rules
- Sync Templates
### If campaign changes should be reviewed before launch
Use:
- Approval Flows
- Environments (for rETL models and syncs only) for staging and production where needed
---
## Next steps
To put governance into practice, start with the feature docs for the controls you need:
- [Core concepts](/workspace-management/overview)
- [User groups](/workspace-management/user-management)
- [Roles](/workspace-management/roles)
- [SSO setup guide](/workspace-management/sso)
- [Subsets](/customer-studio/subsets)
- [Destination Rules](/customer-studio/destination-rules)
- [Data masking](/customer-studio/data-masking)
- [Audience Templates](/customer-studio/templates#audience-templates)
- [Sync Templates](/customer-studio/templates#sync-templates)
- [Approval flows](/workspace-management/approval-flows)
- [Environments](/workspace-management/environments)
- [Audit logs](/workspace-management/audit-logs)
- [Folders and labels](/workspace-management/folders-filters)
- [Spaces](/workspace-management/spaces) — early access feature; contact your Hightouch team to enable
---
## Core concepts
**URL:** https://hightouch.com/docs/workspace-management/overview
**Description:** Understand the core building blocks of Hightouch — organizations, workspaces, user groups, roles, environments, and Spaces.
**Section:** Workspace management
This page covers core concepts. For prescriptive guidance on when to use each control and how to structure your organization, see [Governance best practices](/workspace-management/governance).
Hightouch is built around a set of core concepts: organizations, workspaces, user groups, roles, environments, and Spaces. Understanding how they relate to each other is the foundation for configuring your workspace correctly.
---
## Organizations
An **organization** is the top-level entity in Hightouch. It represents your entire company and is the container for billing, SSO configuration, and user management. All workspaces belong to a single organization.
Most companies should have exactly one organization. An organization is created automatically when the first person at your company signs up — that person becomes the first organization admin. From there, organization admins can manage billing, configure [SSO and SCIM](/workspace-management/sso), [invite users](/workspace-management/user-management#inviting-users), and [transfer admin access](/workspace-management/user-management#transferring-organization-admin-access) as team members change.
See: [Set up an organization and workspace](/workspace-management/workspaces#first-sign-in)
---
## Workspaces
A **workspace** is a fully isolated partition within an organization. Each workspace has its own sources, destinations, models, syncs, audiences, and user roster. By design, nothing is shared across workspace boundaries — not even the connection to your data source.
Workspaces are commonly used to:
- Separate staging and production environments
- Isolate business units or regions that must not share resources or configuration
For most enterprise customers, a single production workspace is sufficient. Hightouch's internal governance controls — including user groups, roles, subsets, and Spaces — allow different teams, regions, or brands to operate within a shared workspace without interfering with each other.
To learn how to set up a new workspace, see [Workspaces](/workspace-management/workspaces). For guidance on when to create additional workspaces versus using [Environments](/workspace-management/environments), see [Do you need another workspace?](/workspace-management/workspaces#do-you-need-another-workspace)
---
## Environments
An **environment** is a linked copy of a workspace used for change management — typically dev, staging, and production. Environments let you test changes to models and syncs before deploying them to production, without needing a completely separate workspace.
Environments share the same organization, user roster, and governance controls as the parent workspace. Use them when you need a safe promotion path but don't need hard data isolation.
See: [Environments](/workspace-management/environments) | [Workspaces vs environments](/workspace-management/workspaces#do-you-need-another-workspace)
---
## Spaces
A **Space** is a visibility boundary within a single workspace. Spaces let you scope which resources (syncs, models, audiences) a team can see without creating a separate workspace. They're useful when multiple teams share a workspace but shouldn't be distracted by each other's work.
Spaces control visibility, not permissions — roles and user groups still govern what actions a user can take.
See: [Spaces](/workspace-management/spaces)
---
## User groups
A **user group** is a collection of users who share the same permissions. Access control in Hightouch is managed at the group level, not for individual users. This makes it easier to keep permissions consistent and to update access as team structures change.
User groups are created within an organization and can be granted access to one or more workspaces. Users can belong to multiple groups and inherit the combined permissions from all of their assigned groups.
In most cases, user-to-group mappings are managed through your identity provider via [SSO and SCIM](/workspace-management/sso). If SSO is not configured, user groups and memberships can be managed manually in the Hightouch app.
See: [User groups](/workspace-management/user-management)
---
## Roles
A **role** defines what actions a user group can perform within a workspace. User groups aggregate users; roles define the scope of what those users can do.
Hightouch offers two types of roles:
- **Pre-built roles** — apply broadly to all resources in a workspace. These are the right starting point for most teams.
- **Admin** — full grants across the workspace, typically assigned to data or IT teams
- **Editor** — can create and configure resources, but cannot delete or update existing source and destination configurations
- **Viewer** — read-only access across the workspace
- **Draft Contributor** — can create and propose changes, but cannot publish; all changes require approval before going live
- **Custom roles** — provide granular, per-resource control. Custom roles let you define specific permissions for individual sources, destinations, and Customer Studio parent models. Useful when pre-built roles are not flexible enough for a large or complex workspace.
See: [Roles](/workspace-management/roles)
---
## Next steps
- [Workspaces](/workspace-management/workspaces) — create, manage, and delete workspaces
- [Set up user groups and members](/workspace-management/user-management)
- [Configure roles](/workspace-management/roles)
- [Set up SSO and SCIM](/workspace-management/sso)
- [Governance best practices](/workspace-management/governance) — when to use workspaces, subsets, Spaces, destination rules, approval flows, and more
---
## Roles
**URL:** https://hightouch.com/docs/workspace-management/roles
**Description:** Learn how role-based access control works in Hightouch, including pre-built and custom roles, permission types, and best practices for managing workspace access.
**Section:** Workspace management
| Field | Content |
| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Audience** | Organization admins |
| **Prerequisites** | Organization admin permissions, or membership in a group with access to [Organization settings](https://app.hightouch.com/settings/organization). |
## Overview
Hightouch uses **role-based access control (RBAC)** to define what actions users can take across workspaces and resources.
Each **group** in your organization is assigned one or more roles that determine permissions for specific workspaces, sources, destinations, and Customer Studio models.
This version of role management was introduced with **Permissions V2**. For older implementations, see the [Access control (legacy)](https://hightouch.com/docs/workspace-management/rbac) documentation.
---
## Why use roles?
Roles help you manage access safely and consistently across your organization. They define who can view data, build or publish syncs, and manage workspace settings.
Using roles keeps your workspaces organized and secure by ensuring each person only has access to the tools and data they need.
**Common examples:**
- Give your **marketing team** access to Customer Studio, but not to source connections.
- Allow your **data team** to manage syncs and schema configurations.
- Limit **view-only** roles for stakeholders who just need to monitor sync health or insights.
---
## Role types
Hightouch offers two kinds of roles:
- [**Pre-built roles**](#pre-built-roles): Ready-made access levels (Admin, Editor, Draft Editor, Viewer) that make onboarding fast.
- [**Custom roles**](#custom-roles): Fine-grained roles you can tailor to match your teams, regions, or workflows.
Start with pre-built roles for onboarding. When your organization grows, switch to custom roles to fine-tune access by workspace or product.
---
## How roles work
Hightouch’s permissions system is built around **data flow** (who can see data and where it can be sent) rather than micromanaging every model or sync.
### Access is based on sources and destinations
Every sync involves multiple resources. For example, syncing BigQuery → Salesforce includes:
- The **source** (BigQuery)
- The **model** built from BigQuery
- The **destination** (Salesforce)
- The **sync** that connects them
Rather than managing permissions for each model or sync, Hightouch groups access by **source** (where data comes from) and **destination** (where it goes).
Models and syncs inherit permissions from these top-level resources.
To create or run a sync, users need permission for **both** the source and the destination. Having access to one side alone isn’t enough.
---
### Governance in context
RBAC defines **who can act** in a workspace, but it’s only _one_ part of Hightouch’s governance framework.
While roles manage user access, they don’t enforce organization-wide policies like redacting PII or honoring consent requirements. Maintaining compliance and data quality at scale often requires additional controls that apply across all users and workspaces.
Hightouch provides complementary governance features to manage data access and operational workflows:
- **[Destination rules](/customer-studio/destination-rules)** – Control what types of data can be sent to specific destinations (for example, ensuring only consented users are synced to email marketing).
- **[Subsets](/customer-studio/subsets)** – Divide your total addressable customer base into smaller segments, which is especially useful in workspaces that span multiple brands or regions.
- **[Approval flows](/workspace-management/approval-flows)** – Add peer review for sync and model changes before publishing.
- **[Staging environments](/workspace-management/environments)** – Test and validate syncs in a dedicated space before going live to reduce the risk of errors in production.
Together, these features create a unified framework for managing visibility, data movement, and change control. You can use them independently or in combination, depending on your organization’s size, compliance needs, and collaboration model.
You don’t need to enable every governance feature—especially when you’re just getting started. Many are built for large enterprises with complex brand or compliance needs. If you’d like help designing the right setup, — we’re here to help.
---
## The anatomy of a role
Each role in Hightouch is made up of **grants** (specific permissions). There are four types:
| Grant Type | Description |
|-------------|--------------|
| **General** | Controls whether users can create new sources or destinations in the workspace. |
| **Sources** | Defines how users interact with connected data sources (view row-level data, manage credentials, create models). |
| **Destinations** | Defines what users can do with destinations (create syncs, trigger sync runs, set up alerts). |
| **Customer Studio** | Controls access specifically to the Customer Studio product. They function just like source-level grants, except they apply to parent models instead of the underlying source. These grants mainly control the types of audiences users can create and determine if users can view data about individual members of those audiences. |
## Reference list of individual grants
When configuring a custom role, you'll see a list of individual grants that can be toggled on or off. These grants are built around two key pillars of governance: managing data visibility and controlling data flow. Below, you'll find a breakdown of each grant, detailing its function and how it affects different features across the Hightouch app.
**Jump to:**
[General](#general) · [Sources](#sources) · [Destinations](#destinations) · [Customer Studio](#customer-studio) · [Subsets](#differences-when-subsets-are-enabled) · [Approval Flows](#differences-when-approval-flows-are-enabled)
## General
**Can this role create new sources?**
Allows users to create new data sources within the workspace. During setup, they can assign access permissions for other groups.
**Can this role create new destinations?**
Allows users to create new destinations within the workspace and assign access permissions for other groups.
Workspace-level resources—such as folders, extensions, custom storage, and API keys—can only be configured by workspace admins.
### Source
Source-level permissions are designed primarily for Reverse ETL use cases, where users need the flexibility to run custom SQL when pulling data from the source. If you're creating a role for marketers in Customer Studio, it's best to avoid enabling any source grants. Instead, use Customer Studio grants, which limit access to specific parent models and subsets, ensuring that marketers work within your pre-approved Customer Studio schema.
**View source data**
With this grant enabled, users can view row-level data from this source across the Hightouch app. This includes:
- Previewing models
- Viewing row-level data when testing rows during sync creation
- Accessing row-level data in the debugger for historical sync runs
Primary keys and error messages are **not** classified as row-level data to preserve basic sync observability. If you choose not to assign this grant, avoid using columns with PII (like email or phone number) as primary keys, as these fields will be visible to all users across the app.
**Configure models & syncs**
When enabled, this grant allows users to:
- Create, edit, or delete models that pull data from the source
- Use these models for syncing (provided the user has permissions to sync data to destinations)
Users can only edit existing models if they can also edit all of the syncs that rely on that model.
**Configure schema**
Applicable only to workspaces with Customer Studio, this grant allows users to configure the schema layer for Customer Studio, including traits, destination rules, and subsets.
**Manage source**
With this grant enabled, users can edit the source configuration, which includes:
- Updating connection details and credentials
- Adjusting the sync engine choice
- Setting source-level defaults for Warehouse Sync Logs
### Destination
**Trigger syncs**
Lets users manually trigger sync runs. Does not allow editing schedules.
**Configure syncs**
Lets users create, edit, or delete syncs to the destination and set their schedules. Does not allow manual triggering.
**Manage destination**
Allows users to modify the destination's configuration, including:
- Updating connection details and credentials
- Configuring defaults for alert triggers and recipients
### Customer Studio
These grants function similarly to source-level grants but are scoped to specific parent models within the Customer Studio schema.
**View parent model data**
Enabling this grant allows users to view row-level data from the parent model throughout the Hightouch app. This includes:
- Previewing audiences to see row-level data for individual members
- Viewing row-level data when testing rows during sync creation
- Accessing row-level data in the debugger for historical sync runs
If a user has the **Configure audiences & syncs** grant but lacks the **View
parent model data** grant, they can still preview audiences, but they'll be
limited to aggregate counts and insights, without row-level data for each
audience member.
**Configure audiences & syncs**
When enabled, this grant allows users to:
- Create, edit, or delete audiences that pull data from the parent model
- Use these audiences for syncing (provided the user has permissions to sync data to destinations)
Users can only edit existing audiences if they can also edit all of the syncs
that rely on that audience.
**Configure journeys**
When enabled, this grant allows users to:
- Create, edit, or delete journeys and journey templates.
**Publish journeys**
When enabled, this grant allows users to:
- Launch, pause, or stop live journeys
### Differences when subsets are enabled
[**Subsets**](/customer-studio/subsets) let you divide a parent model into smaller, permissioned segments (e.g., by brand, region, or consent type).
They're configured in **Customer Studio → Governance → Subset categories**.
For example:
- Your **EU marketing team** can only build audiences using EU data.
- Your **email marketing team** can only use consented audiences.
When subsets are active, you'll see a separate set of checkboxes that let you apply earlier grants (like **View parent model data** and **Configure audiences & syncs**) to specific subsets. The behavior of these checkboxes varies based on whether the subset category is marked as "required."
- **If the subset category is required,** the user will only be able to configure and use audiences within the selected subset values. (If no values are checked, the user will not be able to configure or use any audiences at all.)
- **If the subset category is not required,** the user can configure and use audiences with the selected subset categories, or without subsets altogether.
### Differences when approval flows are enabled
[**Approval flows**](/workspace-management/approval-flows) add review steps before publishing model or sync changes.
They’re enabled in **Settings → Workspace → Configuration → Require approvals**.
When enabled, the **Configure models & syncs** grant splits into:
- **Draft models & syncs** – Create or edit drafts (cannot delete published resources).
- **Approve models & syncs** – Review and approve or deny drafts.
Together, these two grants provide the same functionality as the original **Configure models & syncs** grant.
Approval flows do not apply to audience creation in Customer Studio, as there
is no concept of a draft audience. However, approval may be required to sync
an audience to a destination.
The **Draft** and **Approve** grants are separate and do not overlap. To allow
a user to publish changes without requiring approval, you need to assign both
the **Draft** and **Approve** grants. (Under the hood, the user creates a
draft and then immediately self-approves it.)
To approve a sync, a user needs the **Approve** grant for both the source and
the destination.
## Pre-built roles
We recommend starting with pre-built roles during onboarding. These cover common access levels and make setup fast.
Once your workspace is established, switch to custom roles for more granular control.
There are four pre-built roles:
- **Workspace admin** – Full access to all resources and settings. Can manage, approve, and configure everything. Organization admins automatically inherit this role.
- **Workspace editor** – Can create and edit most resources but cannot approve or manage credentials.
- **Workspace draft editor** – Can create and edit drafts (where supported) but cannot trigger or approve syncs.
- **Workspace viewer** – Read-only access. Cannot see row-level data or make changes.
All built-in roles apply to **all** sources, destinations, and Customer Studio parent models. If you need to scope permissions to specific resources, use a [custom role](#custom-roles).
---
### Pre-built role permissions
#### General
| Permission | Admin | Editor | Draft editor |
| ----------------------- | ----- | ------ | ------------ |
| Create new sources | ✅ | ✅ | ❌ |
| Create new destinations | ✅ | ✅ | ❌ |
#### Source
| Permission | Admin | Editor | Draft editor |
| ---------------------- | ----- | ------ | ------------ |
| View row-level data | ✅ | ✅ | ✅ |
| Draft syncs & models | ✅ | ✅ | ✅ |
| Approve syncs & models | ✅ | ✅ | ❌ |
| Configure schema | ✅ | ❌ | ❌ |
| Manage source | ✅ | ❌ | ❌ |
#### Destination
| Permission | Admin | Editor | Draft editor |
| ---------------------- | ----- | ------ | ------------ |
| Trigger syncs | ✅ | ✅ | ❌ |
| Draft syncs & models | ✅ | ✅ | ✅ |
| Approve syncs & models | ✅ | ✅ | ❌ |
| Manage destination | ✅ | ❌ | ❌ |
#### Customer Studio
Customer Studio does **not** have a “draft audience” concept. Approval flows do not apply to audience creation, but they may be required for **audience syncs**.
| Permission | Admin | Editor | Draft editor |
| ------------------------- | ----- | ------ | ------------ |
| View parent model data | ✅ | ✅ | ✅ |
| Create and edit audiences | ✅ | ✅ | ✅ |
| Trigger audience syncs | ✅ | ✅ | ❌ |
| Approve audience syncs | ✅ | ❌ | ❌ |
* **Create & edit audiences** - No approval step; available to all roles except Viewer.
* **Trigger audience syncs** - Starts a sync run for an audience; requires source and destination permissions.
* **Approve audience syncs** - Required if approval flows are enabled for syncing audiences.
* **Draft Editor** - Can create and edit audiences, but cannot trigger or approve audience syncs.
---
### Assign a pre-built role
Roles are assigned to **user groups**, not individual users. By default, new groups don't have access to any workspaces.
To manage group roles:
1. Go to **Settings → Organization → [Groups](https://app.hightouch.com/organization/groups)**.
2. Select or create a group.
3. In the **Workspaces** tab, use the **Role** dropdown to assign a pre-built role.
4. **Save changes**.
The **Access** column provides an overview of the group's permissions for each workspace, showing the actions they can take for each destination. While it gives a helpful summary, it doesn't display every possible grant or cover permissions related to other resource types.
---
## Custom roles
Custom roles let you define precise permissions for each team, workspace, or use case. They’re ideal once your organization scales beyond standard access levels and you need finer control over **who can view, edit, or publish** data connections and audiences.
### When to use a custom role
Use custom roles when:
- Different teams (e.g., marketing, data, and engineering) share a workspace but need distinct permissions.
- You want to restrict access to certain **sources** or **destinations**.
- You’re managing multiple brands, business units, or regions with different data governance policies.
- You use **Approval flows**, **Subsets**, or **Destination rules** and need to define roles that align with those governance features.
### How to create or edit a custom role
1. Go to **Settings → Organization → [Groups](https://app.hightouch.com/organization/groups)**.
2. Select or create a group.
3. In the **Workspaces** tab, select `Custom...` from the **Role** dropdown.
4. **Save changes**.
5. Click **Edit custom role** to open the configuration panel.
You’ll see four tabs that correspond to different types of permissions:
- **General** — Grants permission to create new sources and destinations.
- **Sources** — Defines what users can do with connected sources (view data, configure models, manage credentials).
- **Destinations** — Defines what users can do with connected destinations (configure or trigger syncs, manage credentials).
- **Customer Studio** — Defines permissions for building and managing audiences.
Changes save automatically when you click **Save changes** in the bottom-right corner.
## Next steps
Once your roles are in place, explore additional governance tools that complement RBAC:
- [Approval flows](/workspace-management/approval-flows)
- [Destination rules](/customer-studio/destination-rules)
- [Subsets](/customer-studio/subsets)
- [Staging environments](/workspace-management/environments)
{
// Note: this is a useful section, but commenting out for now because we should verify what the default permissions are for IDR, AID and events
// ### How do I set permissions for other products?
// Currently, only workspace admins can access Identity Resolution, AI Decisioning, and Events. If you're interested in joining a private preview for custom permissions in these features, please reach out to us.
}
## FAQ
### How do I prevent a user from seeing a resource?
In Hightouch, all resources in a workspace are intentionally visible to all workspace members; configuration details are not hidden. Access control determines who can create or modify resources, but it cannot make resources invisible. If you need to restrict visibility of sensitive resources for certain users, consider placing these resources in a separate workspace.
{/* TODO: [add callout about RBAC supporting row-level hiding] */}
### What happens if a user belongs to more than one group?
Users can belong to multiple groups within the same workspace, allowing them to inherit permissions from more than one role at a time.
This setup is common. When a user is part of multiple groups, they receive the combined permissions of all their group memberships, meaning that they can perform all actions permitted to members of any of those groups.
However, this does not allow them to “mix and match” permissions across groups. In other words, if one group can sync from source A to destination B, and another group can sync from source C to destination D, a user who belongs to both groups can sync from A to B and C to D—but not from A to D or C to B.
---
## SSO setup guide
**URL:** https://hightouch.com/docs/workspace-management/sso
**Description:** Hightouch helps customers manage access control for large organizations via a third-party identity provider.
**Section:** Workspace management
## Overview
Hightouch integrates with identity providers like Okta and Microsoft Entra ID to simplify user management and authentication.
- **SAML SSO** enables users to authenticate through their organization’s identity provider, supporting just-in-time (JIT) provisioning during login.
- **SCIM** automates user management tasks such as group assignments and deactivations based on changes in the identity provider.
Need help with SSO or SCIM setup? Share this page with your IT team.
### Required permissions
To configure these integrations, you'll need the following:
- Admin access to your company's identity provider, which is typically managed by your IT team.
- User membership in the **Organization admins** group within your Hightouch organization.
### Identity model and trust boundaries
- SSO is configured at the **organization level** in Hightouch. This means all workspaces in that organization inherit the same SAML trust and access model.
- Hightouch supports one SAML IdP trust per organization at a time. Updating IdP metadata (like issuer, certificate, or IdP SSO URL) updates that trust for the entire organization.
## Configuring SAML SSO
SAML single sign-on (SSO) enables just-in-time (JIT) provisioning so that Hightouch users are automatically created upon first login. Once SSO is set up, users in your organization can navigate to the [Hightouch login page](https://app.hightouch.com/login) and select **Log in with SSO** to authenticate with your identity provider.
When a user attempts to log in, Hightouch sends a SAML authentication request to your identity provider. If the identity provider validates the user’s credentials and confirms that the user is authorized to access Hightouch, the user is logged in. For first-time users, an account is automatically provisioned based on the information returned from the identity provider, such as name and email address. SSO also ensures that user attributes (like email addresses) are updated during login when changes are detected.
Configuration steps for SAML SSO may vary depending on the identity provider used. Below are detailed instructions for setting up SAML with [Okta](#okta) and [Microsoft Entra ID](#microsoft-entra-id) (formerly known as Azure Active Directory). For other identity providers, similar configuration will apply.
For an overview of SSO and SAML concepts, refer to this [introductory video](https://www.youtube.com/watch?v=O1cRJWYF-g4).
Keep your SAML NameID stable over time, especially during IdP migrations or
tenant changes. If the NameID format or source value changes (for example,
from `email` to an employee ID or UUID), users may be created as duplicate
identities in Hightouch.
### Okta
The first step is to create a new SAML application in Okta. You can follow [this guide](https://help.okta.com/oag/en-us/Content/Topics/Access-Gateway/add-app-saml-pass-thru-add-okta.htm) or the steps outlined below:
1. In your Okta dashboard, navigate to **Applications** and select **Create App Integration**.
2. Choose **SAML 2.0** as the sign-in method and click **Next**.
3. Give the application a descriptive **App Name**, such as "Hightouch." You can also add the [Hightouch logo](https://cdn.sanity.io/images/pwmfmi47/production/acbc037f008ebc7c51a8ba6ca32db440aec03ee2-1600x1600.jpg) if you'd like. Then, click **Next**.
4. In Hightouch, visit the **Single sign-on** tab on the **Organization settings** page. Click **Configure SAML SSO** to display a modal that provides the **Hightouch SSO URL** and **Audience URI**.
5. In Okta, configure the SAML settings by entering the **Hightouch SSO URL** from the Hightouch modal as the **Single sign on URL**, and the **Audience URI** as the **Audience URI (SP Entity ID)**. You can leave the remaining fields at their default settings.
6. Under **Attribute Statements**, map the `name` and `email` attributes. For instance, you might map `name` using `String.join(" ", user.firstName, user.lastName)` and `email` as `user.email`. Ensure that these match the properties defined in your Okta instance. You can refer to the Okta user [profile properties](https://developer.okta.com/docs/reference/api/users/#default-profile-properties) if needed. Click **Next** to continue.
7. For the prompt **Help Okta Support understand how you configured this application**, select **I'm an Okta customer adding an internal app**. Click **Finish**.
8. Then, you’ll be taken to the application overview page in Okta. Under **Metadata details**, click **More details**, then copy the **Sign on URL** and download the **Signing Certificate**. These will be needed for Hightouch.
9. In the Hightouch modal from step 4, paste the **Identity provider SSO URL** and upload the certificate. Click **Save** to finalize the connection.
10. In Okta, go to the **Assignments** tab of the application you just created. Assign users or groups to grant them access to Hightouch.
11. At this point, you've completed the basic SAML SSO setup, allowing your users to log in to Hightouch through Okta. However, you'll still need to manually assign permissions for each user after they join your Hightouch organization. To streamline this process, we highly recommend setting up automatic group assignments in Okta. This ensures users have the right access to workspaces and resources as soon as they log in for the first time.
If SSO group mappings are not configured, new users won't be automatically
assigned to any group, which may result in limited access until group
assignments are manually set up.
12. To configure group mappings, navigate to the **General** tab for your Hightouch application in Okta and click the **Edit** button in the **SAML Settings** section.
13. Scroll down to the **Group Attribute Statements** section. Set the attribute name to `groups` and apply the appropriate filter. For instance, to make all Okta groups available in Hightouch, select the **Matches regex** filter and enter `.*`.
If you want to send only specific groups, you can either map them individually or use a filter like **Starts with `Hightouch`**.
14. Next, you'll want to go back to Hightouch and create mappings between the groups from your identity provider and the corresponding user groups in Hightouch. Navigate to **Organization settings** and click on the **Single sign-on** tab.
15. Scroll to the bottom section called **Group mappings**. In this table, each group from your identity provider can be mapped to any number of user groups in Hightouch. (Users can belong to multiple groups, and when they do, they inherit the combined access from all of their assigned groups.)
Group assignments are refreshed only when a user logs in. This means that new
SSO groups will not appear for mapping until after a member belonging to that
group has logged into Hightouch. If you need to synchronize users and groups
without requiring login, consider [enabling SCIM](#configuring-scim).
15. All done! Members of your organization can access Hightouch by selecting **Log in with SSO**. You can also share your workspace's direct **Hightouch login URL**, which is available in the **Single sign-on** tab on the **Organization settings** page.
### Microsoft Entra ID
1. In the Microsoft Entra admin center, go to the **Enterprise applications** screen and select **New application**, then click **Create your own application**.
2. In your newly created app, select **Set up single sign on**.
3. Choose **SAML** as the sign-on method.
4. In Hightouch, open the **Single sign-on** tab in the **Organization settings** page. Click **Configure SAML SSO** to display a modal with the **Hightouch SSO URL** and **Audience URI**.
5. Configure the SAML settings in Microsoft Entra ID by entering the **Hightouch SSO URL** as the **Reply URL (Assertion Consumer Service URL)** and the **Audience URI** as the **Identifier (Entity ID).**
Leave the **Sign on URL** field empty if you plan to use IdP-initiated
sign-on. Alternatively, grab the value from the **Hightouch login URL** field in Hightouch and enter it here.
6. In Microsoft Entra ID, go to the **Attributes & Claims** section and add two claim mappings:
- **Name**: `email`
**Namespace** (optional): `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/email`
- **Name**: `name`
**Namespace** (optional): `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name`
As indicated above, the `Namespace` fields can be left blank.
The **Unique User Identifier (Name ID)** is the stable value the service provider uses to identify a user across login sessions. Set this to a unique, persistent value in Microsoft Entra (e.g.`userprincipalname`) to ensure consistent user recognition.
Example mapping section:
The **Source attributes** you choose for the `name` and `email` claims will depend on your Entra configuration; in the example above, the `displayname` attribute contains the user's **full name**, while the `mail` property contains their **work email**.
7. In the Hightouch modal from step 4, enter Microsoft Entra ID's **Login URL** as the **Identity provider SSO URL**, and upload the **Certificate Base64** as the **x.509 certificate** in Hightouch. Then, click **Save**.
8. At this point, you've completed the basic SAML SSO setup, allowing your users to log in to Hightouch through your identity provider. However, you'll still need to manually assign permissions for each user after they join your Hightouch organization. To streamline this process, we highly recommend setting up automatic group assignments. This ensures users have the right access to workspaces and resources as soon as they log in for the first time.
If SSO group mappings are not configured, new users won't be automatically
assigned to any group, which may result in limited access until group
assignments are manually set up.
9. In the Microsoft Entra admin center, go to **Enterprise applications** and select the Hightouch application you created earlier. Navigate to **Single Sign On configuration** and then click on **Attributes & Claims** again.
10. Select **Add a group claim**.
11. Make sure to choose `Groups assigned to the application` when defining your scope. That means **only** groups that are both assigned to the application and include the logging-in user as a member will be included in the SAML assertion.
If you select a different option, you won’t be able to use `Cloud-only group display names` as the **Source attribute** - this attribute is required to sync human-readable group names to Hightouch, as described in the [Entra documentation](https://learn.microsoft.com/en-us/entra/identity/hybrid/connect/how-to-connect-fed-group-claims#emit-cloud-only-group-display-name-in-token).
12. Next, set `Cloud-only group display names` as the **Source attribute**. This will send the `displayName` for each group associated with both the application and the user.
13. Optionally, you can also configure custom settings, such as filtering for groups whose `Display name` starts with a specific prefix, e.g. `Hightouch`.
For setups that are on-premises only (but synced to Entra) or hybrid (using both on-premises and cloud groups), you have two options for emitting group claims:
1. For on-premises groups, select an appropriate on-premises source attribute (such as `sAMAccountName`). For cloud-only groups, enable the `Emit group name for cloud-only groups` checkbox (see screenshot below). This configuration emits the selected on-prem group attribute for synced groups and the display name for cloud-only groups. Learn more about this setup in the [Entra documentation](https://learn.microsoft.com/en-us/entra/identity/hybrid/connect/how-to-connect-fed-group-claims?WT.mc_id=AZ-MVP-5003833#emit-cloud-only-group-display-name-in-token).
2. You can also choose to emit the `Group ID` for both group types. In this case, Entra group object UUIDs will appear in the **SSO group from identity provider** section of the **SSO mappings** settings in Hightouch.
14. Next, you'll want to go back to Hightouch and create mappings between the groups from Microsoft Entra and the corresponding user groups in Hightouch. Navigate to **Organization settings** and click on the **SSO mappings** tab.
15. On this page, each group from your identity provider can be mapped to any number of user groups in Hightouch. (Users can belong to multiple groups, and when they do, they inherit the combined access from all of their assigned groups.)
Group assignments are refreshed only when a user logs in. This means that new
SSO groups will not appear for mapping until after a member belonging to that
group has logged into Hightouch. If you need to synchronize users and groups
without requiring login, consider [enabling SCIM](#configuring-scim).
16. All done! Members of your organization can access Hightouch by selecting **Log in with SSO**. You can also share your workspace's direct **Hightouch login URL**, which is available in the **Single sign-on** tab on the **Organization settings** page.
Regardless of your exact Entra configuration, make sure that your Entra SAML user and group attribute mappings are aligned with your Entra SCIM user and group attribute mappings.
### Other identity providers
Hightouch supports all major identity providers, including OneLogin, Rippling, Google, Ping, and more.
The setup instructions are generally similar to those provided above for Okta. If you encounter any issues or need assistance, please don't hesitate to reach out to our support team—we're here to help!
## Configuring SCIM
You must [configure SAML SSO](#saml-single-sign-on) before configuring SCIM.
Unlike SAML SSO, which creates and updates user accounts only when users log in, SCIM automatically synchronizes user account changes from your identity provider to Hightouch without requiring user login. This means that adding, updating, or deactivating user accounts in your identity provider happens in the background and is immediately reflected in Hightouch.
To get started, generate a SCIM API token by following these steps:
1. Visit the **Organization settings** page in Hightouch and navigate to the **Single sign-on** tab.
2. Click **Generate SCIM token**.
3. Copy the generated bearer token and click **Save** to activate the token.
Hightouch displays the SCIM token only once, immediately after you generate
it. Once you close the modal, you won't be able to access the same token
again. If you don't copy it when it's first displayed, you'll need to generate
a new token by clicking **Refresh SCIM token**.
### Okta
After generating your SCIM token, follow these steps:
1. Navigate to the Hightouch application within your Okta admin panel.
2. On the **General** tab, locate the **App Settings** section and click the **Edit** button.
3. Check the box labeled **Enable SCIM provisioning**, then click **Save**.
4. Move to the **Provisioning** tab and click **Edit** in the **SCIM Connection** section.
5. Enter `https://api.hightouch.com/api/scim/v2` as the **SCIM connector base URL**.
6. Enter `userName` as the **Unique identifier field for users**.
7. Under **Supported provisioning actions**, ensure that the following options are selected:
- Import New Users and Profile Updates
- Push New Users
- Push Profile Updates
- Push Groups
8. Set the **Authentication Mode** to **HTTP Header**.
9. Paste the SCIM bearer token you generated in Hightouch into the authentication field, ensuring it includes the `Bearer` prefix.
10. Click **Save** to complete the configuration.
11. Finally, make sure to assign the relevant users and groups to the Hightouch application. In Okta, you must do this from both the **Assignments** tab and the **Push Groups** tab.
For additional information, or if your Okta admin panel differs from the steps
above, please refer to [Okta's documentation on SCIM
provisioning](https://help.okta.com/en-us/content/topics/apps/apps_app_integration_wizard_scim.htm?cshid=ext_Apps_App_Integration_Wizard-scim).
### Microsoft Entra ID
After generating your SCIM token in Hightouch, follow these steps:
1. Navigate to the **SAML SSO application** that you created for logging into Hightouch within the Microsoft Entra admin center.
2. Click on the **Provisioning** tab.
3. To configure SCIM for your application, click **Connect your application**.
4. Set the **Tenant URL** to `https://api.hightouch.com/api/scim/v2`.
5. Paste the SCIM bearer token you generated in Hightouch into the **Secret Token** field. Test the connection to ensure it's working, then click **Save**.
6. At this point, SCIM is configured for your application, but it isn't yet enabled. It's also recommended to set up user and group assignments.
7. Cick on the **Provisioning** tab under the **Manage** section. This will open a nested Provisioning section where you can configure **Users** and **Groups** attribute mappings.
8. Select **Provision Microsoft Entra ID Users** to configure SCIM attribute mappings for users.
9. A Hightouch user object has three key properties relevant to SCIM provisioning: _email_, _name_, and _active_. During provisioning, these fields should receive their values from the following SCIM Target Attributes:
- `emails[type eq "work"].value`
- `displayName`
- `active`
While it’s technically possible to use other Target Attributes (for example, sending a value for email under userName), doing so isn't recommended. Using the Target Attributes above ensures that user provisioning behaves as expected within Hightouch.
If your Entra configuration meets the following conditions:
- Emails are stored in the `mail` attribute and are **unique**.
- Full names are stored in the `displayName` attribute.
- You want to control user access by enabling or disabling accounts in Hightouch without immediately deleting them.
You should proceed with the following setup:
- First, uncheck the **userName** field as a required attribute in the **Edit attribute list for customappsso** section under **Show advanced options**, and save the change.
- For `emails[type eq "work"].value` as the **Target attribute**, use `mail` as the **Source attribute**. Set **Match objects using this attribute** to `Yes` and set **Matching precedence** to `1`.
- For `displayName` as the **Target attribute**, use `displayName` as the **Source attribute** as well.
- Leave `active` as the **Target attribute** mapped to the `Switch([IsSoftDeleted], , "False", "True", "True", "False")` formula as the **Source attribute**.
- Leave the **Primary Key** settings unchanged.
**Example of a valid user attribute mapping configuration**:
You can read more about SCIM with Entra - including the `active` / `IsSoftDeleted` behavior, and the meaning of **Match objects using this attribute**, **Matching precedence**, and **Primary Key** - in the [official Microsoft documentation](https://learn.microsoft.com/en-us/entra/identity/app-provisioning/customize-application-attributes).
10. As mentioned earlier, the exact **Source attribute** configuration depends on how your Entra directory is set up. Once this setup is complete, your **User Attributes Mapping** section is ready!
11. The **Group provisioning** section (`Provision Microsoft Entra ID Groups`) does not require any changes. The default configuration works out of the box. If you want to verify it, ensure that the `displayName` **Target attribute** is mapped to the `displayName` **Source attribute**, and that **Match objects using this attribute** is set to `Yes` with a **Matching precedence** of `1` for this mapping. Additionally, the `members` **Target attribute** should be mapped to the `members` **Source attribute**. Optionally, you can delete the `externalId` (**Target attribute**) to `objectId` (**Source attribute**) mapping.
**Example of a valid group attribute mapping configuration**:
As with the earlier example of the SAML group attribute configuration, this example SCIM setup is also optimized for a cloud-only Entra group environment; for an on-premises or hybrid group setup that includes both **on-premises groups synced to Entra** and **cloud-only groups in Entra**, you need to ensure that the **Source attribute** selected during SCIM provisioning for the `displayname` **Target attribute** fits your Entra configuration and also **aligns with your SAML attribute mapping**.
12. When provisioning **groups with members**, keep in mind that users added with this process **will not** automatically receive the intended Hightouch group assignment, as the SSO group mapping between Entra and Hightouch cannot yet exist at provisioning time.
Let’s assume the corresponding Hightouch group already exists. In that case, to ensure the user is correctly assigned to that group in Hightouch:
1. After the initial group-with-members provisioning, set up the relevant Entra → Hightouch SSO group mapping inside Hightouch.
2. Then, have the user log in to Hightouch via Entra using SAML SSO. The group assignment will be applied at that time (assuming the SAML attribute mapping is correctly configured, as [per our documentation](https://hightouch.com/docs/workspace-management/sso#microsoft-entra-id)).
### Other identity providers
Hightouch supports all major identity providers, including OneLogin, Rippling, Google, Ping, and more.
The setup instructions are generally similar to those provided above for Okta. If you encounter any issues or need assistance, please don't hesitate to reach out to our support team—we're here to help!
## Rollout, migration, and diagnostics best practices
These recommendations help reduce lockouts and access regressions during new SSO deployments, IdP migrations, and certificate rotations.
### Before rollout
- Use the **exact** Hightouch values for **SSO URL** and **Audience URI** from the SSO modal. Even minor typos can cause login loops.
- Plan your IdP group taxonomy and mapping strategy before production rollout.
- During initial validation, keep default access minimal and temporary until group mappings are verified.
### During migration or cutover
1. Recreate your SAML app in the new IdP/tenant with the same **Audience URI**, **NameID format/source**, and required claim mappings.
2. Update Hightouch with the new IdP SSO URL and signing certificate.
3. Generate a new SCIM token and connect SCIM in the new IdP app before disabling SCIM in the old app.
4. Validate with a small pilot (2-3 users), then expand to a full rollout.
Keep an admin validation path during cutovers, for example by testing in a
separate browser session before organization-wide rollout.
### Diagnostics
- Test in an incognito/private window to avoid cached session effects.
- Use a SAML tracer tool to inspect incoming NameID and group claims.
- If users can authenticate in your IdP but have no workspaces, verify SSO group mappings and that expected groups have been sent to Hightouch.
SSO disable/rollback is not self-serve in the app. If you need emergency SSO
disablement for recovery, contact Hightouch Support.
## FAQ
### Where can I find my organization identifier to log in with SSO?
After setting up SSO, you can share your workspace's direct **Hightouch login URL**, which is available in the **Single sign-on** tab on the **Organization settings** page.
The organization identifier is the part that appears after `/sso/`.
### I've enabled SSO in my workspace—how do I invite the rest of my team?
Once SSO is enabled, users no longer need a direct invitation to join your Hightouch organization. When they log in through your identity provider for the first time, Hightouch will automatically create an account for them.
### How do I allow or disallow non-SSO logins?
Go to the **Single sign-on** tab in the **Organization settings** screen. Here, you'll find a toggle labeled **Allow inviting users**. We typically recommend disabling this when using SSO to ensure that all users in your organization are managed through your central identity provider.
However, there may be situations where you'd want to allow non-SSO logins for external users, such as contractors or consultants. In these cases, you might choose to leave the setting on.
### When trying to login, I get redirected to app.hightouch.com/login.
If you see the **Log in to Hightouch** page instead of your Hightouch workspace after logging in with SSO, you need to make sure your IT team has appropriately set the **Audience URI** when [configuring SAML SSO for login](#saml-single-sign-on)).
### I don't see any workspaces after accepting an invitation.
There are a few possible reasons for this:
- If your workspace uses SSO, you shouldn't need to accept an invitation. Instead, use your organization's dedicated login link (shared with you directly or accessible through your identity provider). This ensures that you're logging in to the correct organization.
- If you land on the **Welcome to Hightouch** page, you may have accidentally created a separate Hightouch account. Keep in mind that **Log in with Google** and **Log in with Microsoft** do not connect to your company's central identity provider. To access your company's organization, be sure to select **Log in with SSO**.
- Confirm with your team that your account is mapped to at least one user group in Hightouch. Group assignments might be automatically inherited from your identity provider. Without being part of a user group, you won't have access to any workspaces.
### When I log in to Hightouch after configuring SSO, I am shown an error on the log in page.
In your SAML configuration:
- Check that you have mapped attributes for `name` and `email`. See the [SSO setup instructions](#okta-saml-sso-configuration) for more detailed steps.
- Ensure you used the correct Hightouch SAML URL and audience URL provided in your [dashboard](https://app.hightouch.com/settings/sso).
If you made changes to the SAML app in your identity provider between uploading your self-serve SAML settings, you can try to re-generate a certification and upload the new app settings on the Hightouch [**SSO** tab](https://app.hightouch.com/settings/sso).
### I see an error when logging into Hightouch after configuring SSO.
In your SAML configuration:
- Verify that the attributes for `name` and `email` are correctly mapped. Refer to the [SSO setup instructions](#okta-saml-sso-configuration) for detailed guidance.
- Make sure you are using the correct **Hightouch SSO URL** and **Audience URI** when configuring SAML SSO in your identity provider.
If you made changes to the SAML app in your identity provider after uploading your self-serve SAML settings, try re-generating a certificate and uploading it again.
### I see a duplicate user in Hightouch after logging in for the first time with SSO.
This is expected. SSO users are treated as separate from non-SSO users, so it's possible to have two users with the same email address (but different login methods). If you no longer need the original non-SSO user, you can manually delete it. This will not affect existing syncs or other resources.
### I don't see any groups on the SSO tab of my dashboard.
You will only see groups and their mapped roles on your **SSO** tab if your identity provider has been configured to send the `groups` attribute. Users may need to log out and log back in for changes to take effect.
### My group was updated in my organization's identity provider, but I still belong to the same group in Hightouch.
You will need to log out and log back in for changes to take effect.
### Why can't I remove certain group assignments in the Hightouch app?
Group assignments inherited from your identity provider cannot be overridden. However, you can still manually add additional users beyond those assigned by your identity provider.
### I see a "Matching user not found" error when adding users after enabling SCIM.
When SCIM is enabled, your identity provider needs to be configured to create users in Hightouch. For example, in Okta, you must go to the **Provisioning** tab and ensure your Hightouch integration app has permissions to **Create Users**, **Update User Attributes**, and **Deactivate Users**.
Once these settings are updated, delete any users showing the "Matching user not found" error and re-add them.
---
## Managing users and groups
**URL:** https://hightouch.com/docs/workspace-management/user-management
**Description:** Hightouch xxxx
**Section:** Workspace management
Before you begin, it's essential to understand the structure of a Hightouch organization. Access control is exclusively managed at the group level, so configuring users and groups should be your first step during the onboarding process.
This guide provides an overview of common user management tasks, such as [inviting users](#inviting-users), [assigning them to groups](#assigning-users-to-groups), and [managing group access](#granting-a-group-access-to-a-workspace) to workspaces. Once you've set up users and groups, you can proceed to [define custom roles](/workspace-management/roles) and manage more granular permissions.
If you're planning to connect Hightouch to your company's identity provider
(e.g., Okta), it's recommended to review the [single sign-on (SSO)
documentation](/workspace-management/sso) before inviting other users to your
organization. An identity provider can automate many of the following through
SAML SSO and SCIM.
## Users
To view a list of users in your organization, navigate to the **Organization settings** page and select the **Users** tab. There, you'll find a table of all users in your organization, including their authentication methods and group memberships. You can click on any user in the table to reveal more detailed information.
### Inviting users
If your organization is configured to use SSO, there's no need to manually invite users. Instead, go to **Organization settings** > **Single sign-on** and copy the **Hightouch login URL**. Share this link with your teammates so they can log in through your identity provider—no invite required.
For organizations not using SSO, you can invite others by going to **Organization settings** > **Users** and clicking the **Invite user** button. Only members of the **Organization admins** group can send invites. Make sure to assign the recipient to at least one user group so they receive workspace access upon their first login.
A list of pending invitations is available at the top of **Organization settings** > **Users**, where you can also withdraw invites if necessary. Otherwise, invites will expire automatically after 30 days.
### Removing users
Go to **Organization settings** > **Users**, select the user you wish to remove, and click **Remove user**.
Removing a user doesn't prevent them from rejoining your organization in the
future if they receive another invite or authenticate through your central
identity provider.
## Groups
A **group** is a collection of users. In Hightouch, access control is managed at the group level, making it easy to update permissions for many users at once.
Every Hightouch organization includes two built-in groups that cannot be deleted:
- **Organization admins** have full access to all workspaces and can manage organization-wide settings such as users, groups, billing, SSO, and more.
- **Organization viewers** have read-only access to all workspaces, but they do not have any admin privileges whatsoever.
Each organization must always have at least one admin, while the viewers group is optional and exists purely for convenience.
### Assigning users to groups
User-to-group mappings are best managed by your identity provider (e.g., Okta). However, if SSO group mappings aren't enabled or you need custom groups, you can manually set up user groups and assign users directly within the Hightouch app.
To manage group assignments for a single user, navigate to **Organization settings** > **Users**, select a user from the list, and use the **Assign** toggle for each group to add or remove the user.
To manage multiple members in a group, go to **Organization settings** > **Groups**, choose the group from the sidebar, and select the **Members** tab.
To add more users, click **Add users**; to remove existing users, check the boxes next to their names and click **Remove users**.
If group memberships are controlled by your identity provider via SSO group
mappings, you cannot override or make subtractive changes (e.g., removing
users from groups). However, you can make additive changes, such as assigning
additional members to an existing group.
### Creating a new group
Each group should represent a set of users with similar access to workspaces and their associated resources. For instance, you might create one group for “Data Science” and another for “Growth Marketing”. Groups can also be used to reflect different teams, regions, brands, or any other organizational structure that impacts permissions.
To simplify your onboarding, it's recommended to keep the number of groups to a minimum. Remember, users can belong to multiple groups at the same time, inheriting combined permissions from all their assigned groups. This can help reduce the total number of groups needed in your organization. For example, instead of creating a separate group for “Growth Marketing Analysts”, you might instead assign those users to both the “Data Science” and “Growth Marketing” groups.
To create a new group, go to **Organization settings** > **Groups** and click **Add group**. Name the group and select the members to include.
### Granting a group access to a workspace
By default, new groups don't have access to any workspaces. To grant access, navigate to **Organization settings** > **Groups**, select the group from the sidebar, and open the **Workspaces** tab. Use the **Role** dropdown to assign the appropriate access level for each workspace.
Roles determine the actions a group can perform within a workspace. You can select from built-in roles (e.g., “Workspace viewer”) or create a custom role for more fine-tuned control. For more information, refer to the [documentation for roles](/workspace-management/roles).
The **Access** column provides an overview of the group's permissions for each workspace, showing the actions they can take for each destination. While it gives a helpful summary, it doesn't display every possible grant or cover permissions related to other resource types.
## Transferring organization admin access
As people join, change roles, or leave your company, you may need to transfer **organization admin** privileges. Organization admins control organization-wide settings including SSO/SCIM configuration, billing, user management, and group permissions.
**Best practice:** Keep at least two active organization admins at all times. This prevents lockout situations if one admin leaves unexpectedly.
### Self-serve transfer
If at least one current org admin still has access to Hightouch, the transfer is fully self-serve.
**To add a new organization admin:**
1. If the new admin doesn't have a Hightouch account yet, go to **Organization settings** > **Users** and click **Invite user** to send them an invitation. Check **Organization admins** under "Assign to groups" to grant admin access immediately.
2. Go to **Organization settings** > **Groups** and select the **Organization admins** group.
3. In the **Members** tab, click **+ Add users** and add the new admin.
4. Click **Save changes**.
The new user now has full organization admin access.
**To remove a departing admin:**
1. In the **Organization admins** group, check the box next to the departing admin's name and click **Remove users**.
2. Click **Save changes**.
Hightouch requires at least one organization admin at all times. If you attempt to remove the last remaining admin, the operation will be blocked. Add a replacement admin first, then remove the departing one.
### Transfer via SSO/SCIM
If your organization uses SSO with SCIM provisioning, manage org admin membership through your identity provider instead of directly in Hightouch:
1. In your identity provider (for example, Okta or Entra ID), confirm you have a group mapped to Hightouch's **Organization admins** group.
2. Add the new admin to that IdP group and remove the departing admin.
3. SCIM will automatically sync the membership changes to Hightouch.
This approach keeps your identity provider as the single source of truth for admin access.
### When all admins have left
If no one in your company has organization admin access — for example, because all previous admins have left — you won't be able to make this change self-serve. In that case:
1. **Do not create a new organization.** Your existing organization, workspaces, and configuration are still intact.
2. Have someone from your company from a verified company email address.
3. In the request, include:
- Your **company name** and Hightouch **organization name or workspace URL**
- The **name and email** of the person who should become the new organization admin
- Any relevant context, such as when the previous admins departed
Because organization admins control billing and security settings, Hightouch may require additional verification — such as confirmation from an existing commercial or security contact on file — before assigning a new admin.
---
## Workspaces
**URL:** https://hightouch.com/docs/workspace-management/workspaces
**Description:** Learn how to create, manage, and delete Hightouch workspaces — including what each field on the creation form means, how to switch between workspaces, and common setup mistakes to avoid.
**Section:** Workspace management
*When you first sign up for Hightouch, or when your team needs an additional workspace, you'll use the workspace creation flow. This guide walks through when and how to create a workspace, what each field means, and common mistakes to avoid.*
Before creating a new workspace, make sure you actually need one. For most teams, a single production workspace is sufficient — Hightouch's [governance controls](/workspace-management/governance), including user groups, roles, Spaces, and approval flows, let multiple teams share a workspace safely. See [Do you need another workspace?](#do-you-need-another-workspace) below.
## Who should create workspaces
Workspace creation is restricted to users who have **organization admin** privileges. In most companies, this is the [**IT or platform admin**](/getting-started/it) — typically the person who also owns SSO configuration, billing, and user provisioning. If you're setting up Hightouch for the first time, the person who signs up becomes the first organization admin automatically.
If your organization uses SSO, workspace creation may be further restricted — in that case, contact your IT team or Hightouch administrator.
If you see the message **"SSO users are not allowed to create workspaces"**, your organization has disabled self-serve workspace creation. Reach out to your platform admin to have them create the workspace on your behalf.
## First sign-in
What you see when you first log in to Hightouch depends on whether your company already has an organization set up.
### New company
If no one at your company has used Hightouch before, sign up at [app.hightouch.com](https://app.hightouch.com) using Google, Microsoft, or email.
After signing up, you'll be prompted to:
1. **Create an organization** — this represents your company and is the container for billing, SSO, and user management.
2. **Create your first workspace** — using the form described below.
After creating your workspace, the **Get started** checklist guides you through connecting a data source, adding destinations, and inviting teammates.
### Existing company
If your company already has an organization and workspaces in Hightouch, you should **not** create a new organization. Instead:
- Ask an existing admin to **invite you** to the appropriate workspace, or
- Log in through your company's **SSO provider**, which will route you to the correct organization and workspace automatically.
If you were invited to Hightouch but land on the workspace creation screen instead of your team's workspace, **stop and contact the person who invited you**. Creating a new organization or workspace in this situation will result in a duplicate that needs to be cleaned up.
## Workspace creation form
The creation form has four fields: **Organization**, **Workspace name**, **Workspace slug**, and **Region**.
### Organization
The **Organization** dropdown lets you select which organization the new workspace belongs to.
- **New organization** — if you're the first person at your company signing up for Hightouch, select **New organization**. You'll provide your organization name (typically your company's legal or common name, such as "Acme Inc."), and Hightouch will create the organization and make you its first organization admin.
- **Existing organization** — if your company already has an organization, select it from the dropdown. You'll only see organizations where you have permission to create workspaces.
Most companies should have **exactly one** organization. Multiple organizations are only appropriate when you operate truly separate legal or identity domains — for example, a holding company with independent subsidiaries that don't share users, billing, or SSO configuration.
If your company already uses Hightouch, do not select **New organization**. This creates a duplicate organization that will need to be cleaned up. Instead, ask an existing admin to invite you, or log in via SSO.
### Workspace name
The name appears in the workspace switcher and helps your team identify the workspace. Choose something descriptive:
- **Single workspace:** Your company name, or something like ` Production`.
- **Production + sandbox:** ` Production` and ` Sandbox` (or `Playground`).
- **Multiple workspaces:** Use a pattern that makes the distinction clear, such as ` – US` and ` – EU`, or `` and ``.
You can change the workspace name later in **Settings → Workspace**.
### Workspace URL/slug
The slug becomes part of your workspace URL: `app.hightouch.com/`. It must be globally unique and is auto-generated from the workspace name.
Treat the slug as **permanent**. While workspace admins can technically update it later in **Settings → Workspace**, changing the slug breaks all existing bookmarked URLs, saved links, scripts, and integrations that reference the old slug — and there is no automatic redirect. Choose carefully before submitting the form.
### Region and cloud provider
The region determines where Hightouch processes your data. You'll select both a **geographic region** (United States, European Union, Asia Pacific, or Middle East) and a **cloud provider** (AWS, Google Cloud, or Azure). The available cloud providers vary by region.
When choosing a region:
- **Prefer the region closest to your primary data warehouse** and your end users to reduce latency.
- **Consider data residency requirements.** If compliance mandates that data stay within a specific jurisdiction, select a region in that jurisdiction.
- **Check feature availability.** Not all features are available in every region. Refer to the [Regions](/security/regions) documentation for a current compatibility matrix.
Region cannot be changed after workspace creation. If you create a workspace in the wrong region, you'll need to create a new workspace in the correct region and migrate your configuration. if you need assistance.
Some newer regions may be in limited availability. If the region you need isn't visible in the dropdown, to check availability and have our team help provision your workspace.
## After creation
Once your workspace is created, Hightouch presents a **Get started** checklist that walks you through initial setup:
1. **Invite teammates** — add collaborators so they can help configure the workspace.
2. **Set up storage bucket** *(Business tier only)* — bring your own S3, GCS, or Azure Blob Storage bucket for data-at-rest. This step only appears for Business tier workspaces. See [Storage](/security/storage).
3. **Connect to your data warehouse** — set up access to your data source. See [Sources](/sources/overview).
4. **Add destinations** — connect your marketing, sales, or other downstream tools. See [Destinations](/destinations/overview).
The exact steps shown may vary depending on your plan tier and workspace configuration.
## Switching between workspaces
If you have access to multiple workspaces, you can switch between them in two ways:
- **Workspace switcher** — click the workspace name in the top-left corner of the sidebar to open the workspace dropdown, or press **S** then **W** anywhere in the app. You can also open the command bar with **Cmd+K** (Mac) / **Ctrl+K** (Windows) and type "Switch workspace." Search by name or slug and select the workspace you want.
- **Direct URL** — navigate to a URL containing another workspace's slug (for example, `app.hightouch.com/other-workspace/syncs`). Hightouch will automatically switch your active workspace to match.
## Deleting a workspace
There is no self-serve option to delete a workspace. You can find the deletion option in **Settings → Workspace**, but it routes you to Hightouch support.
If you need to remove a workspace — for example, one created by mistake — . Before submitting your request, remove or disconnect all sources, destinations, syncs, and other resources from the workspace. All current organization or workspace admins will need to approve the deletion.
## Do you need another workspace?
Before creating an additional workspace, consider whether [Environments](/workspace-management/environments) or other governance controls would be a better fit.
**Use Environments when:**
- You need dev / staging / production promotion for the same data and teams.
- You want change control and approval flows without hard security isolation.
**Create a separate workspace when:**
- You have strict isolation requirements — regulatory, regional, or business-unit-level — where sources, destinations, and data must not be visible across boundaries.
- You need fully separate tenants under the same commercial relationship, such as independent subsidiaries.
| Consideration | Environments | Separate workspaces |
| :--- | :--- | :--- |
| Shared sources and destinations | Yes | No — each workspace has its own |
| Shared user roster | Yes | No — managed separately per workspace |
| Approval flows | Yes | Yes, but independent per workspace |
| Data isolation | Logical (same workspace) | Physical (nothing shared) |
| Typical use case | Dev → Staging → Prod | US region vs EU region |
For more on structuring your organization, see [Governance best practices](/workspace-management/governance).
## Troubleshooting
### I was invited but see the create-workspace screen
This usually means the invitation didn't link you to the correct organization or workspace. Don't create a new organization — instead, contact the person who invited you and ask them to re-send the invite or check that your account is correctly associated with their organization.
### I accidentally created a duplicate organization
If you signed up directly instead of using an invitation or SSO, you may have created a second organization for your company. to consolidate your organizations.
### I need to change my workspace region
Region cannot be changed self-serve after creation. to plan a migration. Before you reach out, be aware of the following:
- **Brief outage required.** The migration involves a short period of downtime for the affected workspace. Schedule it during low-traffic hours.
- **Tunnels must be recreated.** Existing [SSH tunnels](/security/ssh-tunneling) cannot be migrated between regions. You'll need to set up new tunnels in the target region after the migration.
- **IP allowlists need updating.** If your sources or destinations are behind a firewall that allowlists Hightouch's [IP addresses](/security/networking#ip-addresses), add the new region's IP range before the migration and remove the old range afterward.
- **Sync history and debugger data.** If you use [self-hosted storage](/security/storage#self-hosted-storage) or the [Lightning sync engine](/syncs/lightning-sync-engine), your sync history and CDC state are preserved. If you use Hightouch-managed storage with the standard sync engine, syncs will need a full resync after the migration and pre-migration debugger data may not carry over.
## Related resources
- [Core concepts](/workspace-management/overview)
- [Governance best practices](/workspace-management/governance)
- [Environments](/workspace-management/environments)
- [Regions](/security/regions)
- [Storage](/security/storage)
- [Managing users and groups](/workspace-management/user-management)
---
## Amazon Web Services
**URL:** https://hightouch.com/docs/security/aws
**Description:** Grant Hightouch access to resources within your AWS account.
**Section:** Security and compliance
## Overview
Hightouch's AWS integration powers several features:
- Workspace deployment on AWS (see [Regions](/security/regions#supported-regions))
- [Self-hosted storage via S3](/security/storage#self-hosted-storage)
- [Athena](/sources/amazon-athena) source
- [S3](/sources/s3) source
- [S3](/destinations/s3) destination
- [AWS PrivateLink](/security/aws/privatelink)
## Setup
To give Hightouch access to resources on your AWS account,
go to the [**Cloud providers**](https://app.hightouch.com/settings/cloud-providers) tab on the Settings page.
Click **Add cloud provider**, and select **Amazon Web Services**.
Give your credential a **Display name**, then select either **Cross-account role** or **Access key**.
### Cross-account role
Cross-account roles are a mechanism provided by AWS to allow you to grant secure access to Hightouch without requiring that you hand over sensitive secrets like Secret Access Keys.
The Hightouch UI displays an **AWS account ID** and a randomly generated **AWS external ID**, as shown below, which you need to enter into an IAM Role you create in your AWS account. Store these values for future reference.
Neither the Account ID nor the external ID are secrets, so you don't need to worry about securely storing them.
1. From your AWS console, navigate to **Identity and Access Management (IAM)** > **Access Management** > **Roles** and click **Create Role**.
2. Under **Select type of trusted entity**, choose **AWS account**. Then under **An AWS account**, select **Another AWS account**.
3. Enter the **Account ID** you copied from the Hightouch UI, select **Require external ID** and enter the external ID you copied from the Hightouch UI. Click **Next**.
4. Attach permissions to the role, choose a role name, then click **Create role**. The exact permission policies you attach depend on which Hightouch features you intend to use. Consult the documentation [for those services](#overview) for further guidance.
5. Copy the **Role ARN** from AWS IAM and paste it into the **Role ARN** field in Hightouch. Click **Create** to complete the process.
For more information, read [AWS's tutorial](https://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_cross-account-with-roles.html) on delegating access cross-account using IAM Roles.
### Access key
The **Access Key** access type allows you to configure Hightouch to use an IAM user by providing the user's **Access Key ID** and **Secret Access Key**.
If you need help generating these keys, consult the [IAM article](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) on this topic.
Once you have an Access Key ID and Secret Access Key, paste those values into the form and click **Create**.
## Tips and troubleshooting
{/* */}
### CredentialsError: Missing credentials in configuration
{/* */}
If you receive the following error `CredentialsError: Missing credentials in config`, it means Hightouch can't assume the role required by your Athena or S3 sync, or [external storage](/security/storage) configuration.
This error is often due to an issue with the [IAM trust policy](https://aws.amazon.com/blogs/security/how-to-use-trust-policies-with-iam-roles/). This policy defines which other AWS accounts can assume roles in your account. When you set up AWS credentials in Hightouch using a [cross-account role](#cross-account-role), we provide you with Hightouch's Account ID and an additional external ID. You should include Hightouch's Account ID in your role trust policy, and can optionally include the external ID.
To troubleshoot this error, follow these steps:
- Ensure that the role exists.
- Ensure that the role has a trust policy attached.
- Ensure that the role's trust policy includes our Hightouch AWS account ID (`324528010823`) as a principal.
- If the role trust policy includes an external ID check, ensure it matches the external ID generated by Hightouch. See the example below:
If you want to check the external ID but misplaced the external ID that Hightouch generates, you need to [re-create the cloud provider integration in Hightouch](#setup). Hightouch only displays this value once during cloud provider setup.
---
## AWS PrivateLink
**URL:** https://hightouch.com/docs/security/aws/privatelink
**Description:** Securely connect your warehouse to Hightouch using AWS PrivateLink.
**Section:** Security and compliance
Using [AWS PrivateLink](https://aws.amazon.com/privatelink/), you can securely connect your AWS-deployed services to Hightouch.
Network traffic between Hightouch and your services connected by AWS PrivateLink stays on the [AWS backbone network](https://aws.amazon.com/about-aws/global-infrastructure/global_network/) and doesn't travel over the public internet. That means that you can restrict public access to your services, and your data is never exposed to the public internet.
Hightouch supports managed AWS PrivateLink integrations for [Snowflake](/sources/snowflake), [Redshift](/sources/amazon-redshift), [Databricks](/sources/databricks), [Amazon MSK](/events/sources/kafka), and other integrations with [VPC Endpoint Service](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-share-your-services.html).
## Snowflake
Hightouch supports connecting to your Snowflake account via AWS PrivateLink.
### Prerequisites
- Hightouch supports AWS PrivateLink on [Business tier](https://hightouch.com/pricing) accounts.
- Snowflake supports AWS PrivateLink on [Business Critical](https://docs.snowflake.com/en/user-guide/intro-editions.html#business-critical-edition) accounts or higher.
- Your Snowflake account must be located in one of the following AWS regions: `us-east-1`, `us-west-2`, `eu-west-1`, `ap-south-1`, or `ap-southeast-2`
- Your Hightouch workspace must be located in the AWS `us-east-1`, `eu-west-1`, or `ap-south-1` region.
### Getting started
Please to our customer success team to get started with AWS PrivateLink for Snowflake.
### Learn more
Check out these AWS and Snowflake docs to learn more:
- [What is AWS PrivateLink?](https://docs.aws.amazon.com/vpc/latest/privatelink/what-is-privatelink.html)
- [AWS PrivateLink & Snowflake](https://docs.snowflake.com/en/user-guide/admin-security-privatelink.html)
## Databricks
Hightouch can use AWS PrivateLink to connect to your Databricks workspace control plane. This allows Hightouch to securely interact with your Databricks workspace without exposing your data to the public internet.
Hightouch uses Databricks "Front-end PrivateLink", which is distinct from Databricks "Back-end PrivateLink", which is used to connect the Databricks control plane to a compute plane running in your AWS VPCs.
### Prerequisites
- Hightouch supports AWS PrivateLink on [Business tier](https://hightouch.com/pricing) accounts.
- Your Hightouch workspace must be located in the AWS `us-east-1`, `eu-west-1`, or `ap-south-1` region.
- Your Databricks workspace must be located in one of the following AWS regions: `us-east-1`, `us-west-2`, `eu-west-1`, `ap-south-1`, or `ap-southeast-2`
- Your Databricks workspace must have been created with a "Private Access Settings" object. Databricks does not support setting a Private Access Settings object on existing workspaces. If you need to use PrivateLink with an existing workspace, you will need to create a new workspace with the appropriate settings.
### Getting started
Please to our customer success team to get started with AWS PrivateLink for Databricks.
### Learn more
- [Enable AWS PrivateLink](https://docs.databricks.com/aws/en/security/network/front-end/front-end-private-connect)
## Redshift
Hightouch supports connecting to both provisioned Redshift clusters and Redshift Serverless workgroups via PrivateLink with [Redshift-managed VPC Endpoints](https://docs.aws.amazon.com/redshift/latest/mgmt/managing-cluster-cross-vpc.html).
### Prerequisites
- Hightouch supports AWS PrivateLink on [Business tier](https://hightouch.com/pricing) accounts.
- Your Redshift cluster or workgroup must be located in one of the following AWS regions: `us-east-1`, `us-west-2`, `eu-west-1`, `ap-south-1`, or `ap-southeast-2`
- Your Hightouch workspace must be located in the AWS `us-east-1`, `eu-west-1`, or `ap-south-1` region.
**For provisioned clusters:**
- Your Redshift cluster must be of the `RA3` node type.
- Your Redshift cluster must have [Cluster Relocation](https://docs.aws.amazon.com/redshift/latest/mgmt/managing-cluster-recovery.html) or Multi-AZ enabled.
### Getting started
Please to our customer success team to get started with AWS PrivateLink for Redshift.
### Learn more
- [What is AWS PrivateLink?](https://docs.aws.amazon.com/vpc/latest/privatelink/what-is-privatelink.html)
- [Redshift-managed VPC endpoint documentation](https://docs.aws.amazon.com/redshift/latest/mgmt/managing-cluster-cross-vpc.html)
- [Connecting to Amazon Redshift Serverless](https://docs.aws.amazon.com/redshift/latest/mgmt/serverless-connecting.html#serverless-cross-vpc)
## Amazon MSK
Hightouch supports [Multi-VPC private connectivity](https://docs.aws.amazon.com/msk/latest/developerguide/aws-access-mult-vpc.html) for Amazon Managed Streaming for Apache Kafka (MSK). This is a managed solution that handles the networking infrastructure automatically, without requiring you to set up a Network Load Balancer or VPC Endpoint Service.
### Prerequisites
- Hightouch supports AWS PrivateLink on [Business tier](https://hightouch.com/pricing) accounts.
- Your MSK cluster must be running Apache Kafka 2.7.1 or later.
- Your MSK cluster must use `SASL_IAM` or `SASL_SCRAM` authentication.
- Your MSK cluster cannot use the `t3.small` instance type.
- You must [turn on multi-VPC connectivity](https://docs.aws.amazon.com/msk/latest/developerguide/mvpc-cluster-owner-action-turn-on.html) for your cluster.
- Your MSK cluster must be located in one of the following AWS regions: `us-east-1`, `us-west-2`, `eu-west-1`, `ap-south-1`, or `ap-southeast-2`
- Your Hightouch workspace must be located in the AWS `us-east-1`, `eu-west-1`, or `ap-south-1` region.
### Getting started
Please to our customer success team to get started with AWS PrivateLink for Amazon MSK.
### Learn more
- [Amazon MSK multi-VPC private connectivity](https://docs.aws.amazon.com/msk/latest/developerguide/aws-access-mult-vpc.html)
## Other integrations
Hightouch can connect to any source or destination exposed via a [VPC Endpoint Service](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-share-your-services.html). This allows you to expose any service running in your VPC to Hightouch without exposing it to the public internet.
### Prerequisites
- Hightouch supports AWS PrivateLink on [Business tier](https://hightouch.com/pricing) accounts.
- Your integration (source / destination) must be exposed via a [VPC Endpoint Service](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-share-your-services.html). This generally requires a [Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) to sit in front of your service.
- Your VPC Endpoint Service must be located in one of the following AWS regions: `us-east-1`, `us-west-2`, `eu-west-1`, `ap-south-1`, or `ap-southeast-2`
- Your Hightouch workspace must be located in the AWS `us-east-1`, `eu-west-1`, or `ap-south-1` region.
### Getting started
Please to our customer success team to get started with AWS PrivateLink for VPC Endpoint Services.
### Learn more
- [What is AWS PrivateLink?](https://docs.aws.amazon.com/vpc/latest/privatelink/what-is-privatelink.html)
- [What is a Network Load Balancer?](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html)
- [Share your services through AWS PrivateLink](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-share-your-services.html)
---
## Microsoft Azure
**URL:** https://hightouch.com/docs/security/azure
**Description:** Grant Hightouch access to resources within your Azure account.
**Section:** Security and compliance
## Overview
Hightouch's Azure integration powers several features:
- Workspace deployment on Azure (see [Regions](/security/regions#supported-regions))
- [Self-hosted storage via Azure Blob Storage](/security/storage#self-hosted-storage)
- [Azure Blob Storage](/sources/azure-blob) source
- [Azure Blob Storage](/destinations/azure-blob) destination
- [Azure Private Link](/security/azure/private-link)
## Configure Azure
Azure configuration includes creating an app registration and granting access to Blob Storage. Once you've done both of these, you can use your Blob Storage information and Azure credentials to connect Hightouch.
### Create an app registration
1. In the [Azure Portal](https://portal.azure.com/), navigate to **Azure Active Directory**.
2. Locate and click **Manage** > **App registrations** in the left-hand menu.
3. Create a new app registration by selecting **+ New registration**. Give the application a descriptive **Name**, for example, "Hightouch-integration-user," and leave the **Supported account type** as **Accounts in this organizational directory only (Default Directory only - Single tenant)**. You don't need a redirect URI for Hightouch storage integration. Click **Register** when done.
4. Next, you need an app secret for this account. While on the app registration screen, navigate to **Manage** > **Certificates & secrets** in the left-hand menu.
In the **Client secrets** tab, click **New client secret**. Enter a **Description** and **Expires** time frame. Click **Add** to create the secret.
Make sure to select a sufficient expiry time frame. If this credential expires, Hightouch can't
access your Azure Blob Storage, and if your workspace is configured to use Azure for log storage, all syncs will fail.
5. Copy and store the secret **Value**, _not_ the **Secret ID**, after you create it. You won't be able to read the value again later.
6. Navigate to the App **Overview** in the left-hand menu. Copy and save the **Application (client) ID** and **Directory (tenant) ID**. You need both of these and the Client secret value
to configure Hightouch.
### Grant access to Blob Storage
1. In the [Azure Portal](https://portal.azure.com/), navigate to your storage account, and then to the Blob Storage container you want to connect to Hightouch. If you don't have a container, [create one](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-portal#create-a-container).
2. In the configuration for your Blob Storage container, go to **Access Control (IAM)** in the left-hand menu.
3. Select **+Add** then **Add role assignment**.
4. Select the role you want to grant to the Hightouch integration. To use the integration for storage, we suggest
using the **Storage Blob Data Contributor** role.
If you want to assign more granular permissions, refer to the for [storage](/security/storage) documentation
and consult your Active Directory administrator.
Once you've selected the role, click **Next** to assign **Members** to the role.
4. Keep **User, group, or service principal** as the access assignment. Click **+ Select members** and search for and select the Hightouch integration user you created during [app registration](#create-an-app-registration).
6. Click **Review + assign** to confirm. The Hightouch user should now have access to your Azure Blob Storage.
## Configure Hightouch
To configure Hightouch to use the Azure integration user,
go to the [**Cloud providers**](https://app.hightouch.com/settings/cloud-providers) tab on the Settings page. Click **Add cloud provider**, and select **Microsoft Azure**.
Give your credential a **Display name**, and enter the **Tenant ID**, **Client ID**, and **Client secret** from your Azure configuration.
## Azure credential usage
Once you've set up your Microsoft Azure credentials in Hightouch, you can select Microsoft Azure when setting up [external storage](/security/storage).
---
## Azure Private Link
**URL:** https://hightouch.com/docs/security/azure/private-link
**Description:** Securely connect to Hightouch using Azure Private Link.
**Section:** Security and compliance
Using [Private Link](https://learn.microsoft.com/en-us/azure/private-link/private-link-overview), you can securely connect your Azure-deployed services to Hightouch.
Network traffic between Hightouch and your services connected by Azure Private Link stays on the Microsoft backbone network and doesn't travel over the public internet. That means that you can restrict public access to your services, and your data is never exposed to the public internet.
Hightouch currently supports Azure Private Link for the following services:
- Azure Databricks
- Azure Blob Storage
- Snowflake (Azure-hosted)
## Azure Databricks
### Prerequisites
- Hightouch supports Azure Private Link on [Business tier](https://hightouch.com/pricing) accounts.
### Getting started
Please to our customer success team to get started with Azure Private Link for Azure Databricks.
## Azure Storage
### Prerequisites
- Hightouch supports Azure Private Link on [Business tier](https://hightouch.com/pricing) accounts.
### Getting started
Please to our customer success team to get started with Azure Private Link for Azure Storage.
## Snowflake (Azure-hosted)
### Prerequisites
- Hightouch supports Azure Private Link on [Business tier](https://hightouch.com/pricing) accounts.
### Getting started
Please to our customer success team to get started with Azure Private Link for Snowflake.
#### Setup steps
1. Run [this command](https://docs.snowflake.com/en/sql-reference/functions/system_get_privatelink_config) in Snowflake to retrieve your PrivateLink config:
```sql
SELECT SYSTEM$GET_PRIVATELINK_CONFIG();
```
2. Once verified, Hightouch will connect to your Snowflake destination privately via Azure Private Link.
For reference, see [Snowflake’s documentation](https://docs.snowflake.com/en/user-guide/privatelink-azure#configure-access-to-snowflake-with-azure-private-link) on Azure Private Link setup.
### Learn more
- [What is Azure Private Link?](https://learn.microsoft.com/en-us/azure/private-link/private-link-overview)
- [Configure private connectivity to Azure Databricks](https://learn.microsoft.com/en-us/azure/databricks/security/network/front-end/front-end-private-connect#private-connectivity-to-azure-databricks-overview)
- [Use private endpoints for Azure Storage](https://learn.microsoft.com/en-us/azure/storage/common/storage-private-endpoints)
---
## Google Cloud Platform
**URL:** https://hightouch.com/docs/security/gcp
**Description:** Grant Hightouch access to resources within your Google Cloud project.
**Section:** Security and compliance
## Overview
Hightouch's Google Cloud integration powers several features:
- Workspace deployment on Google Cloud (see [Regions](/security/regions#supported-regions))
- [Self-hosted storage via GCS](/security/storage#self-hosted-storage)
- [BigQuery](/sources/google-bigquery) source
- [Google Cloud Storage](/sources/gcs) source
- [Google Cloud Storage](/destinations/gcs) destination
- [Google Private Service Connect](/security/gcp/private-service-connect)
## Setup
To give Hightouch access to resources on your GCP account,
go to the [**Cloud providers**](https://app.hightouch.com/settings/cloud-providers) tab on the Settings page.
Click **Add cloud provider**, and select **Google Cloud Platform**.
Give your credential a **Display name**, then select either **Service account managed by Hightouch** or **Bring your own service account**.
### Managed service account
When using the managed service account access type, Hightouch creates a service account within our secure Google Cloud account, to which you can bind your project's IAM policies. IAM policies grant permissions within your Google Cloud project.
When you click **Create a new service account**, Hightouch generates a new service account in our project. The service account is unique to your workspace and this credential. You can then copy the service account email and click **Create** to confirm the service account creation.
Next, you need to bind IAM roles to this service account by calling [`gcloud projects add-iam-policy-binding`](https://cloud.google.com/sdk/gcloud/reference/projects/add-iam-policy-binding) via the Google Cloud CLI. The specific roles you grant depend on the types of resources you want to grant Hightouch access to, such as [Cloud Storage](/security/storage) or [BigQuery](/sources/google-bigquery). Consult the relevant docs for more information on the specific permissions Hightouch needs.
### Bring your own service account
When you **Bring your own service account** you upload your service account's key, which Hightouch securely stores.
First, you need to [create a service account](https://cloud.google.com/iam/docs/creating-managing-service-accounts), [grant it the relevant roles](https://cloud.google.com/iam/docs/granting-changing-revoking-access), and [generate a key](https://cloud.google.com/iam/docs/creating-managing-service-account-keys). Make sure you create **JSON** key, not P12. Once the service account and key have been created, download the key to your machine and drag and drop it into the file selector.
Hightouch then parses the key, and if it's valid, you see the **Client ID** and **Client Email** fields populated. If those correspond with your intended service account, click **Create**.
Make sure the service account you attached has the specific roles you need.
These permissions depend on the types of resources you want to grant Hightouch access to, such as [Cloud Storage](/security/storage) or [BigQuery](/sources/google-bigquery).
Consult the relevant docs for more information on the specific permissions Hightouch needs.
## Google Cloud credential usage
Once you've set up your Google Cloud credentials in Hightouch, you can now use them throughout the app, for example for [external storage](/security/storage), the [BigQuery](/sources/google-bigquery) and [GCS](/sources/gcs) sources, and other Google Cloud integrations like [Google Ad Manager](/destinations/gam360).
Regardless of whether you use a managed service account or being your own, ensure it has required permissions.
Consult the relevant docs for more information on the specific permissions Hightouch needs.
---
## Private Service Connect
**URL:** https://hightouch.com/docs/security/gcp/private-service-connect
**Description:** Securely connect your warehouse to Hightouch using Google Cloud Private Service Connect.
**Section:** Security and compliance
Using [Private Service Connect](https://cloud.google.com/vpc/docs/private-service-connect), you can securely connect your Google Cloud-deployed services to Hightouch.
Network traffic between Hightouch and your services connected by Private Service Connect stays on the Google Cloud network and doesn't travel over the public internet. That means that you can restrict public access to your services, and your data is never exposed to the public internet.
## Cloud SQL
Hightouch supports Google Cloud Private Service Connect for Cloud SQL instances (MySQL, Postgres, SQL Server).
### Prerequisites
- Hightouch supports GCP Private Service Connect on [Business tier](https://hightouch.com/pricing) accounts.
- Your Hightouch workspace must be hosted on **GCP** (any Hightouch GCP region).
- Your Cloud SQL instance must be in the GCP `us-east1`, `us-east4`, or `us-west2` regions. If you're interested in using a different region, please reach out to our customer success team.
- Your Cloud SQL instance must have Private Service Connect enabled.
### Getting started
You will need to enable Private Service Connect on your Cloud SQL instance. You can do this by following the documentation for [MySQL](https://cloud.google.com/sql/docs/mysql/private-service-connect), [PostgreSQL](https://cloud.google.com/sql/docs/postgres/private-service-connect), or [SQL Server](https://cloud.google.com/sql/docs/sqlserver/configure-private-service-connect#create-cloud-sql-instance-psc-enabled). You will also need to authorize the Hightouch Google Cloud Project to create a Private Service Connect endpoint to your Cloud SQL instance.
Please to our customer success team to get started with GCP Private Service Connect for Cloud SQL.
### Learn more
- [What is Private Service Connect?](https://cloud.google.com/vpc/docs/private-service-connect)
- [Configure Private Service Connect - MySQL](https://cloud.google.com/sql/docs/mysql/configure-private-service-connect)
- [Configure Private Service Connect - PostgreSQL](https://cloud.google.com/sql/docs/postgres/configure-private-service-connect)
- [Configure Private Service Connect - SQL Server](https://cloud.google.com/sql/docs/sqlserver/configure-private-service-connect)
---
## Networking
**URL:** https://hightouch.com/docs/security/networking
**Description:** Allow Hightouch to connect to your warehouse, destinations, and other internal services.
**Section:** Security and compliance
Hightouch can connect directly to services running within your private network, including your data warehouse, self-hosted destinations, and even tools like a [private Git server](/extensions/git-sync) or your own [hosted Looker deployment](/sources/looker).
For added security, considering using an [SSH tunnel](/security/ssh-tunneling) to layer on public key-based authentication and encryption.
## IP addresses
Many private networks need to be configured to allow external connections. You can provide Hightouch access by adding Hightouch's IP addresses to your firewall allowlist.
Ensure you configure your network firewall rules to allow access from _only_ the IP addresses corresponding to the [region](/security/regions) you selected when you created your Hightouch workspace.
### Amazon Web Services
| Region | IP Addresses |
|------------------------------|------------------------------------------------------------------------------------------------|
| ap-south-1 (Mumbai, India) | `13.126.190.7` `65.1.90.146` |
| eu-west-1 (Ireland) | `34.255.226.212` `52.16.84.199` `52.211.114.138` |
| us-east-1 (N. Virginia, USA) | `54.196.30.169` `52.72.201.213` `18.213.226.96` `3.224.126.197` `3.217.26.199` |
### Google Cloud Platform
| Region | IP Addresses |
|------------------------------------|------------------------------------------------------------|
| europe-west1 (Belgium) | `35.241.218.183` `35.195.167.179` `104.199.61.185` |
| me-central2 (Dammam, Saudi Arabia) | `34.166.244.44` `34.166.247.32` `34.166.115.20` |
| us-east4 (N. Virginia, USA) | `34.145.155.148` `35.245.150.71` `35.245.58.91` |
### Microsoft Azure
| Region | IP Addresses Ranges |
|-----------------------|---------------------|
| East US (N. Virginia) | `23.100.16.164/30` |
---
## Security
**URL:** https://hightouch.com/docs/security/overview
**Description:** Learn about Hightouch's security practices and infrastructure.
**Section:** Security and compliance
[**Explore the interactive architecture diagram →**](https://architecture.hightouch.com/?from=docs&product=reverse-etl&utm_source=hightouch-docs&utm_medium=referral&utm_campaign=architecture-explorer&utm_content=reverse-etl)
See how data moves through Hightouch — including security controls, private networking, and storage boundaries.
## Overview
At Hightouch, we work with large healthcare, financial, and public enterprises to sync sensitive customer data to downstream tools and systems. As such, we're committed to building a data activation tool that's held to the [highest possible security standards](https://hightouch.com/platform/security).
This document gives an overview of our security processes and infrastructure architecture. It outlines your options for data transfer and storage, including implementing Hightouch so that none of your data is stored at rest on Hightouch's infrastructure.
## Compliance
Hightouch complies with SOC 2 Type 2, HIPAA, GDPR, amd CCPA. Hightouch is also ISO 27001 certified by Schellman, an accredited certification body.
In addition to these certifications, we follow these security best practices:
- Automated vulnerability scanning in our platform
- Regular third-party penetration testing—please for the latest report
### SOC 2 Type 2 audit report
If you are an existing Hightouch customer, or ping us in your dedicated Slack channel to request our SOC 2 audit report. If you are trialing Hightouch, your point of contact can provide you with the SOC 2 audit report under NDA.
## Architecture overview
Hightouch can connect to your warehouse or data [source](/getting-started/concepts#sources) directly or via an [SSH tunnel](/security/ssh-tunneling) or [reverse SSH tunnel](/security/ssh-tunneling#reverse-tunneling). A direct connection is suitable for most use cases. If your source is on a private network or virtual private cloud (VPC), you may need to set up a tunnel, [AWS PrivateLink](/security/aws/privatelink), or [GCP Private Service Connect](/security/gcp/private-service-connect).
Once you've connected Hightouch to your data source, you need to create [models](/getting-started/concepts#models) that define what data to pull from your source. These data models are purely SQL definitions and do **not** store data.
A [sync](/getting-started/concepts#syncs) declares how to map data to a destination and how often it should run. When a Hightouch sync initiates, Hightouch:
- executes the SQL query associated with your model on your source
- identifies the incremental changes to send to the associated downstream tool
- and translates these rows to the appropriate APIs
For more information on your infrastructure options in this process, refer to the [change data capture section](#change-data-capture-architecture).
Customer data flows through Hightouch infrastructure only during an "active sync." The data is encrypted in transit via [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security) as it's flowing through Hightouch. We don't expose our compute instances to the internet, and we secure our infrastructure according to cloud security best practices.
### Transit details
Hightouch uses the [Elastic Load Balancer](https://aws.amazon.com/elasticloadbalancing/) within AWS to distribute incoming traffic across multiple healthy targets and increase application and network availability. To enhance the security of our Application Load Balancers (ALBs) and Network Load Balancers (NLBs), we require that all load balancers that accept HTTPS traffic use at least [TLS 1.2](https://en.wikipedia.org/wiki/Transport_Layer_Security#tls1.2).
Older TLS versions and legacy SSL protocols have known fatal security flaws and don't provide protection for data in transit. Hightouch also follows the [AWS Well-Architected Framework](https://aws.amazon.com/architecture/well-architected/) to achieve best practices related to operational excellence, security, reliability, performance efficiency, and cost optimization.
We use the following best practices for protecting data in transit:
- Enforcing encryption in transit
- Authenticating network communications
- Automating detection of unintended data access
- Using AWS PrivateLink if required
### Data storage
After sending data downstream, Hightouch stores full request/response payloads in a cloud storage bucket. By default, this storage bucket is on Hightouch-managed infrastructure, but you can [use your own cloud bucket](#bring-your-own-bucket) within your infrastructure. Our [live debugger](https://hightouch.com/blog/observability-better-than-ever) queries this cloud bucket for row-level debugging, and if you're self-hosting these logs, you can set custom retention policies.
## Change data capture architecture
To prevent making excessive API requests and send only necessary updates to your destinations, Hightouch uses a process called **change data capture** (CDC) or diffing. For details on this process, check out the [change data capture docs](/getting-started/concepts#change-data-capture).
Two different methods are available to support this process, and it's your choice which one you'd like to use:
- The [**Lightning sync engine**](/syncs/lightning-sync-engine) does the diff computation within your data warehouse. To allow this, Hightouch needs `WRITE` access to specific tables in your warehouse that store metadata from previous syncs. In-warehouse computation allows for faster syncs at higher volumes, and customers syncing over 1 million rows typically use the Lightning sync engine for performance reasons.
- The **Basic sync engine**, also known as "local diffing," is the default method. The diff computation is done on Hightouch infrastructure, so it has slightly slower sync speeds but has the benefit of offloading some compute from your warehouse and not requiring `WRITE` access.
## Bring your own bucket
On our [self-service plans](https://hightouch.com/pricing), Hightouch stores your query results in an encrypted cloud bucket on your behalf with a 30-day retention policy.
On our [business-tier plans](https://hightouch.com/pricing), you can [host your own bucket](/security/storage) on your AWS, Google Cloud, or Azure instance for more control over your data. Please
{" "}
for details on getting access to this feature.
## Personal identifiable information redaction
Sometimes, you need to sync Personal identifiable information (PII) to a destination, but you don't want your entire team to have access to it. For example, you may need to sync PII to ad platforms for the most accurate targeting. You may also want your marketing team to build [audiences](/customer-studio/overview) for these destinations, but want to keep them from having access to the PII of those audiences.
When configuring your [audience models](/customer-studio/schema#columns), you can redact specific columns to hide their values from team members building and previewing audience membership.
When a team member previews a user using [Audience Insights tools](/customer-studio/insights), they see `` or a customized message of your choosing, for example, `Field redacted`.
---
## Regions
**URL:** https://hightouch.com/docs/security/regions
**Description:** You can configure Hightouch to run in multiple regions around the world.
**Section:** Security and compliance
## Overview
Your Hightouch region defines where we host the Hightouch _data plane_, which is the component of our system that directly interacts with your [sources](/getting-started/concepts#sources) and [destinations](/getting-started/concepts#destinations).
## Supported regions
Hightouch operates in the following Amazon Web Services, Google Cloud Platform, and Azure regions:
| Cloud | Region | Location |
|-------|--------------|:---------------------|
| AWS | ap-south-1 | Mumbai, India |
| AWS | eu-west-1 | Ireland |
| AWS | us-east-1 | N. Virginia, USA |
| Azure | East US | N. Virginia, USA |
| GCP | europe-west1 | Belgium |
| GCP | me-central2 | Dammam, Saudi Arabia |
| GCP | us-east4 | N. Virginia, USA |
You may prefer to use a particular region for a variety of reasons, including data residency requirements, availability of features like [AWS PrivateLink](/security/aws/privatelink) or [Private Service Connect](/security/gcp/private-service-connect), or use of a self-managed storage bucket in Amazon S3, Google Cloud Storage.
## Setup
When creating a new workspace, you must select your desired region. Once a workspace has been created, its region can't be changed.
### Data in transit
During syncs, your data flows through Hightouch's data plane on its way to your destinations.
During the in-transit phase of a sync, your data remains within your workspace region. However, Hightouch can't verify where your destinations store your data. If your destination's data region is important to you, verify it with the destination directly.
### Data at rest
Hightouch only stores data needed to perform [change data capture](/getting-started/concepts#change-data-capture) and [debugging](/security/storage#observability-and-debugging). By default, Hightouch stores this data in an encrypted Amazon S3 bucket, Google Cloud Storage bucket, or Azure Storage account, depending on the cloud in which your workspace is created. Your Hightouch bucket, and all access to that bucket, is scoped to your region, ensuring compliance with data residency regulations.
Hightouch customers on the [Business tier](https://hightouch.com/pricing) can set up an [external storage bucket](/security/storage), granting more control over their data at rest. If you use an external storage bucket, Hightouch doesn't validate that the bucket is hosted in the same region as your Hightouch workspace.
---
## SSH Tunneling
**URL:** https://hightouch.com/docs/security/ssh-tunneling
**Description:** Hightouch supports self-service SSH tunneling, allowing you to securely connect to your data warehouse.
**Section:** Security and compliance
## Overview
Tunneling lets Hightouch securely open a connection to a data warehouse in your private network or Virtual Private Cloud (VPC) without exposing it to the internet. SSH tunnels are secure, authenticated, encrypted, and dedicated to your workspace. To learn more about SSH tunneling, check out [this introductory article](https://www.ssh.com/academy/ssh/tunneling).
This feature is supported for the following:
### Sources
- [Azure Synapse](https://hightouch.com/docs/sources/synapse)
- [ClickHouse](https://hightouch.com/docs/sources/clickhouse)
- [Datawarehouse.io](https://hightouch.com/docs/sources/datawarehouseio)
- [Greenplum](https://hightouch.com/docs/sources/greenplum)
- [Metabase](https://hightouch.com/docs/sources/metabase)
- [Mixpanel](https://hightouch.com/docs/sources/mixpanel)
- [MongoDB](https://hightouch.com/docs/sources/mongodb)
- [MySQL](https://hightouch.com/docs/sources/mysql)
- [Oracle DB](https://hightouch.com/docs/sources/oracle)
- [PostgreSQL](https://hightouch.com/docs/sources/postgresql)
- [Redshift](https://hightouch.com/docs/sources/amazon-redshift)
- [SingleStore](https://hightouch.com/docs/sources/single-store)
- [Snowflake](https://hightouch.com/docs/sources/snowflake)
- [Tableau](https://hightouch.com/docs/sources/tableau)
- [Teradata Vantage](https://hightouch.com/docs/sources/teradata-vantage)
- [Trino](https://hightouch.com/docs/sources/trino)
### Event Sources
- [Kafka](https://hightouch.com/docs/events/sources/kafka)
### Destinations
- [AlloyDB](https://hightouch.com/docs/destinations/alloydb)
- [Databricks](https://hightouch.com/docs/destinations/databricks)
- [Elasticsearch](https://hightouch.com/docs/destinations/elasticsearch)
- [MariaDB](https://hightouch.com/docs/destinations/mariadb)
- [MongoDB](https://hightouch.com/docs/destinations/mongodb)
- [MySQL](https://hightouch.com/docs/destinations/mysql)
- [OpenSearch](https://hightouch.com/docs/destinations/opensearch)
- [Oracle DB](https://hightouch.com/docs/destinations/oracle-db)
- [PostgreSQL](https://hightouch.com/docs/destinations/postgres)
- [Redis](https://hightouch.com/docs/destinations/redis)
- [Snowflake](https://hightouch.com/docs/destinations/snowflake)
### Event Destinations
- PostgreSQL
- Redshift
### Standard versus reverse tunneling
Both **Standard** and **Reverse** tunnels open a secure port connection between Hightouch and your data warehouse. However, they differ in implementation, and you may prefer one based on your network specifications.
[Standard tunnels](#standard-tunneling) require you to run `sshd` on a [bastion host](https://aws.amazon.com/quickstart/architecture/linux-bastion/) accessible from the public internet. Our systems open an SSH connection to your bastion, then open a port forwarding connection from your bastion to the private service you specify.
[Reverse tunnels](#reverse-tunneling) let you forward a port by connecting as a client to an SSH server managed by Hightouch. This removes the necessity for a bastion host in your infrastructure but requires you to maintain that connection. That means if your connection goes down for whatever reason, your systems automatically re-open the connection. You can use programs like [`autossh``](https://linux.die.net/man/1/autossh), [Docker container restart policies](https://docs.docker.com/config/containers/start-containers-automatically/), or process supervisors like [supervisord](http://supervisord.org/) to help maintain your connection.
## Standard tunneling
### Requirements
- Allowlist the Hightouch [static IP addresses](/security/networking#ip-addresses) corresponding to your region.
- Allow connections from the bastion host to your warehouse.
- Set up a user on the bastion host named `hightouch`.
### Setup
To get started, navigate to [**Settings** > **Tunnel**](https://app.hightouch.com/settings/tunnels).
1. Select **Add Tunnel**.
1. Enter a **Name** for your tunnel.
1. Fill out the **SSH Host** and **SSH Port**.
- These are the connection details for your public-facing bastion server host.
- The port is most likely 22, standard for `sshd`.
1. Fill out the **Service Host** and **Service Port**.
- These are the connection details for the data warehouse you are connecting to Hightouch.
- Think of your bastion server as a [jump host](https://en.wikipedia.org/wiki/Jump_server). Hightouch jumps through it to connect to your warehouse using these details.
1. Select **Create**.
1. Copy or download the generated SSH public key. Add this to the `~/.ssh/authorized_keys` file for the `hightouch` user on your bastion server. You can use [`ssh-copy-id`](https://www.ssh.com/academy/ssh/copy-id) to help with this.
1. The tunnel **Status** turns green when the connection is established. Your tunnel is now ready for use.
### Troubleshooting
If you're having trouble establishing a connection with a standard tunnel, check the following:
- Check that you've allowlisted the [static Hightouch IP addresses](/security/networking#ip-addresses) on your bastion host. See [requirements](#requirements) for more information.
- Check that the `hightouch` user exists on your bastion host and that the Hightouch public key is in their `~/.ssh/authorized_keys` file.
- Check permissions on the `hightouch` user's SSH files.
- `~/.ssh` directory should be `0700`
- Fix: `chmod 0700 /home/hightouch/.ssh`
- `~/.ssh/authorized_keys` file should be `0644`
- Fix: `chmod 0644 /home/hightouch/.ssh/authorized_keys`
- `~/.ssh` directory and `~/.ssh/authorized_keys` file should be owned by the `hightouch` user
- Fix: `chown -R /home/hightouch/.ssh`
- Check that the bastion host can network to your warehouse.
- `nc -z $warehouse_host $warehouse_port`
- If all else fails, .
## Reverse tunneling
### Requirements
- You need a server within your VPC to act as the SSH client.
- The SSH client server must be able to connect to both the public internet and your warehouse.
- Make sure your Hightouch workspace is in a region where tunneling is supported:
- AWS ap-south-1
- AWS eu-west-1
- AWS us-east-1
- GCP europe-west1
- GCP us-east4
### Setup
To get started, navigate to [**Settings** > **Tunnel**](https://app.hightouch.com/settings/tunnels).
1. Select **Add reverse tunnel**.
1. Enter a **Name** for your tunnel.
1. Select **Create**.
1. Download the private key to your machine.
{" "}
Hightouch doesn't store your private key. As a result, we can't recover a
lost key. Please make a local copy of this key and store it securely.
1. Copy the example `ssh` command and save it. You'll need to run it later.
- **Example**
```bash
ssh -i path/to/key.pem \
-R 0.0.0.0:$TUNNEL_PORT:$SERVICE_HOST:$SERVICE_PORT \
$TUNNEL_ID@tunnel..hightouch.io -p 22 \
-o ExitOnForwardFailure=yes
```
- This command includes the remote Hightouch `sshd` host and port, and remote forwarding port.
- Set or replace the `$SERVICE_HOST` and `$SERVICE_PORT` variables with your internal warehouse service host and port.
- Set or replace the `$TUNNEL_PORT` and `$TUNNEL_ID` variables with the tunnel port and ID provided by Hightouch upon creating a reverse tunnel
- `hightouch-region` will be the [region](#requirements-1) of your Hightouch workspace (e.g. `aws-us-east-1`).
1. Upload the private key to your SSH client server, store it securely, and ensure its permissions are set to `0400`.
1. From your SSH client server, run the modified `ssh` command.
- Ensure the `-i` flag is pointing to the correct path of the private key.
- You most likely want to wrap this `ssh` command with a process manager to restart in case of failure. Consider [`autossh`](https://linux.die.net/man/1/autossh).
1. The tunnel **Status** turns green when the connection is established. Your tunnel is now ready for use.
**Hightouch Reverse SSH Host Key**
If you prefer, you can check that Hightouch's hostname is correctly set to either `tunnel.aws-us-east-1.hightouch.com`, `tunnel.aws-eu-west-1.hightouch.com`, `tunnel.aws-ap-south-1.hightouch.com`, `tunnel.gcp-us-east4.production.hightouch.com`, or `tunnel.gcp-europe-west1.production.hightouch.com` depending on your workspace region and that the Host Key is correctly included in `~/.ssh/known_hosts` as follows:
```
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXjpH+c24rYGDBoUdHGN3Yazp4SrDU8Z0Oqb0WRaKJz0VpSPS+Kqmc0MioY3yK1aPDh32VSEFUykEHIobV7dsH9uKlR3OllTkxN9pIk3xfJBn1/x4TS2VX1I+XM7KE6xJH/QmBVaVyNzsYzZ+lR/hUdrVC3czV+ReJa5j+YS23atS3x9NznxHSBBOLcq0RpFj65/WZt0kJwbIqhrGYIq5qXR/8d7gzyCj6vg2iNNUszTFQvcFeQ8zeXC63ke4C9oCFHbneX2odvsDY5DQM6WF7H6j91qRhGEq+niQg2zCns1kJq1Gfa7bY3OxIHmQ3HP0R8ExJMsBBqdKoL5uNY8UK717yNuFzBQUV8fgJi1HZC6JR6uDVpLadbXUV3WD0LA0YQ1h24KKGtPRDrnyZxNTdn0R19asz31npyFHKrP95OIz2RoGuqsn9fr3FzO1wfXjdzYp5l6AAy04K6yPNQfTC/+LLk8U1PWbSsMBqlQnHkp/eMaSdTEVMVxqhAFKCz8M=
```
### Troubleshooting
If you're having trouble establishing a connection with a reverse tunnel, check the following:
- Check that your SSH client server is running and can access the public internet.
- Check that you've uploaded the private key and that it's only readable by the user initiating the SSH connection (`chmod 0400`).
- Add the `-v` (verbose) flag to your SSH command to see more detailed errors.
- If all else fails, . Please include any errors in the SSH output.
---
## Storage
**URL:** https://hightouch.com/docs/security/storage
**Description:** Learn more about what data Hightouch stores, whether with Hightouch or your own infrastructure, to power your syncs.
**Section:** Security and compliance
Hightouch needs to store some data to power your syncs and to power features like the [sync debugger](/syncs/debugger). You have full control over where this data is stored.
If you prefer more direct control or have specific security concerns, you can configure Hightouch to store this data in your own Amazon S3, Google Cloud Storage, or Azure Blob Storage account. If you prefer convenience, Hightouch can store this data in our own secure, encrypted infrastructure.
## What data does Hightouch store?
Hightouch stores data at-rest for two purposes:
1. [Change data capture](/getting-started/concepts#change-data-capture)
2. [In-app observability and debugging](/syncs/debugger)
Depending on your privacy and compliance needs, you can configure Hightouch to store **all** data-at-rest within your Virtual Private Cloud (VPC), or in a secure, encrypted bucket.
- If you're on the [Free, Starter, or Pro tier](https://hightouch.com/pricing), Hightouch hosts and manages a secure encrypted bucket on your behalf. For more information, see the [managed storage section](#managed-storage).
- If you're on the [Business tier](https://hightouch.com/pricing), you can self-host storage in your own infrastructure. For more information, see the [self-hosted storage section](#self-hosted-storage).
### Change data capture
To prevent making excessive API requests and send only necessary updates to your destinations, Hightouch uses a process called **change data capture** (CDC) or **diffing**.
In this process, Hightouch stores query results and execution plans after each sync run. When the next sync run occurs, Hightouch uses these diff files to determine incremental changes that should be sent downstream.
By default, Hightouch stores the diff files in a secure encrypted bucket hosted by Hightouch. [Business tier](https://hightouch.com/pricing) accounts can use the [Lightning sync engine](/syncs/lightning-sync-engine) to compute and stores diffs directly within their data warehouse.
To learn more about CDC, check out [the core concepts docs](/getting-started/concepts#change-data-capture).
### Observability and debugging
In addition to storing previous query results, Hightouch stores row-level log metadata including successes and failures, operations performed, and API request and response payloads. This data powers the [in-app debugger](/syncs/debugger) and can be stored either in your VPC or Hightouch's encrypted bucket.
## Managed storage
If you're on a [Free, Starter, or Pro plan](https://hightouch.com/pricing) Hightouch stores data-at-rest in a secure, encrypted, Hightouch-managed bucket. For workspaces running in a Hightouch AWS region, this is an Amazon S3 bucket. For workspaces running in a Hightouch Google Cloud region, this is a Google Cloud Storage bucket. No matter your region, all data in Hightouch managed buckets is encrypted at rest. If you require data-at-rest to live entirely in your VPC, see [self-hosted storage](/security/storage/#self-hosted-storage).
### Data retention
Data automatically expires from Hightouch-managed buckets after 30 days. If [change data capture](/getting-started/concepts#change-data-capture) is done in Hightouch-managed buckets, syncs that have not run in over 30 days will require a **Full Resync** or **Reset CDC** sync since Hightouch depends on diffing files to detect changes in the data model.
## Self-hosted storage
[Business tier](https://hightouch.com/pricing) customers can configure Hightouch to store all customer data-at-rest within their own external storage bucket or blob. Hightouch integrates with these cloud storage providers:
- Amazon S3
- Google Cloud Storage (GCS)
- Microsoft Azure Blob Storage
If you choose to self-host your storage, Hightouch only processes data-in-transit. You can select any supported storage provider to store your data, regardless of your [Hightouch region](/security/regions).
When hosting your own storage, Hightouch places full control over object lifecycle, security, and expiration into your hands.
We don't expire objects automatically or change your object encryption settings.
Ensure that you've configured object expiration, encryption, and access control settings according to your needs.
#### Recommended architecture and security practices
Use the following guidelines to reduce setup issues and simplify operations:
- Scope IAM permissions to the exact bucket/container prefix used by Hightouch. Avoid wildcard access to the entire bucket or storage account.
- Use a dedicated bucket/container, or a dedicated top-level prefix, per environment (for example `prod/`, `staging/`, and `dev/`) to avoid cross-environment access.
- Co-locate storage with your warehouse and destinations when possible to reduce latency and egress costs.
- Prefer private connectivity over public internet paths when available: [AWS PrivateLink](/security/aws/privatelink), [Azure Private Link](/security/azure/private-link), or [Google Private Service Connect](/security/gcp/private-service-connect). If using public endpoints, maintain strict IP allowlists.
- Use encryption at rest with your standard key-management policy (for example SSE-KMS, CMEK, or customer-managed keys) and ensure key policies grant your Hightouch principal the required access.
- Apply lifecycle rules that auto-expire objects according to your retention policy, and enable storage access logging when required for audits.
- After initial setup or credential rotation, run a small validation sync and confirm data is accessible in the [sync debugger](/syncs/debugger) before broad rollout.
Setting up self-hosted storage disrupts the [change data capture](/getting-started/concepts#change-data-capture) process for active syncs.
To reset it, after you've configured self-hosted storage, you need to [trigger a full resync](/syncs/overview#resync-full-query) or [reset cdc sync](/syncs/overview#reset-cdc) for all existing syncs that previously ran with [Hightouch-managed storage](#managed-storage).
Make sure that all your syncs satisfy the [full resync prerequisites](/syncs/overview#full-resync-prerequisites) before setting up self-hosted storage.
Don't hesitate to if you have any doubts or concerns.
Once you've run a sync after setting up a custom storage bucket, you can't
make further changes to your storage configuration self-serve, including disabling it.
This is because changing your storage configuration is disruptive to active syncs
and CDC state. If you need to change your bucket, see [Changing your storage bucket](#changing-your-storage-bucket) below.
## Changing your storage bucket
Changing your external storage bucket — for example, moving to a new AWS account, switching regions, or migrating from Hightouch-managed to customer-managed storage — is a **support-assisted operation**. There is no self-serve way to change your bucket configuration after syncs have run.
to start the process. To help the team plan your migration efficiently, include the following information in your request:
**Scope:**
- Workspace name(s) and URL(s) affected
- Whether the workspace is production or non-production
**New bucket details:**
- Cloud provider and region (for example, `AWS us-east-1`)
- Bucket or container name (for example, `my-company-hightouch-prod`)
- Optional prefix, if you want Hightouch to write to a specific path within the bucket
**Access credentials:**
- For AWS: the IAM role ARN (and external ID, if required) with the [permissions listed above](#authenticate-hightouch-with-aws)
- For GCS: the service account email and confirmation that it has the required permissions (`storage.objects.list`, `storage.objects.create`, `storage.objects.get`, `storage.objects.delete`, `storage.buckets.get`)
- For Azure: the storage principal and scope
- If you're using prefix-scoped IAM policies, include the exact prefix so the team can confirm compatibility
**Feature usage:**
- Whether you're using the [Lightning sync engine](/syncs/lightning-sync-engine), [AI Decisioning](/ai-decisioning/overview), or other features that store artifacts in the bucket
- Whether you're using **Snowflake-integrated features** such as **Snowflake Warehouse Export** — these rely on Snowflake external volumes and stages that also need to be updated to point at the new bucket
- Any lifecycle policies on the old or new bucket that automatically delete objects — particularly planner-state files, which can trigger unexpected full resyncs if expired
**Maintenance window:**
- A preferred time window (ideally during low traffic) when syncs can be paused and the configuration updated
### What to expect
Hightouch will update your workspace's storage configuration to point to the new bucket and validate connectivity. The impact on your syncs depends on the sync engine:
- **Lightning sync engine:** CDC state is stored primarily in your data warehouse, not in the bucket. Changing buckets generally does not require re-seeding CDC, but historical debugger logs from the old bucket won't carry over to the new one.
- **Standard sync engine:** CDC relies on snapshots and diff files stored in the bucket. After a bucket change, each affected sync will need a one-time [full resync](/syncs/overview#resync-full-query) or [CDC reset](/syncs/overview#reset-cdc) to establish a clean baseline in the new bucket. Plan for higher warehouse and API load during these initial runs.
- **AI Decisioning:** The bucket stores long-lived ML artifacts and interaction logs. Migrating may require copying these artifacts to the new bucket, or AI Decisioning will rebuild state from your warehouse over time. Call this out in your support request so the team can plan accordingly.
- **Snowflake-integrated features (Snowflake Warehouse Export and others):** If you use features that rely on Snowflake external volumes or stages, the Hightouch team will also update the associated storage integrations to point at the new bucket. Make sure the new bucket's cloud and region are compatible with your Snowflake account.
After the configuration change, run a small test sync and verify that new artifacts and logs appear in the new bucket before resuming all workloads.
### Amazon S3
Before getting started, [connect Hightouch with your Amazon Web Services account](/security/aws).
#### Create your S3 bucket
In Amazon S3, [create your bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html). We recommend the name `-hightouch`.
**Make sure to:**
- Block all public access to the bucket.
- Enable Amazon S3 key encryption \(SSE-S3\). If using [SSE-KMS](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html) for encryption, you may need to [update your IAM policies](https://repost.aws/knowledge-center/cross-account-access-denied-error-s3) to grant Hightouch access.
- Prefer bucket policies with IAM condition scoping over ACL-based access controls.
- Disable bucket versioning.
- Configure your [bucket object lifecycle](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html), to enhance security and cut down on costs.
#### Authenticate Hightouch with AWS
Hightouch supports authenticating with AWS using Cross-account roles (via STS AssumeRole), or with an Access Key ID / Secret Access Key that you provide. We strongly encourage you to use Cross-account roles, as it doesn't require Hightouch to hold any of your secrets.
To set up your Hightouch AWS credential, follow our [connection instructions](/security/aws).
Hightouch needs the following IAM actions to store and retrieve items from your bucket:
| Action | Details |
| :--------------------- | :------------------------------------------------------------------------------------ |
| `s3:GetObject` | Grants permission to retrieve objects from Amazon S3 |
| `s3:GetObjectVersion` | Grants permission to read versioned objects when applicable |
| `s3:PutObject` | Grants permission to add an object to a bucket |
| `s3:ListBucket` | Grants permission to list some or all the objects in an Amazon S3 bucket (up to 1000) |
| `s3:GetBucketLocation` | Grants permission to read bucket region metadata |
| `s3:DeleteObject` | Grants permission to delete objects from an Amazon S3 bucket |
You can use the following JSON sample to create a prefix-scoped IAM policy:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ListBucketOnPrefix",
"Effect": "Allow",
"Action": ["s3:ListBucket", "s3:GetBucketLocation"],
"Resource": ["arn:aws:s3:::${bucketName}"],
"Condition": {
"StringLike": {
"s3:prefix": ["${prefix}/*"]
}
}
},
{
"Sid": "ObjectAccessOnPrefix",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:GetObjectVersion",
"s3:PutObject",
"s3:DeleteObject"
],
"Resource": ["arn:aws:s3:::${bucketName}/${prefix}/*"]
}
]
}
```
#### Configure your bucket in Hightouch
1. In Hightouch, on the [**Storage** tab](https://app.hightouch.com/settings/storage) of the **Settings** page, select **Amazon S3** as the **Cloud provider**.
2. Select your **AWS region**, enter your **Bucket name**, and select the **AWS credentials** you previously set up.
3. Click **Save**.
Once you save your settings, your new syncs automatically start using your bucket. [Run a few syncs](/syncs/create-your-first-sync) and visit your S3 bucket to check files are saving there.
Don't hesitate to if you have any questions.
#### Common errors
If you receive a `CredentialsError: Missing credentials in configuration` error, refer to the [troubleshooting tips](/security/aws#credentialserror-missing-credentials-in-configuration) in the AWS integration docs.
### Google Cloud Storage
Before getting started, [connect Hightouch with your Google Cloud account](/security/gcp).
#### Create a bucket
In the Google Cloud console, [create a new bucket](https://cloud.google.com/storage/docs/creating-buckets). We recommend the name `-hightouch-bucket`. Copy the bucket name and save it for later.
Configure your [bucket object lifecycle](https://cloud.google.com/storage/docs/lifecycle), to enhance security and cut down on costs.
**Make sure to:**
- Enable [Uniform bucket-level access](https://cloud.google.com/storage/docs/uniform-bucket-level-access) and manage access through IAM policies instead of object ACLs.
- Use CMEK where required by your compliance controls, and ensure your service account has access to the key.
- If you use [VPC Service Controls](https://cloud.google.com/vpc-service-controls/docs/overview), include the bucket and service account in the same perimeter to avoid access denials.
#### Authenticate with Google Cloud
Hightouch supports authenticating with GCP using Hightouch-managed service accounts, or by using a service account that you control.
To set up your Hightouch GCP credential, follow our [connection instructions](/security/gcp).
Hightouch needs the following IAM permissions to store and retrieve items from your bucket:
| Permission | Details |
| :----------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `storage.objects.list` | Grants access to view objects and their metadata, excluding ACLs. Can also list the objects in a bucket. |
| `storage.objects.create` | Grants permission to create, replace, and delete objects; list objects in a bucket; read object metadata when listing (excluding IAM policies); and read bucket metadata, excluding IAM policies. |
| `storage.objects.get` | Grants access to view objects and their metadata, excluding ACLs. Can also list the objects in a bucket. |
| `storage.objects.delete` | Grants permission to overwrite existing objects. |
#### Configure your bucket in Hightouch
1. In Hightouch, on the [**Storage** tab](https://app.hightouch.com/settings/storage) of the **Settings** page, select **Google Cloud Storage** as the **Cloud provider**.
2. Enter the **Project ID** and **Bucket name** and select the **Google Cloud credentials** you previously set up.
3. Click **Save**.
Once you save your settings, your new syncs automatically start using your bucket. [Run a few syncs](/syncs/create-your-first-sync) and visit your Google Cloud bucket to check files are saving there.
Don't hesitate to if you have any questions.
### Microsoft Azure Blob Storage
Before getting started, [connect Hightouch with your Azure account](/security/azure).
#### Create a container
In your Azure portal, [create a container](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-portal#create-a-container). We recommend the name `-hightouch-container`. Copy the container name and save it for later.
Configure your [storage lifecycle](https://learn.microsoft.com/en-us/azure/storage/blobs/lifecycle-management-overview), to enhance security and cut down on costs.
#### Authenticate with Azure
The easiest way to grant Hightouch access is to grant the app the **Storage Blob Contributor** role for the storage account.
Alternatively, you may grant only the **Storage Blob Delegator** role at the account level, and **Storage Blob Contributor**
for the storage **container**.
Use a Service Principal with RBAC over long-lived SAS tokens when possible, and rotate client secrets or certificates on your normal security cadence.
If you want to create a custom role with more granular permissions, Hightouch needs the following IAM permissions to store and retrieve items from your Blob Storage **Container**:
- `"actions"."Microsoft.Storage/storageAccounts/blobServices/containers/read"`
- `"dataActions"."Microsoft.Storage/storageAccounts/blobServices/containers/blobs/add/action"`
- `"dataActions"."Microsoft.Storage/storageAccounts/blobServices/containers/blobs/write"`
- `"dataActions"."Microsoft.Storage/storageAccounts/blobServices/containers/blobs/delete"`
Some functionality may require delegation to be granted on the storage Blob Storage **Account**
- `"actions"."Microsoft.Storage/storageAccounts/blobServices/generateUserDelegationKey/action"`
If you're using ADLS Gen2 (hierarchical namespace enabled), ensure list operations target stable folder prefixes (for example with trailing `/`) and validate that your processing handles directory entries correctly.
#### Configure your container in Hightouch
1. In Hightouch, on the [**Storage** tab](https://app.hightouch.com/settings/storage) of the **Settings** page, select **Microsoft Azure** as the **Cloud provider**.
2. Enter the **Account name** and **Container name** from the Azure portal (see [connecting Hightouch with your Azure account](/security/azure)), and select the **Azure credentials** you previously set up.
3. Click **Save**.
Once you save your settings, your new syncs automatically start using your container. [Run a few syncs](/syncs/create-your-first-sync) and visit your Blob Storage container to check files are saving there.
Don't hesitate to if you have any questions.
#### Common errors
Your Azure storage configuration may fail to save for a few reasons:
- `invalid_client`: Check that you've provided your client secret **value**, not client secret ID. See the [configuration docs](/security/azure#create-an-app-registration) for more information.
- `This request is not authorized to perform this operation`: Check that Hightouch has [adequate access](#authenticate-with-azure) to your storage account. If you've configured your storage account's **Network access** to be **from selected virtual networks and IP addresses**, be sure to add Hightouch's [IP addresses](/security/networking#ip-addresses) to your [network rules](https://learn.microsoft.com/en-us/azure/storage/common/storage-network-security?tabs=azure-portal).
---
## Same-session audiences overview
**URL:** https://hightouch.com/docs/real-time/overview
**Description:** Understand what same-session audiences are, when to use them, and how they evaluate user behavior during an active session.
**Section:** Hightouch Real-time
*Same-session audiences allow Hightouch to evaluate audience membership using live user behavior during the current session -- typically in less than one second.*
---
## Overview
Same-session audiences let you evaluate whether a user qualifies for an audience **while they’re actively using your product or website**—usually in less than one second.
Instead of waiting for scheduled updates, you can respond immediately to what someone is doing right now.
Think of it like this:
- Instead of asking *“Who qualified for this audience today?”*
- You’re asking *“Does this user qualify right now?”*
This shift makes it possible to react in the moment—while the user is still engaged.
For example, you can respond when someone:
- views your pricing page multiple times
- searches for a product during their visit
- starts checkout but doesn’t finish
- meets a condition based on both their past behavior and current activity
Same-session audiences still use traits and models defined in your [schema
setup](https://hightouch.com/docs/customer-studio/schema), but do not run new
SQL queries each time an event occurs.
---
## Why use same-session audiences?
Many campaigns depend on **recent user behavior**.
If your audiences only update every few hours (or once a day), you can miss the moment when a user is most engaged—or send messages that are already outdated.
Same-session audiences allow you to react while a user is still active.
This enables you to:
- Personalize website or app experiences in real time
- Stop showing promotions as soon as someone converts
- Trigger follow-ups based on recent actions
- Avoid sending irrelevant or mistimed messages
For example, if a user completes a purchase, they can be removed from a promotional audience within seconds--not hours later.
---
## When to use same-session audiences
Use same-session audiences when timing matters—especially during an active session.
| Scenario | Recommended approach |
|---|---|
| Daily or hourly campaign refresh | Batch audience |
| Event-triggered campaign messaging | [Event streaming](/events/event-streaming) |
| **Personalization during the same session** | **Same-session audiences** |
### Common use cases
- Stop sending messages immediately after a user converts
- Show recommendations based on what someone is browsing right now
- Trigger a push notification when a user abandons checkout
- Adapt the experience based on in-session behavior
If your campaign doesn't require immediate updates, a scheduled audience
refresh may be simpler and more cost-effective.
---
## How same-session audiences work
Same-session audiences combine two types of information:
- **What the user is doing right now** (live events like page views or searches)
- **What you already know about them** (customer data from your warehouse)
When something happens—like a page view or search—Hightouch quickly checks:
- *“Based on this action, does this user now qualify for this audience?”*
Here’s what happens step by step:
1. **The user takes an action**
For example, they view a pricing page or search for a product.
2. **Hightouch checks your audience rules**
It evaluates whether that action meets the conditions you’ve defined.
3. **Customer context is included**
If your audience includes traits like `plan_type` or `region`, those are considered too.
4. **Audience membership updates immediately**
If the user qualifies, they enter the audience.
If they no longer qualify, they exit.
This all happens in about a second, so your campaigns and experiences can respond right away. In other words, you move from asking *"Who qualified earlier today?"* to *"What should we do for this user right now?"*
See [Same-session audience architecture and data handling →](/real-time/architecture) for a deeper dive.
---
## How this differs from batch audiences
Batch audiences are updated on a schedule (for example, every few hours or once a day).
Same-session audiences update continuously, as users interact with your product.
| | Batch audiences | Same-session audiences |
|---|---|---|
| **When they update** | On a schedule | Continuously, in real time |
| **Best for** | Broad campaign targeting | In-session personalization |
| **Speed** | Minutes to hours | Less than a second |
---
## What's next
- [Define your data schema →](/customer-studio/schema)
- [Same-session audiences architecture →](/real-time/architecture)
- [Set up same-session audiences →](/real-time/setup)
- [Personalization API →](/real-time/personalization-api)
---
## Personalization API
**URL:** https://hightouch.com/docs/real-time/personalization-api
**Description:** A fast, flexible API to tailor customer experiences across web, mobile, and email.
**Section:** Hightouch Real-time
This feature is available as an add-on to [Business tier plans](https://hightouch.com/pricing). To get access and learn more about implementation, please .
## Overview
Hightouch Personalization API is a fully managed service that combines the analytical power of the data warehouse with the real-time performance of a low-latency API. You can think of this feature as a cache for your data warehouse.
With the Personalization API, you can serve any data model—customers, products, transactions, recommendations, etc.—to any internal app, web experience, or email marketing platform. Define your dataset with SQL and Hightouch handles the rest.
Example use cases include:
- **Powering personalized marketing campaigns** by enriching marketing touchpoints with dynamic product recommendations
- **Delivering in-app or web personalization** like customized search results, cohort-based pricing and messaging, article recommendations, or nearest store locations
- **Optimizing product experimentation** by delivering up-to-date customer information like audience inclusion/exclusion to decisioning and experimentation systems
---
## Architecture
Under the hood, the Personalization API is backed by a distributed, low-latency database that stores a cache of your model. This cache updates as frequently as you see fit with Hightouch's [scheduling options](/syncs/schedule-sync-ui). Hightouch then serves this cache over HTTP so that it can be queried by your SaaS tools and other services.
Our benchmarks show p99 response times under 30 milliseconds in the same region. The Personalization API can support peak throughput of 1 million requests per second. To obtain the most recent benchmarks for your region, please .
---
## Implementation steps
Using this feature is a three-step process:
1. Enabling the Personalization API in the regions closest to your own infrastructure
2. Creating syncs to define which models should be materialized in the Hightouch-managed cache
3. Using your HTTP endpoints to access data
---
## Getting started
After the Personalization API is enabled for your workspace, navigate to the newly created destination and configure the region from which the API should be served.
Currently, the following AWS regions are available for self-serve deployment:
- `us-east-1`
- `us-west-2`
- `eu-west-1`
For other regions, please to complete setup.
---
## Sync configuration
There are two core concepts in the Personalization API: collections and records.
- **Collections** are conceptually similar to database tables. Each should represent a particular type of data, such as customers or invoices. Collection names must be alphanumeric and will become part of your Personalization API endpoint.
- **Records** are conceptually similar to rows in a database. Each should represent a single item, like a customer profile. Collections consist of one or more records.
You can create multiple syncs to the same collection provided that each sync updates different record fields.
For example, you might have one model representing basic customer demographic data—`age`, `gender`, `location`, etc.—and another model representing account usage data—`last_login_at`, `num_messages_sent`, etc. By creating two syncs to the same `customers` collection, you can create more complex records that contain all attributes necessary for personalization.
Next, specify which column from your model should serve as the primary index for record lookups. This field must uniquely identify each record in the collection and is often the same as your model's primary key.
The Personalization API supports lookups on multiple indices. For example, you might want to retrieve customer profiles using `user_id`, `anonymous_id`, or `email_address`. To enable multiple indices, please .
Use the field mapper to specify which columns from your model should be included in the API response payload. You can rename these fields and use the advanced mapper to apply transformations using the [Liquid template language](https://shopify.github.io/liquid/).
### Delete behavior
You have the following options for how Hightouch should handle rows leaving your query results:
| Behavior | Description |
| ----------------------------------------------------------------------------- | -------------------------------------------------------------------- |
| **Do nothing** | All cached data is preserved indefinitely. This option is *strongly* recommended against. |
| **Clear the fields managed by this sync but do not delete the entire record** | Fields managed by other syncs are preserved. |
| **Delete the entire record** | Requests for deleted records receive `404 Not Found` responses. |
You probably want to delete the entire record if your collection is managed using only one sync. However, if multiple syncs are writing to the same collection, it's best to clear only the fields updated by this sync.
---
## Access data
To start using the API, you must first create an API key for your workspace.
### Create an API key
### Make API requests
Once you have an API key, you can make requests to the Personalization API to retrieve data. The API is accessible at `https://personalization.{region}.hightouch.com`, for example, `https://personalization.us-west-2.hightouch.com`.
#### Authentication
You can specify your API key via a bearer token in an HTTP Authorization header.
```
Authorization: Bearer XXXXX
```
#### Usage
The API has a single endpoint:
| Verb | Path |
| ---- | ------------------------------------|
| GET | /v1/collections/`:collection_name`/records/id/`:index_value` |
Refer to your [sync configuration](#sync-configuration) for your `collection_name`.
A request looks like this:
```bash
curl -X GET \
-H "Authorization: Bearer 55555555-4444-3333-2222-111111111111" \
"https://personalization.us-west-2.hightouch.com/v1/collections/customers/records/id/12345"
```
When the query is successful, the HTTP response is a `200 OK` and returns the record specified by the index value, in this case the customer with `user_id` (the index key) equal to `12345` (the index value). The data in the response payload is entirely flexible and defined by your [sync configuration](#sync-configuration).
For example:
```json
{
"user_id": 12345,
"full_name": "Jane Doe",
"lifetime_value": 1492.18,
"churn_risk": 0.04,
"90_day_summary": {
"num_songs_listened": 813,
"top_genres": [
"house",
"techno",
"ambient"
],
"top_artists": [
"deadmau5",
"Marsh",
"Enamour"
],
},
"recommendations": {
"concerts": [
{
"artist": "Aphex Twin",
"location": "San Francisco, CA",
"event_date": "2023-01-31"
},
{
"artist": "Sultan + Shepard",
"location": "San Francisco, CA",
"event_date": "2023-02-25"
}
],
"upcoming_album_release": {
"title": "Universal Language",
"artist": "Simon Doty",
"label": "Anjunadeep",
"release_date": "2023-04-28"
}
},
"_audiences": {
"top_concert_ticket_purchasers": true,
"attended_concert_last_month": false,
"electronic_music_listeners": true
}
}
```
---
## Tips and troubleshooting
### Unique primary key requirement
The model you select for your sync configuration _must_ have a unique identifier as its [primary key](/getting-started/concepts#unique-primary-key-requirement).
### Recommended usage
The Personalization API should not be called directly from a webpage, mobile app, or any untrusted client. We recommend proxying API calls through a service that you own, such as your application backend. This allows you to enforce reasonable rate limits and ensure that clients cannot retrieve data belonging to other users. Keep in mind that authentication is performed using a Hightouch API key, which grants access to all records in your Personalization API collection.
### Live debugger
### Sync alerts
### Merging records
When [configuring multiple syncs to update the same collection](#sync-configuration),
there are some common errors and misconfigurations to be aware of.
In general, it is simpler and less error-prone to ensure that different syncs do not attempt to manage
the same fields.
#### Merging records with the same field names
If two syncs both update the _same field_ on the record, it will of course take whatever value is written last.
This may have unexpected results if a record is _removed_ from one sync, but not another, as the behavior
is defined by whatever [deletion behavior](#delete-behavior) is specified for the sync that deletes the record.
For example, the record may be deleted, or if two syncs write to the same field name, the field may be cleared
and thus have no value at all until another sync updates the record.
#### Merging records with nested fields
When merging records, e.g. writing an `address` field to a `customer` record created by another sync, the
merging operates simply by replacing the `address` value with the new one. There is no special logic to
merge property _values_, or nested values. For example, if one sync creates a customer record such as
`{ id: 1, name: "John Doe", address: { country: "US" } }`, and another sync updates it with
`{ id: 1, age: 33, address: { zip: 86023 } }`,
the resulting record will be `{ id: 1, age: 33, name: "John Doe", address: { zip: 86023 } }`: it merges fields
from both syncs (`name`, `age`), but the value of the `address` field, originally `{ country: "US" }`, was
overwritten with the new value `{ zip: 86023 }`. It is not possible in the Personalization API to perform
a deep merge, setting the field to `{ country: "US", zip: 86023 }`. Instead, we recommend setting the
entire `address` field using a single sync.
---
## Set up same-session audiences
**URL:** https://hightouch.com/docs/real-time/setup
**Description:** Create audiences that evaluate live user behavior using realtime events and deliver membership updates instantly.
**Section:** Hightouch Real-time
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Marketers, platform admins |
| **Prerequisites** |
An event source sending events to Hightouch (Hightouch Events, Segment, RudderStack, or Snowplow)
A destination that supports realtime syncing, or a system that will call the Membership API
Realtime sync capability enabled for your workspace
|
*Same-session audiences evaluate user behavior during the current session using streaming events and warehouse context.*
---
## What you'll learn
After reading this article, you'll know how to:
- Configure realtime events in your Customer Studio schema
- Build an audience that evaluates live user behavior
- Deliver audience membership through realtime syncs or the Membership API
- Choose the right delivery pattern for your personalization or campaign workflow
---
## Overview
[Same-session audiences](/real-time/overview) evaluate audience membership from live behavioral events during the current session, combining streaming events with warehouse attributes.
This guide walks you through the four steps to create one:
1. Connect an event stream
2. Define a **realtime event** in your schema
3. Build an audience that references that event
4. Configure how audience membership is delivered
---
## Step 1: Connect an event stream
Hightouch must receive realtime events before audiences can be evaluated.
Supported event sources include:
- Hightouch Events
- Snowplow
- RudderStack
- Segment
These events flow through Hightouch’s event pipeline and can be used for realtime audience evaluation once a corresponding **realtime event model** is defined in your schema.
See:
- [Set up Hightouch Events →](/events/event-streaming#setup)
- [Migrate to Hightouch Events →](/events/migration-guide/overview)
---
## Step 2: Create a realtime event
Realtime events define which incoming events should be available for same-session audience evaluation.
To create one:
1. Go to **Customer Studio → Schema**
2. Locate or create a [**parent model**](/customer-studio/schema#1-define-the-parent-model) (usually `Users`)
3. Click the **plus (+)** icon next to the parent model
4. Select **Create a related realtime event**
---
### Configure the event source
In the **Select event source** screen:
- Choose the **event source** (for example Segment or Hightouch Events)
- Enter the **event type** (for example `track`, `page`, or `order_completed`)
- Define event properties by adding columns and optional aliases
- Select the **timestamp column**, which determines when the event occurred
Click **Continue**.
---
### Configure the event model
Next, configure how the event relates to your schema.
Provide:
| Field | Description |
|------|-------------|
| **Name** | Display name for the event in Customer Studio |
| **Description** | Optional explanation of the event |
| **Timestamp column** | Used for time-based audience filters |
| **Foreign key** | Column linking the event to the parent model (for example `user_id`) |
Click **Create realtime event**.
Once created, the realtime event appears as an event filter option in the audience builder.
---
## Step 3: Create an audience
Next, create an audience that references the realtime event.
1. Go to **Customer Studio → [Audiences](https://app.hightouch.com/audiences)**
2. Create a new audience or open an existing one.
3. Select the **parent model** connected to your realtime event.
Any existing audience can become a same-session audience once it includes a realtime event condition.
---
## Step 4: Add a realtime event condition
Add a filter that evaluates the realtime event.
1. In the audience builder, click **Add filter**
2. Select **Event** and choose your realtime event model
3. Define the event condition—for example, set it to "performed within the last 5 minutes" to capture recent activity
4. Click **Continue**
5. Finalize audience settings and click **Finish**
---
## Step 5: Choose a delivery method
Same-session audiences can deliver membership updates through two mechanisms.
| Method | Best for | Description |
|---|---|---|
| **Realtime syncs** | Campaign tools, CRMs, advertising platforms | Push audience entry or exit events to supported destinations |
| **Membership API** | Websites, apps, personalization engines | Query audience membership on demand |
### Option 1: Realtime syncs
Realtime syncs send audience membership changes to supported destinations.
When a user enters or exits an audience, Hightouch forwards a membership event to the destination.
Example event:
`User 123 entered audience "High-intent pricing visitors"`
#### Create a realtime sync
1. Open your audience and go to the **Syncs** tab
2. Click **Add Sync**
3. Select a destination that **supports realtime syncing**
Supported destinations include:
- [Braze](/destinations/braze)
- [Iterable](/destinations/iterable)
- [Meta Custom Audiences](/destinations/facebook)
- [Google Ads Customer Match](/destinations/google)
- [Segment](/destinations/segment)
- [Amplitude](/destinations/amplitude)
- [Webhooks](/destinations/http-request)
4. Complete the sync setup.
#### Configure events
After your sync has been created, open its **Configuration** tab to define which events should be forwarded.
Depending on the destination, you can configure:
- audience entry events
- audience exit events
- custom event mappings
### Option 2: Membership API
The **Membership API** allows applications to check audience membership directly.
Your application sends a `user_id` and receives all same-session audiences the user currently qualifies for.
This is commonly used for:
- website personalization
- feature gating
- experimentation platforms
- in-app messaging
**Example request:**
```bash
GET /v1/audiences/{parent_model_id}/{user_id}
Authorization: Bearer
```
**Example response:**
```json
{
"parent_model_id": "1234567",
"user_id": "123xyz",
"audience_ids": [
"1212121",
"3434343"
]
}
```
The Membership API returns same-session audience membership. To retrieve warehouse-backed traits and profile attributes, use the [Personalization API →](/real-time/personalization-api).
---
## Example: Suppression use case
**Goal:** Hide a promotional banner after a user completes a purchase.
**Audience filters:**
- Event: `order_completed` in the last 5 minutes
- Trait: `LTV > 300`
- Property: `channel = web`
**Delivery:**
Realtime webhook sync to the personalization system
**Result:**
A user completes a purchase and immediately exits the promotional audience. The banner no longer appears during the same session.
This pattern is commonly used for cart abandonment suppression, real-time promotions, and session-based personalization.
---
## Constraints
Keep the following limitations in mind when designing same-session audiences:
- Realtime event conditions **must include a lookback window** (maximum **24 hours**)
- Only event **count** conditions are supported today (no arbitrary aggregations like SUM)
- User property conditions from `identify` calls must go through the warehouse and cannot be evaluated in realtime
For the full list of constraints, see [Constraints and limitations](/real-time/architecture#constraints-and-limitations) in the architecture guide.
---
## Best practices
When designing same-session audiences:
- Focus on **recent session behavior** (minutes rather than hours)
- Keep event conditions **specific and narrow**
- Combine live events with **warehouse traits** for richer targeting
- Monitor delivery through **sync logs** or **API responses**
Realtime event conditions must include a lookback window and support windows up to **24 hours**.
---
## What's next
- [Same-session audiences overview →](/real-time/overview)
- [Same-session audiences architecture →](/real-time/architecture)
- [Personalization API →](/real-time/personalization-api)
---
## Ads
**URL:** https://hightouch.com/docs/campaign-intelligence/ads
**Description:** The Ads view in Hightouch's Campaigns feature helps you understand conversions and campaign data from your connected ad platforms.
**Section:** Campaign Intelligence
## Overview
The ads dashboard is the best way to view performance for ad campaigns across any ad channels. You can report on platform level metrics like ad spend and impressions or see conversions from each campaign with customized attribution models.
## Configuring your view
To configure your view, follow these steps to select the appropriate columns:
### 1. Select columns
- **Asset columns**
Asset columns describe the asset and include attributes such as the created date, author, status, account, channel, and geography. These columns are sourced from the asset model in the schema layer.
- **Asset metrics**
Asset metrics typically include cost, clicks, and impressions. For ads reporting, these metrics are loaded from ad platforms and processed in the warehouse. Note that these metrics are usually reported at the campaign/ad set/ad level and cannot be tied to individuals. Learn more from dbt [docs](https://hub.getdbt.com/fivetran/ad_reporting/latest/). These metrics are created from the ad stats model in the schema layer.
- **Conversion metrics**
Conversion metrics are based on metrics defined in the metrics layer and use the attribution model defined in the attribution layer. Common conversion metrics include the number of transactions, total revenue, unique user transactions, bookings, and product views. These metrics can be derived from any customer event, both online and offline.
### 2. Apply filters
Filters enable you to customize the view to see only the campaigns or ads that are relevant. You can filter on any of the columns defined above. Common filters include filtering by campaign name, campaign dates, or spend amounts. You can apply string-based filters or numeric filters.
### 3. Time range
Campaign Intelligence has four default time ranges: 7 days, 14 days, 30 days, and 90 days.
Note: the time range is not to be confused with the attribution lookback
window. The time range only determines which conversions to report in the UI,
whereas the lookback window in the attribution model determines which
touchpoints should be attributed for a conversion.
### 4. Column sorting
You can customize your view by sorting your columns either alphabetically or numerically.
## Saving your view
You can save a view to revisit later or share across your team. The saved view captures the current filters, sorts, visible columns, and time range for a given table. You can quickly switch between them in the "Views" menu.
{/* */}
Additional documentation is coming soon, please if you're interested in this feature.
{/* */}
---
## Assets
**URL:** https://hightouch.com/docs/campaign-intelligence/assets
**Description:** Assets are the most granular elements of a campaign.
**Section:** Campaign Intelligence
## Overview
**Assets** are the most granular elements of a campaign. For paid channels this might be ads and for lifecycle marketing this might be an email or even a variant.
We support the following **Asset types**:
- Emails
- SMS
- Ads
{/* */}
---
## Attribution Methods
**URL:** https://hightouch.com/docs/campaign-intelligence/attribution-methods
**Description:** Build attribution methods across touchpoints and conversions from all channels.
**Section:** Campaign Intelligence
## Overview
The attribution layer defines the logic for how conversions should be attributed to touchpoints (**Events**).
### Definitions
- **Touchpoint**
A touchpoint is any **Event** or engagement a potential customer has with your brand, product, or service. This can include a variety of actions such as visiting your website, clicking on an advertisement, opening an email, or interacting with your social media posts. Each touchpoint is a step in the customer journey and can influence the customer's decision to convert.
- **Conversion**
A conversion occurs when a potential customer completes a desired action, turning from a prospect into an actual customer. Conversions can be defined in different ways depending on the business goals, such as making a purchase, signing up for a newsletter, filling out a contact form, or downloading a white paper. Conversions are critical metrics for assessing the effectiveness of marketing efforts and overall business success.
### Example conversion path:
In the above path, you can see the user had four touchpoints before converting. An attribution method determines which touchpoints should be included in the conversion path and how you distribute conversion credit across touchpoints.
## Creating an attribution method
### Determining which touchpoints to include
- **Event type**
Define the types of **Events** that qualify as touchpoints. Common **Event types** include:
- Website visits
- Clicks on advertisements
- Email opens
- Email sends
- **Lookback window**
The lookback window is the period during which **Events** are considered relevant for attribution. Define the length of this window based on your business goals. For example, a 30-day lookback window considers all **Events** within the last 30 days prior to a conversion. You can set custom lookback windows for each **Event type**.
### Determining weighting
- **First touch**
First touch attribution gives all the credit for the conversion to the first **Event** a customer has with your brand. This method is useful for understanding which channels or campaigns are effective at creating initial awareness.
- **Last touch**
Last touch attribution gives all the credit for the conversion to the last **Event** before the conversion. This method helps identify which channels or campaigns are effective at driving final conversions.
- **Participation**
Participation, or linear attribution, assigns equal credit to all touchpoints in the customer journey. This method provides a balanced view of how each **Event** contributes to the conversion.
{/* - **Custom multi-touch (Coming soon)** */}
{/* This upcoming feature will allow you to create custom attribution methods that assign different weights to various touchpoints based on their role in the conversion process. This flexible method helps tailor the attribution to better reflect your unique customer journey and marketing strategy. */}
{/* */}
Please if you're interested in metrics and attribution!
{/* */}
---
## Campaign Schema
**URL:** https://hightouch.com/docs/campaign-intelligence/campaign-schema
**Description:** Schema information and diagrams for Hightouch's Campaigns product.
**Section:** Campaign Intelligence
## Overview
Hightouch's Campaigns product leverages two main model types: Events and Assets. This guide walks you through configuring these models in your schema.
### Events
**Events** connect a user to an asset model. **Events** require:
1. a timestamp
2. a relationship with a parent model
3. a relationship with an asset model
Apply one of the standard **Event types** to an **Event** to populate the default metrics on the various asset dashboards of the Campaigns page. We will add more **Event types** over time.
{/* */}
### Assets
**Assets** are the most granular elements of a campaign. For paid channels this might be ads and for lifecycle marketing this might be an email or even a variant.
We support the following **Asset types**:
- Emails
- SMS
- Ads
{/* */}
### Message assets
Campaigns for [messages](/campaign-intelligence/emails), such as emails and SMS, leverages three types of models:
1. **Users** are the [parent model](/customer-studio/schema#1-define-the-parent-model).
2. [**Events**](/campaign-intelligence/events) are events that connect users to an asset, such as email clicks or opens.
3. **Emails** are a type of [asset](/campaign-intelligence/assets).
#### Schema
In the entity relationship diagram (ERD) below, we show a schema for messages using emails as an example. The ERD shows expected model columns and their relationships, with necessary primary (PK) and foreign keys (FK).
In the ERD above, Users are connected to different **Events** -- such as email opens and clicks -- through a unique `user_id`. **Events** require a `timestamp`. They can have additional columns based on the **Event type**, such as a `clicked_url` for email click events.
Each **Event** is also linked to an **Asset model**, either email or SMS, by way of a `message_id`. The message can have a variety of columns, including date columns such as `start_at` and `ended_at` to indicate the dates that email asset was in use.
### Ad assets
Campaigns for [ads](/campaign-intelligence/ads) leverages four types of models:
1. **Users** are the [parent model](/customer-studio/schema#parent-models).
2. **Sessions** are a type of [**Event**](/campaign-intelligence/events) and connect a user to an ad. They necessarily have a timestamp and typically include `utm` parameters.
3. **Ads** are a type of [**Asset**](/campaign-intelligence/assets), and minimally include campaign and platform information.
4. **Ad stats** is a model that indicates engagement with an ad, such as clicks, impressions, and spend. It should fit the format of [this dbt model](https://fivetran.github.io/dbt_ad_reporting/#!/model/model.ad_reporting.ad_reporting__ad_report).
The [relationships](/customer-studio/schema#related-models-and-events) between these models depend on whether your ad model has information about the ad itself, such as an `ad_name` or `ad_group_id` alongside the ad campaign information or just campaign information.
Below are [entity relationship diagrams](https://en.wikipedia.org/wiki/Entity%E2%80%93relationship_model) (ERDs) that show expected columns and relationships in both situations, including necessary primary (PK) and foreign keys (FK). We note differences between the schemas.
#### Schema with only campaign data for ads
The ERD above includes the important features:
- Sessions connect to **Ads** with a campaign identifier based on what's available in the session, in this case through a mapping from `utm_campaign` to `campaign_name`.
- **Ads** also connect to **Ad stats** through a campaign identifier, in this case the `campaign_id`.
- Hightouch generates a primary key in the **Ad stats** model that combines the `date_day` and `campaign_id` columns.
#### Schema with ad, ad group, and campaign data
Two notable differences in the schema are evident when there is additional ad and ad group information.
- Sessions now connect to **Ads** through an identifier at the level of an individual ad, in this case with a mapping from `utm_content` to `ad_name`.
- **Ads** also now connect to **Ad stats** through an individual ad level identifier, here the `ad_id`.
- Both **Ads** and **Ad stats** have additional columns for individual ad level information.
---
## Charts Overview
**URL:** https://hightouch.com/docs/campaign-intelligence/charts
**Description:** Learn what how to configure, save, and update charts
**Section:** Campaign Intelligence
## Chart types
Charts are the building blocks of Hightouch's Intelligence features. We currently support several chart types:
* [Insights](https://hightouch.com/docs/campaign-intelligence/insights): Analyze audience composition, breakdowns, and performance.
* [Funnels](https://hightouch.com/docs/campaign-intelligence/funnels): Identify drop-off points across a sequence of events.
* [Splits](https://hightouch.com/docs/campaign-intelligence/splits): Measure results of experiments between randomized groups.
## Key elements
While the analysis varies between chart types, the general anatomy stays consisent across them.
- **Parent model\:** The dataset for your analysis, defined in the schema setup step.
- **Chart type\:** The analysis type (Insights, Funnels, Splits).
- **Metrics\:** An aggregation over events. Metrics can have an aggregation type, filters, and an attribution window. Metrics can be on the cohort level (sum of purchases for all users) or on the user level (average purchase amount per user).
- **Segment by\:** The segment of users to include in the analysis. Default is to include the entire parent model. You can select multiple existing audiences or create inline audiences. If using an attribution window, events are only included if the user falls within the attribution window for the audience-metric pair.
- **Group by\:** The dimension to breakdown a metric by. There are several types of dimensions:
- User properties: Attributes, traits, or merge columns of a user. Examples: gender, state, loyalty status
- Event properties: Attributes on the specific event. Examples: brand, campaign, utms.
- Related properties: Attributes on a related model. Examples: item, item property.
- **Time interval\:** Time granularity for metric rollup (daily, weekly, monthly). For example, sessions from distinct users per day = DAU, per week = WAU.
- **Time window\:** The time period for analysis. Events outside this window won't contribute to metrics.
- **Saved charts\:** Drawer showing all saved charts across the workspace.
- **Export as CSV\:** Option to download the chart's datapoints as CSV.
- **Sampling\:** For parent models that have Sampling turned on, enable `Fast queries` to query a sampled percentage of your parent model.
- **Mode\:** The type of data aggregation (`Over time` or `Cumulative`) when viewing time-series charts (Line, Column, and Area).
- **Visualization\:** The type of data visualization (Line, Column, Area, Bar, Table, Heat map, Number).
---
## Copilot
**URL:** https://hightouch.com/docs/campaign-intelligence/copilot
**Description:** Hightouch's Campaign Intelligence
**Section:** Campaign Intelligence
Documentation is coming soon. Please see the video below for a preview of our Copilot features.
---
## Dashboards
**URL:** https://hightouch.com/docs/campaign-intelligence/dashboards
**Description:** Hightouch's dashboards feature provides a snapshot of your audience and performance data.
**Section:** Campaign Intelligence
**Audience:** Marketers and analysts
**Prerequisites:** Saved charts created from your schema models
*Use dashboards to combine saved charts into a single, visual workspace where you can analyze trends, validate campaign logic, and monitor performance.*
***
## Overview
Dashboards let you organize saved charts in a flexible grid layout. You can use them to:
* Explore key metrics and user behavior
* Validate audience criteria before launching campaigns
* Monitor engagement, conversions, and lift over time
Dashboards are composed of saved **Charts**. [Each chart](https://hightouch.com/docs/campaign-intelligence/overview#campaign-intelligence-chart-types) is a data visualization created from your models (users, events, and related models). Charts can be reused across dashboards.
All charts in a dashboard must share the same parent model.
## Who uses it
| Role | Responsibilities |
| ----- | ----- |
| **Data, Marketing Ops or MarTech teams** | Add and configure charts and dashboards |
| **Marketers** | Explore and filter dashboards |
## Create a dashboard
### Step 1: Open the Dashboards tab
1. Go to `Intelligence → Dashboards` to see all dashboards in your workspace.
2. Click `Create dashboard` in the top right.
### Step 2: Name your dashboard and choose a parent model
In the modal:
1. Select a parent model (e.g. Users).
2. Enter a dashboard name.
3. Optionally, add a description.
4. Click `Create dashboard` to save and open your new dashboard.
### Step 3: Add charts
Click `Add chart` to choose one of the following options:
* **Saved chart**: Add an existing chart from your library
* [**Insights**](https://hightouch.com/docs/campaign-intelligence/insights) **chart**: Build a new visualization to measure your marketing efforts
* [**Funnel**](https://hightouch.com/docs/campaign-intelligence/funnels) **chart**: Track drop-off across a multi-step flow
After adding charts, use drag-and-drop to arrange and resize them in your layout. Additional charts can be added within the same row or a new row, unless the maximum number of 4 charts for the row has been reached.
Archived charts will still appear on existing dashboards, but can't be added to new dashboards.
## Apply filters
Use the filter bar at the top of the dashboard to apply filters across all charts. You can filter by:
* **Time range**: Default (respects the time filters on the individual charts), 7d, 30d, 60d, 90d, or set a custom range.
* **Properties**: Parent, event, and related model fields like `country`, `device_type`.
* **Audience membership**: One or multiple audiences can be applied to the dashboard.
Filters help you explore performance across specific timeframes, properties, or user segments.
Filters applied in View mode are temporary. Switch to Edit mode and click **Save changes** to make them permanent.
## Edit and manage dashboards
### Modes and editing
Dashboards support two modes:
| Mode | Description |
| ----- | ----- |
| **Edit** | Add, remove, or resize charts. Update filters and layout. |
| **View** | Explore charts and apply temporary filters. No changes saved. |
To save layout or filter changes, switch to Edit mode and click `Save changes`.
### Manual refresh and data retention
* Dashboard data is cached for up to **30 days**
* Each dashboard displays a `Last refreshed` timestamp
* Click the refresh button to manually reload chart data. New filters are not automatically applied, you must refresh the dashboard to apply them.
### Scheduled refresh
You can also schedule dashboard refreshes so data is automatically loaded the next time you open the dashboard. The following options are available:
* Manual: The dashboard can only be refreshed manually.
* Custom recurrence: The dashboard refreshes at a specific time and schedule (for example, every Monday at 9:00 AM).
### Chart behavior
If a saved chart is updated or removed after being added to a dashboard:
| Saved chart status | Behavior in dashboard |
| ----- | ----- |
| **Deleted** | A deleted chart placeholder appears |
| **Updated** | A warning is displayed. Changes appear only after the dashboard is refreshed |
| **Archived** | An archived badge appears until removed or restored |
### Sharing dashboards
Dashboards are visible to anyone in your workspace with access to the parent model and can be shared via link with other workspace members.
### Exporting dashboards
The data displayed in the dashboard charts can be exported. The export generates a ZIP file containing a separate CSV for each individual chart in the dashboard.
### Dashboard performance
* Limit dashboards to **20 charts** for optimal performance
* If performance is a concern, consider leveraging [sampling](https://hightouch.com/docs/customer-studio/sampling) (fast queries). If enabled, it applies to the entire dashboard, not individual charts. Sampling *could* impact the accuracy of certain charts.
## Example: Campaigns performance dashboard
**Goal:** Monitor engagement and conversions for Q3 promo campaigns
1. Create a saved chart for **audience size over time**
2. Create a dashboard named **Q3 campaigns monitoring**
3. Add:
* Saved audience size chart
* Build a chart for "Email delivered"
* Build a chart for "Checkouts" in-line
4. Apply a **Last 30 Days** global filter
5. Refresh as needed to monitor performance throughout the campaign
---
## Emails
**URL:** https://hightouch.com/docs/campaign-intelligence/emails
**Description:** The Emails view in Hightouch's Campaigns feature helps you understand conversions and campaign data from your email messages.
**Section:** Campaign Intelligence
## Overview
The emails dashboard is the best way to view performance for all your email messages. You can view engagement metrics like sends, opens, and clicks along with custom conversion metrics such as transactions.
## Configuring your view
To configure your view, follow these steps to select the appropriate columns:
### 1. Select columns
- **Asset columns**
Asset columns describe the asset and include attributes such as the created date, author, status, variant, and campaign type. These columns are sourced from the asset model in the schema layer.
- **Engagement metrics**
Engagement metrics include sent, delivered, opened, clicked, click rate, unique clicks, and unsubscribed. These events are typically generated from channel tools such as Iterable, Braze, SFMC, Klaviyo, and Attentive and loaded into the warehouse. These metrics are pre-defined based on the **Event type** selected.
- **Conversion metrics**
Conversion metrics are based on metrics defined in the metrics layer and use the attribution model defined in the attribution layer. Common conversion metrics include the number of transactions, total revenue, unique user transactions, bookings, and product views. These metrics can be derived from any customer event, both online and offline.
### 2. Apply filters
Filters enable you to customize the view to see only the campaigns or ads that are relevant. You can filter on any of the columns defined above. Common filters include filtering by campaign name, campaign dates, or spend amounts. You can apply string-based filters or numeric filters.
### 3. Time range
Campaign Intelligence has four default time ranges: 7 days, 14 days, 30 days, and 90 days.
Note: the time range is not to be confused with the attribution lookback
window. The time range only determines which conversions to report in the UI,
whereas the lookback window in the attribution model determines which
touchpoints should be attributed for a conversion.
### 4. Column sorting
You can customize your view by sorting your columns either alphabetically or numerically.
## Saving your view
You can save a view to revisit later or share across your team. The saved view captures the current filters, sorts, visible columns, and time range for a given table. You can quickly switch between them in the “Views” menu.
{/* */}
Additional documentation is coming soon, please if you're interested in this feature.
{/* */}
---
## Events
**URL:** https://hightouch.com/docs/campaign-intelligence/events
**Description:** Hightouch's Campaigns connects user events to asset models in a Schema's parent model.
**Section:** Campaign Intelligence
## Overview
**Events** connect a user to an asset model. **Events** require:
1. a timestamp
2. a relationship with a parent model
3. a relationship with an asset model
Apply one of the standard **Event types** to an **Event** to populate the default metrics on the various asset dashboards of the Campaigns page. We will add more **Event types** over time.
{/* */}
---
## Experiments
**URL:** https://hightouch.com/docs/campaign-intelligence/experiments
**Description:** Measure the impact of your marketing campaigns using experiment results from Audience Experiments.
**Section:** Campaign Intelligence
**Experiments** were previously called **Splits**.
| | |
|----------------------|-------------------------------------------------------------------------------------------------------------------|
| **Audience** | Marketers and analysts who want to compare performance across randomized test and control groups. |
| **Prerequisites** |
Created [Audience Experiments](/customer-studio/splits) in Customer Studio
Audience synced at least once after experiments were created
Event data (such as purchases or clicks) available for measurement
|
*Experiments help you evaluate how different marketing strategies perform by analyzing outcomes across randomized experiment groups. You can measure lift, compare treatment and holdout results, and understand whether your campaigns drive meaningful changes in user behavior.*
***
## Learning objectives
After reading this article, you’ll know how to:
- [Identify how Experiments are created and managed](#how-experiments-are-created)
- [Set up Audience Experiments for measurement](#setup-and-requirements)
- [Navigate to and review all Experiments](#1-view-list-of-experiments)
- [Configure metrics, start dates, and measurement windows](#2-configure-an-experiment)
- [Interpret lift, confidence intervals, and performance trends](#3-interpret-results)
- [Normalize results for clearer comparisons](#4-normalize-results)
---
## Overview
**Experiments** provide a measurement layer for [Audience Experiments](/customer-studio/splits).
Whenever you create an Audience Experiment in Customer Studio or configure an experiment on an [A/B split node in Journeys](/customer-studio/journeys#ab-split), Hightouch automatically generates a corresponding **Experiment** so you can track performance across randomized groups.
Experiments allow you to:
- Compare holdout vs. treatment outcomes
- Analyze lift and confidence intervals
- Visualize performance over time
- Evaluate strategies continuously
| Feature | Description |
|--------|-------------|
| **Experiment results** | Displays lift, performance trends, and confidence intervals |
| **Configuration** | Controls metrics, measurement windows, and start dates |
| **Normalization** | Enables per-member or baseline-scaled comparisons |
Experiment measurement charts have been updated. All experiment reporting now lives in **`Intelligence` → `Experiments`**.
---
## How Experiments are created
Experiments are automatically managed based on Audience Experiments in Customer Studio:
- Creating an **Audience Experiment** automatically creates a corresponding **Experiment**.
- Disabling or deleting an Audience Experiment removes its Experiment.
- Restoring an Audience Experiment restores its Experiment.
This ensures measurement stays aligned with the audiences you're actively using.
---
## Setup and requirements
### 1. Create Audience Experiments
Before measuring experiment results:
1. Create [Audience Experiments](/customer-studio/splits) in Customer Studio.
2. Ensure the audience has synced at least once *after* the experiment groups were created.
3. Confirm that users generate measurable events (such as purchases, page views, or clicks).
For every Audience Experiment created in Customer Studio, Hightouch automatically creates a corresponding Experiment in **`Intelligence` → `Experiments`**. Deleting or restoring an Audience Experiment removes or restores the linked Experiment.
---
## Measure results
The **`Experiments`** section of Intelligence helps you compare outcomes between experiment groups and evaluate the impact of your campaigns.
Navigate to: **`Intelligence` → [`Experiments`](https://app.hightouch.com/experiments)**
---
### 1. View list of Experiments
The Experiments list shows all Experiments, their statuses, and recent updates.
Statuses include:
- **Draft:** Missing one or both required configuration elements (primary metric or start date).
- **Scheduled:** Fully configured; the start date is in the future.
- **Running:** Fully configured; the start date is today or in the past.
---
### 2. Configure an experiment
Open an experiment and select the **`Configuration`** tab.
From here, you can:
- **Choose a primary metric** (required) and optional secondary metrics (e.g., *Conversions*, *Revenue*).
- [Learn how to create a metric](https://hightouch.com/docs/campaign-intelligence/metrics).
- *Optional:* Use the **`Filter by`** dropdown to refine by user properties or events.
- **Set a Start date**
- Determines when measurement begins.
- Does **not** apply retroactively.
- Does *not* affect sync or activation behavior.
- **Choose a Measurement window**
- Example: *Entry → 30 days after entry* measures events from the moment the user enters the audience to 30 days later.
---
### 3. Interpret results
The **`Overview`** tab displays experiment outcomes and performance trends.
Key elements:
- **Lift %:** Percentage difference between treatment and holdout group performance.
- **Lift interval bar:**
- **Green:** Significant positive lift
- **Red:** Significant negative lift
- **Gray:** Not statistically significant (interval overlaps 0%)
- **Performance lines:**
- Solid lines show performance over time
- Shaded regions represent 95% confidence intervals
Lift intervals use a Bayesian method, enabling continuous monitoring without needing to wait for an experiment to complete.
---
### 4. Normalize results
Use the **`Normalization`** dropdown to switch perspectives:
- **Per member (default):** Average performance per user
- **Normalized to baseline group:** Scales performance to compare groups evenly
Hover over the lift card to view raw totals.
---
## Funnels
**URL:** https://hightouch.com/docs/campaign-intelligence/funnels
**Description:** Hightouch's Customer Studio Funnels helps you pinpoint funnel drop-offs and track conversion rates over time.
**Section:** Campaign Intelligence
## Overview
Funnels are an easy way to explore where dropoff occurs across a customer's journey or analyze any sequence of events. This is important to understand which touch points need to be improved to improve overall conversion rate.
### Building a Funnel
To build a funnel analysis in Hightouch, follow these steps:
1. Select Starting Event: Choose the initial event to analyze. This can be any specific or general event.
2. Add Event Properties: Apply filters to events by selecting property names and values.
3. Define Event Sequence: Choose additional events and set the order (exact order, any order, this order).
4. Segment Users: Identify and segment the user base for the analysis by adding properties or importing saved segments.
### Compare Segments
While funnels tell you where your customers are dropping off overall, it doesn't tell you who is dropping off at each point. Get deeper insights by comparing conversion paths against different segments, user properties, or event properties.
### Create an Audience
Once you know who is dropping off where, you are in a great spot to nudge them past this point. You can create an audience by selecting on the chart where you can then sync it to any of our destinations.
---
## Insights
**URL:** https://hightouch.com/docs/campaign-intelligence/insights
**Description:** Hightouch's Intelligence product provides the Insights feature to measure how your audiences are performing.
**Section:** Campaign Intelligence
## Overview
Suppose you've just built a new [audience](/customer-studio/usage) and deployed them to a campaign.
You are probably wondering: What actions are they taking? How has my audience reacted to that new campaign?
The Insights feature enables you to view breakdowns of your members and define campaign success metrics to measure your audience against over time, all within Hightouch. You can track insights on metrics such as:
- Revenue per country
- Number of subscription sign-ups
- Number of inactive members who've recently logged back in
You can then track how these metrics perform over time and compare them across audiences.
Insights also works seamlessly with Hightouch [Splits](/customer-studio/splits). You can set up an A/B test within Hightouch, see how each split group performs, and determine a winner.
This doc provides step-by-step instructions for creating **Insight** charts.
## Understanding the core concepts
#### Custom metrics (“Saved metrics”)
You can define and save metrics using any [**event**](/customer-studio/schema#3-add-event-models) in your [data schema](/customer-studio/schema).
Metrics are defined on a [parent-model](/customer-studio/schema#1-define-the-parent-model) basis, and you can enable them on any audience built off the parent model.
#### Events and related models
Events and related models provide additional characteristics or actions to filter your [parent-model](/customer-studio/schema#1-define-the-parent-model). They're similar, but the key difference is that events require a timestamp.
Events and related models contain **properties** — if you think of each instance as a row, properties are the columns.
#### Properties
Properties that are defined by your parent model. If you think of each member in your parent model as a row, member properties are the columns.
#### Traits
[Traits](/customer-studio/traits) are custom columns added to your parent model.
## Getting started with Insights
Within the **Charts** product, the **Insights** tab allows you to create these charts. You can also view Insights charts that have already been created and saved.
## Metrics and properties
You can base an insight chart off of several different measurement options, and depending on the type you select, you can select an aggregation type, define a measurement window and add filters.
- **Aggregation type:** Defines how individual instances are combined to form the metric – for example, the total number of rows, unique members, or the sum, count or average of a property.
- **Measurement window:** Specifies the time period in which the selected metric must occur - for example, measuring an event from audience entry to 30 days after audience exit.
- **Filters:** Narrow down the data included. Filters are based on the properties of the models you use – for example location, product type, or price.
### Saved metrics
Saved metrics already include an aggregation, so additional aggregations can't be applied within **Charts**. However, you can change the measurement window and filters.
### Events and event properties
Different aggregations are available depending on whether an event property is included.
### Related model and related model properties
Different aggregations are available depending on whether a [related model](/customer-studio/schema#2-add-related-models) property is included.
### Properties
You can use the property you select in an aggregation on the parent model. Different aggregations are available depending on whether the member property is included.
Additional filters cannot be applied, but the metric can be **segmented** and **grouped**, as detailed below.
### Traits
Traits have a per-member aggregation built in, so additional aggregations can't be applied within **Charts**.
## Segmenting and grouping
### Segmenting
Segment data by one or more predefined Audiences, or create ad-hoc member segments, to focus the analysis on specific groups of members.
### Grouping
Group your data based on one or more selected properties or Traits (e.g., gender, location, brand loyalty). This allows you to compare the metric across different segments of members.
## Visualizations
### Line charts
Used to view and compare data trends over-time or cumulatively, across a selected time-frame and interval (e.g., daily, weekly). Ideal for tracking changes over time and comparing multiple series.
**Examples:**
**How many marketing emails did I send to this audience weekly over the last 60 days?**
**How many marketing emails did I send to these two audiences weekly over the last 60 days?**
### Column charts
View and compare aggregated values across categories using vertical bars. This can be done over-time or cumulatively, across a selected time-frame and interval (e.g., daily, weekly). Great for visual side-by-side comparisons.
**Example: Across two audiences, how can I compare how many members of each audience have started a checkout process over the last 7 days?**
### Bar charts
View and compare aggregated data across segments and groupings using horizontal bars for a given time frame. Best for top-N comparisons or summaries.
**Example: How is my audience distributed by state?**
### Area charts
View changes in data volume over-time or cumulatively, across a selected time-frame and interval (e.g., daily, weekly). Useful for visualizing contribution trends and growth.
**Example: What is the cumulative number of marketing emails sent to my audience over the last 30 days?**
### Table view
View aggregated data in rows, allowing for detailed comparison across values of a specific property for a given time-frame. Ideal for in-depth row-by-row analysis.
**Example: How many members by state have received marketing emails over the last 60 days?**
### Number chart
Communicate the most important numbers at a glance, typically used on dashboards to display and track prominent metrics or key performance indicators.
**Example: How many total members are in my audience?**
## Saving a chart
You can use the `Save chart` button to save any chart for future use. Once saved, the chart can be accessed from the Charts panel.
## Exporting a chart
You can use the `Export as CSV` button to download a chart’s data points in CSV format. You can export the data only after the chart has finished running.
---
## Metrics
**URL:** https://hightouch.com/docs/campaign-intelligence/metrics
**Description:** Learn how to create custom metrics to use in the Intelligence product.
**Section:** Campaign Intelligence
## Overview
A metric is a quantifiable measure used to track and assess the status of a specific business process. Metrics help in understanding performance and identifying areas for improvement. Key components of a metric are shown below:
### Creating a Metric
- **Event**
An event is a specific action or occurrence that can be measured. Examples include a user clicking a button, a transaction being completed, or a user becoming a subscriber.
- **Aggregation Type**
The aggregation type defines how individual events are combined to form the metric. Our aggregations include sum, count, unique count, and average.
- **Filter**
Filters are used to narrow down the data included in a metric. Filters can be based on various criteria such as location, product type, or price.
### Adding an Attribution Method
While metrics can stand alone, you must apply an attribution method to tie them back to a campaign, enabling you to understand the impact of different marketing efforts on these metrics.
Metrics can have multiple attribution methods. This is useful for gaining a more comprehensive understanding of how different events contribute to conversions. Learn more about attribution [here](/campaign-intelligence/attribution-methods).
---
## Intelligence overview
**URL:** https://hightouch.com/docs/campaign-intelligence/overview
**Description:** Hightouch Intelligence gives marketers and analysts self-serve access to explore, measure, and visualize customer data directly from the warehouse.
**Section:** Campaign Intelligence
| | |
| ----------------- | ------------------------------------------------------------------------------------------------------ |
| **Audience** | Marketers, analysts, and data teams who want to explore customer data, measure experiments, and track performance in Hightouch. |
| **Prerequisites** | Access to Customer Studio with at least one connected source. |
---
### Learning objectives
After reading this article, you’ll understand:
* What Intelligence is and how it fits into the **Customer Studio** workflow
* Key features available in Intelligence
* When to use Intelligence versus other analytics tools
* How to access Intelligence in the Hightouch app
---
## Overview
**Intelligence** is a set of analytics features that give you answers directly in your workflow.
Instead of switching between platforms or waiting for reports, you can check performance, measure experiments, and explore customer data—all while building audiences and campaigns in Hightouch.
Within Intelligence, you can:
* **Visualize trends and patterns** with **Charts**, including [Insights](/campaign-intelligence/insights), [Funnels](/campaign-intelligence/funnels), and audience comparisons.
* **Measure success consistently** by defining shared KPIs with [Metrics](/campaign-intelligence/metrics).
* **Test and learn** by creating audience [Splits](/customer-studio/splits) and [Split Charts](/campaign-intelligence/splits) to compare results between groups.
* **Monitor performance over time** with [Dashboards](/campaign-intelligence/dashboards), which combine multiple charts into a single view to analyze trends, validate campaign logic, and track outcomes.
Because Intelligence runs directly on your connected warehouse data, you’re always working from the same, up-to-date source of truth used for segmentation and activation.
---
## Key features and examples
| Feature | What it does | Example use case |
| ----- | ----- | ----- |
| [**Metrics**](/campaign-intelligence/metrics) | Define and reuse performance metrics across teams. | Track “Conversion rate” consistently across every campaign so teams use the same calculation. |
| [**Charts**](/campaign-intelligence/charts) | Build visualizations to track audience behavior and outcomes. | Compare purchases from loyalty members vs. non-members over the last 30 days. |
| [**Dashboards**](/campaign-intelligence/dashboards) | Combine saved charts into a single, visual workspace to monitor performance. | Track engagement and conversions for an ongoing promotional campaign. |
---
### Chart types
| Chart type | What it does | Example use case |
| ----- | ----- | ----- |
| [**Insights**](/campaign-intelligence/insights) | Analyze audience composition, breakdown, and performance. | See how many members of an audience have booked a trip in the past 90 days. |
| [**Funnels**](/campaign-intelligence/funnels) | Identify drop-off points across a sequence of events. | See where users abandon the checkout process. |
| [**Splits**](/campaign-intelligence/splits) | Measure results of experiments between randomized groups. | Compare purchase rates between customers who received a discount and those who didn’t. |
---
## Value for marketers
Most marketing and advertising tools only show analytics for campaigns run on their platforms. Business Intelligence tools can provide broader analysis but sit outside the campaign workflow.
**Hightouch Intelligence** bridges that gap—bringing full-funnel analytics into the same workspace where marketers activate campaigns and data teams manage the source of truth.
With Intelligence, you can:
* **Analyze in context** – View performance and data insights while building or optimizing audiences and journeys.
* **Measure across channels** – Use consistent metric definitions for email, ads, and other touchpoints.
* **Act on results immediately** – Move from insight to activation without switching platforms.
* **Monitor campaign health** – Use Dashboards to track engagement, conversions, and lift over time.
---
## Access Intelligence
1. Go to [**app.hightouch.com/analytics**](https://app.hightouch.com/analytics).
2. Select **Charts**, **Metrics**, or **Dashboards** to begin your analysis.
---
## Ads Schema
**URL:** https://hightouch.com/docs/campaign-intelligence/schema/ads
**Description:** Schema information and diagrams for Hightouch's Campaign Intelligence for ads.
**Section:** Campaign Intelligence
## Overview
Campaign Intelligence for [ads](https://hightouch.com/docs/campaign-intelligence/ads) leverages four types of models:
1. **Users** are the [parent model](https://hightouch.com/docs/customer-studio/schema#1-define-the-parent-model).
2. **Sessions** are a type of [**Event**](https://hightouch.com/docs/campaign-intelligence/events) and connect a user to an ad. They necessarily have a timestamp and typically include `utm` parameters.
3. **Ads** are a type of [**Asset**](https://hightouch.com/docs/campaign-intelligence/assets), and minimally include campaign and platform information.
4. **Ad stats** is a model that indicates engagement with an ad, such as clicks, impressions, and spend. It should fit the format of [this dbt model](https://fivetran.github.io/dbt_ad_reporting/#!/model/model.ad_reporting.ad_reporting__ad_report).
The [relationships](https://hightouch.com/docs/customer-studio/schema#relationships) between these models depend on whether your ad model has information about the ad itself, such as an `ad_name` or `ad_group_id` alongside the ad campaign information or just campaign information.
Below are [entity relationship diagrams](https://en.wikipedia.org/wiki/Entity%E2%80%93relationship_model) (ERDs) that show expected columns and relationships in both situations, including necessary primary (PK) and foreign keys (FK). We note differences between the schemas.
## Schema with only campaign data for ads
The ERD above includes the important features:
- Sessions connect to **Ads** with a campaign identifier based on what's available in the session, in this case through a mapping from `utm_campaign` to `campaign_name`.
- **Ads** also connect to **Ad stats** through a campaign identifier, in this case the `campaign_id`.
- Hightouch generates a primary key in the **Ad stats** model that combines the `date_day` and `campaign_id` columns.
## Schema with ad, ad group, and campaign data
Two notable differences in the schema are evident when there is additional ad and ad group information.
- Sessions now connect to **Ads** through an identifier at the level of an individual ad, in this case with a mapping from `utm_content` to `ad_name`.
- **Ads** also now connect to **Ad stats** through an individual ad level identifier, here the `ad_id`.
- Both **Ads** and **Ad stats** have additional columns for individual ad level information.
---
## Messages Schema
**URL:** https://hightouch.com/docs/campaign-intelligence/schema/messages
**Description:** Schema information and diagrams for Hightouch's Campaign Intelligence for email and SMS messages.
**Section:** Campaign Intelligence
## Overview
Campaign Intelligence for [messages](https://hightouch.com/docs/campaign-intelligence/emails), such as emails and SMS, leverages three types of models:
1. **Users** are the [parent model](https://hightouch.com/docs/customer-studio/schema#1-define-the-parent-model).
2. [**Events**](https://hightouch.com/docs/campaign-intelligence/events) are events that connect users to an asset, such as email clicks or opens.
3. **Emails** are a type of [asset](https://hightouch.com/docs/campaign-intelligence/assets).
## Schema
In the entity relationship diagram (ERD) below, we show a schema for messages using emails as an example. The ERD shows expected model columns and their relationships, with necessary primary (PK) and foreign keys (FK).
In the ERD above, Users are connected to different **Events** -- such as email opens and clicks -- through a unique `user_id`. **Events** require a `timestamp`. They can have additional columns based on the **Event type**, such as a `clicked_url` for email click events.
Each **Event** is also linked to an **Asset model**, either email or SMS, by way of a `message_id`. The message can have a variety of columns, including date columns such as `start_at` and `ended_at` to indicate the dates that email asset was in use.
---
## Business tier pricing
**URL:** https://hightouch.com/docs/pricing/bt-pricing
**Description:** Learn about Hightouch's Business tier pricing, including platform fees, consumption-based pricing, and feature add-ons.
**Section:** Pricing
{/* */}
{/* Allow for use of "We" in sensitive topics like pricing and security --> */}
Hightouch's Business tier pricing varies depending on your use case and required features.
It typically includes:
- A **platform fee**, which could include consumption-based pricing based on [active syncs](https://hightouch.com/docs/pricing/bt-pricing#active-syncs).
- Optional **feature add-ons**, such as [Customer Studio](/customer-studio/overview) features or the [Personalization API](/destinations/personalization-api).
This allows us to create a plan that fits your specific feature usage and consumption needs.
For the complete list of features included in Business tier packages, please visit our [pricing page](https://hightouch.com/pricing).
Don't hesitate to to receive a quote or discuss whether the Business tier is a good fit for you.
{/* */}
## SOC 2 report
If you are an existing Hightouch Business Tier customer, you can request a copy of our full SOC 2 report at the [Hightouch Trust Center](https://trust.hightouch.com/).
If you are not yet a Business Tier customer but are considering buying Business Tier, we can provide you the report under NDA.
## Active syncs
An **active sync** is any sync that has run at least once in the **previous calendar month**.
This includes syncs that are later deleted or disabled—if a sync ran successfully during that month, it is still counted as active. At the start of each new calendar month, the active sync count resets.
---
## Self-serve pricing
**URL:** https://hightouch.com/docs/pricing/ss-pricing
**Description:** Learn about our self-serve pricing and how Hightouch calculates usage based on active syncs.
**Section:** Pricing
{/* */}
{/* Allow for use of "We" in sensitive topics like pricing and security --> */}
This page describes Self-serve pricing (Free and Self-serve tiers).
Visit the [pricing page](https://hightouch.com/pricing) for a full feature comparison, or see the
[Business tier pricing](/pricing/bt-pricing) page to learn about Business plans.
## Overview
Hightouch’s self-serve pricing is based on the **number of [active syncs](https://hightouch.com/docs/pricing/ss-pricing#active-syncs)** in your organization.
## Plan limits
- **Free plan**: 2 active syncs per month.
- **Self-serve plan**: 10 active syncs per month.
There are **no limits** on destinations, destination types, user seats, or sync runs.
Free and Self-serve plans are limited to **hourly sync frequency**.
To ensure we can serve all customers sustainably, we impose an operations cap of 100,000,000 operations per month. Most workspaces fall under this limit. Please get in touch with us if you’re concerned about usage.
### Workspaces
[Organizations](/workspace-management/overview#core-concepts) on self-serve plans are limited to **one workspace per org**.
## Active syncs
An **active sync** is any sync that has run at least once in the **previous calendar month**.
This includes syncs that are later deleted or disabled—if a sync ran successfully during that month, it is still counted as active. At the start of each new calendar month, the active sync count resets.
{/* */}