Overview
- Categorize all the errors in your sync using regular expressions, and find unexpected errors.
- Filter out previously failed rows from your model using a JOIN.
- Aggregate the sync history to see what rows are changing the most (flapping rows can be a sign of data problems).
- Visualize how rows in your models (e.g. targeted users in ad campaigns) are changing over time.
Schema
hightouch_audit
schema:sync_changelog
: This table contains a row for every operation performed by Hightouch. It includes the result of the operation, as well as any error messages from syncing.sync_snapshot
: This table contains the latest status of each row in your model. The information is very similar to thesync_changelog
table, but is easier to query for some use cases.sync_runs
: This table contains a log of all the sync runs. The changelog and snapshot tables can be JOINed to this table for more information on when the sync occurred, and how it was configured.
sync_id
column).Setup
- Snowflake
- BigQuery
- Redshift
- Databricks
- Postgres
Required permissions
hightouch_audit
schema. You should not require any additional permissions if you have set up
Warehouse
Planning.Enabling Warehouse Sync Logs for a sync
- Make sure Warehouse Planning is enabled for the source.

-
Go to the "Sync Logs" tab in your sync.
-
Enable your desired history tables.

Example queries
Get the most common sync error
select
failure_reason,
count(*) as c
from hightouch_audit.sync_snapshot
where failure_reason is not null
group by failure_reason
order by c desc

Track when users entered and exited a model
with details as (
select
model_name,
row_id,
op_type as type,
started_at as timestamp,
lag(op_type) over(partition by model_name, row_id order by started_at) as lag_type
from hightouch_audit.sync_changelog c
join hightouch_audit.sync_runs r on c.sync_id = r.sync_id
where op_type != 'changed'
order by model_name, row_id
)
select
row_id as user_id,
model_name as audience,
type,
timestamp
from details
where (lag_type != type or lag_type is null)
order by model_name, row_id, timestamp

Get the current rows in all models
with model_names as (
select distinct
sync_id,
model_name
from hightouch_audit.sync_runs
)
select
model_name,
row_id as user_id
from hightouch_audit.sync_snapshot s
join model_names r on s.sync_id = r.sync_id
where s.status != 'failed'
qualify row_number() over (partition by user_id, model_name order by null) = 1
order by user_id

Detailed schema
sync_snapshot
, sync_changelog
, and sync_runs
tables after each sync.hightouch_audit.sync_changelog
Column | Description |
---|---|
sync_id | The id of the sync |
sync_run_id | The id of the sync run |
op_type | Whether the row was added , changed , or removed relative to the last run. This is computed by Hightouch when planning the sync run |
row_id | The value of the row's primary key (as defined from the model) |
status | Whether the row was successfully synced into destination. They value may be: succeeded (the row was successfully synced), failed (Hightouch attempted to sync the row, but it failed to sync), and aborted (Hightouch planned to sync the row, but did not attempt to sync. This may happen if the sync may have been cancelled, or the sync encountered a fatal error that terminated the run early). |
failed_reason | If the status is failed , this will contain a string describing why the row failed to sync. |
fields | A JSON object of the raw data from the model that is being synced into destination. Note that this is the raw data from the warehouse, not the payload that Hightouch sent to the destination. This column is not supported on Redshift. |
hightouch_audit.sync_snapshot
Column | Description |
---|---|
sync_id | The id of the sync |
op_type | Whether the row was added , changed , or unchanged relative to the last run |
row_id | The value of the row's primary key (as defined from the model) |
status | The status of the row. See the sync_changelog.status description for a list of possible statuses |
failed_reason | If the status is failed , this will contain a string describing why the row failed to sync |
fields | The fields from the model for this row. See the sync_changelog.fields description for more information |
hightouch_audit.sync_runs
sync_changelog
and sync_snapshot
tables may be joined with this table using
the sync_id
column.Column | Description |
---|---|
sync_id | The id of the sync |
sync_run_id | The id of the sync run |
primary_key | The primary key column of your sync (as defined on the model attached to the sync) |
destination | The destination type (e.g. salesforce or braze) |
model_name | The name of the model attached to the sync |
status | The status of the sync run. This will be either succeeded or failed. In general, the per-row results of the sync are a better indication of status. |
error | The sync-level error if the sync terminated early |
started_at | When the sync run started |
finished_at | When the sync run finished |
num_planned_add | The number of planned adds. |
num_planned_change | The number of planned changes. |
num_planned_remove | The number of planned removes. |
num_attempted_add | The number of planned adds that were actually attempted. |
num_attempted_change | The number of planned changes that were actually attempted. |
num_attempted_remove | The number of planned removes that were actually attempted. |
num_succeeded_add | The number of planned adds that were successfully synced to the destination |
num_succeeded_change | The number of planned changes that were successfully synced to the destination |
num_succeeded_remove | The number of planned removes that were successfully synced to the destination |
num_failed_add | The number of planned adds that were attempted, but failed to get synced into destination |
num_failed_change | The number of planned changes that were attempted, but failed to get synced into destination |
num_failed_remove | The number of planned removes that were attempted, but failed to get synced into destination |
Frequently asked questions
What's the performance impact?
- Connection issues with your data warehouse?
- Confusing API responses from destination systems?
- Unsupported destination objects or modes?
- Help with complex SQL queries?
Need help?
Our team is relentlessly focused on your success. We're ready to jump on a call to help unblock you.
or
Feature Requests?
If you see something that's missing from our app, let us know and we'll work with you to build it!
We want to hear your suggestions for new sources, destinations, and other features that would help you activate your data.