Lookup table usage

After Hightouch runs identity resolution (IDR) on your graph, Hightouch outputs tables into your warehouse so they can be used for any business need. These are found in the hightouch_planner schema (or the override name of the internal schema for the source you're using.)

Your marketing team can use them for paid campaigns, your sales team for deduplicating outreach efforts, or data scientists for fraud dection.

The output of IDR can also helpful in constructing models or parent models in Hightouch.

Hightouch creates three output tables:

1. A RESOLVED table (<output_prefix>_resolved) that labels each row from your input tables with a HT_ID representing a profile. This table tells you all the input rows associated with a profile by its HT_ID. The SOURCE is the slug of an input model. The PRIMARY_KEY is the primary key value from the input model. The LATEST_TIMESTAMP is the timestamp of the row from the input model.

2. A RESOLVED_IDENTIFIERS table (<output_prefix>_resolved_identities) includes all the unique identifiers associated with each profile. Each row represents a unique identifiers associated with a profile. COUNT represents the number of times the identifier was seen in an input row, including updated rows if the same row gets seen multiple times due to updates. IDENTIFIER is the type of identifier. VALUE is the identifier value for the profile.

3. A UNRESOLVED table (<output_prefix>_unresolved) which lists all the input rows we couldn't process because they primary key of the rows were not unique within the model it came from.

4. GOLDEN_RECORD table (<output_prefix>_golden_records):
   (Optional) Only appears if you enable Golden Record in your IDR project's settings. Golden Record flattens all possible values of each identifier into either:

   • A single "best" or "primary" value based on survivorship rules (most recent, most frequent, or source priority)
   • An array of all unique values when using the array survivorship rule

   See our Golden Record doc for more details about survivorship rules and array fields.

   Each row corresponds to a single HT_ID (one identity). For example, a row might look like:

   {
     "ht_id": "ht1",
     "most_recent_email": "user@example.com",                -- using most recent survivorship
     "all_phone_numbers": ["123-456-7890", "098-765-4321"],  -- using array survivorship
     "primary_user_id": "user_123"                           -- using source priority survivorship
   }
   

State tables

In addition to the output tables, Hightouch generates and maintains state tables in the hightouch_planner schema for each identity graph. These tables are typically prefixed with IDR_XXX... where XXX represents a series of alphanumeric characters. These state tables are required for incremental runs and maintaining the consistency of your identity graphs. We may also store a couple of backup tables for each identity graph, prefixed by IDR_BACKUP_... in the hightouch_planner schema.