Event Streaming

Event streaming lets you push events collected by Hightouch Events directly to destinations without going through the warehouse. Events stream in seconds, which is useful for latency-sensitive use cases. Event streaming is a type of Event Sync that takes events from an Event Source and syncs them to an Event Destination in real-time.

Use Cases

Post-purchase workflow to send upsell offers to users immediately following a purchase
Ad impression measurements so you can immediately pause ads that surpass their budget
Other time-sensitive emails or messages sent to users after significant events like a demo request, new user sign up, and missed call follow-up
Update users’ consent or contact status immediately after a change

When NOT to use event streaming

Generally, we recommend reverse ETL for most use cases since it allows for better enrichment, governance, and monitoring of the data you’re sending to different systems. For example, we recommend reverse ETL instead of event streaming when you’re syncing:

Conversion events which benefit from having as much identity information associated with the event as possible
Audiences which benefit from easily adding or removing users based on historical behaviors, not just single events

Event streaming should only be considered when sub-minute latency matters.

Setup

Required

Event Source already set up

Steps

Set up an event source first if you haven’t already
Create a new event streaming destination (or use an existing reverse ETL destination)

Note: Event streaming is referred to as event forwarding on the destination creation and selection page. We’re working to update the UI of any ambiguity here.

Enter credentials for the event streaming destination

Most destinations will ask for an API key—these are found and created in the event destination’s own website or app, not in Hightouch.

Name your event streaming destination
Create a new event sync
Select the event streaming destination you created
Configure event streaming destination

Within the setup wizard and configuration tab for the event destination, you’ll be able to configure settings like which events to sync, fields to map in the sync, and more

That’s it! Your event streaming sync will now appear in your list of events syncs, and events should start streaming into your destination. Visit the event destination’s website or app to check that events are flowing.

How it works

With event streaming, we send events to destinations in real-time fashion as opposed to grouping them into larger batches before syncing them to the destination on a schedule.

Like all other syncs, event streaming syncs take data from an event source and sync them to a destination. You can configure syncs to stream only a filtered set of events to your destination, control which event properties to sync, and define other sync behavior such as object creation behavior. These settings live within each of the event types you configure to stream within the sync. We’ll go over these below.

Streamed events go through contract validation before getting filtered, mapped, and then synced to the destination. With event functions, you can also transform the event after validation but before filtering and mapping.

Event types

You can configure different types of events from a source to stream to the destination. Most destinations have a few standard types like a Track Purchase and one custom event type to catch all non-standard events.

⚠️ We recommend making sure to configure the custom events type so all your events get streamed to the destination.

Event filters

Event filters allow you to filter which events get streamed as a specific type of event. For example, you would want to filter for only purchase-related events when configuring the Track Purchase event type. You can filter using any of the properties in the event payload. Check the event source’s debugger tab to inspect available properties.

Field mappings

Field mappings are used to match events to an object in the destination and event properties to fields in the destination. These include required fields for standard events accepted by the destination and also custom fields you may want to sync.

For the custom events type, we recommend passing the entire properties JSON blob in as custom event fields for syncing. Otherwise, it can be unwieldy to have separate configurations for every event you’re sending downstream. There are two ways to do this depending on the event destination:

Option A (new)

Set the properties column as the source field to map to the destination Attributes field.

Option B (legacy)

Select the “As an object in a single column” option when asked where event properties are defined and provide the properties column containing the JSON blob of event properties.

Referencing fields

In event filters and field mappings, you can reference fields on the event payload to filter on and match to fields in the destination object. Here are a few tips to keep in mind when doing so:

You can reference fields using dot notation, e.g. properties.price or event.
You can apply liquid templating language to cast types, e.g. {{ row['properties'].price | cast : 'number'}}.
You should escape periods/dots . and backslashes \ with a backslash \. For example, to refer to the innermost field of { "foo . bar": { "baz": 1 } }, you would use the field reference foo \. bar.baz.
You don’t need to escape spaces.
The event property refers to the event’s name for track events
The type property refers to the SDK and API event type which are track, identify, page, and group.

Errors and retries

If event syncs fail for whatever reason, we retry those events for a period of time and multiple times before archiving the event. You can find events that failed in the Errors tab, including ones that got archived.