Search documentation...

K
ChangelogBook a demoSign up

Datadog

Overview

Hightouch integrates with Datadog to help you gain greater observability into your syncs and monitor their health. Once you've authorized access to Datadog in Hightouch, we send real-time sync metrics and events to your Datadog account. You can then aggregate these statistics to gain insight into the health of your syncs.

Setup

You need your Datadog site and API key to set up the extension. You may want to create a new API key for Hightouch. Make sure that the API key creator in Datadog has the necessary write permissions for custom metrics (Metric Tags Write, Host Tags Write, Metrics Metadata Write) and events (Event Config Write).

Make sure to use the API Key itself and not the Key ID. Also ensure you use an API Key and not an Application Key to avoid the Datadog configuration is invalid error. You can find more information about the difference between these keys in Datadog's documentation.

  1. Go to the Monitoring section of the Extensions page and select Configure extension.
  2. Select your Datadog site and enter your API Key.
  3. Click Save.

Once you've saved your Datadog credentials in Hightouch, your syncs automatically start emitting metrics and events to Datadog.

Hightouch metrics and events don't immediately show up in Datadog, but only once sync runs are triggered.

Emitted metrics and events

Hightouch emits both metrics and events. Metrics are best suited for dashboards and statistical aggregations, while events are best for filtering. Don't hesitate to if there's another metric or event you're interested in using.

Metrics

hightouch.sync.row_processed

This metric is a COUNT type metric and is incremented for each row processed in the sync.

It provides the following tags:

  • sync - The sync slug in Hightouch.
  • workspace - The slug of the Hightouch workspace that contains the sync.
  • op (added/changed/removed) - The operation performed: whether the row was added, changed, or removed in the query.
  • success (true/false) - Whether the row was successfully synced to the destination.

hightouch.sync.complete

This metric is a COUNT type metric and is incremented each time the sync completes.

It supports the following tags:

  • sync - The sync slug in Hightouch.
  • workspace - The slug of the Hightouch workspace that contains the sync.
  • status (success/failed/warning/cancelled) - The overall status of the sync run.

hightouch.sync.query_time

This metric tracks your syncs' querying times.

It supports the following tags:

  • sync - The sync slug in Hightouch.
  • workspace - The slug of the Hightouch workspace that contains the sync.

hightouch.sync.execution_time

This metric tracks your syncs' execution times.

It supports the following tags:

  • sync - The sync slug in Hightouch.
  • workspace - The slug of the Hightouch workspace that contains the sync.

hightouch.sync.total_time

This metric tracks your syncs' total run times.

It supports the following tags:

  • sync - The sync slug in Hightouch.
  • workspace - The slug of the Hightouch workspace that contains the sync.

Events

hightouch.sync.complete

Hightouch emits this event each time a sync completes. It's identical to the metric of the same name.

It supports the following tags:

  • sync - The sync slug in Hightouch.
  • workspace - The slug of the Hightouch workspace that contains the sync.
  • status (success/failed/warning/canceled) - The overall status of the sync run.

Use cases

With just a few metrics, you can build complex dashboards and monitors. For example, you can create dashboards that target specific syncs that are critical to your operations and report the total number of operations or top syncs that have warnings or failures.

Example Datadog dashboard

You can import the JSON for a basic dashboard using this sample code to get started.

Completed sync status

One common use case is to build a basic dashboard that tracks completed syncs by status over time.

Datadog sync dashboard

The preceding chart uses the hightouch.sync.complete metric and sums the counts by status. You can filter further by including only statuses that are status:success or NOT status:success.

Anomaly detection

Anomaly detection can help flag significant changes, like many users suddenly being deleted or created. You can use the Datadog extension for anomaly detection in added, changed, or removed rows.

Anomaly detection setup

  1. In Datadog, create a metric monitor with the Anomaly Detection method.

Datadog metric monitor configuration

  1. Define the metric by selecting the hightouch.sync.row_processed metric. Specify the op_type, for example, removed, and sync name. Select the metric aggregation as sum by and enter sync.

Datadog metric monitor configuration with sum by sync

  1. In Alert Conditions, set the Alert threshold and Warning threshold to values that make sense for your use case.

Datadog metric monitor configuration

  1. (Optional) You can inject the sync name into the monitor's subject or message body using {{sync.name}}.

Datadog metric monitor configuration with sum by sync

Other use cases

You're not limited to the preceding examples. Here are some other use cases to consider:

  • Metric monitors for when the number of failed rows exceeds a threshold
  • Metric monitors for when a critical sync hasn't run over a specified time period

Ready to get started?

Jump right in or a book a demo. Your first destination is always free.

Book a demoSign upBook a demo

Need help?

Our team is relentlessly focused on your success. Don't hesitate to reach out!

Feature requests?

We'd love to hear your suggestions for integrations and other features.

Last updated: Aug 28, 2023

On this page

OverviewSetupEmitted metrics and eventsMetricsEventsUse casesCompleted sync statusAnomaly detectionOther use cases

Was this page helpful?