Sequences allow you to execute a series of related syncs in a predefined order. Sequences are particularly useful when you have dependencies between syncs. For example, you may have two syncs:
- Sync A creates a set of new contacts in your CRM
- Sync B adds the new contacts to their associated accounts
Without sequences, you would have to schedule the two syncs individually to give the first enough time to finish before the second starts. It would be difficult to guarantee correct behavior, since sometimes syncs take longer than expected for various reasons. Another potential workaround could involve using an external orchestrator such as Airflow or Dagster to trigger syncs through the Hightouch API.
Sequences make orchestration a first-class feature in Hightouch so you can control how and when data is activated to your destinations. You can schedule sequences to run on a variety of schedules (such as triggered by dbt Cloud runs or cron schedules), just as you would with syncs, and sequences can also be initiated manually as needed.
Here are four syncs with dependencies. The table of staff members needs to be updated first, followed by staff permissions. Separately, a table of customer prospects needs to be updated. The final sync updates staff member assignments to customer prospects.
To represent this sequence of syncs, below we see a sequence that executes these four syncs in order:
- The first sync is configured to terminate the sequence if it fatally errors or has too many rejected rows
- The second and third syncs only terminate the sequence if they fatally error
- The fourth sync does not check for errors, so the sequence will complete successfully even if this sync fails.
To create a new sequence, select “Sequences” on the sidebar, then click the “Add sequence” button on the sequences page.
Select the “Add syncs” button to choose syncs for the new sequence. You can include as many syncs as you want, and you can add more later as needed.
Once syncs have been added to the sequence, they can be re-ordered using the up / down arrows.
Each sync’s error handling behavior can be configured using the “Terminate” drop-down menu.
On the final page, you can give your sequence a name and define its execution schedule.
Once you save your new sequence, it will be visible on the main Sequences page.
You can click on a sequence to go to its details page. Here you can see the sequence’s run history, disable or enable it, execute it manually, or update its configuration or schedule.
Clicking on one of the sequence runs will show its details in a pop-up, including the execution time and final state of each sync run belonging to the sequence run.