When building your team’s data pipelines, it can be difficult to have visibility into how your tables and data are used downstream. This lack of visibility can lead to unintentional breaking changes, wasted time as you try to maintain unused tables and frustration as you try to piece together which syncs are associated with certain tables. dbt recently launched a new feature called Exposures to give you a way to express the downstream utilization of your sources and models in data docs and lineage graphs.
We’re excited to announce that Hightouch now supports dbt Exposures natively in our app. With a simple toggle switch, you can automatically update your dbt repos with Exposures to see where models are being used in your team’s Hightouch syncs. This means better end-to-end visibility of your data lineages, giving you clearer context for how data moves through your pipelines so you can avoid breaking changes.
🎥 Check out this video for a quick demo of how it works.
dbt Exposures
dbt + Hightouch = ❤️
dbt continues to take the data world by storm, with no signs of stopping. This launch further deepens our integration with the popular data transformation tool, making it easier than ever for our customers to use the warehouse as their single source of truth for Data Activation. By enabling our dbt integration, you can:
- Pull existing dbt models via git and sync modeled data directly from the warehouse to 200+ business tools
- Trigger Hightouch syncs to execute after dbt Cloud jobs run
- Get complete context for how Hightouch syncs are related to dbt models
If you’re new to dbt (we’ve been there!), we recently wrote a primer on dbt with some simple tips on how you can jumpstart your knowledge.
“Analytics Engineers are responsible for building and delivering data assets that increasingly power business operations in addition to reporting. With the Hightouch dbt Exposures integration, dbt users can easily visualize their data lineage and understand how their dbt models and Hightouch syncs interact. We are pleased that Hightouch continues to build deep integrations into dbt, making it easier than ever for our joint customers to power workflows off their dbt modeled data.”
Nikhil Kothari
Head of Technology Partnerships at dbt Labs
Setting up dbt Exposures in Hightouch
To activate dbt Exposures in Hightouch, head to the Extensions tab and select the dbt-compatible source you want to set up. If you have not yet activated dbt model sync for your source, you should do so now. Enable the toggle for dbt Exposures Sync, which will create commits in your dbt repo to create the Hightouch exposures file (just make sure Hightouch has write access to your dbt branch).
Hightouch Extensions
Exposures Commit
Once you have enabled dbt Exposures, you should see the sync status indicator turn green and a commit created in your dbt repo. dbt Exposure syncs will run whenever there are changes to a Hightouch sync that use a dbt model.
dbt Exposures Sync
If Hightouch encounters an error updating your dbt Exposures, you can see the error details under the “activation toggle,” helping you debug faster.
Error Details
Now, when you rebuild your lineage graphs in dbt on your latest commit, you’ll be able to see exactly which Hightouch syncs are using your dbt models.
Anytime a Hightouch sync is created or updated that uses one of your dbt models, Hightouch will update the Exposures file in your dbt repo, showing you your updated Exposures when you next rebuild your dbt docs. You can also quickly visualize where you need to run your DAG up to to ensure your Hightouch syncs are using the freshest data.
Looking Ahead
In the next few months, we will continue to invest in our dbt integration to further surface the connection between your dbt models and your downstream data flows. We want to help you ship changes to dbt models with confidence by providing better validation and simulations of the impact radius on all dependent syncs.
We’re also working closely with our partners in data observability to better flag when changes upstream can impact Hightouch syncs. Check out our recent integration with Datafold to learn more.
Get Started
dbt Exposures for Hightouch is GA for all Hightouch customers. Simply navigate to the Extensions tab, click the dbt Models tile, and toggle the dbt Exposures button to “Enable.” Check out our Docs to learn more. We’ll also be at dbt Coalesce October 17-21 in New Orleans…hope to see you there!
