Search documentation...

K
ChangelogBook a demoSign up

Materialize

Materialize is a streaming database powered by Timely and Differential Dataflow, purpose-built for low-latency applications. It lets you ask complex questions about your data using SQL, and incrementally maintains the results of these SQL queries as the underlying data changes.

View Materialize's documentation.

Overview

Hightouch lets you pull data stored in your Materialize database and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to your Materialize instance for information like your host, port, database name, cluster, and credentials.

You may need to allowlist Hightouch's IP addresses to let our systems connect to your warehouse. Reference our networking docs to determine which IP addresses you need to allowlist.

Connection configuration

To get started, go to the Sources overview page and click the Add source button. Select Materialize and follow the steps below.

Configure your source

Enter the following required fields into Hightouch:

  • Host: The hostname or IP address of your Materialize server.
  • Port: The port number of your Materialize server. The default port number is 6875, but yours may be different.
  • Database: This specifies the database to use when Hightouch executes queries in Materialize.

Provide credentials

Enter the following fields into Hightouch:

  • User: This can be your personal Materialize login or a dedicated user for Hightouch. At minimum, this user must have read access to the data you wish to sync.
  • Password: The password for the user specified above.

Test your connection

When setting up a source for the first time, Hightouch validates the following:

  • Network connectivity
  • Materialize credentials
  • Permission to list schemas and tables
  • Permission to write to hightouch_planner schema
  • Permission to write to hightouch_audit schema

All configurations must pass the first three, while those with the Lightning engine must pass all of them.

Some sources may initially fail connection tests due to timeouts. Once a connection is established, subsequent API requests should happen more quickly, so it's best to retry tests if they first fail. You can do this by clicking Test again.

If you've retried the tests and verified your credentials are correct but the tests are still failing, don't hesitate to .

Next steps

Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up models to define which data you want to pull from Materialize.

The Materialize source supports these modeling methods:

You may also want to consider storing sync logs in Materialize. Like using the Lightning sync engine versus the standard one, this feature lets you use Materialize instead of Hightouch infrastructure. Rather than performance gains, it makes your sync log data available for more complex analysis. Refer to the warehouse sync logs docs to learn more.

You must enable the Lightning sync engine to store sync logs in your warehouse.

Tips and troubleshooting

If you encounter an error or question not listed below and need assistance, don't hesitate to . We're here to help.

Canceling statement due to statement timeout

This error occurs when the required execution time of your Materialize query exceeds the timeout limit for the database. To address the error, increase the timeout by executing the following query:

set statement_timeout = '300 s'; -- 300 seconds, 5 minutes

Be sure to adjust the time to be as long as it takes for your query to execute.

Ready to get started?

Jump right in or a book a demo. Your first destination is always free.

Book a demoSign upBook a demo

Need help?

Our team is relentlessly focused on your success. Don't hesitate to reach out!

Feature requests?

We'd love to hear your suggestions for integrations and other features.

Last updated: Aug 14, 2023

On this page

OverviewConnection configurationConfigure your sourceProvide credentialsTest your connectionNext stepsTips and troubleshootingCanceling statement due to statement timeout

Was this page helpful?