Hightouch lets you pull data stored in your Amazon Redshift data warehouse and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to Redshift for information like your host, port, database name, and credentials.
You need to allowlist Hightouch's IP addresses to let our systems contact your warehouse. Reference our networking docs to determine which IPs you need to allowlist.
To get started, go to the Sources overview page and click the Add source button. Select Amazon Redshift and follow the steps below.
Choose connection type
Hightouch can connect directly to Redshift over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your Redshift instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's tunneling documentation.
Configure your source
Enter the following required fields into Hightouch:
- Host: The hostname or IP address of your Redshift cluster. The hostname can be found by visiting the Redshift web console, navigating to the Clusters panel, and clicking your cluster. Copy the Endpoint string, excluding the port and database name.
- Port: The port number of your Redshift cluster. The default is 5439, but yours may be different. To confirm, visit the Redshift web console, navigate to the Clusters panel, and click your cluster. The port number is shown in the Properties tab.
- Database: The name of the database in your Redshift cluster. Most clusters have only one database. Visit the Redshift web console, navigate to the Clusters panel, and click your cluster. The database name is shown in the Properties tab.
Choose your sync engine
For optimal performance, Hightouch tracks incremental changes in your data model—such as added, changed, or removed rows—and only syncs those records. You can choose between two different sync engines for this work.
The standard engine requires read-only access to Redshift. Hightouch executes a query in your database, reads all query results, and then determines incremental changes using Hightouch's infrastructure. This engine is easier to set up since it requires read—not write—access to Redshift.
The Lightning engine requires read and write access to Redshift. The engine stores previously synced data in a separate schema in Redshift managed by Hightouch. In other words, the engine uses Redshift to track incremental changes to your data rather than performing these calculations in Hightouch. Therefore, these computations are completed more quickly.
If you select the standard engine, you can switch to the Lightning engine later. Once you've configured the Lightning engine, you can't move back to the standard engine without recreating Redshift as a source.
To learn more, including migration steps and tips, check out the Lightning sync engine docs.
Standard versus Lightning engine comparison
|Standard sync engine||Lightning sync engine|
|Required permissions in Redshift||Read-only||Read and write|
|Location of change data capture||Hightouch infrastructure||Redshift schemas managed by Hightouch|
|Ease of setup||Simpler||More involved|
|Ability to switch||You can move to the Lightning engine at any time||You can't move to the standard engine once you've configured Lightning|
Lightning engine setup
To set up the Lightning engine, you need to grant Hightouch write access to Redshift. You can do so by running the following SQL snippet.
CREATE USER hightouch_user WITH PASSWORD '********'; CREATE SCHEMA IF NOT EXISTS hightouch_audit; CREATE SCHEMA IF NOT EXISTS hightouch_planner; GRANT CREATE, USAGE ON SCHEMA hightouch_audit TO hightouch_user; GRANT CREATE, USAGE ON SCHEMA hightouch_planner TO hightouch_user;
The snippet creates a dedicated Amazon Redshift user for Hightouch. It also provisions two schemas,
hightouch_audit, for storing logs of previously synced data.
Enter the following fields into Hightouch:
- User: This can be your personal Redshift login or a dedicated user for Hightouch. At minimum, this user must have read access to the data you wish to sync. If using the Lightning sync engine, you must also grant this user additional permissions as described above.
- (Optional) Password: The password for the user specified above.
Test your connection
When setting up a source for the first time, Hightouch validates the following:
- Network connectivity
- Redshift credentials
- Permission to list schemas and tables
- Permission to write to
- Permission to write to
All configurations must pass the first three, while those with the Lightning engine must pass all of them.
Some sources may initially fail connection tests due to timeouts. Once a connection is established, subsequent API requests should happen more quickly, so it's best to retry tests if they first fail. You can do this by clicking Test again.
If you've retried the tests and verified your credentials are correct but the tests are still failing, don't hesitate to .
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up models to define which data you want to pull from Amazon Redshift.
The Amazon Redshift source supports these modeling methods:
- writing a query in the SQL editor
- using the visual table selector
- leveraging existing dbt models
- leveraging existing Looker Looks
- leveraging existing Sigma workbooks
You may also want to consider storing sync logs in Redshift. Like using the Lightning sync engine versus the standard one, this feature lets you use Redshift instead of Hightouch infrastructure. Rather than performance gains, it makes your sync log data available for more complex analysis. Refer to the warehouse sync logs docs to learn more.