My morning routine as a product manager at Hightouch begins with a lot of scrolling. You see, our integrations backlog is really long. Sometimes, it feels like it could stretch to the moon and back.
How are we going to build all of these destinations? Can it be done? The data world is messy: it’s rife with niche SaaS and internal tools for which we can’t offer pre-built connectors.
Dealing with the “long tail” of integrations is one of the most difficult challenges we’re facing on our journey to make data actionable for everyone. Does it stress me out? Of course. But it also excites me and reaffirms what I’ve always known: that the use cases we enable are endless. Our customers are already using Hightouch to accomplish so much. They’re building data-driven growth engines, personalizing in-app experiences, and waving goodbye to traditional CDPs. Data activation is the most powerful—and empowering—piece of the modern data stack.
Hightouch supports 100+ native destinations, more than any other reverse ETL platform. Still, we’re not even close to scratching the surface of what’s possible. Every day, I’m reminded that the “long tail” isn’t ignorable; it’s the epicenter of interesting and impactful data work. To unleash the full potential of the Hightouch platform, we need to make it easy and fast for anyone to create their own custom destinations.
It’s my privilege to share what we’ve built to make this possible.
🚀 Introducing the custom destination toolkit
Today, we’re releasing a suite of developer-focused destinations that make connecting Hightouch to internal tools and third-party APIs effortless. If you can dream it, you can build it… in minutes. And often without writing a single line of code.
Our custom destination toolkit offers first-class support for:
- Making HTTP requests using our visual request builder
- Invoking serverless functions via AWS Lambda, Google Cloud Functions, and Azure Functions
- Publishing messages into Apache Kafka, Google Cloud Pub/Sub, RabbitMQ, Amazon SQS, Amazon SNS, Amazon EventBridge, and Amazon Kinesis
- Writing to transactional databases like PostgreSQL, MySQL, SQL Server, MongoDB, Firestore, Rockset, CockroachDB, and more
- Creating custom UIs using our embedded destination framework
Our goal is simple yet ambitious: give customers the tools they need to build their own integrations without sacrificing reliability, observability, or security. Data teams should not feel limited by their vendors’ roadmaps and priorities. Do you need to sync data into legacy APIs or niche, industry-specific software? Just pick the right tool for the job and get it done.
HTTP requests
Our custom destination toolkit exposes the building blocks of software-to-software communication, so where else could we begin but with the venerable HTTP request?
Most data activation tools offer some basic HTTP functionality, usually as a simple “webhook” destination. We quickly learned with our customers that these webhook connectors—ours included—lacked the flexibility required to interface with many popular APIs. Rate limits, retries, dynamic URLs? Never heard of ‘em. Support for nested objects and arrays was also poor or nonexistent. In fact, the most common use case for our webhook destination involved making requests to internal services that would modify the requests before proxying them to third-party services. 🥴
One customer gave it to me straight: “Right now, I need to redesign my API to accept Hightouch webhooks. I want it to work the other way around. Why can’t I configure my Hightouch webhooks to interface with existing APIs?”
That question presented an irresistible challenge for our engineering team, leading to today’s announcement.
Our all-new HTTP Request destination is—by far—the most powerful and flexible HTTP connector on the market. Using our no-code visual interface, you can exercise granular control over:
- Request triggers, like rows added, changed, and/or deleted
- Methods, such as POST, PUT, PATCH, and DELETE
- Dynamic endpoints, including templated URLs with variable injection
- Dynamic payloads, supporting JSON objects, XML documents, and URL encoded forms
- Rate limits, up to 10,000 requests per second
- Concurrency limits, up to 1,000 requests in parallel
- Timeout thresholds, up to 60 seconds per request
- Custom error handling logic, including retries across multiple sync runs
- And much more…
You can compose nested JSON objects and arrays, or create XML documents using our new Liquid-based templating engine. Our payload builder supports variable injection, control flow, and loops. Gone are the old days of building complex payloads in SQL.
Check out the docs to see it in action!
Simplified mock-up of the HTTP Request destination
Serverless functions
If you’re still itching to write code, I’m delighted to share that Hightouch now integrates directly with AWS Lambda, Google Cloud Functions, and Azure Functions.
These destinations make it possible to build your own custom connector by invoking a serverless function whenever rows are added, changed, or removed in your data model. Here’s why you might want to do that:
- Sometimes, it’s easier to transform and filter data using code instead of SQL. With serverless functions, you can use your favorite language to get the job done fast.
- Client libraries help you interact with complex APIs. (No need to read—and reread—confusing specs to understand HTTP endpoints.)
- Your functions can also enrich warehouse data using external sources like Clearbit and ZoomInfo.
Serverless functions offer the ideal environment to execute custom code during a sync. Hightouch now integrates natively with the three major cloud providers: AWS, Google Cloud, and Azure. All of our serverless function destinations offer first-class support for custom triggers, batching rules, rate limits, concurrency limits, and error handling. What more could you ask for?
Simplified mock-up of a serverless function destination
Message queues
To support high-throughput, distributed, and asynchronous workloads, Hightouch can now publish messages into Apache Kafka, Google Cloud Pub/Sub, RabbitMQ, Amazon SQS, Amazon SNS, Amazon EventBridge, and Amazon Kinesis.
Message queues might be “dumb pipes,” but our integrations are anything but basic. With the Kafka destination, for example, you can connect to multiple brokers, authenticate with SASL, bring your own certificate authority, publish into different topics for each message trigger, and define custom ordering and partition keys. Hightouch supports all managed Kafka services (Amazon MSK, Confluent Cloud, etc.) and can also connect to self-hosted instances. That covers only one of the seven message queue destinations we're releasing today. 🤯
Simplified mock-up of a message queue destination
Transactional databases
Hightouch can also write into transactional databases like PostgreSQL, MySQL, SQL Server, MongoDB, Firestore, Rockset, CockroachDB, and many others. Today, we’re shipping a bunch of optimizations to make these syncs up to 20 times faster, so you can power internal tools and in-app experiences using the freshest data from your warehouse.
Internally at Hightouch, we built our customer-facing billing portal by syncing product usage data from Snowflake into Postgres. If you’re curious about other use cases, we’ve written extensively about combining the analytical power of the data warehouse with the low-latency performance of transactional databases.
Simplified mock-up of a transactional database destination
Customizing the user experience
Our embedded destination framework helps you build a custom connector that looks and feels just like a native destination. It offers programmatic control over the entire sync configuration UI/UX, including questions, answers, and validation rules.
Embedded destinations are meant for data teams tasked with building connectors for less technical users. By customizing the form’s appearance, developers can hide technical complexity and present a point-and-click interface to end users in marketing, sales, and other departments.
Is a custom UI overkill for most use cases? Most definitely. But if you need this level of control, it’s really nice to have these advanced features in your back pocket.
Michelangelo the sculptor used his iron chisel. Picasso the painter used his horsehair brush. And you, the data activator, can use Hightouch’s embedded destination framework to create your visual masterpiece.
Read the docs to learn how it works.
Simplified mock-up of a form created using the embedded destination framework
What’s next?
Starting today, our custom destination toolkit is available for all Hightouch workspaces. I’m excited to see what you build! If you’re new to Hightouch, you can request a demo or create a free account.
Want to chat about your use case? Need help creating your first custom destination? Please don’t hesitate to reach out: omeed@hightouch.com (or ask for me on Intercom!)
