Step 1: Preparing for migration

Organizational size and technical complexity can impact each of these steps, but during a migration process, you'll typically:

1. Form a migration team and communication plan: Gather a cross-functional team, including data and analytics engineers, analysts, and stakeholders that will carry out the migration and verify data quality. Ensure that you inform affected teams of upcoming changes and provide a channel for concerns or questions.
2. Determine the scope of your migration: Decide whether you're modifying your data model during the migration or keeping it the same, whether you're merging Hightouch Event data with historical data, and whether you want to replace existing event tables or create new ones.
3. Map downstream destinations: Document any data destinations, dashboards, reports, or other consumers of event data that may need to be updated or validated as part of the migration.
4. Establish a timeline: Set realistic deadlines for each migration phase based on available personnel.

Determining the scope of migration

Data model and tracking plan

Decide on your approach to migration. We think there are two main options:

1. Simple migration: Keep your existing data model and event structure as much as possible. This makes validation and updating downstream applications and destinations more straightforward, and may require little or no data wrangling to fit new event data into the existing data model.
2. Complex migration: Address issues or concerns in your data model during the migration process. Changes during a migration process could also include re-instrumenting events for a new tracking plan or integrating new tooling.

While these changes can have positive impacts on long-term data quality, each change increases the complexity of the migration and should be evaluated for return on the increased implementation time.

For most organizations, we strongly recommend a simple migration. Minimizing changes increases the likelihood of timely completion and cuts down on the roadblocks that can develop through substantial changes in the data model.

If data quality is a significant issue, a migration to Hightouch Events can be a good occasion to build for long-term data quality and usability.

Historical data handling

During planning, you should decide where you want the new event data to appear in your warehouse and whether you intend to union the historical and new data.

Regardless of your choice on unioning, we recommend writing the new Hightouch Event data into a separate schema in your warehouse. This makes it easier to isolate the events while you test and validate your migration, comparing them to historical data. We'll discuss where and how to unify historical and new data in Step 5 of the guide.

Mapping Sources and Downstream Destinations

During planning, identify:

• Sources: All the applications you collect events from that you want to migrate to Hightouch Events.
• Destinations: All the places that should receive or consume event data, including your warehouse, BI and analysis tools, and marketing and advertising platforms.

While most organizations send their events into their warehouse, then reverse ETL (rETL) data to other tools, Hightouch Events also supports streaming events directly to other destinations.

It's worthwhile to create a table or other structure to map each source to its destinations and downstream consumers along with the relevant transfer method (rETL or streaming).

Once your team has decided on any changes to the data model and mapped data pipelines from sources to destinations, you can plan for any updates that will be needed for each downstream consumer or tool.

Estimating time to migrate

Once you've broken down the migration according to key phases on the roadmap, you can estimate effort based on personnel, number of events, and number of sources and destinations.

For organizations with 10 to 30 event sources, we often see 4 to 6 calendar weeks for a migration. Organizations with 70 to 100 sources can migrate over 8 to 12 weeks.

1. Planning and preparation: 1 week effort for a data engineer and data analyst for research, planning, and documenting the plan, though this scales with changes to the data model and tracking plan.
2. SDK implementation: 1 week (or less) for an engineer to implement, test, and deploy new SDK code and initial data contracts.
3. Configuring destinations: 1 week (or less) for a data analyst to configure, test, and do initial monitoring.
4. Validating and unioning data: 1 week for a data analyst, though this scales with number and complexity of events and destinations and how you're unioning historical data.
5. Updating downstream applications and deprecating your prior event collection tool: 1 week or less for an engineer and a data analyst to update downstream data consumers, replace analytics code, re-validate data, and sunset your prior event collection tool.

Remember, these are rough estimates. Your actual timeline may vary based on the complexity of your current setup, the scope of changes you're implementing, and the number of dedicated personnel you have working on the migration.

By thoroughly preparing and planning your migration, you'll set a strong foundation for the technical implementation phases that follow. In the next section, we'll dive into the process of switching SDKs to start collecting data with Hightouch Events.