Planning…. It’s a word that is used often in marketing and quite often with CRM Integrations. It’s so important to start with a strong plan when thinking through your integration. You want to ensure you’ve given proper brainpower to your use cases and data architecture requirements. You’ll also want to develop a strong test plan to ensure data is flowing and updating properly. When it comes to integrations, there are a few common best practice guidelines you should follow.
Plan, plan, plan
Not everyone is a project manager, but it’s really important to plan your integration flow and dependencies. Here are some things to think about.
- What are your use cases?
- Is this a one way or bi-directional integration?
- What are the data types being transferred?
- Are picklists being utilized?
- How will you filter the data?
- How does your destination source handle duplicate records?
- Do you have data from multiple sources that needs to relate to a single source in the destination system?
- Do you have a testing plan?
What are your uses cases?
Why do you want to Integrate your data? How will you use the data? Creating use cases for how you will use the data once it gets to its destination will help determine what data you need in your destination system.
- Who will use the data?
- How will you use the data?
- Will you use the data to make decisions in Campaigns?
- Will you use the data in Reporting?
- What data fields will you need for Data Segmentation?
- Document what data you need to accomplish a particular task.
- Will you export data?
Is this a one way or bi-directional integration?
You may have one or more source systems integrating to a single destination or you may have one source and one destination. Either way you need to decide if this is a one way or bi-directional flow.
- One direction flow
- Develop use cases for how you will use the data.
- Keep your flow limited to only data needed in your destination. No need to clutter with unnecessary data.
- How will you match to existing records?
- Will you create new records if they do not already exist?
- Bi-directional flow
- Develop use cases for how you will use the data in each system.
- Think about what data should be priority in each system.
- Document what data should flow from system 1 to system 2 and what data should go from system 2 to system 1.
- Which system will be used as the “Source of Truth” for each piece of data?
- When configuring a bi-directional integration you will need to think about which systems data is priority. For instance, each system may hold Email Opt out data. You need to determine which system will be the most updated. If system 1 is a CRM platform and system 2 is an Email Marketing platform, and all opt out data is captured in system2, then you would only want system 2 updating system 1.
- Use advanced features to ensure good data is kept and poor data doesn’t overwrite priority data.
- What will you use as your Unique identifier is each system?
- Will you create new records if they do not already exist?
What are the data types being transferred? Make sure the source matches the destination data type.
When you are transferring data, you will need to make sure you are transferring to a compatible field. This is where your use cases will come in handy. You need to think about how you will use the data in these fields.
- Will you need to use these fields in a query or filter?
- What filter options are available for each data type?
- If using a Check box, make sure to check the value of checked and not checked
- Do the systems accept multi-select lists?
- Can they be written to a large text file?
Are picklists being utilized?
Using picklists can help to keep your data clean. In some cases, your destination system will accept values not in the picklist but will not display the values in the user interface. In other cases, records or files will be rejected if there is not a match.
- Using picklists are helpful and recommended when using the field as criteria in filters.
- Make sure the database values and the user interface values in the picklist are acceptable to the destination system.
How will you filter the data?
You may not want all the data from the source flowing to the destination. Make sure to identify filter criteria so you are only sending useful data.
- Will you use incremental data? This is only the data that is new or has been modified since the last update.
- Do you only want data created by a certain user?
- Make sure you plan for the first import; it may be large depending on your filters.
How does your destination source handle duplicate records?
Many Systems contain duplicate records. Sometimes this is by design and sometimes it is due to dirty data. Either way, you need to understand how the systems handle duplicates.
- Make sure you understand how your destination handles duplicate records.
- Platforms such as Oracle Eloqua, by default, use Email Address as the unique identifier and may merge duplicate records in an ASCII hierarchical fashion. This will merge duplicate records with the same email address and replace data based on hierarchy rules.
- Best rule is to keep your data clean if possible.
- If duplicates are intentional make sure you have a plan to route duplicate data respectively into Custom Tables if available.
Do you have data from multiple sources that need to relate to a single source in the destination system?
In some cases, you may have data coming from several sources that relate to one record in your destination system and you need a way to link them.
- Do you have a unique identifier on your data?
- Think about what key data field you can use amongst your multiple sources to tie them to your destination record.
- Consider creating a Global Unique identifier if one does not exist
Do you have a testing plan?
As with any project, the testing plan can be the most pivotal piece. If you have access to a sandbox environment, this is a safe way to test. In some cases, integrating with a sandbox will be required by your marketing or IT team. For scoping reasons, be sure to estimate time for developing in a sandbox first if you are not able to push your sandbox environment to production.
- Your Use Cases should include a Mapping Document
- Create test users for each use case you have developed
- Document Source data and confirm destination results
- Document any changes in the Integration plan, use cases and configurations as a result of testing or any other reason.
Building out an Integration of any size should first and foremost start with a plan. A plan that includes use cases, documentation and journey mapping if possible. Take the time to get this right and make sure you have a good test plan. Need assistance? Please, reach out!