A Crawl, Walk, Run Approach to Data Integration

In today's multichannel world, there are few things more exciting (and daunting) than integrating visitor data across all of your customer touch points. While you might be tempted to pull everything in all at once so you’ll have it, ‘just in case’—be mindful that this is, in fact, a very counterproductive and flawed approach to data collection. That ‘just-in-case’ data will quickly turn into an obnoxious overload of data and cause major downfalls.
In attempting to pull in everything and nail it on the first try you will encounter 3 initial problems:
  1. Not All Data is Created Equal.
    As any analyst worth his weight in gold will tell you, there is a lot of data out there that just isn't all that interesting as far as analysis is concerned. As a result, it's really important to consider the potential value of pulling in particular data points and narrowing them accordingly.
  2. Data Comes At a High Cost.
    The cost associated with pulling in every spare scrap of data can be extremely high - not only do you have to pay to store/archive it, but you also have to consider the time that you and all of the teams working on this project are going to spend getting the data into the warehouse and then setting it up in such a way that you can actually make use of it. 
  3. I Want a Re-do.
    You are going to learn a lot during the time you spend pulling, housing and sorting the data.  After it’s all said and done, it is likely there will be things that you will want to change or go back and do differently. 
The reasons listed above are why we strongly advocate taking a phased approach to data integration, and the requirements that you define should absolutely reflect that.
So, what does a crawl, walk, run approach look like when it comes to integrating data? 
We've had the opportunity to do this with hundreds of clients over the last 13+ years, and each process looks different. When starting out, it's best to begin with a small subset of data and go through the entire process as a POC, thereby giving you the opportunity to work through this process and learn from doing it. I've taken the liberty of highlighting some of the key points that came out of each one of these integrations.
  1. Interview Your Stakeholders.
    Avoid starting by asking them what data they want in the data warehouse. Instead, talk to them about actual use cases for the data - what do you want to know about your users? What is most valuable to you in terms of your business and your initiatives? By starting at the highest level, you will gain the opportunity to start from the question and then determine which metrics are best suited to provide the answer.
  2. Get Your Business Users Talking.
    Perhaps your business users are having a tough time coming up with use cases. Here are a few examples of things that you can use to get them talking:
    Product performance - who interacts with which products? Who puts what type of products in their cart and what cross-sold and up-sold products are they interacting with? You can personalize the user experience by seeing what they like, and recommending similar products and ancillary components that might appeal to them.
    Content engagement - who is viewing what types of content? Who downloads particular articles? Who shares what content? Pay attention to VOC (read more about it HERE). You can use this information for personalization purposes and can recommend articles and tailor your marketing messages based on the content that garners the most interaction. 
    Marketing channel effectiveness - what types of channels, messages, and frequencies do your users respond to, either by clicking through to the site or by converting? By evaluating this, you can better communicate with particular users and user groups.
    Key event contribution - what do people interact with before completing a goal or key event on your site? Are they looking at particular products or content, or interacting with key features of the site like videos, product tours, or other information? Evaluating these contributors can help to provide insight into what drives users to convert and can also help to inform testing outside of your conversion funnel.
  3. Assess Segments.
    Most importantly, when talking to your stakeholders, remember to assess which segments are most important to them! All of this information is not particularly useful if you have to look at it on an individual level, so rather than attempting to do that, you are best served to figure out which constituencies are most valuable to your business users. It is also very useful to consider what some of the expected behavior might be for these segments for later analysis - what do you expect particular user groups to do and are they doing it?
  4. Define Metrics.
    Once you've got your business cases outlined, take some time to determine which metrics will be most useful in terms of analyzing that data. Remember not to put in everything but the kitchen sink - you'll want to be very judicious in choosing what goes into your data warehouse and what stays put. For example, when evaluating content trends, you shouldn’t aim to bring in every single page and page view that is associated with a user. Not only would that be data overload, but there are much more elegant ways to get that information. Instead, if trying to assess which types of articles someone views on your site, you'd be better off to pull in content groups or content tags, if you have them available. 

    You may need to update your implementation in order to start actively tracking this information, in that case, you can assess the level of effort required to do that and determine if that requirement can stay in this phase. If it's simply too complex or time consuming, roll it into phase two. Also, don’t be afraid to embark on new areas of metrics, such as mobile analytics (read more HERE).

  5. Start Analyzing!
    Once you've finally got all of your data, start analyzing! You should have a solid foundation to get started if you've clearly defined your business use cases, segments, and hypotheses. If you don't quite know where to start, I always like to make a plan by assessing impact and level of effort. 

    *One final tip - as you're analyzing, when you find questions that you can't answer with the current data set, make a note of it and add them to the list for the next phase.  With the right scalable and flexible approach, you’ll be able to add these data points in incrementally without much effort.

In the quest to visitor data integration, the importance of a phased approach is abundantly clear. With our extensive client examples, we’ve learned a thing or two when it comes to staying on a crawl, walk, run track to keeping data integration efficient, valuable, measureable, and deliberate. Look to Stratigent to help you make sense of your data; we can help simplify the complex and ensure that you’re leveraging your existing datasets to the best of their ability. 
Questions? Comments?
Reach out to info@stratigent.com or comment below.
By Erin Cropper
About the Author:

Erin Cropper is the Director of Client Services at Stratigent.

Contact Us Now