Whether you call it multi-channel, omni-channel, data lake, data ocean, data pond, data puddle, or some other ridiculous term it is safe to say that your big data project is probably not going as well as you'd like it to. According to Gartner, 85% of Fortune 500 organizations will be unable to gain a competitive advantage with big data in 2015.
Everyone understands the value of the data being collected, and we all understand how much data is being created daily about an individual. So, why aren't organizations able to get value from this data? Innovation is all around us and the technology sector is seemingly blowing up with new options to help organizations crunch the numbers. With that toolkit, you would think a higher percentage of organizations would derive value in 2015 from big data.
If we were to hop in a time machine and go back to 2008, we'd be in a world where pontificates argued on conference stages whether or not analytics was easy. I'm here today to tell you that big data isn't easy and it requires a whole, new way of thinking in order to ensure your success in the coming year. From our interactions with clients, to the roadblocks we've had to plow through while creating a big data architecture, we have come across three key reasons for big data failure:
Reason #1: Lack of Use Cases
Too often, data integration projects of any shape and size are immediately driven by the mentality of simply bringing all of the datasets together. This is where you end up boiling that data ocean of yours, and it never ends well. I'm consistently amazed by the "business" decision to build an overtly complex architecture without any consideration of the actual business use cases for that data and the overall vision for what business decisions should be positively impacted by the integration of the data.
Recommendation: Start with use cases and build an environment that can be flexible down the road if you decide to add in more data. Don't spend the time, money, and effort building the Titanic. Spoiler alert: it sinks.
Reason #2: Old World Thinking
Rack 'em and stack 'em, Johnny. Sure, that used to be the old way of doing things, but it isn't going to help you add value to the business long-term. There is a reason that the stalwarts of data storage are struggling for growth and it's because a real-time personalization architecture isn't going to come from a server farm in your data center. The cloud is your friend, not your enemy.
You simply can't let your "big data committee" be comprised solely of IT stakeholders. This isn't a data storage project; it's a data activation project. Activation will take many forms but that predominantly will involve real-time segmentation/personalization and econometric modeling.
Recommendation: Build a steering committee that is predominantly chaired by business stakeholders to build the vision and the use cases outlined above. From there, partner with IT to develop an architecture that solves for the exact needs of the business.
Reason #3: Shiny Object Syndrome
Ooh, look at that! If you follow the #bigdata trends on Twitter you know exactly what I'm referring to. Everyone raved about Hadoop and suddenly there were Hadoop conferences all over the world. This isn't going to stop anytime soon. Rapid expansion, followed by contraction in the form of consolidation and acquisition is the name of the game. It happens in every, single tech sector. If everyone jumped off a bridge, would you do it, too? (In my parent voice)
Recommendation: Don't follow the trends, follow your vision. Just because a technology is all the rage doesn't mean you wouldn't be better served using something like AWS + Redshift, SQL, or some other data platform. Map your requirements against the technology options you are considering and POC the "final" selections with data volumes that mirror a production environment to kick the tires and stress-test the system.
The goal behind the big data revolution is not one to be ignored, but it feels a lot like running in mud when you're just getting started. I've outlined some of the common reasons these projects end up failing, but there are a lot of other roadblocks to avoid along the way in addition to these. We'd be happy to sit down and provide you with the guidance you need to ensure you're able to better understand the customer journey and ultimately do something about it.
If you’d like to discuss this in greater detail, please leave a comment below or reach out to me directly. If you'd like to meet us, visit us at the IN Analytics Conference coming up this October in Chicago, attending is free but space is filling up so reserve your spot today!