To understand the current and future state of big data, we spoke to 31 IT executives from 28 organizations. We asked them, "What’s the future of big data ingestion, management, and analysis from your perspective – where do the greatest opportunities lie?" Here’s what they told us:
- We’ll see the transition from on-prem to the cloud and subsequently see traditional Hadoop make the transition to the cloud. This will lead to higher adoption of AI/ML.
- Just drive the digitization agenda of the company. You have sufficient compute power and data – what could you do? Take advantage of the capability. Use AI/ML to filter through the data. Enable more people to get involved.
- AI protected systems. Maintain and keep the data safer. Create ethical and moral dilemmas for humans. Protect the data because at some point it will be turned over to machines which is terrifying because you don’t know what the machine may do with it and you cannot recover.
- There is a continued drive for standardization of data ingestion, with many companies looking to Kafka for on-premises or private cloud or Kinesis for AWS cloud. Data management and analytic tools then become sinks and sources for data for those data movement frameworks, which creates a sort of data utility grid for these companies, sort of like the electrical system in a house. If you need electricity, you just need an appliance with a standard plug and you plug in. The same is occurring with data access — and is already in place at some companies — if you need to get use of data or provide data to someone else, you just plug your application into the data grid using their standard “plug” (or interface). This will also allow for more “best of breed” components to be used, like the best BI or analytics tool or best database for a particular workload rather than having to compromise on an inferior all-in-one product since the data integration will be more standard than custom. Localization of data is a great opportunity, too. That is, having data located in the world where it is needed rather than needed to traverse long networks in order to retrieve it, process it, change it, or analyze it. That means more master-master, active-active architectures which can create application challenges for any enterprise, so the right choice of components will be important.
- Data analysts and scientists don’t care where the data is, they just want the data and the tools they need to analyze it. Catalog and know where the data is. Next step just want data where I want it. Build a virtual catalog to access delivery. There’s a logical progression of what we’re doing.
- The future is big data analytical platforms that provide proven capabilities for ingestion, management, and analysis at the speed and scale that enables businesses to compete and win. The greatest opportunities are for businesses to no longer be constrained by the imagination of the business in getting accurate insight so that they can act on all opportunities – understand exactly which customers are likely to churn and grow your business, establish entirely new business models based on the revenue-generating capabilities of data (think pay-as-you-drive insurance, as an example). Every single industry can differentiate itself based on the valuable insight of the data. Make an investment in a proven data analytical platform that offers no compromises and prepares you for whatever the future holds in terms of deployment models (clouds, on-premises, on Hadoop, etc.) or seamless integration with emerging technologies in the data ecosystem.
- We see three mission-critical opportunities in the future of data-driven marketing and sales. 1) Cord-Cutters — Our clients’ customers are more mobile and digital than ever. Traditional data elements and IDs such as home phone, home address, business extension, etc. have to be complemented with digital IDs such as mobile phone number, GPS coordinates, cookie ID, device ID, MAIDs, etc. 2) Predictive World — Artificial intelligence is woven throughout our everyday lives and experiences. Our phones predict the next few words in the sentence we are texting. Our thermostats predict what temperature is optimal for personal warmth and cost savings. Our cars brake for us before an accident happens. Consumers now expect marketing and sales experiences will also be predictive, using data and intelligence to improve their brand experiences in real-time. 3) B2B2C Life — There is a blending of our business and consumer selves. Research shows that approximately 43% of consumer work remotely and the number of people that spend > 50% of their time working at home has grown 115% over the past 10 years. Therefore, marketers must be able to connect the data IDs, attributes and behaviors of individuals versus siloed B2B or B2C targeting.
Here’s who we spoke to: