The value of Big Data
Over the last few months, it seems that there is some re-thinking, with an emphasis away from the hype and more towards the value of Big Data. The whole cycle started with large silicon valley tech companies such as Yahoo!, Google and Facebook creating and deploying Big Data applications to great success. However, as large enterprises started down the path, it was clear that the technologies were not yet mature – especially in the realm of security. The early enterprise adopters struggled, taking well over a year for their first projects.
Things have improved in the last couple of years with robust integrations in the stack and bundled distributions. The next wave of enterprise IT adoptions found it easier to build their big data warehouses – yet many still remain as development projects. The most successful projects have been where Hadoop has been used as a data repository and/or for backup purposes. Many early adopters did this just because it was the “next cool thing” they could impresses their bosses with, or worse to pad their resumes! But where were the promised “new” insights? What is the true value of Big Data?
It is no wonder than that many analysts and other industry veterans have started rolling the drums claiming the end of the Big Data hype.
What is Big Data anyway?
Consider this: Most enterprises have barely even scratched the surface of this technology revolution. Some think that these technologies are irrelevant, others worry that if they don’t jump on the bandwagon, they will be left behind. But in many cases, the basic question that customers continue to ask is this: What can I do with this technology? What is the value of Big Data to my organization?
This is where we in the Big Data technology community must step up our efforts. We need to stop talking about the Yahoos and the Googles and instead realize that most businesses are nothing like tech companies, they don’t need 1000 node clusters or have petabytes of data. How can they still benefit from this revolution that lets them integrate multiple structured and unstructured data sources? What is the easiest way for them to genuinely gain insights from their data?
The industry should be focusing on answering these questions, rather than trying to invent yet another platform or API to shave off a few milliseconds on a query. Do we really need yet another NoSQL database?
Off my soap box. Thanks for reading.