Preparedness – The Big Data way

By November 2, 2015 Blog 2 Comments

Preparedness has been a quality that is usually understated when it comes to describing personality of a person or corporation unlike innovation,  hard work, agility, perseverance, intelligence. However preparedness (or lack of it) has been an integral part of one of some major breakthroughs and colossal failures that mankind has witnessed over decades and centuries.

Hitler’s troops were ill-prepared to handle Russian forces and Russian people and they eventually retreated. Arab Spring impacted different Arab rulers and leaders differently. The Arab leaders who prepared themselves for embracing social media and in general media explosion prepared themselves survived the change. World history is full of it and so is corporate history. Nokia could have made smartphones and not only survived but maintained its leadership position. Kodak could have made digital cameras early on instead of living in denial mode calling digital cameras a fad. To quote a very recent instance, GM, BMW or for that matter BP, Shell or Chevron could have developed a fully electric car. They could have avoided the heat they are facing due to Tesla’s phenomenal success by themselves working on the less heat generating eco-friendly energy technology early on. There is a lowest common denominator in all these stories and that is “Preparedness”.


Now, when it comes to Big Data, significance of preparedness attains a whole new dimension and scale. Corporations are faced with the challenge of not just a new technology to be leveraged but a whole new platform to be embraced. There is data everywhere- years of data from transactional OLTP platforms, data from Social platforms, data from mobile and sensors & IoT sources. Data is now at the center of every enterprise’s IT strategy. The more they leverage it, the better edge they have over their competition. The key to overcoming all this is “Preparedness to embrace Big Data”. It involves getting started with the Big Data technology sooner than later. No angels are going to descend from heavens and announce the use cases to be taken up or infrastructure to be built to take advantage of Big Data technology for the organization. The more you wait, the harder it would be to fit a “data platform” like Hadoop into your overall IT infrastructure.

“Luck is a dividend of sweat. The more you sweat, the luckier you get.”

— Ray Kroc,  founder of McDonalds

“Preparedness” in the context of Big Data implies building a versatile Big Data platform and democratically opening and exposing it to all the business units and end users. Preparedness in this context also implies inculcating a culture of out-of-the-box thinking about the current pain points and challenges that the organization is facing. This will whet the desires of business users and make them talk to their IT counterparts to start addressing the current issues that they face with existing IT infrastructure. Here are few facts worth noticing about Big Data:

  • The cost of periodically purging data has surpassed the cost of maintaining data. Add to that the cost of missed opportunities and innovation.
  • The cost of maintaining data center is higher by several orders of magnitude than cost of running a system on a pay-per-use cloud.
  • The licensing cost of enterprise softwares is way out of whack if you compare the same with corresponding Big Data stack. Moreover traditional enterprise tools are quite silo-ed. These tools need at least one engineer for each enterprise tool at a minimum with barely any fungibility between the resources making devops really difficult.
  • The cost of commodity hardware is much lower than high end servers and only heading southwards with every passing year.

The cost of not recognizing the impact of SMAC (Social, Mobile, Analytics and Cloud) is not easily quantifiable. However not doing anything about it can result in irreparable losses due to missed opportunities. With open source taking the center stage and making tinkering with data-driven technologies extremely affordable, it is imperative that enterprises get started with it at the earliest.

“The more we sweat in peace, the less we bleed in war.”

—  Vijaya Lakshmi Pandit

It is about time enterprises get started with a Big Data platform to manage their data. It should all start with identifying some pilot use cases that have been thought of as cost prohibitive and thus have been traditionally kept out of scope of current budget again and again.

  • It could be a clickstream analysis.
  • It could be leveraging new age technologies like machine learning for predictive analytics.
  • It could be offloading of heavy batch processes that are run on mainframes. They can be replaced by way of re-engineering CICS-COBOL based batch processes to the Big Data platform.
  • It could be long term storage of data for compliance and regulatory purpose.
  • It could be big reports and downloads that are regularly purged due to lack of space.
  • It could be mounting of volumes in a mainframe every time there is an ad-hoc audit or regulatory query about something that occurred in the past.

There are no bad use cases here. A robust data platform can go a long way in addressing various needs of businesses and their end users. It could store data perennially and still keep it available at the click of a button. The cost of ownership is definitely reduced to a non-issue. All the current enterprise applications can keep running as usual while the enterprise prepares its data platform based IT infrastructure. It is a paradigm shift that will happen either by choice or be forced upon at the last minute. You can put on a beret today or wear a fireman’s hat at the last minute.

What is your Big Data strategy, how far are you in your Hadoop journey. Share with us. Contact us for a free assessment at or visit Or give us a call at +1.925.400.8475 to discuss the possibilities  and opportunities waiting to be exploited by enterprises in this realm.

About Trehanz

At Trehanz, we are a team of professionals who are Big Data, Hadoop and Cloud technology enthusiasts. We currently have products that help our clients start their journey of Hadoop fairly quickly and see the ROI on their Big Data investments very early on in the process. Reach us at


Leave a Reply