Intolerance: It works everywhere, even in Big Data

By January 5, 2016 Uncategorized No Comments


Intolerance and its base word ‘Tolerance’ are getting a little too much attention here in India these days. Its primary driver is how high echelons of Nehru-vian elitism are shattered after son of a chai-wala (Tea stall owner) became prime minister. To make matters worse, for this elitist Lutyen’s Delhi crowd, the man is still living a simple life as a PM.  He has cut back on government funding and endorsement for foreign trips for writers and creative community folks as he perhaps believes that just like any other business, creative community should find its funding by itself and not be parasites to government or International funding. Hence the elites are calling the PM and his government intolerant for lack of any other issues due to his clean track record.

PM Modi works 18+ hours a day and hence people around him have to put in good number of hours at work and that hogs media and social media as well. He is not married and doesn’t elevate his family members into any lucrative government positions unlike all other elitist politicians who take it up as their prime & solemn responsibility once they are in power and hence there is no room for dynastic-ism or filial favors in his style of functioning. Moreover he is extremely business oriented and a rightist. He is eager to replicate all the good things that Chinese, Americans and all other fast growing economies did during their high growth periods.  Anyway, so much for Indian politics but the word of the day i.e. ‘Intolerance’ has never been more relevant than now in Big Data space as well.

Previously, it was okay for enterprise applications to hit a performance bottleneck and companies used alternate ways to handle additional load on the systems. Some of those techniques were:

  • Provide alternate ways of paying bills, accessing account information.
  • Segregating data centers by city, state or some other geographical criteria.
  • Not integrating similar existing systems as the combined load might jeopardize availability of a mission critical system.
  • Telecom providers offer cheaper calls during nights and struggle to explain the low speed during peak times. Needless to mention that Mothers’ day clogging of telecom networks.
  • Retailers always expressed their inability to handle customers at stores and the perennial delays observed on Black Fridays and Christmas Eve.
  • Highways can handle only so many automobiles and hence Thanksgiving weekend getaway can easily turn out to be a nightmare.

To solve all of the above, what you need is zero tolerance or Intolerance towards these in your organization. Technology is now in place to enable use cases such as:

  • Timely feedback about traffic conditions on various highways,
  • Replacing Black Friday shopping with online deal hunting without the queues. If 2015’s Black Friday shopping pattern is any indicator of things to come, then it would be fair to say that the modern day shoppers are Intolerant and for all the good reasons. If the following graph is any indicator, the intolerance has already made its way in the psyche of consumers.


Big Data has made online experience a breeze. Customers are totally spoilt for choices that e-commerce platforms offer these days. Driving all this under the hood is new age technology platforms broadly categorized as Big Data. Elasticity and the ability to scale horizontally on-demand is theoretically infinite in these data platforms. The good news here is that it is not just the digital natives that have access to these technologies.

Every company today can leverage the Big Data and Cloud technologies in order to leverage and compete in this intolerant marketplace. The customers’ patience levels are really low especially when they are using these systems via a mobile device. Customers access these anytime & anywhere and still expect a delightful experience. It is about time that every brick-and-mortar business takes a note of this massive change in data-driven platforms and enterprise architecture that is being necessitated by the intolerant customers.  Following is a list of items that CIO or CDO (Chief Digital Officer) should include in his upcoming IT roadmap and budget.

  1. Cloud strategy – Are we using enough of cloud technologies for development, testing, staging and production.
  2. Big Data strategy – Are we leveraging Big Data to the extent that we can handle any peaks and volume fluctuations.
  3. Mobility – What is mobility strategy for both customer-facing as well as supply side facing functions? Is our UX more nifty and friendly that the competition. Are we capturing enough metrics about user experience?
  4. IoT – Do we need to handle and take advantage or IoT to improve our operations. Is our supply chain process, our logistics framework, our HR processes optimized using ubiquitous IoT technologies or not?

Big Data adoption is resisted as every change is hard and uncertain. Analytics and business intelligence use cases are low hanging fruits when it comes to Big Data adoption. Although customers are becoming increasingly intolerant but enterprises have their own apprehensions that need to be addressed. For starters, enterprises want to know where to start. Enterprises also want to start small especially when comes to funding a Big Data initiative. They don’t want to hire a bunch of Big Data professionals with an open ended agenda. Enterprises don’t want to take the big plunge without having numbers and data that they can use to justify to the management and stakeholders about the radical shift.

Here are few baby steps to take in order to ensure that your initial foray into big data doesn’t turn out to be non-starter killing all the strategy change upfront.

  1. Pick a analytics or data crunching and integration use case as the first one to be addressed using Big Data. The use cases should involve processing of millions of records if not on day one but definitely accumulated over a period of time.
  2. Start with tools that can help you quickly ingest or import your data inside a Hadoop cluster. Some examples of this are HortonWorks HDF, Hue, Trehanz EDI.
  3. Now, look for tools that can help you slice and dice data. This is important as Hadoop’s power stems from schema on read as opposed a cookie-cutter upfront schema that you are forced to work in traditional systems.
  4. Charting and reporting – Now that you have all your data and also nicely massaged and evaluated pre-aggregates ready for use, you can start charting and reporting using this data.
  5. Export – This is another integration piece that you should look for in the suite of tools that you will be using to get started with your Hadoop journey. This will help bridge the data back to the sources and systems that it came from, thus completing the loop. This will ensure that your current systems and IT investments are really getting well utilized and even more efficiently with the help of addition of Hadoop to the technology mix.

So don’t wait for more industry success stories before starting your Hadoop journey. It is about time you start taking advantage of the edge that big data brings to the table. Hadoop helps you get orthogonal views of customers, sales, operational data, and social data-streams or just about any type of data especially if high volume, variety or velocity has been the concern prohibiting you in the past from embarking on any such initiatives.

With the kind of technology maturity that is now available, it is about time you revisit your big data strategy as an integral part of IT infrastructure optimization. What is your Big Data strategy? Do you have any plans to take advantage of Big Data in 2016 to grow your business? Feel free to contact us for a free assessment at Or give us a call at +1.925.400.8475 to discuss possibilities.​

About JTrehanz

At Trehanz, we are a team of professionals who are Big Data, Hadoop and Cloud technology enthusiasts. We currently have products that help our clients start their journey of Hadoop fairly quickly and see the ROI on their Big Data investments very early on in the process. Reach us at

Leave a Reply