MUMBAI: Marketers may still be coming to terms with ‘big data’ but in future they are going to have to understand ‘dark data’ and combine that with behavioural psychology to predict human behaviour.

 

Consumers and marketers are accustomed to using search engines like Google, Yahoo or Bing as the first point of reference when carrying out research but these only skim the surface of what is available.

 

A search of the “surface web” – that part of the worldwide web that is readily available to the general public and searchable with standard web search engines – returns a fraction of the information that exists online.

“It’s like fishing in the top two feet of the ocean – you miss the virtual Mariana Trench below,” according to an article in Popular Science.

 

In India, Quantta Analytics is trawling the depths of the ‘deep web’ and the ‘dark web’ and supplying insights to several major Indian financial institutions, including State Bank of India (SBI) and Kotak, as well as QSR businesses like McDonald’s and Starbucks.

 

“We saw ourselves as people who could aggregate data from diverse sources, combine them in a homogeneous way, stack them up geographically – so that I can see where every ATM, bank, school, hospital is located, and then make the dark data around it useful,” co-founder Ritesh Bawri explained to Tech in Asia.

 

A retailer like Nike, he said, with hundreds of stores across India, gathers data from each, including things like what customers bought, how much time they spent in store, what time of day they bought.

 

“If Nike gives that data to me, I will add data about what else is happening in the world around those Nike stores,” he said.

 

“I can tell the company that around this store, where you clocked sales of a million Indian rupees, are potentially 500 customers who have not bought from you so far.”

 

Non-commercial applications might include mapping incidents of TB in a city against relevant locations, such as water bodies, restaurants and malls, and then using Quantta’s algorithms to “derive meaning by looking at all the different data sets stacked together and see the cause”.