Big Data Must Give Way to Useful Information

business data analyst

The term 'big data' has become a nearly universal catchphrase intended to describe the large-scale collection and analysis of data for predictive purposes. Still, its use does not indicate the ability of those who use it to actually make the most of the data they collect. Therein lies the biggest reason the big data paradigm continues to be criticised.

When the concept of big data was first introduced decades ago, the idea was to develop systems that can collect an infinite amount of data with the hope of learning how to use that data at some point in the future. Since the 1980s, we have mastered our means of collecting and storing data. We have not even come close to figuring out how to use it. Suffice to say that big data must give way to useful information if it is to be used to create business intelligence reports, predictive analysis, etc.

Core Aspects of Business Intelligence

Any business data analyst who knows how to use big data effectively understands that there are a few core aspects of business intelligence around which the entire data universe revolves. Those core aspects include:

optimisation of key performance indicators

aggregation and allocation of data in multiple dimensions

integration of unstructured data sources

predictive simulations utilising statistical inference.

It goes without saying that business intelligence and data warehousing are two separate entities. When data analysts treat them as such, business intelligence reports take on a more detailed and specified role in helping businesses successfully predict the future and adapt to it. When analysts fail to draw that distinction, intelligence reports become muddled with a tremendous amount of data irrelevant to effective predictions.

Knowing What to Analyse

The critique of big data as a principal generally falls in one of two camps: the usefulness of the concept and the execution thereof. It is the execution that business data analysts like me are interested in.

One of the biggest execution challenges we face is knowing what to analyse. In short, data sets that are too large tend to offer too shallow an analysis to generate effective business intelligence reports. Smaller data sets are preferred. However, those smaller data sets may not provide the complete picture necessary for accurately predicting the future.

It is the job of business data analysts to not only know what kinds of data to analyse but also how big data sets and samples should be. Unfortunately, there is no university programme or certification course to teach this. Getting it right is a combination of experience, practice, observation, and real-time learning.

The concept of big data has been around for more than 40 years. And with our capacity for collecting and warehousing data growing by the day, the big data paradigm is not going away. Therefore, using data to generate effective business intelligence reports dictates we find better and more innovative ways to create useful information from the data we gather.

Send Chris Scanlon an Email
Profile picture of Chris Scanlon

Author: Chris Scanlon

Hullo.  I'm a graduate chartered accountant with 25 years experience in blue chip businesses and the last 15 years in owner manged businesses.  My particular skill is turning data into information. Bringing the performance management of the business alive so that ...