The Uncertainties of Big Data (Part 1)

We might be on the beginning on the big data “epoch”. There are certainly great opportunities not only for business decision makers, but also for those, who like to work with big analytics. But the forecasting uncertainties are NOT eliminated by big data. In fact, the big data makes us even blinder to something potential, different, and not recorded.  This is a source if big boundaries, limits and diminished opportunities.

By Olena Denysyuk

Isn’t fun to be able to predict the future and change the direction of a business? Not the way the clairvoyants do, but like economists/ data analysts do. There are no big differences in these 2 groups, some would say. Maybe, it’s right. Nobody can see the next 10 years.  A 10-year business forecast is a good structuring tool, but as seeing- into-the- future tool, there is not so much value in it. Sadly.

But there should be some underlying reasoning, which these two groups share, since they share the idea of prophecy.

The later one predict the future (form the opinions, make the decisions) basing any judgment on data, technical/statistical/accounting/graphical. The former one predicts the future…well… I guess they have their own access to data they need: social media, Google among others.  No matter what analytics you go after in the realm of predicting the future, data is the central component.

So the word” data” is not new…     

The financial agents were playing with it for decades. Bank, government, corporate finance departments and academics: from deriving correlations between assets and the market in the daily run, to deriving correlations between the wine prices and the weather in the 10-year distance.  So the idea of Big Data (or data crunching) is actually as old as capitalism. The only difference now is that is also comes from the social media; before it came from statistics databases, ERP, questionnaires, or equations. Before it was stored in excel sheets and managed by a controller; now it’s stored in terabytes and managed by the data warehouses.

I think some articles I read about big data have a focusing illusion about how data, especially how big data from the social media can predict the future. I must admit, some of the big data analytics are way out of my reach (these from the fields of IT: machine learning techniques, compression, filtering, outlier alerts, and sampling).

 Therefore, I will write about the big data prospects from the conventional strategic business approach, where my core competences are. I acknowledge the fact that I am not qualified to analyze/criticize the approaches that require extensive knowledge of data mining or programming that are alternatives to data crunching.

But I would like to add my own rational reasoning to the “revolutionary thinking”.  My major learning from the subject “Bubbles and Corporate Greed and Failure” is that we have to spot the irrational thinking at the early stage, the potential sources of corporate and market risks, greed and failures. So, now it’s time to apply these learning.

My point of departure is that the logic behind the big data is the same, as the logic behind data crunching in the disciplines of statistical analysis and finance. Statistical analysis is a basic tool to extract meaning from data. As soon as you calculate an "average" (this would be the core of big data crunching), you've entered the realm of statistics. You enter the realm of uncertainty, assumption, and subjectivity.

As you enter the realm of big data, you enter the realm of statistics. Then your GREAT OPPORTUNITIES from the data is starting diminishing. There are risks, weaknesses and uncertainties, noise and biases, generated by the data.

Forecasting Uncertainties

Conventionally, no one decision, from a rational perspective, should take a departure without a proper data crunching, supported by the mathematical and statistical certainties. The data crunching has always been a way to eliminate the business uncertainty.

One of the widely used models in finance is discounted cash flow (DCF) approach.  Applying the model, the value of the company equals the cash flows of different projects, discounted by its cost of capital (for example, the interest rate). In the decisions using the DCF approach, we assess uncertainty by adjusting its risk in a discount rate (using cost of capital). 

We also project cash flows, where uncertainty is embedded as a base- case, worst- case and best -case scenarios.

Underestimating/ overestimating these inputs, we can easily increase the likelihood of lower/higher net present (NPV) values, which it turn, leads to higher/lower rate of rejecting/accepting “good”/”bad” investment opportunities.

This is what we call the intrinsic firm value, which captures certain growth opportunities.

Worth mentioning, the forecast of future cash flows is supported by the historical data – looking backwards (but it can also be forward looking, almost disregarding the history). So, if you make a projection, you will form your subjective opinions, based on the history. It is assumed that your future will repeat the past (pattern based forecast).

So my point is that corporate finance uses data to make decisions by applying financial tools. However, under the DCF approach we do not capture the value of different/other options the company may face. Thus, models may be applied well in relatively stable business environment. 

The analysis is focused on something that has already been defined.

The same applies to big data analysis: you can use it in business-as-usual situations (at a time being). But very few companies operate in a stable business environment. So the data from the social media doesn’t capture the value of potential, biased or not yet existing customer behavior. It doesn’t capture the potential value of something that has not been recorded yet. This biases the Big Data, which, in turn, increases the likelihood of missed opportunities.

So the forecasting uncertainties are NOT eliminated by big data. In fact, the big data makes us even blinder to something potential, different, and not recorded.

Not to forget to mention the cost of extracting the big data.

Technical Uncertainties

So, the availability of big data is not yet a guarantee that it will help to spot the right customer, or the right market. The data should be founded, collected, analyzed, shared, transferred, presented, understood, and applied. And the more data we have, the more noise and uncertainty there are in that data.

Needless to mention, that the speed of generation and aging of the data/information is enormous. Do “normal” companies have the right resources to collect, analyze, present and adapt the data?

Thus, data itself is also a subject to technical limitations as well.

David Smith, on his blog, mentions that big data is becoming a new oil to companies, but- like a crude oil – it’s messy and requires a lot of resources to extract it. Also the mess of big data is being associated with uncertainty, isn’t it?

And the word uncertainty does not inspire confidence in us, nor in decision makers.

Not to forget to mention that interpretation of any data visualization would now require new skills. Traditionally, the management used the script data, which is basically the record of the past. There is the accounting information, databases, supply- or- demand- driven information, different correlations. Now big data is being presented in funny and by any means creative ways, and it’s understanding would of course require new skills, not only in programming and statistics, but also in design thinking, pattern and relationship seeing.

This is a step forward from a technical and traditional 2-dimentional presentation of data (qualitative) towards more visualized and playful (quantitative). This is a step forward from equation-based decision making towards closer to human many- dimensional judgment.

But this, in turn, throws us into human behavioral uncertainty.  (Read here about the behavioral uncertainty,part 2).