Big Data Opportunities – Are We There Yet? (Part 2)

Decision making based on the data from the social media can be biased, because the data do not capture what the decision makers need to minimize their own biases. The Big Data is biased and noisy. This is a source of big mistakes, risks and uncertainties.

By Olena Denysyuk

(for part 1, see here)

There is a new and big buzz-word is bubbling in many diversities of blogosphere, academics, forums and journalism, social influencers. The big word is BIG DATA (not just data, but BIG data). And the thinking is that the big data can give us BIG OPPORTUNITIES, and even visions to revolutionize the way we “stare” into the future. To me, there is a slight risk that we are moving towards wishful thinking, once again. Why wishful thinking? Well, the risks and uncertainties around big data are being underestimated, once again. We always had and will have a need in socially- sharing -thinking about something positive:  a belief that there is something that will bring us new opportunities; a new era towards a better future. And now all stare at big data bright future.

I have already mentioned, that the big data makes us even blinder to something potential, different, and not recorded (forecasting and technical uncertainties), so no matter which angle I would like to focus on when writing about big data, something else data had/has/will always have on its challenging side: the uncertainty generated by the human factor and by….big data itself.

But first allow me to mention what are these big opportunities I would like to focus on.

Strategic Improvements

There is a common belief about the big data, i.e. that companies can get a more detailed understanding of consumer behavior and attitudes and more precisely identify consumer segments, improving their ability to target the highest-value opportunities. But first, some of the “derivatives” of the strategic advantages from the social media:

·         it spots the right consumers, because they /(we) ourselves define the areas of our interests when contributing into social media profiles;

·         therefore companies promote the goods more effectively and more targeted;

·         it decreases the marketing cost when a company study the behavior this way (where qualitative questionnaire is becoming the outdated tool);

·         when our income, spending and preferences are studied, this allows price discrimination;

·         more efficient advertising will allow retailers to reduce out-of-stocks problems;

·         it provides a better shoppingexperience for consumers, and boost customer satisfaction.
 

·         understanding of customer needs and preferences  can contribute and change customer journey experience for the better

·         IT could help us get early clues tohard-to-predict, high-impact black swan events, so we can dig deeper into these clues and assess their validity (a statement that generates a lot of heat recently) (investigate catastrophic black swan events, be they airline crashes, financial crises, or terrorist attacks)

·         some customers can now interact with a company brand without ever interacting with a customer service team.

·         Companies can now predict customer satisfaction by simply following what people say about a brand (or a brand from a competitor)

·         today’s data is also the basis for sophisticated projections of the relationships between users.

So one of the advantages of big social data, is that is now capturing the human factor at a time (quantitative information), and showing it graphically (qualitatively) and creatively (visualized) at the same time: you would know what people want to share, how they feel about it, and from what part of the world it comes from. Here it is possible to measure what people talk about at the time being: which terms do they use; what people say about your competitor, what country is engaging mostly, etc. The companies can measure positive/negative chain of reactions. They can find out the potential sources of mavens (people with a special gift for bringing the world together; a gift of combining personality, curiosity, self-confidence, sociability and energy to send the relevant information in a thousand directions at once), the source of a connector (who accumulate knowledge and information), and who has the natural ability to attract interest. This is a platform of connecting the supplier with the buyer…

This is a step forward from a technical and traditional 2-dimentional presentation of data (qualitative) towards more visualized and playful (quantitative). This is a step forward from equation- based decision making towards closer to human many- dimensional judgment.

However, any decision making based on the big social data accepts not only forecasting, technical (see here), but also human behavioral uncertainties.

The behavioral uncertainty (quantitative uncertainty)

It is believed that big data is a good tool in predicting/understanding the behavioral human factors (how people respond, what they feel, what is our mood, the quantitative information). However, this belief, in fact, is driven by wishful thinking. The big data doesn’t capture our psychological biases in our behavior on the social media platform.

From another side of the story: one really popular method is visual presentation of data (charts, graphs), allowing human’s right side of brains to apply their amazing power to discern patterns and relationships.

This poses enormous risks and biases in understanding the data and decision making linked to that data. The technical side of the data presentation is a challenge, but the data are a source of biased interpretation of reality too.

For example, what you do, see and hear on the social media is not a microcosm of reality.  We might pretend the way they are not; we use our selective attention to post and updates; we also tend to create a more positive image of ourselves, neglecting the negativity and challenges we face. The majority of the time, it’s a place for entertainment, defined by our own impulses (not the consumer choices). It’s a platform to go beyond the normal, or stay silent and observing. It’s a parallel of our normal lives, just in a desired edition.

All this creates data noise.

So the data illusion (uncertainty) comes from our illusionary (biased) behavior on the social media platform. For example, we tend to believe that people not only think like us, but that they also agree with us (false-consensus bias). It's a bias where we overestimate how typical and normal we are, and assume that an agreement exists on matters when there may be none. If somebody like your post, we tend to assume, the do actually like it. But is that so?

The “like-chain”, will be biased by the false-consensus effect. In words of Wikipedia, “this bias is especially prevalent in group settings where one thinks the collective opinion of their own group matches that of the larger population. Since the members of a group reach a consensus and rarely encounter those who dispute it, they tend to believe that everybody thinks the same way.”

The false-consensus effect biases the big social data.

Moreover, it can also create the effect where the members of a fundamental or a leading group assume that more people on the outside agree with them. It can create an exaggerated confidence among the members of the group.

Additionally, we might end up wanting to be part of the group, therefore, accepting the fact that they might be wrong. The false-consensus effect can be contrasted with pluralistic ignorance, “an error in which people privately disapprove but publicly support what seems to be the majority view (regarding a norm or belief), when the majority in fact shares their (private) disapproval”.

While the false consensus effect leads people to wrongly believe that they agree with the majority (when the majority, in fact, openly disagrees with them), the pluralistic ignorance effect leads people to wrongly believe that they disagree with the majority (when the majority, in fact, covertly agrees with them).

Another example, when people talk about a particular brand, the conversation, or the words they use can be worthless for the data analytics, because it is being taking out of the situation context. The conversation could simply be limited to the image of themselves people want to create. Would the data analyst know, what is the purpose of conversation, how many people are involved (is there group thinking vs. individual thinking), who started the conversation and what was the starting point of the conversation (what is the anchoring effect). What was the tone of conversation (framing effect).

It’s a long explanation to state that the big social data (or group data) will always be biased by the false-consensus, the pluralistic ignorance, the anchoring and framing effects.

Our understanding of it is a subject to our cognitive biases. So making your decision linked to social data might be “smashed” by psychological biases. Therefore, it will not have any value, and in the worst case, lead to big mistakes.

So, decision making based on the data from the social media can be biased, because the data will not capture what the decision makers need to minimize their own biases. So some data is better than no data, one would say. But it’s important to remember that it is our psychological bias that can makes decision makers individually and collectively blind to uncertainty of big data.

So the Big Data is not a microcosm of our lives, not at all.

Conclusions

So the logic behind the big data is the same, as the logic behind data crunching in the disciplines of statistical analysis and finance. Statistical analysis is a basic tool to extract meaning from data. As soon as you calculate an "average" (this would be the core of big data crunching), you've entered the realm of statistics. You enter the realm of uncertainty, assumption, and subjectivity.

So, as you enter the realm of big data, you enter the realm of statistics. Then your GREAT OPPORTUNITIES from the data is starting diminishing. There are risks, weaknesses and uncertainties, noise and biases, generated by the data.

We might be on the beginning on the big data “epoch”. There are certainly great opportunities not only for business decision makers, but also for those, who like to work with big analytics. And there is a huge amount of information, biased or précised, uncorrelated or correlated, asymmetric or honest, conventional or innovative, insufficient or valuable, dirty and clean, logical and creative.
There is a data ocean out there. Some might be lucky to get meaning out of it, and some not. Somebody might well become very rich, if he/she finds out how to get meaning of it and how to prevent the uncertainties, which come with big data.

However, one of the characteristics of a bubble economy is wishful thinking towards something new: a new epoch that can revolutionize the word for the better. The opportunities of big data can certainly give us valuable opportunities for doing business, however, the risk and uncertainty should not be forgotten. The new sources for big mistakes should not be ignored.