Cognitive Bias in Data-Driven Decision-Making

"The numbers don't lie," is an oft-repeated phrase to reiterate that any decision arrived on the basis of data cannot be biased. But is this assertion itself a product of bias?

Let's examine it a little closely. Below is a statement and two questions with options. Pick the option you think is the most probable answer. Try to answer as quickly as possible and without any external assistance.

"Out of a group of 10 individuals, five have received their graduate degree from an Ivy League university, and five have dropped out. All 10 individuals have started businesses."

Question 1: Given that all individuals have studied in Ivy League colleges, how many individuals' businesses will likely survive for more than five years?

1. More than five

2. Less than five

3. Five

Question 2: Given that Ivy League dropouts, including Bill Gates and Mark Zuckerberg, have started successful businesses, how many college dropouts' businesses will likely survive for more than five years, compared to those who have received their degrees?

1. The numbers are equal

2. More college dropouts' businesses will likely survive for more than five years than those with a college degree

3. Fewer college dropouts' businesses will likely survive for more than five years than those with a college degree

If you chose option 3 for both questions, you are probably closest to the right answer. The U.S. Small Business Association pegs the likelihood of a new business surviving for more than five years at about 50%. Similarly, a Harvard Business Review study showed that only about 4% of college dropouts had started successful businesses, compared to more than 60% of graduates.

But don't beat yourself up if you picked the wrong option. Most people are in the same boat as you because most people come with inherent biases. The bias that may have affected you in this case was the 'anchoring bias.'

Anchoring bias occurs when individuals rely too heavily on the first piece of information encountered, using it as a reference point for subsequent judgments. In the instances above, the first piece of information you got in Question 1 was that all are Ivy League students, and you may have immediately anchored that information to companies and brands founded by such individuals, including Google and Facebook. Similarly, in Question 2, your anchor was the fact that dropouts such as Gates and Zuckerberg have founded successful companies, which may have led you to believe that many Ivy League dropouts are successful entrepreneurs.

In data analysis, the anchoring bias can lead to an overreliance on initial data points or statistics, even if they are not representative of the overall dataset. For example, a financial analyst evaluating a company's stock performance might anchor their analysis on the stock's price at the beginning of the year, failing to account for subsequent fluctuations or market trends. Addressing anchoring bias requires you to consider a wide range of data points and avoid fixating on any single piece of information.

Biases Affect Decisions

In modern business, data-driven decision-making is key to strategic planning and operational efficiency. Organizations invest heavily in data collection, analysis, and development of advanced algorithms to gain a competitive edge. However, cognitive biases inherent in human thinking can compromise these efforts by distorting our interpretation of data and leading to suboptimal outcomes.

One common bias is confirmation bias, the inclination to seek and favor information that confirms preexisting beliefs. In data analysis, this means selectively focusing on data points that align with preconceived notions while downplaying or disregarding contradictory evidence. For instance, when evaluating a marketing campaign, a company might prioritize data indicating increased website traffic or social media engagement while overlooking data suggesting a decline in customer satisfaction.

The availability heuristic is another cognitive bias that can impact decision-making. This bias refers to the tendency to overestimate the likelihood of events based on their ease of recall. In data analysis, this can result in overemphasizing recent or salient events and neglecting less memorable but potentially more relevant data.

Finally, survivorship bias occurs when we focus exclusively on successful outcomes and ignore failures, leading to a distorted perception of reality. In data analysis, this manifests as analyzing only the data of successful projects, products, or individuals while neglecting data on failures. For example, a study examining the factors contributing to business success might focus solely on companies that have achieved growth or profitability while overlooking failed ventures.

To ensure effective data-driven decision-making, it is imperative to be aware of these cognitive biases and take steps to mitigate their impact. This can involve actively seeking out disconfirming evidence, systematically collecting and analyzing data over extended periods, and including data on both successes and failures in any analysis. By doing so, organizations can make more informed decisions based on a comprehensive understanding of the available data.

Mitigating Cognitive Bias for Better Decisions

While cognitive biases are inherent in human thinking, their influence on data-driven decision-making can be mitigated through various strategies. Gaining awareness of these biases among decision-makers and data analysts is the first step. Education and training programs can equip individuals with the knowledge and skills necessary to identify and address biases in their work.

The second step is establishing clear decision criteria before analyzing data, which can help ensure that decisions are based on objective evidence rather than subjective interpretations. Creating clear decision criteria involves defining specific metrics, thresholds, and success criteria that will guide the decision-making process.

Furthermore, fostering a culture of collaboration can be instrumental in mitigating cognitive biases. By involving individuals with diverse backgrounds, experiences, and perspectives in the decision-making process, organizations can gain a more comprehensive understanding of the data and reduce the likelihood of groupthink. Encouraging open discussion and debate, and actively seeking out dissenting opinions can also help expose and challenge biases.

Another key step is analyzing the data and forming hypotheses that can be confirmed and disproved using the available data. By actively employing a methodological approach to hypothesis testing in two scenarios of confirming and disproving the set hypothesis, one can challenge assumptions and reduce confirmation bias. Following this process encourages a comprehensive exploration of data, including metrics that contradict initial beliefs, potentially revealing hidden patterns or unexpected relationships. A rigorous approach fosters a more objective understanding of the data, leading to more robust insights and data-driven decisions grounded in reality rather than clouded by preconceived notions.

Finally, external validation can play a crucial role in ensuring the objectivity and reliability of data-driven decisions. Validation can involve peer review of findings, engaging external consultants, or utilizing independent data verification services.

Cognitive biases pose a significant challenge to effective data-driven decision-making. By understanding and actively addressing these biases, organizations can harness the power of data to make more informed, objective, and ultimately successful choices. Commit to ongoing learning, foster a culture of collaboration and diversity, and be willing to challenge assumptions and seek out unconfirming evidence. By embracing these principles, businesses can unlock the full potential of data-driven decision-making and achieve their strategic objectives.

About the Authors:

Richa Gupta, Business Unit Head at Mu Sigma, and Todd Wandtke Head of Marketing and Customer Success at Mu Sigma.