Big data requires better judgement

Peter Chomowicz, program director of the master’s in technology leadership (MTL) program, argues that big data and AI can create false certainty, making human judgment more important than ever. Through scenario planning, he teaches leaders to test assumptions and prepare for multiple possible futures rather than relying on a single forecast.

Illustration of a robot and human standing on a scale, showing balancing AI

When I work with corporate or government clients to develop long-range strategies, I begin with a simple question: “What keeps you awake at night?” 

The answers usually range from the mundane, such as raccoons getting into the garbage, to existential concerns, such as nuclear war. Thoughtful executives, however, typically identify challenges that fall between those extremes — issues critical to their organization’s future with outcomes that remain highly uncertain. In scenario planning, this is known as the focal question. 

Each year, in Brown University’s master’s in technology leadership (MTL) program, I teach a futurism course focused on scenario planning. Students are divided into industry-based groups and develop a focal question that anchors their work for the semester. They then create four distinct visions of the future, exploring forces and conditions beyond any organization’s control. In the final step, they place their focal question into each of these possible futures and ask: if this future unfolds, how will the organization respond? 

The primary goal of scenario planning is to reshape the mental map of an organization’s key decision-makers. 

“ Throughout the course, we challenge each other to accept that tomorrow may not look like yesterday. While forecasting remains a powerful tool, periods of uncertainty require leaders to consider possibilities that fall outside of conventional expectations. ”

The illusion of certainty

Think back to the years leading up to the 2008 financial crisis. Many financial institutions relied on increasingly sophisticated models to assess risk, but those models were built on assumptions about how markets would behave. When those assumptions proved wrong, the consequences were significant.

The lesson is not that models are inherently flawed. Rather, models are limited by the assumptions that underpin them. Data can help us understand the past and identify patterns, but it cannot eliminate uncertainty about the future.

Leaders often place too much confidence in forecasts because numbers create the appearance of certainty. Yet strategy requires more than analysis; it requires judgment. The challenge is not simply interpreting the data in front of us, but questioning the assumptions behind it.

We are all hallucinating

Today, I see a similar risk emerging in how organizations think about artificial intelligence (AI). In classrooms and boardrooms alike, there is a growing tendency to assume that AI will continue to deliver rapid, exponential gains across every domain. 

This is beginning to sound like the next bubble, because everyone seems unable to imagine alternative futures where that trajectory slows, levels off or produces more uneven outcomes than expected.

This matters because strategic planning depends on the ability to consider multiple plausible futures, not just the one that currently feels most likely. If leaders become anchored to a single narrative about AI, they risk overlooking important uncertainties and constraints. 

There are real constraints and complications in how AI is being adopted that are often overlooked. For example, some recent AI-related jobs may reflect over-hiring rather than purely productivity gains from AI, even though it is often more appealing for organizations to frame them as technology-driven efficiencies (Rotman, 2026).

There are also unresolved questions about data and consent use. Many creators and publishers are still weighing whether it makes sense to contribute content into systems that may use it in ways they do not fully control or benefit from. 

Villalobos et al. (2022) suggest that there may be limits to how much useful, high-quality information is left for training AI systems, which could slow future progress.

“ The real question is not whether AI will matter — it almost certainly will — but how its impact unfolds, and how prepared organizations are for outcomes that differ from today’s expectations. ”

Beyond the rearview mirror

The real goal of strategy is not to predict the future, but rather to change the mental map of an organization's key decision-makers and thereby prepare for a range of possible futures. Big data models and AI can strengthen decision-making, but they can also create a false sense of certainty where their underlying assumptions go unchallenged.

The organizations that will be most resilient are not necessarily those with the most advanced tools, but those that maintain the strongest capacity for judgment. Those willing to question prevailing narratives and ask consistently: what if we’re wrong?

In the MTL program, we focus less on teaching students to read the map of the future, and more on helping them draw it. Because in uncertain environments, the ability to imagine alternative futures is itself a strategic advantage.

Related news

Peter Chomowicz, program director of the master's in technology leadership program, explains that today’s leaders must build a “second brain” by partnering with agentic AI to navigate rapid change. Reflecting this shift, the MTL program now includes a new course, “Agentic AI for Technology Leaders,” which equips professionals to build and deploy their own AI agents.
Read Article
Most AI deployments fail not because the technology is flawed, but because leadership responsibility is. Professor of the Practice of Leadership, Baba Prasad, outlines four levels every leader must master to deploy AI wisely, sustainably and competitively.
Read Article