Why embracing regret is the best decision you can make in the age of ‘radical uncertainty’

In short, we have now entered an era of “radical uncertainty,” a term central to my work, coined by my friends, former Bank of England governor Mervyn King and economist John Kay, in their influential book of the same name.

In this context, the past is a poor predictor of the future and the shelf life of knowledge reduces dramatically. It is little wonder that decision-making feels hard.

People don’t feel like they can plan for the future

This is one of the key findings from Seizing Uncertainty, a major new study from HSBC on which I acted as adviser. We recruited more than 17,000 participants, including 3,000 business leaders, across 12 markets with the aim of understanding how radical uncertainty is impacting decision-making around the world.

Across the world, 68% of respondents said they were finding it hard to plan for the future. For businesses, the study highlighted myriad problems, including supply chain disruption, cybersecurity concerns, and the evolving needs of customers. Many executives expect uncertainty to increase dramatically over the next five years.

Strikingly, the study also revealed the lack of support many people feel when facing uncertainty. Among business leaders, 64% wish they had more support when making decisions, compared to 57% of the general population. Around the world, one in three people (36%) said they’d experienced increased feelings of detachment from others in the last five years. More than half of respondents (52%) expect to experience these feelings over the next five years.

The upshot is diminished confidence—and it’s particularly acute among business leaders. More than two-thirds said they wished they felt more capable of tackling decisions head-on. And more than a third said the smallest doubt could stop them from acting. Fear of failure and ensuing regret weigh heavy.

However, the confidence to take action and stay committed to it is central to all major decisions in life—and absolutely crucial in business contexts. Widespread decision paralysis among our business leaders should be a concern for us all as it has consequences for employees, customers, and, ultimately, our economies.

Seizing Uncertainty found that half of business leaders said they often feel regret over missed opportunities and a similar proportion regret the decisions they make. For them to succeed, additional tools, understanding, and support become essential.

The double-edged sword of narratives

I have always been interested in uncertainty and the difficulty academic disciplines like economics have in accommodating it.

It was this interest that led me to develop Conviction Narrative Theory (CNT)—a cross-disciplinary framework that tracks how social, psychological, and neurological processes to enable confident decision-making despite radical uncertainty.

Central to CNT is the idea that we have evolved the capacity to overcome our fear of failure, and this is grounded in our ability to form narratives. 

We choose narratives in our businesses, families, and wider networks, to envision how certain actions can produce certain results. They ebb and flow in local popularity but are essential for helping us turn information into predictions about the future and how we’d feel if they came true.

CNT suggests that when uncertain, decision-makers use narratives to support the actions they imagine will work, testing alternative futures both cognitively and emotionally.

When we do decide to act, it’s because a narrative has helped us achieve conviction: the motivation to approach an imagined outcome via a certain action, rather than avoid it. In this way, uncertainty and fear are sidelined.

But how does regret fit into all this? It’s about confidence and whether we build it in one of two ways. One way is to select a convenient narrative to simply blot out doubt, and the feeling we may regret a certain choice. In the short term, it can work. But eventually, it will lead us over a cliff. CNT calls this a “divided state.”

Another way is to accept uncomfortable emotions but investigate them—we call this an “integrated state.” Here, uncertainty becomes a blend of opportunity and risk, and courses of action become monitored experiments. We look back on what we’ve done, connect with how we feel about it, discuss it with those around us, and use the ensuing narratives to adjust the path forward.

Anyone who is struggling with fear of failure but needs to make high-stakes decisions must build stronger, more trusted networks, comprised of diverse advisers who speak their minds. Above all, they must investigate, not ignore, their feelings.

In a world increasingly dominated by artificial intelligence and automation, using emotions as a prompt to interrogate our potential decisions and social connections as well as data to test them out can deliver a strategic advantage.

Face and investigate, don’t avoid, the fear of regret and its paralyzing effects. To become a better decision-maker in this era of radical uncertainty, you must challenge the stigma surrounding failure and view it as an opportunity to learn.

To quote Samuel Beckett: “Try again. Fail again. Fail better.”

More must-read commentary published by Fortune:

The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.