Delphi Paradox: New Study Showed Disagreements Intensification Under Expert Surveys

Researchers from the Higher School of Economics (HSE) have systematically criticized one of the pillars of global technology foresight — the Delphi method. Their study ‘Biases in expert judgements in large-scale S&T Delphi Surveys: How to cope with them?’, published in 2025 in ‘Technological Forecasting and Social Change’ journal, shows that classical expert surveys not only do not guarantee objective consensus achieving but often implicitly reinforce bias and dispersion of opinions. Researchers have proposed supplementing the Delphi survey methodology with built-in “protective mechanisms” against potential distortions.

AI generated picture

Shedevrum.AI

International research team led by Dr. Alexander Sokolov has conducted detailed mathematical analysis of four major national Foresight-studies in Japan, Germany, the UK and Russia. Key finding was paradoxical as it doubted one of the basic Delphi principles that a second round of polling, conducted to bring experts' opinions closer together, often only increases the divergence. It turned out that most cases dispersion of expert assessments in the second round was higher than in the first one. At the same time, there is a shift in experts’ responses towards increasing assessment of importance and implementation timeframes of assessed topics. According to researchers, the problem lies in complex impact of cognitive, social and procedural distortions, which traditional methods ignore.

Authors described these distortions in details:

Cognitive

An expert's overconfidence in his/her own correctness and a tendency to stick to initial assessment even after receiving new information (the "anchoring effect").

Social

Authoritative opinion pressure, group solidarity, cultural stereotypes and prejudices dominating expert community.

Procedural

Errors inherent in survey design, such as poorly worded questions or incorrect ordering of questions.

"Blindly relying on collective expert opinion without taking these biases into account can lead to significant strategic errors," the study's authors warn.

To "protect" foresight from systemic deviations, Russian scientists proposed building distortion-filtering mechanisms into each Delphi stage.

Before survey: careful balancing of expert pool across disciplines, schools, and countries, as well as mandatory piloting of a questionnaire.

During survey: use of special scales that neutralize the “anchoring effect,” strict anonymity, and questions that test the expert’s level of confidence.

After survey: statistically data “cleaning” of identified biases and focusing on analyzing of counterarguments and “dissident” opinions instead of their mechanically averaging.

The proposed approach reflects a transition to a new, more rigorous generation of Foresight studies, which involves abandoning a dogma of expert assessments inevitable convergence.

Delphi surveys based on evidence-based methods of correction will significantly increase reliability of one of the most well-known strategic planning tools. This is especially relevant for Russia, which actively uses Foresight methods within the framework of National Technology Initiative development (NTI) and state science and technology programmes designing. The tools proposed by Russian researchers can improve strategic decisions validity in the context of intense global technological competition.