Skip to main content
SpringerLink
Log in

The Delphi technique and judgmental forecasting

  • Published:
Climatic Change Aims and scope Submit manuscript

Abstract

The Delphi technique for judgmental forecasting by expert groups is described and the controversy surrounding its use is summarized. The technique clearly does not eliminate all unwanted psychological effects on group judgment. Furthermore, the design of most Delphi studies makes it impossible to separate the signal from the noise in expert judgment. A methodological standard for evaluating judgmental forecasts is proposed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Armstrong, J. Scott: 1978, Long-Range Forecasting from Crystal Ball to Computer, New York, Wiley.

    Google Scholar 

  • Auboy, J. R.: 1981, ‘A New Tool for Fishery Managers: The Delphi Technique’, North American Journal of Fisheries Management, 1, 55–59.

    Google Scholar 

  • Axelrod, R. (Ed.): 1976, Structure of Decision: The Cognitive Maps of Political Elites Princeton, Princeton University Press.

    Google Scholar 

  • Baldwin, G. H.: 1982, ‘The Delphi Technique and the Forecasting of Specific Fringe Benefits’, Futures, 14, 319–325.

    Google Scholar 

  • Bardecki, M. J.: 1984, ‘Participants' Response to the Delphi Method: An Attitudinal Perspective’, Technological Forecasting and Social Change, 5, 281–292.

    Google Scholar 

  • Camerer, C. F.: 1981, ‘General Conditions for the Success of Bootstrapping Models’, Organizational Behavior and Human Performance, 27, 411–422.

    Google Scholar 

  • Chan, S.: 1982, ‘Expert Judgments Under Uncertainty: Some Evidence and Suggestions’, Social Science Quarterly, 63, 428–444.

    Google Scholar 

  • Cicarelli, J.: 1984, ‘The Future of Economics: A Delphi Study’, Forecasting and Social Change, 25, 139–157.

    Google Scholar 

  • Coates, J. F.: 1975, ‘In Defense of Delphi’, Technological Forecasting and Social Change, 7, 193–194.

    Google Scholar 

  • Dalkey, N.: 1969, ‘An Experimental Study of Group Opinion: The Delphi Method’, Futures, 1, 408–426.

    Google Scholar 

  • Dawes, R. M., and B. Corrigan: 1974, ‘Linear Models in Decision Making’, Psychological Bulletin, 81, 95–106.

    Google Scholar 

  • Delbecq, A. L., A. H. Van de Ven, and D. H. Gustafson: 1975, Group Techniques for Program Planning, Glenview, IL, Scott, Foresman.

    Google Scholar 

  • Ebbesen, E. B., and V. J. Konecni: 1980, ‘On the External Validity of Decision-Making Research: What Do We Know About Decisions in the Real World?’, In S. Wallsten (Ed.), Cognitive Processes in Choice and Decision Behavior, Hillsdale, New Jersey, Erlbaum.

    Google Scholar 

  • Einhorn, H., and R. M. Hogarth: 1981, ‘Behavioral Decision Theory: Processes of Judgment and Choice’, Annual Review of Psychology, 32, 53–88.

    Google Scholar 

  • Einhorn, H. J., and R. M. Hogarth: 1982, ‘Prediction, Diagnosis, and Causal Thinking in Forecasting’, Journal of Forecasting, 1, 23–36.

    Google Scholar 

  • Einhorn, H. J., D. N. Kleinmuntz, and B. Kleinmuntz: 1979, ‘Linear Regression and Processing-Tracing Models of Judgment’, Psychological Review, 86, 465–485.

    Google Scholar 

  • Ericsson, K. A., and H. A. Simon: 1980, ‘Verbal Reports as Data’, Psychological Review, 87, 215–251.

    Google Scholar 

  • Evans, J. St. B. T.: 1982, ‘Psychological Pitfalls in Forecasting’, Futures, 14, 258–265.

    Google Scholar 

  • Fischer, G. W.: 1981, ‘When Oracles Fail—A Comparison of Four procedures for Aggregating Subjective Probability Forecasts’, Organizational Behavior and Human Performance, 28, 96–110.

    Google Scholar 

  • Fischhoff, B.: 1982, ‘Debiasing’, In D. Kahneman, P. Slovic, and A. Tversky (Eds.), Judgment Under Uncertainty: Heuristics and Biases, New York, Cambridge University Press.

    Google Scholar 

  • Fischhoff, B.: 1983, ‘Predicting Frames’, Journal of Experimental Psychology: Learning Memory and Cognition, 9, 103–116.

    Google Scholar 

  • Fry, F. L.: 1980, ‘The End of Affirmative Action’, Business Horizons, 23, 34–40.

    Google Scholar 

  • Goldschmidt, P. G.: 1975, ‘Scientific Inquiry or Political Critique?’, Technological Forecasting and Social Change, 7, 195–213.

    Google Scholar 

  • Hammond, K. R.: 1976, ‘Externalizing the Parameters of Quasirational Thought’, In M. Zeleny (Ed.), Multiple Criteria Decision Making, Kyoto 1975, New York, Springer-Verlag, 1976.

    Google Scholar 

  • Hammond, K. R.: 1981, ‘Principles of Organization in Intuitive and Analytical Cognition’, University of Colorado at Boulder, Center for Research on Judgment and Policy, Report No. 231.

  • Hammond, K. R., and L. Adelman: 1976, ‘Science, Values, and Human Judgment’, Science, 194, 389–396.

    Google Scholar 

  • Hammond, K. R., B. F. Anderson, J. Sutherland, and B. Marvin: 1984, ‘Improving Scientists Judgments of Risk’, Risk Analysis, 4, 69–78.

    Google Scholar 

  • Hammond, K. R., G. H. McClelland, and J. Mumpower: 1980, Human Judgment and Decision Making: Theories, Methods, and Procedures, New York, Praeger.

    Google Scholar 

  • Hammond, K. R., T. R. Stewart, B. Brehmer, and D. O. Steinman: 1975, ‘Social Judgment Theory’, In M. F. Kaplan and S. Schwartz (Eds.), Human Judgment and Decision Processes, New York, Academic Press.

    Google Scholar 

  • Hill, K. Q. and J. Fowles: 1975, ‘The Methodological Worth of the Delphi Forecasting Technique’, Technological Forecasting and Social Change, 7, 179–192.

    Google Scholar 

  • Hogarth, R. M.: 1980, Judgement and Choice: The Psychology of Decision, Chichester, England, Wiley.

    Google Scholar 

  • Hogarth, R. M. and S. Makridakis: 1981, ‘Forecasting and Planning: An Evaluation’, Management Science, 27, 115–138.

    Google Scholar 

  • Kahneman, D., and A. Tversky: 1979, ‘Intuitive Prediction: Biases and Corrective Procedures’, In S. Makridakis and S. C. Wheelwright (Eds.), Forecasting, TIMS, Studies in Management Science, 12, 313–327.

  • Kahneman, D., P. Slovic, and A. Tversky (Eds.): 1982, Judgment under Uncertainty: Heuristics and Biases, New York, Cambridge University Press.

    Google Scholar 

  • Keeney, R. L.: 1982, ‘Decision Analysis: An Overview’, Operations Research, 30, 803–838.

    Google Scholar 

  • Keeney, R. L., and H. Raiffa: 1976, Decisions with Multiple Objectives: Preferences and Value Tradeoffs, New York, Wiley.

    Google Scholar 

  • Kleinmuntz, B. (Ed.): 1968, Formal Representation of Human Judgment, New York, Wiley.

    Google Scholar 

  • Kruglanski, A. W., N. Friedland, and E. Farkash: 1984, ‘Lay Persons' Sensitivity to Statistical Information: The Case of High Perceived Applicability’, Journal of Personality and Social Psychology, 46, 503–518.

    Google Scholar 

  • Levin, I. P., J. J. Louviere, A. A. Schepanski, and K. L. Norman: 1983, ‘External Validity Tests of Laboratory Studies of Information Integration’, Organizational Behavior and Human Performance, 31, 173–193.

    Google Scholar 

  • Linstone, H. A.: 1975, ‘Eight Basic Pitfalls: A Checklist’, In Linstone, H. D., and M. Turoff (Eds), The Delphi Method: Techniques and Applications, Reading, MA, Addison-Wesley, 573–586.

    Google Scholar 

  • Linstone, H. A. and M. Turoff: 1975, ‘Introduction’, In Linstone, H. D., and M. Turoff (Eds), The Delphi Method: Techniques and Applications, Reading, MA, Addison-Wesley, 3–12.

    Google Scholar 

  • Martino, J. P.: 1983, Technological Forecasting for Decision Making, (Second edition), New York, North-Holland.

    Google Scholar 

  • Mitroff, I. A. and M. Turoff: 1975, ‘Philosophical and Methodological Foundations of Delphi’, In Linstone, H. D., and M. Turoff (Eds), The Delphi Method: Techniques and Applications, Reading, MA, Addison-Wesley, 17–36.

    Google Scholar 

  • Morgan, D. R., J. P. Pelissero, and R. E. England: 1979, ‘Urban Planning: Using Delphi as a Decision-Making Aid’, Public Administration Review, 39, 380–384.

    Google Scholar 

  • Nisbett, R. E., D. H. Krantz, C. Jepson, and Z. Kunda: 1983, ‘The Use of Statistical Heuristics in Everyday Inductive Reasoning’, Psychological Review, 90, 339–363.

    Google Scholar 

  • Paliwoda, S. J.: 1983, ‘Predicting the Future Using Delphi’, Management Decision, 21, 31–38.

    Google Scholar 

  • Parente, F. J., J. K. Anderson, P. Myers, and T. O'Brien: 1984, ‘An Examination of Factors Contributing to Delphi Accuracy’, Journal of Forecasting, 3, 173–182.

    Google Scholar 

  • Pill, J.: 1971, ‘The Delphi Method: Substance, Context, a Critique and an Annotated Bibliography’, Socio-economic Planning Sciences, 5, 57–71.

    Google Scholar 

  • Preble, J. F.: 1983, ‘Public Sector Use of the Delphi Technique’, Technological Forecasting and Social Change, 23, 75–88.

    Google Scholar 

  • Raiffa, Howard: 1968, Decision Analysis: Introductory Lectures on Choices Under Uncertainty, Reading, MA, Addison-Wesley.

    Google Scholar 

  • Riggs, W. E.: 1983,‘The Delphi Technique: An Experimental Evaluation’, Technological Forecasting and Social Change, 23,89–94.

    Google Scholar 

  • Rohrbaugh, J.: 1979. ‘Improving the Quality of Group Judgment: Social Judgment Analysis and the Delphi Technique’, Organizational Behavior and Human Performance, 24, 79–92.

    Google Scholar 

  • Sackman, H.: 1975, Delphi Critique, Lexington, MA, D.C. Heath and Company.

    Google Scholar 

  • Sahel, D. and K. Yee: 1975, ‘Delphi: An Investigation from a Bayesian Viewpoint’, Technological Forecasting and Social Change, 7, 165–178.

    Google Scholar 

  • Scheele, D. S.: 1975, ‘Consumerism Comes to Delphi’, Technological Forecasting and Social Change, 7, 215–219.

    Google Scholar 

  • Sjoberg, L.: 1982, ‘Aided and Unaided Decision Making: Improving Intuitive Judgment’, Journal of Forecasting, 1, 349–363.

    Google Scholar 

  • Slovic, P., B. Fischhoff, and S. Lichtenstein: 1977, ‘Behavioral Decision Theory’, Annual Review of Psychology, 28, 1–39.

    Google Scholar 

  • Slovic, P., and S. Lichtenstein: 1971, ‘Comparison of Bayesian and Regression Approaches to the Study of Information Processing in Judgment’, Organizational Behavior and Human Performance, 6, 649–744.

    Google Scholar 

  • Spinelli, T.: 1983, ‘The Delphi Decision-Making Process’, The Journal of Psychology, 113, 73–80.

    Google Scholar 

  • Steiner, I. D.: 1972, Group Processes and Productivity, New York, Academic Press.

    Google Scholar 

  • Stewart, T. R., and M. H. Glantz: 1985, ‘Expert Judgment and Climate Forecasting: A Methodological Critique of “Climate Change to the Year 2000”’, Climatic Change, 7, 159–183.

    Google Scholar 

  • Tversky, A., and D. Kahneman: 1974, ‘Judgment Under Uncertainty: Heuristics and Biases’, Science, 185, 1124–1131.

    Google Scholar 

  • Tversky, A., and D. Kahneman: 1981, ‘The Framing of Decisions and the Rationality of Choice’, Science, 211, 453–458.

    Google Scholar 

  • Tversky, A., and D. Kahneman: 1983, ‘Extensional vs. Intuitive Reasoning: The Conjunction Fallacy in Probability Judgment’, Psychological Review, 90, 293–315.

    Google Scholar 

Download references

Author information

Author notes

  1. Thomas R. Stewart

    Present address: Center for Research on Judgment and Policy, University of Colorado, Campus Box 344, 80309, Boulder, Colorado, USA

Authors and Affiliations

  1. Environmental and Societal Impacts Group, National Center for Atmospheric Research, P.O. Box 3000, 80307, Boulder, CO, USA

    Thomas R. Stewart

Authors
  1. Thomas R. Stewart
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Stewart, T.R. The Delphi technique and judgmental forecasting. Climatic Change 11, 97–113 (1987). https://doi.org/10.1007/BF00138797

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00138797

Keywords

Search

Navigation