An evaluation of the reliability of probability judgments across response modes and over time

Kathleen M. Whitcomb, Dilek Önkal, P. George Benson, Shawn P. Curley

Research output: Contribution to journalArticlepeer-review

6 Scopus citations


Despite the importance of probability assessment methods in behavioral decision theory and decision analysis, little attention has been directed at evaluating their reliability and validity. In fact, no comprehensive study of reliability has been undertaken. Since reliability is a necessary condition for validity, this oversight is significant. The present study was motivated by that oversight. We investigated the reliability of probability measures derived from three response modes: numerical probabilities, pie diagrams, and odds. Unlike previous studies, the experiment was designed to distinguish systematic deviations in probability judgments, such as those due to experience or practice, from random deviations. It was found that subjects assessed probabilities reliably for all three assessment methods regardless of the reliability measures employed. However, a small but statistically significant decrease over time in the magnitudes of assessed probabilities was observed. This effect was linked to a decrease in subjects overconfidence during the course of the experiment.

Original languageEnglish (US)
Pages (from-to)283-296
Number of pages14
JournalJournal of Behavioral Decision Making
Issue number4
StatePublished - Dec 1993


  • Probability assessment
  • Probability judgment
  • Reliability
  • Subjective probability


Dive into the research topics of 'An evaluation of the reliability of probability judgments across response modes and over time'. Together they form a unique fingerprint.

Cite this