TY - JOUR
T1 - An evaluation of the reliability of probability judgments across response modes and over time
AU - Whitcomb, Kathleen M.
AU - Önkal, Dilek
AU - Benson, P. George
AU - Curley, Shawn P.
N1 - Copyright:
Copyright 2016 Elsevier B.V., All rights reserved.
PY - 1993/12
Y1 - 1993/12
N2 - Despite the importance of probability assessment methods in behavioral decision theory and decision analysis, little attention has been directed at evaluating their reliability and validity. In fact, no comprehensive study of reliability has been undertaken. Since reliability is a necessary condition for validity, this oversight is significant. The present study was motivated by that oversight. We investigated the reliability of probability measures derived from three response modes: numerical probabilities, pie diagrams, and odds. Unlike previous studies, the experiment was designed to distinguish systematic deviations in probability judgments, such as those due to experience or practice, from random deviations. It was found that subjects assessed probabilities reliably for all three assessment methods regardless of the reliability measures employed. However, a small but statistically significant decrease over time in the magnitudes of assessed probabilities was observed. This effect was linked to a decrease in subjects overconfidence during the course of the experiment.
AB - Despite the importance of probability assessment methods in behavioral decision theory and decision analysis, little attention has been directed at evaluating their reliability and validity. In fact, no comprehensive study of reliability has been undertaken. Since reliability is a necessary condition for validity, this oversight is significant. The present study was motivated by that oversight. We investigated the reliability of probability measures derived from three response modes: numerical probabilities, pie diagrams, and odds. Unlike previous studies, the experiment was designed to distinguish systematic deviations in probability judgments, such as those due to experience or practice, from random deviations. It was found that subjects assessed probabilities reliably for all three assessment methods regardless of the reliability measures employed. However, a small but statistically significant decrease over time in the magnitudes of assessed probabilities was observed. This effect was linked to a decrease in subjects overconfidence during the course of the experiment.
KW - Probability assessment
KW - Probability judgment
KW - Reliability
KW - Subjective probability
UR - http://www.scopus.com/inward/record.url?scp=84980162954&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84980162954&partnerID=8YFLogxK
U2 - 10.1002/bdm.3960060406
DO - 10.1002/bdm.3960060406
M3 - Article
AN - SCOPUS:84980162954
SN - 0894-3257
VL - 6
SP - 283
EP - 296
JO - Journal of Behavioral Decision Making
JF - Journal of Behavioral Decision Making
IS - 4
ER -