Reliability and Validity in Binary Ratings: Areas of Common Misunderstanding in Diagnosis and Symptom Ratings

Gregory Carey, Irving I Gottesman

Research output: Contribution to journalArticlepeer-review

72 Scopus citations

Abstract

Confusion may exist between the reliability of a binary rating (for example, schizophrenia versus not-schizophrenia) and its implications for validity. High reliability does not guarantee validity, but paradoxically, low reliability does not imply poor validity in all contexts. Changes in the base rate or in experimental design may indicate high validity even when the reliability was thought to be low. Attempts to improve the psychiatric nomanclature by increasing only reliability run the risk of the “attenuation paradox,” where further increases in reliability will make the ratings less valid. Finally, the assumption of random error in making diagnoses does not always hold, so that statistical analyses must be adjusted accordingly. New statistical methods are needed to index only false-positive or false-negative rates in order to quantify the error that will reduce some validity coefficients.

Original languageEnglish (US)
Pages (from-to)1454-1459
Number of pages6
JournalArchives of General Psychiatry
Volume35
Issue number12
DOIs
StatePublished - Dec 1978

Fingerprint Dive into the research topics of 'Reliability and Validity in Binary Ratings: Areas of Common Misunderstanding in Diagnosis and Symptom Ratings'. Together they form a unique fingerprint.

Cite this