The use of best visual acuity over several encounters as an outcome variable: An analysis of systematic bias

Dara Koozekanani, Douglas J. Covert, David V. Weinberg

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

PURPOSE. To investigate whether the use of the best of multiple measures of visual acuity as an endpoint introduces bias into study results. METHODS. Mathematical models and Monte Carlo simulations were used. A model was designed in which a hypothetical intervention did not influence the visual acuity. The best of one or more postintervention measures was used as the outcome variable and was compared to the baseline measure. Random test-retest variability was included in the model. RESULTS. When the better of two postintervention measures was used as the outcome variable with a sample size of 25, the model falsely rejected the null hypothesis 55% of the time. When the best of three measures was used, the false-positive rate increased to 90%. The probability of falsely rejecting the null hypothesis increased with increasing sample size, also increasing the number of measures used to select the outcome variable. CONCLUSIONS. Using the best of multiple measures as an outcome variable introduces a systematic bias resulting in false conclusions of improvement in that variable. The use of best of multiple measures of visual acuity as an outcome variable should be avoided.

Original languageEnglish (US)
Pages (from-to)3909-3912
Number of pages4
JournalInvestigative Ophthalmology and Visual Science
Volume51
Issue number8
DOIs
StatePublished - Aug 1 2010

Fingerprint

Visual Acuity
Sample Size
Theoretical Models
Outcome Assessment (Health Care)

Cite this

The use of best visual acuity over several encounters as an outcome variable : An analysis of systematic bias. / Koozekanani, Dara; Covert, Douglas J.; Weinberg, David V.

In: Investigative Ophthalmology and Visual Science, Vol. 51, No. 8, 01.08.2010, p. 3909-3912.

Research output: Contribution to journalArticle

@article{e28842faa957462db0641bdfbbb4c983,
title = "The use of best visual acuity over several encounters as an outcome variable: An analysis of systematic bias",
abstract = "PURPOSE. To investigate whether the use of the best of multiple measures of visual acuity as an endpoint introduces bias into study results. METHODS. Mathematical models and Monte Carlo simulations were used. A model was designed in which a hypothetical intervention did not influence the visual acuity. The best of one or more postintervention measures was used as the outcome variable and was compared to the baseline measure. Random test-retest variability was included in the model. RESULTS. When the better of two postintervention measures was used as the outcome variable with a sample size of 25, the model falsely rejected the null hypothesis 55{\%} of the time. When the best of three measures was used, the false-positive rate increased to 90{\%}. The probability of falsely rejecting the null hypothesis increased with increasing sample size, also increasing the number of measures used to select the outcome variable. CONCLUSIONS. Using the best of multiple measures as an outcome variable introduces a systematic bias resulting in false conclusions of improvement in that variable. The use of best of multiple measures of visual acuity as an outcome variable should be avoided.",
author = "Dara Koozekanani and Covert, {Douglas J.} and Weinberg, {David V.}",
year = "2010",
month = "8",
day = "1",
doi = "10.1167/iovs.09-4643",
language = "English (US)",
volume = "51",
pages = "3909--3912",
journal = "Investigative Ophthalmology and Visual Science",
issn = "0146-0404",
publisher = "Association for Research in Vision and Ophthalmology Inc.",
number = "8",

}

TY - JOUR

T1 - The use of best visual acuity over several encounters as an outcome variable

T2 - An analysis of systematic bias

AU - Koozekanani, Dara

AU - Covert, Douglas J.

AU - Weinberg, David V.

PY - 2010/8/1

Y1 - 2010/8/1

N2 - PURPOSE. To investigate whether the use of the best of multiple measures of visual acuity as an endpoint introduces bias into study results. METHODS. Mathematical models and Monte Carlo simulations were used. A model was designed in which a hypothetical intervention did not influence the visual acuity. The best of one or more postintervention measures was used as the outcome variable and was compared to the baseline measure. Random test-retest variability was included in the model. RESULTS. When the better of two postintervention measures was used as the outcome variable with a sample size of 25, the model falsely rejected the null hypothesis 55% of the time. When the best of three measures was used, the false-positive rate increased to 90%. The probability of falsely rejecting the null hypothesis increased with increasing sample size, also increasing the number of measures used to select the outcome variable. CONCLUSIONS. Using the best of multiple measures as an outcome variable introduces a systematic bias resulting in false conclusions of improvement in that variable. The use of best of multiple measures of visual acuity as an outcome variable should be avoided.

AB - PURPOSE. To investigate whether the use of the best of multiple measures of visual acuity as an endpoint introduces bias into study results. METHODS. Mathematical models and Monte Carlo simulations were used. A model was designed in which a hypothetical intervention did not influence the visual acuity. The best of one or more postintervention measures was used as the outcome variable and was compared to the baseline measure. Random test-retest variability was included in the model. RESULTS. When the better of two postintervention measures was used as the outcome variable with a sample size of 25, the model falsely rejected the null hypothesis 55% of the time. When the best of three measures was used, the false-positive rate increased to 90%. The probability of falsely rejecting the null hypothesis increased with increasing sample size, also increasing the number of measures used to select the outcome variable. CONCLUSIONS. Using the best of multiple measures as an outcome variable introduces a systematic bias resulting in false conclusions of improvement in that variable. The use of best of multiple measures of visual acuity as an outcome variable should be avoided.

UR - http://www.scopus.com/inward/record.url?scp=77955864125&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77955864125&partnerID=8YFLogxK

U2 - 10.1167/iovs.09-4643

DO - 10.1167/iovs.09-4643

M3 - Article

C2 - 20335608

AN - SCOPUS:77955864125

VL - 51

SP - 3909

EP - 3912

JO - Investigative Ophthalmology and Visual Science

JF - Investigative Ophthalmology and Visual Science

SN - 0146-0404

IS - 8

ER -