Adaptive Validation Design: A Bayesian Approach to Validation Substudy Design with Prospective Data Collection

Lindsay J. Collin, Richard F. MacLehose, Thomas P. Ahern, Rebecca Nash, Darios Getahun, Douglas Roblin, Michael J. Silverberg, Michael Goodman, Timothy L. Lash

Research output: Contribution to journalArticlepeer-review

6 Scopus citations


An internal validation substudy compares an imperfect measurement of a variable with a gold-standard measurement in a subset of the study population. Validation data permit calculation of a bias-adjusted estimate, which has the same expected value as the association that would have been observed had the gold-standard measurement been available for the entire study population. Existing guidance on optimal sampling for validation substudies assumes complete enrollment and follow-up of the target cohort. No guidance exists for validation substudy design while cohort data are actively being collected. In this article, we use the framework of Bayesian monitoring methods to develop an adaptive approach to validation study design. This method monitors whether sufficient validation data have been collected to meet predefined criteria for estimation of the positive and negative predictive values. We demonstrate the utility of this method using the Study of Transition, Outcomes and Gender- A cohort study of transgender and gender nonconforming people. We demonstrate the method's ability to determine efficacy (when sufficient validation data have accumulated to obtain estimates of the predictive values that fall above a threshold value) and futility (when sufficient validation data have accumulated to conclude the mismeasured variable is an untenable substitute for the gold-standard measurement). This proposed method can be applied within the context of any parent epidemiologic study design and modified to meet alternative criteria given specific study or validation study objectives. Our method provides a novel approach to effective and efficient estimation of classification parameters as validation data accrue.

Original languageEnglish (US)
Pages (from-to)509-516
Number of pages8
Issue number4
StatePublished - Jul 1 2020

Bibliographical note

Funding Information:
Supported in part by the US National Cancer Institute (F31CA239566 awarded to L.J.C., and R01CA234538 awarded to T.L.L.) and the US National Library of Medicine (R01LM013049 awarded to T.L.L.). T.P.A. was supported by an award from the US National Institute of General Medical Sciences (P20 GM103644). STRONG cohort data were col-lected with support from Contract AD-12-11-4532 from the Patient Cen-tered Outcome Research Institute and by the National Institute of Child Health and Human Development R21HD076387 awarded to M.G.

Publisher Copyright:
© 2020 Lippincott Williams and Wilkins. All rights reserved.


  • Bayesian methods
  • Validation study design


Dive into the research topics of 'Adaptive Validation Design: A Bayesian Approach to Validation Substudy Design with Prospective Data Collection'. Together they form a unique fingerprint.

Cite this