Conceptualising and classifying validity evidence for simulation

CONTEXT The term 'validity' is used pervasively in medical education, especially as it relates to curriculum, assessment, measurement and instrumentation. Exactly what is meant by the term 'validity' in the medical education literature is not always clearly defined. OBJECTIVES This study attempts to clarify, conceptualise and classify how validity fits within the context of assessment and to provide a framework for medical educators to deterimine the type and degree of validity evidence required for their specific assessment and evaluation needs. METHODS We apply a structure for considering validity, and its association with validation, in medical education. We build this discussion j around the use of simulation in medical training because of its rapid growth as a foundation for numeric measurement of performance in the development of clinical skills and reasoning. We explain why validity is inextricably tied to the assessment process in both simulation based medical training and traditional medical education. RESULTS This logical framework structures the type and degree of validity evidence for various assessment and evaluation needs. We also provide an example for medical educators to reference and follow in collecting and reviewing their own needs for validity evidence in all aspects of medical education. CONCLUSIONS Assessment is integral to measurement and decision making in medical education. The implications of assessment results are variably dependent on the inferences and decisions made from them. As such, validity evidence is critical, but is also flexibly tied to those decisions and not all assessments require the same degree of validity rigor. The framework described herein reinforces a model for medical educators to use in developing their assessment and evaluation needs and associated requirements for validity evidence.