Can We Learn From Student Mistakes in a Formative, Reading Comprehension Assessment?

This article describes an ongoing project to develop a formative, inferential reading comprehension assessment of causal story comprehension. It has three features to enhance classroom use: equated scale scores for progress monitoring within and across grades, a scale score to distinguish among low-scoring students based on patterns of mistakes, and a reading efficiency index. Instead of two response types for each multiple-choice item, correct and incorrect, each item has three response types: correct and two incorrect response types. Prior results on reliability, convergent and discriminant validity, and predictive utility of mistake subscores are briefly described. The three-response-type structure of items required rethinking the item response theory (IRT) modeling. IRT-modeling results are presented, and implications for formative assessments and instructional use are discussed.