Gaze-based predictive models of deep reading comprehension

Eye gaze patterns can reveal user attention, reading fluency, corrective responding, and other reading processes, suggesting they can be used to develop automated, real-time assessments of comprehension. However, past work has focused on modeling factual comprehension, whereas we ask whether gaze patterns reflect deeper levels of comprehension where inferencing and elaboration are key. We trained linear regression and random forest models to predict the quality of users’ open-ended self-explanations (SEs) collected both during and after reading and scored on a continuous scale by human raters. Our models use theoretically grounded eye tracking features (number and duration of fixations, saccade distance, proportion of regressive and horizontal saccades, spatial dispersion of fixations, and reading time) captured from a remote, head-free eye tracker (Tobii TX300) as adult users read a long expository text (6500 words) in two studies (N = 106 and 131; 247 total). Our models: (1) demonstrated convergence with human-scored SEs (r =.322 and.354), by capturing both within-user and between-user differences in comprehension; (2) were distinct from alternate models of mind-wandering and shallow comprehension; (3) predicted multiple-choice posttests of inference-level comprehension (r =.288,.354) measured immediately after reading and after a week-long delay beyond the comparison models; and (4) generalized across new users and datasets. Such models could be embedded in digital reading interfaces to improve comprehension outcomes by delivering interventions based on users’ level of comprehension.