Development and preliminary validation of the evaluation use scale for evaluation systems (EUS-ES)

The usefulness of evaluation is critical. Evaluation use occurs when, from its results or process, decisions are made about the program, it changes people's mindsets, or persuasive or legitimation actions happen (instrumental, conceptual, and symbolic uses respectively). Few quantitative evaluation use studies have been conducted in recent years. Most existing evaluation use scales are outdated due to theoretical advances and were project or program specific, not designed for evaluation systems. This study was designed to develop the Evaluation Use Scale for Evaluation Systems (EUS-ES) and gather initial validity evidence. We collected and analyzed data from Brazilian professors and program coordinators who were consultants of the Brazilian Graduate Evaluation System. Validity evidence was collected based on instrument content and internal structure. CFA results supported a three-factor-correlated model with adequate score reliability estimates. The EUS-ES is valuable for diagnostic purposes to identify low/high evaluation uses in an evaluation system setting.