The generalizability of CBM survey-level mathematics assessments: Just how many samples do we need?

This study examined the extent to which CBM survey-level mathematics assessment varies as a function of type of survey-level assessment used (i.e., single-skill and multiple-skill math probes) and the generalizability of results both within and across assessment methodologies. Participants included 67 students enrolled in first through fifth grades. All students were assessed using grade-appropriate single- and multiple-skill CBM math survey-level probes. Findings indicate that single- and multiple-skill survey level CBM math assessments do indeed measure two distinct constructs. Results of a generalizability analysis suggest that for single-skill survey-level CBM math assessments the administration of one single-skill math probe provides information of sufficient quality to make both criterion- and norm-referenced decisions. Similar results were also observed for multiple-skill probes. However, given variations in performance the administration of three multiple-skill math probes as part of a survey-level CBM math assessment continues to be recommended.