Validation of automated scoring of science assessments

Ou Lydia Liu, Joseph A. Rios, Michael Heilman, Libby Gerard, Marcia C. Linn

Research output: Contribution to journalArticlepeer-review

87 Scopus citations


Constructed response items can both measure the coherence of student ideas and serve as reflective experiences to strengthen instruction. We report on new automated scoring technologies that can reduce the cost and complexity of scoring constructed-response items. This study explored the accuracy of c-rater-ML, an automated scoring engine developed by Educational Testing Service, for scoring eight science inquiry items that require students to use evidence to explain complex phenomena. Automated scoring showed satisfactory agreement with human scoring for all test takers as well as specific subgroups. These findings suggest that c-rater-ML offers a promising solution to scoring constructed-response science items and has the potential to increase the use of these items in both instruction and assessment.

Original languageEnglish (US)
Pages (from-to)215-233
Number of pages19
JournalJournal of Research in Science Teaching
Issue number2
StatePublished - Feb 1 2016
Externally publishedYes

Bibliographical note

Funding Information:
National Science Foundation; Contract grant number: 1119670. This material is based upon work supported by the National Science Foundation under Grant No. 1119670. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

Publisher Copyright:
© 2016 Wiley Periodicals, Inc.


  • automated scoring
  • c-rater-ML
  • science assessment


Dive into the research topics of 'Validation of automated scoring of science assessments'. Together they form a unique fingerprint.

Cite this