Does topic matter? topic influences on linguistic and rubric-based evaluation of writing

Nia Dowell, Sidney K. D'Mello, Caitlin Mills, Art Graesser

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Although writing is an integral part of education, there is limited knowledge on how assigned topics influence writing quality both in terms of micro-level linguistic features and macro-level subjective evaluations by human judges. We addressed this question by conducting a study in which 44 students wrote short essays on three different topics: traditional academic-based topics such as the ones used in standardized tests, personal emotional experiences, and socially charged topics. The essays were automatically scored on five linguistic dimensions (narrativity, situation model cohesion, referential cohesion, syntactic complexity, and word abstractness). They were also manually scored by human judges based on a rubric focusing on macro-level dimensions (i.e., introduction, thesis, and conclusion). The results indicated that topic-related differences were observed on both the rubric-based and linguistic assessments, although there were weak relationships between these two measures.

Original languageEnglish (US)
Title of host publicationArtificial Intelligence in Education - 15th International Conference, AIED 2011
Pages450-452
Number of pages3
DOIs
StatePublished - 2011
Externally publishedYes
Event15th International Conference on Artificial Intelligence in Education, AIED 2011 - Auckland, New Zealand
Duration: Jun 28 2011Jul 1 2011

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume6738 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference15th International Conference on Artificial Intelligence in Education, AIED 2011
Country/TerritoryNew Zealand
CityAuckland
Period6/28/117/1/11

Keywords

  • Coh-Metrix
  • Coherence
  • Cohesion
  • Linguistics
  • Writing quality

Fingerprint

Dive into the research topics of 'Does topic matter? topic influences on linguistic and rubric-based evaluation of writing'. Together they form a unique fingerprint.

Cite this