Automated Claim Identification Using NLP Features in Student Argumentative Essays

Qian Wan; Scott Crossley; Michelle Banawan; Renu Balyan; Yu Tian; Danielle McNamara; Laura Allen

Automated Claim Identification Using NLP Features in Student Argumentative Essays

Qian Wan, Scott Crossley, Michelle Banawan, Renu Balyan, Yu Tian, Danielle McNamara, Laura Allen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

The current study explores the ability to predict argumentative claims in structurally-annotated student essays to gain insights into the role of argumentation structure in the quality of persuasive writing. Our annotation scheme specified six types of argumentative components based on the well-established Toulmin’s model of argumentation. We developed feature sets consisting of word count, frequency data of key n-grams, positionality data, and other lexical, syntactic, semantic features based on both sentential and suprasentential levels. The suprasentential Random Forest model based on frequency and positionality features yielded the best results, reporting an accuracy of 0.87 and kappa of 0.73. This model will be included in an online writing assessment tool to generate feedback for student writers.

Original language	English (US)
Title of host publication	Proceedings of the 14th International Conference on Educational Data Mining, EDM 2021
Editors	I-Han Hsiao, Shaghayegh Sahebi, Francois Bouchet, Jill-Jenn Vie
Publisher	International Educational Data Mining Society
Pages	375-383
Number of pages	9
ISBN (Electronic)	9781733673624
State	Published - 2021
Externally published	Yes
Event	14th International Conference on Educational Data Mining, EDM 2023 - Paris, France Duration: Jun 29 2021 → Jul 2 2021

Publication series

Name	Proceedings of the 14th International Conference on Educational Data Mining, EDM 2021

Conference

Conference	14th International Conference on Educational Data Mining, EDM 2023
Country/Territory	France
City	Paris
Period	6/29/21 → 7/2/21

Bibliographical note

Publisher Copyright:
© EDM 2021.All rights reserved.

Keywords

Argumentation
Argumentative writing
Claim identification

Find It

Find It at U of M Libraries

Cite this

Wan, Q., Crossley, S., Banawan, M., Balyan, R., Tian, Y., McNamara, D., & Allen, L. (2021). Automated Claim Identification Using NLP Features in Student Argumentative Essays. In I.-H. Hsiao, S. Sahebi, F. Bouchet, & J.-J. Vie (Eds.), Proceedings of the 14th International Conference on Educational Data Mining, EDM 2021 (pp. 375-383). (Proceedings of the 14th International Conference on Educational Data Mining, EDM 2021). International Educational Data Mining Society.

Automated Claim Identification Using NLP Features in Student Argumentative Essays. / Wan, Qian; Crossley, Scott; Banawan, Michelle et al.
Proceedings of the 14th International Conference on Educational Data Mining, EDM 2021. ed. / I-Han Hsiao; Shaghayegh Sahebi; Francois Bouchet; Jill-Jenn Vie. International Educational Data Mining Society, 2021. p. 375-383 (Proceedings of the 14th International Conference on Educational Data Mining, EDM 2021).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Wan, Q, Crossley, S, Banawan, M, Balyan, R, Tian, Y, McNamara, D & Allen, L 2021, Automated Claim Identification Using NLP Features in Student Argumentative Essays. in I-H Hsiao, S Sahebi, F Bouchet & J-J Vie (eds), Proceedings of the 14th International Conference on Educational Data Mining, EDM 2021. Proceedings of the 14th International Conference on Educational Data Mining, EDM 2021, International Educational Data Mining Society, pp. 375-383, 14th International Conference on Educational Data Mining, EDM 2023, Paris, France, 6/29/21.

Wan Q, Crossley S, Banawan M, Balyan R, Tian Y, McNamara D et al. Automated Claim Identification Using NLP Features in Student Argumentative Essays. In Hsiao IH, Sahebi S, Bouchet F, Vie JJ, editors, Proceedings of the 14th International Conference on Educational Data Mining, EDM 2021. International Educational Data Mining Society. 2021. p. 375-383. (Proceedings of the 14th International Conference on Educational Data Mining, EDM 2021).

Wan, Qian ; Crossley, Scott ; Banawan, Michelle et al. / Automated Claim Identification Using NLP Features in Student Argumentative Essays. Proceedings of the 14th International Conference on Educational Data Mining, EDM 2021. editor / I-Han Hsiao ; Shaghayegh Sahebi ; Francois Bouchet ; Jill-Jenn Vie. International Educational Data Mining Society, 2021. pp. 375-383 (Proceedings of the 14th International Conference on Educational Data Mining, EDM 2021).

@inproceedings{47260901118f40a294c67dd35857a1e4,

title = "Automated Claim Identification Using NLP Features in Student Argumentative Essays",

abstract = "The current study explores the ability to predict argumentative claims in structurally-annotated student essays to gain insights into the role of argumentation structure in the quality of persuasive writing. Our annotation scheme specified six types of argumentative components based on the well-established Toulmin{\textquoteright}s model of argumentation. We developed feature sets consisting of word count, frequency data of key n-grams, positionality data, and other lexical, syntactic, semantic features based on both sentential and suprasentential levels. The suprasentential Random Forest model based on frequency and positionality features yielded the best results, reporting an accuracy of 0.87 and kappa of 0.73. This model will be included in an online writing assessment tool to generate feedback for student writers.",

keywords = "Argumentation, Argumentative writing, Claim identification",

author = "Qian Wan and Scott Crossley and Michelle Banawan and Renu Balyan and Yu Tian and Danielle McNamara and Laura Allen",

year = "2021",

language = "English (US)",

series = "Proceedings of the 14th International Conference on Educational Data Mining, EDM 2021",

publisher = "International Educational Data Mining Society",

pages = "375--383",

editor = "I-Han Hsiao and Shaghayegh Sahebi and Francois Bouchet and Jill-Jenn Vie",

booktitle = "Proceedings of the 14th International Conference on Educational Data Mining, EDM 2021",

}

TY - GEN

T1 - Automated Claim Identification Using NLP Features in Student Argumentative Essays

AU - Wan, Qian

AU - Crossley, Scott

AU - Banawan, Michelle

AU - Balyan, Renu

AU - Tian, Yu

AU - McNamara, Danielle

AU - Allen, Laura

PY - 2021

Y1 - 2021

N2 - The current study explores the ability to predict argumentative claims in structurally-annotated student essays to gain insights into the role of argumentation structure in the quality of persuasive writing. Our annotation scheme specified six types of argumentative components based on the well-established Toulmin’s model of argumentation. We developed feature sets consisting of word count, frequency data of key n-grams, positionality data, and other lexical, syntactic, semantic features based on both sentential and suprasentential levels. The suprasentential Random Forest model based on frequency and positionality features yielded the best results, reporting an accuracy of 0.87 and kappa of 0.73. This model will be included in an online writing assessment tool to generate feedback for student writers.

AB - The current study explores the ability to predict argumentative claims in structurally-annotated student essays to gain insights into the role of argumentation structure in the quality of persuasive writing. Our annotation scheme specified six types of argumentative components based on the well-established Toulmin’s model of argumentation. We developed feature sets consisting of word count, frequency data of key n-grams, positionality data, and other lexical, syntactic, semantic features based on both sentential and suprasentential levels. The suprasentential Random Forest model based on frequency and positionality features yielded the best results, reporting an accuracy of 0.87 and kappa of 0.73. This model will be included in an online writing assessment tool to generate feedback for student writers.

KW - Argumentation

KW - Argumentative writing

KW - Claim identification

UR - http://www.scopus.com/inward/record.url?scp=85130123006&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85130123006&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85130123006

T3 - Proceedings of the 14th International Conference on Educational Data Mining, EDM 2021

SP - 375

EP - 383

BT - Proceedings of the 14th International Conference on Educational Data Mining, EDM 2021

A2 - Hsiao, I-Han

A2 - Sahebi, Shaghayegh

A2 - Bouchet, Francois

A2 - Vie, Jill-Jenn

PB - International Educational Data Mining Society

T2 - 14th International Conference on Educational Data Mining, EDM 2023

Y2 - 29 June 2021 through 2 July 2021

ER -

Automated Claim Identification Using NLP Features in Student Argumentative Essays

Abstract

Publication series

Conference

Bibliographical note

Keywords

Find It

Other files and links

Fingerprint

Cite this