Generalizing expert misconception diagnoses through common wrong answer embedding

John Kolb, Scott Farrar, Zachary A. Pardos

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Misconceptions have been an important area of study in STEM education towards improving our understanding of learners' construction of knowledge. The advent of large-scale tutoring systems has given rise to an abundance of data in the form of learner question-answer logs in which signatures of misconceptions can be mined. In this work, we explore the extent to which collected expert misconception diagnoses can be generalized to held-out questions to add misconception semantics. We attempt this generalization by way of a question-answer neural embedding trained on chronological sequences of learner answers. As part of our study, we collect natural language misconception diagnoses from math educators for a sampling of student answers to questions within four topics on Khan Academy. Drawing inspiration from machine translation, we use a multinomial logistic regression model to explore how well the expert misconception semantics, in the form of bag-of-words vectors, can be mapped onto the learned embedding space and interpolated. We evaluate the ability of the space to generalize expert diagnoses using three levels of cross-fold validation in which we measure the recall of predicted natural language diagnoses across rater, topics, and questions. We find that the embedding provides generalization performance substantially beyond baseline approaches.

Original languageEnglish (US)
Title of host publicationEDM 2019 - Proceedings of the 12th International Conference on Educational Data Mining
EditorsCollin F. Lynch, Agathe Merceron, Michel Desmarais, Roger Nkambou
PublisherInternational Educational Data Mining Society
Pages342-347
Number of pages6
ISBN (Electronic)9781733673600
StatePublished - 2019
Externally publishedYes
Event12th International Conference on Educational Data Mining, EDM 2019 - Montreal, Canada
Duration: Jul 2 2019Jul 5 2019

Publication series

NameEDM 2019 - Proceedings of the 12th International Conference on Educational Data Mining

Conference

Conference12th International Conference on Educational Data Mining, EDM 2019
Country/TerritoryCanada
CityMontreal
Period7/2/197/5/19

Bibliographical note

Funding Information:
We thank Khan Academy for sharing anonymized exercise data. This work was supported, in part, by a grant from the National Science Foundation (#1547055).

Publisher Copyright:
© EDM 2019 - Proceedings of the 12th International Conference on Educational Data Mining. All rights reserved.

Fingerprint

Dive into the research topics of 'Generalizing expert misconception diagnoses through common wrong answer embedding'. Together they form a unique fingerprint.

Cite this