This paper describes the DuluthNLP system that participated in Task 7 of SemEval-2022 on Identifying Plausible Clarifications of Implicit and Underspecified Phrases in Instructional Texts. Given an instructional text with an omitted token, the task requires models to classify or rank the plausibility of potential fillers. To solve the task, we fine-tuned the models BERT, RoBERTa, and ELECTRA on training data where potential fillers are rated for plausibility. This is a challenging problem, as shown by BERT-based models achieving accuracy less than 45%. However, our ELECTRA model with tuned class weights on CrossEntropyLoss achieves an accuracy of 53.3% on the official evaluation test data, which ranks 6 out of the 8 total submissions for Subtask A.
|Original language||English (US)|
|Title of host publication||SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop|
|Editors||Guy Emerson, Natalie Schluter, Gabriel Stanovsky, Ritesh Kumar, Alexis Palmer, Nathan Schneider, Siddharth Singh, Shyam Ratan|
|Publisher||Association for Computational Linguistics (ACL)|
|Number of pages||5|
|State||Published - 2022|
|Event||16th International Workshop on Semantic Evaluation, SemEval 2022 - Seattle, United States|
Duration: Jul 14 2022 → Jul 15 2022
|Name||SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop|
|Conference||16th International Workshop on Semantic Evaluation, SemEval 2022|
|Period||7/14/22 → 7/15/22|
Bibliographical noteFunding Information:
The authors would like to thank the organizers for the opportunity to participate in SemEval-2021 Task 7. We are also grateful to the three anonymous reviewers for their thoughtful comments and feedback. Finally, we would like to thank Dr. Alexis Elder for her contributions regarding the ethical considerations of this work.
© 2022 Association for Computational Linguistics.