Classifying Individuals with ASD Through Facial Emotion Recognition and Eye-Tracking

Ming Jiang, Sunday M. Francis, Diksha Srishyla, Christine Conelea, Qi Zhao, Suma Jacob

Research output: Chapter in Book/Report/Conference proceedingConference contribution

53 Scopus citations

Abstract

Individuals with Autism Spectrum Disorder (ASD) have been shown to have atypical scanning patterns during face and emotion perception. While previous studies characterized ASD using eye-tracking data, this study examined whether the use of eye movements combined with task performance in facial emotion recognition could be helpful to identify individuals with ASD. We tested 23 subjects with ASD and 35 controls using a Dynamic Affect Recognition Evaluation (DARE) task that requires an individual to recognize one of six emotions (i.e., anger, disgust, fear, happiness, sadness, and surprise) while observing a slowly transitioning face video. We observed differences in response time and eye movements, but not in the recognition accuracy. Based on these observations, we proposed a machine learning method to distinguish between individuals with ASD and typically developing (TD) controls. The proposed method classifies eye fixations based on a comprehensive set of features that integrate task performance, gaze information, and face features extracted using a deep neural network. It achieved an 86% classification accuracy that is comparable with the standardized diagnostic scales, with advantages of efficiency and objectiveness. Feature visualization and interpretations were further carried out to reveal distinguishing features between the two subject groups and to understand the social and attentional deficits in ASD.

Original languageEnglish (US)
Title of host publication2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages6063-6068
Number of pages6
ISBN (Electronic)9781538613115
DOIs
StatePublished - Jul 2019
Event41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2019 - Berlin, Germany
Duration: Jul 23 2019Jul 27 2019

Publication series

NameProceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
ISSN (Print)1557-170X

Conference

Conference41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2019
Country/TerritoryGermany
CityBerlin
Period7/23/197/27/19

Bibliographical note

Publisher Copyright:
© 2019 IEEE.

Fingerprint

Dive into the research topics of 'Classifying Individuals with ASD Through Facial Emotion Recognition and Eye-Tracking'. Together they form a unique fingerprint.

Cite this