Audio-visual spontaneous emotion recognition

Zhihong Zeng, Yuxiao Hu, Glenn I. Roisman, Zhen Wen, Yun Fu, Thomas S. Huang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

34 Scopus citations

Abstract

Automatic multimodal recognition of spontaneous emotional expressions is a largely unexplored and challenging problem. In this paper, we explore audio-visual emotion recognition in a realistic human conversation setting-the Adult Attachment Interview (AAI). Based on the assumption that facial expression and vocal expression are at the same coarse affective states, positive and negative emotion sequences are labeled according to Facial Action Coding System. Facial texture in visual channel and prosody in audio channel are integrated in the framework of Adaboost multi-stream hidden Markov model (AdaMHMM) in which the Adaboost learning scheme is used to build component HMM fusion. Our approach is evaluated in AAI spontaneous emotion recognition experiments.

Original languageEnglish (US)
Title of host publicationArtifical Intelligence for Human Computing, ICMI 2006 and IJCAI 2007 International Workshops, Banff, Canada, November 3, 2006 and Hyderabad, India, January 6, 2007, Revised Seleced and Invited Papers
Pages72-90
Number of pages19
DOIs
StatePublished - 2007
Externally publishedYes
Event20th International Joint Conference on Artificial Intelligence, IJCAI 2007 - Workshop on Artifical Intelligence for Human Computing - Hyderabad, India
Duration: Jan 6 2007Jan 6 2007

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume4451 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other20th International Joint Conference on Artificial Intelligence, IJCAI 2007 - Workshop on Artifical Intelligence for Human Computing
CountryIndia
CityHyderabad
Period1/6/071/6/07

Keywords

  • Affect recognition
  • Affective computing
  • Emotion recognition
  • Multimodal human-computer interaction

Fingerprint Dive into the research topics of 'Audio-visual spontaneous emotion recognition'. Together they form a unique fingerprint.

Cite this