MEC-Enabled Hierarchical Emotion Recognition and Perturbation-Aware Defense in Smart Cities

Yi Zhao, Ke Xu, Haiyang Wang, Bo Li, Meina Qiao, Haobin Shi

Research output: Contribution to journalArticlepeer-review

4 Scopus citations


With the explosive growth of Internet of Things (IoT) devices and various emerging network technologies, IoT-enabled smart cities are further refined into health smart cities. For example, IoT devices can automatically recognize emotional states through collected facial expressions, which can further serve mental health assessment, human-computer interaction, etc. On the other hand, existing facial expression recognition algorithms emphasize the application of deep neural networks (DNNs), and it is difficult for resource-constrained IoT devices to provide sufficient computing resources to optimize parameters for DNN-based structures. To solve the challenge of resource constraints, we propose the hierarchical emotion recognition system enabled by mobile edge computing (MEC). Specifically, MEC nodes provide IoT devices with short-delay and high-performance computing services, satisfying the requirements of training DNN-based algorithms. Moreover, our proposed emotion recognition system leverages a pretrained feature extraction module on the remote cloud to accelerate optimization and provides a localization module for specific tasks of IoT devices. In addition to evaluating the accuracy and efficiency, we also clarify that the DNN-based emotion recognition system exposes obvious vulnerability to perturbation. Due to the uncertainty of the environment, it is common for facial expressions collected by IoT devices to be accompanied by perturbation. To address this issue, we propose the proactive perturbation-aware defense mechanism. It has been demonstrated that the newly proposed defense mechanism can maintain state-of-the-art performance on the publicly available LIRIS-CSE dataset while defending against known and unknown perturbation. This can promote the deployment of our proposed MEC-enabled hierarchical emotion recognition system and defense mechanism in real-world scenarios.

Original languageEnglish (US)
Pages (from-to)16933-16945
Number of pages13
JournalIEEE Internet of Things Journal
Issue number23
StatePublished - Dec 1 2021

Bibliographical note

Funding Information:
This work was supported in part by the China National Funds for Distinguished Young Scientists under Grant 61825204; in part by NSFC Project under Grant 61932016; in part by Beijing Outstanding Young Scientist Program under Grant BJJWZYJH01201910003011; in part by the National Key R&D Program of China under Grant 2018YFB0803405; in part by Beijing National Research Center for Information Science and Technology (BNRist) under Grant BNR2019RC01011; and in part by PCL Future Greater- Bay Area Network Facilities for Largescale Experiments and Applications under Grant LZC0019.

Publisher Copyright:
© 2014 IEEE.


  • Emotion recognition
  • Internet of Things (IoT)
  • facial expression
  • mobile edge computing (MEC)
  • proactive perturbation-aware defense
  • robustness


Dive into the research topics of 'MEC-Enabled Hierarchical Emotion Recognition and Perturbation-Aware Defense in Smart Cities'. Together they form a unique fingerprint.

Cite this