Unobtrusive measurement of subtle nonverbal behaviors with the Microsoft Kinect

Nathan Burba, Mark Bolas, David M. Krum, Evan A. Suma

Research output: Chapter in Book/Report/Conference proceedingConference contribution

38 Scopus citations


We describe two approaches for unobtrusively sensing subtle nonverbal behaviors using a consumer-level depth sensing camera. The first signal, respiratory rate, is estimated by measuring the visual expansion and contraction of the user's chest cavity during inhalation and exhalation. Additionally, we detect a specific type of fidgeting behavior, known as "leg jiggling," by measuring high-frequency vertical oscillations of the user's knees. Both of these techniques rely on the combination of skeletal tracking information with raw depth readings from the sensor to identify the cyclical patterns in jittery, low-resolution data. Such subtle nonverbal signals may be useful for informing models of users' psychological states during communication with virtual human agents, thereby improving interactions that address important societal challenges in domains including education, training, and medicine.

Original languageEnglish (US)
Title of host publicationIEEE Virtual Reality Conference 2012, VR 2012 - Proceedings
StatePublished - 2012
Externally publishedYes
Event19th IEEE Virtual Reality Conference, VR 2012 - Costa Mesa, CA, United States
Duration: Mar 4 2012Mar 8 2012

Publication series

NameProceedings - IEEE Virtual Reality


Other19th IEEE Virtual Reality Conference, VR 2012
Country/TerritoryUnited States
CityCosta Mesa, CA

Bibliographical note

Copyright 2012 Elsevier B.V., All rights reserved.


  • breathing
  • depth sensors
  • fidgeting
  • nonverbal behavior


Dive into the research topics of 'Unobtrusive measurement of subtle nonverbal behaviors with the Microsoft Kinect'. Together they form a unique fingerprint.

Cite this