A nonintrusive system for behavioral analysis of children using multiple RGB+depth sensors

In developmental disorders such as autism and schizophrenia, observing behavioral precursors in very early childhood can allow for early intervention and can improve patient outcomes. While such precursors open the possibility of broad and large-scale screening, until now they have been identified only through experts' painstaking examinations and their manual annotations of limited, unprocessed video footage. Here we introduce a system to automate and assist in such procedures. Employing multiple inexpensive real-time rgb+depth (rgb+d) sensors recording from multiple viewpoints, our non-invasive systemnow installed at the Shirley G. Moore Lab School, a research preschool-is being developed to monitor and reconstruct the play and interactions of preschoolers. The system's role is to help in assessing the growing volumes of its on-site recordings and to provide the data needed to uncover additional neuromotor behavioral markers via techniques such as data mining.