This paper presents a method to predict the future movements (location and gaze direction) of basketball players as a whole from their first person videos. The predicted behaviors reflect an individual physical space that affords to take the next actions while conforming to social behaviors by engaging to joint attention. Our key innovation is to use the 3D reconstruction of multiple first person cameras to automatically annotate each other's visual semantics of social configurations. We leverage two learning signals uniquely embedded in first person videos. Individually, a first person video records the visual semantics of a spatial and social layout around a person that allows associating with past similar situations. Collectively, first person videos follow joint attention that can link the individuals to a group. We learn the egocentric visual semantics of group movements using a Siamese neural network to retrieve future trajectories. We consolidate the retrieved trajectories from all players by maximizing a measure of social compatibility-the gaze alignment towards joint attention predicted by their social formation, where the dynamics of joint attention is learned by a longterm recurrent convolutional network. This allows us to characterize which social configuration is more plausible and predict future group trajectories.
|Original language||English (US)|
|Title of host publication||Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017|
|Publisher||Institute of Electrical and Electronics Engineers Inc.|
|Number of pages||10|
|State||Published - Nov 6 2017|
|Event||30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 - Honolulu, United States|
Duration: Jul 21 2017 → Jul 26 2017
|Name||Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017|
|Other||30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017|
|Period||7/21/17 → 7/26/17|
Bibliographical noteFunding Information:
This work is partially supported by the National Science Foundation (IIS 1651389) and Facebook/Oculus gift.