Training deep learning based video classifiers for action recognition requires a large amount of labeled videos. The labeling process is labor-intensive and time-consuming. On the other hand, large amount of weakly-labeled images are uploaded to the Internet by users everyday. To harness the rich and highly diverse set of Web images, a scalable approach is to crawl these images to train deep learning based classifier, such as Convolutional Neural Networks (CNN). However, due to the domain shift problem, the performance of Web images trained deep classifiers tend to degrade when directly deployed to videos. One way to address this problem is to fine-tune the trained models on videos, but sufficient amount of annotated videos are still required. In this work, we propose a novel approach to transfer knowledge from image domain to video domain. The proposed method can adapt to the target domain (i.e. video data) with limited amount of training data. Our method maps the video frames into a low-dimensional feature space using the class-discriminative spatial attention map for CNNs. We design a novel Siamese EnergyNet structure to learn energy functions on the attention maps by jointly optimizing two loss functions, such that the attention map corresponding to a ground truth concept would have higher energy. We conduct extensive experiments on two challenging video recognition datasets (i.e. TVHI and UCF101), and demonstrate the efficacy of our proposed method.
|Original language||English (US)|
|Title of host publication||MM 2017 - Proceedings of the 2017 ACM Multimedia Conference|
|Publisher||Association for Computing Machinery, Inc|
|Number of pages||9|
|State||Published - Oct 23 2017|
|Event||25th ACM International Conference on Multimedia, MM 2017 - Mountain View, United States|
Duration: Oct 23 2017 → Oct 27 2017
|Name||MM 2017 - Proceedings of the 2017 ACM Multimedia Conference|
|Other||25th ACM International Conference on Multimedia, MM 2017|
|Period||10/23/17 → 10/27/17|
Bibliographical noteFunding Information:
This research is supported by the National Research Foundation, Prime Minister’s Office, Singapore under its International Research Centre in Singapore Funding Initiative.
- Action recognition
- Attention map
- Domain adaptation