Multiview supervision by registration

Yilun Zhang, Hyun Soo Park

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper presents a semi-supervised learning framework to train a keypoint detector using multiview image streams given the limited number of labeled instances (typically <4%). We leverage three self-supervisionary signals in multiview tracking to utilize the unlabeled data: (1) a keypoint in one view can be supervised by other views via epipolar geometry; (2) a keypoint detection must be consistent across time; (3) a visible keypoint in one view is likely to be visible in the adjacent view. We design a new end-to-end network that can propagate these self-supervisionary signals across the unlabeled data from the labeled data in a differentiable manner. We show that our approach outperforms existing detectors including DeepLabCut tailored to the keypoint detection of non-human species such as monkeys, dogs, and mice.

Original languageEnglish (US)
Title of host publicationProceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages409-417
Number of pages9
ISBN (Electronic)9781728165530
DOIs
StatePublished - Mar 2020
Event2020 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2020 - Snowmass Village, United States
Duration: Mar 1 2020Mar 5 2020

Publication series

NameProceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020

Conference

Conference2020 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2020
CountryUnited States
CitySnowmass Village
Period3/1/203/5/20

Fingerprint Dive into the research topics of 'Multiview supervision by registration'. Together they form a unique fingerprint.

  • Cite this

    Zhang, Y., & Park, H. S. (2020). Multiview supervision by registration. In Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020 (pp. 409-417). [9093591] (Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/WACV45572.2020.9093591