TY - GEN
T1 - MAP visibility estimation for large-scale dynamic 3D reconstruction
AU - Joo, Hanbyul
AU - Park, Hyun Soo
AU - Sheikh, Yaser
N1 - Publisher Copyright:
© 2014 IEEE.
Copyright:
Copyright 2015 Elsevier B.V., All rights reserved.
PY - 2014/9/24
Y1 - 2014/9/24
N2 - Many traditional challenges in reconstructing 3D motion, such as matching across wide baselines and handling occlusion, reduce in significance as the number of unique viewpoints increases. However, to obtain this benefit, a new challenge arises: estimating precisely which cameras observe which points at each instant in time. We present a maximum a posteriori (MAP) estimate of the time-varying visibility of the target points to reconstruct the 3D motion of an event from a large number of cameras. Our algorithm takes, as input, camera poses and image sequences, and outputs the time-varying set of the cameras in which a target patch is visibile and its reconstructed trajectory. We model visibility estimation as a MAP estimate by incorporating various cues including photometric consistency, motion consistency, and geometric consistency, in conjunction with a prior that rewards consistent visibilities in proximal cameras. An optimal estimate of visibility is obtained by finding the minimum cut of a capacitated graph over cameras. We demonstrate that our method estimates visibility with greater accuracy, and increases tracking performance producing longer trajectories, at more locations, and at higher accuracies than methods that ignore visibility or use photometric consistency alone.
AB - Many traditional challenges in reconstructing 3D motion, such as matching across wide baselines and handling occlusion, reduce in significance as the number of unique viewpoints increases. However, to obtain this benefit, a new challenge arises: estimating precisely which cameras observe which points at each instant in time. We present a maximum a posteriori (MAP) estimate of the time-varying visibility of the target points to reconstruct the 3D motion of an event from a large number of cameras. Our algorithm takes, as input, camera poses and image sequences, and outputs the time-varying set of the cameras in which a target patch is visibile and its reconstructed trajectory. We model visibility estimation as a MAP estimate by incorporating various cues including photometric consistency, motion consistency, and geometric consistency, in conjunction with a prior that rewards consistent visibilities in proximal cameras. An optimal estimate of visibility is obtained by finding the minimum cut of a capacitated graph over cameras. We demonstrate that our method estimates visibility with greater accuracy, and increases tracking performance producing longer trajectories, at more locations, and at higher accuracies than methods that ignore visibility or use photometric consistency alone.
UR - http://www.scopus.com/inward/record.url?scp=84911381635&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84911381635&partnerID=8YFLogxK
U2 - 10.1109/CVPR.2014.147
DO - 10.1109/CVPR.2014.147
M3 - Conference contribution
AN - SCOPUS:84911381635
T3 - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
SP - 1122
EP - 1129
BT - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
PB - IEEE Computer Society
T2 - 27th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014
Y2 - 23 June 2014 through 28 June 2014
ER -