Synchronization of Video Sequences Through 3D Trajectory Reconstruction

Xue Wang, Jian Bo Shi, Hyun Soo Park, Qing Wang

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


We present an algorithm for synchronization of an arbitrary number of videos captured by cameras indepen- dently moving in a dynamic 3D scene. Assuming the 3D spatial poses of the cameras are known for each frame, we first reconstruct the 3D trajectory of a moving point using the trajectory basis-based method. The trajectory coefficients are computed for each sequence separately. Point correspondences across sequences are not required, or even it is possible to track different points in different sequences, only if every 3D point tracked in the second sequence is a linear combination of subsets of the 3D points tracked in the first sequence. Then we propose use a robust rank constraint of the coefficient matrices to measure the spatio-temporal alignment quality for every feasible pair of video fragments. Finally, the optimal temporal mapping is found using a graph-based approach. Our algorithm can use both short and long feature trajectories, and it is robust to mild outliers. We verify the robustness and performance of the proposed approach on synthetic data as well as on challenging real video sequences.

Original languageEnglish (US)
Pages (from-to)1759-1772
Number of pages14
JournalZidonghua Xuebao/Acta Automatica Sinica
Issue number10
StatePublished - Oct 1 2017

Bibliographical note

Funding Information:
Manuscript received August 10, 2016; accepted March 2, 2017 国家自然科学基金 (61531014) 资助 Supported by National Natural Science Foundation of China (61531014) 本文责任编委 黄庆明 Recommended by Associate Editor HUANG Qing-Ming 1. 西北工业大学计算机学院 西安 710072 中国 2. 宾夕法尼亚大学 工程与应用科学学院 费城 19104 美国 1. School of Computer Science, Northwestern Polytechinical University, Xi′an 710072, China 2. School of Engineering and Applied Science, University of Pennsylvania, Philadelphia PA 19104, USA

Publisher Copyright:
Copyright © 2017 Acta Automatica Sinica. All rights reserved.


  • Independently-moving cameras
  • Non-rigid structure from motion
  • Rank constraint
  • Trajectory basis
  • Video synchronization


Dive into the research topics of 'Synchronization of Video Sequences Through 3D Trajectory Reconstruction'. Together they form a unique fingerprint.

Cite this