3D Trajectory Reconstruction under Perspective Projection

Hyun Soo Park, Takaaki Shiratori, Iain Matthews, Yaser Sheikh

Research output: Contribution to journalArticlepeer-review

31 Scopus citations


We present an algorithm to reconstruct the 3D trajectory of a moving point from its correspondence in a collection of temporally non-coincidental 2D perspective images, given the time of capture that produced each image and the relative camera poses at each time instant. Triangulation-based solutions do not apply, as multiple views of the point may not exist at each time instant. We represent a 3D trajectory using a linear combination of compact trajectory basis vectors, such as the discrete cosine transform basis, that have been shown to approximate object independence. We note that such basis vectors are also coordinate independent, which allows us to directly use camera poses estimated from stationary areas in the scene (in contrast to nonrigid structure from motion techniques where cameras are simultaneously estimated). This reduces the reconstruction optimization to a linear least squares problem, allowing us to robustly handle missing data that often occur due to motion blur, texture deformation, and self occlusion. We present an algorithm to determine the number of trajectory basis vectors, individually for each trajectory via a cross validation scheme and refine the solution by minimizing the geometric error. The relationship between point and camera motion can cause degeneracies to occur. We geometrically analyze the problem by studying the relationship of the camera motion, point motion, and trajectory basis vectors. We define the reconstructability of a 3D trajectory under projection, and show that the estimate approaches the ground truth when reconstructability approaches infinity. This analysis enables us to precisely characterize cases when accurate reconstruction is achievable. We present qualitative results for the reconstruction of several real-world scenes from a series of 2D projections where high reconstructability can be guaranteed, and report quantitative results on motion capture sequences.

Original languageEnglish (US)
Pages (from-to)115-135
Number of pages21
JournalInternational Journal of Computer Vision
Issue number2
StatePublished - Nov 1 2015

Bibliographical note

Funding Information:
This work was supported by NSF Grant IIS-0916272.

Publisher Copyright:
© 2015, Springer Science+Business Media New York.


  • Dynamic 3D reconstruction
  • Reconstructability
  • Trajectory space
  • Trajectory triangulation


Dive into the research topics of '3D Trajectory Reconstruction under Perspective Projection'. Together they form a unique fingerprint.

Cite this