We consider the problem of building accurate three dimensional (3D)reconstructions of orchard rows. This problem arises in many applications including yield mapping and measuring traits (e.g. trunk diameters)for phenotyping. While 3D reconstructions of side views can be obtained using standard methods, merging the two side-views is difficult due to the lack of overlap between the two partial reconstructions. We present a novel method that utilizes global features to constrain the solution. Specifically, we use information from the silhouettes and the ground plane for alignment. The method is evaluated using multiple simulated and real datasets. For additional information and demonstration of experimental results please see https://www.youtube.com/watch?v=6mGMF2gFv4M.