We present computer vision algorithms to collect yield related information in an apple orchard using images collected from a single camera. The goal of our system is to give farmers the capability to use their phones or digital cameras to record images and obtain yield related parameters. There are two challenges in this setup which necessitate novel methods: (i) It is very difficult to generate dense matches using standard image features. (ii) The constrained geometry of the setup causes existing structure from motion algorithms to fail. We present a novel piecewise incremental structure from motion technique to register and reconstruct the apples which is used for extracting count and diameter information. We validate our approach by presenting results from multiple field trials.