Grasping real objects using virtual images

This paper describes a fundamentally new approach to vision-based control in robotics. In particular, the problem of grasping different objects using a robot with a singe camera mounted on the end-effector (an eye-in-hand system) is considered. In the proposed approach the recognition of the object, and thereby the identification of its grasp position, the subsequent translational and rotational pose alignment of the manipulator with the object, and its movement in depth are controlled by using image morphing. We use a model-based framework where the image of each object at a graspable pose is stored in a database. Given an unknown object in the workspace, its identity is established by morphing its contours to the shapes in the database and using a quantification of the morph as a dissimilarity measure. From the morph a sequence of virtual (synthesized) images are obtained, which describe the progressive transformation of the input (both in terms of its shape and pose) to the template. These images are used as sub goals to guide the eye-in-hand robotic system to attain the desired orientation and height from which the grasp can be executed.