Integrating computer vision and control for vision-assisted robotic tasks

One of the most desirable characteristics of a robotic manipulator is its flexibility. Flexibility and adaptability can be achieved by incorporating vision and generally, sensory information in the feedback loop. Our research introduces a framework called controlled active vision for efficient integration of the vision sensor in the feedback loop. This framework was applied to the problem of robotic visual tracking and servoing and the results were very promising. Full 3-D robotic visual tracking was achieved at rates of 30 Hz. Most importantly, the tracking was successful even under the assumption of poor calibration of the hand-eye system. This paper extends this framework to other problems of sensor-based robotics such as the derivation of depth maps from controlled motion; the vision-assisted grasping; the active calibration of the system robot-camera; and the computation of the relative pose of the target from the camera. We dealt with these problems by combining adaptive control techniques with computer vision algorithms. The paper concludes with a discussion on several relative issues such as the stability and robustness of the proposed algorithms.