Applying the controlled active vision framework to the visual servoing problem

Our previous work has introduced the framework of Controlled Active Vision that provides a bridge between Computer Vision and Control. This paper presents some of the vision techniques that were employed in order to automatically select features, measure features' displacements, and evaluate measurements during the various applications of the Controlled Active Vision framework. We experimented with a lot of different techniques, but the most robust proved to be the Sum-of-Squared Differences (SSD) optical flow technique. In addition, several techniques for the evaluation of the measurements are presented. One important characteristic of these techniques is that they can also be used for the selection of features for tracking in conjunction with several numerical criteria that guarantee the robustness of the servoing. These techniques are important aspects of our work since they can be implemented either on-line of off-line. An extension of the SSD measure to color images is presented and the results from the application of these techniques to real images are discussed. Finally, the derivation of depth maps through the controlled motion of the hand-eye system is outlined.