Theory and experiments in vision-based grasping

Flexible operation of a robotic agent requires interaction with an uncalibrated or partially calibrated environment through the use of sensing. Much of the recent work in robotics and computer vision has concentrated upon the active observation of dynamic targets by the robotic agent. This paper focuses on autonomous interaction with moving targets in the environment. In particular, we propose a system that performs autonomous grasping of a moving target in an uncalibrated environment. The proposed system is derived using the Controlled Active Vision framework and provides the flexibility to robustly interact with the environment in the presence of uncertainty. The proposed work is experimentally verified using the Minnesota Robotic Visual Tracker (MRVT) to select targets of interest, to derive estimates of unknown environmental parameters, and to supply a control vector based upon these estimates to guide the manipulator in both the tracking and the grasping of a target.