Adaptive Robotic Visual Tracking: Theory and Experiments

Nikolaos P. Papanikolopoulos, Pradeep K. Khosla

Research output: Contribution to journalArticlepeer-review

209 Scopus citations


This paper addresses the use of a vision sensor in the feedback loop within the Controlled Active Vision frame-, work. Using this framework, algorithms are proposed for the solution of the robotic (eye-in-hand configuration) visual tracking and servoing problem. We state the problem of visual tracking as a problem of combining control with computer vision. We use the sum-of-squared differences (SSD) optical flow for the computation of the vector of discrete displacements. The measurements can be derived either from a single big window or from multiple small windows. These displacements are fed to an adaptive controller (selftuning regulator) that creates commands for a robot control system. The whole algorithm is based on the Online estimation of the relative distance of the target with respect to the camera. An important contribution of this work is that it requires only partial knowledge of the relative distance of the target with respect to the camera. This fact obviates the need for off-line calibration of the eye-in-hand robotic system. We have implemented, both in simulation and in experiments, three different adaptive control schemes, and the results are presented in this paper. The computational complexity and the experimental results demonstrate that the proposed algorithms can be implemented in real time.

Original languageEnglish (US)
Pages (from-to)429-445
Number of pages17
JournalIEEE Transactions on Automatic Control
Issue number3
StatePublished - Mar 1993

Bibliographical note

Funding Information:
Manuscript received July 30, 1991; revised May 15, 1992. Paper recommended by Past Associate Editor, B. K. Ghosh. This work was supported by the Defense Advanced Research Projects Agency through ARPA Order DAAA-21-89C-0001.T he views and conclusions contained herein are those of the authors, and should not be interpreted as representing the official policies, either expressed or implied, of the funding agencies. N. P. Papanikolopoulos is with the Department of Computer Science, University of Minnesota, Minneapolis, MN 55455. P. K. Khosla is with the Department of Electrical and Computer Engineering, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213. IEEE Log Number 9207144.


Dive into the research topics of 'Adaptive Robotic Visual Tracking: Theory and Experiments'. Together they form a unique fingerprint.

Cite this