Semantically-Aware Strategies for Stereo-Visual Robotic Obstacle Avoidance

Jungseok Hong, Karin de Langis, Cole Wyeth, Christopher Walaszek, Junaed Sattar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Scopus citations


Mobile robots in unstructured, mapless environments must rely on an obstacle avoidance module to navigate safely. The standard avoidance techniques estimate the locations of obstacles with respect to the robot but are unaware of the obstacles' identities. Consequently, the robot cannot take advantage of semantic information about obstacles when making decisions about how to navigate. We propose an obstacle avoidance module that combines visual instance segmentation with a depth map to classify and localize objects in the scene. The system avoids obstacles differentially, based on the identity of the objects: for example, the system is more cautious in response to unpredictable objects such as humans. The system can also navigate closer to harmless obstacles and ignore obstacles that pose no collision danger, enabling it to navigate more efficiently. We validate our approach in two simulated environments: one terrestrial and one underwater. Results indicate that our approach is feasible and can enable more efficient navigation strategies.

Original languageEnglish (US)
Title of host publication2021 IEEE International Conference on Robotics and Automation, ICRA 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages7
ISBN (Electronic)9781728190778
StatePublished - 2021
Event2021 IEEE International Conference on Robotics and Automation, ICRA 2021 - Xi'an, China
Duration: May 30 2021Jun 5 2021

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
ISSN (Print)1050-4729


Conference2021 IEEE International Conference on Robotics and Automation, ICRA 2021

Bibliographical note

Funding Information:
The authors are with the Department of Computer Science and Engineering, Minnesota Robotics Institute, University of Minnesota–Twin Cities, 100 Union St SE, Minneapolis, MN, 55455, USA. {1jungseok,2dento019,3wyeth008, 4walas013,5junaed} *This work was supported by the US National Science Foundation awards IIS-#1637875 & IIS-#1845364, the UMII-MnDRIVE Fellowship, the MnRI Seed Grant, and Nvidia GPU Grant.

Publisher Copyright:
© 2021 IEEE


Dive into the research topics of 'Semantically-Aware Strategies for Stereo-Visual Robotic Obstacle Avoidance'. Together they form a unique fingerprint.

Cite this