Spatial Action Maps Augmented with Visit Frequency Maps for Exploration Tasks

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Reinforcement learning has been widely applied in exploration, navigation, manipulation, and other fields. Most of the relevant techniques generate kinematic commands (e.g., move, stop, turn) for agents based on the current state information. However, recent dense action representations based research, such as spatial action maps, pointing way-points to the agent in the same domain as its observation of the state shows great promise in mobile manipulation tasks. Inspired by that, we make the first step towards using a spatial action maps based method to effectively explore novel environmental spaces. To reduce the chance of redundant exploration, the visit frequency map (VFM) and its corresponding reward function are introduced to direct the agent to actively search previously unexplored areas. In the experimental section, our work was compared to the same method without VFM and the method based on traditional steering commands with the same input data in various environments. The results show conclusively that our method is more efficient than other methods. The project page is: https://github.com/zxwang96/sam-exploration

Original languageEnglish (US)
Title of host publicationIEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3175-3181
Number of pages7
ISBN (Electronic)9781665417143
DOIs
StatePublished - 2021
Event2021 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2021 - Prague, Czech Republic
Duration: Sep 27 2021Oct 1 2021

Publication series

NameIEEE International Conference on Intelligent Robots and Systems
ISSN (Print)2153-0858
ISSN (Electronic)2153-0866

Conference

Conference2021 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2021
Country/TerritoryCzech Republic
CityPrague
Period9/27/2110/1/21

Bibliographical note

Funding Information:
This material is based upon work partially supported by the Minnesota Robotics Institute and NSF through grants #CNS-1439728, #CNS-1531330, #CNS-1544887, and #CNS-1939033. USDA/NIFA has also supported this work through grant 2020-67021-30755.

Publisher Copyright:
© 2021 IEEE.

Fingerprint

Dive into the research topics of 'Spatial Action Maps Augmented with Visit Frequency Maps for Exploration Tasks'. Together they form a unique fingerprint.

Cite this