Reinforcement learning has been widely applied in exploration, navigation, manipulation, and other fields. Most of the relevant techniques generate kinematic commands (e.g., move, stop, turn) for agents based on the current state information. However, recent dense action representations based research, such as spatial action maps, pointing way-points to the agent in the same domain as its observation of the state shows great promise in mobile manipulation tasks. Inspired by that, we make the first step towards using a spatial action maps based method to effectively explore novel environmental spaces. To reduce the chance of redundant exploration, the visit frequency map (VFM) and its corresponding reward function are introduced to direct the agent to actively search previously unexplored areas. In the experimental section, our work was compared to the same method without VFM and the method based on traditional steering commands with the same input data in various environments. The results show conclusively that our method is more efficient than other methods. The project page is: https://github.com/zxwang96/sam-exploration
|Original language||English (US)|
|Title of host publication||IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2021|
|Publisher||Institute of Electrical and Electronics Engineers Inc.|
|Number of pages||7|
|State||Published - 2021|
|Event||2021 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2021 - Prague, Czech Republic|
Duration: Sep 27 2021 → Oct 1 2021
|Name||IEEE International Conference on Intelligent Robots and Systems|
|Conference||2021 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2021|
|Period||9/27/21 → 10/1/21|
Bibliographical noteFunding Information:
This material is based upon work partially supported by the Minnesota Robotics Institute and NSF through grants #CNS-1439728, #CNS-1531330, #CNS-1544887, and #CNS-1939033. USDA/NIFA has also supported this work through grant 2020-67021-30755.
© 2021 IEEE.