Abstract
Autonomous search is an essential research topic for rescue and other robotic applications. However, searching for targets efficiently is still an unsolved problem. To achieve this objective, a robot needs to simultaneously maximize environmental coverage, maximize probability of detection (PD) and minimize motion cost. The problems associated with these objectives are NP-hard. This research reformulates the three objective functions as a maximum cumulative PD problem with motion cost. Since the PD function depends on the environment, the robot needs to both learn the PD function and the cost-to-go (CTG) function. This research proposes a reinforcement learning algorithm to learn the PD and CTG functions simultaneously. Since the PD function is sparse in the Fourier domain under certain subgoal patterns, spatial Fourier sparse set is proposed to learn PD functions based on the compressed sensing technique. The learned PD and CTG functions can then be used to generate subgoals that achieve (1 - 1 / e) of the optimum due to the submodularity. Experiments conducted with this algorithm demonstrate that the robot can search for the target faster than prior learning approaches (e.g., PMAC and FSS) and the benchmark model (e.g., PD).
Original language | English (US) |
---|---|
Pages (from-to) | 329-351 |
Number of pages | 23 |
Journal | Autonomous Robots |
Volume | 42 |
Issue number | 2 |
DOIs | |
State | Published - Feb 1 2018 |
Bibliographical note
Funding Information:This research was completed thanks to the financial support from ONR Grant 1361538 and NSF CAREER CMMI 1254906. Kuo-Shih would like to thank his daughter, Chin-Chun Tseng. The hide-and-seek game they played inspires the learning concept for search problems. This is one of several papers published in Autonomous Robots comprising the Special Issue on Active Perception.
Funding Information:
Acknowledgements This research was completed thanks to the financial support from ONR Grant 1361538 and NSF CAREER CMMI 1254906. Kuo-Shih would like to thank his daughter, Chin-Chun Tseng. The hide-and-seek game they played inspires the learning concept for search problems.
Publisher Copyright:
© 2017, Springer Science+Business Media New York.
Keywords
- Compressed sensing
- Probabilistic search
- Q-learning
- Sparse learning
- Submodularity