Egocentric Spatial Memory

Mengmi Zhang, Keng Teck Ma, Shih Cheng Yen, Joo Hwee Lim, Qi Zhao, Jiashi Feng

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations


Egocentric spatial memory (ESM) defines a memory system with encoding, storing, recognizing and recalling the spatial information about the environment from an egocentric perspective. We introduce an integrated deep neural network architecture for modeling ESM. It learns to estimate the occupancy state of the world and progressively construct top-down 2D global maps from egocentric views in a spatially extended environment. During the exploration, our proposed ESM model updates belief of the global map based on local observations using a recurrent neural network. It also augments the local mapping with a novel external memory to encode and store latent representations of the visited places over longterm exploration in large environments which enables agents to perform place recognition and hence, loop closure. Our proposed ESM network contributes in the following aspects: (1) without feature engineering, our model predicts free space based on egocentric views efficiently in an end-to-end manner; (2) different from other deep learning-based mapping system, ESMN deals with continuous actions and states which is vitally important for robotic control in real applications. In the experiments, we demonstrate its accurate and robust global mapping capacities in 3D virtual mazes and realistic indoor environments by comparing with several competitive baselines.

Original languageEnglish (US)
Title of host publication2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages8
ISBN (Electronic)9781538680940
StatePublished - Dec 27 2018
Event2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018 - Madrid, Spain
Duration: Oct 1 2018Oct 5 2018

Publication series

NameIEEE International Conference on Intelligent Robots and Systems
ISSN (Print)2153-0858
ISSN (Electronic)2153-0866


Conference2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018

Bibliographical note

Publisher Copyright:
© 2018 IEEE.


Dive into the research topics of 'Egocentric Spatial Memory'. Together they form a unique fingerprint.

Cite this