Memory Segment Matching Network Based Image Geo-Localization

Jienan Chen, Yunzhi Duan, Gerald E. Sobelman, Cong Zhang

Research output: Contribution to journalArticlepeer-review

3 Scopus citations


Humans and other animals can easily perform self-localization by means of vision. However, that remains a challenging task for computer vision algorithms with traditional image matching methods. In this paper, we propose a memory segment matching network for image geo-localization that is inspired by the discovery of the place cell in the brain by using artificial intelligence. The place cell becomes active when an animal enters a particular location, where the external sensory information in the environment matches features stored in the hippocampus. In order to emulate the operation of the place cell, we employ a convolutional neural network (CNN) and a long-short term memory (LSTM) to extract the visual features of the environment. The extracted features are stored as segmented memory bounded with a location tag. A matching network is utilized to calculate the cross firing probability of the memory segment and the current input visual data. The final prediction of the location is obtained by sending the cross firing probability to an inference engine that uses a hidden Markov model (HMM). According to the simulation results, the localization accuracy reaches up to 95% for the datasets tested, which outperforms the state-of-the-art by 17% in localization detection accuracy.

Original languageEnglish (US)
Article number8735687
Pages (from-to)77448-77459
Number of pages12
JournalIEEE Access
StatePublished - 2019
Externally publishedYes

Bibliographical note

Funding Information:
This work was supported in part by the National Nature Science Foundation of China, under Grant 6197011787 and in part by State Key Lab. of ASIC and Systems, under Grant 2018GF016.

Publisher Copyright:
© 2013 IEEE.


  • Computer vision
  • artificial intelligence
  • geo-localization
  • hidden Markov model (HMM)
  • image matching
  • memory segment matching network


Dive into the research topics of 'Memory Segment Matching Network Based Image Geo-Localization'. Together they form a unique fingerprint.

Cite this