Optimal Dynamic Proactive Caching Via Reinforcement Learning

Alireza Sadeghi, Fatemeh Sheikholeslami, Georgios B Giannakis

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

Storage of popular reusable data at the edge of a heterogeneous wireless cellular network (HetNet) offers the premise of shifting the load on low-rate, unreliable backhaul links during peak traffic hours to off-peak periods. In order to intelligently capitalize on the limited available caching capacity, a content-agnostic small base station (SB) needs to proactively learn what and when to cache. An important challenge in a realistic network scenario is the spatio-temporal dynamics, inherent to the unknown content popularity profiles. To cope with such dynamics, local and global Markov processes are exploited to model user demands, whose structure and transition probabilities are assumed unknown. A reinforcement learning framework is put forth, through which a cache control unit (CCU) at the SB can continuously learn, track, and possibly adapt to the underlying dynamics of user demands. A Q-learning algorithm is developed to solve the proposed reinforcement learning task, unraveling the optimal caching policy in an online fashion. Simulated tests demonstrate the effectiveness of the proposed proactive caching scheme under spatio-temporal dynamic demands.

Original languageEnglish (US)
Title of host publication2018 IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Print)9781538635124
DOIs
StatePublished - Aug 24 2018
Event19th IEEE International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2018 - Kalamata, Greece
Duration: Jun 25 2018Jun 28 2018

Publication series

NameIEEE Workshop on Signal Processing Advances in Wireless Communications, SPAWC
Volume2018-June

Other

Other19th IEEE International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2018
CountryGreece
CityKalamata
Period6/25/186/28/18

Bibliographical note

Funding Information:
This work was supported by USA NSF grants 1423316, 1508993, 1514056, 1711471.

Keywords

  • Proactive caching
  • dynamic popularity profile
  • dynamic user demand
  • reinforcement learning

Fingerprint Dive into the research topics of 'Optimal Dynamic Proactive Caching Via Reinforcement Learning'. Together they form a unique fingerprint.

Cite this