Storage of popular reusable data at the edge of a heterogeneous wireless cellular network (HetNet) offers the premise of shifting the load on low-rate, unreliable backhaul links during peak traffic hours to off-peak periods. In order to intelligently capitalize on the limited available caching capacity, a content-agnostic small base station (SB) needs to proactively learn what and when to cache. An important challenge in a realistic network scenario is the spatio-temporal dynamics, inherent to the unknown content popularity profiles. To cope with such dynamics, local and global Markov processes are exploited to model user demands, whose structure and transition probabilities are assumed unknown. A reinforcement learning framework is put forth, through which a cache control unit (CCU) at the SB can continuously learn, track, and possibly adapt to the underlying dynamics of user demands. A Q-learning algorithm is developed to solve the proposed reinforcement learning task, unraveling the optimal caching policy in an online fashion. Simulated tests demonstrate the effectiveness of the proposed proactive caching scheme under spatio-temporal dynamic demands.
|Original language||English (US)|
|Title of host publication||2018 IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2018|
|Publisher||Institute of Electrical and Electronics Engineers Inc.|
|State||Published - Aug 24 2018|
|Event||19th IEEE International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2018 - Kalamata, Greece|
Duration: Jun 25 2018 → Jun 28 2018
|Name||IEEE Workshop on Signal Processing Advances in Wireless Communications, SPAWC|
|Other||19th IEEE International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2018|
|Period||6/25/18 → 6/28/18|
Bibliographical noteFunding Information:
This work was supported by USA NSF grants 1423316, 1508993, 1514056, 1711471.
- Proactive caching
- dynamic popularity profile
- dynamic user demand
- reinforcement learning