Kernel estimation and model combination in a bandit problem with covariates

Wei Qian, Yuhong Yang

Research output: Contribution to journalArticlepeer-review

7 Scopus citations

Abstract

Multi-armed bandit problem is an important optimization game that requires an explorationexploitation tradeoff to achieve optimal total reward. Motivated from industrial applications such as online advertising and clinical research, we consider a setting where the rewards of bandit machines are associated with covariates, and the accurate estimation of the corresponding mean reward functions plays an important role in the performance of allocation rules. Under a flexible problem setup, we establish asymptotic strong consistency and perform a finite-time regret analysis for a sequential randomized allocation strategy based on kernel estimation. In addition, since many nonparametric and parametric methods in supervised learning may be applied to estimating the mean reward functions but guidance on how to choose among them is generally unavailable, we propose a model combining allocation strategy for adaptive performance. Simulations and a real data evaluation are conducted to illustrate the performance of the proposed allocation strategy.

Original languageEnglish (US)
JournalJournal of Machine Learning Research
Volume17
StatePublished - Oct 1 2016

Bibliographical note

Publisher Copyright:
© 2016 Wei Qian and Yuhong Yang.

Keywords

  • Contextual bandit problem
  • Exploration-exploitation tradeoff
  • Nonparametric regression
  • Regret bound
  • Upper confidence bound

Fingerprint

Dive into the research topics of 'Kernel estimation and model combination in a bandit problem with covariates'. Together they form a unique fingerprint.

Cite this