Multi-armed bandit problem is an important optimization game that requires an explorationexploitation tradeoff to achieve optimal total reward. Motivated from industrial applications such as online advertising and clinical research, we consider a setting where the rewards of bandit machines are associated with covariates, and the accurate estimation of the corresponding mean reward functions plays an important role in the performance of allocation rules. Under a flexible problem setup, we establish asymptotic strong consistency and perform a finite-time regret analysis for a sequential randomized allocation strategy based on kernel estimation. In addition, since many nonparametric and parametric methods in supervised learning may be applied to estimating the mean reward functions but guidance on how to choose among them is generally unavailable, we propose a model combining allocation strategy for adaptive performance. Simulations and a real data evaluation are conducted to illustrate the performance of the proposed allocation strategy.
|Original language||English (US)|
|Journal||Journal of Machine Learning Research|
|State||Published - Oct 1 2016|
Bibliographical notePublisher Copyright:
© 2016 Wei Qian and Yuhong Yang.
- Contextual bandit problem
- Exploration-exploitation tradeoff
- Nonparametric regression
- Regret bound
- Upper confidence bound