Importance Weighting Without Importance Weights: An Efficient Algorithm for Combinatorial Semi-Bandits

Gergely Neu 1, 2 Bartók Gábor 3, 4
1 SEQUEL - Sequential Learning
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189
Abstract : We propose a sample-efficient alternative for importance weighting for situations where one only has sample access to the probability distribution that generates the observations. Our new method, called Geometric Resampling (GR), is described and analyzed in the context of online combinatorial optimization under semi-bandit feedback, where a learner sequentially selects its actions from a combinatorial decision set so as to minimize its cumulative loss. In particular, we show that the well-known Follow-the-Perturbed-Leader (FPL) prediction method coupled with Geometric Resampling yields the first computationally efficient reduction from offline to online optimization in this setting. We provide a thorough theoretical analysis for the resulting algorithm, showing that its performance is on par with previous, inefficient solutions. Our main contribution is showing that, despite the relatively large variance induced by the GR procedure, our performance guarantees hold with high probability rather than only in expectation. As a side result, we also improve the best known regret bounds for FPL in online combinatorial optimization with full feedback, closing the perceived performance gap between FPL and exponential weights in this setting.
Document type :
Journal articles
Complete list of metadatas

Cited literature [28 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01380278
Contributor : Gergely Neu <>
Submitted on : Wednesday, October 12, 2016 - 5:11:54 PM
Last modification on : Thursday, June 27, 2019 - 1:36:06 PM
Long-term archiving on : Saturday, February 4, 2017 - 8:23:05 PM

File

15-091.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01380278, version 1

Citation

Gergely Neu, Bartók Gábor. Importance Weighting Without Importance Weights: An Efficient Algorithm for Combinatorial Semi-Bandits. Journal of Machine Learning Research, Microtome Publishing, 2016, 17 (154), pp.1 - 21. ⟨hal-01380278⟩

Share

Metrics

Record views

365

Files downloads

141