An adaptive stochastic optimization algorithm for resource allocation

Xavier Fontaine (1) , Shie Mannor (2) , Vianney Perchet (3, 4)

1 CB - CB - Centre Borelli - UMR 9010
2 EE-Technion - Department of Electrical Engineering - Technion [Haïfa]
3 CREST - Centre de Recherche en Économie et Statistique
4 Criteo AI Lab

We consider the classical problem of sequential resource allocation where a decision maker must repeatedly divide a budget between several resources, each with diminishing returns. This can be recast as a specific stochastic optimization problem where the objective is to maximize the cumulative reward, or equivalently to minimize the regret. We construct an algorithm that is adaptive to the complexity of the problem, expressed in term of the regularity of the returns of the resources, measured by the exponent in the Łojasiewicz inequality (or by their universal concavity parameter). Our parameter-independent algorithm recovers the optimal rates for strongly-concave functions and the classical fast rates of multi-armed bandit (for linear reward functions). Moreover, the algorithm improves existing results on stochastic optimization in this regret minimization setting for intermediate cases.

Mots clés: Stochastic optimization; online learning; adaptive algorithms; resource allocation

Domaines: Apprentissage [cs.LG]; Optimisation et contrôle [math.OC]; Machine Learning [stat.ML]

Xavier Fontaine, Shie Mannor, Vianney Perchet. An adaptive stochastic optimization algorithm for resource allocation. The 31st International Conference on Algorithmic Learning Theory, Feb 2020, San Diego, United States. pp.1 - 45. ⟨hal-02475130⟩