Online optimization and regret guarantees for non-additive long-term constraints

Rodolphe Jenatton (1) , Jim Huang (2) , Dominik Csiba (3) , Cedric Archambeau (1)

1 Amazon, Berlin
2 Amazon, Seattle
3 School of Mathematics - University of Edinburgh

We consider online optimization in the 1-lookahead setting, where the objective does not decompose additively over the rounds of the online game. The resulting formulation enables us to deal with non-stationary and/or long-term constraints , which arise, for example, in online display advertising problems. We propose an on-line primal-dual algorithm for which we obtain dynamic cumulative regret guarantees. They depend on the convexity and the smoothness of the non-additive penalty, as well as terms capturing the smoothness with which the residuals of the non-stationary and long-term constraints vary over the rounds. We conduct experiments on synthetic data to illustrate the benefits of the non-additive penalty and show vanishing regret convergence on live traffic data collected by a display advertising platform in production.

Mots clés: online learning; dynamic regret; convex optimization

Domaines: Machine Learning [stat.ML]; Optimisation et contrôle [math.OC]; Statistiques [math.ST]

Rodolphe Jenatton, Jim Huang, Dominik Csiba, Cedric Archambeau. Online optimization and regret guarantees for non-additive long-term constraints. 2016. ⟨hal-01273728v2⟩