Sharpness, Restart and Acceleration

Vincent Roulet (1, 2) , Alexandre d'Aspremont (1, 2)

1 SIERRA - Statistical Machine Learning and Parsimony
2 DI-ENS - Département d'informatique - ENS Paris

The Łojasievicz inequality shows that sharpness bounds on the minimum of convex optimization problems hold almost generically. Here, we show that sharpness directly controls the performance of restart schemes. The constants quantifying sharpness are of course unobservable, but we show that optimal restart strategies are fairly robust, and searching for the best scheme only increases the complexity by a logarithmic factor compared to the optimal bound. Overall then, restart schemes generically accelerate accelerated methods.

Mots clés: Acceleration; Restart; Lojasiewicz inequality

Domaines: Optimisation et contrôle [math.OC]

Vincent Roulet, Alexandre d'Aspremont. Sharpness, Restart and Acceleration. 2017. ⟨hal-01474362⟩