Duality between subgradient and conditional gradient methods

Francis Bach (1, 2)

1 SIERRA - Statistical Machine Learning and Parsimony
2 LIENS - Laboratoire d'informatique de l'école normale supérieure

In this paper, we show the equivalence between mirror descent algorithms and algorithms generalizing the conditional gradient method. This is done through convex duality, and implies notably that for certain problems (such as the support vector machine), the primal subgradient method and the dual conditional gradient method are formally equivalent. The dual interpretation leads to a form of line search for mirror descent, as well as guarantees of convergence for primal-dual certificates.

Domaines: Apprentissage [cs.LG]; Optimisation et contrôle [math.OC]

Francis Bach. Duality between subgradient and conditional gradient methods. 2012. ⟨hal-00757696v1⟩