First Order Methods for Nonsmooth Convex Large-Scale Optimization, I: General Purpose Methods

Anatoli B. Juditsky (1) , Arkadii S. Nemirovski (2)

1 SMS - Statistique et Modélisation Stochatisque
2 ISyE - School of Industrial and Systems Engineering [Georgia Tech]

We discuss several state-of-the-art computationally cheap, as opposed to the polynomial time Interior Point algorithms, rst order methods for minimizing convex objectives over "simple" large-scale feasible sets. Our emphasis is on the general situation of a nonsmooth convex objective represented by deterministic/stochastic First Order oracle and on the methods which, under favorable circumstances, exhibit (nearly) dimension-independent convergence rate.

Domaines: Statistiques [math.ST]; Théorie [stat.TH]

Anatoli B. Juditsky, Arkadii S. Nemirovski. First Order Methods for Nonsmooth Convex Large-Scale Optimization, I: General Purpose Methods. Suvrit Sra, Sebastian Nowozin, Stephen J. Wright. Optimization for Machine Learning, MIT Press, pp.1-28, 2010. ⟨hal-00981863⟩