Conditional Gradient Algorithms for Norm-Regularized Smooth Convex Optimization

Zaid Harchaoui 1 Anatoli Juditsky 2 Arkadii S. Nemirovski 3
1 LEAR - Learning and recognition in vision
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
2 SAM - Statistique Apprentissage Machine
LJK - Laboratoire Jean Kuntzmann
Abstract : Motivated by some applications in signal processing and machine learning, we consider two convex optimization problems where, given a cone K, a norm ∥⋅∥ and a smooth convex function f, we want either 1) to minimize the norm over the intersection of the cone and a level set of f, or 2) to minimize over the cone the sum of f and a multiple of the norm. We focus on the case where (a) the dimension of the problem is too large to allow for interior point algorithms, (b) ∥⋅∥ is "too complicated" to allow for computationally cheap Bregman projections required in the first-order proximal gradient algorithms. On the other hand, we assume that {it is relatively easy to minimize linear forms over the intersection of K and the unit ∥⋅∥-ball}. Motivating examples are given by the nuclear norm with K being the entire space of matrices, or the positive semidefinite cone in the space of symmetric matrices, and the Total Variation norm on the space of 2D images. We discuss versions of the Conditional Gradient algorithm capable to handle our problems of interest, provide the related theoretical efficiency estimates and outline some applications.
Type de document :
Article dans une revue
Mathematical Programming, Series A, Springer, 2015, 152 (1), pp.75-112. <10.1007/s10107-014-0778-9>
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-00978368
Contributeur : Anatoli Juditsky <>
Soumis le : lundi 14 avril 2014 - 10:18:48
Dernière modification le : mercredi 23 novembre 2016 - 01:04:29

Identifiants

Citation

Zaid Harchaoui, Anatoli Juditsky, Arkadii S. Nemirovski. Conditional Gradient Algorithms for Norm-Regularized Smooth Convex Optimization. Mathematical Programming, Series A, Springer, 2015, 152 (1), pp.75-112. <10.1007/s10107-014-0778-9>. <hal-00978368>

Partager

Métriques

Consultations de la notice

569