Generalized Conditional Gradient with Augmented Lagrangian for Composite Optimization

Antonio Silveti-Falls (1, 2, 3) , Cesare Molinari (1) , Jalal M. Fadili (1, 2, 4, 3)

1 Equipe Image - Laboratoire GREYC - UMR6072
2 NU - Normandie Université
3 GREYC - Groupe de Recherche en Informatique, Image et Instrumentation de Caen
4 ENSICAEN - École Nationale Supérieure d'Ingénieurs de Caen

In this paper we propose a splitting scheme which hybridizes generalized conditional gradient with a proximal step which we call CGALP algorithm, for minimizing the sum of closed, convex, and proper functions over a bounded subset of a Hilbert space. The minimization is subject to an affine constraint, which we address by the augmented Lagrangian approach, that allows in particular to deal with composite problems of sum of three or more functions by the usual product space technique. We allow for possibly nonsmooth functions which are simple, i.e., the associated proximal mapping is easily computable. Our analysis is carried out for a wide choice of algorithm parameters satisfying so called open loop rules. As main results, under mild conditions, we show asymptotic feasibility with respect to the affine constraint, weak convergence of the dual variable to a solution of the dual problem, and convergence of the Lagrangian values to the saddle-point optimal value. We also provide (subsequential) rates of convergence for both the feasibility gap and the Lagrangian values. Experimental results in signal processing are finally reported.

Domaines: Traitement du signal et de l'image [eess.SP]; Théorie de l'information [cs.IT]; Théorie [stat.TH]; Machine Learning [stat.ML]; Analyse numérique [math.NA]; Analyse fonctionnelle [math.FA]; Statistiques [math.ST]; Optimisation et contrôle [math.OC]

Antonio Silveti-Falls, Cesare Molinari, Jalal M. Fadili. Generalized Conditional Gradient with Augmented Lagrangian for Composite Optimization. SPARS 2019, Jul 2019, Toulouse, France. ⟨hal-02268280⟩