ON QUASI-NEWTON FORWARD-BACKWARD SPLITTING: PROXIMAL CALCULUS AND CONVERGENCE

Stephen Becker (1) , Jalal M. Fadili (2, 3, 4, 5) , Peter Ochs (6)

1 University of Colorado - Department of Applied Mathematics
2 Equipe Image - Laboratoire GREYC - UMR6072
3 NU - Normandie Université
4 ENSICAEN - École Nationale Supérieure d'Ingénieurs de Caen
5 GREYC - Groupe de Recherche en Informatique, Image et Instrumentation de Caen
6 Saarland University [Saarbrücken]

We introduce a framework for quasi-Newton forward--backward splitting algorithms (proximal quasi-Newton methods) with a metric induced by diagonal $\pm$ rank-$r$ symmetric positive definite matrices. This special type of metric allows for a highly efficient evaluation of the proximal mapping. The key to this efficiency is a general proximal calculus in the new metric. By using duality, formulas are derived that relate the proximal mapping in a rank-$r$ modified metric to the original metric. We also describe efficient implementations of the proximity calculation for a large class of functions; the implementations exploit the piece-wise linear nature of the dual problem. Then, we apply these results to acceleration of composite convex minimization problems, which leads to elegant quasi-Newton methods for which we prove convergence. The algorithm is tested on several numerical examples and compared to a comprehensive list of alternatives in the literature. Our quasi-Newton splitting algorithm with the prescribed metric compares favorably against state-of-the-art. The algorithm has extensive applications including signal processing, sparse recovery, machine learning and classification to name a few.

Domaines: Traitement du signal et de l'image [eess.SP]; Théorie de l'information [cs.IT]; Théorie [stat.TH]; Machine Learning [stat.ML]; Analyse numérique [math.NA]; Analyse fonctionnelle [math.FA]; Statistiques [math.ST]; Optimisation et contrôle [math.OC]

Stephen Becker, Jalal M. Fadili, Peter Ochs. ON QUASI-NEWTON FORWARD-BACKWARD SPLITTING: PROXIMAL CALCULUS AND CONVERGENCE. SIAM Journal on Optimization, In press. ⟨hal-02268169⟩