Non-smooth Non-convex Bregman Minimization: Unification and new Algorithms

Peter Ochs (1) , Jalal M. Fadili (2, 3, 4) , Thomas Brox (5)

1 Saarland University [Saarbrücken]
2 Equipe Image - Laboratoire GREYC - UMR6072
3 NU - Normandie Université
4 ENSICAEN - École Nationale Supérieure d'Ingénieurs de Caen
5 Computer Vision Lab [Freiburg]

We propose a unifying algorithm for non-smooth non-convex optimization. The algorithm approximates the objective function by a convex model function and finds an approximate (Bregman) proximal point of the convex model. This approximate minimizer of the model function yields a descent direction, along which the next iterate is found. Complemented with an Armijo-like line search strategy, we obtain a flexible algorithm for which we prove (subsequential) convergence to a stationary point under weak assumptions on the growth of the model function error. Special instances of the algorithm with a Euclidean distance function are, for example, Gradient Descent, Forward-Backward Splitting, ProxDescent, without the common requirement of a "Lipschitz continuous gradient". In addition, we consider a broad class of Bregman distance functions (generated by Legendre functions), replacing the Euclidean distance. The algorithm has a wide range of applications including many linear and non-linear inverse problems in signal/image processing and machine learning.

Domaines: Traitement du signal et de l'image [eess.SP]; Théorie de l'information [cs.IT]; Théorie [stat.TH]; Machine Learning [stat.ML]; Analyse numérique [math.NA]; Analyse fonctionnelle [math.FA]; Statistiques [math.ST]; Optimisation et contrôle [math.OC]

Peter Ochs, Jalal M. Fadili, Thomas Brox. Non-smooth Non-convex Bregman Minimization: Unification and new Algorithms. Journal of Optimization Theory and Applications, In press. ⟨hal-01921804⟩