Descentwise inexact proximal algorithms for smooth optimization

Marc Fuentes (1) , Jérôme Malick (2) , Claude Lemaréchal (2)

1 Inria Nancy - Grand Est
2 BIPOP - Modelling, Simulation, Control and Optimization of Non-Smooth Dynamical Systems

The proximal method is a standard regularization approach in optimization. Practical implementations of this algorithm require (i) an algorithm to compute the proximal point, (ii) a rule to stop this algorithm, (iii) an update formula for the proximal parameter. In this work we focus on (ii), when smoothness is present - so that Newton-like methods can be used for (i): we aim at giving adequate stopping rules to reach overall efficiency of the method. Roughly speaking, usual rules consist in stopping inner iterations when the current iterate is close to the proximal point. By contrast, we use the standard paradigm of numerical optimization: the basis for our stopping test is a "sufficient" decrease of the objective function, namely a fraction of the ideal decrease. We establish convergence of the algorithm thus obtained and we illustrate it on some ill-conditioned functions. The experiments show that combining a standard smooth optimization algorithm with the proposed inexact proximal scheme improves numerical behaviour for those problems.

Mots clés: Proximal algorithm; Quasi-Newton algorithms; Convex analysis

Domaines: Optimisation et contrôle [math.OC]

Marc Fuentes, Jérôme Malick, Claude Lemaréchal. Descentwise inexact proximal algorithms for smooth optimization. Computational Optimization and Applications, 2012, 53 (3), pp.755-769. ⟨10.1007/s10589-012-9461-3⟩. ⟨hal-00628777⟩