Fast convex optimization via a third-order in time evolution equation

Hedy Attouch (1) , Zaki Chbani (2) , Hassan Riahi (2)

1 IMAG - Institut Montpelliérain Alexander Grothendieck
2 LIBMA - Laboratoire Ibn Al-Banna de Mathématiques et Applications

In a Hilbert space H, we develop fast convex optimization methods, which are based on a third order in time evolution system. The function to minimize f : H → R is convex, continuously differentiable, with argmin f = ∅, and enters the dynamic via its gradient. On the basis of Lyapunov's analysis and temporal scaling techniques, we show a convergence rate of the values of the order 1/t 3 , and obtain the convergence of the trajectories towards optimal solutions. When f is strongly convex, an exponential rate of convergence is obtained. We complete the study of the continuous dynamic by introducing a damping term induced by the Hessian of f. This allows the oscillations to be controlled and attenuated. Then, we analyze the convergence of the proximal-based algorithms obtained by temporal discretization of this system, and obtain similar convergence rates. The algorithmic results are valid for a general convex, lower semicontinuous, and proper function f : H → R ∪ {+∞}.

Domaines: Optimisation et contrôle [math.OC]

Hedy Attouch, Zaki Chbani, Hassan Riahi. Fast convex optimization via a third-order in time evolution equation. 2020. ⟨hal-02432351⟩