CONVERGENCE RATES OF INERTIAL FORWARD-BACKWARD ALGORITHMS

Abstract : In a Hilbert space $\mathcal H$, given $(\alpha_k)$ a general sequence of nonnegative numbers, we analyze the convergence properties of the inertial forward-backward algorithm $$(IFB) \quad\left\{ \begin{array}{l} y_k=x_k+\alpha_k(x_k-x_{k-1})\\ x_{k+1}=\mbox{prox}_{s\Psi}(y_k-s\nabla \Phi(y_k)), \end{array} \right.$$ where $\Psi: \mathcal H \to \mathbb R \cup \lbrace + \infty \rbrace $ is a proper lower-semicontinuous convex function, and $\Phi: \mathcal H \to \mathbb R$ is a differentiable convex function, whose gradient is Lipschitz continuous. Various options for the sequence $(\alpha_k)$ are considered in the literature. Among them, the Nesterov choice leads to the FISTA algorithm, and accelerates convergence from $\mathcal O(1/k)$ to $\mathcal O(1/k^2)$ for the values. Several variants are used to guarantee the convergence of the iterates, or also to improve the rate of convergence for the values. For the design of fast optimization methods, the tuning of the sequence $(\alpha_k)$ is a subtle issue, which we deal with in this paper in general. We show that the convergence rate of the algorithm can be obtained simply by analyzing the sequence of positive real numbers $(\alpha_k)$. In addition to the case $\alpha_k= 1 -\frac{\alpha}{k} $ with $\alpha\geq 3$, our results apply equally well to $\alpha_k = 1- \frac{\alpha}{k^r}$, with an exponent $r\in ]0,1[$, and to Polyak's heavy ball method. Thus, we unify most of the existing results based on the accelerated gradient method of Nesterov. In the process, we improve some of them, and discover new ones.
Type de document :
Pré-publication, Document de travail
2018
Liste complète des métadonnées

Littérature citée [37 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-01453170
Contributeur : Alexandre Cabot <>
Soumis le : samedi 7 avril 2018 - 14:55:55
Dernière modification le : vendredi 8 juin 2018 - 14:50:24

Fichier

AC-SIOPT-Final R2, Jan. 7, 201...
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01453170, version 3

Collections

Citation

Hedy Attouch, Alexandre Cabot. CONVERGENCE RATES OF INERTIAL FORWARD-BACKWARD ALGORITHMS . 2018. 〈hal-01453170v3〉

Partager

Métriques

Consultations de la notice

63

Téléchargements de fichiers

101