Convergence of a Constrained Vector Extrapolation Scheme

Mathieu Barré (1, 2) , Adrien Taylor (1, 2) , Alexandre d'Aspremont (1, 2)

1 SIERRA - Statistical Machine Learning and Parsimony
2 PSL - Université Paris Sciences et Lettres

Among extrapolation methods, Anderson acceleration (AA) is a popular technique for speeding up convergence of iterative processes toward their limit points. AA proceeds by extrapolating a better approximation of the limit using a weighted combination of previous iterates. Whereas AA was originally developed in the context of nonlinear integral equations, or to accelerate the convergence of iterative methods for solving linear systems, it is also used to extrapolate the solution of nonlinear systems. Simple additional stabilization strategies can be used in this context to control conditioning issues. In this work, we study a constrained vector extrapolation scheme based on an offline version of AA with fixed window size, for solving nonlinear systems arising in optimization problems, where the stabilization strategy consists in bounding the magnitude of the extrapolation weights. We provide explicit convergence bounds for this method and, as a by-product, upper bounds on a constrained version of the Chebyshev problem on polynomials.

Domaines: Optimisation et contrôle [math.OC]

Mathieu Barré, Adrien Taylor, Alexandre d'Aspremont. Convergence of a Constrained Vector Extrapolation Scheme. SIAM Journal on Mathematics of Data Science, 2022, ⟨10.1137/21M1428030⟩. ⟨hal-03044520⟩