Studying convergence of gradient algorithms via optimal experimental design theory

Rebecca Haycroft (1) , Luc Pronzato (2) , Henry P. Wynn (3) , Anatoly A. Zhigljavsky (1)

1 School of Mathematics [Cardiff]
2 Laboratoire d'Informatique, Signaux, et Systèmes de Sophia-Antipolis (I3S) / Equipe SYSTEMES
3 LSE - Department of Statistics - London School of Economics

We study the family of gradient algorithms for solving quadratic optimization problems, where the step-length gamma_k is chosen according to a particular procedure. In order to carry out the study, we re-write the algorithms in a normalized form and make a connection with the theory of optimum experimental design. We provide the results of a numerical study which shows that some of the proposed algorithms are extremely efficient.

Mots clés: Gradient algorithms; steepest descent algorithm; rate of convergence; design of experiments; optimality criteria

Domaines: Statistiques [math.ST]

Rebecca Haycroft, Luc Pronzato, Henry P. Wynn, Anatoly A. Zhigljavsky. Studying convergence of gradient algorithms via optimal experimental design theory. Luc Pronzato, Anatoly Zhigljavsky. Optimal Design and Related Areas in Optimization and Statistics, Springer, pp.13-37, 2009, Springer Optimization and its Applications, ⟨10.1007/978-0-387-79936-0_2⟩. ⟨hal-00358757⟩