Average Performance Analysis of the Stochastic Gradient Method for Online PCA

Stephane Chretien (1, 2, 3) , Christophe Guyeux (4) , Zhen-Wai Olivier Ho

1 LMB - Laboratoire de Mathématiques de Besançon (UMR 6623)
2 ERIC - Entrepôts, Représentation et Ingénierie des Connaissances
3 NPL - National Physical Laboratory [Teddington]
4 FEMTO-ST - Franche-Comté Électronique Mécanique, Thermique et Optique - Sciences et Technologies (UMR 6174)

This paper studies the complexity of the stochastic gradient algorithm for PCA when the data are observed in a streaming setting. We also propose an online approach for selecting the learning rate. Simulation experiments confirm the practical relevance of the plain stochastic gradient approach and that drastic improvements can be achieved by learning the learning rate.

Domaines: Machine Learning [stat.ML]

Stephane Chretien, Christophe Guyeux, Zhen-Wai Olivier Ho. Average Performance Analysis of the Stochastic Gradient Method for Online PCA. Machine Learning, Optimization, and Data Science. LOD 2018, vol 11331., Springer, 2019, Lecture Notes in Computer Science, ⟨10.1007/978-3-030-13709-0_19⟩. ⟨hal-02515921⟩