Affine Invariant Covariance Estimation for Heavy-Tailed Distributions

Dmitrii Ostrovskii 1, 2 Alessandro Rudi 1, 2
2 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, CNRS - Centre National de la Recherche Scientifique, Inria de Paris
Abstract : In this work we provide an estimator for the covariance matrix of a heavy-tailed random vector. We prove that the proposed estimator $\widehat{\mathbf{S}}$ admits $\textit{affine_invariant}$ bounds of the form $$ (1-\varepsilon) \mathbf{S} \preccurlyeq \widehat{\mathbf{S}} \preccurlyeq (1+\varepsilon) \mathbf{S} $$ in high probability, where $\mathbf{S}$ is the unknown covariance matrix, and $\preccurlyeq$ is the positive semidefinite order on symmetric matrices. The result only requires the existence of fourth-order moments, and allows for $\varepsilon = O(\sqrt{\kappa^4 d/n})$ where $\kappa^4$ is some measure of kurtosis of the distribution, $d$ is the dimensionality of the space, and $n$ is the sample size. More generally, we can allow for regularization with level~$\lambda$, then $\varepsilon$ depends on the degrees of freedom number which is generally smaller than $d$. The computational cost of the proposed estimator is essentially~$O(d^2 n + d^3)$, comparable to the computational cost of the sample covariance matrix in the statistically interesting regime~$n \gg d$. Its applications to eigenvalue estimation with relative error and to ridge regression with heavy-tailed random design are discussed.
Complete list of metadatas

Cited literature [30 references]  Display  Hide  Download
Contributor : Dmitrii Ostrovskii <>
Submitted on : Friday, February 8, 2019 - 3:06:54 PM
Last modification on : Thursday, March 14, 2019 - 3:40:38 PM
Long-term archiving on : Thursday, May 9, 2019 - 1:03:11 PM


Public Domain


  • HAL Id : hal-02011464, version 1
  • ARXIV : 1902.03086



Dmitrii Ostrovskii, Alessandro Rudi. Affine Invariant Covariance Estimation for Heavy-Tailed Distributions. 2019. ⟨hal-02011464⟩



Record views


Files downloads