A note on covariance estimation in the unbiased estimator of risk framework
Résumé
The estimation of covariance matrices is an important area in multivariate statistics and arises naturally in many applications. Stein’s covariance estimator is regarded as a benchmark in the literature, as it generally yields remarkable risk reductions compared to the maximum likelihood estimator (MLE) in small sample sizes. In its original or raw form, however, Stein’s estimator becomes unbounded when two sample eigenvalues approach each other. Thus, in some settings, Stein’s raw estimator has greater risk than the MLE. This implies that Stein’s raw estimator is not uniformly better than the MLE and thus cannot always be used as an alternative. The problem of the unbounded behavior of Stein’s raw estimator is often mitigated by employing an ad hoc isotonizing algorithm which has no formal statistical basis. By leveraging Stein’s unbiased estimator of risk framework, in this paper we propose a general approach that prevents the unbounded behavior of the unbiased estimator of risk as two sample eigenvalues approach each other. We then employ this framework to obtain a proof-of-concept for constructing covariance estimators which retain the attractive properties of Stein’s estimator and are simultaneously uniformly better than the MLE.