Generalized Robust Shrinkage Estimator and Its Application to STAP Detection Problem

Frédéric Pascal (1) , Yacine Chitour (2, 1) , Yihui Quek (3)

1 SONDRA - Sondra, CentraleSupélec, Université Paris-Saclay (COmUE)
2 L2S - Laboratoire des signaux et systèmes
3 MIT - Massachusetts Institute of Technology

Recently, in the context of covariance matrix estimation, in order to improve as well as to regularize the performance of the Tyler's estimator [1] also called the Fixed-Point Estimator (FPE) [2], a “shrinkage” fixed-point estimator has been originally introduced in [3]. First, this work extends the results of [4], [5] by giving the general solution of the “shrinkage” fixed-point algorithm. Secondly, by analyzing this solution, called the generalized robust shrinkage estimator, we prove that this solution converges to a unique solution when the shrinkage parameter (losing factor) tends to 0. This solution is exactly the FPE with the trace of its inverse equal to the dimension of the problem. This general result allows one to give another interpretation of the FPE and more generally, on the Maximum Likelihood approach for covariance matrix estimation when constraints are added. Then, some simulations illustrate our theoretical results as well as the way to choose an optimal shrinkage factor. Finally, this work is applied to a Space-Time Adaptive Processing (STAP) detection problem on real STAP data.

Mots clés: robust shrinkage estimation; Tyler's Estimator

Domaines: Traitement du signal et de l'image [eess.SP]

Frédéric Pascal, Yacine Chitour, Yihui Quek. Generalized Robust Shrinkage Estimator and Its Application to STAP Detection Problem. IEEE Transactions on Signal Processing, 2014, 62 (21), pp.5640 - 5651. ⟨10.1109/TSP.2014.2355779⟩. ⟨hal-01104004⟩