Skip to Main content Skip to Navigation
Journal articles

Stability of feature selection in classification issues for high-dimensional correlated data

Abstract : Handling dependence or not in feature selection is still an open question in supervised classification issues where the number of covariates exceeds the number of observations. Some recent papers surprisingly show the superiority of naive Bayes approaches based on an obviously erroneous assumption of independence, whereas others recommend to infer on the dependence structure in order to decorrelate the selection statistics. In the classical linear discriminant analysis (LDA) framework, the present paper first highlights the impact of dependence in terms of instability of feature selection. A second objective is to revisit the above issue using a flexible factor modeling for the covariance. This framework introduces latent components of dependence, conditionally on which a new Bayes consistency is defined. A procedure is then proposed for the joint estimation of the expectation and variance parameters of the model. The present method is compared to recent regularized diagonal discriminant analysis approaches, assuming independence among features, and regularized LDA procedures, both in terms of classification performance and stability of feature selection. The proposed method is implemented in the R package FADA, freely available from the R repository CRAN.
Complete list of metadata

Cited literature [34 references]  Display  Hide  Download
Contributor : Chloé Friguet Connect in order to contact the contributor
Submitted on : Thursday, June 4, 2020 - 5:43:54 PM
Last modification on : Wednesday, November 3, 2021 - 6:05:44 AM
Long-term archiving on: : Thursday, December 3, 2020 - 2:20:28 PM


Stability of feature selection...
Publication funded by an institution


Distributed under a Creative Commons Attribution 4.0 International License



Emeline Perthame, Chloé Friguet, David Causeur. Stability of feature selection in classification issues for high-dimensional correlated data. Statistics and Computing, Springer Verlag (Germany), 2016, 26 (4), pp.783-796. ⟨10.1007/s11222-015-9569-2⟩. ⟨hal-01256508⟩



Record views


Files downloads