Beyond Support in Two-Stage Variable Selection

Abstract : Numerous variable selection methods rely on a two-stage procedure, where a sparsity-inducing penalty is used in the first stage to predict the support, which is then conveyed to the second stage for estimation or inference purposes. In this framework, the first stage screens variables to find a set of possibly relevant variables and the second stage operates on this set of candidate variables, to improve estimation accuracy or to assess the uncertainty associated to the selection of variables. We advocate that more information can be conveyed from the first stage to the second one: we use the magnitude of the coefficients estimated in the first stage to define an adaptive penalty that is applied at the second stage. We give two examples of procedures that can benefit from the proposed transfer of information, in estimation and inference problems respectively. Extensive simulations demonstrate that this transfer is particularly efficient when each stage operates on distinct subsamples. This separation plays a crucial role for the computation of calibrated p-values, allowing to control the False Discovery Rate. In this setup, the proposed transfer results in sensitivity gains ranging from 50% to 100% compared to state-of-the-art.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-01145426
Contributor : Jean-Michel Bécu <>
Submitted on : Friday, April 24, 2015 - 11:11:52 AM
Last modification on : Friday, July 20, 2018 - 11:13:37 AM
Long-term archiving on : Wednesday, April 19, 2017 - 4:53:10 AM

Files

main_jcgs.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01145426, version 1
  • ARXIV : 1505.07281

Citation

Jean-Michel Bécu, Yves Grandvalet, Christophe Ambroise, Cyril Dalmasso. Beyond Support in Two-Stage Variable Selection. 2015. ⟨hal-01145426⟩

Share

Metrics

Record views

598

Files downloads

130