Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Aggregated hold out for sparse linear regression with a robust loss function

Guillaume Maillard 1 
1 CELESTE - Statistique mathématique et apprentissage
Inria Saclay - Ile de France, LMO - Laboratoire de Mathématiques d'Orsay
Abstract : Sparse linear regression methods generally have a free hyperparameter which controls the amount of sparsity, and is subject to a bias-variance tradeoff. This article considers the use of Aggregated hold-out to aggregate over values of this hyperparameter, in the context of linear regression with the Huber loss function. Aggregated hold-out (Agghoo) is a procedure which averages estimators selected by hold-out (cross-validation with a single split). In the theoretical part of the article, it is proved that Agghoo satisfies a non-asymptotic oracle inequality when it is applied to sparse estimators which are parametrized by their zero-norm. In particular , this includes a variant of the Lasso introduced by Zou, Hastié and Tibshirani. Simulations are used to compare Agghoo with cross-validation. They show that Agghoo performs better than CV when the intrinsic dimension is high and when there are confounders correlated with the predictive covariates.
Document type :
Preprints, Working Papers, ...
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-02485694
Contributor : Guillaume Maillard Connect in order to contact the contributor
Submitted on : Tuesday, May 4, 2021 - 3:56:11 PM
Last modification on : Wednesday, April 20, 2022 - 3:44:13 AM

File

EJS2002-027R1A0.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02485694, version 2
  • ARXIV : 2002.11553

Citation

Guillaume Maillard. Aggregated hold out for sparse linear regression with a robust loss function. 2021. ⟨hal-02485694v2⟩

Share

Metrics

Record views

55

Files downloads

43