Dynamic Screening: Accelerating First-Order Algorithms for the Lasso and Group-Lasso

Antoine Bonnefoy 1 Valentin Emiya 1 Liva Ralaivola 1 Rémi Gribonval 2
1 QARMA - éQuipe AppRentissage et MultimediA [Marseille]
LIF - Laboratoire d'informatique Fondamentale de Marseille
2 PANAMA - Parcimonie et Nouveaux Algorithmes pour le Signal et la Modélisation Audio
Inria Rennes – Bretagne Atlantique , IRISA-D5 - SIGNAUX ET IMAGES NUMÉRIQUES, ROBOTIQUE
Abstract : Recent computational strategies based on screening tests have been proposed to accelerate algorithms addressing penalized sparse regression problems such as the Lasso. Such approaches build upon the idea that it is worth dedicating some small computational effort to locate inactive atoms and remove them from the dictionary in a preprocessing stage so that the regression algorithm working with a smaller dictionary will then converge faster to the solution of the initial problem. We believe that there is an even more efficient way to screen the dictionary and obtain a greater acceleration: inside each iteration of the regression algorithm, one may take advantage of the algorithm computations to obtain a new screening test for free with increasing screening effects along the iterations. The dictionary is henceforth dynamically screened instead of being screened statically, once and for all, before the first iteration. We formalize this dynamic screening principle in a general algorithmic scheme and apply it by embedding inside a number of first-order algorithms adapted existing screening tests to solve the Lasso or new screening tests to solve the Group-Lasso. Computational gains are assessed in a large set of experiments on synthetic data as well as real-world sounds and images. They show both the screening efficiency and the gain in terms running times.
Complete list of metadatas

Cited literature [29 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01084986
Contributor : Antoine Bonnefoy <>
Submitted on : Tuesday, November 25, 2014 - 12:04:19 PM
Last modification on : Monday, March 4, 2019 - 2:04:22 PM
Long-term archiving on : Thursday, February 26, 2015 - 10:05:12 AM

Files

DynScreen_Long_singlecol.pdf
Files produced by the author(s)

Identifiers

Citation

Antoine Bonnefoy, Valentin Emiya, Liva Ralaivola, Rémi Gribonval. Dynamic Screening: Accelerating First-Order Algorithms for the Lasso and Group-Lasso. IEEE Transactions on Signal Processing, Institute of Electrical and Electronics Engineers, 2015, 63 (19), pp.20. ⟨10.1109/TSP.2015.2447503⟩. ⟨hal-01084986⟩

Share

Metrics

Record views

1608

Files downloads

591