Skip to Main content Skip to Navigation
New interface
Conference papers

Task Agnostic and Task Specific Self-Supervised Learning from Speech with LeBenchmark

Abstract : Self-Supervised Learning (SSL) has yielded remarkable improvements in many different domains including computer vision, natural language processing and speech processing by leveraging large amounts of unlabeled data. In the specific context of speech, however, and despite promising results, there exists a clear lack of standardization in the evaluation process for comprehensive comparisons of these models. This issue gets even worse with the investigation of SSL approaches for other languages than English. We present LeBenchmark, an open-source and reproducible framework for assessing SSL from French speech data. It includes a documented, large-scale and heterogeneous corpora, seven pre-trained SSL wav2vec 2.0 models shared with the community, and a clear evaluation protocol made of four downstream tasks along with their scoring scripts: automatic speech recognition, spoken language understanding, automatic speech translation and automatic emotion recognition. For the first time, SSL models are analyzed and compared on the latter domains both from a task-agnostic (i.e. frozen) and task-specific (i.e. fine-tuned w.r.t the downstream task) perspectives. We report state-of-the-art performance on most considered French tasks and provide a readable evaluation set-up for the development of future SSL models for speech processing.
Document type :
Conference papers
Complete list of metadata
Contributor : Solène Evain Connect in order to contact the contributor
Submitted on : Monday, February 7, 2022 - 4:39:28 PM
Last modification on : Wednesday, July 6, 2022 - 4:22:30 AM
Long-term archiving on: : Sunday, May 8, 2022 - 7:08:16 PM


Files produced by the author(s)


  • HAL Id : hal-03407172, version 1


Solène Evain, Manh Ha Nguyen, Hang Le, Marcely Zanon Boito, Salima Mdhaffar, et al.. Task Agnostic and Task Specific Self-Supervised Learning from Speech with LeBenchmark. Thirty-fifth Conference on Neural Information Processing Systems ( NeurIPS 2021), Dec 2021, on-line, United States. ⟨hal-03407172⟩



Record views


Files downloads