Audiovisual Speech Source Separation: An overview of key methodologies

Bertrand Rivet (1) , Wenwu Wang (2) , Syed Mohsen Naqvi (3) , Jonathon Chambers (4)

1 GIPSA-VIBS - GIPSA - Vision and Brain Signal Processing
2 Centre for Vision Speech and Signal Processing
3 Advanced Signal Processing Group
4 School of Electronic, Electrical and Systems Engineering

The separation of speech signals measured at multiple microphones in noisy and reverberant environments using only the audio modality has limitations because there is generally insufficient information to fully dis- criminate the different sound sources. Humans miti- gate this problem by exploiting the visual modality, which is insensitive to background noise and can provide contextual information about the audio scene. This advantage has inspired the creation of the new field of audiovisual (AV) speech source separation that targets exploiting visual modality alongside the microphone measurements in a machine. Success in this emerging field will expand the application of voice- based machine interfaces, such as Siri,theintelligentpersonal assistant on the iPhone and iPad, to much more realis- tic settings and thereby provide more natural human-machine interfaces.

Mots clés: blind source separation; speech extraction; audiovisual speech

Domaines: Traitement du signal et de l'image [eess.SP]; Traitement du signal et de l'image [eess.SP]

Bertrand Rivet, Wenwu Wang, Syed Mohsen Naqvi, Jonathon Chambers. Audiovisual Speech Source Separation: An overview of key methodologies. IEEE Signal Processing Magazine, 2014, 31 (3), pp.125-134. ⟨10.1109/MSP.2013.2296173⟩. ⟨hal-00990000⟩