Semi-supervised multichannel speech enhancement with variational autoencoders and non-negative matrix factorization

Simon Leglaive (1) , Laurent Girin (2) , Radu Horaud (1)

1 PERCEPTION - Interpretation and Modelling of Images and Videos
2 GIPSA-CRISSP - GIPSA - Cognitive Robotics, Interactive Systems, & Speech Processing

In this paper we address speaker-independent multichannel speech enhancement in unknown noisy environments. Our work is based on a well-established multichannel local Gaussian modeling framework. We propose to use a neural network for modeling the speech spectro-temporal content. The parameters of this supervised model are learned using the framework of variational autoencoders. The noisy recording environment is supposed to be unknown, so the noise spectro-temporal modeling remains unsupervised and is based on non-negative matrix factorization (NMF). We develop a Monte Carlo expectation-maximization algorithm and we experimentally show that the proposed approach outperforms its NMF-based counterpart , where speech is modeled using supervised NMF.

Mots clés: Monte Carlo expectation-maximization; Variational autoencoders; Local Gaussian modeling; Multichannel speech enhancement; Non-negative matrix factorization

Domaines: Traitement du signal et de l'image [eess.SP]; Réseau de neurones [cs.NE]

Simon Leglaive, Laurent Girin, Radu Horaud. Semi-supervised multichannel speech enhancement with variational autoencoders and non-negative matrix factorization. IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP 2019), May 2019, Brighton, United Kingdom. pp.101-105, ⟨10.1109/ICASSP.2019.8683704⟩. ⟨hal-02005102v1⟩