Hider-Finder-Combiner: An Adversarial Architecture for General Speech Signal Modification

Jacob J Webber (1) , Olivier Perrotin (2) , Simon King (1)

1 CSTR - The Centre for Speech Technology Research [Edinburgh]
2 GIPSA-CRISSP - GIPSA - Cognitive Robotics, Interactive Systems, & Speech Processing

We introduce a prototype system for modifying an arbitrary parameter of a speech signal. Unlike signal processing approaches that require dedicated methods for different parameters, our system can-in principle-modify any control parameter that the signal can be annotated with. Our system comprises three neural networks. The 'hider' removes all information related to the control parameter, outputting a hidden embedding. The 'finder' is an adversary used to train the 'hider', attempting to detect the value of the control parameter from the hidden embedding. The 'combiner' network recombines the hidden embedding with a desired new value of the control parameter. The input and output to the system are mel-spectrograms and we employ a neural vocoder to generate the output speech waveform. As a proof of concept, we use F0 as the control parameter. The system was evaluated in terms of control parameter accuracy and naturalness against a high quality signal processing method of F0 modification that also works in the spectrogram domain. We also show that, with modifications only to training data, the system is capable of modifying the 1 st and 2 nd vocal tract for-mants, showing progress towards universal signal modification.

Mots clés: speech synthesis; adversarial networks; speech modification

Domaines: Traitement du signal et de l'image [eess.SP]; Interface homme-machine [cs.HC]

Jacob J Webber, Olivier Perrotin, Simon King. Hider-Finder-Combiner: An Adversarial Architecture for General Speech Signal Modification. Interspeech 2020 - 21st Annual Conference of the International Speech Communication Association, Oct 2020, Shanghai (Virtual Conf), China. ⟨10.21437/interspeech.2020-2558⟩. ⟨hal-02987956⟩