Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Beyond Voice Identity Conversion: Manipulating Voice Attributes by Adversarial Learning of Structured Disentangled Representations

Laurent Benaroya 1 Nicolas Obin 1 Axel Roebel 1 
1 Analyse et synthèse sonores [Paris]
STMS - Sciences et Technologies de la Musique et du Son
Abstract : Voice conversion (VC) consists of digitally altering the voice of an individual to manipulate part of its content, primarily its identity, while maintaining the rest unchanged. Research in neural VC has accomplished considerable breakthroughs with the capacity to falsify a voice identity using a small amount of data with a highly realistic rendering. This paper goes beyond voice identity and presents a neural architecture that allows the manipulation of voice attributes (e.g., gender and age). Leveraging the latest advances on adversarial learning of structured speech representation, a novel structured neural network is proposed in which multiple auto-encoders are used to encode speech as a set of idealistically independent linguistic and extra-linguistic representations, which are learned adversariarly and can be manipulated during VC. Moreover, the proposed architecture is time-synchronized so that the original voice timing is preserved during conversion which allows lip-sync applications. Applied to voice gender conversion on the real-world VCTK dataset, our proposed architecture can learn successfully gender-independent representation and convert the voice gender with a very high efficiency and naturalness.
Document type :
Preprints, Working Papers, ...
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03569608
Contributor : Axel Roebel Connect in order to contact the contributor
Submitted on : Sunday, February 13, 2022 - 12:23:16 AM
Last modification on : Tuesday, March 15, 2022 - 3:22:34 AM

Links full text

Identifiers

  • HAL Id : hal-03569608, version 1
  • ARXIV : 2107.12346

Citation

Laurent Benaroya, Nicolas Obin, Axel Roebel. Beyond Voice Identity Conversion: Manipulating Voice Attributes by Adversarial Learning of Structured Disentangled Representations. 2021. ⟨hal-03569608⟩

Share

Metrics

Record views

12