Audio style transfer - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2018

Audio style transfer

Eric Grinstein
  • Fonction : Auteur
  • PersonId : 1021935
Ngoc Q K Duong
  • Fonction : Auteur
  • PersonId : 864978
Alexey Ozerov
  • Fonction : Auteur
  • PersonId : 882775
Patrick Pérez
  • Fonction : Auteur
  • PersonId : 1022281

Résumé

"Style transfer" among images has recently emerged as a very active research topic, fuelled by the power of convolution neural networks (CNNs), and has become fast a very popular technology in social media. This paper investigates the analogous problem in the audio domain: How to transfer the style of a reference audio signal to a target audio content? We propose a flexible framework for the task, which uses a sound texture model to extract statistics characterizing the reference audio style, followed by an optimization-based audio texture synthesis to modify the target content. In contrast to mainstream optimization-based visual transfer method, the proposed process is initialized by the target content instead of random noise and the optimized loss is only about texture, not structure. These differences proved key for audio style transfer in our experiments. In order to extract features of interest, we investigate different architectures, whether pre-trained on other tasks, as done in image style transfer, or engineered based on the human auditory system. Experimental results on different types of audio signal confirm the potential of the proposed approach.
Fichier principal
Vignette du fichier
manuscript.pdf (731.05 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01626389 , version 1 (30-10-2017)
hal-01626389 , version 2 (17-05-2018)
hal-01626389 , version 3 (06-11-2018)

Identifiants

Citer

Eric Grinstein, Ngoc Q K Duong, Alexey Ozerov, Patrick Pérez. Audio style transfer. ICASSP 2018 - 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Apr 2018, Calgary, France. ⟨10.1109/ICASSP.2018.8461711⟩. ⟨hal-01626389v3⟩
831 Consultations
2227 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More