What you can cram into a single \$&!#* vector: Probing sentence embeddings for linguistic properties

Abstract : Although much effort has recently been devoted to training high-quality sentence embeddings, we still have a poor understanding of what they are capturing. "Downstream" tasks, often based on sentence classification, are commonly used to evaluate the quality of sentence representations. The complexity of the tasks makes it however difficult to infer what kind of information is present in the representations. We introduce here 10 probing tasks designed to capture simple linguistic features of sentences, and we use them to study embeddings generated by three different encoders trained in eight distinct ways, uncovering intriguing properties of both encoders and training methods.
Type de document :
Communication dans un congrès
ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Jul 2018, Melbourne, Australia. Association for Computational Linguistics, 1, pp.2126-2136, 2018
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01898412
Contributeur : Loïc Barrault <>
Soumis le : jeudi 18 octobre 2018 - 13:56:16
Dernière modification le : vendredi 16 novembre 2018 - 02:20:43

Lien texte intégral

Identifiants

  • HAL Id : hal-01898412, version 1
  • ARXIV : 1805.01070

Citation

Alexis Conneau, German Kruszewski, Guillaume Lample, Loïc Barrault, Marco Baroni. What you can cram into a single \$&!#* vector: Probing sentence embeddings for linguistic properties. ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Jul 2018, Melbourne, Australia. Association for Computational Linguistics, 1, pp.2126-2136, 2018. 〈hal-01898412〉

Partager

Métriques

Consultations de la notice

24