Breaking Sticks and Ambiguities with Adaptive Skip-gram

Sergey Bartunov 1 Dmitry Kondrashkin 1 Anton Osokin 2, 3 Dmitry Vetrov 1
3 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, CNRS - Centre National de la Recherche Scientifique, Inria de Paris
Abstract : Recently proposed Skip-gram model is a powerful method for learning high-dimensional word representations that capture rich semantic relationships between words. However, Skip-gram as well as most prior work on learning word representations does not take into account word ambiguity and maintain only single representation per word. Although a number of Skip-gram modifications were proposed to overcome this limitation and learn multi-prototype word representations, they either require a known number of word meanings or learn them using greedy heuristic approaches. In this paper we propose the Adaptive Skip-gram model which is a nonparametric Bayesian extension of Skip-gram capable to automatically learn the required number of representations for all words at desired semantic resolution. We derive efficient online variational learning algorithm for the model and empirically demonstrate its efficiency on word-sense induction task.
Type de document :
Communication dans un congrès
Proceedings of the 19th International Conference on Artificial Intelligence and Statistics (AISTATS), May 2016, Cadiz, Spain. Proceedings of the 19th International Conference on Artificial Intelligence and Statistics (AISTATS), pp.130-138
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01404056
Contributeur : Anton Osokin <>
Soumis le : lundi 28 novembre 2016 - 12:01:48
Dernière modification le : jeudi 26 avril 2018 - 10:29:04

Lien texte intégral

Identifiants

  • HAL Id : hal-01404056, version 1
  • ARXIV : 1502.07257

Collections

Citation

Sergey Bartunov, Dmitry Kondrashkin, Anton Osokin, Dmitry Vetrov. Breaking Sticks and Ambiguities with Adaptive Skip-gram. Proceedings of the 19th International Conference on Artificial Intelligence and Statistics (AISTATS), May 2016, Cadiz, Spain. Proceedings of the 19th International Conference on Artificial Intelligence and Statistics (AISTATS), pp.130-138. 〈hal-01404056〉

Partager

Métriques

Consultations de la notice

175