Learning Word Importance with the Neural Bag-of-Words Model

Imran Sheikh 1 Irina Illina 1 Dominique Fohr 1 Georges Linares 2
1 MULTISPEECH - Speech Modeling for Facilitating Oral-Based Communication
Inria Nancy - Grand Est, LORIA - NLPKD - Department of Natural Language Processing & Knowledge Discovery
Abstract : The Neural Bag-of-Words (NBOW) model performs classification with an average of the input word vectors and achieves an impressive performance. While the NBOW model learns word vectors targeted for the classification task it does not explicitly model which words are important for given task. In this paper we propose an improved NBOW model with this ability to learn task specific word importance weights. The word importance weights are learned by introducing a new weighted sum composition of the word vectors. With experiments on standard topic and sentiment classification tasks, we show that (a) our proposed model learns meaningful word importance for a given task (b) our model gives best accuracies among the BOW approaches. We also show that the learned word importance weights are comparable to tf-idf based word weights when used as features in a BOWSVM classifier.
Document type :
Conference papers
Liste complète des métadonnées

Cited literature [40 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01331720
Contributor : Dominique Fohr <>
Submitted on : Thursday, October 20, 2016 - 9:52:52 AM
Last modification on : Wednesday, April 3, 2019 - 1:22:59 AM

File

repl4nlp_draft21Jun16.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01331720, version 1

Citation

Imran Sheikh, Irina Illina, Dominique Fohr, Georges Linares. Learning Word Importance with the Neural Bag-of-Words Model. ACL, Representation Learning for NLP (Repl4NLP) workshop, Aug 2016, Berlin, Germany. ⟨hal-01331720⟩

Share

Metrics

Record views

734

Files downloads

451