Skip to Main content Skip to Navigation
Conference papers

Learning Word Importance with the Neural Bag-of-Words Model

Imran Sheikh 1 Irina Illina 1 Dominique Fohr 1 Georges Linares 2
1 MULTISPEECH - Speech Modeling for Facilitating Oral-Based Communication
Inria Nancy - Grand Est, LORIA - NLPKD - Department of Natural Language Processing & Knowledge Discovery
Abstract : The Neural Bag-of-Words (NBOW) model performs classification with an average of the input word vectors and achieves an impressive performance. While the NBOW model learns word vectors targeted for the classification task it does not explicitly model which words are important for given task. In this paper we propose an improved NBOW model with this ability to learn task specific word importance weights. The word importance weights are learned by introducing a new weighted sum composition of the word vectors. With experiments on standard topic and sentiment classification tasks, we show that (a) our proposed model learns meaningful word importance for a given task (b) our model gives best accuracies among the BOW approaches. We also show that the learned word importance weights are comparable to tf-idf based word weights when used as features in a BOWSVM classifier.
Document type :
Conference papers
Complete list of metadata

Cited literature [40 references]  Display  Hide  Download
Contributor : Dominique Fohr <>
Submitted on : Thursday, October 20, 2016 - 9:52:52 AM
Last modification on : Tuesday, December 8, 2020 - 9:38:28 AM


Files produced by the author(s)


  • HAL Id : hal-01331720, version 1


Imran Sheikh, Irina Illina, Dominique Fohr, Georges Linares. Learning Word Importance with the Neural Bag-of-Words Model. ACL, Representation Learning for NLP (Repl4NLP) workshop, Aug 2016, Berlin, Germany. ⟨hal-01331720⟩



Record views


Files downloads