Investigating the stability of concrete nouns in word embeddings

Bénédicte Pierrejean (1) , Ludovic Tanguy (2)

1 CLLE-ERSS - Cognition, Langues, Langage, Ergonomie
2 ERSS - Equipe de Recherche en Syntaxe et Sémantique

We know that word embeddings trained using neural-based methods (such as word2vec SGNS) are sensitive to stability problems and that across two models trained using the exact same set of parameters, the nearest neighbors of a word are likely to change. All words are not equally impacted by this internal instability and recent studies have investigated features influencing the stability of word embeddings. This stability can be seen as a clue for the reliability of the semantic representation of a word. In this work, we investigate the influence of the degree of concreteness of nouns on the stability of their semantic representation. We show that for English generic corpora, abstract words are more affected by stability problems than concrete words. We also found that to a certain extent, the difference between the degree of concreteness of a noun and its nearest neighbors can partly explain the stability or instability of its neighbors.

Domaines: Linguistique; Informatique et langage [cs.CL]

Bénédicte Pierrejean, Ludovic Tanguy. Investigating the stability of concrete nouns in word embeddings. 13th International Conference on Computational Semantics, May 2019, Gothenburg, Sweden. ⟨hal-02073705⟩