Geometry and Analogies: A Study and Propagation Method for Word Representations

Abstract : In this paper we discuss the well-known claim that language analogies yield almost parallel vector differences in word embeddings. On the one hand, we show that this property, while it does hold for a handful of cases, fails to hold in general especially in high dimension, using the best known publicly available word embeddings. On the other hand, we show that this property is not crucial for basic natural language processing tasks such as text classification. We achieve this by a simple algorithm which yields updated word embeddings where this property holds: we show that in these word representations, text classification tasks have about the same performance.
Document type :
Book sections
Complete list of metadatas

Cited literature [22 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02350281
Contributor : Leo Liberti <>
Submitted on : Wednesday, November 6, 2019 - 5:46:47 AM
Last modification on : Friday, November 8, 2019 - 1:39:40 AM

File

slsp19.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Sammy Khalife, Leo Liberti, Michalis Vazirgiannis. Geometry and Analogies: A Study and Propagation Method for Word Representations. Statistical Language and Speech Processing, pp.100-111, 2019, ⟨10.1007/978-3-030-31372-2_9⟩. ⟨hal-02350281⟩

Share

Metrics

Record views

5

Files downloads

8