Word Embedding-Based Approaches for Measuring Semantic Similarity of Arabic-English Sentences

Abstract : Semantic Textual Similarity (STS) is an important component in many Natural Language Processing (NLP) applications, and plays an important role in diverse areas such as information retrieval, machine translation, information extraction and plagiarism detection. In this paper we propose two word embedding-based approaches devoted to measuring the semantic similarity between Arabic-English cross-language sentences. The main idea is to exploit Machine Translation (MT) and an improved word embedding representations in order to capture the syntactic and semantic properties of words. MT is used to translate English sentences into Arabic language in order to apply a classical monolingual comparison. Afterwards, two word embedding-based methods are developed to rate the semantic similarity. Additionally, Words Alignment (WA), Inverse Document Frequency (IDF) and Part-of-Speech (POS) weighting are applied on the examined sentences to support the identification of words that are most descriptive in each sentence. The performances of our approaches are evaluated on a cross-language dataset containing more than 2400 Arabic-English pairs of sentence. Moreover, the proposed methods are confirmed through the Pearson correlation between our similarity scores and human ratings.
Liste complète des métadonnées

Cited literature [42 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01683494
Contributor : Didier Schwab <>
Submitted on : Saturday, January 13, 2018 - 7:34:11 PM
Last modification on : Monday, February 11, 2019 - 4:36:02 PM
Document(s) archivé(s) le : Monday, May 7, 2018 - 1:28:20 AM

File

Lecture_Notes_in_Computer_Scie...
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01683494, version 1

Collections

Citation

El Moatez Billah Nagoudi, Jérémy Ferrero, Didier Schwab, Hadda Cherroun. Word Embedding-Based Approaches for Measuring Semantic Similarity of Arabic-English Sentences. 6th International Conference on Arabic Language Processing, Oct 2017, Fez, Morocco. ⟨hal-01683494⟩

Share

Metrics

Record views

195

Files downloads

416