CALYOU: A Comparable Spoken Algerian Corpus Harvested from YouTube

Karima Abidi (1) , Mohamed Amine Menacer (1) , Kamel Smaili (1)

1 SMarT - Statistical Machine Translation and Speech Modelization and Text

This paper addresses the issue of comparability of comments extracted from Youtube. The comments concern spoken Algerian which could be either local Arabic, Modern Standard Arabic or French. This diversity of expression arises a huge number of problems concerning the data processing. In this article, several methods of alignment will be proposed and tested. The method which permits to best align is Word2Vec-based approach that will be used iteratively. This recurrent call of Word2Vec allows to improve significantly the results of comparability. In fact, a dictionary-based approach leads to a Recall of 4, while our approach allows to get a Recall of 33 at rank 1. Thanks to this approach, we built from Youtube CALYOU, a Comparable Corpus of the spoken Algerian.

Mots clés: Comparability; Arabic dialect; Word2Vec; Youtube

Domaines: Informatique et langage [cs.CL]

Karima Abidi, Mohamed Amine Menacer, Kamel Smaili. CALYOU: A Comparable Spoken Algerian Corpus Harvested from YouTube. 18th Annual Conference of the International Communication Association (Interspeech), Aug 2017, Stockholm, Sweden. ⟨hal-01531591⟩