Handling Unknown Words in Statistical Latent-Variable Parsing Models for Arabic, English and French

Mohammed Attia (1) , Jennifer Foster (1) , Deirdre Hogan (1) , Joseph Le Roux (1) , Lamia Tounsi (1) , Josef van Genabith (1)

1 NCLT - National Centre for Language Technology

This paper presents a study of the impact of using simple and complex morphological clues to improve the classification of rare and unknown words for parsing. We compare this approach to a language-independent technique often used in parsers which is based solely on word frequencies. This study is ap- plied to three languages that exhibit different levels of morphological expressiveness: Arabic, French and English. We integrate infor- mation about Arabic affixes and morphotac- tics into a PCFG-LA parser and obtain state- of-the-art accuracy. We also show that these morphological clues can be learnt automati- cally from an annotated corpus.

Mots clés: parsing

Domaines: Informatique et langage [cs.CL]

Mohammed Attia, Jennifer Foster, Deirdre Hogan, Joseph Le Roux, Lamia Tounsi, et al.. Handling Unknown Words in Statistical Latent-Variable Parsing Models for Arabic, English and French. First Workshop on Statistical Parsing of Morphologically Rich Languages (SPMRL 2010), 2010, United States. pp.67-75. ⟨hal-00702414⟩