Evaluating the Impact of External Lexical Resources into a CRF-based Multiword Segmenter and Part-of-Speech Tagger

Mathieu Constant (1) , Isabelle Tellier (2)

1 LIGM - Laboratoire d'Informatique Gaspard-Monge
2 Lattice - Lattice - Langues, Textes, Traitements informatiques, Cognition - UMR 8094

This paper evaluates the impact of external lexical resources into a CRF-based joint Multiword Segmenter and Part-of-Speech Tagger. We especially show different ways of integrating lexicon-based features in the tagging model. We display an absolute gain of 0.5\% in terms of f-measure. Moreover, we show that the integration of lexicon-based features significantly compensates the use of a small training corpus.

Mots clés: Multiword Expressions; Part Of Speech tagging; Statistical and machine learning methods; Conditional Random Fields; Morphosyntactic lexicons; lexical segmentation

Domaines: Traitement du texte et du document

Mathieu Constant, Isabelle Tellier. Evaluating the Impact of External Lexical Resources into a CRF-based Multiword Segmenter and Part-of-Speech Tagger. 8th International Conference on Language Resources and Evaluation (LREC'12), May 2012, Turkey. pp.646-650. ⟨hal-00790624⟩