Overview of the SPMRL 2013 Shared Task: A Cross-Framework Evaluation of Parsing Morphologically Rich Languages

Djamé Seddah (1) , Reut Tsarfaty (2) , Sandra Kübler (3) , Marie Candito (1) , Jinho D. Choi (4) , Richárd Farkas (5) , Jennifer Foster (6) , Iakes Goenaga (7) , Koldo Gojenola Galletebeitia (7) , Yoav Goldberg (8) , Spence Green (9) , Nizar Habash (10) , Marco Kuhlmann (11) , Wolfgang Maier (12) , Joakim Nivre (11) , Adam Przepiórkowski (13) , Ryan Roth (10) , Wolfgang Seeker (14) , Yannick Versley (15) , Veronika Vincze (16) , Marcin Wolińsk (13) , Alina Wróblewska (13) , Éric Villemonte de La Clergerie (1)

1 ALPAGE - Analyse Linguistique Profonde à Grande Echelle ; Large-scale deep linguistic processing
2 Faculty of Mathematics and Computer Science
3 Computational Linguistics
4 Ipsoft - IPSOFT
5 Department of Computer Algorithms and Artificial Intelligence.
6 NCLT - National Centre for Language Technology
7 Computer Languages and Systems
8 Department of Computer Science
9 Stanford NLP Group
10 CCLS - Center for Computational Learning Systems
11 Computational Linguistics and Language Technology
12 Abteilung für Computerlinguistik [Düsseldorf ]
13 Institute of Computer Science [Warszawa]
14 Institut für Maschinelle Sprachverarbeitung
15 Institute for Computational Linguistics
16 Department of Computer Algorithms and Artificial Intelligence.

This paper reports on the first shared task on statistical parsing of morphologically rich lan- guages (MRLs). The task features data sets from nine languages, each available both in constituency and dependency annotation. We report on the preparation of the data sets, on the proposed parsing scenarios, and on the eval- uation metrics for parsing MRLs given dif- ferent representation types. We present and analyze parsing results obtained by the task participants, and then provide an analysis and comparison of the parsers across languages and frameworks, reported for gold input as well as more realistic parsing scenarios.

Mots clés: Statistical Parsing; Morphologically-rich languages; phrase based parsing; dependency parsing; non gold scenarios

Mots clés: Parsing statistiques; langues à morphologie riche; parsing en constituents; parsing en dépendance; analyse syntaxique à large couverture; treebanking; evaluation; non gold tokenization

Domaines: Linguistique; Informatique et langage [cs.CL]

Djamé Seddah, Reut Tsarfaty, Sandra Kübler, Marie Candito, Jinho D. Choi, et al.. Overview of the SPMRL 2013 Shared Task: A Cross-Framework Evaluation of Parsing Morphologically Rich Languages. Proceedings of the Fourth Workshop on Statistical Parsing of Morphologically-Rich Languages, Oct 2013, Seattle, Washington, United States. pp.146--182. ⟨hal-00877096⟩