Inter-annotator agreement for a speech corpus pronounced by French and German language learners

Odile Mella 1 Dominique Fohr 1 Anne Bonneau 1
1 MULTISPEECH - Speech Modeling for Facilitating Oral-Based Communication
Inria Nancy - Grand Est, LORIA - NLPKD - Department of Natural Language Processing & Knowledge Discovery
Abstract : This paper presents the results of an investigation of inter-annotator agreement for the non-native and native French part of the IFCASL corpus. This large bilingual speech corpus for French and German language learners was manually annotated by several annotators. This manual annotation is the starting point which will be used both to improve the automatic segmentation algorithms and derive diagnosis and feedback. The agreement is evaluated by comparing the manual alignments of seven annotators to the manual alignment of an expert, for 18 sentences. Whereas results for the presence of the devoicing diacritic show a certain degree of disagreement between the annotators and the expert, there is a very good consistency between annotators and the expert for temporal boundaries as well as insertions and deletions. We find a good overall agreement for boundaries between annotators and expert with a mean deviation of 7.6 ms and 93% of boundaries within 20 ms.
Type de document :
Communication dans un congrès
Workshop on Speech and Language Technology in Education, Sep 2015, Leipzig, Germany
Liste complète des métadonnées

Littérature citée [10 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-01185194
Contributeur : Odile Mella <>
Soumis le : mercredi 19 août 2015 - 12:13:36
Dernière modification le : samedi 22 décembre 2018 - 17:26:06
Document(s) archivé(s) le : vendredi 20 novembre 2015 - 10:44:12

Fichier

slate_agreement_v7.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01185194, version 1

Collections

Citation

Odile Mella, Dominique Fohr, Anne Bonneau. Inter-annotator agreement for a speech corpus pronounced by French and German language learners. Workshop on Speech and Language Technology in Education, Sep 2015, Leipzig, Germany. 〈hal-01185194〉

Partager

Métriques

Consultations de la notice

374

Téléchargements de fichiers

170