Perceptual evaluation for automatic anomaly detection in disordered speech: Focus on ambiguous cases

Abstract : Perceptual evaluation is still the most common method in clinical practice for diagnosing and following the condition progression of people suffering from dysarthria (or speech disorders more generally). Such evaluations are frequently described as non-trivial, subjective and highly time-consuming (depending on the evaluation level). Most of the time, perceptual assessment is performed individually by clinicians which can be problematic since judgment may vary from one clinician to the other. Clinicians have therefore expressed the need for new objective evaluation tools better adapted to longitudinal studies, the observation of small units and rehabilitation context to monitor patients' progress. We have previously proposed an automatic approach to the anomaly detection at the phone level for dysarthric speech. The system behavior was studied and validated with different corpora and speech styles and shows good results in this specific task. Nonetheless, the lack of annotated French dysarthric speech corpora has limited our ability to analyze some aspects of its behavior, such as severity, more precisely (more anomalies are detected automatically compared with human experts). To overcome this limitation, we proposed an original perceptual evaluation protocol applied to a limited set of decisions made by the automatic system, relating to the presence of anomalies. Particularly, we intended to focus our analyses on some ambiguous cases in order to enrich our knowledge about the system behavior. This evaluation was carried out by a jury of 29 non-naive individuals. Results confirm the relevance of the system for the anomaly detection, and place it within the most severe juries. Besides interesting information related to the system behavior, the evaluation protocol highlighted main differences between human process and the automatic system: humans have difficulties in focusing on small units and they are influenced by con-textual information, while the system only focuses on small units. In a way, both approaches are probably complementary.
Type de document :
Article dans une revue
Speech Communication, Elsevier : North-Holland, 2018, 105, pp.23-33. 〈10.1016/j.specom.2018.10.003〉
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01959385
Contributeur : Christine Meunier <>
Soumis le : mardi 18 décembre 2018 - 17:05:52
Dernière modification le : vendredi 22 mars 2019 - 11:34:07
Document(s) archivé(s) le : mercredi 20 mars 2019 - 11:03:38

Fichier

perceptual-evaluation-automati...
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

Citation

Imed Laaridh, Christine Meunier, Corinne Fredouille. Perceptual evaluation for automatic anomaly detection in disordered speech: Focus on ambiguous cases. Speech Communication, Elsevier : North-Holland, 2018, 105, pp.23-33. 〈10.1016/j.specom.2018.10.003〉. 〈hal-01959385〉

Partager

Métriques

Consultations de la notice

23

Téléchargements de fichiers

22