HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Journal articles

Perceptual evaluation for automatic anomaly detection in disordered speech: Focus on ambiguous cases

Abstract : Perceptual evaluation is still the most common method in clinical practice for diagnosing and following the condition progression of people suffering from dysarthria (or speech disorders more generally). Such evaluations are frequently described as non-trivial, subjective and highly time-consuming (depending on the evaluation level). Most of the time, perceptual assessment is performed individually by clinicians which can be problematic since judgment may vary from one clinician to the other. Clinicians have therefore expressed the need for new objective evaluation tools better adapted to longitudinal studies, the observation of small units and rehabilitation context to monitor patients' progress. We have previously proposed an automatic approach to the anomaly detection at the phone level for dysarthric speech. The system behavior was studied and validated with different corpora and speech styles and shows good results in this specific task. Nonetheless, the lack of annotated French dysarthric speech corpora has limited our ability to analyze some aspects of its behavior, such as severity, more precisely (more anomalies are detected automatically compared with human experts). To overcome this limitation, we proposed an original perceptual evaluation protocol applied to a limited set of decisions made by the automatic system, relating to the presence of anomalies. Particularly, we intended to focus our analyses on some ambiguous cases in order to enrich our knowledge about the system behavior. This evaluation was carried out by a jury of 29 non-naive individuals. Results confirm the relevance of the system for the anomaly detection, and place it within the most severe juries. Besides interesting information related to the system behavior, the evaluation protocol highlighted main differences between human process and the automatic system: humans have difficulties in focusing on small units and they are influenced by con-textual information, while the system only focuses on small units. In a way, both approaches are probably complementary.
Document type :
Journal articles
Complete list of metadata

Cited literature [3 references]  Display  Hide  Download

Contributor : Christine Meunier Connect in order to contact the contributor
Submitted on : Tuesday, December 18, 2018 - 5:05:52 PM
Last modification on : Friday, February 4, 2022 - 3:27:45 AM
Long-term archiving on: : Wednesday, March 20, 2019 - 11:03:38 AM


Files produced by the author(s)




Imed Laaridh, Christine Meunier, Corinne Fredouille. Perceptual evaluation for automatic anomaly detection in disordered speech: Focus on ambiguous cases. Speech Communication, Elsevier : North-Holland, 2018, 105, pp.23-33. ⟨10.1016/j.specom.2018.10.003⟩. ⟨hal-01959385⟩



Record views


Files downloads