Online classification accuracy is a poor metric to study mental imagery-based bci user learning: an experimental demonstration and new metrics

Fabien Lotte 1, 2 Camille Jeunet 3
2 Potioc - Popular interaction with 3d content
LaBRI - Laboratoire Bordelais de Recherche en Informatique, Inria Bordeaux - Sud-Ouest
3 Hybrid - 3D interaction with virtual environments using body and mind
Inria Rennes – Bretagne Atlantique , IRISA_D6 - MEDIA ET INTERACTIONS
Abstract : While promising for many applications, Electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs) are still scarcely used outside laboratories , due to a poor reliability. It is thus necessary to study and fix this reliability issue. Doing so requires to use appropriate reliability metrics to quantify both signal processing and user learning performances. So far, Classification Accuracy (CA) is the typical metric used for both aspects. However, we argue in this paper that CA is a poor metric to study how well users are learning to use the BCI. Indeed CA is notably unspecific, discrete, training data and classifier dependent, and as such may not always reflect successful EEG pattern self-modulation by the user. We thus propose new performance metrics to specifically measure how distinct and stable the EEG patterns produced by the user are. By re-analyzing EEG data with these metrics, we indeed confirm that CA may hide some learning effects or hide the user inability to self-modulate a given EEG pattern.
Document type :
Conference papers
Complete list of metadatas

Cited literature [15 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01519478
Contributor : Camille Jeunet <>
Submitted on : Sunday, May 7, 2017 - 11:44:44 PM
Last modification on : Thursday, May 9, 2019 - 4:16:13 PM
Long-term archiving on : Tuesday, August 8, 2017 - 12:23:31 PM

File

lotte-jeunet_graz2017_metrics_...
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01519478, version 1

Citation

Fabien Lotte, Camille Jeunet. Online classification accuracy is a poor metric to study mental imagery-based bci user learning: an experimental demonstration and new metrics . 7th International BCI Conference, Sep 2017, Graz, Austria. ⟨hal-01519478⟩

Share

Metrics

Record views

1039

Files downloads

359