Online classification accuracy is a poor metric to study mental imagery-based bci user learning: an experimental demonstration and new metrics

Fabien Lotte (1, 2) , Camille Jeunet (3)

1 LaBRI - Laboratoire Bordelais de Recherche en Informatique
2 Potioc - Popular interaction with 3d content
3 Hybrid - 3D interaction with virtual environments using body and mind

While promising for many applications, Electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs) are still scarcely used outside laboratories , due to a poor reliability. It is thus necessary to study and fix this reliability issue. Doing so requires to use appropriate reliability metrics to quantify both signal processing and user learning performances. So far, Classification Accuracy (CA) is the typical metric used for both aspects. However, we argue in this paper that CA is a poor metric to study how well users are learning to use the BCI. Indeed CA is notably unspecific, discrete, training data and classifier dependent, and as such may not always reflect successful EEG pattern self-modulation by the user. We thus propose new performance metrics to specifically measure how distinct and stable the EEG patterns produced by the user are. By re-analyzing EEG data with these metrics, we indeed confirm that CA may hide some learning effects or hide the user inability to self-modulate a given EEG pattern.

Mots clés: Brain-computer interfaces; Performance; Metric

Domaines: Sciences cognitives

Fabien Lotte, Camille Jeunet. Online classification accuracy is a poor metric to study mental imagery-based bci user learning: an experimental demonstration and new metrics . 7th International BCI Conference, Sep 2017, Graz, Austria. ⟨hal-01519478⟩