Reward-­based online learning in non­stationary environments: adapting a P300­-speller with a ``Backspace’’ key

Abstract : We adapt a policy gradient approach to the problem of reward-based online learning of a non-invasive EEG-based ``P300''-speller. We first clarify the nature of the P300-speller classification problem and present a general regularized gradient ascent formula. We then show that when the reward is immediate and binary (namely ``bad response'' or ``good response''), each update is expected to improve the classifier accuracy, whether the actual response is correct or not. We also estimate the robustness of the method to occasional mistaken rewards, i.e. show that the learning efficacy may only linearly decrease with the rate of invalid rewards. The effectiveness of our approach is tested in a series of simulations reproducing the conditions of real experiments. We show in a first experiment that a systematic improvement of the spelling rate is obtained for all subjects in the absence of initial calibration. In a second experiment, we consider the case of the online recovery that is expected to follow unforeseen impairments. Combined with a specific failure detection algorithm, the spelling error information (typically contained in a ``backspace'' hit), is shown useful for the policy gradient to adapt the P300 classifier to the new situation, provided the feedback is reliable enough (namely having a reliability greater than 70%).
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-01196513
Contributor : Liva Ralaivola <>
Submitted on : Wednesday, September 9, 2015 - 11:01:06 PM
Last modification on : Monday, March 4, 2019 - 2:04:23 PM

Identifiers

  • HAL Id : hal-01196513, version 1

Collections

Citation

Emmanuel Daucé, Timothée Proix, Liva Ralaivola. Reward-­based online learning in non­stationary environments: adapting a P300­-speller with a ``Backspace’’ key. IJCNN 2015, Jul 2015, Killarney, Ireland. ⟨hal-01196513⟩

Share

Metrics

Record views

122