A deep reinforcement learning approach for early classification of time series

Coralie Martinez (1, 2) , Guillaume Perrin (2) , E Ramasso (3) , Michèle Rombaut (1)

1 GIPSA-AGPIG - GIPSA - Architecture, Géométrie, Perception, Images, Gestes
2 Bio-Mérieux [Marcy l'Etoile]
3 FEMTO-ST - Franche-Comté Électronique Mécanique, Thermique et Optique - Sciences et Technologies (UMR 6174)

—In many real-world applications, ranging from pre-dictive maintenance to personalized medicine, early classification of time series data is of paramount importance for supporting decision makers. In this article, we address this challenging task with a novel approach based on reinforcement learning. We introduce an early classifier agent, an end-to-end reinforcement learning agent (deep Q-network, DQN) [1] able to perform early classification in an efficient way. We formulate the early classification problem in a reinforcement learning framework: we introduce a suitable set of states and actions but we also define a specific reward function which aims at finding a compromise between earliness and classification accuracy. While most of the existing solutions do not explicitly take time into account in the final decision, this solution allows the user to set this trade-off in a more flexible way. In particular, we show experimentally on datasets from the UCR time series archive [2] that this agent is able to continually adapt its behavior without human intervention and progressively learn to compromise between accurate and fast predictions.

Keywords: time sensitive applications; early classification; Index Terms—time series; reinforcement learning; Deep Q-Network

Domains: Signal and Image Processing

Coralie Martinez, Guillaume Perrin, E Ramasso, Michèle Rombaut. A deep reinforcement learning approach for early classification of time series. EUSIPCO 2018 - 26th European Signal Processing Conference, Sep 2018, Rome, Italy. ⟨10.23919/EUSIPCO.2018.8553544⟩. ⟨hal-01825472⟩