Empirical Bernstein Inequality for Martingales : Application to Online Learning

Thomas Peel (1, 2) , Sandrine Anthoine (1) , Liva Ralaivola (2)

1 LATP - Laboratoire d'Analyse, Topologie, Probabilités
2 QARMA - éQuipe AppRentissage et MultimediA [Marseille]

In this article we present a new empirical Bernstein inequality for bounded martingale difference sequences. This inequality refines the one by Freedman [1975] and is then used in order to bound the average risk of the hypotheses during an online learning process. We show theoretical and empirical evidences of the tightness of our result compared with the state of the art bound provided by Cesa-Bianchi and Gentile [2008].

Mots clés: Empirical Inequality; Bernstein Inequality; Online Learning; Martingale; Martingale Difference Sequence

Domaines: Machine Learning [stat.ML]

Thomas Peel, Sandrine Anthoine, Liva Ralaivola. Empirical Bernstein Inequality for Martingales : Application to Online Learning. 2013. ⟨hal-00879909⟩