Fano's inequality for random variables

Sebastien Gerchinovitz (1) , Pierre Ménard (1) , Gilles Stoltz (2, 3)

1 IMT - Institut de Mathématiques de Toulouse UMR5219
2 LMO - Laboratoire de Mathématiques d'Orsay
3 CELESTE - Statistique mathématique et apprentissage

We extend Fano's inequality, which controls the average probability of events in terms of the average of some f--divergences, to work with arbitrary events (not necessarily forming a partition) and even with arbitrary [0,1]--valued random variables, possibly in continuously infinite number. We provide two applications of these extensions, in which the consideration of random variables is particularly handy: we offer new and elegant proofs for existing lower bounds, on Bayesian posterior concentration (minimax or distribution-dependent) rates and on the regret in non-stochastic sequential learning.

Keywords: Information theory; Lower bounds; Multiple-hypotheses testing; Bayesian posterior concentration; Bayesian posterior concentration *

Domains: Statistics Theory [stat.TH]; Information Theory [cs.IT]

Sebastien Gerchinovitz, Pierre Ménard, Gilles Stoltz. Fano's inequality for random variables. Statistical Science, 2020, 35 (2), pp.178-201. ⟨hal-01470862v3⟩