Laplace's rule of succession in information geometry

Yann Ollivier 1, 2
1 TAO - Machine Learning and Optimisation
CNRS - Centre National de la Recherche Scientifique : UMR8623, Inria Saclay - Ile de France, UP11 - Université Paris-Sud - Paris 11, LRI - Laboratoire de Recherche en Informatique
Abstract : Laplace's "add-one" rule of succession modifies the observed frequencies in a sequence of heads and tails by adding one to the observed counts. This improves prediction by avoiding zero probabilities and corresponds to a uniform Bayesian prior on the parameter. The canonical Jeffreys prior corresponds to the "add-one-half" rule. We prove that, for exponential families of distributions, such Bayesian predictors can be approximated by taking the average of the maximum likelihood predictor and the \emph{sequential normalized maximum likelihood} predictor from information theory. Thus in this case it is possible to approximate Bayesian predictors without the cost of integrating or sampling in parameter space.
Type de document :
Communication dans un congrès
Frank Nielsen; Frédéric Barbarescro. Geometric science of information, Oct 2015, Palaiseau, France. LNCS 9389, Springer Verlag, Lecture notes in computer science, pp.311-319, 2015, Geometric science of information
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01228952
Contributeur : Yann Ollivier <>
Soumis le : dimanche 15 novembre 2015 - 18:13:11
Dernière modification le : jeudi 5 avril 2018 - 12:30:12

Lien texte intégral

Identifiants

  • HAL Id : hal-01228952, version 1
  • ARXIV : 1503.04304

Citation

Yann Ollivier. Laplace's rule of succession in information geometry. Frank Nielsen; Frédéric Barbarescro. Geometric science of information, Oct 2015, Palaiseau, France. LNCS 9389, Springer Verlag, Lecture notes in computer science, pp.311-319, 2015, Geometric science of information. 〈hal-01228952〉

Partager

Métriques

Consultations de la notice

164