Skip to Main content Skip to Navigation
Conference papers

Laplace's rule of succession in information geometry

Yann Ollivier 1, 2
1 TAO - Machine Learning and Optimisation
CNRS - Centre National de la Recherche Scientifique : UMR8623, Inria Saclay - Ile de France, UP11 - Université Paris-Sud - Paris 11, LRI - Laboratoire de Recherche en Informatique
Abstract : Laplace's "add-one" rule of succession modifies the observed frequencies in a sequence of heads and tails by adding one to the observed counts. This improves prediction by avoiding zero probabilities and corresponds to a uniform Bayesian prior on the parameter. The canonical Jeffreys prior corresponds to the "add-one-half" rule. We prove that, for exponential families of distributions, such Bayesian predictors can be approximated by taking the average of the maximum likelihood predictor and the \emph{sequential normalized maximum likelihood} predictor from information theory. Thus in this case it is possible to approximate Bayesian predictors without the cost of integrating or sampling in parameter space.
Document type :
Conference papers
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-01228952
Contributor : Yann Ollivier <>
Submitted on : Sunday, November 15, 2015 - 6:13:11 PM
Last modification on : Tuesday, April 21, 2020 - 1:08:33 AM

Links full text

Identifiers

  • HAL Id : hal-01228952, version 1
  • ARXIV : 1503.04304

Citation

Yann Ollivier. Laplace's rule of succession in information geometry. Geometric science of information, Oct 2015, Palaiseau, France. pp.311-319. ⟨hal-01228952⟩

Share

Metrics

Record views

219