A vanilla Rao–Blackwellization of Metropolis–Hastings algorithms

Randal Douc (1, 2, 3) , Christian Robert (4, 5, 6)

1 TSP - CITI - Communications, Images et Traitement de l'Information
2 TIPIC-SAMOVAR - Traitement de l'Information Pour Images et Communications
3 CNRS - Centre National de la Recherche Scientifique
4 CEREMADE - CEntre de REcherches en MAthématiques de la DEcision
5 CREST - Centre de Recherche en Économie et Statistique
6 IUF - Institut universitaire de France

Casella and Robert (1996) presented a general Rao--Blackwellisation principle for accept-reject and Metropolis-Hastings schemes that leads to significant decreases in the variance of the resulting estimators, but at a high cost in computing and storage. Adopting a completely different perspective, we introduce instead a universal scheme that guarantees variance reductions in all Metropolis-Hastings~based estimators while keeping the computing cost under control. We establish a central limit theorems for the improved estimators and illustrate their performances on toy examples.

Mots clés: Central limit theorem; Markov Chain Monte Carlo (MCMC); Metropolis-Hastings algorithm; Variance reduction; Conditioning

Domaines: Calcul [stat.CO]; Statistiques [math.ST]

Randal Douc, Christian Robert. A vanilla Rao–Blackwellization of Metropolis–Hastings algorithms. Annals of Statistics, 2011, 39 (1), pp.261-277. ⟨10.1214/10-AOS838⟩. ⟨hal-00375377⟩