Stochastic approximation of quasi-stationary distributions on compact spaces and applications

Michel Benaim (1) , Bertrand Cloez (2) , Fabien Panloup (3)

1 UNINE - Institut de Mathématiques
2 MISTEA - Mathématiques, Informatique et STatistique pour l'Environnement et l'Agronomie
3 LAREMA - Laboratoire Angevin de Recherche en Mathématiques

In the continuity of a recent paper ([6]), dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact metric space killed in finite time. The idea is to run the process until extinction and then to bring it back to life at a position randomly chosen according to the (possibly weighted) empirical occupation measure of its past positions. General conditions are given ensuring the convergence of this measure to the quasi-stationary distribution of the chain. We then apply this method to the numerical approximation of the quasi-stationary distribution of a diffusion process killed on the boundary of a compact set and to the estimation of the spectral gap of irreducible Markov processes. Finally, the sharpness of the assumptions is illustrated through the study of the algorithm in a non-irreducible setting.

Mots clés: Quasi-stationary distributions; stochastic approximation; reinforced random walks; random perturba-tions of dynamical systems; extinction rate; spectral gap; Euler scheme AMS-MSC 65C20; 60B12; 60J10; Secondary 34F05; 60J20; 60J60

Domaines: Probabilités [math.PR]

Michel Benaim, Bertrand Cloez, Fabien Panloup. Stochastic approximation of quasi-stationary distributions on compact spaces and applications. 2016. ⟨hal-01334603v2⟩