Convergence of Markovian Stochastic Approximation with Discontinuous Dynamics

Gersende Fort (1, 2) , Éric Moulines (3, 2) , Amandine Schreck (1, 2) , Matti Vihola (2, 4)

1 LTCI - Laboratoire Traitement et Communication de l'Information
2 CMAP - Centre de Mathématiques Appliquées - Ecole Polytechnique
3 XPOP - Modélisation en pharmacologie de population
4 JYU - University of Jyväskylä

This paper is devoted to the convergence analysis of stochastic approximation algorithms of the form $\theta_{n+1} = \theta_n + \gamma_n H(\theta_n,X_{n+1})$ where $\{{\theta_n,n \in \nset\}$ is a $\rset^d$-valued sequence, {γn, n ∈ N} is a deterministic step-size sequence and {Xn, n ∈ N} is a controlled Markov chain. We study the convergence under weak assumptions on smoothness-in-θ of the function $\theta \to H(\theta,x)$. It is usually assumed that this function is continuous for any x; in this work, we relax this condition. Our results are illustrated by considering stochastic approximation algorithms for (adaptive) quantile estimation and a penalized version of the vector quantization.

Mots clés: State-dependent noise; Stochastic approximat; Discontinuous dynamics; Controlled Markov chain

Domaines: Statistiques [math.ST]

Gersende Fort, Éric Moulines, Amandine Schreck, Matti Vihola. Convergence of Markovian Stochastic Approximation with Discontinuous Dynamics. SIAM Journal on Control and Optimization, 2016, 54 (2), pp.866-893. ⟨10.1137/140962723⟩. ⟨hal-01418857⟩