Nonlinear Randomized Urn Models: a Stochastic Approximation Viewpoint

Sophie Laruelle (1, 2) , Gilles Pagès (3)

1 LAMA - Laboratoire d'Analyse et de Mathématiques Appliquées
2 MAS - Mathématiques Appliquées aux Systèmes - EA 4037
3 LPMA - Laboratoire de Probabilités et Modèles Aléatoires

This paper extend the link between stochastic approximation and randomized urn models investigated in Laruelle and Pagès (AAP 2013) for application in clinical trials introduced in Bai and Hu (AAP 2005) or Bai, Hu ans Shen (JMA 2002). The idea is that the drawing rule is not necessary uniform on the urn composition, but can be reinforced by a function f. Firstly, by considering that f is concave or convex and by reformulating the dynamics of the urn composition as a standard stochastic approximation (SA) algorithm (with remainder), we derive the a.s. convergence and the asymptotic normality (Central Limit Theorem CLT) of the normalized procedure by calling upon the ODE and SDE methods. An in-depth analysis of this reinforced drawing rule in dimension d=2 exhibit two different behaviours: either a single equilibrium point when f is concave, or a single, two or three ones when f is convex. The last setting is solved using results on traps for SA to remove the repulsive point and to deduce the a.s. towards one of the attractive point. Secondly the Polya urn is investigated with the point of view of bandit algorithm. Finally, these results are applied to Finance for optimal allocation and to the case where f has regular variation.

Mots clés: Stochastic approximation; extended Pólya urn models; reinforcement; nonhomogeneous generating matrix; strong consistency; asymptotic normality; bandit algorithm

Domaines: Probabilités [math.PR]

Sophie Laruelle, Gilles Pagès. Nonlinear Randomized Urn Models: a Stochastic Approximation Viewpoint. 2013. ⟨hal-00910902v1⟩