Markov Property for a Function of a Markov chain: a linear algebra approach

Leonid Gurvits (1) , James Ledoux (2, 3)

1 LANL - Los Alamos National Laboratory
2 IRMAR - Institut de Recherche Mathématique de Rennes
3 INSA Rennes - Institut National des Sciences Appliquées - Rennes

In this paper, we address whether a (probabilistic) function of a finite homogeneous Markov chain still enjoys a Markov-type property. We propose a complete answer to this question using a linear algebra approach. At the core of our approach is the concept of invariance of a set under a matrix. In that sense, the framework of this paper is related to the so-called "geometric approach" in control theory for linear dynamical systems. This allows us to derive a collection of new results under generic assumptions on the original Markov chain. In particular, we obtain a new criterion for a function of a Markov chain to be a homogeneous Markov chain. We provide a deterministic polynomial-time algorithm for checking this criterion. Moreover, a non-standard notion of observability for a linear system will be used. This allows one to show that the set of all stochastic matrices for which our criterion holds, is nowhere dense in the set of stochastic matrices.

Mots clés: k th-order Markov chain; Rogers-Pitman matrix; Hidden Markov chain

Domaines: Probabilités [math.PR]; Performance et fiabilité [cs.PF]

Leonid Gurvits, James Ledoux. Markov Property for a Function of a Markov chain: a linear algebra approach. Linear Algebra and its Applications, 2005, 404, pp.85-117. ⟨hal-00837503⟩