Continuous-time Markov chains as transformers of unbounded observables

Abstract : This paper provides broad sufficient conditions for the com-putability of time-dependent averages of stochastic processes of the form f (Xt) where Xt is a continuous-time Markov chain (CTMC), and f is a real-valued function (aka an observable). We consider chains with values in a countable state space S, and possibly unbounded f s. Observables are seen as generalised predicates on S and chains are interpreted as transformers of such generalised predicates, mapping each observable f to a new observable Ptf defined as (Ptf)(x) = Ex(f (Xt)), which represents the mean value of f at time time t as a function of the initial state x. We obtain three results. First, the well-definedness of this operator interpretation is obtained for a large class of chains and observables by restricting Pt to judiciously chosen rescalings of the basic Banach space C0(S) of S-indexed sequences which vanish at infinity. We prove, under appropriate assumptions, that the restricted family Pt forms a strongly continuous operator semigroup (equivalently the time evolution map t → Pt is continuous w.r.t. the usual topology on bounded operators). The computability of the time evolution map follows by generic arguments of constructive analysis. A key point here is that the assumptions are flexible enough to accommodate unbounded observables, and we give explicit examples of such using stochastic Petri nets and stochastic string rewriting. Thirdly, we show that if the rate matrix (aka the q-matrix) of the CTMC is locally algebraic on a subspace containing f , the time evolution of projections t → (Ptf)(x) is PTIME computable for each x. These results provide a functional analytic alternative to Monte Carlo simulation as test bed for mean-field approximations, moment closure, and similar techniques that are fast, but lack absolute error guarantees.
Type de document :
Pré-publication, Document de travail
Acceped to the 20th International Conference on Foundations of Software Science and Computation S.. 2016
Liste complète des métadonnées

Littérature citée [4 références]  Voir  Masquer  Télécharger
Contributeur : Ilias Garnier <>
Soumis le : lundi 9 janvier 2017 - 09:25:59
Dernière modification le : samedi 14 janvier 2017 - 01:20:13
Document(s) archivé(s) le : lundi 10 avril 2017 - 13:02:20


Fichiers produits par l'(les) auteur(s)


  • HAL Id : hal-01429664, version 1



Vincent Danos, Tobias Heindel, Ilias Garnier, Jakob Simonsen. Continuous-time Markov chains as transformers of unbounded observables. Acceped to the 20th International Conference on Foundations of Software Science and Computation S.. 2016. 〈hal-01429664〉



Consultations de
la notice


Téléchargements du document