Comparing Markov chains: Combining aggregation and precedence relations applied to sets of states

Ana Bušić (1, 2) , Ingrid M. H. Vliegen (3) , Alan Scheller-Wolf (4)

1 LIAFA - Laboratoire d'informatique Algorithmique : Fondements et Applications
2 MESCAL - Middleware efficiently scalable
3 Department of Industrial Engineering and Innovation Sciences
4 Tepper School of Business

Numerical methods for solving Markov chains are in general inefficient if the state space of the chain is very large (or infinite) and lacking a simple repeating structure. One alternative to solving such chains is to construct models that are simple to analyze and that provide bounds for a reward function of interest. We present a new bounding method for Markov chains inspired by Markov reward theory; our method constructs bounds by redirecting selected sets of transitions, facilitating an intuitive interpretation of the modifications on the original system. We show that our method is compatible with strong aggregation of Markov chains; thus we can obtain bounds for the initial chain by analyzing a much smaller chain. We illustrate our method on a problem of order fill rates for an inventory system of service tools.

Mots clés: Markov chains; bounds; aggregation

Domaines: Mathématique discrète [cs.DM]; Performance et fiabilité [cs.PF]

Ana Bušić, Ingrid M. H. Vliegen, Alan Scheller-Wolf. Comparing Markov chains: Combining aggregation and precedence relations applied to sets of states. 2009. ⟨hal-00361430v1⟩