Commutative Stochastic Games

Xavier Venel (1)

1 CES - Centre d'économie de la Sorbonne

We are interested in the convergence of the value of n-stage games as n goes to infinity and the existence of the uniform value in stochastic games with a general set of states and finite sets of actions where the transition is commutative. This means that playing an action profile a 1 followed by an action profile a 2 , leads to the same distribution on states as playing first the action profile a 2 and then a 1. For example, absorbing games can be reformulated as commutative stochastic games. When there is only one player and the transition function is deterministic, we show that the existence of a uniform value in pure strategies implies the existence of 0-optimal strategies. In the framework of two-player stochastic games, we study a class of games where the set of states is R m and the transition is deterministic and 1-Lipschitz for the L 1-norm, and prove that these games have a uniform value. A similar proof shows the existence of an equilibrium in the non zero-sum case. These results remain true if one considers a general model of finite repeated games, where the transition is commutative and the players observe the past actions but not the state.

Mots clés: Repeated games; 0-optimality; Markov Decision Process; Stochastic games; Uniform value

Domaines: Optimisation et contrôle [math.OC]

Xavier Venel. Commutative Stochastic Games. Mathematics of Operations Research, 2015, ⟨10.1287/moor.2014.0676⟩. ⟨hal-01302525⟩