Skip to Main content Skip to Navigation
Journal articles

Bounding Fixed Points of Set-based Bellman Operator and Nash Equilibria of Stochastic Games

Abstract : Motivated by uncertain parameters encountered in Markov decision processes (MDPs) and stochastic games, we study the effect of parameter uncertainty on Bellman operator-based algorithms under a set-based framework. Specifically, we first consider a family of MDPs where the cost parameters are in a given compact set; we then define a Bellman operator acting on a set of value functions to produce a new set of value functions as the output under all possible variations in the cost parameter. We prove the existence of a fixed point of this set-based Bellman operator by showing that it is contractive on a complete metric space, and explore its relationship with the corresponding family of MDPs and stochastic games. Additionally, we show that given interval set-bounded cost parameters, we can form exact bounds on the set of optimal value functions. Finally, we utilize our results to bound the value function trajectory of a player in a stochastic game.
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03187698
Contributor : Pierre-Loïc Garoche Connect in order to contact the contributor
Submitted on : Thursday, April 1, 2021 - 11:40:38 AM
Last modification on : Tuesday, October 19, 2021 - 11:02:50 AM
Long-term archiving on: : Friday, July 2, 2021 - 6:36:35 PM

Files

root.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Sarah Li, Assalé Adjé, Pierre-Loïc Garoche, Behçet Açıkmeşe. Bounding Fixed Points of Set-based Bellman Operator and Nash Equilibria of Stochastic Games. Automatica, Elsevier, 2021, 130, ⟨10.1016/j.automatica.2021.109685⟩. ⟨hal-03187698⟩

Share

Metrics

Les métriques sont temporairement indisponibles