Skip to Main content Skip to Navigation
Journal articles

Max-Plus Linear Approximations for Deterministic Continuous-State Markov Decision Processes

Eloïse Berthier 1, 2 Francis Bach 1, 2
1 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, CNRS - Centre National de la Recherche Scientifique, Inria de Paris
Abstract : We consider deterministic continuous-state Markov decision processes (MDPs). We apply a max-plus linear method to approximate the value function with a specific dictionary of functions that leads to an adequate state-discretization of the MDP. This is more efficient than a direct discretization of the state space, typically intractable in high dimension. We propose a simple strategy to adapt the discretization to a problem instance, thus mitigating the curse of dimensionality. We provide numerical examples showing that the method works well on simple MDPs.
Document type :
Journal articles
Complete list of metadata

Cited literature [17 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02617479
Contributor : Eloïse Berthier <>
Submitted on : Monday, May 25, 2020 - 12:01:20 PM
Last modification on : Tuesday, May 4, 2021 - 2:06:02 PM

File

max-plus-linear-approximations...
Files produced by the author(s)

Identifiers

Collections

Citation

Eloïse Berthier, Francis Bach. Max-Plus Linear Approximations for Deterministic Continuous-State Markov Decision Processes. IEEE Control Systems Letters, IEEE, 2020, 4 (3), pp.767-772. ⟨10.1109/LCSYS.2020.2973199⟩. ⟨hal-02617479⟩

Share

Metrics

Record views

6805

Files downloads

361