Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications

Achref Bachouch (1) , Côme Huré (2) , Nicolas Langrené (3) , Huyen Pham (2)

1 UiO - University of Oslo
2 LPSM (UMR_8001) - Laboratoire de Probabilités, Statistique et Modélisation
3 CSIRO - Data61 [Canberra]

This paper presents several numerical applications of deep learning-based algorithms that have been introduced in [HPBL18]. Numerical and comparative tests using TensorFlow illustrate the performance of our different algorithms, namely control learning by performance iteration (algorithms NNcontPI and ClassifPI), control learning by hybrid iteration (algorithms Hybrid-Now and Hybrid-LaterQ), on the 100-dimensional nonlinear PDEs examples from [EHJ17] and on quadratic backward stochastic differential equations as in [CR16]. We also performed tests on low-dimension control problems such as an option hedging problem in finance, as well as energy storage problems arising in the valuation of gas storage and in microgrid management. Numerical results and comparisons to quantization-type algorithms Qknn, as an efficient algorithm to numerically solve low-dimensional control problems, are also provided; and some corresponding codes are available on https://github.com/comeh/.

Mots clés: reinforcement learning; Policy iteration algorithm; Deep learning; value iteration; quantization

Domaines: Probabilités [math.PR]; Optimisation et contrôle [math.OC]; Finance quantitative [q-fin.CP]; Machine Learning [stat.ML]

Achref Bachouch, Côme Huré, Nicolas Langrené, Huyen Pham. Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications. Methodology and Computing in Applied Probability, In press. ⟨hal-01949221v3⟩