Stability analysis of discrete-time finite-horizon discounted optimal control

Mathieu Granzotto (1) , Romain Postoyan (1) , Lucian Busoniu (2) , Dragan Nesic (3) , Jamal Daafouz (1)

1 CRAN - Centre de Recherche en Automatique de Nancy
2 UTCN - Universitatea Tehnica din Cluj-Napoca
3 Department of Electrical and Electronic Engineering [Melbourne]

Discounted costs are considered in many fields, like reinforcement learning, for which various algorithms can be used to obtain optimal inputs for finite horizons. The related literature mostly concentrates on optimality and largely ignores stability. In this context, we study stability of general nonlinear discrete-time systems controlled by an optimal sequence of inputs that minimizes a finite-horizon discounted cost computed in a receding horizon fashion. Assumptions are made related to the stabilizability of the system and its detectability with respect to the stage cost. Then, a Lyapunov function for the closed-loop system with the receding horizon controller is constructed and a uniform semiglobal stability property is ensured, where the adjustable parameters are both the discount factor and the horizon length. Uniform global exponential stability is guaranteed by strengthening the initial assumptions, in which case explicit bounds on the discount factor and the horizon length are provided. We compare the obtained bounds in the particular cases where there is no discount or the horizon is infinite, respectively, with related results in the literature and we show our bounds improve existing ones on the examples considered.

Domaines: Automatique / Robotique

Mathieu Granzotto, Romain Postoyan, Lucian Busoniu, Dragan Nesic, Jamal Daafouz. Stability analysis of discrete-time finite-horizon discounted optimal control. 57th IEEE Conference on Decision and Control, CDC 2018, Dec 2018, Miami, FL, United States. ⟨hal-01877140⟩