Skip to Main content Skip to Navigation

Exploiting Imprecise Information Sources in Sequential Decision Making Problems under Uncertainty

Abstract : Partially Observable Markov Decision Processes (POMDPs) define a useful formalism to express probabilistic sequential decision problems under uncertainty. When this model is used for a robotic mission, the system is defined as the features of the robot and its environment, needed to express the mission. The system state is not directly seen by the agent (the robot). Solving a POMDP consists thus in computing a strategy which, on average, achieves the mission best i.e. a function mapping the information known by the agent to an action. Some practical issues of the POMDP model are first highlighted in the robotic context: it concerns the modeling of the agent ignorance, the imprecision of the observation model and the complexity of solving real world problems. A counterpart of the POMDP model, called π-POMDP, simplifies uncertainty representation with a qualitative evaluation of event plausibilities. It comes from Qualitative Possibility Theory which provides the means to model imprecision and ignorance. After a formal presentation of the POMDP and π-POMDP models, an update of the possibilistic model is proposed. Next, the study of factored π-POMDPs allows to set up an algorithm named PPUDD which uses Algebraic Decision Diagrams to solve large structured planning problems. Strategies computed by PPUDD, which have been tested in the context of the competition IPPC 2014, can be more efficient than those produced by probabilistic solvers when the model is imprecise or for high dimensional problems. We show next that the π-Hidden Markov Processes (π-HMP), i.e. π-POMDPs without action, produces useful diagnosis in the context of Human-Machine interactions. Finally, a hybrid POMDP benefiting from the possibilistic and the probabilistic approach is built: the qualitative framework is only used to maintain the agent’s knowledge. This leads to a strategy which is pessimistic facing the lack of knowledge, and computable with a solver of fully observable Markov Decision Processes (MDPs). This thesis proposes some ways of using Qualitative Possibility Theory to improve computation time and uncertainty modeling in practice.
Complete list of metadata

Cited literature [156 references]  Display  Hide  Download
Contributor : Cécile André Connect in order to contact the contributor
Submitted on : Friday, April 1, 2016 - 3:07:16 PM
Last modification on : Thursday, October 28, 2021 - 3:16:37 PM
Long-term archiving on: : Monday, November 14, 2016 - 12:33:33 PM


  • HAL Id : tel-01296749, version 1



Nicolas Drougard. Exploiting Imprecise Information Sources in Sequential Decision Making Problems under Uncertainty. Space Physics []. Institut Supérieur de l'Aéronautique et de l'Espace (ISAE), 2015. English. ⟨tel-01296749⟩



Record views


Files downloads