Exploiting locality and translational invariance to design effective Deep Reinforcement Learning control of the 1-dimensional unstable falling liquid film

Vincent Belus (1, 2) , Jean Rabault (1, 2) , Jonathan Viquerat (2, 3) , Zhizhao Che (4) , Elie Hachem (2) , Ulysse Reglade (2, 1)

1 UiO - University of Oslo
2 CEMEF - Centre de Mise en Forme des Matériaux
3 NACHOS - Numerical modeling and high performance computing for evolution problems in complex domains and heterogeneous media
4 TJU - Tianjin University

Instabilities arise in a number of flow configurations. One such manifestation is the development of interfacial waves in multiphase flows, such as those observed in the falling liquid film problem. Controlling the development of such instabilities is a problem of both academic and industrial interest. However, this has proven challenging in most cases due to the strong nonlinearity and high dimensionality of the underlying equations. In the present work, we successfully apply Deep Reinforcement Learning (DRL) for the control of the one-dimensional (1D) depth-integrated falling liquid film. In addition, we introduce for the first time translational invariance in the architecture of the DRL agent, and we exploit locality of the control problem to define a dense reward function. This allows to both speed up learning considerably, and to easily control an arbitrary large number of jets and overcome the curse of dimensionality on the control output size that would take place using a naive approach. This illustrates the importance of the architecture of the agent for successful DRL control, and we believe this will be an important element in the effective application of DRL to large two-dimensional (2D) or three-dimensional (3D) systems featuring translational, axisymmetric or other invariance.

Domaines: Mécanique des fluides [physics.class-ph]; Physique mathématique [math-ph]; Modélisation et simulation; Intelligence artificielle [cs.AI]

Vincent Belus, Jean Rabault, Jonathan Viquerat, Zhizhao Che, Elie Hachem, et al.. Exploiting locality and translational invariance to design effective Deep Reinforcement Learning control of the 1-dimensional unstable falling liquid film. AIP Advances, 2019, 9, pp.125014. ⟨10.1063/1.5132378⟩. ⟨hal-02401501⟩