Skip to Main content Skip to Navigation
Conference papers

Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Prediction

Abstract : Leveraging physical knowledge described by partial differential equations (PDEs) is an appealing way to improve unsupervised video prediction methods. Since physics is too restrictive for describing the full visual content of generic videos, we introduce PhyDNet, a two-branch deep architecture , which explicitly disentangles PDE dynamics from unknown complementary information. A second contribution is to propose a new recurrent physical cell (PhyCell), inspired from data assimilation techniques, for performing PDE-constrained prediction in latent space. Extensive experiments conducted on four various datasets show the ability of PhyDNet to outperform state-of-the-art methods. Ab-lation studies also highlight the important gain brought out by both disentanglement and PDE-constrained prediction. Finally, we show that PhyDNet presents interesting features for dealing with missing data and long-term forecasting.
Complete list of metadata

Cited literature [95 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02947331
Contributor : Vincent Le Guen <>
Submitted on : Wednesday, September 23, 2020 - 10:13:05 PM
Last modification on : Friday, October 2, 2020 - 10:32:02 PM
Long-term archiving on: : Thursday, December 3, 2020 - 4:24:50 PM

File

Phydnet_paper.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Vincent Le Guen, Nicolas Thome. Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Prediction. Computer Vision and Pattern Recognition 2020 (CVPR), Jun 2020, Seattle, United States. ⟨10.1109/CVPR42600.2020.01149⟩. ⟨hal-02947331⟩

Share

Metrics