On the Applications of Robust PCA in Image and Video Processing

Thierry Bouwmans (1) , Sajid Javed (2) , Hongyang Zhang , Zhouchen Lin , Ricardo Otazo

1 MIA - Mathématiques, Image et Applications - EA 3165
2 KNU - Kyungpook National University [Daegu]

Robust PCA (RPCA) via decomposition into low-rank plus sparse matrices offers a powerful framework for a large variety of applications such as image processing, video processing and 3D computer vision. Indeed, most of the time these applications require to detect sparse outliers from the observed imagery data that can be approximated by a low-rank matrix. Moreover, most of the time experiments show that RPCA with additional spatial and/or temporal constraints often outperforms the state-of-the-art algorithms in these applications. Thus, the aim of this paper is to survey the applications of RPCA in computer vision. In the first part of this paper, we review representative image processing applications as follows: (1) low-level imaging such as image recovery and denoising, image composition , image colorization, image alignment and rectification, multi-focus image and face recognition, (2) medical imaging like dynamic Magnetic Resonance Imaging (MRI) for acceleration of data acquisition, background suppression and learning of inter-frame motion fields, and (3) imaging for 3D computer vision with additional depth information like in Structure from Motion (SfM) and 3D motion recovery. In the second part, we present the applications of RPCA in video processing which utilize additional spatial and temporal information compared to image processing. Specifically, we investigate video denoising and restoration, hyperspectral video and background/foreground separation. Finally, we provide perspectives on possible future research directions and algorithmic frameworks that are suitable for these applications.

Mots clés: Robust PCA; Image processing; Video process-ing; 3D Computer Vision; Medical Imaging

Domaines: Traitement des images [eess.IV]

Thierry Bouwmans, Sajid Javed, Hongyang Zhang, Zhouchen Lin, Ricardo Otazo. On the Applications of Robust PCA in Image and Video Processing. Proceedings of the IEEE, 2018, 106 (8), pp.1427 - 1457. ⟨10.1109/JPROC.2018.2853589⟩. ⟨hal-01891028⟩