DIP-VBTV: A Color Image Restoration Model combining a Deep Image Prior and a Vector Bundle Total Variation

Thomas Batard (1) , Gloria Haro (2) , Coloma Ballester (2)

1 CVC - Computer Vision Center (Centre de visio per computador)
2 UPF - Universitat Pompeu Fabra [Barcelona]

In this paper, we introduce a new variational model for color image restoration, called DIP-VBTV, which combines two priors: a deep image prior (DIP), which assumes that the restored image can be generated through a neural network and a Vector Bundle Total Variation (VBTV) which generalizes the Vectorial Total Variation (VTV) on vector bundles. VBTV is determined by a geometric triplet: a Riemannian metric on the base manifold, a covariant derivative and a metric on the vector bundle. Whereas VTV prior encourages the restored images to be piece-wise constant, VBTV prior encourages them to be piece-wise parallel with respect to a covariant derivative. For well-chosen geometric triplets, we show that the minimization of VBTV encourages the solutions of the restoration model to share some visual content with the clean image. Then, we show on experiments that DIP-VBTV benefits from this property by outperforming DIP-VTV and state-of-the-art unsupervised methods, which demonstrates the relevance of combining DIP and VBTV priors.

Domaines: Traitement des images [eess.IV]

Thomas Batard, Gloria Haro, Coloma Ballester. DIP-VBTV: A Color Image Restoration Model combining a Deep Image Prior and a Vector Bundle Total Variation. 2021. ⟨hal-02994439v3⟩