ModeNet: Mode Selection Network For Learned Video Coding

Théo Ladune (1, 2) , Pierrick Philippe (2) , Wassim Hamidouche (1) , Lu Zhang (1) , Olivier Déforges (1)

1 IETR - Institut d'Électronique et des Technologies du numéRique
2 Orange Labs R&D [Rennes]

In this paper, a mode selection network (ModeNet) is proposed to enhance deep learning-based video compression. Inspired by traditional video coding, ModeNet purpose is to enable competition among several coding modes. The proposed ModeNet learns and conveys a pixel-wise partitioning of the frame, used to assign each pixel to the most suited coding mode. ModeNet is trained alongside the different coding modes to minimize a rate-distortion cost. It is a flexible component which can be generalized to other systems to allow competition between different coding tools. Mod-eNet interest is studied on a P-frame coding task, where it is used to design a method for coding a frame given its prediction. ModeNet-based systems achieve compelling performance when evaluated under the Challenge on Learned Image Compression 2020 (CLIC20) P-frame coding track conditions.

Mots clés: Video Coding; Autoencoder; Deep Learning; Coding Mode Selection

Domaines: Traitement du signal et de l'image [eess.SP]; Réseau de neurones [cs.NE]

Théo Ladune, Pierrick Philippe, Wassim Hamidouche, Lu Zhang, Olivier Déforges. ModeNet: Mode Selection Network For Learned Video Coding. Machine Learning for Signal Processing (MLSP) 2020, Sep 2020, Espoo, Finland. ⟨hal-02888453v2⟩