Realtime fusion of depth maps with a film camera in a popular game engine framework for pre-visualization compositing

Timothée de Goussencourt (1) , Pascal Bertolino (1)

1 GIPSA-AGPIG - GIPSA - Architecture, Géométrie, Perception, Images, Gestes

The global context of our work is the virtual production film industry. We present an efficient framework to merge a low resolution depth map sensor with a high resolution film camera. The depth sensor used is a Kinect 2, based on time of flight technology. Our method is especially designed for film production requiring live (pre) visualization. To achieve real-time performance we are not using a specific customized solution software but the very popular game engine Unity 3D. Our method is directly implemented into this game engine to give the user all the facilities of a traditional game engine.

Mots clés: Depth map; Real-time; Compositing; Virtual Production; Previz on-set; Unity

Domaines: Traitement du signal et de l'image [eess.SP]

Timothée de Goussencourt, Pascal Bertolino. Realtime fusion of depth maps with a film camera in a popular game engine framework for pre-visualization compositing. International Conference on Image Processing (ICIP), Sep 2015, Québec, Canada. ⟨hal-01208180v1⟩