Temporally Coherent Segmentation of 3D Reconstructions

Kiran Varanasi (1) , Edmond Boyer (1)

1 PERCEPTION - Interpretation and Modelling of Images and Videos

In this paper, we address the problem of segmenting consistently an evolving 3D scene reconstructed individually at different time-frames. The spatial reconstruction of 3D objects from multiple views has been studied extensively in the recent past. Various approaches exist to capture the performance of real actors into voxel-based or mesh-based representations. However, without any knowledge about the nature of the scene being observed, such reconstructions suffer from artifacts such as holes and topological inconsistencies. We explore the problem of segmenting such reconstructions in a temporally coherent manner, as a decomposition into rigidly moving parts. We work with mesh-based representations, though our method can be extended to other 3D representations as well. Unlike related works, our method is independent of the scene being observed, and can handle multiple actors interacting with each other. We also do not require an 'a priori' motion-estimate, which we compute simultaneously as we segment the scene. We individually segment each of the reconstructed scenes into approximately convex parts, and compute their reliability through rigid motion estimates over the sequence. We finally merge these various parts together into a holistic and consistent segmentation over the sequence. We present results on publicly available data sets.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Kiran Varanasi, Edmond Boyer. Temporally Coherent Segmentation of 3D Reconstructions. 3DPVT 2010 - 5th International Symposium on 3D Data Processing, Visualization and Transmission, May 2010, Paris, France. ⟨inria-00568905⟩