Interactive Multicut Video Segmentation

Evgeny E Levinkov (1) , James Tompkin (2) , Nicolas Bonneel (3, 4) , Steffen Kirchhoff (2) , Bjoern E Andres (1) , Hanspeter Pfister (2)

1 Max Planck Institute for Informatics [Saarbrücken]
2 SEAS - Harvard John A. Paulson School of Engineering and Applied Sciences
3 GeoMod - Modélisation Géométrique, Géométrie Algorithmique, Fractales
4 M2DisCo - Geometry Processing and Constrained Optimization

Video segmentation requires separating foreground from background, but the general problem extends to more complicated scene segmentations of different objects and their multiple parts. We develop a new approach to interactive multi-label video segmentation where many objects are segmented simultaneously with consistent spatio-temporal boundaries, based on intuitive multi-colored brush scribbles. From these scribbles, we derive constraints to define a combinatorial problem known as the multicut—a problem notoriously difficult and slow to solve. We describe a solution using efficient heuristics to make multi-label video segmentation interactive. As our solution generalizes typical binary segmentation tasks, while also improving efficiency in multi-label tasks, our work shows the promise of multicuts for interactive video segmentation.

Domaines: Traitement des images [eess.IV]; Vision par ordinateur et reconnaissance de formes [cs.CV]; Mathématique discrète [cs.DM]

Evgeny E Levinkov, James Tompkin, Nicolas Bonneel, Steffen Kirchhoff, Bjoern E Andres, et al.. Interactive Multicut Video Segmentation. Pacific Graphics, Oct 2016, Okinawa, Japan. pp.33-38. ⟨hal-01378800⟩