Silhouette-based Pose Estimation for Deformable Organs Application to Surgical Augmented Reality

Yinoussa Adagolodjo (1) , Raffaella Trivisonne (2, 3) , Nazim Haouchine (3) , Stéphane Cotin (3) , Hadrien Courtecuisse (1)

1 ICube - Laboratoire des sciences de l'ingénieur, de l'informatique et de l'imagerie
2 MAGRIT - Visual Augmentation of Complex Environments
3 MIMESIS - Computational Anatomy and Simulation for Medicine

— In this paper we introduce a method for semi-automatic registration of 3D deformable models using 2D shape outlines (silhouettes) extracted from a monocular camera view. Our framework is based on the combination of a biomechanical model of the organ with a set of projective constraints influencing the deformation of the model. To enforce convergence towards a global minimum for this ill-posed problem we interactively provide a rough (rigid) estimation of the pose. We show that our approach allows for the estimation of the non-rigid 3D pose while relying only on 2D information. The method is evaluated experimentally on a soft silicone gel model of a liver, as well as on real surgical data, providing augmented reality of the liver and the kidney using a monocular laparoscopic camera. Results show that the final elastic registration can be obtained in just a few seconds, thus remaining compatible with clinical constraints. We also evaluate the sensitivity of our approach according to both the initial alignment of the model and the silhouette length and shape.

Domaines: Sciences de l'ingénieur [physics]; Informatique [cs]; Mathématiques [math]

Yinoussa Adagolodjo, Raffaella Trivisonne, Nazim Haouchine, Stéphane Cotin, Hadrien Courtecuisse. Silhouette-based Pose Estimation for Deformable Organs Application to Surgical Augmented Reality. IROS 2017 - IEEE/RSJ International Conference on Intelligent Robots and Systems, Sep 2017, Vancouver, Canada. ⟨hal-01578815⟩