AssemblyNet: A Novel Deep Decision-Making Process for Whole Brain MRI Segmentation

Pierrick Coupé (1, 2) , Boris Mansencal (1) , Michaël Clément (3) , Rémi Giraud (1) , Baudouin Denis de Senneville (4) , Vinh-Thong Ta (1) , Vincent Lepetit (1) , José V Manjón (5)

1 LaBRI - Laboratoire Bordelais de Recherche en Informatique
2 PICTURA - Patch-based processing for medical and natural images
3 LIPADE - EA 2517 - Laboratoire d'Informatique Paris Descartes
4 IMB - Institut de Mathématiques de Bordeaux
5 UPV - Universitat Politècnica de València = Universitad Politecnica de Valencia = Polytechnic University of Valencia

Whole brain segmentation using deep learning (DL) is a very challenging task since the number of anatomical labels is very high compared to the number of available training images. To address this problem, previous DL methods proposed to use a global convolution neural network (CNN) or few independent CNNs. In this paper, we present a novel ensemble method based on a large number of CNNs processing different overlapping brain areas. Inspired by parliamentary decision-making systems, we propose a framework called Assem-blyNet, made of two "assemblies" of U-Nets. Such a parliamentary system is capable of dealing with complex decisions and reaching a consensus quickly. AssemblyNet introduces sharing of knowledge among neighboring U-Nets, an "amendment" procedure made by the second assembly at higher-resolution to refine the decision taken by the first one, and a final decision obtained by majority voting. When using the same 45 training images, AssemblyNet outperforms global U-Net by 28% in terms of the Dice metric, patch-based joint label fusion by 15% and SLANT-27 by 10%. Finally, AssemblyNet demonstrates high capacity to deal with limited training data to achieve whole brain segmentation in practical training and testing times.

Mots clés: Whole brain segmentation; CNN; Ensemble learning; transfer learning; multiscale framework

Domaines: Imagerie médicale; Intelligence artificielle [cs.AI]

Pierrick Coupé, Boris Mansencal, Michaël Clément, Rémi Giraud, Baudouin Denis de Senneville, et al.. AssemblyNet: A Novel Deep Decision-Making Process for Whole Brain MRI Segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Oct 2019, Shenzhen, China. pp.466-474, ⟨10.1007/978-3-030-32248-9_52⟩. ⟨hal-02358626⟩