Skip to Main content Skip to Navigation
Conference papers

Auto-CNNp: a component-based framework for automating CNN parallelism

Abstract : Effectively training of Convolutional Neural Networks (CNNs) is a computationally intensive and time-consuming task. Therefore, scaling up the training of CNNs has become a key approach to decrease the training duration and train CNN models in a reasonable time. Nevertheless, introducing parallelism to CNNs is a laborious task in practice. It is a manual, repetitive and error-prone process. In this paper, we present Auto-CNNp, a novel framework that aims to address this challenge by automating CNNs training parallelization task. To achieve this goal, the Auto-CNNp introduces a key component which is called CNN-Parallelism-Generator. The latter component aims to streamline routine tasks throughout (1) capturing cumbersome CNNs parallelization tasks within a backbone structure while (2) keeping the framework flexible enough and extensible for user-specific personalization. Our proposed reference implementation provides a high level of abstraction over MPI-based CNNs parallelization process, despite the CNN-based imaging task and its related architecture and training dataset. We introduce the design and the core building blocks of Auto-CNNp. We further conduct an extensive assessment of our proposal that not only shows its effectiveness in accelerating the process of scaling up CNNs training, but also its generalization for a wider variety of use cases.
Complete list of metadata

Cited literature [44 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02894478
Contributor : Soulaimane Guedria <>
Submitted on : Thursday, July 9, 2020 - 3:12:31 PM
Last modification on : Wednesday, December 9, 2020 - 3:51:15 PM
Long-term archiving on: : Monday, November 30, 2020 - 6:04:48 PM

File

Auto_CNNp_IEEE_Extended.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Soulaimane Guedria, Noel de Palma, Felix Renard, Nicolas Vuillerme. Auto-CNNp: a component-based framework for automating CNN parallelism. 2019 IEEE International Conference on Big Data (Big Data), Dec 2019, Los Angeles, United States. pp.3330-3339, ⟨10.1109/BigData47090.2019.9006175⟩. ⟨hal-02894478⟩

Share

Metrics

Record views

96

Files downloads

93