Performance of precision auto-tuned neural networks

Quentin Ferro (1) , Stef Graillat (1) , Thibault Hilaire (1) , Fabienne Jézéquel (1, 2)

1 PEQUAN - Performance et Qualité des Algorithmes Numériques
2 Université Paris-Panthéon-Assas

While often used in embedded systems, neural networks can be costly in terms of memory and execution time. Reducing the precision used in neural networks can be beneficial in terms of performance and energy consumption. After having applied a floating-point auto-tuning tool, PROMISE, on various neural networks, we obtained versions using lower precision while keeping a required accuracy on the results. In this article, we present results regarding the memory and computation time gains obtained thanks to reduced precision, using vectorized and non-vectorized code. We also show the impact on the execution time of PROMISE of the parallelization of the Delta Debug algorithm it implements.

Mots clés: Precision; Neural Networks; Auto-Tuning; Floating-Point; Stochastic Arithmetic

Domaines: Réseau de neurones [cs.NE]; Arithmétique des ordinateurs; Analyse numérique [cs.NA]; Performance et fiabilité [cs.PF]

Quentin Ferro, Stef Graillat, Thibault Hilaire, Fabienne Jézéquel. Performance of precision auto-tuned neural networks. MCSoC 2023 (16th IEEE International Symposium on Embedded Multicore/Manycore Systems-on-Chip), special session POAT (Performance Optimization and Auto-Tuning of Software on Multicore/Manycore Systems), Dec 2023, Singapore, Singapore. ⟨hal-04149501⟩