Neural Network Precision Tuning Using Stochastic Arithmetic

Quentin Ferro (1) , Stef Graillat (1) , Thibault Hilaire (1) , Fabienne Jézéquel (1, 2) , Basile Lewandowski (1)

1 PEQUAN - Performance et Qualité des Algorithmes Numériques
2 Université Paris-Panthéon-Assas

Neural networks can be costly in terms of memory and execution time. Reducing their cost has become an objective, especially when integrated in an embedded system with limited resources. A possible solution consists in reducing the precision of their neurons parameters. In this article, we present how to use auto-tuning on neural networks to lower their precision while keeping an accurate output. To do so, we use a floating-point auto-tuning tool on different kinds of neural networks. We show that, to some extent, we can lower the precision of several neural network parameters without compromising the accuracy requirement.

Mots clés: Stochastic Arithmetic; Floating-Point; Auto-Tuning; Neural Networks; Precision

Domaines: Analyse numérique [cs.NA]; Arithmétique des ordinateurs

Quentin Ferro, Stef Graillat, Thibault Hilaire, Fabienne Jézéquel, Basile Lewandowski. Neural Network Precision Tuning Using Stochastic Arithmetic. NSV'22, 15th International Workshop on Numerical Software Verification,, Aug 2022, Haifa, Israel. ⟨hal-03682645v2⟩