PhD Forum: Why TanH can be a Hardware Friendly Activation Function for CNNs

Abstract : Convolutional Neural Network (CNN) techniques improved accuracy and robustness of machine vision systems at the price of a very high computational cost. This motivated multiple research efforts to investigate the applicability of approximate computing and more particularly, fixed point-arithmetic for CNNs. In all this approaches, a recurrent problem is that the learned parameters in deep CNN layers have a significantly lower numerical dynamic range when compared to the feature maps. This problem prevents from using of a low bit-width representation in deep layers. In this paper, we demonstrate that using the TanH activation function is way to prevent this issue. To support this demonstration, three benchmark CNN models are trained with the TanH function. These models are then quantized using the same bit-width across all the layers. Efficiency of this method is demonstrated on an FPGA based accelerator, by inferring CNNs with the minimal amount of logic elements.
Type de document :
Communication dans un congrès
Proceedings of the 11th International Conference on Distributed Smart Cameras - ICDSC 2017, Sep 2017, Stanford, CA, United States
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01654697
Contributeur : Kamel Abdelouahab <>
Soumis le : lundi 4 décembre 2017 - 11:57:22
Dernière modification le : vendredi 16 novembre 2018 - 01:25:31

Fichier

Abdelouahab17-ICDSC.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01654697, version 1

Citation

Kamel Abdelouahab, Maxime Pelcat, François Berry. PhD Forum: Why TanH can be a Hardware Friendly Activation Function for CNNs. Proceedings of the 11th International Conference on Distributed Smart Cameras - ICDSC 2017, Sep 2017, Stanford, CA, United States. 〈hal-01654697〉

Partager

Métriques

Consultations de la notice

538

Téléchargements de fichiers

2512