Investigating current-based and gating approaches for accurate and e-efficient spiking recurrent neural networks

Manon Dampfhoffer (1, 2) , Thomas Mesquida (2) , Alexandre Valentian (2) , Lorena Anghel (1)

1 SPINTEC - SPINtronique et TEchnologie des Composants
2 LSTA - Laboratoire Systèmes-sur-puce et Technologies Avancées

Spiking Neural Networks (SNNs) with spike-based computations and communications may be more energy-efficient than Artificial Neural Networks (ANNs) for embedded applications. However, SNNs have mostly been applied to image processing, although audio applications may better fit their temporal dynamics. We evaluate th e accuracy and energy-efficiency of Leaky Integrate-and-Fire (LIF) models on spiking audio datasets compared to ANNs. We demonstrate that, for processing temporal sequences, the Current-based LIF (Cuba-LIF) outperforms the LIF. Moreover, gated recurrent networks have demonstrated superior accuracy than simple recurrent networks for such tasks. Therefore, we introduce SpikGRU, a gated version of the Cuba-LIF. SpikGRU achieves higher accuracy than other recurrent SNNs on the most difficult task studied in this work. The Cuba-LIF and SpikGRU reach state-of-theart accuracy, only <1.1% below the accuracy of the best ANNs, while showing up to a 49x reduction in the number of operations compared to ANNs, due to the high spike sparsity.

Domaines: Intelligence artificielle [cs.AI]; Micro et nanotechnologies/Microélectronique

Manon Dampfhoffer, Thomas Mesquida, Alexandre Valentian, Lorena Anghel. Investigating current-based and gating approaches for accurate and e-efficient spiking recurrent neural networks. Lecture Notes in Computer Science, 2022, Artificial Neural Networks and Machine Learning – ICANN 2022, 13531, pp.359-370. ⟨10.1007/978-3-031-15934-3_30⟩. ⟨hal-03823943v2⟩