Constraining the Connectivity of Sparse Neural Associative Memories

Philippe Tigréat (1, 2) , Vincent Gripon (2) , Pierre-Henri Horrein (2, 1)

1 ELEC - Département Electronique
2 Lab-STICC_TB_CACS_IAS

Willshaw networks are a type of associative memories with a storing mechanism characterized by a strong redundancy. Namely, all the subparts of a message get connected to one another. We introduce an additional specificity, by imposing the constraint of a minimal space separating every two elements of a message. This approach results from biological observations, knowing that in some brain regions, a neuron receiving a stronger stimulation can inhibit its neighbors within a given radius. Theoretical arguments are derived to quantify the benefits of this method in terms of memory usage as well as pattern completion ability. We experiment with different values of the inhibition radius introduced, and we study its impact on the error rate in the retrieval of stored messages. We show that this added constraint can result in significatively better performance of the Willshaw network, either when reducing its set of connections, or when extending its set of neurons while maintaining the memory resource.

Mots clés: Willshaw Networks; Clique-Based Neural Networks; Content-Adressable Memory; Lateral Inhibition; Sparse neural networks

Domaines: Réseau de neurones [cs.NE]; Algorithme et structure de données [cs.DS]

Philippe Tigréat, Vincent Gripon, Pierre-Henri Horrein. Constraining the Connectivity of Sparse Neural Associative Memories. International Journal On Advances in Life Sciences, 2016, 8 (3), pp.203-213. ⟨hal-01499953⟩