Spatiotemporal features for asynchronous event-based data

Xavier Lagorce (1) , Sio-Hoi Ieng (1) , Xavier Clady (1) , Michael Pfeiffer (2) , Ryad B. Benosman (1)

1 Institut de la Vision
2 INI - Institute of Neuroinformatics [Zürich]

Bio-inspired asynchronous event-based vision sensors are currently introducing a paradigm shift in visual information processing. These new sensors rely on a stimulus-driven principle of light acquisition similar to biological retinas. They are event-driven and fully asynchronous, thereby reducing redundancy and encoding exact times of input signal changes, leading to a very precise temporal resolution. Approaches for higher-level computer vision often rely on the reliable detection of features in visual frames, but similar definitions of features for the novel dynamic and event-based visual input representation of silicon retinas have so far been lacking. This article addresses the problem of learning and recognizing features for event-based vision sensors, which capture properties of truly spatiotemporal volumes of sparse visual event information. A novel computational architecture for learning and encoding spatiotemporal features is introduced based on a set of predictive recurrent reservoir networks, competing via winner-take-all selection. Features are learned in an unsupervised manner from real-world input recorded with event-based vision sensors. It is shown that the networks in the architecture learn distinct and task-specific dynamic visual features, and can predict their trajectories over time.

Mots clés: echo-state networks; spatiotemporal; feature extraction; recognition; silicon retinas

Domaines: Neurosciences [q-bio.NC]

Xavier Lagorce, Sio-Hoi Ieng, Xavier Clady, Michael Pfeiffer, Ryad B. Benosman. Spatiotemporal features for asynchronous event-based data. Frontiers in Aging Neuroscience, 2015, 9, pp.46. ⟨10.3389/fnins.2015.00046⟩. ⟨hal-01221778⟩