Visuo-Tactile Recognition of Daily-Life Objects Never Seen or Touched Before

Zineb Abderrahmane (1) , Gowrishankar Ganesh (2, 1) , André Crosnier (1) , Andrea Cherubini (1)

1 IDH - Interactive Digital Humans
2 CNRS-AIST JRL - Joint Robotics Laboratory

This study proposes a visuo-tactile Zero-Shot object recognition framework. The proposed framework recognizes a set of novel objects for which no tactile or visual training data are available. It uses visuo-tactile training data collected from known objects to recognize the novel ones, given their attributes. This framework extends the haptic Zero-Shot Learning framework that we proposed in [1] with vision, which enables a multimodal recognition system. In our test with the PHAC-2 dataset, the system was able to get a recognition accuracy of 72% among 6 objects that were never touched or seen during the training phase.

Domaines: Robotique [cs.RO]; Vision par ordinateur et reconnaissance de formes [cs.CV]; Apprentissage [cs.LG]

Zineb Abderrahmane, Gowrishankar Ganesh, André Crosnier, Andrea Cherubini. Visuo-Tactile Recognition of Daily-Life Objects Never Seen or Touched Before. ICARCV 2018 - 15th International Conference on Control, Automation, Robotics and Vision, Nov 2018, Singapore, Singapore. pp.1765-1770, ⟨10.1109/ICARCV.2018.8581230⟩. ⟨hal-01869015⟩