A Multimodal Human-Robot Interaction Dataset

Abstract : This works presents a multimodal dataset for Human-Robot Interactive Learning. 1 The dataset contains synchronized recordings of several human users, from a stereo 2 microphone and three cameras mounted on the robot. The focus of the dataset is 3 incremental object learning, oriented to human-robot assistance and interaction. To 4 learn new object models from interactions with a human user, the robot needs to 5 be able to perform multiple tasks: (a) recognize the type of interaction (pointing, 6 showing or speaking), (b) segment regions of interest from acquired data (hands and 7 objects), and (c) learn and recognize object models. We illustrate the advantages 8 of multimodal data over camera-only datasets by presenting an approach that 9 recognizes the user interaction by combining simple image and language features.
Document type :
Poster communications
Complete list of metadatas

Cited literature [12 references]  Display  Hide  Download

https://hal.inria.fr/hal-01402479
Contributor : Florian Golemo <>
Submitted on : Wednesday, December 7, 2016 - 4:10:42 PM
Last modification on : Wednesday, July 3, 2019 - 10:48:04 AM
Long-term archiving on : Wednesday, March 22, 2017 - 11:49:41 PM

File

multimodal-dataset-nips(1).pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01402479, version 1

Citation

Pablo Azagra, Yoan Mollard, Florian Golemo, Ana Cristina Murillo, Manuel Lopes, et al.. A Multimodal Human-Robot Interaction Dataset. NIPS 2016, workshop Future of Interactive Learning Machines, Dec 2016, Barcelona, Spain. ⟨hal-01402479⟩

Share

Metrics

Record views

409

Files downloads

972