A Framework for Real-Time Physical Human-Robot Interaction using Hand Gestures

Osama Mazhar (1) , Sofiane Ramdani (1) , Benjamin Navarro (1) , Robin Passama (1) , Andrea Cherubini (1)

1 IDH - Interactive Digital Humans

A physical Human-Robot Interaction (pHRI) framework is proposed using vision and force sensors for a two-way object hand-over task. Kinect v2 is integrated with the state-of-the-art 2D skeleton extraction library namely Openpose to obtain a 3D skeleton of the human operator. A robust and rotation invariant (in the coronal plane) hand gesture recognition system is developed by exploiting a convolutional neural network. This network is trained such that the gestures can be recognized without the need to pre-process the RGB hand images at run time. This work establishes a firm basis for the robot control using hand-gestures. This will be extended for the development of intelligent human intention detection in pHRI scenarios to efficiently recognize a variety of static as well as dynamic gestures.

Domaines: Robotique [cs.RO]; Vision par ordinateur et reconnaissance de formes [cs.CV]; Automatique / Robotique

Osama Mazhar, Sofiane Ramdani, Benjamin Navarro, Robin Passama, Andrea Cherubini. A Framework for Real-Time Physical Human-Robot Interaction using Hand Gestures. ARSO: Advanced Robotics and its Social Impacts, Sep 2018, Genova, Italy. pp.46-47, ⟨10.1109/ARSO.2018.8625753⟩. ⟨hal-01827254⟩