A Reliable Framework for Real-Time Physical Human-Robot Interaction using Hand Gestures

Osama Mazhar 1 Sofiane Ramdani 1 Benjamin Navarro 1 Robin Passama 1 Andrea Cherubini 1
1 IDH - Interactive Digital Humans
LIRMM - Laboratoire d'Informatique de Robotique et de Microélectronique de Montpellier
Abstract : A physical Human-Robot Interaction (pHRI) framework is proposed using vision and force sensors for a two-way object handover task. Kinect v2 is integrated with the state-of-the-art 2D skeleton extraction library namely Openpose to obtain a 3D skeleton of the human operator. A robust and rotation invariant (in the coronal plane) hand gesture recognition system is developed by exploiting a convolutional neural network. This network is trained such that the gestures can be recognized without the need to pre-process the RGB hand images at run time. This work establishes a firm basis for the robot control using hand-gestures. This will be extended for the development of intelligent human intention detection in pHRI scenarios to efficiently recognize a variety of static as well as dynamic gestures.
Complete list of metadatas

Cited literature [8 references]  Display  Hide  Download

Contributor : Andrea Cherubini <>
Submitted on : Sunday, March 25, 2018 - 5:51:57 AM
Last modification on : Sunday, April 14, 2019 - 2:37:43 PM
Long-term archiving on : Thursday, September 13, 2018 - 2:28:41 AM


Files produced by the author(s)


  • HAL Id : hal-01742458, version 1



Osama Mazhar, Sofiane Ramdani, Benjamin Navarro, Robin Passama, Andrea Cherubini. A Reliable Framework for Real-Time Physical Human-Robot Interaction using Hand Gestures. ARSO: Advanced Robotics and its Social Impacts, Sep 2018, Genova, Italy. ⟨hal-01742458⟩



Record views


Files downloads