Towards Force Sensing From Vision: Observing Hand-Object Interactions to Infer Manipulation Forces

Tu-Hoa Pham 1, 2 Abderrahmane Kheddar 1, 2 Ammar Qammaz 3 Antonis Argyros 3, 4
1 CNRS-AIST JRL - Joint Robotics Laboratory [Japan]
2 IDH - Interactive Digital Humans
LIRMM - Laboratoire d'Informatique de Robotique et de Microélectronique de Montpellier
3 ICS-FORTH - Institute of Computer Science
4 Computer Science Department, University of Crete, Greece
Abstract : We present a novel, non-intrusive approach for estimating contact forces during hand-object interactions relying solely on visual input provided by a single RGB-D camera. We consider a manipulated object with known geometrical and physical properties. First, we rely on model-based visual tracking to estimate the object's pose together with that of the hand manipulating it throughout the motion. Following this, we compute the object's first and second order kinematics using a new class of numerical differentiation operators. The estimated kinematics is then instantly fed into a second-order cone program that returns a minimal force distribution explaining the observed motion. However, humans typically apply more forces than mechanically required when manipulating objects. Thus, we complete our estimation method by learning these excessive forces and their distribution among the fingers in contact. We provide a full validity analysis of the proposed method by evaluating it based on ground truth data from additional sensors such as accelerometers, gyroscopes and pressure sensors. Experimental results show that force sensing from vision (FSV) is indeed feasible.
Keywords : Force sensing manipulation Tracking
Type de document :
Communication dans un congrès
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015), Jun 2015, Boston, MA, United States. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, 〈http://www.pamitc.org/cvpr15/〉. 〈10.1109/CVPR.2015.7298898〉
Liste complète des métadonnées

Littérature citée [51 références]  Voir  Masquer  Télécharger
https://hal.archives-ouvertes.fr/hal-01356136
Contributeur : Tu-Hoa Pham <>
Soumis le : jeudi 25 août 2016 - 08:26:05
Dernière modification le : mardi 12 février 2019 - 12:28:03
Document(s) archivé(s) le : samedi 26 novembre 2016 - 12:57:51

Fichier

fsv_cvpr2015.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

IDH | LIRMM | MIPS | UNIV-MONTPELLIER

Citation

Tu-Hoa Pham, Abderrahmane Kheddar, Ammar Qammaz, Antonis Argyros. Towards Force Sensing From Vision: Observing Hand-Object Interactions to Infer Manipulation Forces. IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015), Jun 2015, Boston, MA, United States. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, 〈http://www.pamitc.org/cvpr15/〉. 〈10.1109/CVPR.2015.7298898〉. 〈hal-01356136〉

Exporter

BibTeX TEI DC DCterms EndNote

Partager

Métriques

Consultations de la notice

204

Téléchargements de fichiers

311

Contact

support.ccsd.cnrs.fr
hal.support@ccsd.cnrs.fr
Logo CCSD