Skip to Main content Skip to Navigation
Journal articles

Hand-Object Contact Force Estimation From Markerless Visual Tracking

Abstract : We consider the problem of computing realistic contact forces during manipulation, backed with ground-truth measurements, using vision alone. Interaction forces are traditionally measured by mounting force transducers onto the manipulated objects or the hands. Those are costly, cumbersome, and alter the objects' physical properties and their perception by the human sense of touch. Our work establishes that interaction forces can be estimated in a cost-effective, reliable, non-intrusive way using vision. This is a complex and challenging problem. Indeed, in multi-contact, a given trajectory can generally be caused by an infinity of possible distributions. To alleviate the limitations of traditional models based on inverse optimization, we collect and release the first large-scale dataset on manipulation kinodynamics as 3.2 hours of synchronized force and motion measurements under 193 object-grasp configurations. We learn a mapping between high-level kinematic features based on the equations of motion and the underlying manipulation forces using recurrent neural networks (RNN). The RNN predictions are consistently refined using physics-based optimization through second-order cone programming (SOCP). We show that our method can successfully capture interaction forces compatible with both the observations and the way humans naturally manipulate objects, on an acquisition system no more complex than a single RGB-D camera.
Complete list of metadata

Cited literature [77 references]  Display  Hide  Download
Contributor : Tu-Hoa Pham Connect in order to contact the contributor
Submitted on : Monday, September 25, 2017 - 5:53:01 AM
Last modification on : Wednesday, November 3, 2021 - 7:45:32 AM
Long-term archiving on: : Tuesday, December 26, 2017 - 12:34:57 PM


Files produced by the author(s)




Tu-Hoa Pham, Nikolaos Kyriazis, Antonis Argyros, Abderrahmane Kheddar. Hand-Object Contact Force Estimation From Markerless Visual Tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers, 2018, 40 (12), pp.2883-2896. ⟨10.1109/TPAMI.2017.2759736⟩. ⟨hal-01356138v2⟩



Les métriques sont temporairement indisponibles