Hand-Object Contact Force Estimation From Markerless Visual Tracking

Abstract : We consider the problem of computing realistic contact forces during manipulation, backed with ground-truth measurements, using vision alone. Interaction forces are traditionally measured by mounting force transducers onto the manipulated objects or the hands. Those are costly, cumbersome, and alter the objects' physical properties and their perception by the human sense of touch. Our work establishes that interaction forces can be estimated in a cost-effective, reliable, non-intrusive way using vision. This is a complex and challenging problem. Indeed, in multi-contact, a given trajectory can generally be caused by an infinity of possible distributions. To alleviate the limitations of traditional models based on inverse optimization, we collect and release the first large-scale dataset on manipulation kinodynamics as 3.2 hours of synchronized force and motion measurements under 193 object-grasp configurations. We learn a mapping between high-level kinematic features based on the equations of motion and the underlying manipulation forces using recurrent neural networks (RNN). The RNN predictions are consistently refined using physics-based optimization through second-order cone programming (SOCP). We show that our method can successfully capture interaction forces compatible with both the observations and the way humans naturally manipulate objects, on an acquisition system no more complex than a single RGB-D camera.
Complete list of metadatas

Cited literature [77 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01356138
Contributor : Tu-Hoa Pham <>
Submitted on : Monday, September 25, 2017 - 5:53:01 AM
Last modification on : Wednesday, June 12, 2019 - 10:07:42 PM
Long-term archiving on : Tuesday, December 26, 2017 - 12:34:57 PM

File

fsv_tpami.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Tu-Hoa Pham, Nikolaos Kyriazis, Antonis Argyros, Abderrahmane Kheddar. Hand-Object Contact Force Estimation From Markerless Visual Tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers, 2018, 40 (12), pp.2883-2896. ⟨10.1109/TPAMI.2017.2759736⟩. ⟨hal-01356138v2⟩

Share

Metrics

Record views

603

Files downloads

532