A. Gupta, A. Kembhavi, and L. Davis, Observing Human-Object Interactions: Using Spatial and Functional Compatibility for Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.31, issue.10, 2009.
DOI : 10.1109/TPAMI.2009.83

Y. Zhu, Y. Zhao, and S. C. Zhu, Understanding tools: Taskoriented object modeling, learning and recognition, IEEE Conference on Computer Vision and Pattern Recognition, 2015.

C. Ye, Y. Yang, C. Fermuller, and Y. Aloimonos, What can i do around here? deep functional scene understanding for cognitive robots, 2016.

X. Niu, A. V. Terekhov, M. L. Latash, and V. M. Zatsiorsky, Reconstruction of the Unknown Optimization Cost Functions from Experimental Recordings during Static Multi-Finger Prehension, Motor Control, vol.16, issue.2, 2012.
DOI : 10.1123/mcj.16.2.195

G. Slota, M. Latash, and V. Zatsiorsky, Grip forces during object manipulation: experiment, mathematical model, and validation, Experimental Brain Research, vol.30, issue.1, 2011.
DOI : 10.1007/s00221-011-2784-y

T. Pham, A. Kheddar, A. Qammaz, and A. A. Argyros, Towards force sensing from vision: Observing hand-object interactions to infer manipulation forces, IEEE Conference on Computer Vision and Pattern Recognition, 2015.
URL : https://hal.archives-ouvertes.fr/hal-01356136

S. A. Mascaro and H. H. Asada, Photoplethysmograph fingernail sensors for measuring finger forces without haptic obstruction, IEEE Transactions on Robotics and Automation, vol.17, issue.5, 2001.
DOI : 10.1109/70.964669

Y. Sun, J. M. Hollerbach, and S. A. Mascaro, Estimation of fingertip force direction with computer vision, IEEE Trans. on Robotics, vol.25, issue.6, 2009.

S. Urban, J. Bayer, C. Osendorfer, G. Westling, B. B. Edin et al., Computing grip force and torque from finger nail images using Gaussian processes, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013.
DOI : 10.1109/IROS.2013.6696933

P. G. Kry and D. K. Pai, Interaction capture and synthesis, ACM Trans. on Graphics, vol.25, issue.3, 2006.
URL : https://hal.archives-ouvertes.fr/Inria-00402113

J. M. Rehg and T. Kanade, Visual tracking of high DOF articulated structures: An application to human hand tracking, European Conference on Computer Vision, 1994.
DOI : 10.1007/BFb0028333

I. Oikonomidis, N. Kyriazis, and A. A. Argyros, Efficient modelbased 3d tracking of hand articulations using kinect, British Machine Vision Conference. BMVA, 2011.

M. De-la-gorce, D. J. Fleet, and N. Paragios, Model-Based 3D Hand Pose Estimation from Monocular Video, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.33, issue.9, 2011.
DOI : 10.1109/TPAMI.2011.33
URL : https://hal.archives-ouvertes.fr/hal-00856313

C. Keskin, F. K?raç, Y. E. Kara, and L. Akarun, Hand Pose Estimation and Hand Shape Classification Using Multi-layered Randomized Decision Forests, European Conference on Computer Vision, 2012.
DOI : 10.1007/978-3-642-33783-3_61

C. Qian, X. Sun, Y. Wei, X. Tang, and J. Sun, Realtime and Robust Hand Tracking from Depth, 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014.
DOI : 10.1109/CVPR.2014.145

D. Tang, H. J. Chang, A. Tejani, and T. Kim, Latent Regression Forest: Structured Estimation of 3D Articulated Hand Posture, 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014.
DOI : 10.1109/CVPR.2014.490

J. Tompson, M. Stein, Y. Lecun, and K. Perlin, Real-Time Continuous Pose Recovery of Human Hands Using Convolutional Networks, ACM Transactions on Graphics, vol.33, issue.5, 2014.
DOI : 10.1145/2629500

P. Krejov, A. Gilbert, and R. Bowden, Combining discriminative and model based approaches for hand pose estimation, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 2015.
DOI : 10.1109/FG.2015.7163141

G. Rogez, J. S. Supancic, and D. Ramanan, Understanding Everyday Hands in Action from RGB-D Images, 2015 IEEE International Conference on Computer Vision (ICCV), 2015.
DOI : 10.1109/ICCV.2015.443
URL : https://hal.archives-ouvertes.fr/hal-01237011

T. Sharp, C. Keskin, D. Robertson, J. Taylor, J. Shotton et al., Accurate, Robust, and Flexible Real-time Hand Tracking, Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI '15, 2015.
DOI : 10.1145/2702123.2702179

J. Taylor, L. Bordeaux, T. Cashman, B. Corish, C. Keskin et al., Efficient and precise interactive hand tracking through joint, continuous optimization of pose and correspondences, ACM Transactions on Graphics, vol.35, issue.4, 2016.
DOI : 10.1145/2897824.2925965

I. Oikonomidis, N. Kyriazis, and A. Argyros, Full dof tracking of a hand interacting with an object by modeling occlusions and physical constraints Tracking the articulated motion of two strongly interacting hands, IEEE International Conference on Computer Vision IEEE Conference on Computer Vision and Pattern Recognition, 2011.

D. Tzionas, L. Ballan, A. Srikantha, P. Aponte, M. Pollefeys et al., Capturing Hands in Action Using Discriminative Salient Points and Physics Simulation, International Journal of Computer Vision, vol.28, issue.3, 2015.
DOI : 10.1007/s11263-016-0895-4

S. Sridhar, F. Mueller, M. Zollhöferzollh¨zollhöfer, D. Casas, A. Oulasvirta et al., Real-Time Joint Tracking of a Hand Manipulating an Object from RGB-D Input, European Conference on Computer Vision, 2016.
DOI : 10.1007/978-3-319-11752-2_22

N. Kyriazis and A. Argyros, Physically Plausible 3D Scene Tracking: The Single Actor Hypothesis, 2013 IEEE Conference on Computer Vision and Pattern Recognition, 2013.
DOI : 10.1109/CVPR.2013.9

W. Zhao, J. Zhang, J. Min, and J. Chai, Robust realtime physicsbased motion control for human grasping, ACM Trans. on Graphics, vol.32, issue.6, 2013.

Y. Wang, J. Min, J. Zhang, Y. Liu, F. Xu et al., Videobased hand manipulation capture through composite motion control, ACM Trans. on Graphics, vol.32, issue.4, 2013.

M. A. Arbib, T. Iberall, and D. Lyons, Coordinated Control Programs for Movements of the Hand, 1985.
DOI : 10.1007/978-3-642-70105-4_7

F. Gao, M. L. Latash, and V. M. Zatsiorsky, Internal forces during object manipulation, Experimental Brain Research, vol.30, issue.1, 2005.
DOI : 10.1007/s00221-005-2282-1

J. Kerr and B. Roth, Analysis of Multifingered Hands, The International Journal of Robotics Research, vol.1, issue.1, 1986.
DOI : 10.1177/027836498600400401

T. Yoshikawa and K. Nagai, Manipulating and grasping forces in manipulation by multifingered robot hands, IEEE Transactions on Robotics and Automation, vol.7, issue.1, 1991.
DOI : 10.1109/70.68071

M. T. Mason and J. K. Salisbury, Robot Hands and the Mechanics of Manipulation, Journal of Dynamic Systems, Measurement, and Control, vol.111, issue.1, 1985.
DOI : 10.1115/1.3153010

R. M. Murray, S. S. Sastry, and L. Zexiang, A Mathematical Introduction to Robotic Manipulation, 1994.

J. R. Flanagan and R. S. Johansson, Hand Movements, Encyclopedia of the human brain, 2002.
DOI : 10.1016/B0-12-227210-2/00157-6

S. L. Gorniak, V. M. Zatsiorsky, and M. L. Latash, Manipulation of a fragile object, Experimental Brain Research, vol.13, issue.2, 2010.
DOI : 10.1007/s00221-009-2148-z

J. Park, T. Singh, V. M. Zatsiorsky, and M. L. Latash, Optimality versus variability: effect of fatigue in multi-finger redundant tasks, Experimental Brain Research, vol.99, issue.4, 2012.
DOI : 10.1007/s00221-011-2963-x

B. I. Prilutsky and V. M. Zatsiorsky, Optimization-Based Models of Muscle Coordination, Exercise and sport sciences reviews, 2002.
DOI : 10.1097/00003677-200201000-00007

M. S. Lobo, L. Vandenberghe, S. P. Boyd, and H. Lebret, Applications of second-order cone programming, Linear Algebra and its Applications, vol.284, issue.1-3, 1998.
DOI : 10.1016/S0024-3795(98)10032-0

S. P. Boyd and B. Wegbreit, Fast Computation of Optimal Contact Forces, IEEE Transactions on Robotics, vol.23, issue.6, 2007.
DOI : 10.1109/TRO.2007.910774

I. Lenz, H. Lee, and A. Saxena, Deep learning for detecting robotic grasps, International Journal of Robotics Research, vol.34, issue.4, 2015.

H. S. Koppula and A. Saxena, Anticipating human activities using object affordances for reactive robotic response, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol.38, issue.1, 2016.

Y. Bengio, A. Courville, and P. Vincent, Representation Learning: A Review and New Perspectives, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.35, issue.8, 2013.
DOI : 10.1109/TPAMI.2013.50

M. A. Brubaker, L. Sigal, and D. J. Fleet, Estimating contact dynamics, 2009 IEEE 12th International Conference on Computer Vision, 2009.
DOI : 10.1109/ICCV.2009.5459407

M. Mohammadi, T. L. Baldi, S. Scheggi, and D. Prattichizzo, Fingertip force estimation via inertial and magnetic sensors in deformable object manipulation, 2016 IEEE Haptics Symposium (HAPTICS), 2016.
DOI : 10.1109/HAPTICS.2016.7463191

Y. Zhu, C. Jiang, Y. Zhao, D. Terzopoulos, and S. Zhu, Inferring Forces and Learning Human Utilities from Videos, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
DOI : 10.1109/CVPR.2016.415

H. S. Koppula, A. Anand, T. Joachims, and A. Saxena, Semantic labeling of 3d point clouds for indoor scenes, Advances in Neural Information Processing Systems, 2011.

K. Lai, L. Bo, and D. Fox, Unsupervised feature learning for 3D scene labeling, 2014 IEEE International Conference on Robotics and Automation (ICRA), 2014.
DOI : 10.1109/ICRA.2014.6907298

J. Sung, C. Ponce, B. Selman, and A. Saxena, Unstructured human activity detection from RGBD images, IEEE International Conference on Robotics and Automation, 2012.

A. Saxena, J. Driemeyer, and A. Y. Ng, Robotic Grasping of Novel Objects using Vision, The International Journal of Robotics Research, vol.13, issue.3, 2008.
DOI : 10.1177/0278364907087172

B. , A. Walsman, A. Singh, S. Srinivasa, P. Abbeel et al., Benchmarking in manipulation research: The YCB object and model set and benchmarking protocols, ICRA Tutorial, 2015.

C. Schedlinski and M. Link, A SURVEY OF CURRENT INERTIA PARAMETER IDENTIFICATION METHODS, Mechanical Systems and Signal Processing, 2001.
DOI : 10.1006/mssp.2000.1345

K. S. Bhat, S. M. Seitz, J. Popovi´cpopovi´c, and P. K. Khosla, Computing the Physical Parameters of Rigid-Body Motion from Video, European Conference on Computer Vision, 2002.
DOI : 10.1007/3-540-47969-4_37

T. Feix, J. Romero, H. B. Schmiedmayer, A. M. Dollar, and D. Kragic, The GRASP Taxonomy of Human Grasp Types, IEEE Transactions on Human-Machine Systems, vol.46, issue.1, 2016.
DOI : 10.1109/THMS.2015.2470657

A. Erdemir, S. Mclean, W. Herzog, A. J. Van-den, and . Bogert, Model-based estimation of muscle forces exerted during movements, Clinical Biomechanics, vol.22, issue.2, 2007.
DOI : 10.1016/j.clinbiomech.2006.09.005

J. L. Elman, Finding Structure in Time, Cognitive Science, vol.49, issue.2, 1990.
DOI : 10.1207/s15516709cog1402_1

S. Hochreiter and J. Schmidhuber, Long Short-Term Memory, Neural Computation, vol.4, issue.8, 1997.
DOI : 10.1016/0893-6080(88)90007-X

R. Collobert, K. Kavukcuoglu, and C. Farabet, Torch7: A matlablike environment for machine learning, BigLearn, NIPS Workshop, 2011.

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, Dropout: A simple way to prevent neural networks from overfitting, Journal of Machine Learning Research, vol.15, issue.1, 2014.

N. Kyriazis and A. Argyros, Scalable 3D Tracking of Multiple Interacting Objects, 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014.
DOI : 10.1109/CVPR.2014.438

M. Fliess and H. Sira-ramírez, An algebraic framework for linear identification, ESAIM: Control, Optimisation and Calculus of Variations, 2003.
DOI : 10.1051/cocv:2003008

M. Mboup, C. Join, and M. Fliess, Numerical differentiation with annihilators in noisy environment, Numerical Algorithms, vol.14, issue.12, 2009.
DOI : 10.1007/s11075-008-9236-1
URL : https://hal.archives-ouvertes.fr/inria-00319240

S. Stassi, V. Cauda, G. Canavese, and C. F. Pirri, Flexible Tactile Sensing Based on Piezoresistive Composites: A Review, Sensors, vol.14, issue.3, 2014.
DOI : 10.3390/s140305296

M. Andersen, J. Dahl, and L. Vandenberghe, Cvxopt: A python package for convex optimization, 2013.

A. Krull, F. Michel, E. Brachmann, S. Gumhold, S. Ihrke et al., 6-DOF Model Based Tracking via Object Coordinate Regression, Asian Conference on Computer Vision, 2014.
DOI : 10.1007/978-3-319-16817-3_25

J. Issac, M. W. ¨-uthrich, C. Garcia-cifuentes, J. Bohg, S. Trimpe et al., Depth-based object tracking using a Robust Gaussian Filter, 2016 IEEE International Conference on Robotics and Automation (ICRA), 2016.
DOI : 10.1109/ICRA.2016.7487184

T. Pham, A. Kheddar, A. Qammaz, and A. A. Argyros, Capturing and reproducing hand-object interactions through visionbased force sensing, IEEE ICCV Workshop on Object Understanding for Interaction, 2015.
URL : https://hal.archives-ouvertes.fr/hal-01372238

Y. Zheng and K. Yamane, Evaluation of grasp force efficiency considering hand configuration and using novel generalized penetration distance algorithm, 2013 IEEE International Conference on Robotics and Automation, 2013.
DOI : 10.1109/ICRA.2013.6630781