A. Gupta, A. Kembhavi, and L. Davis, Observing Human-Object Interactions: Using Spatial and Functional Compatibility for Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.31, issue.10, pp.1775-1789, 2009.
DOI : 10.1109/TPAMI.2009.83

Y. Zhu, Y. Zhao, and S. C. Zhu, Understanding tools: Taskoriented object modeling, learning and recognition, Proc. IEEE Conf. Comput. Vis, pp.2855-2864, 2015.
DOI : 10.1109/cvpr.2015.7298903

C. Ye, Y. Yang, C. Fermuller, and Y. Aloimonos, What can i do around here? Deep functional scene understanding for cognitive robots, 2017 IEEE International Conference on Robotics and Automation (ICRA), pp.4604-4611, 2017.
DOI : 10.1109/ICRA.2017.7989535

X. Niu, A. V. Terekhov, M. L. Latash, and V. M. Zatsiorsky, Reconstruction of the Unknown Optimization Cost Functions from Experimental Recordings during Static Multi-Finger Prehension, Motor Control, vol.16, issue.2, pp.195-228, 2012.
DOI : 10.1123/mcj.16.2.195

G. Slota, M. Latash, and V. Zatsiorsky, Grip forces during object manipulation: experiment, mathematical model, and validation, Experimental Brain Research, vol.30, issue.1, pp.125-139, 2011.
DOI : 10.1109/3477.826959

URL : http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3212984

T. Pham, A. Kheddar, A. Qammaz, and A. A. Argyros, Towards force sensing from vision: Observing hand-object interactions to infer manipulation forces, Proc. IEEE Conf. Comput. Vis, pp.2810-2819, 2015.
URL : https://hal.archives-ouvertes.fr/hal-01356136

S. A. Mascaro and H. H. Asada, Photoplethysmograph fingernail sensors for measuring finger forces without haptic obstruction, IEEE Transactions on Robotics and Automation, vol.17, issue.5, pp.698-708, 2001.
DOI : 10.1109/70.964669

Y. Sun, J. M. Hollerbach, and S. A. Mascaro, Estimation of Fingertip Force Direction With Computer Vision, IEEE Transactions on Robotics, vol.25, issue.6, pp.1356-1369, 2009.
DOI : 10.1109/TRO.2009.2032954

S. Urban, J. Bayer, C. Osendorfer, G. Westling, B. B. Edin et al., Computing grip force and torque from finger nail images using Gaussian processes, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp.4034-4039, 2013.
DOI : 10.1109/IROS.2013.6696933

URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.667.3533

P. G. Kry and D. K. Pai, Interaction capture and synthesis, ACM Transactions on Graphics, vol.25, issue.3, pp.872-880, 2006.
DOI : 10.1145/1141911.1141969

URL : https://hal.archives-ouvertes.fr/Inria-00402113

J. M. Rehg and T. Kanade, Visual tracking of high DOF articulated structures: An application to human hand tracking, Proc. Eur. Conf. Comput. Vis, pp.35-46, 1994.
DOI : 10.1007/BFb0028333

I. Oikonomidis, N. Kyriazis, and A. A. Brit, Efficient modelbased 3d tracking of hand articulations using kinect, pp.101-102, 2011.
DOI : 10.5244/c.25.101

M. De-la-gorce, D. J. Fleet, and N. Paragios, Model-Based 3D Hand Pose Estimation from Monocular Video, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.33, issue.9, pp.1793-1805, 2011.
DOI : 10.1109/TPAMI.2011.33

URL : https://hal.archives-ouvertes.fr/hal-00856313

C. Keskin, F. K?raç, Y. E. Kara, and L. Akarun, Hand Pose Estimation and Hand Shape Classification Using Multi-layered Randomized Decision Forests, Proc. Eur. Conf. Comput. Vis, pp.852-863, 2012.
DOI : 10.1007/978-3-642-33783-3_61

C. Qian, X. Sun, Y. Wei, X. Tang, and J. Sun, Realtime and Robust Hand Tracking from Depth, 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp.1106-1113, 2014.
DOI : 10.1109/CVPR.2014.145

URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.454.4572

D. Tang, H. J. Chang, A. Tejani, and T. Kim, Latent Regression Forest: Structured Estimation of 3D Articulated Hand Posture, 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp.3786-3793, 2014.
DOI : 10.1109/CVPR.2014.490

J. Tompson, M. Stein, Y. Lecun, and K. Perlin, Real-Time Continuous Pose Recovery of Human Hands Using Convolutional Networks, ACM Transactions on Graphics, vol.33, issue.5, p.169, 2014.
DOI : 10.1145/1531326.1531369

P. Krejov, A. Gilbert, and R. Bowden, Combining discriminative and model based approaches for hand pose estimation, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG)
DOI : 10.1109/FG.2015.7163141

G. Rogez, J. S. Supancic, and D. Ramanan, Understanding Everyday Hands in Action from RGB-D Images, 2015 IEEE International Conference on Computer Vision (ICCV), pp.3889-3897, 2015.
DOI : 10.1109/ICCV.2015.443

URL : https://hal.archives-ouvertes.fr/hal-01237011

M. Cai, K. M. Kitani, and Y. Sato, A scalable approach for understanding the visual structures of hand grasps, Proc, pp.1360-1366, 2015.

D. Huang, M. Ma, W. Ma, and K. M. Kitani, How do we use our hands? discovering a diverse set of common grasps, Proc. IEEE Conf. Comput. Vis, pp.666-675, 2015.

T. Sharp, C. Keskin, D. Robertson, J. Taylor, J. Shotton et al., Accurate, Robust, and Flexible Real-time Hand Tracking, Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI '15, pp.3633-3642, 2015.
DOI : 10.1016/j.tics.2006.05.002

J. Taylor, L. Bordeaux, T. Cashman, B. Corish, C. Keskin et al., Efficient and precise interactive hand tracking through joint, continuous optimization of pose and correspondences, ACM Transactions on Graphics, vol.35, issue.4, p.143, 2016.
DOI : 10.1007/3-540-44480-7_21

J. S. Supancic, G. Rogez, Y. Yang, J. Shotton, and D. Ramanan, Depth-Based Hand Pose Estimation: Data, Methods, and Challenges, 2015 IEEE International Conference on Computer Vision (ICCV), pp.1868-1876, 2015.
DOI : 10.1109/ICCV.2015.217

URL : https://hal.archives-ouvertes.fr/hal-01237023

O. Koller, H. Ney, and R. Bowden, Deep Hand: How to Train a CNN on 1 Million Hand Images When Your Data is Continuous and Weakly Labelled, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.3793-3802, 2016.
DOI : 10.1109/CVPR.2016.412

L. Ge, H. Liang, J. Yuan, and D. Thalmann, Robust 3D Hand Pose Estimation in Single Depth Images: From Single-View CNN to Multi-View CNNs, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.3593-3601, 2016.
DOI : 10.1109/CVPR.2016.391

URL : http://arxiv.org/pdf/1606.07253

A. Sinha, C. Choi, and K. Ramani, DeepHand: Robust Hand Pose Estimation by Completing a Matrix Imputed with Deep Features, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.4150-4158, 2016.
DOI : 10.1109/CVPR.2016.450

C. Wan, T. Probst, L. Van-gool, and A. Yao, Crossing nets: Combining gans and vaes with a shared latent space for hand pose estimation, Proc. IEEE Conf. Comput. Vis. Pattern Recognit, 2017.

L. Ge, H. Liang, J. Yuan, and D. Thalmann, 3d convolutional neural networks for efficient and robust hand pose estimation from single depth images, Proc. IEEE Conf. Comput. Vis. Pattern Recognit, 2017.

T. Simon, H. Joo, I. Matthews, and Y. Sheikh, Hand keypoint detection in single images using multiview bootstrapping, Proc. IEEE Conf. Comput. Vis. Pattern Recognit, 2017.

I. Oikonomidis, N. Kyriazis, and A. Argyros, Full DOF tracking of a hand interacting with an object by modeling occlusions and physical constraints, 2011 International Conference on Computer Vision, pp.2088-2095, 2011.
DOI : 10.1109/ICCV.2011.6126483

D. Tzionas, L. Ballan, A. Srikantha, P. Aponte, M. Pollefeys et al., Capturing Hands in Action Using Discriminative Salient Points and Physics Simulation, International Journal of Computer Vision, vol.28, issue.3, pp.1-22, 2015.
DOI : 10.1007/978-3-642-44964-2_8

URL : http://arxiv.org/pdf/1506.02178

S. Sridhar, F. Mueller, M. Zollhöferzollh¨zollhöfer, D. Casas, A. Oulasvirta et al., Real-Time Joint Tracking of a Hand Manipulating an Object from RGB-D Input, Proc. Eur. Conf. Comput. Vis, pp.294-310, 2016.
DOI : 10.1145/2601097.2601165

N. Kyriazis and A. Argyros, Physically Plausible 3D Scene Tracking: The Single Actor Hypothesis, 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp.9-16, 2013.
DOI : 10.1109/CVPR.2013.9

URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.309.5263

W. Zhao, J. Zhang, J. Min, and J. Chai, Robust realtime physicsbased motion control for human grasping, ACM Trans. Graphics, vol.32, issue.207, pp.1-20712, 2013.

Y. Wang, J. Min, J. Zhang, Y. Liu, F. Xu et al., Videobased hand manipulation capture through composite motion control, ACM Trans. Graphics, vol.32, issue.4, p.43, 2013.
DOI : 10.1145/2461912.2462000

URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.648.5127

M. A. Arbib, T. Iberall, and D. Lyons, Coordinated Control Programs for Movements of the Hand, Hand function and the neocortex, pp.111-129, 1985.
DOI : 10.1007/978-3-642-70105-4_7

F. Gao, M. L. Latash, and V. M. Zatsiorsky, Internal forces during object manipulation, Experimental Brain Research, vol.30, issue.1, pp.69-83, 2005.
DOI : 10.3758/BF03213054

URL : http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2847586

J. Kerr and B. Roth, Analysis of Multifingered Hands, The International Journal of Robotics Research, vol.1, issue.1, pp.3-17, 1986.
DOI : 10.1177/027836498200100102

T. Yoshikawa and K. Nagai, Manipulating and grasping forces in manipulation by multifingered robot hands, IEEE Transactions on Robotics and Automation, vol.7, issue.1, pp.67-77, 1991.
DOI : 10.1109/70.68071

M. T. Mason and J. K. Salisbury, Robot Hands and the Mechanics of Manipulation, Journal of Dynamic Systems, Measurement, and Control, vol.111, issue.1, 1985.
DOI : 10.1115/1.3153010

R. M. Murray, S. S. Sastry, and L. Zexiang, A Mathematical Introduction to Robotic Manipulation, 1994.

J. R. Flanagan and R. S. Johansson, Hand Movements, Encyclopedia of the human brain, pp.399-414, 2002.
DOI : 10.1016/B0-12-227210-2/00157-6

S. L. Gorniak, V. M. Zatsiorsky, and M. L. Latash, Manipulation of a fragile object, Experimental Brain Research, vol.13, issue.2, 2010.
DOI : 10.1123/mcj.13.3.251

J. Park, T. Singh, V. M. Zatsiorsky, and M. L. Latash, Optimality versus variability: effect of fatigue in multi-finger redundant tasks, Experimental Brain Research, vol.99, issue.4, pp.591-607, 2012.
DOI : 10.1152/jn.01029.2007

B. I. Prilutsky and V. M. Zatsiorsky, Optimization-Based Models of Muscle Coordination, Exercise and Sport Sciences Reviews, vol.30, issue.1, p.32, 2002.
DOI : 10.1097/00003677-200201000-00007

C. Fermüllerferm¨fermüller, F. Wang, Y. Yang, K. Zampogiannis, Y. Zhang et al., Prediction of manipulation actions, Int. J. Comput. Vis, pp.1-17, 2017.

M. S. Lobo, L. Vandenberghe, S. P. Boyd, and H. Lebret, Applications of second-order cone programming, Linear Algebra and its Applications, vol.284, issue.1-3, pp.193-228, 1998.
DOI : 10.1016/S0024-3795(98)10032-0

S. P. Boyd and B. Wegbreit, Fast Computation of Optimal Contact Forces, IEEE Transactions on Robotics, vol.23, issue.6, pp.1117-1132, 2007.
DOI : 10.1109/TRO.2007.910774

URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.137.4980

I. Lenz, H. Lee, and A. Saxena, Deep learning for detecting robotic grasps, Int. J. Robot. Res, vol.34, pp.4-5, 2015.
DOI : 10.15607/rss.2013.ix.012

URL : http://arxiv.org/abs/1301.3592

H. S. Koppula and A. Saxena, Anticipating Human Activities Using Object Affordances for Reactive Robotic Response, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.38, issue.1, pp.14-29, 2016.
DOI : 10.1109/TPAMI.2015.2430335

URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.307.5387

Y. Bengio, A. Courville, and P. Vincent, Representation Learning: A Review and New Perspectives, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.35, issue.8, pp.1798-1828, 2013.
DOI : 10.1109/TPAMI.2013.50

URL : http://arxiv.org/pdf/1206.5538

M. A. Brubaker, L. Sigal, and D. J. Fleet, Estimating contact dynamics, 2009 IEEE 12th International Conference on Computer Vision, pp.2389-2396, 2009.
DOI : 10.1109/ICCV.2009.5459407

URL : http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.154.5133&rep=rep1&type=pdf

M. Mohammadi, T. L. Baldi, S. Scheggi, and D. Prattichizzo, Fingertip force estimation via inertial and magnetic sensors in deformable object manipulation, 2016 IEEE Haptics Symposium (HAPTICS), pp.284-289, 2016.
DOI : 10.1109/HAPTICS.2016.7463191

Y. Zhu, C. Jiang, Y. Zhao, D. Terzopoulos, and S. Zhu, Inferring Forces and Learning Human Utilities from Videos, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.3823-3833, 2016.
DOI : 10.1109/CVPR.2016.415

H. S. Koppula, A. Anand, T. Joachims, and A. Saxena, Semantic labeling of 3d point clouds for indoor scenes, Proc. Adv. Neural Inf. Process. Syst, pp.244-252, 2011.

K. Lai, L. Bo, and D. Fox, Unsupervised feature learning for 3D scene labeling, 2014 IEEE International Conference on Robotics and Automation (ICRA), pp.3050-3057, 2014.
DOI : 10.1109/ICRA.2014.6907298

URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.646.8686

J. Sung, C. Ponce, B. Selman, and A. Saxena, Unstructured human activity detection from RGBD images, Proc, pp.842-849, 2012.

A. Saxena, J. Driemeyer, and A. Y. Ng, Robotic Grasping of Novel Objects using Vision, The International Journal of Robotics Research, vol.13, issue.3, pp.157-173, 2008.
DOI : 10.1177/027836499601500302

B. , A. Walsman, A. Singh, S. Srinivasa, P. Abbeel et al., Benchmarking in manipulation research: The YCB object and model set and benchmarking protocols, ICRA Tutorial, 2015.

C. Schedlinski and M. Link, A SURVEY OF CURRENT INERTIA PARAMETER IDENTIFICATION METHODS, Mechanical Systems and Signal Processing, vol.15, issue.1, pp.189-211, 2001.
DOI : 10.1006/mssp.2000.1345

K. S. Bhat, S. M. Seitz, J. Popovi´cpopovi´c, and P. K. Khosla, Computing the Physical Parameters of Rigid-Body Motion from Video, Proc. Eur. Conf. Comput. Vis, pp.551-565, 2002.
DOI : 10.1007/3-540-47969-4_37

M. Ciocarlie, C. Lackner, and P. Allen, Soft Finger Model with Adaptive Contact Geometry for Grasping and Manipulation Tasks, Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC'07), pp.219-224, 2007.
DOI : 10.1109/WHC.2007.103

URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.137.466

D. Kraft, Algorithm 733; TOMP---Fortran modules for optimal control calculations, ACM Transactions on Mathematical Software, vol.20, issue.3, pp.262-281, 1994.
DOI : 10.1145/192115.192124

URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.512.2567

J. L. Elman, Finding Structure in Time, Cognitive Science, vol.49, issue.2, pp.179-211, 1990.
DOI : 10.1007/BF00308682

S. Hochreiter and J. Schmidhuber, Long Short-Term Memory, Neural Computation, vol.4, issue.8, pp.1735-1780, 1997.
DOI : 10.1016/0893-6080(88)90007-X

R. Collobert, K. Kavukcuoglu, and C. Farabet, Torch7: A matlablike environment for machine learning, BigLearn, NIPS Workshop, 2011.

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, Dropout: A simple way to prevent neural networks from overfitting, J. Mach. Learn. Res, vol.15, issue.1, pp.1929-1958, 2014.

J. P. Scholz, F. Danion, M. L. Latash, and G. Schönersch¨schöner, Understanding finger coordination through analysis of the structure of force variability, Biological Cybernetics, vol.86, issue.1, pp.29-39, 2002.
DOI : 10.1007/s004220100279

N. Kyriazis and A. Argyros, Scalable 3D Tracking of Multiple Interacting Objects, 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp.3430-3437, 2014.
DOI : 10.1109/CVPR.2014.438

URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.476.5558

M. Mboup, C. Join, and M. Fliess, Numerical differentiation with annihilators in noisy environment, Numerical Algorithms, vol.14, issue.12, pp.439-467, 2009.
DOI : 10.1007/978-1-4612-1118-1

URL : https://hal.archives-ouvertes.fr/inria-00319240

S. Stassi, V. Cauda, G. Canavese, and C. F. Pirri, Flexible Tactile Sensing Based on Piezoresistive Composites: A Review, Sensors, vol.59, issue.116, pp.5296-5332, 2014.
DOI : 10.1109/TCSI.2012.2188954

URL : http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4003994

M. Andersen, J. Dahl, and L. Vandenberghe, Cvxopt: A python package for convex optimization, 2013.

A. Krull, F. Michel, E. Brachmann, S. Gumhold, S. Ihrke et al., 6-DOF Model Based Tracking via Object Coordinate Regression, Proc. Asian Conf. Comput. Vis, pp.384-399, 2014.
DOI : 10.1007/978-3-319-16817-3_25

URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.673.1169

J. Issac, M. W. ¨-uthrich, C. Garcia-cifuentes, J. Bohg, S. Trimpe et al., Depth-based object tracking using a robust gaussian filter Capturing and reproducing hand-object interactions through visionbased force sensing, Proc IEEE ICCV Workshop on Object Understanding for Interaction, pp.608-615, 2015.
DOI : 10.1109/icra.2016.7487184

URL : http://arxiv.org/abs/1602.06157

Y. Zheng and K. Yamane, Evaluation of grasp force efficiency considering hand configuration and using novel generalized penetration distance algorithm, 2013 IEEE International Conference on Robotics and Automation, pp.1580-1587, 2013.
DOI : 10.1109/ICRA.2013.6630781

T. Pham, S. Caron, and A. Kheddar, Multi-contact interaction force sensing from whole-body motion capture, 2017.
DOI : 10.1109/sii.2016.7843975

URL : https://hal.archives-ouvertes.fr/hal-01372531/document