Vision-based calibration, position control and force sensing for soft robots

Zhongkai Zhang 1, 2
2 DEFROST - Deformable Robots Simulation Team
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189
Abstract : Soft robots can interact with the environment in a safe and compliant way because of their deformable structures. However, the modeling of soft robots which have, theoretically, infinite degrees of freedom, are extremely difficult especially when the robots have complex configurations. This difficulty of modeling leads to new challenges for the calibration and the control design of the robots, but also new opportunities with possible new force sensing strategies. This dissertation aims to provide new and general solutions using modeling and vision. The thesis at first presents a discrete-time kinematic model for soft robots based on the real-time Finite Element (FE) method. Then, a vision-based simultaneous calibration of sensor-robot system and actuators is investigated. Two closed-loop position controllers are designed and the robust stability of the closed-loop system is analyzed using Lyapunov stability theory. Besides, to deal with the problem of image feature loss, a switched control strategy is proposed by combining both the open-loop controller and the closed-loop controller. Using soft robot itself as a force sensor is available due to the deformable feature of soft structures. Two methods (marker-based and marker-free) of external force sensing for soft robots are proposed based on the fusion of vision-based measurements and FE model. Using both methods, not only the intensities but also the locations of the external forces can be estimated. The marker-based approach is proposed to find the correct locations of external forces among several possible ones. If there are no obvious feature points on the surface of the soft robot, the marker-free force sensing strategy is available using an RGB-D camera. As a specific application, a cable-driven continuum catheter robot through contacts is modeled based on FE method. Then, the robot is controlled by a decoupled control strategy which allows to control insertion and bending independently. Both the control inputs and the contact forces along the entire catheter can be computed by solving a quadratic programming (QP) problem with a linear complementarity constraint (QPCC). A simplified solution is proposed for the computation of QPCC by converting it into a standard QP problem.
Liste complète des métadonnées

Cited literature [13 references]  Display  Hide  Download
Contributor : Zhongkai Zhang <>
Submitted on : Wednesday, January 23, 2019 - 2:26:52 PM
Last modification on : Friday, March 22, 2019 - 1:37:21 AM


Zhongkai Zhang-PhD thesis.pdf
Files produced by the author(s)


  • HAL Id : tel-01990867, version 1


Zhongkai Zhang. Vision-based calibration, position control and force sensing for soft robots. Robotics [cs.RO]. Université de Lille, 2019. English. ⟨tel-01990867⟩



Record views


Files downloads