Understanding perceptual-gestural knowledge in TEL systems with eye-tracking
Résumé
This paper presents our methodology to capture and model multimodal interactions in Intelligent Tutoring Systems (ITS). We are specifically interested in perceptual-gestural interactions combining perceptions, gestures and other type of actions.Traces of such interactions are multisource and heterogeneous. The challenge is to foster their representation into sequences that render their multimodal nature. In this work, we want to show how the proposed representation yields the analysis of the influence of visual perceptions on learners’ performance. Our case study is the ITS TELEOS, a simulation based Intelligent Tutoring System dedicated to percutaneous orthopedic surgery. We also conducted an experiment on PILOTE 2, a flight simulation environment, to give a proof of concept of the genericity of our propositions.