Help Me to Help You: how to Learn Intentions, Actions and Plans

Abstract : The collaboration between a human and a robot is here understood as a learning process mediated by the instructor prompt behaviours and the apprentice collecting information from them to learn a plan. The instructor wears the Gaze Machine, a wearable device gathering and conveying visual and audio input from the instructor while executing a task. The robot, on the other hand, is eager to learn both the best sequence of actions , their timing and how they interlace. The cross relation among actions is specified both in terms of time intervals for their execution, and in terms of location in space to cope with the instruction interaction with people and objects in the scene. We outline this process: how to transform the rich information delivered by the Gaze Machine into a plan. Specifically, how to obtain a map of the instructor positions and his gaze position, via visual slam and gaze fixations; further, how to obtain an action map from the running commentaries and the topological maps and, finally, how to obtain a temporal net of the relevant actions that have been extracted. The learned structure is then managed by the flexible time paradigm of flexible planning in the Situation Calculus for execution monitoring and plan generation.
Document type :
Conference papers
Liste complète des métadonnées

Cited literature [24 references]  Display  Hide  Download
Contributor : Harmish Khambhaita <>
Submitted on : Monday, July 17, 2017 - 12:00:33 PM
Last modification on : Thursday, February 7, 2019 - 3:48:44 PM
Document(s) archivé(s) le : Saturday, January 27, 2018 - 2:59:04 PM


Files produced by the author(s)


  • HAL Id : hal-01563110, version 1


Harmish Khambhaita, Geert-Jan Kruijff, Matei Mancas, Mario Gianni, Panagiotis Papadakis, et al.. Help Me to Help You: how to Learn Intentions, Actions and Plans. The AAAI Spring Symposium Help Me Help You: Bridging the Gaps in Human-Agent Collaboration, Mar 2011, Palo Alto, California, United States. ⟨hal-01563110⟩



Record views


Files downloads