Skip to Main content Skip to Navigation
Conference papers

The Usability of Speech and/or Gestures in Multi-Modal Interface Systems

Abstract : Multi-Modal Interface Systems (MMIS) have proliferated in the last few decades, since they provide a direct interface for both Human Computer Interaction (HCI) and face-to-face communication. Our aim is to provide users without any prior 3D modelling experience, with a multi-modal interface to create a 3D object. The system also incorporates help throughout the drawing process and identifies simple words and gestures to accomplish a range of (simple to complex) modeling tasks. We have developed a multi-modal interface that allows users to design objects in 3D, using AutoCAD commands as well as speech and gesture. We have used a microphone to collect speech input and a Leap Motion sensor to collect gesture input in real time. Two sets of experiments were conducted to investigate the usability of the system and evaluate the system performance using Leap Motion versus keyboard and mouse. Our results indicate that performing a task using speech is perceived exhausting, when there is no shared vocabulary between man and machine, and the usability of traditional input devices supersedes the usability of speech and gestures. Only a small ratio of participants, less than 7% in our experiments were able to carry out the tasks with appropriate precision.
Complete list of metadatas

Cited literature [10 references]  Display  Hide  Download
Contributor : Compte de Service Administrateur Ensam <>
Submitted on : Wednesday, April 5, 2017 - 5:01:14 PM
Last modification on : Monday, March 30, 2020 - 8:44:41 AM
Document(s) archivé(s) le : Thursday, July 6, 2017 - 1:53:24 PM


Files produced by the author(s)


  • HAL Id : hal-01502555, version 1
  • ENSAM :


Farzana Alibay, Manolya Kavakli, Jean-Rémy Chardonnet, Muhammad Zeeshan Baig. The Usability of Speech and/or Gestures in Multi-Modal Interface Systems. International Conference on Computer and Automation Engineering (ICCAE 2017), Feb 2017, Sydney, Australia. pp.1-5. ⟨hal-01502555⟩



Record views


Files downloads