Gesture Elicitation as a Computational Optimization Problem

Theophanis Tsandilas (1) , Pierre Dragicevic (2)

1 EX-SITU - Extreme Situated Interaction - LISN
2 Potioc - Popular interaction with 3d content

Gesture elicitation studies are commonly used for designing novel gesture-based interfaces. There is a rich methodology literature on metrics and analysis methods that helps researchers understand and characterize data arising from such studies. However, deriving concrete gesture vocabularies from this data, which is often the ultimate goal, remains largely based on heuristics and ad hoc methods. In this paper, we treat the problem of deriving a gesture vocabulary from gesture elicitation data as a computational optimization problem. We show how to formalize it as an optimal assignment problem and discuss how to express objective functions and custom design constraints through integer programs. In addition, we introduce a set of tools for assessing the uncertainty of optimization outcomes due to random sampling, and for supporting researchers' decisions on when to stop collecting data from a gesture elicitation study. We evaluate our methods on a large number of simulated studies.

Mots clés: Gesture elicitation; design optimization; assignment problems; optimization uncertainty; learning curves; bootstrap

Domaines: Informatique [cs]

Theophanis Tsandilas, Pierre Dragicevic. Gesture Elicitation as a Computational Optimization Problem. ACM Conference on Human Factors in Computing Systems (CHI ’22), Apr 2022, New Orleans, United States. ⟨10.1145/3491102.3501942⟩. ⟨hal-03559392⟩