The Phone with the Flow: Combining Touch + Optical Flow in Mobile Instruments

Abstract : Mobile devices have been a promising platform for musical performance thanks to the various sensors readily available on board. In particular, mobile cameras can provide rich input as they can capture a wide variety of user gestures or environment dynamics. However, this raw camera input only provides continuous parameters and requires expensive computation. In this paper, we propose combining camera based motion/gesture input with the touch input, in order to filter movement information both temporally and spatially , thus increasing expressiveness while reducing computation time. We present a design space which demonstrates the diversity of interactions that our technique enables. We also report the results of a user study in which we observe how musicians appropriate the interaction space with an example instrument.
Liste complète des métadonnées

Cited literature [17 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01809210
Contributor : Cagan Arslan <>
Submitted on : Wednesday, June 6, 2018 - 3:01:22 PM
Last modification on : Friday, March 22, 2019 - 1:36:34 AM
Document(s) archivé(s) le : Friday, September 7, 2018 - 1:59:51 PM

File

Nime_18_PwF.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01809210, version 1

Citation

Cagan Arslan, Florent Berthaut, Jean Martinet, Ioan Marius Bilasco, Laurent Grisoni. The Phone with the Flow: Combining Touch + Optical Flow in Mobile Instruments. NIME 2018 - 18th International Conference on New Interfaces for Musical Expression, Jun 2018, Blacksburg, VA, United States. ⟨hal-01809210⟩

Share

Metrics

Record views

190

Files downloads

146