Co-Sound: An interactive medium with WebAR and spatial synchronization

Kazuma Inokuchi (1) , Manabu Tsukada (2) , Hiroshi Esaki (1)

1 UTokyo - The University of Tokyo
2 Department of Creative Informatics

An Internet-based media service platform can control recording processes and manage video and audio data. Furthermore, the design and implementation of an object-based system for recording enable the flexible playback of the viewing contents. Augmented Reality (AR) is a three-dimensional video projection technology. However, there are few examples of its use as a method for audiovisual media platforms. In this study, we propose Co-Sound, which is designed as a multimodal interface that renders object-based AR dynamically in response to various actions from viewers on a web browser by sharing AR objects among multiple devices in real time. We confirmed that the system was developed as an object-based interactive medium with AR, achieved the general acceptance of the system was very high through a questionnaire survey, and low-latency synchronization to accept operations from multiple users in real time.

Mots clés: Interactive media, Object-based audio, Augmented reality, Software defined media; Interactive media, Object-based audio, Augmented reality, Software defined media

Domaines: Réseaux et télécommunications [cs.NI]; Informatique mobile; Multimédia [cs.MM]

Kazuma Inokuchi, Manabu Tsukada, Hiroshi Esaki. Co-Sound: An interactive medium with WebAR and spatial synchronization. 19th International Conference on Entertainment Computing (ICEC), Nov 2020, Xi'an, China. pp.255-263, ⟨10.1007/978-3-030-65736-9_22⟩. ⟨hal-02942505⟩