A pipeline for motion capture recording and analysis of movement data using an intrinsic frame of reference
Résumé
Goldin-Meadow & Brentari (2015) highlighted the study analysis of movement through motion capture systems (mocap) as one of thea promising means to study the quality of movement with good accuracy. Malaia & Wilbur (2012) have conducted such research, analyzing the difference between telic and atelic verbs in ASL. In the project discussed in this panel, the hypothesis concerning the boundedness of gestures or not, associated with the expression of perfectivity versus imperfectivity, respectively, was supported for French, but not for the two other languages in the study (German and Russian). The German and Russian speakers employed many more bounded gestures than the French speakers did, and more bounded gestures with both verb types studied (perfect[ive] and imperfect[ive]). Preliminary results of another manner to codeof coding, according to the propagation of the movement along the upper limb, give even better results for French than those we have with a visual coding of the pulse of effort (Boutet et al. 2016). The kinesiological manner of coding explores the flow of propagation of the movement. Taking into account the flow could move beyond the visual quality of boundedness, overrepresented in German and Russian gestures, to reach a kinesiological source directly for the expression of perfectivity. We distinguish a proximal-distal flow for which the movement goes toward a distal segment (e.g. hand, fingers), which we hypothesize is more likely to express perfectivity, and the distal-proximal flow, which is expected to match more frequently with verbs in the imperfect(ive) form.
This talk will present two kinds of results: 1/ A pipeline of the framework to record, visualize and analyze a multimodal corpus, including audio, video and MOCAP mocap data; and 2/ measurement of the flow of the movement, with a mocap device consisting of Inertial Measurement Units (IMUs).
The pipeline, made for linguists, is conceived in terms of methodological guidelines. The hardware and the open-source softwares used will be listed. These guidelines begin before the recording step, presenting the conditions of the environment, the precautions that must be taken into account for the alignment of the three modes of recording (audio, video and mocap); and the naming of the files which must contain, for example, the gender and the height of the subject to visualize the body with a proper virtual skeleton after the recording. The way to visualize the mocap and its inclusion along with video data – in order to check the correspondence between the mocap data and the video – will be explained. Beyond the first results of our project presented here, these guidelines and the open-source softwares used are being created to help members of any project which uses mocap in gesture studies.
Bibliography:
-Boutet, D., Morgenstern, A., & Cienki, A. (2016). Grammatical Aspect and Gesture in French: A kinesiological approach. Russian Journal of Linguistics, 20(3), 132‑151.
-Goldin-Meadow, S., Brentari, D., Goldin-Meadow, S., & Brentari, D. (2015). Gesture, sign and language: The coming of age of sign language and gesture studies. The Behavioral and brain sciences, 1‑82.
-Malaia, E., & Wilbur, R. B. (2012). Kinematic signatures of telic and atelic events in ASL predicates. Language and speech, 55(3), 407–421.