Skip to Main content Skip to Navigation
Poster communications

No fonts: no advantage for transcriptions and comparisons

Abstract : Two issues are still at stake for the transcription of the gestures in a broad sense —cospeech gestures and SL —: i/the time used to transcribe and ii/the common ground of these transcriptions (Slobin et al 2001). The time needed to annotate gestural phenomena restricts drastically the size of the corpus and the possible generalizations. Whereas the absence of common transcription of the form between gestures and signs makes comparisons difficult across studies (Pointing gestures with a functional point of view for Kita 2003 and buoys in SL for Liddell 2003) and across the means of expression (part of cospeech gestures in SL) increasing the differences (Schembri et al 2005, Goldin-Meadow & Brentari 2017) and sometimes essentialising the nature of the linguistic or cospeech productions (Singleton et al 1995, McNeill 2014). A formal system of transcription designed for SL (project XX) but available for cospeech gestures as well depicts each segment (hand, forearm, arm and shoulder) of the upper limb, the orientation of the palm, the handshapes and the non-manual parameters including the mouth actions, the face, the head and the torso. This system is grouped in a family of fonts (YY et al 2018) which discretizes the positions of each articulator, for instance a digit in the handsaphe font. The glyphs obtained for handshape’s font are readable, easily writable and surely searchable. They contain informations (genuine characters in Unicode’s sense) about the shape, the angle and the close relations with other fingers, for each digit (ZZ et al 2018). Because of their formal and modular approaches, these transcriptions allow i/a comparison between SL and gestures/mimic in multimodal discourses, ii/in SL, profound subparameters querries for instance about the angle of a segment (e. g. the hand) according to a degree of freedom (e.g. Flexion/Extension) and up to the relations between segments or even a pointing form regardless of the finger. A corpus of multimodal discourses and FSL of several minutes each has been transcribed (in ELAN) using these fonts in two different ways: by a manual annotation and by a semi-automatic way. This last transcription in XX fonts is made from an IMU device put on the locutors and through the processing of OpenFace (Baltrušaitis et al 2018) and OpenPose (Cao et al 2018) from the video. In this talk, we present the family fonts system, its structure, the features annotated, the possibles queries made at two levels for the glyphs and for their ligatured components (see the informations, supra.). Some transcriptions will be exposed and some results too. We will put a focus on the presentation of results according to the time used to transcribe. The ratio of transcription: the duration for the transcription in the two different ways already mentioned compared to the length of the corpus. We propose to set up a tutorial session beside the conference (3 hours, group of 15 people) to learn how to use this transcription environment and offer to download the fonts and the Virtual Keyboard.
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-02394497
Contributor : Dominique Boutet <>
Submitted on : Wednesday, December 4, 2019 - 5:57:51 PM
Last modification on : Thursday, December 5, 2019 - 1:40:05 AM

Identifiers

  • HAL Id : hal-02394497, version 1

Citation

Dominique Boutet, Claudia Savina Bianchini, Patrick Doan, Morgane Rébulard, Claire Danet, et al.. No fonts: no advantage for transcriptions and comparisons. Sign CAFE 2: Second international workshop on cognitive and functional explorations in sign language linguistics, May 2020, Raguse, Italy. ⟨hal-02394497⟩

Share

Metrics

Record views

46