Preliminary study exploring the relation between visual prosody and the prosodic components in sign language

Abstract

Type enriched with visual prosody is a powerful tool to encourage expressive reading. Visual prosody adds cues to text to guide vocal variations in loudness, duration, and pitch. More vocal variations result in a less monotonous voice and thus more expression. A positive effect of visual prosody is known on the voice of normal hearing readers and of signed bilingual deaf readers who developed signed language and spoken language. These deaf readers rely on speech as well as sign language and both modalities can be used interchangeably to compensate each other. This preliminary study explores visual prosody in text in relation to Flemish Sign Language to see if sign language can be used to explain prosody. We asked deaf readers between 7 and 18 to relate prosodic cues to videos presenting prosodic components of Flemish Sign Language. We found that those readers connect the prosodic cues with the components in Flemish Sign Language as intended. Larger word-spacing correlates with a pause between signs, a wider font with a sign with ‘longer duration’, a thicker font with more ‘displacement’ in the sign, a raised font with a ‘faster velocity’ in the sign. However, some confusion occurred as participants seemed to extract only two prosodic components in the sign language: both the ‘faster velocity’ and ‘longer duration’ were referred to in terms of 'speed' and were not perceived as separate prosodic components. Participants were confused about why there were three cues in the text. Therefore, it is advised to re-evaluate and to re-design visual prosody for sign language with only ‘displacement’ and ‘speed’ in mind.