Audiovisual perception of openness and lip rounding in front vowels
Traunmüller, H., & Öhrström, N.
Journal of Phonetics 35 (2007): 244-258.
Swedish nonsense syllables /gig/, /gyg/, /geg/ and /gøg/, produced by four speakers, were video-recorded and presented to male and female subjects in auditory, visual and audiovisual mode and also in cross-dubbed audiovisual form with incongruent cues to vowel openness, roundedness, or both. With audiovisual stimuli, subjects perceived openness nearly always by ear. Most subjects perceived roundedness by eye rather than by ear although the auditory conditions were optimal and the sensation was an auditory one. This resulted in fused percepts such as when an acoustic /geg/ dubbed onto an optic /gyg/ was predominantly perceived as /gøg/. Since the acoustic cues to openness are prominent, while those to roundedness are less reliable, this lends support to the “information reliability hypothesis” in multisensory perception: The perception of a feature is dominated by the modality that provides the more reliable information. A mostly male minority relied less on vision. The between-gender difference was significant. Presence of lip rounding (a visibly marked feature) was noticed more easily than its absence. The influence of optic information was not fully explicable on the basis of the subjects’ success rates in lipreading compared with auditory perception. It was highest in stimuli produced by a speaker who smiled.