PDF(Pubmed)
Abstract：
The visual cues of lexical tones are more implicit and much less investigated than consonants and vowels, and it is still unclear what facial areas contribute to facial tones identification. This study investigated Chinese and English speakers\' eye movements when they were asked to identify audiovisual Mandarin lexical tones. The Chinese and English speakers were presented with an audiovisual clip of Mandarin monosyllables (for instance, /ă/, /à/, /ĭ/, /ì/) and were asked to identify whether the syllables were a dipping tone (/ă/, / ĭ/) or a falling tone (/ à/, /ì/). These audiovisual syllables were presented in clear, noisy and silent (absence of audio signal) conditions. An eye-tracker recorded the participants\' eye movements. Results showed that the participants gazed more at the mouth than the eyes. In addition, when acoustic conditions became adverse, both the Chinese and English speakers increased their gaze duration at the mouth rather than at the eyes. The findings suggested that the mouth is the primary area that listeners utilise in their perception of audiovisual lexical tones. The similar eye movements between the Chinese and English speakers imply that the mouth acts as a perceptual cue that provides articulatory information, as opposed to social and pragmatic information.
摘要：
与辅音和元音相比，词汇音调的视觉线索更含蓄，研究也少得多，目前还不清楚哪些面部区域有助于识别面部色调。这项研究调查了中文和英文使用者被要求识别视听普通话词汇音调时的眼球运动。向中英文使用者展示了普通话单音节的视听剪辑（例如，//,/à/,//,/µ/），并被要求确定音节是否为浸渍音（/边沿/，//)或下降音(/à/,/µ/)。这些视听音节清晰地呈现，嘈杂和无声（没有音频信号）的条件。眼球追踪器记录了参与者的眼球运动。结果显示，参与者盯着嘴比盯着眼睛。此外,当声学条件变得不利时，说中文和英文的人都增加了对嘴而不是眼睛的凝视时间。研究结果表明，嘴巴是听众感知视听词汇音调的主要区域。中文和英文使用者之间相似的眼球运动意味着嘴巴充当了提供发音信息的感知线索，与社会和务实的信息相反。
