PDF(Pubmed)
Abstract：
The current study investigated the neuro mechanisms of emoji processing as sentence predicate in written context. In the hybrid textuality which is more cognitively engaging, emojis in sentential intermediate positions were designed as either congruent or incongruent to the context. The results showed that incongruent words led to a robust N400 effect, while incongruent emojis only elicited the P600 effect. It implies that semantics and syntax of words can be separated while those of emojis seem to be integrated together. That is, when the meaning of the emoji is violated to the sentential context, its grammatical role cannot be well interpreted, especially when it is used as a key grammatical component in a sentence, such as the predicate. Thus, it shows that even though the meaning of emojis can be interpreted by readers, their syntactic and semantic functions cannot be clearly separated. In comparison with word processing, the larger amplitude with emojis in the time window of 350-500 ms shows more cognitive efforts in emoji semantic processing, possibly arising from the switch of modalities within the visual channel, that is, the multimodal cognitive load.
摘要：
当前的研究调查了表情符号在书面语境中作为句子谓词处理的神经机制。在更具认知吸引力的混合语态中，句子中间位置的表情符号被设计为与上下文一致或不一致。结果表明，不一致的单词导致了强烈的N400效应，而不一致的表情符号仅引起P600效应。这意味着单词的语义和语法可以分开，而表情符号的语义和语法似乎可以集成在一起。也就是说,当表情符号的含义违反了句子上下文时，它的语法作用不能很好地解释，尤其是当它被用作句子中的关键语法成分时，如谓词。因此，它表明，即使表情符号的含义可以被读者解释，它们的句法和语义功能不能明确分开。与文字处理相比，在350-500毫秒的时间窗口中，表情符号的振幅更大，在表情符号语义处理中显示出更多的认知努力，可能是由于视觉通道内的模态转换引起的，也就是说,多模态认知负荷。
