PDF(Pubmed)
Abstract：
A core aspect of human speech comprehension is the ability to incrementally integrate consecutive words into a structured and coherent interpretation, aligning with the speaker\'s intended meaning. This rapid process is subject to multidimensional probabilistic constraints, including both linguistic knowledge and non-linguistic information within specific contexts, and it is their interpretative coherence that drives successful comprehension. To study the neural substrates of this process, we extract word-by-word measures of sentential structure from BERT, a deep language model, which effectively approximates the coherent outcomes of the dynamic interplay among various types of constraints. Using representational similarity analysis, we tested BERT parse depths and relevant corpus-based measures against the spatiotemporally resolved brain activity recorded by electro-/magnetoencephalography when participants were listening to the same sentences. Our results provide a detailed picture of the neurobiological processes involved in the incremental construction of structured interpretations. These findings show when and where coherent interpretations emerge through the evaluation and integration of multifaceted constraints in the brain, which engages bilateral brain regions extending beyond the classical fronto-temporal language system. Furthermore, this study provides empirical evidence supporting the use of artificial neural networks as computational models for revealing the neural dynamics underpinning complex cognitive processes in the brain.
摘要：
人类语音理解的一个核心方面是能够将连续的单词逐步整合成结构化和连贯的解释，与演讲者的预期含义保持一致。这个快速过程受到多维概率约束，包括特定语境中的语言知识和非语言信息，他们的解释连贯性推动了成功的理解。为了研究这个过程的神经基质，我们从BERT中提取句子结构的逐字度量，一个深层的语言模型，这有效地逼近了各种类型的约束之间动态相互作用的连贯结果。使用代表性相似性分析，我们测试了BERT解析深度和相关的基于语料库的测量,以及参与者在听相同句子时通过脑电/脑磁图记录的时空分辨的大脑活动.我们的结果提供了结构化解释的增量构建中涉及的神经生物学过程的详细图片。这些发现表明，通过评估和整合大脑中多方面的约束，何时何地出现连贯的解释，它涉及双边大脑区域，延伸到经典的前颞叶语言系统之外。此外，这项研究提供了经验证据，支持使用人工神经网络作为计算模型来揭示支撑大脑复杂认知过程的神经动力学。
