Abstract:
Intramodal and cross-modal perceptual grouping based on the spatial proximity and temporal closeness between multiple sensory stimuli, as an operational principle has built a coherent and meaningful representation of the multisensory event/object. To implement and investigate the cross-modal perceptual grouping, researchers have employed excellent paradigms of spatial/temporal ventriloquism and cross-modal dynamic capture and have revealed the conditional constraints as well as the functional facilitations among various correspondence of sensory properties, with featured behavioral evidence, computational framework as well as brain oscillation patterns. Typically, synesthetic correspondence as a special type of cross-modal correspondence can shape the efficiency and effect-size of cross-modal interaction. For example, factors such as pitch/loudness in the auditory dimension with size/brightness in the visual dimension could modulate the strength of the cross-modal temporal capture. The empirical behavioral findings, as well as psychophysical and neurophysiological evidence to address the cross-modal perceptual grouping and synesthetic correspondence, were summarized in this review. Finally, the potential applications (such as artificial synesthesia device) and how synesthetic correspondence interface with semantics (sensory linguistics), as well as the promising research questions in this field have been discussed.
摘要:
基于多个感官刺激之间的空间接近度和时间接近度的模态内和跨模态感知分组,作为一种操作原理,建立了多感官事件/对象的连贯和有意义的表示。为了实现和研究跨模态感知分组,研究人员采用了出色的空间/时间口技和跨模态动态捕获范式,并揭示了条件约束以及各种感官属性对应之间的功能促进,有特色的行为证据,计算框架以及大脑振荡模式。通常,联觉对应作为跨模态对应的一种特殊类型,可以塑造跨模态交互的效率和效果大小。例如,诸如听觉维度中的音调/响度与视觉维度中的大小/亮度的因素可以调节跨模态时间捕获的强度。实证行为发现,以及解决跨模态知觉分组和联觉对应的心理物理学和神经生理学证据,在这篇综述中进行了总结。最后,潜在的应用(如人工联觉装置)以及联觉对应如何与语义(感觉语言学),以及在这一领域有希望的研究问题进行了讨论。