交叉模态对应关系,一种感觉形态中的感觉特征/属性的倾向(无论是身体上的还是仅仅想象的),与另一种感觉形态中的感觉特征相关联,已经被广泛研究,揭示一致的模式,例如甜味与粉红色和各种语言的圆形有关。本研究探讨了ChatGPT是否捕获了这样的对应关系,由OpenAI开发的大型语言模型。在十二项研究中,这项研究调查了ChatGPT-3.5和-4o中的颜色/形状-味道交叉模态对应关系,专注于形状/颜色与三种语言的五种基本品味之间的关联(英语,日本人,和西班牙语)。研究1A-F检查了味形关联,使用三种语言的提示来评估ChatGPT的圆形和棱角形状与五种基本口味的关联。结果表明,一致,形状和味道之间的联系,with,例如,与甜/鲜味强烈相关的圆形和带有苦/咸/酸味的棱角形状。ChatGPT-4o中的形状-味道匹配程度似乎大于ChatGPT-3.5中的形状-味道匹配程度,并且ChatGPT用英语和西班牙语提示的比ChatGPT用日语提示的要大。研究2A-F专注于颜色-味道对应,使用ChatGPT评估11种颜色与五种基本口味之间的关联。结果表明,ChatGPT-4o,而不是ChatGPT-3.5,通常复制先前在人类参与者中观察到的颜色-味道对应的模式。具体来说,ChatGPT-4o将甜味与粉红色联系起来,黄色的酸,咸白色/蓝色,苦涩的黑色,和不同语言的红色鲜味。然而,在ChatGPT-4o中观察到的形状/颜色-味道匹配的幅度/相似性似乎更明显(即,方差很小,较大的平均差异),这不能充分反映通常在人类形状/颜色-味道对应关系中看到的细微细微差别。这些发现表明,ChatGPT捕获颜色/形状-味道对应关系,使用语言和GPT版本特定的变体,尽管与以前涉及人类参与者的研究相比有一些差异。这些发现为跨模态对应关系领域贡献了宝贵的知识,探索类似于人类感知系统和跨语言认知的生成AI的可能性,并提供对捕获人类跨模态对应关系的生成AI系统的开发和演变的见解。
Crossmodal correspondences, the tendency for a sensory feature / attribute in one sensory modality (either physically present or merely imagined), to be associated with a sensory feature in another sensory modality, have been studied extensively, revealing consistent patterns, such as sweet tastes being associated with pink colours and round shapes across languages. The present research explores whether such correspondences are captured by ChatGPT, a large language model developed by OpenAI. Across twelve studies, this research investigates colour/shapes-taste crossmodal correspondences in ChatGPT-3.5 and -4o, focusing on associations between shapes/colours and the five basic tastes across three languages (English, Japanese, and Spanish). Studies 1A-F examined taste-
shape associations, using prompts in three languages to assess ChatGPT\'s association of round and angular shapes with the five basic tastes. The results indicated significant, consistent, associations between
shape and taste, with, for example, round shapes strongly associated with sweet/umami tastes and angular shapes with bitter/salty/sour tastes. The magnitude of
shape-taste matching appears to be greater in ChatGPT-4o than in ChatGPT-3.5, and ChatGPT prompted in English and Spanish than ChatGPT prompted in Japanese. Studies 2A-F focused on colour-taste correspondences, using ChatGPT to assess associations between eleven colours and the five basic tastes. The results indicated that ChatGPT-4o, but not ChatGPT-3.5, generally replicates the patterns of colour-taste correspondences that have previously been observed in human participants. Specifically, ChatGPT-4o associates sweet tastes with pink, sour with yellow, salty with white/blue, bitter with black, and umami with red across languages. However, the magnitude/similarity of
shape/colour-taste matching observed in ChatGPT-4o appears to be more pronounced (i.e., having little variance, large mean difference), which does not adequately reflect the subtle nuances typically seen in human
shape/colour-taste correspondences. These findings suggest that ChatGPT captures colour/shapes-taste correspondences, with language- and GPT version-specific variations, albeit with some differences when compared to previous studies involving human participants. These findings contribute valuable knowledge to the field of crossmodal correspondences, explore the possibility of generative AI that resembles human perceptual systems and cognition across languages, and provide insight into the development and evolution of generative AI systems that capture human crossmodal correspondences.