背景:大型语言模型(LLM)具有心理健康应用的潜力。然而,他们不透明的对齐过程可能会嵌入偏见,形成有问题的观点。评估嵌入在LLM中指导其决策的价值观具有道德重要性。施瓦茨的基本价值观理论(STBV)为量化文化价值取向提供了一个框架,并显示了在心理健康环境中检查价值观的效用。包括文化,诊断,和治疗师-客户动态。
目的:这项研究旨在(1)评估STBV是否可以测量领先的LLM中的价值样构建体,以及(2)确定LLM是否表现出与人类和彼此不同的价值样模式。
方法:总共,4名法学硕士(吟游诗人,克劳德2,生成预训练变压器[GPT]-3.5,GPT-4)被拟人化,并指示完成肖像值问卷修订(PVQ-RR)以评估类似价值的构造。对他们在10项试验中的反应进行了信度和效度分析。要对LLM值配置文件进行基准测试,将他们的结果与来自49个国家的53,472名完成PVQ-RR的不同样本的已发表数据进行比较.这使我们能够评估LLM是否与跨文化群体的既定人类价值模式有所不同。还通过统计检验比较了模型之间的值概况。
结果:PVQ-RR显示出良好的信度和效度,用于量化LLM内的价值式基础设施。然而,LLM的价值概况和人口数据之间出现了很大的差异。这些模型缺乏共识,表现出明显的动机偏见,反映不透明的对齐过程。例如,所有模式都优先考虑普遍主义和自我导向,在不强调成就的同时,电源,和相对于人类的安全。成功的判别分析区分了4个不同的LLM值概况。进一步的检查发现,当出现心理健康困境时,有偏见的价值概况强烈预测了LLM的反应,需要在相反的价值之间进行选择。这为嵌入塑造其决策的独特动机价值样结构的模型提供了进一步的验证。
结论:这项研究利用了STBV来映射激励领先LLM的类价值基础设施。尽管研究表明STBV可以有效地表征LLM中的类价值基础设施,与人类价值观的巨大分歧引发了人们对将这些模型与心理健康应用保持一致的道德担忧。如果在没有适当保障措施的情况下进行整合,对某些文化价值集的偏见会带来风险。例如,即使在临床上不明智的情况下,优先考虑普遍性也可以促进无条件接受。此外,LLM之间的差异强调了标准化调整过程以捕获真正的文化多样性的必要性。因此,任何负责任的将LLM整合到精神卫生保健中都必须考虑到其嵌入的偏见和动机不匹配,以确保跨不同人群的公平交付。实现这一目标将需要透明和完善对齐技术,以灌输全面的人类价值观。
BACKGROUND: Large language models (LLMs) hold potential for mental health applications. However, their opaque alignment processes may embed biases that shape problematic perspectives. Evaluating the values embedded within LLMs that guide their decision-making have ethical importance. Schwartz\'s theory of basic values (STBV) provides a framework for quantifying cultural value orientations and has shown utility for examining values in mental health contexts, including cultural, diagnostic, and therapist-client dynamics.
OBJECTIVE: This
study aimed to (1) evaluate whether the STBV can measure value-like constructs within leading LLMs and (2) determine whether LLMs exhibit distinct value-like patterns from humans and each other.
METHODS: In total, 4 LLMs (Bard, Claude 2, Generative Pretrained Transformer [GPT]-3.5, GPT-4) were anthropomorphized and instructed to complete the Portrait Values Questionnaire-Revised (PVQ-RR) to assess value-like constructs. Their responses over 10 trials were analyzed for reliability and validity. To benchmark the LLMs\' value profiles, their results were compared to published data from a diverse sample of 53,472 individuals across 49 nations who had completed the PVQ-RR. This allowed us to assess whether the LLMs diverged from established human value patterns across cultural groups. Value profiles were also compared between models via statistical tests.
RESULTS: The PVQ-RR showed good reliability and validity for quantifying value-like infrastructure within the LLMs. However, substantial divergence emerged between the LLMs\' value profiles and population data. The models lacked consensus and exhibited distinct motivational biases, reflecting opaque alignment processes. For example, all models prioritized universalism and self-direction, while de-emphasizing achievement, power, and security relative to humans. Successful discriminant analysis differentiated the 4 LLMs\' distinct value profiles. Further examination found the biased value profiles strongly predicted the LLMs\' responses when presented with mental health dilemmas requiring choosing between opposing values. This provided further validation for the models embedding distinct motivational value-like constructs that shape their decision-making.
CONCLUSIONS: This
study leveraged the STBV to map the motivational value-like infrastructure underpinning leading LLMs. Although the
study demonstrated the STBV can effectively characterize value-like infrastructure within LLMs, substantial divergence from human values raises ethical concerns about aligning these models with mental health applications. The biases toward certain cultural value sets pose risks if integrated without proper safeguards. For example, prioritizing universalism could promote unconditional acceptance even when clinically unwise. Furthermore, the differences between the LLMs underscore the need to standardize alignment processes to capture true cultural diversity. Thus, any responsible integration of LLMs into mental health care must account for their embedded biases and motivation mismatches to ensure equitable delivery across diverse populations. Achieving this will require transparency and refinement of alignment techniques to instill comprehensive human values.