背景:大型语言模型(LLM)是从文本数据推断的机器学习模型,该模型捕获了上下文中语言使用的微妙模式。现代LLM基于结合了变压器方法的神经网络架构。它们允许模型通过关注文本序列中的多个单词来将单词联系在一起。LLM已被证明对自然语言处理(NLP)中的一系列任务非常有效,包括分类和信息提取任务以及生成应用程序。
目的:这项改编的Delphi研究的目的是收集研究人员关于LLM如何影响医疗保健和优势的意见,弱点,机遇,以及LLM在医疗保健中使用的威胁。
方法:我们邀请了健康信息学领域的研究人员,护理信息学,和医学NLP分享他们对医疗保健中LLM使用的看法。我们从第一轮开始,根据我们的优势提出了开放的问题,弱点,机遇,威胁框架。在第二轮和第三轮,参与者对这些项目进行了评分。
结果:第一个,第二,第三轮有28、23和21名参与者,分别。几乎所有参与者(26/28,第一轮93%和20/21,第三轮95%)都隶属于学术机构。就与用例相关的103项达成了协议,好处,风险,可靠性,采用方面,以及LLM在医疗保健领域的未来。参与者提供了几个用例,包括支持临床任务,文档任务,医学研究和教育,并同意基于LLM的系统将充当患者教育的健康助手。商定的好处包括提高数据处理和提取的效率,提高流程的自动化程度,提高医疗保健服务质量和整体健康结果,提供个性化护理,加速诊断和治疗过程,并改善患者和医疗保健专业人员之间的互动。总的来说,总体上确定了5种医疗保健风险:网络安全漏洞,潜在的病人错误信息,伦理问题,有偏见的决策的可能性,以及与不准确沟通相关的风险。基于LLM的系统中的过度自信被认为是对医学界的风险。6个商定的隐私风险包括使用不受监管的云服务,损害数据安全。暴露敏感的患者数据,违反保密规定,欺诈性使用信息,数据存储和通信中的漏洞,以及对患者数据的不当访问或使用。
结论:与LLM相关的未来研究不仅应专注于测试其与NLP相关任务的可能性,还应考虑模型可能有助于的工作流程以及有关质量的要求,一体化,以及在实践中成功实施所需的法规。
A large language model (LLM) is a machine learning model inferred from text data that captures subtle patterns of language use in context. Modern LLMs are based on neural network architectures that incorporate transformer methods. They allow the model to relate words together through attention to multiple words in a text sequence. LLMs have been shown to be highly effective for a range of tasks in natural language processing (NLP), including classification and information extraction tasks and generative applications.
The aim of this adapted Delphi
study was to collect researchers\' opinions on how LLMs might influence health care and on the strengths, weaknesses, opportunities, and threats of LLM use in health care.
We invited researchers in the fields of health
informatics, nursing
informatics, and medical NLP to share their opinions on LLM use in health care. We started the first round with open questions based on our strengths, weaknesses, opportunities, and threats framework. In the second and third round, the participants scored these items.
The first, second, and third rounds had 28, 23, and 21 participants, respectively. Almost all participants (26/28, 93% in round 1 and 20/21, 95% in round 3) were affiliated with academic institutions. Agreement was reached on 103 items related to use cases, benefits, risks, reliability, adoption aspects, and the future of LLMs in health care. Participants offered several use cases, including supporting clinical tasks, documentation tasks, and medical research and education, and agreed that LLM-based systems will act as health assistants for patient education. The agreed-upon benefits included increased efficiency in data handling and extraction, improved automation of processes, improved quality of health care services and overall health outcomes, provision of personalized care, accelerated diagnosis and treatment processes, and improved interaction between patients and health care professionals. In total, 5 risks to health care in general were identified: cybersecurity breaches, the potential for patient misinformation, ethical concerns, the likelihood of biased decision-making, and the risk associated with inaccurate communication. Overconfidence in LLM-based systems was recognized as a risk to the medical profession. The 6 agreed-upon privacy risks included the use of unregulated cloud services that compromise data security, exposure of sensitive patient data, breaches of confidentiality, fraudulent use of information, vulnerabilities in data storage and communication, and inappropriate access or use of patient data.
Future research related to LLMs should not only focus on testing their possibilities for NLP-related tasks but also consider the workflows the models could contribute to and the requirements regarding quality, integration, and regulations needed for successful implementation in practice.