■尽管通过电子健康记录直接接触临床医生,患者越来越多地转向互联网获取与他们的健康相关的信息,尤其是敏感的泌尿系统疾病,如佩罗尼病(PD)。大型语言模型(LLM)聊天机器人是一种人工智能形式,它依赖于用户提示来模仿对话,他们表现出了非凡的能力。这些聊天机器人的对话性质有可能回答与PD相关的患者问题;然而,准确性,全面性,这些与PD相关的LLM的可读性仍然未知。
■为了评估从4个LLM生成的与PD相关的搜索信息的质量和可读性;查看用户是否可以改善响应;并评估准确性,完整性,以及在接受PD手术之前通过电子健康记录发送的人工术前患者问题的可读性。
■美国国立卫生研究院与PD有关的常见问题被输入到4个LLM中,无提示和提示。通过先前验证的DISCERN问卷评估回答的总体质量。使用先前接受的Likert量表评估了LLM对11条术前患者信息的反应的准确性和完整性。所有评估均在2023年10月由3名独立审稿人进行,所有审查均在2024年4月重复进行。进行描述性统计和分析。
■没有提示,所有LLM的信息质量中等,但在提示下提高到高质量。LLM是准确和完整的,平均得分为6.0中的5.5分(SD,0.8)和3.0的2.8(SD,0.4),分别。Flesch-Kincaid的平均阅读水平为12.9级(SD,2.1).聊天机器人在收到提示时无法以8年级的阅读水平进行交流,他们的引用只有42.5%的时间是合适的。
■LLM可能成为PD患者教育的宝贵工具,但它们目前依赖于临床背景和人类的适当提示。不幸的是,他们的先决条件阅读水平仍然高于普通患者,他们的引用是不可信的。然而,鉴于它们越来越多的吸收和可及性,患者和医生应接受如何与这些LLM互动的教育,以引发最合适的反应.在未来,LLM可以通过帮助医生对患者信息做出回应来减少倦怠。
UNASSIGNED: Despite direct access to clinicians through the electronic health record, patients are increasingly turning to the internet for information related to their health, especially with sensitive urologic conditions such as Peyronie\'s disease (PD). Large language model (LLM) chatbots are a form of artificial intelligence that rely on user prompts to mimic conversation, and they have shown remarkable capabilities. The conversational nature of these chatbots has the potential to answer patient questions related to PD; however, the accuracy, comprehensiveness, and readability of these LLMs related to PD remain unknown.
UNASSIGNED: To assess the quality and readability of information generated from 4 LLMs with searches related to PD; to see if users could improve responses; and to assess the accuracy, completeness, and readability of responses to artificial preoperative patient questions sent through the electronic health record prior to undergoing PD surgery.
UNASSIGNED: The National Institutes of Health\'s frequently asked questions related to PD were entered into 4 LLMs, unprompted and prompted. The responses were evaluated for overall quality by the previously validated DISCERN questionnaire. Accuracy and completeness of LLM responses to 11 presurgical patient messages were evaluated with previously accepted Likert scales. All evaluations were performed by 3 independent reviewers in October 2023, and all reviews were repeated in April 2024. Descriptive statistics and analysis were performed.
UNASSIGNED: Without prompting, the quality of information was moderate across all LLMs but improved to high quality with prompting. LLMs were accurate and complete, with an average score of 5.5 of 6.0 (SD, 0.8) and 2.8 of 3.0 (SD, 0.4), respectively. The average Flesch-Kincaid reading level was grade 12.9 (SD, 2.1). Chatbots were unable to communicate at a grade 8 reading level when prompted, and their citations were appropriate only 42.5% of the time.
UNASSIGNED: LLMs may become a valuable tool for patient education for PD, but they currently rely on clinical context and appropriate prompting by humans to be useful. Unfortunately, their prerequisite reading level remains higher than that of the average patient, and their citations cannot be trusted. However, given their increasing uptake and accessibility, patients and physicians should be educated on how to interact with these LLMs to elicit the most appropriate responses. In the future, LLMs may reduce burnout by helping physicians respond to patient messages.