PDF(Pubmed)
Abstract：
BACKGROUND: Large language models (LLMs) are computational artificial intelligence systems with advanced natural language processing capabilities that have recently been popularized among health care students and educators due to their ability to provide real-time access to a vast amount of medical knowledge. The adoption of LLM technology into medical education and training has varied, and little empirical evidence exists to support its use in clinical teaching environments.
OBJECTIVE: The aim of the study is to identify and qualitatively evaluate potential use cases and limitations of LLM technology for real-time ward-based educational contexts.
METHODS: A brief, single-site exploratory evaluation of the publicly available ChatGPT-3.5 (OpenAI) was conducted by implementing the tool into the daily attending rounds of a general internal medicine inpatient service at a large urban academic medical center. ChatGPT was integrated into rounds via both structured and organic use, using the web-based \"chatbot\" style interface to interact with the LLM through conversational free-text and discrete queries. A qualitative approach using phenomenological inquiry was used to identify key insights related to the use of ChatGPT through analysis of ChatGPT conversation logs and associated shorthand notes from the clinical sessions.
RESULTS: Identified use cases for ChatGPT integration included addressing medical knowledge gaps through discrete medical knowledge inquiries, building differential diagnoses and engaging dual-process thinking, challenging medical axioms, using cognitive aids to support acute care decision-making, and improving complex care management by facilitating conversations with subspecialties. Potential additional uses included engaging in difficult conversations with patients, exploring ethical challenges and general medical ethics teaching, personal continuing medical education resources, developing ward-based teaching tools, supporting and automating clinical documentation, and supporting productivity and task management. LLM biases, misinformation, ethics, and health equity were identified as areas of concern and potential limitations to clinical and training use. A code of conduct on ethical and appropriate use was also developed to guide team usage on the wards.
CONCLUSIONS: Overall, ChatGPT offers a novel tool to enhance ward-based learning through rapid information querying, second-order content exploration, and engaged team discussion regarding generated responses. More research is needed to fully understand contexts for educational use, particularly regarding the risks and limitations of the tool in clinical settings and its impacts on trainee development.
OBJECTIVE: The aim of the study is to identify and qualitatively evaluate potential use cases and limitations of LLM technology for real-time ward-based educational contexts.
METHODS: A brief, single-site exploratory evaluation of the publicly available ChatGPT-3.5 (OpenAI) was conducted by implementing the tool into the daily attending rounds of a general internal medicine inpatient service at a large urban academic medical center. ChatGPT was integrated into rounds via both structured and organic use, using the web-based \"chatbot\" style interface to interact with the LLM through conversational free-text and discrete queries. A qualitative approach using phenomenological inquiry was used to identify key insights related to the use of ChatGPT through analysis of ChatGPT conversation logs and associated shorthand notes from the clinical sessions.
RESULTS: Identified use cases for ChatGPT integration included addressing medical knowledge gaps through discrete medical knowledge inquiries, building differential diagnoses and engaging dual-process thinking, challenging medical axioms, using cognitive aids to support acute care decision-making, and improving complex care management by facilitating conversations with subspecialties. Potential additional uses included engaging in difficult conversations with patients, exploring ethical challenges and general medical ethics teaching, personal continuing medical education resources, developing ward-based teaching tools, supporting and automating clinical documentation, and supporting productivity and task management. LLM biases, misinformation, ethics, and health equity were identified as areas of concern and potential limitations to clinical and training use. A code of conduct on ethical and appropriate use was also developed to guide team usage on the wards.
CONCLUSIONS: Overall, ChatGPT offers a novel tool to enhance ward-based learning through rapid information querying, second-order content exploration, and engaged team discussion regarding generated responses. More research is needed to fully understand contexts for educational use, particularly regarding the risks and limitations of the tool in clinical settings and its impacts on trainee development.
摘要：
背景：大型语言模型（LLM）是具有高级自然语言处理能力的计算人工智能系统，由于其能够提供对大量医学知识的实时访问，最近在医疗保健学生和教育工作者中普及。LLM技术在医学教育和培训中的应用各不相同，几乎没有经验证据支持其在临床教学环境中的使用。
目的：研究的目的是确定和定性评估LLM技术在实时基于病房的教育环境中的潜在用例和局限性。
方法：简短，通过在大型城市学术医疗中心的普通内科住院服务的每日就诊中实施该工具，对公开可用的ChatGPT-3.5（OpenAI）进行了单站点探索性评估。ChatGPT通过结构化和有机使用整合到回合中，使用基于Web的“chatbot”样式界面通过对话自由文本和离散查询与LLM交互。通过分析ChatGPT对话日志和临床会话中的相关速记注释，使用现象学查询的定性方法来识别与使用ChatGPT相关的关键见解。
结果：确定的ChatGPT集成用例包括通过离散的医学知识查询来解决医学知识差距，建立鉴别诊断和参与双过程思维，具有挑战性的医学公理，使用认知辅助手段来支持急性护理决策，并通过促进与亚专科的对话来改善复杂的护理管理。潜在的额外用途包括与患者进行艰难的对话，探索伦理挑战和一般医学伦理教学，个人继续医学教育资源，开发基于病房的教学工具，支持和自动化临床文档，并支持生产力和任务管理。LLM偏见，错误信息,伦理,健康公平被确定为临床和培训使用的关注领域和潜在限制。还制定了有关道德和适当使用的行为准则，以指导团队在病房中的使用。
结论：总体而言，ChatGPT提供了一种新颖的工具，可以通过快速的信息查询来增强基于病房的学习，二阶内容探索，并就生成的响应进行团队讨论。需要更多的研究来充分了解教育用途的背景，特别是关于该工具在临床环境中的风险和局限性及其对培训生发展的影响。
目的：研究的目的是确定和定性评估LLM技术在实时基于病房的教育环境中的潜在用例和局限性。
方法：简短，通过在大型城市学术医疗中心的普通内科住院服务的每日就诊中实施该工具，对公开可用的ChatGPT-3.5（OpenAI）进行了单站点探索性评估。ChatGPT通过结构化和有机使用整合到回合中，使用基于Web的“chatbot”样式界面通过对话自由文本和离散查询与LLM交互。通过分析ChatGPT对话日志和临床会话中的相关速记注释，使用现象学查询的定性方法来识别与使用ChatGPT相关的关键见解。
结果：确定的ChatGPT集成用例包括通过离散的医学知识查询来解决医学知识差距，建立鉴别诊断和参与双过程思维，具有挑战性的医学公理，使用认知辅助手段来支持急性护理决策，并通过促进与亚专科的对话来改善复杂的护理管理。潜在的额外用途包括与患者进行艰难的对话，探索伦理挑战和一般医学伦理教学，个人继续医学教育资源，开发基于病房的教学工具，支持和自动化临床文档，并支持生产力和任务管理。LLM偏见，错误信息,伦理,健康公平被确定为临床和培训使用的关注领域和潜在限制。还制定了有关道德和适当使用的行为准则，以指导团队在病房中的使用。
结论：总体而言，ChatGPT提供了一种新颖的工具，可以通过快速的信息查询来增强基于病房的学习，二阶内容探索，并就生成的响应进行团队讨论。需要更多的研究来充分了解教育用途的背景，特别是关于该工具在临床环境中的风险和局限性及其对培训生发展的影响。
