背景:随着人工智能(AI)的快速发展,特别是大型语言模型(LLM),如ChatGPT-4(OpenAI),人们对他们协助学术任务的潜力越来越感兴趣,包括进行文献综述。然而,与传统的人类主导方法相比,人工智能生成的综述的有效性仍未得到充分探索.
目的:本研究旨在比较ChatGPT-4模型与人类研究人员进行的文献综述的质量,专注于医生和患者之间的关系动态。
方法:我们在同一主题的研究中纳入了2篇文献综述,即,探索法医学背景下影响医生和患者关系动态的因素。其中一项研究使用了GPT-4,最后更新于2021年9月,另一项研究是由人类研究人员进行的。人类评论涉及使用OvidMEDLINE中的医学主题词和关键词进行全面的文献检索,然后对文献进行主题分析,从选定的文章中综合信息。人工智能生成的审查使用了一种新的即时工程方法,使用迭代和顺序提示生成结果。比较分析基于定性措施,如准确性,响应时间,一致性,知识的广度和深度,上下文理解,和透明度。
结果:GPT-4迅速产生了广泛的关系因素列表。人工智能模型展示了令人印象深刻的知识广度,但在深度和上下文理解方面表现出局限性,偶尔产生不相关或不正确的信息。相比之下,人类研究人员提供了一个更细致入微和上下文相关的综述。比较分析根据包括准确性在内的标准评估评论,响应时间,一致性,知识的广度和深度,上下文理解,和透明度。虽然GPT-4在响应时间和知识广度方面显示出优势,以人为主导的评论在准确性方面表现出色,知识的深度,和上下文理解。
结论:研究表明,GPT-4具有结构化的即时工程,通过快速提供广泛的主题概述,可以成为进行初步文献综述的宝贵工具。然而,它的局限性需要仔细的专家评估和完善,使其成为综合文献综述中人类专业知识的助手而不是替代品。此外,这项研究强调了在学术研究中使用GPT-4等人工智能工具的潜力和局限性,特别是在卫生服务和医学研究领域。它强调了将AI的快速信息检索能力与人类专业知识相结合的必要性,以获得更准确和上下文丰富的学术产出。
BACKGROUND: With the rapid evolution of artificial intelligence (AI), particularly large language models (LLMs) such as ChatGPT-4 (OpenAI), there is an increasing interest in their potential to assist in scholarly tasks, including conducting literature reviews. However, the efficacy of AI-generated reviews compared with traditional human-led approaches remains underexplored.
OBJECTIVE: This study aims to compare the quality of literature reviews conducted by the ChatGPT-4 model with those conducted by human researchers, focusing on the relational dynamics between physicians and patients.
METHODS: We included 2 literature reviews in the study on the same topic, namely, exploring factors affecting relational dynamics between physicians and patients in medicolegal contexts. One review used GPT-4, last updated in September 2021, and the other was conducted by human researchers. The human review involved a comprehensive literature search using medical subject headings and keywords in Ovid MEDLINE, followed by a thematic analysis of the literature to synthesize information from selected articles. The AI-generated review used a new prompt engineering approach, using iterative and sequential prompts to generate results. Comparative analysis was based on qualitative measures such as accuracy, response time, consistency, breadth and depth of knowledge, contextual understanding, and transparency.
RESULTS: GPT-4 produced an extensive list of relational factors rapidly. The AI model demonstrated an impressive breadth of knowledge but exhibited limitations in in-depth and contextual understanding, occasionally producing irrelevant or incorrect information. In comparison, human researchers provided a more nuanced and contextually relevant review. The comparative analysis assessed the reviews based on criteria including accuracy, response time, consistency, breadth and depth of knowledge, contextual understanding, and transparency. While GPT-4 showed advantages in response time and breadth of knowledge, human-led reviews excelled in accuracy, depth of knowledge, and contextual understanding.
CONCLUSIONS: The study suggests that GPT-4, with structured prompt engineering, can be a valuable tool for conducting preliminary literature reviews by providing a broad overview of topics quickly. However, its limitations necessitate careful expert evaluation and refinement, making it an assistant rather than a substitute for human expertise in comprehensive literature reviews. Moreover, this research highlights the potential and limitations of using AI tools like GPT-4 in academic research, particularly in the fields of health services and medical research. It underscores the necessity of combining AI\'s rapid information retrieval capabilities with human expertise for more accurate and contextually rich scholarly outputs.