PDF(Pubmed)
Abstract：
UNASSIGNED: ChatGPT is an advanced chatbot based on Large Language Model that has the ability to answer questions. Undoubtedly, ChatGPT is capable of transforming communication, education, and customer support; however, can it play the role of a doctor? In Poland, prior to obtaining a medical diploma, candidates must successfully pass the Medical Final Examination.
UNASSIGNED: The purpose of this research was to determine how well ChatGPT performed on the Polish Medical Final Examination, which passing is required to become a doctor in Poland (an exam is considered passed if at least 56% of the tasks are answered correctly). A total of 2138 categorized Medical Final Examination questions (from 11 examination sessions held between 2013-2015 and 2021-2023) were presented to ChatGPT-3.5 from 19 to 26 May 2023. For further analysis, the questions were divided into quintiles based on difficulty and duration, as well as question types (simple A-type or complex K-type). The answers provided by ChatGPT were compared to the official answer key, reviewed for any changes resulting from the advancement of medical knowledge.
UNASSIGNED: ChatGPT correctly answered 53.4%-64.9% of questions. In 8 out of 11 exam sessions, ChatGPT achieved the scores required to successfully pass the examination (60%). The correlation between the efficacy of artificial intelligence and the level of complexity, difficulty, and length of a question was found to be negative. AI outperformed humans in one category: psychiatry (77.18% vs. 70.25%, p = 0.081).
UNASSIGNED: The performance of artificial intelligence is deemed satisfactory; however, it is observed to be markedly inferior to that of human graduates in the majority of instances. Despite its potential utility in many medical areas, ChatGPT is constrained by its inherent limitations that prevent it from entirely supplanting human expertise and knowledge.
UNASSIGNED: The purpose of this research was to determine how well ChatGPT performed on the Polish Medical Final Examination, which passing is required to become a doctor in Poland (an exam is considered passed if at least 56% of the tasks are answered correctly). A total of 2138 categorized Medical Final Examination questions (from 11 examination sessions held between 2013-2015 and 2021-2023) were presented to ChatGPT-3.5 from 19 to 26 May 2023. For further analysis, the questions were divided into quintiles based on difficulty and duration, as well as question types (simple A-type or complex K-type). The answers provided by ChatGPT were compared to the official answer key, reviewed for any changes resulting from the advancement of medical knowledge.
UNASSIGNED: ChatGPT correctly answered 53.4%-64.9% of questions. In 8 out of 11 exam sessions, ChatGPT achieved the scores required to successfully pass the examination (60%). The correlation between the efficacy of artificial intelligence and the level of complexity, difficulty, and length of a question was found to be negative. AI outperformed humans in one category: psychiatry (77.18% vs. 70.25%, p = 0.081).
UNASSIGNED: The performance of artificial intelligence is deemed satisfactory; however, it is observed to be markedly inferior to that of human graduates in the majority of instances. Despite its potential utility in many medical areas, ChatGPT is constrained by its inherent limitations that prevent it from entirely supplanting human expertise and knowledge.
摘要：
■ChatGPT是基于大型语言模型的高级聊天机器人，具有回答问题的能力。毫无疑问,ChatGPT能够改变通信，教育,和客户支持；然而，它能扮演医生的角色吗？在波兰，在获得医学文凭之前，考生必须顺利通过医学期末考试。
■这项研究的目的是确定ChatGPT在波兰医学最终检查中的表现，在波兰成为医生需要通过（如果至少56％的任务回答正确，则认为考试通过）。2023年5月19日至26日，共有2138个分类的医学期末考试问题（来自2013-2015年和2021-2023年之间举行的11次考试）提交给ChatGPT-3.5。为了进一步分析，这些问题根据难度和持续时间分为五分位数，以及问题类型（简单A型或复杂K型）。ChatGPT提供的答案与官方答案键进行了比较，审查因医学知识的进步而产生的任何变化。
■ChatGPT正确回答了53.4%-64.9%的问题。在11次考试中的8次，ChatGPT达到了成功通过考试所需的分数（60％）。人工智能的功效与复杂性水平之间的相关性，困难,一个问题的长度被发现是否定的。人工智能在一个类别中优于人类：精神病学(77.18%vs.70.25%,p=0.081)。
■人工智能的性能被认为是令人满意的；然而，在大多数情况下，它明显低于人类毕业生。尽管它在许多医疗领域具有潜在的效用，ChatGPT受到其固有局限性的限制，这些局限性使其无法完全取代人类的专业知识和知识。
■这项研究的目的是确定ChatGPT在波兰医学最终检查中的表现，在波兰成为医生需要通过（如果至少56％的任务回答正确，则认为考试通过）。2023年5月19日至26日，共有2138个分类的医学期末考试问题（来自2013-2015年和2021-2023年之间举行的11次考试）提交给ChatGPT-3.5。为了进一步分析，这些问题根据难度和持续时间分为五分位数，以及问题类型（简单A型或复杂K型）。ChatGPT提供的答案与官方答案键进行了比较，审查因医学知识的进步而产生的任何变化。
■ChatGPT正确回答了53.4%-64.9%的问题。在11次考试中的8次，ChatGPT达到了成功通过考试所需的分数（60％）。人工智能的功效与复杂性水平之间的相关性，困难,一个问题的长度被发现是否定的。人工智能在一个类别中优于人类：精神病学(77.18%vs.70.25%,p=0.081)。
■人工智能的性能被认为是令人满意的；然而，在大多数情况下，它明显低于人类毕业生。尽管它在许多医疗领域具有潜在的效用，ChatGPT受到其固有局限性的限制，这些局限性使其无法完全取代人类的专业知识和知识。
