PDF(Pubmed)
Abstract：
BACKGROUND: Academic paper writing holds significant importance in the education of medical students, and poses a clear challenge for those whose first language is not English. This study aims to investigate the effectiveness of employing large language models, particularly ChatGPT, in improving the English academic writing skills of these students.
METHODS: A cohort of 25 third-year medical students from China was recruited. The study consisted of two stages. Firstly, the students were asked to write a mini paper. Secondly, the students were asked to revise the mini paper using ChatGPT within two weeks. The evaluation of the mini papers focused on three key dimensions, including structure, logic, and language. The evaluation method incorporated both manual scoring and AI scoring utilizing the ChatGPT-3.5 and ChatGPT-4 models. Additionally, we employed a questionnaire to gather feedback on students\' experience in using ChatGPT.
RESULTS: After implementing ChatGPT for writing assistance, there was a notable increase in manual scoring by 4.23 points. Similarly, AI scoring based on the ChatGPT-3.5 model showed an increase of 4.82 points, while the ChatGPT-4 model showed an increase of 3.84 points. These results highlight the potential of large language models in supporting academic writing. Statistical analysis revealed no significant difference between manual scoring and ChatGPT-4 scoring, indicating the potential of ChatGPT-4 to assist teachers in the grading process. Feedback from the questionnaire indicated a generally positive response from students, with 92% acknowledging an improvement in the quality of their writing, 84% noting advancements in their language skills, and 76% recognizing the contribution of ChatGPT in supporting academic research.
CONCLUSIONS: The study highlighted the efficacy of large language models like ChatGPT in augmenting the English academic writing proficiency of non-native speakers in medical education. Furthermore, it illustrated the potential of these models to make a contribution to the educational evaluation process, particularly in environments where English is not the primary language.
METHODS: A cohort of 25 third-year medical students from China was recruited. The study consisted of two stages. Firstly, the students were asked to write a mini paper. Secondly, the students were asked to revise the mini paper using ChatGPT within two weeks. The evaluation of the mini papers focused on three key dimensions, including structure, logic, and language. The evaluation method incorporated both manual scoring and AI scoring utilizing the ChatGPT-3.5 and ChatGPT-4 models. Additionally, we employed a questionnaire to gather feedback on students\' experience in using ChatGPT.
RESULTS: After implementing ChatGPT for writing assistance, there was a notable increase in manual scoring by 4.23 points. Similarly, AI scoring based on the ChatGPT-3.5 model showed an increase of 4.82 points, while the ChatGPT-4 model showed an increase of 3.84 points. These results highlight the potential of large language models in supporting academic writing. Statistical analysis revealed no significant difference between manual scoring and ChatGPT-4 scoring, indicating the potential of ChatGPT-4 to assist teachers in the grading process. Feedback from the questionnaire indicated a generally positive response from students, with 92% acknowledging an improvement in the quality of their writing, 84% noting advancements in their language skills, and 76% recognizing the contribution of ChatGPT in supporting academic research.
CONCLUSIONS: The study highlighted the efficacy of large language models like ChatGPT in augmenting the English academic writing proficiency of non-native speakers in medical education. Furthermore, it illustrated the potential of these models to make a contribution to the educational evaluation process, particularly in environments where English is not the primary language.
摘要：
背景：学术论文写作在医学生的教育中具有重要意义，并对母语不是英语的人提出了明显的挑战。本研究旨在调查采用大型语言模型的有效性，尤其是ChatGPT,提高这些学生的英语学术写作能力。
方法：招募了25名来自中国的三年级医学生。该研究包括两个阶段。首先,学生们被要求写一篇迷你论文。其次,要求学生在两周内使用ChatGPT修改迷你论文。对微型文件的评估集中在三个关键方面，包括结构，逻辑,和语言。评估方法结合了使用ChatGPT-3.5和ChatGPT-4模型的手动评分和AI评分。此外,我们采用问卷收集学生使用ChatGPT的经验反馈。
结果：在实施ChatGPT进行写作帮助后，人工得分显着增加了4.23分。同样，基于ChatGPT-3.5模型的AI评分增加了4.82分，而ChatGPT-4模型显示增加3.84点。这些结果凸显了大型语言模型在支持学术写作方面的潜力。统计学分析显示人工评分与ChatGPT-4评分无显著差异,表明ChatGPT-4在评分过程中协助教师的潜力。问卷的反馈表明，学生的反应总体上是积极的，92%的人承认他们的写作质量有所改善，84%的人注意到他们语言技能的进步，76%的人认识到ChatGPT在支持学术研究方面的贡献。
结论：该研究强调了像ChatGPT这样的大型语言模型在医学教育中提高非母语人士的英语学术写作能力的功效。此外，它说明了这些模型对教育评估过程做出贡献的潜力，特别是在英语不是主要语言的环境中。
方法：招募了25名来自中国的三年级医学生。该研究包括两个阶段。首先,学生们被要求写一篇迷你论文。其次,要求学生在两周内使用ChatGPT修改迷你论文。对微型文件的评估集中在三个关键方面，包括结构，逻辑,和语言。评估方法结合了使用ChatGPT-3.5和ChatGPT-4模型的手动评分和AI评分。此外,我们采用问卷收集学生使用ChatGPT的经验反馈。
结果：在实施ChatGPT进行写作帮助后，人工得分显着增加了4.23分。同样，基于ChatGPT-3.5模型的AI评分增加了4.82分，而ChatGPT-4模型显示增加3.84点。这些结果凸显了大型语言模型在支持学术写作方面的潜力。统计学分析显示人工评分与ChatGPT-4评分无显著差异,表明ChatGPT-4在评分过程中协助教师的潜力。问卷的反馈表明，学生的反应总体上是积极的，92%的人承认他们的写作质量有所改善，84%的人注意到他们语言技能的进步，76%的人认识到ChatGPT在支持学术研究方面的贡献。
结论：该研究强调了像ChatGPT这样的大型语言模型在医学教育中提高非母语人士的英语学术写作能力的功效。此外，它说明了这些模型对教育评估过程做出贡献的潜力，特别是在英语不是主要语言的环境中。
