%0 Journal Article
%T Capability of GPT-4V(ision) in the Japanese National Medical Licensing Examination: Evaluation Study.
%A Nakao T
%A Miki S
%A Nakamura Y
%A Kikuchi T
%A Nomura Y
%A Hanaoka S
%A Yoshikawa T
%A Abe O
%J JMIR Med Educ
%V 10
%N 0
%D 2024 Mar 12
%M 38470459
暂无%R 10.2196/54393
%X <B>BACKGROUND: </B>Previous research applying large language models (LLMs) to medicine was focused on text-based information. Recently, multimodal variants of LLMs acquired the capability of recognizing images.<BR><B>OBJECTIVE: </B>We aim to evaluate the image recognition capability of generative pretrained transformer (GPT)-4V, a recent multimodal LLM developed by OpenAI, in the medical field by testing how visual information affects its performance to answer questions in the 117th Japanese National Medical Licensing Examination.<BR><B>METHODS: </B>We focused on 108 questions that had 1 or more images as part of a question and presented GPT-4V with the same questions under two conditions: (1) with both the question text and associated images and (2) with the question text only. We then compared the difference in accuracy between the 2 conditions using the exact McNemar test.<BR><B>RESULTS: </B>Among the 108 questions with images, GPT-4V's accuracy was 68% (73/108) when presented with images and 72% (78/108) when presented without images (P=.36). For the 2 question categories, clinical and general, the accuracies with and those without images were 71% (70/98) versus 78% (76/98; P=.21) and 30% (3/10) versus 20% (2/10; P≥.99), respectively.<BR><B>CONCLUSIONS: </B>The additional information from the images did not significantly improve the performance of GPT-4V in the Japanese National Medical Licensing Examination.