PDF(Pubmed)
Abstract：
UNASSIGNED: To assess the performance of Chat Generative Pre-Trained Transformer-4 in providing accurate diagnoses to retina teaching cases from OCTCases.
UNASSIGNED: Cross-sectional study.
UNASSIGNED: Retina teaching cases from OCTCases.
UNASSIGNED: We prompted a custom chatbot with 69 retina cases containing multimodal ophthalmic images, asking it to provide the most likely diagnosis. In a sensitivity analysis, we inputted increasing amounts of clinical information pertaining to each case until the chatbot achieved a correct diagnosis. We performed multivariable logistic regressions on Stata v17.0 (StataCorp LLC) to investigate associations between the amount of text-based information inputted per prompt and the odds of the chatbot achieving a correct diagnosis, adjusting for the laterality of cases, number of ophthalmic images inputted, and imaging modalities.
UNASSIGNED: Our primary outcome was the proportion of cases for which the chatbot was able to provide a correct diagnosis. Our secondary outcome was the chatbot\'s performance in relation to the amount of text-based information accompanying ophthalmic images.
UNASSIGNED: Across 69 retina cases collectively containing 139 ophthalmic images, the chatbot was able to provide a definitive, correct diagnosis for 35 (50.7%) cases. The chatbot needed variable amounts of clinical information to achieve a correct diagnosis, where the entire patient description as presented by OCTCases was required for a majority of correctly diagnosed cases (23 of 35 cases, 65.7%). Relative to when the chatbot was only prompted with a patient\'s age and sex, the chatbot achieved a higher odds of a correct diagnosis when prompted with an entire patient description (odds ratio = 10.1, 95% confidence interval = 3.3-30.3, P < 0.01). Despite providing an incorrect diagnosis for 34 (49.3%) cases, the chatbot listed the correct diagnosis within its differential diagnosis for 7 (20.6%) of these incorrectly answered cases.
UNASSIGNED: This custom chatbot was able to accurately diagnose approximately half of the retina cases requiring multimodal input, albeit relying heavily on text-based contextual information that accompanied ophthalmic images. The diagnostic ability of the chatbot in interpretation of multimodal imaging without text-based information is currently limited. The appropriate use of the chatbot in this setting is of utmost importance, given bioethical concerns.
UNASSIGNED: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
UNASSIGNED: Cross-sectional study.
UNASSIGNED: Retina teaching cases from OCTCases.
UNASSIGNED: We prompted a custom chatbot with 69 retina cases containing multimodal ophthalmic images, asking it to provide the most likely diagnosis. In a sensitivity analysis, we inputted increasing amounts of clinical information pertaining to each case until the chatbot achieved a correct diagnosis. We performed multivariable logistic regressions on Stata v17.0 (StataCorp LLC) to investigate associations between the amount of text-based information inputted per prompt and the odds of the chatbot achieving a correct diagnosis, adjusting for the laterality of cases, number of ophthalmic images inputted, and imaging modalities.
UNASSIGNED: Our primary outcome was the proportion of cases for which the chatbot was able to provide a correct diagnosis. Our secondary outcome was the chatbot\'s performance in relation to the amount of text-based information accompanying ophthalmic images.
UNASSIGNED: Across 69 retina cases collectively containing 139 ophthalmic images, the chatbot was able to provide a definitive, correct diagnosis for 35 (50.7%) cases. The chatbot needed variable amounts of clinical information to achieve a correct diagnosis, where the entire patient description as presented by OCTCases was required for a majority of correctly diagnosed cases (23 of 35 cases, 65.7%). Relative to when the chatbot was only prompted with a patient\'s age and sex, the chatbot achieved a higher odds of a correct diagnosis when prompted with an entire patient description (odds ratio = 10.1, 95% confidence interval = 3.3-30.3, P < 0.01). Despite providing an incorrect diagnosis for 34 (49.3%) cases, the chatbot listed the correct diagnosis within its differential diagnosis for 7 (20.6%) of these incorrectly answered cases.
UNASSIGNED: This custom chatbot was able to accurately diagnose approximately half of the retina cases requiring multimodal input, albeit relying heavily on text-based contextual information that accompanied ophthalmic images. The diagnostic ability of the chatbot in interpretation of multimodal imaging without text-based information is currently limited. The appropriate use of the chatbot in this setting is of utmost importance, given bioethical concerns.
UNASSIGNED: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
摘要：
■评估ChatGenerativePre-TrainedTransformer-4在为OCTCases的视网膜教学案例提供准确诊断方面的性能。
■横断面研究。
■视网膜教学案例来自OCTCases。
■我们提示了一个定制的聊天机器人，其中有69个视网膜病例包含多模式眼科图像，要求它提供最有可能的诊断。在敏感性分析中，我们输入了与每个病例相关的越来越多的临床信息，直到聊天机器人获得正确的诊断。我们对Statav17.0（StataCorpLLC）进行了多变量逻辑回归，以调查每个提示输入的基于文本的信息量与聊天机器人实现正确诊断的几率之间的关联。调整案件的横向度，输入的眼科图像数量，和成像模式。
■我们的主要结果是聊天机器人能够提供正确诊断的病例比例。我们的次要结果是聊天机器人的表现与眼科图像附带的基于文本的信息量有关。
■在69个视网膜病例中，总共包含139张眼科图像，聊天机器人能够提供一个明确的信息，正确诊断35例(50.7%)。聊天机器人需要可变数量的临床信息来实现正确的诊断，对于大多数正确诊断的病例（35例中的23例，65.7%）。相对于聊天机器人只被提示患者的年龄和性别，当提示完整的患者描述时,聊天机器人获得了较高的正确诊断几率(比值比=10.1,95%置信区间=3.3-30.3,P<0.01).尽管对34例（49.3%）病例提供了不正确的诊断，在这些错误回答的病例中，chatbot在其鉴别诊断中列出了7例（20.6%）的正确诊断。
■这个定制的聊天机器人能够准确诊断大约一半需要多模态输入的视网膜病例，尽管严重依赖伴随眼科图像的基于文本的上下文信息。聊天机器人在没有基于文本的信息的多模态成像解释中的诊断能力目前是有限的。在此设置中适当使用聊天机器人是至关重要的，考虑到生物伦理问题。
■专有或商业披露可在本文末尾的脚注和披露中找到。
■横断面研究。
■视网膜教学案例来自OCTCases。
■我们提示了一个定制的聊天机器人，其中有69个视网膜病例包含多模式眼科图像，要求它提供最有可能的诊断。在敏感性分析中，我们输入了与每个病例相关的越来越多的临床信息，直到聊天机器人获得正确的诊断。我们对Statav17.0（StataCorpLLC）进行了多变量逻辑回归，以调查每个提示输入的基于文本的信息量与聊天机器人实现正确诊断的几率之间的关联。调整案件的横向度，输入的眼科图像数量，和成像模式。
■我们的主要结果是聊天机器人能够提供正确诊断的病例比例。我们的次要结果是聊天机器人的表现与眼科图像附带的基于文本的信息量有关。
■在69个视网膜病例中，总共包含139张眼科图像，聊天机器人能够提供一个明确的信息，正确诊断35例(50.7%)。聊天机器人需要可变数量的临床信息来实现正确的诊断，对于大多数正确诊断的病例（35例中的23例，65.7%）。相对于聊天机器人只被提示患者的年龄和性别，当提示完整的患者描述时,聊天机器人获得了较高的正确诊断几率(比值比=10.1,95%置信区间=3.3-30.3,P<0.01).尽管对34例（49.3%）病例提供了不正确的诊断，在这些错误回答的病例中，chatbot在其鉴别诊断中列出了7例（20.6%）的正确诊断。
■这个定制的聊天机器人能够准确诊断大约一半需要多模态输入的视网膜病例，尽管严重依赖伴随眼科图像的基于文本的上下文信息。聊天机器人在没有基于文本的信息的多模态成像解释中的诊断能力目前是有限的。在此设置中适当使用聊天机器人是至关重要的，考虑到生物伦理问题。
■专有或商业披露可在本文末尾的脚注和披露中找到。
