■为了提高各种医学问答(QA)任务的准确性和可靠性,并研究部署大型语言模型(LLM)技术的有效方法,我们利用最先进的LLM开发了一种新颖的集成学习管道,专注于提高不同医疗QA数据集的性能。
■我们的研究采用了三个医学QA数据集:PubMedQA,MedQA-USMLE,和MedMCQA,每个都在生物医学问答中提出了独特的挑战。拟议的LLM-Synergy框架,专注于使用LLM的零镜头案例,结合了两种主要的集成方法。第一个是基于Boosting的加权多数投票合奏,其中,通过增强算法为不同的LLM分配可变权重来加快和完善决策。第二种方法是基于集群的动态模型选择,为每个查询动态选择最合适的LLM投票,根据问题语境的特点,使用聚类方法。
■多数加权投票和动态模型选择方法在三个医疗QA数据集中与单独的LLM相比表现优异。具体来说,准确率为35.84%,96.21%,MedMCQA为37.26%,PubMedQA,和MedQA-USMLE,分别,多数加权投票。相应地,动态模型选择的精度略高,为38.01%,96.36%,38.13%。
■具有两种集成方法的LLM-Synergy框架,代表了利用LLM进行医疗QA任务的显着进步,并提供了有效利用LLMTechnologies开发的创新方式,在生物医学和健康信息学研究中,为现有和潜在的未来挑战任务定制。
UNASSIGNED: To enhance the accuracy and reliability of diverse medical question-answering (QA) tasks and investigate efficient approaches deploying the Large Language Models (LLM) technologies, We developed a novel ensemble learning pipeline by utilizing state-of-the-art LLMs, focusing on improving performance on diverse medical QA datasets.
UNASSIGNED: Our study employs three medical QA datasets: PubMedQA, MedQA-USMLE, and MedMCQA, each presenting unique challenges in biomedical question-answering. The proposed LLM-Synergy framework, focusing exclusively on zero-shot cases using LLMs, incorporates two primary ensemble methods. The first is a Boosting-based weighted majority vote ensemble, where decision-making is expedited and refined by assigning variable weights to different LLMs through a boosting algorithm. The second method is Cluster-based Dynamic Model Selection, which dynamically selects the most suitable LLM votes for each query, based on the characteristics of question contexts, using a clustering approach.
UNASSIGNED: The Majority Weighted Vote and Dynamic Model Selection methods demonstrate superior performance compared to individual LLMs across three medical QA datasets. Specifically, the accuracies are 35.84%, 96.21%, and 37.26% for MedMCQA, PubMedQA, and MedQA-USMLE, respectively, with the Majority Weighted Vote. Correspondingly, the Dynamic Model Selection yields slightly higher accuracies of 38.01%, 96.36%, and 38.13%.
UNASSIGNED: The LLM-Synergy framework with two ensemble methods, represents a significant advancement in leveraging LLMs for medical QA tasks and provides an innovative way of efficiently utilizing the development with LLM Technologies, customing for both existing and potentially future challenge tasks in biomedical and health informatics research.