背景:在肿瘤学中,电子健康记录包含用于诊断的文本关键信息,分期,和癌症患者的治疗计划。然而,文本数据处理需要大量的时间和精力,这限制了这些数据的利用。自然语言处理(NLP)技术的最新进展,包括大型语言模型,可用于癌症研究。特别是,从手术病理报告中提取病理阶段所需的信息可用于根据最新的癌症分期指南更新癌症分期。
目的:本研究有两个主要目的。第一个目标是评估从基于文本的手术病理报告中提取信息并基于所提取的信息使用针对肺癌患者的微调生成语言模型(GLM)来确定病理阶段的性能。第二个目标是确定在资源受限的计算环境中利用相对较小的GLM进行信息提取的可行性。
方法:从首尔国立大学邦当医院(SNUBH)的通用数据模型数据库收集肺癌手术病理报告,韩国的一家三级医院。我们根据这些报告选择了肿瘤淋巴结(TN)分类所需的42个描述符,并通过两名临床专家的验证创建了黄金标准。病理报告和金标准用于生成用于训练和评估GLM的提示反应对,然后将其用于从病理报告中提取分期所需的信息。
结果:我们使用提取的信息评估了六个训练模型的信息提取性能以及它们在TN分类中的性能。演绎的Mistral-7B模型,用演绎数据集预先训练,整体表现最好,信息提取问题的精确匹配率为92.24%,分类精度为0.9876(同时预测T和N分类)。
结论:这项研究表明,使用演绎数据集训练GLM可以提高信息提取性能,和GLM具有相对较少数量的大约70亿个参数可以在这个问题上实现高性能。提出的基于GLM的信息提取方法有望在临床决策支持中有用,肺癌分期和研究。
BACKGROUND: In oncology, electronic health records contain textual key information for the diagnosis, staging, and treatment planning of patients with cancer. However, text data processing requires a lot of time and effort, which limits the utilization of these data. Recent advances in natural language processing (NLP) technology, including large language models, can be applied to cancer research. Particularly, extracting the information required for the pathological stage from surgical pathology reports can be utilized to update cancer staging according to the latest cancer staging guidelines.
OBJECTIVE: This study has two main objectives. The first objective is to evaluate the performance of extracting information from text-based surgical pathology reports and determining pathological stages based on the extracted information using fine-tuned generative language models (GLMs) for patients with lung cancer. The second objective is to determine the feasibility of utilizing relatively small GLMs for information extraction in a resource-constrained computing environment.
METHODS: Lung cancer surgical pathology reports were collected from the Common Data Model database of Seoul National University Bundang Hospital (SNUBH), a tertiary hospital in Korea. We selected 42 descriptors necessary for tumor-node (TN) classification based on these reports and created a gold standard with validation by two clinical experts. The pathology reports and gold standard were used to generate prompt-response pairs for training and evaluating GLMs which then were used to extract information required for staging from pathology reports.
RESULTS: We evaluated the information extraction performance of six trained models as well as their performance in TN classification using the extracted information. The Deductive Mistral-7B model, which was pre-trained with the deductive dataset, showed the best performance overall, with an exact match ratio of 92.24% in the information extraction problem and an accuracy of 0.9876 (predicting T and N classification concurrently) in classification.
CONCLUSIONS: This study demonstrated that training GLMs with deductive datasets can improve information extraction performance, and GLMs with a relatively small number of parameters at approximately seven billion can achieve high performance in this problem. The proposed GLM-based information extraction method is expected to be useful in clinical decision-making support, lung cancer staging and research.