背景:随机对照试验(RCT)的系统评价是循证医学范式的重要组成部分。然而,由临床专家创建这样的系统评价既昂贵又耗时,结果在出版后很快就会过时。大多数RCT都是基于患者的结构,干预,比较,结果(PICO)框架,并且存在许多旨在自动提取PICO元素的方法。从RCT中自动提取PICO信息有可能显着加快系统评价的创建过程,这种方式也有利于循证医学领域。
结果:以前的工作已经解决了提取PICO元素作为识别相关文本跨度或句子的任务,但没有填充试验的结构化表示。相比之下,在这项工作中,我们将PICO元素视为带有插槽的结构化模板,以公正地处理它们所代表的信息的复杂性质。我们提出了两种不同的方法来从RCT摘要中提取这些结构化信息。第一种方法是基于我们以前的工作的提取方法,该方法扩展为捕获完整的文档表示,并通过聚类步骤来推断每种模板类型的实例数量。第二种方法是基于seq2seq模型的生成方法,该模型对描述RCT的摘要进行编码,并使用解码器来推断包括其臂在内的试验的结构化表示,治疗,终点和结果。两种方法在手动注释的数据集上用不同的基础模型进行评估,该数据集由现有数据集上的RCT摘要组成,该现有数据集包括211个注释的2型糖尿病和青光眼的临床试验摘要。对于这两种疾病,提取方法(用flan-t5-base)达到了最好的F1评分,即2型糖尿病为0.547(±0.006),青光眼为0.636(±0.006)。一般来说,青光眼的F1评分高于2型糖尿病,而生成性方法的标准差更高.
结论:在我们的实验中,这两种方法都显示出从RCT中提取结构化PICO信息的良好性能,特别是考虑到大多数相关的工作都集中在预测结构较少的对象这个容易得多的任务上。在我们的实验结果中,提取方法在这两种情况下都表现最好,尽管青光眼的铅含量高于2型糖尿病。为了将来的工作,基础模型大小如何影响两种方法的性能还有待研究。尽管采掘方法目前为直接改进留下了更多空间,生成方法可能会受益于更大的模型。
BACKGROUND: Systematic reviews of Randomized Controlled Trials (RCTs) are an important part of the evidence-based medicine paradigm. However, the creation of such systematic reviews by clinical experts is costly as well as time-consuming, and results can get quickly outdated after publication. Most RCTs are structured based on the Patient, Intervention, Comparison, Outcomes (
PICO) framework and there exist many approaches which aim to extract
PICO elements automatically. The automatic extraction of
PICO information from RCTs has the potential to significantly speed up the creation process of systematic reviews and this way also benefit the field of evidence-based medicine.
RESULTS: Previous work has addressed the extraction of
PICO elements as the task of identifying relevant text spans or sentences, but without populating a structured representation of a trial. In contrast, in this work, we treat
PICO elements as structured templates with slots to do justice to the complex nature of the information they represent. We present two different approaches to extract this structured information from the abstracts of RCTs. The first approach is an extractive approach based on our previous work that is extended to capture full document representations as well as by a clustering step to infer the number of instances of each template type. The second approach is a generative approach based on a seq2seq model that encodes the abstract describing the RCT and uses a decoder to infer a structured representation of a trial including its arms, treatments, endpoints and outcomes. Both approaches are evaluated with different base models on a manually annotated dataset consisting of RCT abstracts on an existing dataset comprising 211 annotated clinical trial abstracts for Type 2 Diabetes and Glaucoma. For both diseases, the extractive approach (with flan-t5-base) reached the best F 1 score, i.e. 0.547 ( ± 0.006 ) for type 2 diabetes and 0.636 ( ± 0.006 ) for glaucoma. Generally, the F 1 scores were higher for glaucoma than for type 2 diabetes and the standard deviation was higher for the generative approach.
CONCLUSIONS: In our experiments, both approaches show promising performance extracting structured PICO information from RCTs, especially considering that most related work focuses on the far easier task of predicting less structured objects. In our experimental results, the extractive approach performs best in both cases, although the lead is greater for glaucoma than for type 2 diabetes. For future work, it remains to be investigated how the base model size affects the performance of both approaches in comparison. Although the extractive approach currently leaves more room for direct improvements, the generative approach might benefit from larger models.