Abstract：
Health protection agencies require scientific information for evidence-based decision-making and guideline development. However, vetting and collating large quantities of published research to identify relevant high-quality studies is a challenge. One approach to address this issue is the use of adverse outcome pathways (AOPs) that provide a framework to assemble toxicological knowledge into causally linked chains of key events (KEs) across levels of biological organization to culminate in an adverse health outcome of significance to regulatory decision-making. Traditionally, AOPs have been constructed using a narrative review approach where the collection of evidence that supports each pathway is based on prior knowledge of influential studies that can also be supplemented by individually selecting and reviewing relevant references.
We aimed to create a protocol for AOP weight of evidence gathering that harnesses elements of both scoping review methods and artificial intelligence (AI) tools to increase transparency while reducing bias and workload of human screeners.
To develop this protocol, an existing space-health AOP in the workplan of the Organisation for Economic Co-operation and Development (OECD) AOP Programme was used as a case example. To balance the benefits of both scoping review tools and narrative approaches, a study protocol outlining a screening and search strategy was developed, and three reference collection workflows were tested to identify the most efficient method to inform weight of evidence. The workflows differed in their literature search strategies, and combinations of software tools used.
Across the three tested workflows, over 59 literature searches were completed, retrieving over 34,000 references of which over 3300 were human reviewed. The most effective of the three methods used a search strategy with searches across each component of the AOP network, SWIFT Review as a pre-filtering software, and DistillerSR to create structured screening and data extraction forms. This methodology effectively retrieved relevant studies while balancing efficiency in data retrieval without compromising transparency, leading to a well-synthesized evidence base to support the AOP.
The workflow is still exploratory in the context of AOP development, and we anticipate adaptations to the protocol with further experience. To further the systematicity, future iterations of the workflow could include structured quality assessment and risk of bias analysis. Overall, the workflow provides a transparent and documented approach to support AOP development, which in turn will support the need for rigorous methods to identify relevant scientific evidence while being practical to allow uptake by the broader community.
We aimed to create a protocol for AOP weight of evidence gathering that harnesses elements of both scoping review methods and artificial intelligence (AI) tools to increase transparency while reducing bias and workload of human screeners.
To develop this protocol, an existing space-health AOP in the workplan of the Organisation for Economic Co-operation and Development (OECD) AOP Programme was used as a case example. To balance the benefits of both scoping review tools and narrative approaches, a study protocol outlining a screening and search strategy was developed, and three reference collection workflows were tested to identify the most efficient method to inform weight of evidence. The workflows differed in their literature search strategies, and combinations of software tools used.
Across the three tested workflows, over 59 literature searches were completed, retrieving over 34,000 references of which over 3300 were human reviewed. The most effective of the three methods used a search strategy with searches across each component of the AOP network, SWIFT Review as a pre-filtering software, and DistillerSR to create structured screening and data extraction forms. This methodology effectively retrieved relevant studies while balancing efficiency in data retrieval without compromising transparency, leading to a well-synthesized evidence base to support the AOP.
The workflow is still exploratory in the context of AOP development, and we anticipate adaptations to the protocol with further experience. To further the systematicity, future iterations of the workflow could include structured quality assessment and risk of bias analysis. Overall, the workflow provides a transparent and documented approach to support AOP development, which in turn will support the need for rigorous methods to identify relevant scientific evidence while being practical to allow uptake by the broader community.
摘要：
健康保护机构需要科学信息，以循证决策和指南制定。然而,审查和整理大量已发表的研究以确定相关的高质量研究是一个挑战。解决这一问题的一种方法是使用不良结果途径（AOP），该途径提供了一个框架，将毒理学知识整合到跨生物组织水平的关键事件（KE）的因果关系链中，最终导致对监管决策具有重要意义的不良健康结果。传统上,AOP是使用叙述性审查方法构建的，其中支持每个途径的证据收集是基于有影响力的研究的先验知识，也可以通过单独选择和审查相关参考文献来补充。
我们的目标是创建一个AOP重量证据收集协议，利用范围审查方法和人工智能(AI)工具的元素来提高透明度，同时减少人类筛查人员的偏见和工作量。
为了开发这个协议，经济合作与发展组织（经合组织）AOP计划的工作计划中现有的空间健康AOP被用作案例示例。为了平衡范围界定审查工具和叙述方法的好处，制定了概述筛查和搜索策略的研究方案，并测试了三个参考收集工作流程，以确定最有效的方法来告知证据的权重。工作流程的文献检索策略不同，以及使用的软件工具的组合。
在三个经过测试的工作流程中，完成了超过59个文献检索，检索超过34,000个参考文献，其中超过3300个是人类审查。这三种方法中最有效的是使用搜索策略，在AOP网络的每个组件上进行搜索，SWIFTReview作为预过滤软件，和DistillerSR创建结构化筛选和数据提取表单。这种方法有效地检索了相关研究，同时平衡了数据检索的效率，而不影响透明度，导致一个综合良好的证据基础来支持AOP。
在AOP开发的背景下，工作流程仍然是探索性的，我们预计会有更多的经验来适应协议。为了进一步系统化，工作流程的未来迭代可能包括结构化质量评估和偏差风险分析。总的来说,工作流提供了一种透明且有文档记录的方法来支持AOP开发，这反过来将支持需要严格的方法来识别相关的科学证据，同时切实可行，允许更广泛的社区采用。
我们的目标是创建一个AOP重量证据收集协议，利用范围审查方法和人工智能(AI)工具的元素来提高透明度，同时减少人类筛查人员的偏见和工作量。
为了开发这个协议，经济合作与发展组织（经合组织）AOP计划的工作计划中现有的空间健康AOP被用作案例示例。为了平衡范围界定审查工具和叙述方法的好处，制定了概述筛查和搜索策略的研究方案，并测试了三个参考收集工作流程，以确定最有效的方法来告知证据的权重。工作流程的文献检索策略不同，以及使用的软件工具的组合。
在三个经过测试的工作流程中，完成了超过59个文献检索，检索超过34,000个参考文献，其中超过3300个是人类审查。这三种方法中最有效的是使用搜索策略，在AOP网络的每个组件上进行搜索，SWIFTReview作为预过滤软件，和DistillerSR创建结构化筛选和数据提取表单。这种方法有效地检索了相关研究，同时平衡了数据检索的效率，而不影响透明度，导致一个综合良好的证据基础来支持AOP。
在AOP开发的背景下，工作流程仍然是探索性的，我们预计会有更多的经验来适应协议。为了进一步系统化，工作流程的未来迭代可能包括结构化质量评估和偏差风险分析。总的来说,工作流提供了一种透明且有文档记录的方法来支持AOP开发，这反过来将支持需要严格的方法来识别相关的科学证据，同时切实可行，允许更广泛的社区采用。
