PDF(Pubmed)
Abstract：
BACKGROUND: Digital health interventions (DHIs) have the potential to enable public end users, such as citizens and patients, to manage and improve their health. Although the number of available DHIs is increasing, examples of successfully established DHIs in public health systems are limited. To counteract the nonuse of DHIs, they should be comprehensively evaluated while integrating end users. Unfortunately, there is a wide variability and heterogeneity according to the approaches of evaluation, which creates a methodological challenge.
OBJECTIVE: This scoping review aims to provide an overview of the current established processes for evaluating DHIs, including methods, indicators, and end-user involvement. The review is not limited to a specific medical field or type of DHI but offers a holistic overview.
METHODS: This scoping review was conducted following the JBI methodology for scoping reviews based on the framework by Arksey & O\'Malley and complies with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. Three scientific databases (PubMed, Scopus, and Science Direct) were searched in April 2023. English and German studies between 2008 and 2023 were considered when evaluating DHIs that explicitly address public end users. The process of study selection was carried out by several researchers to avoid reviewer bias.
RESULTS: The search strategy identified 9618 publications, of which 160 were included. Among these included articles, 200 evaluations were derived and analyzed. The results showed that there is neither a consensus on the methods to evaluate DHIs nor a commonly agreed definition or usage of the evaluated indicators, which results in a broad variety of evaluation practices. This aligns with observations of the existing literature. It was found that there is a lack of references to existing frameworks for the evaluation of DHIs. The majority of the included studies referred to user-centered approaches and involved end users in the evaluation process. As assistance for people developing and evaluating DHIs and as a basis for thinking about appropriate ways to evaluate DHIs, a results matrix was created where the findings were combined per DHI cluster. Additionally, general recommendations for the evaluators of DHIs were formulated.
CONCLUSIONS: The findings of this scoping review offer a holistic overview of the variety and heterogeneity according to the approaches of evaluation of DHIs for public end users. Evaluators of these DHIs should be encouraged to reference established frameworks or measurements for justification. This would ease the transferability of the results among similar evaluation studies within the digital health sector, thereby enhancing the coherence and comparability of research in this area.
OBJECTIVE: This scoping review aims to provide an overview of the current established processes for evaluating DHIs, including methods, indicators, and end-user involvement. The review is not limited to a specific medical field or type of DHI but offers a holistic overview.
METHODS: This scoping review was conducted following the JBI methodology for scoping reviews based on the framework by Arksey & O\'Malley and complies with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. Three scientific databases (PubMed, Scopus, and Science Direct) were searched in April 2023. English and German studies between 2008 and 2023 were considered when evaluating DHIs that explicitly address public end users. The process of study selection was carried out by several researchers to avoid reviewer bias.
RESULTS: The search strategy identified 9618 publications, of which 160 were included. Among these included articles, 200 evaluations were derived and analyzed. The results showed that there is neither a consensus on the methods to evaluate DHIs nor a commonly agreed definition or usage of the evaluated indicators, which results in a broad variety of evaluation practices. This aligns with observations of the existing literature. It was found that there is a lack of references to existing frameworks for the evaluation of DHIs. The majority of the included studies referred to user-centered approaches and involved end users in the evaluation process. As assistance for people developing and evaluating DHIs and as a basis for thinking about appropriate ways to evaluate DHIs, a results matrix was created where the findings were combined per DHI cluster. Additionally, general recommendations for the evaluators of DHIs were formulated.
CONCLUSIONS: The findings of this scoping review offer a holistic overview of the variety and heterogeneity according to the approaches of evaluation of DHIs for public end users. Evaluators of these DHIs should be encouraged to reference established frameworks or measurements for justification. This would ease the transferability of the results among similar evaluation studies within the digital health sector, thereby enhancing the coherence and comparability of research in this area.
摘要：
背景：数字健康干预措施（DHIs）具有使公共最终用户，比如公民和病人，管理和改善他们的健康。尽管可用DHI的数量正在增加，在公共卫生系统中成功建立DHI的例子有限。为了抵消不使用DHI，在整合最终用户的同时，应该对它们进行全面评估。不幸的是,根据评价方法，存在很大的变异性和异质性，这就带来了方法论上的挑战。
目的：本范围审查旨在概述当前已建立的DHI评估流程，包括方法，指标,和最终用户的参与。该审查不仅限于特定的医学领域或DHI类型，还提供了整体概述。
方法：本范围审查是根据Arksey&O'Malley框架的JBI范围审查方法进行的，并符合PRISMA-ScR（系统审查的首选报告项目和范围审查的Meta分析扩展）指南。三个科学数据库(PubMed，Scopus,和ScienceDirect)于2023年4月进行了搜索。在评估明确针对公共最终用户的DHI时，考虑了2008年至2023年之间的英语和德语研究。研究选择的过程是由几位研究人员进行的，以避免审阅者的偏见。
结果：搜索策略确定了9618种出版物，其中包括160个。在这些包括的文章中，得出并分析了200项评估。结果表明，在评估DHI的方法上没有共识，也没有公认的评估指标的定义或用法。这导致了各种各样的评估实践。这与现有文献的观察结果一致。发现缺乏对评估DHI的现有框架的参考。大多数纳入的研究都涉及以用户为中心的方法，并在评估过程中涉及最终用户。作为对开发和评估DHI的人员的协助，并作为思考评估DHI的适当方法的基础，我们创建了一个结果矩阵,每个DHI集群将这些结果合并在一起.此外,为DHI评估人员制定了一般性建议。
结论：根据公共最终用户的DHI评估方法，本范围审查的结果提供了多样性和异质性的整体概述。应鼓励这些DHI的评估者参考已建立的框架或衡量标准进行论证。这将缓解数字卫生部门类似评估研究中结果的可转移性，从而增强该领域研究的连贯性和可比性。
目的：本范围审查旨在概述当前已建立的DHI评估流程，包括方法，指标,和最终用户的参与。该审查不仅限于特定的医学领域或DHI类型，还提供了整体概述。
方法：本范围审查是根据Arksey&O'Malley框架的JBI范围审查方法进行的，并符合PRISMA-ScR（系统审查的首选报告项目和范围审查的Meta分析扩展）指南。三个科学数据库(PubMed，Scopus,和ScienceDirect)于2023年4月进行了搜索。在评估明确针对公共最终用户的DHI时，考虑了2008年至2023年之间的英语和德语研究。研究选择的过程是由几位研究人员进行的，以避免审阅者的偏见。
结果：搜索策略确定了9618种出版物，其中包括160个。在这些包括的文章中，得出并分析了200项评估。结果表明，在评估DHI的方法上没有共识，也没有公认的评估指标的定义或用法。这导致了各种各样的评估实践。这与现有文献的观察结果一致。发现缺乏对评估DHI的现有框架的参考。大多数纳入的研究都涉及以用户为中心的方法，并在评估过程中涉及最终用户。作为对开发和评估DHI的人员的协助，并作为思考评估DHI的适当方法的基础，我们创建了一个结果矩阵,每个DHI集群将这些结果合并在一起.此外,为DHI评估人员制定了一般性建议。
结论：根据公共最终用户的DHI评估方法，本范围审查的结果提供了多样性和异质性的整体概述。应鼓励这些DHI的评估者参考已建立的框架或衡量标准进行论证。这将缓解数字卫生部门类似评估研究中结果的可转移性，从而增强该领域研究的连贯性和可比性。
