务实的(即,实用)量化实施措施的质量近年来在实施科学文献中受到越来越多的关注。被执行利益相关者认为是务实的执行措施被认为更有可能在研究和实践中应用。尽管需要采取务实的执行措施,关于什么是务实措施的模糊性和不确定性仍然存在。本研究旨在识别和批判性地评估已发表的文献,以了解(i)实用主义如何定义为实施决定因素和结果工具的测量结构/质量;(ii)如何评估工具的实用质量;(iii)确定当前证据基础的关键差距和局限性,以及(iv)确定未来研究的建议。我们还采用批判性审查的方法对文献进行了范围审查。PubMed和PsycINFO数据库,使用OVID接口,搜索了2010年1月至2020年9月发表的相关文章。包含定义和/或描述“实用主义”特征的文章作为定量实施结果(由Proctor的实施结果分类法定义)和/或实施决定因素的衡量结构,有资格纳入。九篇文章符合纳入标准。发现用于描述定量实施决定因素和结果工具的实用质量的定义和术语存在一定程度的重叠。最常被引用的实用主义描述词是“不累赘”,\"brief\",\"可靠\",\"有效\"和\"对更改敏感\"。在9条纳入的条款中,有3条涉及国际执行利益攸关方界定和概念化实用主义,并采用了具体的方法来做到这一点,包括系统的文献综述,利益相关者访谈,概念映射,和Delphi过程。所有其他文章都定义了实用主义,有或没有引用相关文献。一篇文章客观地评估了语用素质,超越心理测量品质,执行措施,使用心理测量学和语用证据评定量表(论文)。实施工具文献中关于实用主义是什么以及如何评估实用主义的证据基础是有限的。审查中确定的一些研究提供了坚实的基础,通过测试其在其他环境(包括医疗保健领域和国家)和更多样化的利益相关者群体中的适用性。我们讨论了与实施决定因素和结果的衡量有关的实用主义概念的进一步发展方向。
需要务实(即,实际)在实施科学文献中一再呼吁评估实施工作的措施。这篇文献综述的重点是理解实用主义,作为实施措施的测量结构和质量,已定义,概念化和评估。我们发现很少有文章(n=9)有助于我们理解实用主义是如何定义和评估的。我们发现,描述实施措施的务实品质的最常用术语包括“不负担”,"brief","可靠"“有效”和“对变化敏感”。我们确定了一个尺度,心理测量学和语用证据评定量表(论文),用来衡量语用质量,以及心理测量质量,执行措施。我们确定了当前证据基础的几个差距和局限性,并提供了进一步发展实用主义概念和评估的方向。具体来说,我们建议未来的研究重点是让更广泛的执行利益攸关方参与界定和概念化实用主义,并对现有的务实评估措施进行更严格和广泛的评估。
The pragmatic (i.e., practical) quality of quantitative implementation measures has received increased attention in the implementation science literature in recent years. Implementation measures that are judged to be pragmatic by implementation stakeholders are thought to be more likely to be applied in research and practice. Despite the need for pragmatic implementation measures, ambiguity and uncertainty regarding what constitutes a pragmatic measure remains. This study sought to identify and critically appraise the published literature to understand (i) how pragmatism is defined as a measurement construct/quality of implementation determinants and outcome instruments; (ii) how pragmatic qualities of instruments are evaluated; (iii) identify key gaps and limitations of the current evidence-base and (iv) identify recommendations for future research. We conducted a scoping
review of the literature also employing methods of critical
review. PubMed and PsycINFO databases, using the OVID interface, were searched for relevant articles published between January 2010 and September 2020. Articles that contained a definition and/or described characteristics of \"pragmatism\" as a measurement construct of quantitative implementation outcomes (as defined by Proctor\'s Implementation Outcomes taxonomy) and/or implementation determinants were eligible for inclusion. Nine articles met inclusion criteria. A degree of overlap in definitions and terms used to describe the pragmatic qualities of quantitative implementation determinant and outcome instruments were found. The most frequently cited descriptors of pragmatism were \"not burdensome\", \"brief\", \"reliable\", \"valid\" and \"sensitive to change\". 3 of the 9 included articles involved international implementation stakeholders in defining and conceptualizing pragmatism and employed specific methods to do so, including a systematic literature
review, stakeholder interviews, concept mapping, and a Delphi process. All other articles defined pragmatism, with or without citing relevant literature. One article objectively assessed the pragmatic qualities, above and beyond the psychometric qualities, of implementation measures, using the Psychometric and Pragmatic Evidence Rating Scale (PAPERS). The evidence base within the implementation instrumentation literature on what pragmatism is and how it might be assessed is limited. Some of the research identified in the
review provides a strong foundation to build upon, by testing its applicability in other settings (including healthcare areas and countries) and among a more diverse group of stakeholders. We discuss directions for further development of the concept of pragmatism relating to the measurement of implementation determinants and outcomes.
The need for pragmatic (i.e., practical) measures to evaluate implementation efforts has been repeatedly called for in the implementation science literature. This literature
review focuses on understanding how pragmatism, as a measurement construct and quality of implementation measures, is defined, conceptualized and evaluated. We identified few articles (n = 9) that contribute to our understanding of how pragmatism is defined and evaluated. We found that the most frequently used terms to describe pragmatic qualities of implementation measures include “not burdensome”, “brief”, “reliable”, “valid” and “sensitive to change”. We identified one scale, the Psychometric and Pragmatic Evidence Rating Scale (PAPERS), developed to measure the pragmatic quality, as well as psychometric quality, of implementation measures. We identified several gaps and limitations of the current evidence-base and offer directions to further develop the concept and appraisal of pragmatism. Specifically, we recommend that future research focus on engaging and involving a wider diversity of implementation stakeholders in defining and conceptualizing pragmatism as well as subjecting existing pragmatic assessment measures to more rigorous and extensive evaluation.