目的:为了评估和改进MATCH-IT-数字,交互式决策支持工具显示用于多重比较的结构化证据摘要-以帮助医师解释和应用来自网络荟萃分析(NMA)的证据用于其临床决策.
方法:我们进行了一项定性的用户测试研究,在迭代开发过程中应用以用户为中心的设计原则。我们在挪威招募了一个方便的执业医生样本,比利时和加拿大,并要求他们解释MATCH-IT中显示的用于多重比较的结构化证据摘要-与临床指南建议相关。用户测试包括(A)介绍临床场景,(b)与参与者工具互动的大声思考会议和(c)半结构化访谈。我们录制了视频,转录,并使用定向内容分析分析用户测试。结果为MATCH-IT提供了新的更新。
结果:分布在五个开发周期中,我们与26名医生一起测试了MATCH-IT。其中,24(94%)报告没有或很少有NMA解释的经验。医生认为MATCH-IT很容易解释,导航并赞赏其提供证据概述的能力。象形图效果的可视化和治疗负担的信息(“实际问题”)被强调为与患者互动的潜在有用特征。我们还发现了问题,包括未发现的功能(拖放),次优教程,和繁琐的导航工具。此外,医生希望定义/解释关键术语(例如,结果和“确定性”),并且有人担心来自大型NMA的压倒性证据会使临床实践的适用性复杂化。这导致了一些更新,开发新的起始页,教程,更新的用户界面,以更有效地操纵,显示关键术语定义和FAQ部分的解决方案。为了便于解释大型网络,我们使用颜色编码和增加过滤功能改进了结果分类.这些修改使医生能够专注于感兴趣的干预措施并减少信息过载。
结论:这项研究提供了医生可以使用MATCH-IT来理解NMA证据的概念证明。MATCH-IT在临床背景下的主要特征包括提供证据的概述,效果的可视化和治疗负担信息的显示。然而,不熟悉等级概念,时间限制和护理点的可及性可能是使用的挑战。我们的结果在多大程度上可以转移到现实世界的临床环境中还有待探索。
OBJECTIVE: To evaluate and improve \"Making Alternative Treatment Choices Intuitive and Trustworthy\" (MATCH-IT)-a digital, interactive decision support tool displaying structured evidence summaries for multiple comparisons-to help physicians interpret and apply evidence from network meta-analysis (NMA) for their clinical decision-making.
METHODS: We conducted a qualitative user testing study, applying principles from user-centered design in an iterative development process. We recruited a convenience sample of practicing physicians in Norway, Belgium, and Canada, and asked them to interpret structured evidence summaries for multiple comparisons-linked to clinical guideline recommendations-displayed in MATCH-IT. User testing included (a) introduction of a clinical scenario, (b) a think-aloud session with participant-tool interaction, and (c) a semistructured interview. We video recorded, transcribed, and analyzed user tests using directed content analysis. The results informed new updates in MATCH-IT.
RESULTS: Distributed across 5 development cycles we tested MATCH-IT with 26 physicians. Of these, 24 (94%) reported either no or sparse prior experience with interpretation of NMA. Physicians perceived MATCH-IT as easy to interpret and navigate, and appreciated its ability to provide an overview of the evidence. Visualization of effects in pictograms and inclusion of information on burden of treatment (\"practical issues\") were highlighted as potentially useful features in interacting with patients. We also identified problems, including undiscovered functionalities (drag and drop), suboptimal tutorial, and cumbersome navigation of the tool. In addition, physicians wanted definition/explanation of key terms (eg, outcomes and \"certainty\"), and there were concerns that overwhelming evidence from a large NMA would complicate applicability to clinical practice. This led to several updates with development of a new start page, tutorial, updated user interface for more efficient maneuvering, solutions to display definition of key terms and a \"frequently asked questions\" section. To facilitate interpretation of large networks, we improved categorization of results using color coding and added filtering functionality. These modifications allowed physicians to focus on interventions of interest and reduce information overload.
CONCLUSIONS: This study provides proof of concept that physicians can use MATCH-IT to understand NMA evidence. Key features of MATCH-IT in a clinical context include providing an overview of the evidence, visualization of effects, and the display of information on burden of treatments. However, unfamiliarity with the Grading of Recommendations Assessment, Development and Evaluation concepts, time constraints, and accessibility at the point of care may be challenges for use. To what extent our results are transferable to real-world clinical contexts remains to be explored.